27:
technique that taps into the expert minority opinion within a crowd. For a given question, a group is asked both "What do you think the right answer is?" and "What do you think the popular answer will be?" The answer that maximizes the average difference between the "right" answer and the "popular"
135:
Both groups B and D think they know something others do not, but B is wrong and D is right. In cases where people feel like they have "inside" knowledge, it is more often the case that it is because they are correct and knowledgeable (group D), not because they are misled (group B).
144:
For m>2 candidates, the
Surprisingly Popular Algorithm requires votes from an infinite number of voters on all possible ranked permutations (m!) of the alternatives to recover the ground-truth ranking with complete certainty, as discussed in the
100:
answer is surprisingly popular (10% > โ10%). Because of the relatively high margin of 10%, there can be high confidence that the correct answer is
419:
340:
132:
This technique causes groups A and C to be eliminated from consideration and measures the difference in size between groups B and D.
313:
Hosseini, Hadi; Mandal, Debmalya; Shah, Nisarg; Shi, Kevin (2021). "Surprisingly
Popular Voting Recovers Rankings, Surprisingly!".
199:
360:
Prelec, Draลพen; Seung, H. Sebastian; McCoy, John (25 January 2017). "A solution to the single-question crowd wisdom problem".
28:
answer is the "surprisingly popular" answer. The term "surprisingly popular" was coined in a 2017 paper published in
151:
article. Hosseini et al. (2021) extended the previous work to recover rankings using various elicitation formats.
177:
165:
160:
414:
369:
263:
128:
D โ "Philadelphia is not the capital, but most others won't know that." (This group answers no/yes.)
24:
424:
393:
318:
295:
34:
entitled "A solution to the single-question crowd wisdom problem", which outlined the technique.
225:
122:
B โ "Philadelphia is the capital, but most others won't know that". (This group answers yes/no.)
385:
336:
287:
279:
377:
328:
271:
147:
29:
429:
125:
C โ "Philadelphia is not the capital, and others will agree." (This group answers no/no.)
373:
267:
119:
A โ "Philadelphia is the capital, and others will agree." (This group answers yes/yes.)
315:
Proceedings of the
Thirtieth International Joint Conference on Artificial Intelligence
408:
397:
299:
105:
47:
43:
171:
251:
109:
332:
283:
115:
An illustrative breakdown of this follows. There are four groups of people.
389:
291:
50:? The two questions asked of the group, and the numbers of responses, are:
381:
275:
323:
140:
Surprisingly
Popular Algorithm for ground-truth ranking recovery
65:
What do you think most people will respond to that question?
250:
Prelec, Draลพen; Seung, H. Sebastian; McCoy, John (2017).
252:"A solution to the single-question crowd wisdom problem"
16:Algorithm for extracting wisdom from the crowd
54:Is Philadelphia the capital of Pennsylvania?
42:Suppose the question to be determined is: Is
8:
77:The difference between the answers to the
322:
190:
7:
245:
243:
219:
217:
224:Dizikes, Peter (January 25, 2017).
198:Akst, Daniel (February 16, 2017).
14:
200:"The Wisdom of Even Wiser Crowds"
180:, also known as Schelling point
1:
420:Social information processing
104:. (The capital is indeed not
226:"Better wisdom from crowds"
446:
202:. The Wall Street Journal
178:Focal point (game theory)
166:Guess 2/3 of the average
161:Keynesian beauty contest
333:10.24963/ijcai.2021/35
89:Yes: 65% โ 75% = โ10%
317:. pp. 245โ251.
21:surprisingly popular
382:10.1038/nature21054
374:2017Natur.541..532P
276:10.1038/nature21054
268:2017Natur.541..532P
92:No: 35% โ 25% = 10%
25:wisdom of the crowd
368:(7638): 532โ535.
342:978-0-9992411-9-6
262:(7638): 532โ535.
81:question and the
437:
401:
347:
346:
326:
310:
304:
303:
247:
238:
237:
235:
233:
221:
212:
211:
209:
207:
195:
445:
444:
440:
439:
438:
436:
435:
434:
405:
404:
359:
356:
354:Further reading
351:
350:
343:
312:
311:
307:
249:
248:
241:
231:
229:
223:
222:
215:
205:
203:
197:
196:
192:
187:
157:
142:
46:the capital of
40:
17:
12:
11:
5:
443:
441:
433:
432:
427:
422:
417:
407:
406:
403:
402:
355:
352:
349:
348:
341:
305:
239:
213:
189:
188:
186:
183:
182:
181:
175:
168:
163:
156:
153:
141:
138:
130:
129:
126:
123:
120:
94:
93:
90:
75:
74:
73:
72:
69:
63:
62:
61:
58:
39:
36:
15:
13:
10:
9:
6:
4:
3:
2:
442:
431:
428:
426:
423:
421:
418:
416:
415:Crowdsourcing
413:
412:
410:
399:
395:
391:
387:
383:
379:
375:
371:
367:
363:
358:
357:
353:
344:
338:
334:
330:
325:
320:
316:
309:
306:
301:
297:
293:
289:
285:
281:
277:
273:
269:
265:
261:
257:
253:
246:
244:
240:
227:
220:
218:
214:
201:
194:
191:
184:
179:
176:
174:
173:
169:
167:
164:
162:
159:
158:
154:
152:
150:
149:
139:
137:
133:
127:
124:
121:
118:
117:
116:
113:
111:
107:
103:
99:
91:
88:
87:
86:
84:
80:
70:
67:
66:
64:
59:
56:
55:
53:
52:
51:
49:
45:
37:
35:
33:
32:
26:
22:
365:
361:
314:
308:
259:
255:
230:. Retrieved
204:. Retrieved
193:
170:
146:
143:
134:
131:
114:
106:Philadelphia
101:
97:
95:
82:
78:
76:
48:Pennsylvania
44:Philadelphia
41:
30:
23:answer is a
20:
18:
172:Family Feud
409:Categories
324:2105.09386
228:. MIT News
185:References
110:Harrisburg
96:Thus, the
85:question:
425:Knowledge
284:1476-4687
390:28128245
292:28128245
155:See also
68:Yes: 75%
57:Yes: 65%
398:4452604
370:Bibcode
300:4452604
264:Bibcode
83:popular
71:No: 25%
60:No: 35%
38:Example
430:Crowds
396:
388:
362:Nature
339:
298:
290:
282:
256:Nature
232:16 May
206:16 May
148:Nature
108:, but
31:Nature
394:S2CID
319:arXiv
296:S2CID
79:right
386:PMID
337:ISBN
288:PMID
280:ISSN
234:2018
208:2018
19:The
378:doi
366:541
329:doi
272:doi
260:541
112:.)
411::
392:.
384:.
376:.
364:.
335:.
327:.
294:.
286:.
278:.
270:.
258:.
254:.
242:^
216:^
102:No
98:No
400:.
380::
372::
345:.
331::
321::
302:.
274::
266::
236:.
210:.
Text is available under the Creative Commons Attribution-ShareAlike License. Additional terms may apply.