Approximating the crowd

Date

2014-09-01

Author

Ertekin Bolelli, Şeyda
Hirsh, Haym

Metadata

Show full item record

This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International License.

Item Usage Stats

77
views

0
downloads

The problem of "approximating the crowd" is that of estimating the crowd's majority opinion by querying only a subset of it. Algorithms that approximate the crowd can intelligently stretch a limited budget for a crowdsourcing task. We present an algorithm, "CrowdSense," that works in an online fashion where items come one at a time. CrowdSense dynamically samples subsets of the crowd based on an exploration/exploitation criterion. The algorithm produces a weighted combination of the subset's votes that approximates the crowd's opinion. We then introduce two variations of CrowdSense that make various distributional approximations to handle distinct crowd characteristics. In particular, the first algorithm makes a statistical independence approximation of the labelers for large crowds, whereas the second algorithm finds a lower bound on how often the current subcrowd agrees with the crowd's majority vote. Our experiments on CrowdSense and several baselines demonstrate that we can reliably approximate the entire crowd's vote by collecting opinions from a representative subset of the crowd.

Subject Keywords

Computer Networks and Communications, Information Systems, Computer Science Applications

URI

https://hdl.handle.net/11511/43802

Journal

DATA MINING AND KNOWLEDGE DISCOVERY

DOI

https://doi.org/10.1007/s10618-014-0354-1

Collections

Department of Computer Engineering, Article

Suggestions

OpenMETU
Core

Approximating the wisdom of the crowd Ertekin Bolelli, Şeyda; Rudin, Cynthia (2011-12-17) The problem of “approximating the crowd” is that of estimating the crowd’s majority opinion by querying only a subset of it. Algorithms that approximate the crowd can intelligently stretch a limited budget for a crowdsourcing task. We present an algorithm, “CrowdSense,” that works in an online fashion to dynamically sample subsets of labelers based on an exploration/exploitation criterion. The algorithm produces a weighted combination of the labelers’ votes that approximates the crowd’s opinion.
The Augustin Capacity and Center Nakiboğlu, Barış (Pleiades Publishing Ltd, 2019-10-01) For any channel, the existence of a unique Augustin mean is established for any positive order and probability mass function on the input set. The Augustin mean is shown to be the unique fixed point of an operator defined in terms of the order and the input distribution. The Augustin information is shown to be continuously differentiable in the order. For any channel and convex constraint set with finite Augustin capacity, the existence of a unique Augustin center and the associated van Erven-Harremoes boun...
Selective sampling of labelers for approximating the crowd Ertekin Bolelli, Şeyda; Rudin, Cynthia (null; 2012-11-05) In this paper, we present CrowdSense, an algorithm for estimating the crowd’s majority opinion by querying only a subset of it. CrowdSense works in an online fashion where examples come one at a time and it dynamically samples subsets of labelers based on an exploration/exploitation criterion. The algorithm produces a weighted combination of a subset of the labelers’ votes that approximates the crowd’s opinion. We also present two probabilistic variants of CrowdSense that are based on different assumptions ...
Search for Boolean functions with excellent profiles in the rotation symmetric class Kavut, Selcuk; Maitra, Subhamoy; Yucel, Melek D. (Institute of Electrical and Electronics Engineers (IEEE), 2007-05-01) For the first time Boolean functions on 9 variables having nonlinearity 241 are discovered, that remained as an open question in literature for almost three decades. Such functions are found by heuristic search in the space of rotation symmetric Boolean functions (RSBFs). This shows that there exist Boolean functions on n (odd) variables having non, linearity > 2(n-1) - 2 (n-1/2) if and only if n > 7. Using similar search technique, balanced Boolean functions on 9, 10, and 11 variables are attained having a...
"Keep it simple!": an eye-tracking study for exploring complexity and distinguishability of web pages for people with autism Eraslan, Sukru; Yesilada, Yeliz; Yaneva, Victoria; Ha, Le An (Springer Science and Business Media LLC, 2020-02-03) A major limitation of the international well-known standard web accessibility guidelines for people with cognitive disabilities is that they have not been empirically evaluated by using relevant user groups. Instead, they aim to anticipate issues that may arise following the diagnostic criteria. In this paper, we address this problem by empirically evaluating two of the most popular guidelines related to the visual complexity of web pages and the distinguishability of web-page elements. We conducted a compa...

Citation Formats

Ş. Ertekin Bolelli and H. Hirsh, “Approximating the crowd,” DATA MINING AND KNOWLEDGE DISCOVERY, pp. 1189–1221, 2014, Accessed: 00, 2020. [Online]. Available: https://hdl.handle.net/11511/43802.