CERN Accelerating science

Aggregating Labels in Crowdsourcing Data

Date published: 
Tuesday, 1 September, 2015
Document type: 
Summer student report
Author(s): 
M. Priisalu
Crowdsourcing is a method in which multiple individuals with possibly no prior knowledge in the field solve a number of tasks. The solutions given by the individuals are then aggregated to infer the true solution from the common knowledge of the individuals. In this paper we give a short overview of some of the aggregation methods and hybrid crowdsourcing solutions used. We then implement the label aggregation model proposed by Dawid and Skene [Dawid 1979] for open source and open science websites such as Crowdcrafting.org [Lombraña, 2015] and the UNOSAT project GeoTagX [UNOSAT, 2015]. Finally we also discuss the organization and results of the CERN Webfest 2015, a hackathon for CERN Summer Students.