Featured Research

from universities, journals, and other organizations

Making crowdsourcing more reliable

Date:
October 10, 2012
Source:
University of Southampton
Summary:
Researchers in the UK are designing incentives for collection and verification of information to make crowdsourcing more reliable.

Researchers from the University of Southampton are designing incentives for collection and verification of information to make crowdsourcing more reliable.

Crowdsourcing is a process of outsourcing tasks to the public, rather than to employees or contractors. In recent years, crowdsourcing has provided an unprecedented ability to accomplish tasks that require the involvement of a large number of people, often across wide-spread geographies, expertise, or interests.

The world's largest encyclopaedia, Wikipedia, is an example of a task that can only be achieved through crowd participation. Crowdsourcing is not limited to volunteer efforts. For example, Amazon Mechanical Turk (AMT) and CrowdFlower are 'labour on demand' markets that allow people to get paid for micro-tasks, as simple as labelling an image or translating a piece of text.

Recently, crowdsourcing has demonstrated effectiveness in large-scale, information-gathering tasks, across very wide geographies. For example, the Ushahidi platform allowed volunteers to perform rapid crisis mapping in real-time in the aftermath of disasters such as the Haiti earthquake.

One of the main obstacles in crowdsourcing information gathering is reliability of collected reports. Now Dr Victor Naroditskiy and Professor Nick Jennings from the University of Southampton, together with Masdar Institute's Professor Iyad Rahwan and Dr Manuel Cebrian, Research Scientist at the University of California, San Diego (UCSD), have developed novel methods for solving this problem through crowdsourcing. The work, which is published in the academic journal PLOS ONE, shows how to crowdsource not just gathering, but also verification of information.

Dr Victor Naroditskiy of the Agents, Interaction and Complexity group at the University of Southampton, and lead author of the paper, says: "The success of an information gathering task relies on the ability to identify trustworthy information reports, while false reports are bound to appear either due to honest mistakes or sabotage attempts. This information verification problem is a difficult task, which, just like the information-gathering task, requires the involvement of a large number of people."

Sites like Wikipedia have existing mechanisms for quality assurance and information verification. However, those mechanisms rely partly on reputation, as more experienced editors can check whether an article conforms to the Wikipedia objectivity criteria, has sufficient citations, etc. In addition, Wikipedia has policies for resolving conflicts between editors in cases of disagreement.

However, in time-critical tasks, there is no established hierarchy of participants, and little basis for judging credibility of volunteers who are recruited on the fly. In this kind of scenario, special incentives are needed to carry out verification. The research presented in the PLOS ONE paper provides such incentives.

Professor Iyad Rahwan of Masdar Institute in Abu Dhabi and a co-author of the paper, explains: "We showed how to combine incentives to recruit participants to verify information. When a participant submits a report, the participant's recruiter becomes responsible for verifying its correctness. Compensations to the recruiter and to the reporting participant for submitting the correct report, as well as penalties for incorrect reports, ensure that the recruiter will perform verification."

Incentives to recruit participants have previously been proposed by Dr Manuel Cebrian from UCSD, and a co-author of the paper, to win the DARPA Red Balloon Challenge, where teams had to locate 10 weather balloons positioned at random locations throughout the United States. In that scheme, where the person who found the balloons received a pre-determined compensation, for example $1,000, his recruiter received $500 and the recruiter of the recruiter got $250. Dr Manuel Cebrian says: "The results on incentives to encourage verification provide theoretical justification for the incentives used to win the Red Balloon Challenge."


Story Source:

The above story is based on materials provided by University of Southampton. Note: Materials may be edited for content and length.


Cite This Page:

University of Southampton. "Making crowdsourcing more reliable." ScienceDaily. ScienceDaily, 10 October 2012. <www.sciencedaily.com/releases/2012/10/121010171946.htm>.
University of Southampton. (2012, October 10). Making crowdsourcing more reliable. ScienceDaily. Retrieved October 23, 2014 from www.sciencedaily.com/releases/2012/10/121010171946.htm
University of Southampton. "Making crowdsourcing more reliable." ScienceDaily. www.sciencedaily.com/releases/2012/10/121010171946.htm (accessed October 23, 2014).

Share This



More Science & Society News

Thursday, October 23, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Chameleon Camouflage to Give Tanks Cloaking Capabilities

Chameleon Camouflage to Give Tanks Cloaking Capabilities

Reuters - Innovations Video Online (Oct. 22, 2014) Inspired by the way a chameleon changes its colour to disguise itself; scientists in Poland want to replace traditional camouflage paint with thousands of electrochromic plates that will continuously change colour to blend with its surroundings. The first PL-01 concept tank prototype will be tested within a few years, with scientists predicting that a similar technology could even be woven into the fabric of a soldiers' clothing making them virtually invisible to the naked eye. Matthew Stock reports. Video provided by Reuters
Powered by NewsLook.com
Jet Sales Lift Boeing Profit 18 Pct.

Jet Sales Lift Boeing Profit 18 Pct.

Reuters - Business Video Online (Oct. 22, 2014) Strong jet demand has pushed Boeing to raise its profit forecast for the third time, but analysts were disappointed by its small cash flow. Fred Katayama reports. Video provided by Reuters
Powered by NewsLook.com
U.S. Issues Ebola Travel Restrictions, Are Visa Bans Next?

U.S. Issues Ebola Travel Restrictions, Are Visa Bans Next?

Newsy (Oct. 22, 2014) Now that the U.S. is restricting travel from West Africa, some are dropping questions about a travel ban and instead asking about visa bans. Video provided by Newsy
Powered by NewsLook.com
US to Track Everyone Coming from Ebola Nations

US to Track Everyone Coming from Ebola Nations

AP (Oct. 22, 2014) Stepping up their vigilance against Ebola, federal authorities said Wednesday that everyone traveling into the US from Ebola-stricken nations will be monitored for symptoms for 21 days. (Oct. 22) Video provided by AP
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:

Strange & Offbeat Stories


Science & Society

Business & Industry

Education & Learning

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins