Featured Research

from universities, journals, and other organizations

Views you can use? How online ratings affect your judgment

Date:
August 8, 2013
Source:
Massachusetts Institute of Technology
Summary:
Positive comments create an illusory snowball effect, while negative responses get cancelled out.

Are you influenced by the opinions of other people -- say, in the comments sections of websites? If your answer is no, here's another question: Are you sure? A new study shows that positive comments create an illusory snowball effect, while negative responses get cancelled out.
Credit: Illustration: Christine Daniloff/MIT

Are you influenced by the opinions of other people -- say, in the comments sections of websites? If your answer is no, here's another question: Are you sure?

A new study co-authored by an MIT professor suggests that many people are, in fact, heavily influenced by the positive opinions other people express online -- but are much less swayed by negative opinions posted in the same venues. Certain topics, including politics, see much more of this "herding" effect than others.

The results, published today in the journal Science, detail a five-month experiment conducted on a major news-aggregation web site. The research group systematically altered the favorability ratings given to certain comments on the site, to see how perceptions of favorability affected people's judgment about those comments. They found that comments whose ratings were manipulated in a favorable direction saw their popularity snowball, receiving a 25 percent higher average rating from other site users.

"This herding behavior happens systematically on positive signals of quality and ratings," says Sinan Aral, an associate professor at the MIT Sloan School of Management, and one of three authors of the study. At the same time, Aral notes, the results "were asymmetric between positive and negative herding." Comments given negative ratings attracted more negative judgments, but that increase was drowned out by what the researchers call a "correction effect" of additional positive responses.

"People are more skeptical of negative social influence," Aral says. "They're more likely to 'correct' a negative vote and give it a positive vote."

While this phenomenon of social positivity sounds pleasant enough on the surface, Aral warns that there are pitfalls to it, such as the manipulation of online ratings by some political operatives, marketers or anyone who stands to profit by creating an exaggerated appearance of popularity.

"These positive ratings also represent bias and inflation," Aral says. "The housing bubble was a spread of positivity, but when it burst, some people lost their savings and their houses went underwater. Stock bubbles represent a positive herding, and they can be dramatically bad in the wrong context."

Still, the experiment also revealed topical limitations in herding: Stories under the rubrics of "politics," "culture and society" and "business" generated positive herding, but stories posted under the topics of "economics," "IT," "fun" and "general news" did not.

More wisdom about crowds

In turn, Aral suggests, we should be as analytical as possible when it comes to harnessing collective judgments.

"We have to be careful about the design and analysis of systems that try to aggregate the wisdom of crowds," Aral says.

The research was conducted by Lev Muchnik of the Hebrew University of Jerusalem; Sean Taylor of New York University; and Aral, who joined MIT this summer.

The experiment was conducted on a news-aggregation site whose identity the researchers cannot disclose for legal reasons, although Aral allows that it operates along the lines of popular sites such as Reddit. Over the five-month period, the researchers randomly manipulated the ratings given to 101,281 comments to the site. In this way, they could see how readers evaluated the same comments when those comments were given different ratings.

This approach was necessary, Aral points out, because in most circumstances, "It's hard to distinguish the effect of high quality from the effect of social influence bias. It could be that past positive ratings have snowballed to create a high score, or it could just be that those items likely to get high scores are just of high quality."

The researchers also found that comments manipulated to have positive ratings were 32 percent more likely than untreated comments to receive a favorable rating from the very next viewer of those comments, and 30 percent more likely than untreated comments to obtain a very high favorable rating.

Positive ratings for the research -- but more is needed

Other researchers who are familiar with the experiment say it is valuable, and helps suggest additional studies that could be conducted.

Duncan Watts of Microsoft Research, who has conducted pioneering work on social influence, believes the study is "well done," and says it is an "interesting finding that positive manipulations persist whereas negative ones tend to be counteracted by 'friends' stepping in to correct them."

That asymmetry would be more explicable, Watts suggests, if the site in question had a general positive bias in its comment threads overall. "One question is whether the positive bias is specific to this site, or is a general feature of social feedback," he adds.

Both Watts and Matthew O. Jackson, an economist at Stanford University, say that more research is also needed to explore why opinion herding might be more prevalent in certain areas of life but not others.

"What accounts for the fact that some topics exhibit herding while others don't?" Jackson asks. "Is it related to the subjectivity of the issue, or to emotional content of an issue? This provides a rich agenda for both theory and empirical studies of herding going forward."

For his part, Aral agrees that the experiment "opens up as many questions as it answers." He suggests that it would also be valuable to have more work "explaining the psychology of the correction effect on the negative side," as a way of understanding how collective judgments are formed.

"Our message is not that we should do away with crowd-based opinion aggregation," Aral says. "Our point is that you need solid science under the hood trying to understand exactly how these mechanisms work in a broad population, what that means for the diffusion of opinion, and how can we design the systems to be fair, to have less incentives for manipulation and fraud, and be safe in aggregating opinions."

Funding for the research was provided in part by a National Science Foundation Career Award and by a Microsoft Research faculty fellowship.


Story Source:

The above story is based on materials provided by Massachusetts Institute of Technology. The original article was written by Peter Dizikes. Note: Materials may be edited for content and length.


Journal Reference:

  1. L. Muchnik, S. Aral, S. J. Taylor. Social Influence Bias: A Randomized Experiment. Science, 2013; 341 (6146): 647 DOI: 10.1126/science.1240466

Cite This Page:

Massachusetts Institute of Technology. "Views you can use? How online ratings affect your judgment." ScienceDaily. ScienceDaily, 8 August 2013. <www.sciencedaily.com/releases/2013/08/130808142136.htm>.
Massachusetts Institute of Technology. (2013, August 8). Views you can use? How online ratings affect your judgment. ScienceDaily. Retrieved July 26, 2014 from www.sciencedaily.com/releases/2013/08/130808142136.htm
Massachusetts Institute of Technology. "Views you can use? How online ratings affect your judgment." ScienceDaily. www.sciencedaily.com/releases/2013/08/130808142136.htm (accessed July 26, 2014).

Share This




More Science & Society News

Saturday, July 26, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Virginia Governor Tours Tornado Aftermath

Virginia Governor Tours Tornado Aftermath

AP (July 25, 2014) Virginia Gov. Terry McAuliffe toured the Cherrystone Family Camping and RV Resort on the Chesapeake Bay today, a day after it was hit by a tornado. The storm claimed two lives and injured dozens of others. (July 25) Video provided by AP
Powered by NewsLook.com
Richard III's Car Park Burial Site Opens to Public

Richard III's Car Park Burial Site Opens to Public

AFP (July 25, 2014) Visitors will be able to look down from a glass walkway on the grave of King Richard III when a new centre opens in the English cathedral city of Leicester, where the infamous hunchback was found under a car park in 2012. Duration: 00:35 Video provided by AFP
Powered by NewsLook.com
Mobile App Gives Tour of Battle of Atlanta Sites

Mobile App Gives Tour of Battle of Atlanta Sites

AP (July 25, 2014) Emory University's Center for Digital Scholarship has launched a self-guided mobile tour app to coincide with the 150th anniversary of the Civil War's Battle of Atlanta. (July 25) Video provided by AP
Powered by NewsLook.com
University Quiz Implies Atheists Are Smarter Than Christians

University Quiz Implies Atheists Are Smarter Than Christians

Newsy (July 25, 2014) An online quiz from a required course at Ohio State is making waves for suggesting atheists are inherently smarter than Christians. Video provided by Newsy
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:
from the past week

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins