Featured Research

from universities, journals, and other organizations

Crowdsourcing nutrition in a snap: Counting calories in photos, PlateMate proves the wisdom of the (well-managed) crowd

Date:
November 2, 2011
Source:
Harvard University
Summary:
If keeping a food diary seems like too much effort, despair not: Computer scientists ahave devised a tool that lets you snap a photo of your meal and let the crowd do the rest. PlateMate's calorie estimates have proved, in tests, to be just as accurate as those of trained nutritionists, and more accurate than the user's own logs.

Computations and algorithms cannot yet evaluate a meal, but it turns out that they can build an effective workforce. The PlateMate project proves that a well-managed crowd can play the role of a nutritional expert.
Credit: Image courtesy of Eric Hysen

Americans spend upwards of $40 billion a year on dieting advice and self-help books, but the first step in any healthy eating strategy is basic awareness -- what's on the plate.

If keeping a food diary seems like too much effort, despair not: computer scientists at the Harvard School of Engineering and Applied Sciences (SEAS) have devised a tool that lets you snap a photo of your meal and let the crowd do the rest.

PlateMate's calorie estimates have proved, in tests, to be just as accurate as those of trained nutritionists, and more accurate than the user's own logs. The research was presented at the 24th ACM Symposium on User Interface Software and Technology, a leading conference on human-computer interaction.

"We can take things that used to require experts and do them with crowds," says Jon Noronha '11, who co-developed PlateMate as an undergraduate at Harvard and now works at Microsoft. "Estimating the nutritional value of a meal is a fairly complex task, from a computational standpoint, but with a structured workflow and some cultural awareness, we've expanded what crowdsourcing can achieve."

When Noronha and his classmate Eric Hysen '11 were looking for a real-world challenge to explore, healthy eating was an obvious choice.

"Nutrition is such a pervasive issue in our society, from counting calories at the dinner table to burning them on the treadmill," says Hysen, who now works at Google. "People worry about whether they're doing the right thing. It seemed like a really good opportunity for crowdsourcing to make a difference."

Often, individuals who claim they are trying to lose weight will underestimate their caloric intake, so PlateMate's advantage is that it allows the user to quickly consult impartial observers, without having to pay for the advice and supervision of an expert nutritionist.

Reproducing the accuracy of an expert in a crowd of untrained strangers, however, was not straightforward.

"Computer scientists normally focus on the computational aspects of a problem, but the HR issues of working with crowds can be just as challenging," says Krzysztof Gajos, Assistant Professor of Computer Science at SEAS and the students' adviser.

PlateMate works in coordination with Amazon Mechanical Turk, a system originally intended to help improve product listings on Amazon.com. Turkers, as the crowd workers call themselves, receive a few cents for each puzzle-like task they complete.

PlateMate divides nutrition analysis into several iterative tasks, asking groups of Turkers to distinguish between foods in the photo, identify what they are, and estimate quantities. The nutrition totals for the meal are then automatically calculated.

The researchers did encounter some common-sense problems with sending photographs to strangers without any context. A latte made with whole milk looks no different than one made with skim milk, a fast-food burger might pack in more calories than one cooked at home, and a close-up photo of a bag of chips could indicate either a sample-sized snack or a late-night binge on a bag designed for 12.

Early tests also identified some cultural limitations. Overseas Turkers routinely identified a burger bun with ketchup as a muffin with jam.

Even after restricting the tests to American workers, Noronha and Hysen discovered that portions of chicken were being characterized as "chicken feet," again and again. The puzzling result drew their attention to another significant and common problem in crowdsourcing: worker laziness. "Chicken feet" was simply the first option in a list of chicken-related foods, so lazy Turkers were just clicking it and moving on to a new task.

Noronha and Hysen solved these problems by designing simple, clearly defined tasks, and algorithms that compare several answers, selecting the best one. They provided warnings about common errors, and vetted their Turkers to weed out those with a history of poor work.

The resulting tool is easier and more accurate than keeping a food diary and cheaper than consulting a nutritionist.

"Just taking pictures won't make you healthier," warns Gajos. "You have to actually reflect on this information. You have to be motivated to change. But if you have this motivation, then PlateMate will make it easier for you to follow through."

In the future, he suggests, some of the contextual problems could be avoided by pairing the photos with location data.

Intended primarily as a foray into the capabilities of human-computer systems, PlateMate may not solve, once and for all, the challenge of eating well. It is, however, one of the first attempts to use multiple human-computational approaches to solve a very complex, real-world problem.

"A lot of prior crowdsourcing research has been about making crowds do things that we wish computers could do, like shorten an 800-word essay to 500 words and have it still make sense," explains Noronha. "That's something computers can almost do, but it's just beyond their reach."

"What makes the nutrition application so interesting as a problem in crowdsourcing is that computers are so very far away from doing it on their own -- because food is such a human thing."

Computations and algorithms cannot yet evaluate a meal, but it turns out that they can build an effective workforce. The PlateMate project proves that a well-managed crowd can play the role of an expert, and that opens the door to a wealth of new opportunities.

"Any problem that can be broken down into logical steps is a great candidate for crowdsourcing," says Haoqi Zhang '07, a doctoral candidate at SEAS who brought crowdsourcing expertise to the PlateMate project. "The only question is, what would you like the crowd to do for you?"

"Take, for example, comparing travel packages or making slides for a presentation -- most people spend a lot of time on that kind of thing, but if you can effectively organize a crowd to help, it's like having an expert as a trusty assistant, ready to help at all times."


Story Source:

The above story is based on materials provided by Harvard University. Note: Materials may be edited for content and length.


Cite This Page:

Harvard University. "Crowdsourcing nutrition in a snap: Counting calories in photos, PlateMate proves the wisdom of the (well-managed) crowd." ScienceDaily. ScienceDaily, 2 November 2011. <www.sciencedaily.com/releases/2011/11/111101171040.htm>.
Harvard University. (2011, November 2). Crowdsourcing nutrition in a snap: Counting calories in photos, PlateMate proves the wisdom of the (well-managed) crowd. ScienceDaily. Retrieved July 22, 2014 from www.sciencedaily.com/releases/2011/11/111101171040.htm
Harvard University. "Crowdsourcing nutrition in a snap: Counting calories in photos, PlateMate proves the wisdom of the (well-managed) crowd." ScienceDaily. www.sciencedaily.com/releases/2011/11/111101171040.htm (accessed July 22, 2014).

Share This




More Health & Medicine News

Tuesday, July 22, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Courts Conflicted Over Healthcare Law

Courts Conflicted Over Healthcare Law

AP (July 22, 2014) Two federal appeals courts issued conflicting rulings Tuesday on the legality of the federally-run healthcare exchange that operates in 36 states. (July 22) Video provided by AP
Powered by NewsLook.com
Why Do People Believe We Only Use 10 Percent Of Our Brains?

Why Do People Believe We Only Use 10 Percent Of Our Brains?

Newsy (July 22, 2014) The new sci-fi thriller "Lucy" is making people question whether we really use all our brainpower. But, as scientists have insisted for years, we do. Video provided by Newsy
Powered by NewsLook.com
Scientists Find New Way To Make Human Platelets

Scientists Find New Way To Make Human Platelets

Newsy (July 22, 2014) Boston scientists have discovered a new way to create fully functioning human platelets using a bioreactor and human stem cells. Video provided by Newsy
Powered by NewsLook.com
Gilead's $1000-a-Pill Drug Could Cure Hep C in HIV-Positive People

Gilead's $1000-a-Pill Drug Could Cure Hep C in HIV-Positive People

TheStreet (July 21, 2014) New research shows Gilead Science's drug Sovaldi helps in curing hepatitis C in those who suffer from HIV. In a medical study, the combination of Gilead's Hep C drug with anti-viral drug Ribavirin cured 76% of HIV-positive patients suffering from the most common hepatitis C strain. Hepatitis C and related complications have been a top cause of death in HIV-positive patients. Typical medication used to treat the disease, including interferon proteins, tended to react badly with HIV drugs. However, Sovaldi's %1,000-a-pill price tag could limit the number of patients able to access the treatment. TheStreet's Keris Lahiff reports from New York. Video provided by TheStreet
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:
from the past week

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins