Featured Research

from universities, journals, and other organizations

Mathematicians Prove New Way To Build A Better Estimate

Date:
March 2, 2008
Source:
Brown University
Summary:
Brown applied mathematicians have found a new way to sift through mountains of data and draw reliable inferences from it -- a Holy Grail in science and technology. Their pioneering work, the development of a new class of statistical estimators, could lead to better methods for analyzing the large data sets that are increasingly common in fields from biology to business.

How do you sift through hundreds of billions of bits of information and make accurate inferences from such gargantuan sets of data? Brown University mathematician Charles “Chip” Lawrence and graduate student Luis Carvalho have arrived at a fresh answer with broad applications in science, technology and business.

In new work published in the Proceedings of the National Academy of Sciences, Lawrence and Carvalho describe a new class of statistical estimators and prove four theorems concerning their properties. Their work shows that these “centroid” estimators allow for better statistical predictions – and, as a result, better ways to extract information from the immense data sets used in computational biology, information technology, banking and finance, medicine and engineering.

“What’s exciting about this work – what makes it every scientist’s dream – is that it’s so fundamental,” Lawrence said. “These new estimators have applications in biology and beyond and they advance a statistical method that’s been around for decades.”

For more than 80 years, one of the most common methods of statistical prediction has been maximum likelihood estimation (MLE). This method is used to find the single most probable solution, or estimate, from a set of data.

But new technologies that capture enormous amounts of data – human genome sequencing, Internet transaction tracking, instruments that beam high-resolution images from outer space – have opened opportunities to predict discrete “high dimensional” or “high-D” unknowns. The huge number of combinations of these “high-D” unknowns produces enormous statistical uncertainty. Data has outgrown data analysis.

This discrepancy creates a paradox. Instead of producing more precise predictions about gene activity, shopping habits or the presence of faraway stars, these large data sets are producing more unreliable predictions, given current procedures. That’s because maximum likelihood estimators use data to identify the single most probable solution. But because any one data point swims in an increasingly immense sea, it’s not likely to be representative.

Lawrence, a professor of applied mathematics and a faculty member in the Center for Computational Molecular Biology at Brown, first came upon this paradox and a potential way around it while working on predicting the structure of RNA molecules. If you want to predict the structure of these molecules – how the molecule will look when it folds onto itself – you’d have billions and billions of possible shapes to choose from.

“Using maximum likelihood estimation, the most likely outcome would be very, very, very unlikely,” Lawrence said, “so we knew we needed a better estimation method.”

Lawrence and Carvahlo used statistical decision theory to understand the limitations of the old procedure when faced with new “high-D” problems. They also used statistical decision-making theory to find an estimation procedure that applies to a broad range of statistical problems. These “centroid” estimators identify not the single most probable solution, but the solution that is most representative of all the data in a set.

Lawrence and Carvahlo went on to prove four theorems that illustrate the favorable properties of these estimators and show that they can be easily computed in many important applications.

“This new procedure should benefit any field that needs to reliably make predictions of large-scale, high-D unknowns,” Lawrence said.

The U.S. Department of Energy and the National Institutes of Health funded the work.


Story Source:

The above story is based on materials provided by Brown University. Note: Materials may be edited for content and length.


Journal Reference:

  1. Luis E. Carvalho and Charles E. Lawrence. Centroid estimation in discrete high-dimensional spaces with applications in biology. Proceedings of the National Academy of Sciences, 105: 3209-3214 DOI: 10.1073/pnas.0712329105

Cite This Page:

Brown University. "Mathematicians Prove New Way To Build A Better Estimate." ScienceDaily. ScienceDaily, 2 March 2008. <www.sciencedaily.com/releases/2008/02/080229090817.htm>.
Brown University. (2008, March 2). Mathematicians Prove New Way To Build A Better Estimate. ScienceDaily. Retrieved April 23, 2014 from www.sciencedaily.com/releases/2008/02/080229090817.htm
Brown University. "Mathematicians Prove New Way To Build A Better Estimate." ScienceDaily. www.sciencedaily.com/releases/2008/02/080229090817.htm (accessed April 23, 2014).

Share This



More Computers & Math News

Wednesday, April 23, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

High Court to Hear Dispute of TV Over Internet

High Court to Hear Dispute of TV Over Internet

AP (Apr. 22, 2014) The future of Aereo, an online service that provides over-the-air TV channels, hinges on a battle with broadcasters that goes before the U.S. Supreme Court on Tuesday. (April 22) Video provided by AP
Powered by NewsLook.com
Aereo Takes on Broadcast TV Titans in Supreme Court Today

Aereo Takes on Broadcast TV Titans in Supreme Court Today

TheStreet (Apr. 22, 2014) Aereo heads to the Supreme Court today to fight for its right to stream broadcast TV over the Internet -- against broadcasters who say the start-up infringes upon copyright law. TheStreet Deputy Managing Editor Leon Lazaroff explains the importance of the case in the TV industry and details what the outcome of it could mean for broadcasters and for cloud storage services -- as Aereo allows its subscribers to not just watch live TV shows but also store content to a DVR in the cloud. Video provided by TheStreet
Powered by NewsLook.com
Lytro Introduces 'Illum,' A Professional Light-Field Camera

Lytro Introduces 'Illum,' A Professional Light-Field Camera

Newsy (Apr. 22, 2014) The light-field photography engineers at Lytro unveiled their next innovation: a professional DSLR-like camera called "Illum." Video provided by Newsy
Powered by NewsLook.com
Netflix To Raise Prices For New Subscribers

Netflix To Raise Prices For New Subscribers

Newsy (Apr. 21, 2014) Netflix executives say they don't think a $1 or $2 price hike will hurt the service, and they have their sites set on overtaking HBO. Video provided by Newsy
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:
from the past week

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins