Featured Research

from universities, journals, and other organizations

Computer Program Learns Language Rules And Composes Sentences, All Without Outside Help

September 1, 2005
Cornell University
Shimon Edelman of Cornell University and colleagues have developed a method for enabling a computer program to scan text, infer the grammar behind it and generate new sentences. It works for different languages, music and protein sequences. (PNAS: Vol. 102:33, 2005)

ITHACA, N.Y. -- Cornell University and Tel Aviv Universityresearchers have developed a method for enabling a computer program toscan text in any of a number of languages, including English andChinese, and autonomously and without previous information infer theunderlying rules of grammar. The rules can then be used to generate newand meaningful sentences. The method also works for such data as sheetmusic or protein sequences.

The development -- which has a patentpending -- has implications for speech recognition and for otherapplications in natural language engineering, as well as for genomicsand proteomics. It also offers new insights into language acquisitionand psycholinguistics.

"The algorithm -- the computational method-- for language learning and processing that we have developed can takea body of text, abstract from it a collection of recurring patterns orrules and then generate new material," explained Shimon Edelman, acomputer scientist who is a professor of psychology at Cornell andco-author of a new paper, "Unsupervised Learning of Natural Languages,"published in the Proceedings of the National Academy of Sciences (PNAS,Vol. 102, No. 33).

"This is the first time an unsupervisedalgorithm is shown capable of learning complex syntax, generatinggrammatical new sentences and proving useful in other fields that callfor structure discovery from raw data, such as bioinformatics," he said.

Unlikeprevious attempts at developing computer algorithms for languagelearning, the new method, called Automatic Distillation of Structure(ADIOS), successfully identifies complex patterns in raw texts. Thealgorithm discovers the patterns by repeatedly aligning sentences andlooking for overlapping parts.

For example, the sentences I wouldlike to book a first-class flight to Chicago, I want to book afirst-class flight to Boston and Book a first-class flight for me,please may give rise to the pattern book a first-class flight -- ifthis candidate pattern passes the novel statistical significance testthat is the core of the algorithm.

If the system also encountersthe sentences I need to book a direct flight from New York to Tel AvivandI would like to book an economy flight , it may infer that thephrases first-class, direct and economy are equivalent in the contextof the new pattern. "Because such equivalence sets can contain otherpatterns -- in turn containing further patterns, and so on -- theresulting body of knowledge grows recursively, as a sort of forest ofbranching trees of possibilities," said Edelman.

He added, "ADIOSrelies on a statistical method for pattern extraction and on structuredgeneralization -- two processes that have been implicated in languageacquisition. Our experiments show that it can acquire intricatestructures from raw data, including transcripts of parents' speechdirected at 2- or 3-year-olds. This may eventually help researchersunderstand how children, who learn language in a similar item-by-itemfashion and with very little supervision, eventually master the fullcomplexities of their native tongue."

In addition tochild-directed language, the algorithm has been tested on the full textof the Bible in several languages, on artificial context-free languageswith thousands of rules and on musical notation. It also has beenapplied to biological data, such as nucleotide base pairs and aminoacid sequences. In analyzing proteins, for example, the algorithm wasable to extract from amino acid sequences patterns that were highlycorrelated with the functional properties of the proteins.

Thenew method was developed jointly with David Horn and Eytan Ruppin,professors of physics and computer science, respectively, at Tel AvivUniversity, and with Zach Solan, a doctoral student there and the leadauthor on the paper. Their collaboration with Edelman was supported inpart by the U.S.-Israel Binational Science Foundation.

Story Source:

The above story is based on materials provided by Cornell University. Note: Materials may be edited for content and length.

Cite This Page:

Cornell University. "Computer Program Learns Language Rules And Composes Sentences, All Without Outside Help." ScienceDaily. ScienceDaily, 1 September 2005. <www.sciencedaily.com/releases/2005/09/050901072808.htm>.
Cornell University. (2005, September 1). Computer Program Learns Language Rules And Composes Sentences, All Without Outside Help. ScienceDaily. Retrieved July 31, 2014 from www.sciencedaily.com/releases/2005/09/050901072808.htm
Cornell University. "Computer Program Learns Language Rules And Composes Sentences, All Without Outside Help." ScienceDaily. www.sciencedaily.com/releases/2005/09/050901072808.htm (accessed July 31, 2014).

Share This

More Computers & Math News

Thursday, July 31, 2014

Featured Research

from universities, journals, and other organizations

Featured Videos

from AP, Reuters, AFP, and other news services

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.


Breaking News:
from the past week

In Other News

... from NewsDaily.com

Science News

Health News

    Environment News

    Technology News


      Free Subscriptions

      Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

      Get Social & Mobile

      Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

      Have Feedback?

      Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
      Mobile: iPhone Android Web
      Follow: Facebook Twitter Google+
      Subscribe: RSS Feeds Email Newsletters
      Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins