Featured Research

from universities, journals, and other organizations

Applying information theory to linguistics

Date:
October 10, 2012
Source:
Massachusetts Institute of Technology
Summary:
Researchers believe that information theory -- the discipline that gave us digital communication -- can explain differences between human languages.

The majority of languages -- roughly 85 percent of them -- can be sorted into two categories: those, like English, in which the basic sentence form is subject-verb-object ("the girl kicks the ball"), and those, like Japanese, in which the basic sentence form is subject-object-verb ("the girl the ball kicks").

The reason for the difference has remained somewhat mysterious, but researchers from MIT's Department of Brain and Cognitive Sciences now believe that they can account for it using concepts borrowed from information theory, the discipline, invented almost singlehandedly by longtime MIT professor Claude Shannon, that led to the digital revolution in communications. The researchers will present their hypothesis in an upcoming issue of the journal Psychological Science.

Shannon was largely concerned with faithful communication in the presence of "noise" -- any external influence that can corrupt a message on its way from sender to receiver. Ted Gibson, a professor of cognitive sciences at MIT and corresponding author on the new paper, argues that human speech is an example of what Shannon called a "noisy channel."

"If I'm getting an idea across to you, there's noise in what I'm saying," Gibson says. "I may not say what I mean -- I pick up the wrong word, or whatever. Even if I say something right, you may hear the wrong thing. And then there's ambient stuff in between on the signal, which can screw us up. It's a real problem." In their paper, the MIT researchers argue that languages develop the word order rules they do in order to minimize the risk of miscommunication across a noisy channel.

Gibson is joined on the paper by Rebecca Saxe, an associate professor of cognitive neuroscience; Steven Piantadosi, a postdoc at the University of Rochester who did his doctoral work with Gibson; Leon Bergen, a graduate student in Gibson's group; research affiliate Eunice Lim; and Kimberly Brink, who graduated from MIT in 2010.

Mixed signals

The researchers' hypothesis was born of an attempt to explain the peculiar results of an experiment reported in the Proceedings of the National Academy of Sciences in 2008; Brink reproduced the experiment as a class project for a course taught by Saxe. In the experiment, native English speakers were shown crude digital animations of simple events and asked to describe them using only gestures. Oddly, when presented with events in which a human acts on an inanimate object, such as a girl kicking a ball, volunteers usually attempted to convey the object of the sentence before trying to convey the verb -- even though, in English, verbs generally precede objects. With events in which a human acts on another human, such as a girl kicking a boy, however, the volunteers would generally mime the verb before the object.

"It's not subtle at all," Gibson says. "It's about 70 percent each way, so it's a shift of about 40 percent."

The tendency even of speakers of a subject-verb-object (SVO) language like English to gesture subject-object-verb (SOV), Gibson says, may be an example of an innate human preference for linguistically recapitulating old information before introducing new information. The "old before new" theory -- which, according to the University of Pennsylvania linguist Ellen Price, is also known as the given-new, known-new, and presupposition-focus theory -- has a rich history in the linguistic literature, dating back to at least the work of the German philosopher Hermann Paul, in 1880.

Imagine, for instance, the circumstances in which someone would actually say, in ordinary conversation, "the girl kicked the ball." Chances are, the speaker would already have introduced both the girl and the ball -- say, in telling a story about a soccer game. The sole new piece of information would be the fact of the kick.

Assuming a natural preference for the SOV word order, then -- at least in cases where the verb is the new piece of information -- why would the volunteers in the PNAS experiments mime SVO when both the subject and the object were people? The MIT researchers' explanation is that the SVO ordering has a better chance of preserving information if the communications channel is noisy.

Suppose that the sentence is "the girl kicked the boy," and that one of the nouns in the sentence -- either the subject or the object -- will be lost in transmission. If the word order is SOV, then the listener will receive one of two messages: either "the girl kicked" or "the boy kicked." If the word order is SVO, however, the two possible messages on the receiving end are "the girl kicked" and "kicked the boy": More information will have made it through the noisy channel.

Down to cases

That is the MIT researchers' explanation for the experimental findings reported in the 2008 PNAS paper. But how about the differences in word order across languages? A preliminary investigation, Gibson says, suggests that there is a very strong correlation between word order and the strength of a language's "case markings." Case marking means that words change depending on their syntactic function: In English, for instance, the pronoun "she" changes to "her" if the kicker becomes the kicked. But case marking is rare in English, and English is an SVO language. Japanese, a strongly case-marked language, is SOV. That is, in Japanese, there are other cues as to which noun is subject and which is object, so Japanese speakers can default to their natural preference for old before new.

Gibson adds that, in fact, some languages have case markings only for animate objects -- an observation that accords particularly well with the MIT researchers' theory.

"It's an extremely valuable study," says Steven Pinker, the Johnstone Family Professor in the Department of Psychology at Harvard University. "The design of any language reflects a compromise between properties that make it more useful -- clarity, expressiveness, ease of articulation -- and properties that are standardized across a community of speakers so that everyone is using the same code. Most grammatical theorists have focused on the arbitrary nature of the community-wide grammar. Gibson has now shed light on how each of these grammars has evolved, in a few predictable ways, to maximize clarity in communicating who did what to whom. That is, much more can be said than just 'That's the way English is; that's the way Turkish is,' and so on. Gibson's study shows that there is a great deal of functional design in seemingly arbitrary patterns of variation across languages."

In order to make their information-theoretical model of word order more rigorous, Gibson says, he and his colleagues need to better characterize the "noise characteristics" of spoken conversation -- what types of errors typically arise, and how frequent they are. That's the topic of ongoing experiments, in which the researchers gauge people's interpretations of sentences in which words have been deleted or inserted.


Story Source:

The above story is based on materials provided by Massachusetts Institute of Technology. The original article was written by Larry Hardesty. Note: Materials may be edited for content and length.


Journal Reference:

  1. E. Gibson, S.T. Piantadosi, K. Brink, L. Bergen, E. Lim, and R. Saxe. A noisy-channel account of crosslinguistic word order variation. Psychological Science, (accepted) 2012

Cite This Page:

Massachusetts Institute of Technology. "Applying information theory to linguistics." ScienceDaily. ScienceDaily, 10 October 2012. <www.sciencedaily.com/releases/2012/10/121010131534.htm>.
Massachusetts Institute of Technology. (2012, October 10). Applying information theory to linguistics. ScienceDaily. Retrieved October 20, 2014 from www.sciencedaily.com/releases/2012/10/121010131534.htm
Massachusetts Institute of Technology. "Applying information theory to linguistics." ScienceDaily. www.sciencedaily.com/releases/2012/10/121010131534.htm (accessed October 20, 2014).

Share This



More Mind & Brain News

Monday, October 20, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Your Birth Season Might Determine Your Temperament

Your Birth Season Might Determine Your Temperament

Newsy (Oct. 20, 2014) A new study says the season you're born in can determine your temperament — and one season has a surprising outcome. Video provided by Newsy
Powered by NewsLook.com
Court Ruling Means Kids' Online Activity Could Be On Parents

Court Ruling Means Kids' Online Activity Could Be On Parents

Newsy (Oct. 17, 2014) In a ruling attorneys for both sides agreed was a first of its kind, a Georgia appeals court said parents can be held liable for what kids put online. Video provided by Newsy
Powered by NewsLook.com
The Best Foods To Boost Your Mood

The Best Foods To Boost Your Mood

Buzz60 (Oct. 17, 2014) Feeling down? Reach for the refrigerator, not the medicine cabinet! TC Newman (@PurpleTCNewman) shares some of the best foods to boost your mood. Video provided by Buzz60
Powered by NewsLook.com
You Can Get Addicted To Google Glass, Apparently

You Can Get Addicted To Google Glass, Apparently

Newsy (Oct. 15, 2014) Researchers claim they’ve diagnosed the first example of the disorder in a 31-year-old U.S. Navy serviceman. Video provided by Newsy
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:

Breaking News:

Strange & Offbeat Stories


Health & Medicine

Mind & Brain

Living & Well

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile: iPhone Android Web
Follow: Facebook Twitter Google+
Subscribe: RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins