Featured Research

from universities, journals, and other organizations

A reward is valued more if you choose it yourself: New quirky byproduct of learning from reward

Date:
July 24, 2014
Source:
Brown University
Summary:
Many people value rewards they choose themselves more than rewards they merely receive, even when the rewards are actually equivalent. A new study provides evidence that this long-observed quirk of behavior is a byproduct of how the brain reinforces learning from reward.

For some people, the plums are sweeter if they picked them.
Credit: © photopitu / Fotolia

The price of learning from rewarding choices may be just a touch of self-delusion, according to a new study in Neuron. The research by Brown University brain scientists links a fundamental problem in neuroscience called "credit assignment" -- how the brain reinforces learning only in the exact circuits that caused the rewarding choice -- to an oft-observed quirk of behavior called "choice bias" -- we value the rewards we choose more than equivalent rewards we don't choose. The researchers used computational modeling and behavioral and genetic experiments to discover evidence that choice bias is essentially a byproduct of credit assignment.

Related Articles


"We weren't looking to explain anything about choice bias to start off with," said lead author Jeffrey Cockburn, a graduate student in the research group of senior author Michael Frank, associate professor of cognitive, linguistic and psychological sciences. "This just happened to be the behavioral phenomenon we thought would emerge out of this credit assignment model."

So the next time a friend raves about the movie he chose and is less enthusiastic about the just-as-good one that you chose, you might be able to chalk it up to his basic learning circuitry and a genetic difference that affects it.

Modeled mechanism

The model, developed by Frank, Cockburn and co-author Anne Collins, was based on prior research on the function of the striatum, a part of the brain's basal ganglia (BG) that is principally involved in representing reward values of actions and picking one. "An interaction between three key BG regions moderates that decision making process. When a rewarding choice has been made, the substantia nigra pars compacta (SNc) releases dopamine into the striatum to reinforce connections between cortex and striatum, so that rewarded actions are more likely to be repeated. But how does the SNc reinforce just the circuits that made the right call? The authors proposed a mechanism by which another part of the subtantia nigra, the SNr, detects when actions are worth choosing and then simultaneously amplifies any dopamine signal coming from the SNc."

"The novel part here is that we have proposed a mechanism by which the BG can detect when it has selected an action and should therefore amplify the dopamine reinforcing event specifically at that time," Frank said. "When the SNr decides that striatal valuation signals are strong enough for one action, it releases the brakes not only on downstream structures that allow actions to be executed, but also on the SNc dopamine system, so any unexpected rewards are amplified."

Specifically, dopamine provides reinforcement by enhancing the responsiveness of connections between cells so that a circuit can more easily repeat its rewarding behavior in the future.

But along with that process of reinforcing the action of choosing, the value placed on the resulting reward becomes elevated compared to rewards not experienced this way.

Experimental evidence

That prediction seemed intriguing, but it still had to be tested. The authors, who also included postdoctoral researcher Anne Collins, identified both behavioral and genetic tests that would be telling.

They recruited 80 people at Brown and elsewhere in Providence to play a behavioral game and to donate some saliva for genetic testing.

The game first presented the subjects pictures of arbitrary Japanese characters that would have different probabilities of rewards if chosen ranging from a 20 percent to 80 percent chance of winning a point or losing a point. For some characters, the player could choose a character to discover its resulting reward or penalty, whereas for others, its result was simply given to them. After that learning phase, the subjects were then presented the characters in pairs and instructed to pick the one they thought had the highest chance of winning based on what they had learned.

The researchers built the game so that for every character a player could choose, there was an equally rewarding one had merely been given to them. On average, players showed a clear choice bias in that they were more likely to prefer rewarding characters that they had chosen over equally rewarding characters they had been given.

Notably, they exhibited no choice bias between unrewarding characters suggesting that choice bias emerges only in relation to reward, one of the key predictions of their model. But they wanted to further test whether the impact of reward on choice bias was related to the proposed biological mechanism: that striatal dopaminergic learning is enhanced to chosen rewards.

The genetic tests focused on single-letter differences in a gene called DARPP-32, which governs how well cells in the striatum respond to the reinforcing influence of dopamine.

People with one version of the gene have been shown in previous research to be less able to learn from rewards, while people with other versions were less driven by reward in learning.

"The reason why this gene is interesting is because we know something about the biology of what it does and where it is expressed in the brain," Frank said. "It's predominant in the striatum and specifically affects synaptic plasticity induced by dopamine signaling. It's related to the imbalance to which you learn from really good things, or not so good things.

"The logic was if the mechanism that we think describes this choice bias and credit assignment problem is accurate then that gene should predict the impact of how good something was on this choice bias phenomenon," he said.

Indeed, that's what the data showed. People with the form of the gene that predisposed them to be responsive to big rewards also showed more choice bias from the most strongly rewarded characters. Interestingly, the other people also showed choice bias, but more strongly for those characters that were more mediocre. This pattern was mirrored by the authors' model when it simulated the effects of DARPP-32 on reward learning imbalances from positive vs. negative outcomes.

For some people, the plums are sweeter if they picked them.


Story Source:

The above story is based on materials provided by Brown University. Note: Materials may be edited for content and length.


Journal Reference:

  1. Jeffrey Cockburn, Anne G.E. Collins, Michael J. Frank. A Reinforcement Learning Mechanism Responsible for the Valuation of Free Choice. Neuron, 2014; DOI: 10.1016/j.neuron.2014.06.035

Cite This Page:

Brown University. "A reward is valued more if you choose it yourself: New quirky byproduct of learning from reward." ScienceDaily. ScienceDaily, 24 July 2014. <www.sciencedaily.com/releases/2014/07/140724124431.htm>.
Brown University. (2014, July 24). A reward is valued more if you choose it yourself: New quirky byproduct of learning from reward. ScienceDaily. Retrieved October 31, 2014 from www.sciencedaily.com/releases/2014/07/140724124431.htm
Brown University. "A reward is valued more if you choose it yourself: New quirky byproduct of learning from reward." ScienceDaily. www.sciencedaily.com/releases/2014/07/140724124431.htm (accessed October 31, 2014).

Share This



More Mind & Brain News

Friday, October 31, 2014

Featured Research

from universities, journals, and other organizations


Featured Videos

from AP, Reuters, AFP, and other news services

Studying Effects of Music on Dementia Patients

Studying Effects of Music on Dementia Patients

AP (Oct. 30, 2014) — The University of Wisconsin-Milwaukee is studying the popular Music and Memory program to see if music, which helps improve the mood of Alzheimer's patients, can also reduce the use of prescription drugs for those suffering from dementia. (Oct. 30) Video provided by AP
Powered by NewsLook.com
Techy Tots Are Forefront of London's Baby Show

Techy Tots Are Forefront of London's Baby Show

AP (Oct. 28, 2014) — Moms and Dads get a more hands-on approach to parenting with tech-centric products for raising their little ones. (Oct. 28) Video provided by AP
Powered by NewsLook.com
Cocoa Could Be As Good For Memory As It Is For A Sweet Tooth

Cocoa Could Be As Good For Memory As It Is For A Sweet Tooth

Newsy (Oct. 27, 2014) — Researchers have come up with another reason why dark chocolate is good for your health. A substance in the treat can reportedly help with memory. Video provided by Newsy
Powered by NewsLook.com
Five-Year-Olds Learn Coding as Britain Eyes Digital Future

Five-Year-Olds Learn Coding as Britain Eyes Digital Future

AFP (Oct. 27, 2014) — Coding has become compulsory for children as young as five in schools across the UK. Making it the first major world economy to overhaul its IT teaching and put programming at its core. Duration: 02:19 Video provided by AFP
Powered by NewsLook.com

Search ScienceDaily

Number of stories in archives: 140,361

Find with keyword(s):
 
Enter a keyword or phrase to search ScienceDaily for related topics and research stories.

Save/Print:
Share:  

Breaking News:

Strange & Offbeat Stories

 

Health & Medicine

Mind & Brain

Living & Well

In Other News

... from NewsDaily.com

Science News

Health News

Environment News

Technology News



Save/Print:
Share:  

Free Subscriptions


Get the latest science news with ScienceDaily's free email newsletters, updated daily and weekly. Or view hourly updated newsfeeds in your RSS reader:

Get Social & Mobile


Keep up to date with the latest news from ScienceDaily via social networks and mobile apps:

Have Feedback?


Tell us what you think of ScienceDaily -- we welcome both positive and negative comments. Have any problems using the site? Questions?
Mobile iPhone Android Web
Follow Facebook Twitter Google+
Subscribe RSS Feeds Email Newsletters
Latest Headlines Health & Medicine Mind & Brain Space & Time Matter & Energy Computers & Math Plants & Animals Earth & Climate Fossils & Ruins