scholarly journals Subgoal- and Goal-related Reward Prediction Errors in Medial Prefrontal Cortex

2019 ◽  
Vol 31 (1) ◽  
pp. 8-23 ◽  
Author(s):  
José J. F. Ribas-Fernandes ◽  
Danesh Shahnazian ◽  
Clay B. Holroyd ◽  
Matthew M. Botvinick
Keyword(s):  
Prefrontal Cortex ◽  
Medial Prefrontal Cortex ◽  
Animal Behavior ◽  
Prediction Error ◽  
Prediction Errors ◽  
Task Context ◽  
Reward Prediction ◽  
Human Participants ◽  
Superordinate Goals ◽  
Different Levels

A longstanding view of the organization of human and animal behavior holds that behavior is hierarchically organized—in other words, directed toward achieving superordinate goals through the achievement of subordinate goals or subgoals. However, most research in neuroscience has focused on tasks without hierarchical structure. In past work, we have shown that negative reward prediction error (RPE) signals in medial prefrontal cortex (mPFC) can be linked not only to superordinate goals but also to subgoals. This suggests that mPFC tracks impediments in the progression toward subgoals. Using fMRI of human participants engaged in a hierarchical navigation task, here we found that mPFC also processes positive prediction errors at the level of subgoals, indicating that this brain region is sensitive to advances in subgoal completion. However, when subgoal RPEs were elicited alongside with goal-related RPEs, mPFC responses reflected only the goal-related RPEs. These findings suggest that information from different levels of hierarchy is processed selectively, depending on the task context.

scholarly journals Subgoal- and Goal-Related Prediction Errors in Medial Prefrontal Cortex

2018 ◽  
Author(s):  
José J. F. Ribas Fernandes ◽  
Danesh Shahnazian ◽  
Clay B. Holroyd ◽  
Matthew M. Botvinick
Keyword(s):  
Prefrontal Cortex ◽  
Medial Prefrontal Cortex ◽  
Animal Behavior ◽  
Prediction Error ◽  
Ventral Striatum ◽  
Prediction Errors ◽  
Task Context ◽  
Past Work ◽  
Superordinate Goals ◽  
Different Levels

AbstractA longstanding view of the organization of human and animal behavior holds that behavior is hierarchically organized, meaning that it can be understood as directed towards achieving superordinate goals through subordinate goals, or subgoals. For example, the superordinate goal of making coffee can be broken down as accomplishing a series of subgoals, namely boiling water, grinding coffee, pouring cream, etc.Learning and behavioral adaptation depend on prediction-error signals, which have been observed in ventral striatum (VS) and medial prefrontal cortex (mPFC). In past work, we have shown that prediction error signals (PEs) can be linked not only to superordinate goals, but also to subgoals.Here we present two functional magnetic resonance imagining experiments that replicate and extend these findings. In the first experiment, we replicated the finding that mPFC signals subgoal-related PEs, independently of goal PEs. Together with our past work, this experiment reveals that BOLD responses to PEs in mPFC are unsigned. In the second experiment, we showed that when a task involves both goal and subgoal PEs, mPFC shows only goal-related PEs, suggesting that context or attention can strongly impact hierarchical PE coding. Furthermore, we observed a dissociation between the coding of PEs in mPFC and VS. These experiments suggest that the mPFC selectively attends to information at different levels of hierarchy depending on the task context.

Mediofrontal Negativity Signals Unexpected Timing of Salient Outcomes

2017 ◽  
Vol 29 (4) ◽  
pp. 718-727 ◽  
Author(s):  
Sara Garofalo ◽  
Christopher Timmermann ◽  
Simone Battaglia ◽  
Martin E. Maier ◽  
Giuseppe di Pellegrino
Keyword(s):  
Prefrontal Cortex ◽  
Computational Model ◽  
Medial Prefrontal Cortex ◽  
Prediction Error ◽  
Critical Role ◽  
Aversive Conditioning ◽  
Prediction Errors ◽  
Per Se ◽  
Future Outcomes

The medial prefrontal cortex (mPFC) and ACC have been consistently implicated in learning predictions of future outcomes and signaling prediction errors (i.e., unexpected deviations from such predictions). A computational model of ACC/mPFC posits that these prediction errors should be modulated by outcomes occurring at unexpected times, even if the outcomes themselves are predicted. However, unexpectedness per se is not the only variable that modulates ACC/mPFC activity, as studies reported its sensitivity to the salience of outcomes. In this study, mediofrontal negativity, a component of the event-related brain potential generated in ACC/mPFC and coding for prediction errors, was measured in 48 participants performing a Pavlovian aversive conditioning task, during which aversive (thus salient) and neutral outcomes were unexpectedly shifted (i.e., anticipated or delayed) in time. Mediofrontal ERP signals of prediction error were observed for outcomes occurring at unexpected times but were specific for salient (shock-associated), as compared with neutral, outcomes. These findings have important implications for the theoretical accounts of ACC/mPFC and suggest a critical role of timing and salience information in prediction error signaling.

scholarly journals Reward prediction error in the ERP following unconditioned aversive stimuli

2021 ◽  
Vol 11 (1) ◽  
Author(s):  
Harry J. Stewardson ◽  
Thomas D. Sambrook
Keyword(s):  
Reinforcement Learning ◽  
Prediction Error ◽  
Dopamine Neurons ◽  
Prediction Errors ◽  
Temporal Difference ◽  
Dopamine System ◽  
Reward Prediction Error ◽  
Reward Prediction ◽  
Midbrain Dopamine ◽  
Human Participants

AbstractReinforcement learning in humans and other animals is driven by reward prediction errors: deviations between the amount of reward or punishment initially expected and that which is obtained. Temporal difference methods of reinforcement learning generate this reward prediction error at the earliest time at which a revision in reward or punishment likelihood is signalled, for example by a conditioned stimulus. Midbrain dopamine neurons, believed to compute reward prediction errors, generate this signal in response to both conditioned and unconditioned stimuli, as predicted by temporal difference learning. Electroencephalographic recordings of human participants have suggested that a component named the feedback-related negativity (FRN) is generated when this signal is carried to the cortex. If this is so, the FRN should be expected to respond equivalently to conditioned and unconditioned stimuli. However, very few studies have attempted to measure the FRN’s response to unconditioned stimuli. The present study attempted to elicit the FRN in response to a primary aversive stimulus (electric shock) using a design that varied reward prediction error while holding physical intensity constant. The FRN was strongly elicited, but earlier and more transiently than typically seen, suggesting that it may incorporate other processes than the midbrain dopamine system.

scholarly journals The Medial Prefrontal Cortex Shapes Dopamine Reward Prediction Errors under State Uncertainty

Neuron ◽  
2018 ◽  
Vol 98 (3) ◽  
pp. 616-629.e6 ◽  
Author(s):  
Clara Kwon Starkweather ◽  
Samuel J. Gershman ◽  
Naoshige Uchida
Keyword(s):  
Prefrontal Cortex ◽  
Medial Prefrontal Cortex ◽  
Prediction Errors ◽  
Reward Prediction ◽  
State Uncertainty

Abnormal prefrontal cortex processing of reward prediction errors in recently diagnosed patients with bipolar disorder and their unaffected relatives

2020 ◽  
Vol 22 (8) ◽  
pp. 849-859
Author(s):  
Julian Macoveanu ◽  
Hanne L. Kjærstad ◽  
Henry W. Chase ◽  
Sophia Frangou ◽  
Gitte M. Knudsen ◽  
...  
Keyword(s):  
Bipolar Disorder ◽  
Prefrontal Cortex ◽  
Prediction Errors ◽  
Reward Prediction

scholarly journals Neural Mechanisms of Reward Prediction Error in Autism Spectrum Disorder

2019 ◽  
Vol 2019 ◽  
pp. 1-10 ◽  
Author(s):  
Maya G. Mosner ◽  
R. Edward McLaurin ◽  
Jessica L. Kinard ◽  
Shabnam Hakimi ◽  
Jacob Parelman ◽  
...  
Keyword(s):  
Prediction Error ◽  
Autism Spectrum ◽  
Neural Mechanisms ◽  
Reward Learning ◽  
Prediction Errors ◽  
Frontal Pole ◽  
Neural Basis ◽  
Reward Prediction ◽  
Behavioral Impairments ◽  
Adults With Asd

Few studies have explored neural mechanisms of reward learning in ASD despite evidence of behavioral impairments of predictive abilities in ASD. To investigate the neural correlates of reward prediction errors in ASD, 16 adults with ASD and 14 typically developing controls performed a prediction error task during fMRI scanning. Results revealed greater activation in the ASD group in the left paracingulate gyrus during signed prediction errors and the left insula and right frontal pole during thresholded unsigned prediction errors. Findings support atypical neural processing of reward prediction errors in ASD in frontostriatal regions critical for prediction coding and reward learning. Results provide a neural basis for impairments in reward learning that may contribute to traits common in ASD (e.g., intolerance of unpredictability).

scholarly journals Electrophysiological correlates of reward prediction error recorded in the human prefrontal cortex

2005 ◽  
Vol 102 (23) ◽  
pp. 8351-8356 ◽  
Author(s):  
H. Oya ◽  
R. Adolphs ◽  
H. Kawasaki ◽  
A. Bechara ◽  
A. Damasio ◽  
...  
Keyword(s):  
Prefrontal Cortex ◽  
Prediction Error ◽  
Reward Prediction Error ◽  
Reward Prediction

scholarly journals Frontal Theta Oscillatory Activity Is a Common Mechanism for the Computation of Unexpected Outcomes and Learning Rate

2014 ◽  
Vol 26 (3) ◽  
pp. 447-458 ◽  
Author(s):  
Ernest Mas-Herrero ◽  
Josep Marco-Pallarés
Keyword(s):  
Prediction Error ◽  
Learning Task ◽  
Learning Rate ◽  
Oscillatory Activity ◽  
Common Mechanism ◽  
Prediction Errors ◽  
Reward Prediction Error ◽  
Reward Prediction ◽  
Medial Pfc ◽  
The Impact

In decision-making processes, the relevance of the information yielded by outcomes varies across time and situations. It increases when previous predictions are not accurate and in contexts with high environmental uncertainty. Previous fMRI studies have shown an important role of medial pFC in coding both reward prediction errors and the impact of this information to guide future decisions. However, it is unclear whether these two processes are dissociated in time or occur simultaneously, suggesting that a common mechanism is engaged. In the present work, we studied the modulation of two electrophysiological responses associated to outcome processing—the feedback-related negativity ERP and frontocentral theta oscillatory activity—with the reward prediction error and the learning rate. Twenty-six participants performed two learning tasks differing in the degree of predictability of the outcomes: a reversal learning task and a probabilistic learning task with multiple blocks of novel cue–outcome associations. We implemented a reinforcement learning model to obtain the single-trial reward prediction error and the learning rate for each participant and task. Our results indicated that midfrontal theta activity and feedback-related negativity increased linearly with the unsigned prediction error. In addition, variations of frontal theta oscillatory activity predicted the learning rate across tasks and participants. These results support the existence of a common brain mechanism for the computation of unsigned prediction error and learning rate.

scholarly journals Beyond Reward Prediction Errors: Human Striatum Updates Rule Values During Learning

2017 ◽  
Author(s):  
Ian Ballard ◽  
Eric M. Miller ◽  
Steven T. Piantadosi ◽  
Noah Goodman ◽  
Samuel M. McClure
Keyword(s):  
Prediction Error ◽  
Rule Learning ◽  
Prediction Errors ◽  
Rule Generation ◽  
Functional Magnetic Resonance ◽  
Resonance Imaging ◽  
Stimulus Response ◽  
Reward Prediction ◽  
The World ◽  
Explicit Rule

ABSTRACTHumans naturally group the world into coherent categories defined by membership rules. Rules can be learned implicitly by building stimulus-response associations using reinforcement learning (RL) or by using explicit reasoning. We tested if the striatum, in which activation reliably scales with reward prediction error, would track prediction errors in a task that required explicit rule generation. Using functional magnetic resonance imaging during a categorization task, we show that striatal responses to feedback scale with a “surprise” signal derived from a Bayesian rule-learning model and are inconsistent with RL prediction error. We also find that striatum and caudal inferior frontal sulcus (cIFS) are involved in updating the likelihood of discriminative rules. We conclude that the striatum, in cooperation with the cIFS, is involved in updating the values assigned to categorization rules when people learn using explicit reasoning.

Sign in / Sign up

Export Citation Format

Share Document