Cresa.eu

Confidence in value-based choice
Benedetto De Martino1–3,6, Stephen M Fleming2,4–6, Neil Garrett1 & Raymond J Dolan2
Decisions are never perfect, with confidence in one's choices fluctuating over time. How subjective confidence and valuation of choice options interact at the level of brain and behavior is unknown. Using a dynamic model of the decision process, we show that confidence reflects the evolution of a decision variable over time, explaining the observed relation between confidence, value, accuracy and reaction time. As predicted by our dynamic model, we show that a functional magnetic resonance imaging signal in human ventromedial prefrontal cortex (vmPFC) reflects both value comparison and confidence in the value comparison process. Crucially, individuals varied in how they related confidence to accuracy, allowing us to show that this introspective ability is predicted by a measure of functional connectivity between vmPFC and rostrolateral prefrontal cortex. Our findings provide a mechanistic link between noise in value comparison and metacognitive awareness of choice, enabling us both to want and to express knowledge of what we want.
The subjective confidence we have in our decision-making, and that confidence from value, and in so doing provide evidence that confi- of others, has far-reaching consequences. For example, the recom- dence reflects an assessment of choice accuracy.
mendations of a financial advisor who expresses high confidence in To explore systematic relationships between confidence, accuracy, a particular investment option will carry more weight than one who choice and reaction time, we modeled our data using a variant of a All rights reserved.
is ambivalent. An expression of doubt in or caution concerning a par- race mode (one of a larger class of dynamic models of decision- ticular course of action can lead one to question or revisit a previous making. This model predicts that subjective confidence reflects decision. Previous work has established that the vmPFC has a central the stochastic accumulation of evidence during the value comparison role in computing the value of potential choice option, with activ- process. As is consistent with this prediction, we show that the same ity in this region reflecting the dynamic evolution of a value com- anatomical region in ventromedial prefrontal cortex (vmPFC) not only America, Inc.
parison However, this work has focused exclusively on the choice reflects a difference in value between available options, but also the process, without considering the subject's level of confidence in the confidence associated with a value comparison process. Final y, we decision. Consequently, it is unknown how a process of value com- show that individual differences in participants' abilities to relate con- parison, instantiated in vmPFC, relates to subjective confidence.
fidence to decision performance is linked to increased functional con- Previous studies have reported neural correlates of decision con- nectivity between vmPFC and rostrolateral prefrontal cortex (RLPFC), 2013 Nature
fidence in brain regions associated with a value representation. For a region previously shown to function in metacognitive appraisal example, firing rates in rat orbitofrontal corte and functional magnetic resonance imaging (fMRI) signal in human vmPFCshow RESULTS graded changes as perceptual decisions become more difficult. We scanned twenty hungry participants while they made choices However, as these studies delineate confidence in terms of factors gov- between food items that they could consume later . After erning choice, they are unable to tease apart the relationship between making each choice, participants reported the degree of confidence in trial-to-trial subjective confidence and decision value. In contrast, the their decision (choice confidence). Note that confidence, or certainty, field of perceptual decision-making has noted that confidence can in the present study is conceptually distinct from risk, in that each be measured independently of the choice process itself where it choice determined a known outcome. Confidence here reflects the is conceptualized as reflecting a ‘second-order' metacognitive evalu- degree of subjective certainty in having made the best choice, which ation. Critically, dissociating confidence from other features of the equates to choosing the higher valued item. To establish value for indi- decision process requires acquisition of separate measures of choice vidual items, we asked participants at the end of the scanning session and confidenc.
to place a bid for each food item using a standard incentive-compat- Here we implement such an approach to dissociate value and confi- ible procedure, the Becker-DeGroot-Marschak (BDM) mechanis. dence during decision-making and to identify their respective neural BDM is widely used in behavioral economics and neuroeconomics substrates. We col ected trial-by-trial estimates of decision confidence to elicit nonstrategic reservation prices, also known as willingness- while healthy volunteers chose between pairs of snack items. We also to-pay. In this phase subjects were required to state their maximum measured the subjective value of each snack item by means of a standard willingness-to-pay for each food item (see Online Methods). Several incentive-compatible bidding procedure. This allowed us to dissociate studies have shown that this mechanism reliably elicits goal values 1Psychology and Language Sciences, University College London, London, UK. 2Wellcome Trust Center for Neuroimaging, at University College London, UK. 3Division of the Humanities and Social Sciences, California Institute of Technology, Pasadena, California, USA. 4Center for Neural Science, New York University, New York, New York, USA. 5Department of Experimental Psychology, University of Oxford, Oxford, UK. 6These authors contributed equally to this work. Correspondence should be addressed to B.D.M. ([email protected]) or S.M.F. ([email protected]).
Received 6 August; accepted 15 November; published online 9 December NATURE NEUROSCIENCE VOLUME 16 NUMBER 1 JANUARY 2013 Figure 1 Task and behavioral results. (a) fMRI Confidence rating task (top): subjects were presented with a choice between two snacks and were then required to High confidenceAll data choose (2.5 s) one item to consume at the end of the experiment. After each choice, subjects indicated their level of confidence in having made a correct decision (choice confidence). Confidence rating Post-scanning task (bottom): subjects were (chose right item) presented with each item individually and had to submit a bid to buy each item. After each bid, they were asked to rate their level of confidence in having provided a correct bid price (bid Value right item–value left item confidence). (b) Probability of choosing the item on the right as a function of DV (that is, bid price) between the two items (logistic fit) for an exemplar subject (see Supplementary Fig. 2 for all individual subjects). Dashed line, all choices; black line, low-confidence choices; gray line, high-confidence choices. The red double-headed arrow indicates the increase in choice accuracy (change in slope) for high- versus low-confidence trials used in the between-subjects analyses (). (c) The slope of the logistic Reaction time (quantile) fit is systematically higher (sharper) in high-confidence compared to low-confidence trials (**P < 0.005; ***P < 0.0001). (d) Average RT as a function of confidence and DV . (e) Heat map showing mean z-scored confidence (color bar) across subjects, as a function of subject-specific DV and RT quantiles. Error bars represent s.e.m.
that are used by the decision maker to guide choice. Participants model 2 provided a better account (that is, lower Bayesian information also provided a rating of their confidence in each bid (bid confi- criterion (BIC)) of participants' choices than the other four models All rights reserved.
dence). Participants' bids for the leftmost items were then subtracted (, as shown by the difference in BIC relative to model 2: model from bids for the rightmost items to calculate a signed difference 1, 214.6; model 3, 196.2; model 4, 251.7; model 5, 111.9. Furthermore, in value (DV) between each pair of items, which was then entered model 2 was a better fit than the canonical model 1 in 19 of 20 par- into a logistic regression to predict the probability that the subject ticipants as assessed by a likelihood ratio test (A = 0.05). This analysis chose the rightmost item on each trial (. In line with previous confirms that a critical modulator of choice accuracy is second-order America, Inc.
studie we found that DV was a reliable predictor of participants' confidence arising in the context of the comparison process (model 2) choices, with the slope of the logistic regression being a measure of as opposed to first-order confidence in the item values (models 3–5).
choice accuracy, or noise in the choice process Stability of confidence over time Choice, confidence and reaction time We next examined whether the relationship between confidence and 2013 Nature
Unsigned DV only accounted for an average of 17.7% of the vari- choice was stable over time. Splitting the logistic regression analysis into ance in participants' confidence ratings (r = 0.42 o 0.19, s.d.). This separate sessions revealed a robust main effect of confidence (F1,19 = partial independence between confidence and DV al owed us to ask 39.75; P < 0.0001) but a nonsignificant main effect of session (F3,57 = whether confidence reflects changes in choice accuracy (the selec- 0.3; P = 0.7) and a lack of interaction between session and confidence tion of items with higher subjective value). By splitting our logistic (F3,57 = 0.13; P = 0.9; Supplementary Fig. 2). To examine whether local
regression fit into high- and low-confidence trials, we showed that fluctuations in attention affected confidence, we constructed a serial higher confidence was consistently associated with increased choice autocorrelation regression model that predicted the current confidence accuracy nd Supplementary Fig. 1). This effect of confi- rating from the confidence ratings given on the immediately preceding
dence on choice was also reflected in reaction time (RT), with main five trials, in addition to DV . None of the autocorrelation coefficients effects of both DV and confidence (both P < 0.001) but no interac- reached group-level significance (al t < 1.2, P > 0.27). Together these tion (). The three-way relationship between DV , confidence results indicate that confidence is a stable predictor of choice accuracy and RT is plotted inWe recognize that other factors (inter- and that it does not reflect local changes in attention.
nal and external) besides DV and RT are likely to affect subjective As each item pairing was presented twice (once in each spatial con- confidence. We report a limited set of these factors (Supplementary figuration), it was also possible to examine the relationship between
Table 1) for which we could exercise good experimental control.
confidence ratings given for identical choice pairs. As confidence is Using logistic regression, we next compared models of the interaction partly determined by absolute difference in value ( DV , which does between confidence and value comparison. Choice confidence, unlike not vary across choice pairs), we expected some stability purely driven by DV, is in itself not a predictor of choice (right or left item) but instead DV. Thus, to address this question, we computed the partial correlation refers to accuracy of the decision. We thus expected choice confidence between first and second confidence ratings, control ing for DV. There to modulate the link between DV and choice. Model 1 predicted choice was no significant difference between mean confidence ratings for the using DV alone; model 2 included choice confidence (that is, confi- first and second presentations of the same item pairs (t19 = −0.64, P = dence at the decision time) as a modulator of DV (DV × confidence); 0.53). For 19 of 20 subjects, there was a significant partial correlation (P <
models 3–5 examined whether bid confidence (that is, confidence at 0.05) between confidence ratings for repeated item pairs after control ing the bid time) could explain additional variance in the link between DV for the influence of DV , indicating stability in confidence for judgments and choice (see Online Methods). In accordance with our predictions, of particular item pairs that cannot be accounted for by DV alone.
VOLUME 16 NUMBER 1 JANUARY 2013 NATURE NEUROSCIENCE

(chose right item) Reaction time (a.u.) Low High Low High Value right item–value left item Model 2Model 3Model 4Model 5 Figure 2 Computational model. (a) Comparison of regression models. Plotted are BIC scores (model 1: 5,424; model 2: 4,995; model 3: 5,388; model 4: 5,498; model 5: 5,291). Smaller numbers indicate a better model fit. See Results and Online Methods for details of each model. (b) Dynamic (race) model of value comparison for an example trial. Evidence in favor of each option accumulates over time, with a choice in favor of one or other option being made when threshold is reached. In this model, decision confidence is derived from the absolute difference between the two accumulators at the time of the decision ($e). a.u., arbitrary units. (c–e) Model predictions. (c) When $e is large (that is, high confidence) choice accuracy is predicted to increase, reflected by a sharper curve in the logistic regression. (d) RT values are predicted to decrease when either DV or $e increase. (e) Matrix representing how model Reaction time (quantile) confidence changes across DV and RT quantiles. Note the similarity between the model predictions and behavior ().
We then examined whether choices were stable over time. On aver- inter-trial noise in the value comparison process results in some trials age, 14.7% of choices (o 5.7% s.d.) were reversed on the second pres- having greater final DV values (higher confidence) than others. Such entation. Choices that would be subsequently reversed were associated decisions wil tend to be made more quickly, be more accurate and be with significantly lower initial confidence than those that would sub- associated with higher confidence ). Indeed, this predicted rela- All rights reserved.
sequently be repeated (in arbitrary units: reversal confidence = 3.11 tionship among RT, DV and confidence closely matched the behavioral o 0.72 (s.d.); repetition confidence = 4.40 o 0.54 (s.d.); t19 = 12.1, data ). Final y, since the model predicts that confidence reflects P < 10−10). In a logistic regression model predicting subsequent the stochastic evolution of a value comparison process, it wil only be reversal from both DV and initial confidence, initial confidence weakly related to initial DV. This feature of the model provides a parsi- was a significant negative predictor of choice reversal (mean stand- monious explanation for why DV and confidence are dissociable in our America, Inc.
ardized regression coefficient −0.99 o 0.40 (s.d.); one-sample behavioral data.
t-test t19 = −11.2, P < 10−9). These data support a hypothesis that low confidence is associated with subsequent changes of mind.
Confidence and value in vmPFCWe next hypothesized that if choice confidence is an emergent prop- erty of a value comparison process, the same brain regions involved 2013 Nature
Our best-fitting regression model suggested that confidence reflects accuracy in a value comparison. This led us to explore in more detail the precise mechanism by which confi- dence and value interact during the decision process. We adapted a race model wherein evidence in favor of each of the options (the snacks presented on the left and right sides of the screen) is accumulated over time and the decision is made on the basis of the first option to reach a threshold . In this model, confidence is defined as the absolute difference between the two accumulators at decision time ($e). Such a model predicts Percentage signal change in vmPFC that when $e is large, then choice accuracy is increased, reflected by a sharper slope in the logistic regression (. Thus, the Figure 3 vmPFC. (a) Brain activity in precuneus and vmPFC. vmPFC (Montreal Neurological race model neatly accounts for an increase in Institute (MNI) space coordinates (x, y, z) = (12, 56, 4); circled) correlating with increases in DV choice accuracy we observed behavioral y in between the two items presented (P < 0.05, FWE corrected at cluster level). Left image, sagittal the high-confidence condition and section; right image, coronal. (b) Brain activity in precuneus and vmPFC (12, 47, −11) correlating Supplementary Table 2). Furthermore this with increases in subjective confidence (P < 0.05 FWE corrected at the cluster level). (c) Signal in
model predicts a decrease in RT when either vmPFC (6-mm sphere centered at the peak of the DV main effect (12, 56, 4)), showing main effects of DV and degree of confidence in the absence of an interaction. The plot (extracted from GLM 2; DV or $e are increased (), as seen in see Online Methods) is shown only to clarify the signal pattern in vmPFC (that is, lack of interaction the behavioral data (. The intuition is between confidence and DV) and to confirm statistical inference (from GLM 1) regarding the main that, even within a particular level of initial DV, effects of DV and confidence. *P < 0.01. Error bars represent s.e.m.
NATURE NEUROSCIENCE VOLUME 16 NUMBER 1 JANUARY 2013

Figure 4 RLPFC. (a) Brain activity in rRLPFC correlating with decreases in subjective confidence (P < 0.005, small-volume FWE corrected). Coronal section; R, right. (b) Signal in rRLPFC (6-mm sphere MNI space coordinates (x, y, z) = (39, 41, 16)) showing a main effect of confidence but not DV. The plot (extracted from GLM 2; see Online Methods) is shown only to clarify the signal pattern in rRLPFC (that is, absence of main effect of DV). n.s., not significant. (c) Between-subjects regression analysis considering the change in choice accuracy (slope of the logistic fit) between low- and high-confidence trials (see red double-headed arrow in as a covariate for confidence-related activity in rRLPFC (peak (x, y, z) = (27, 44, 16); P < 0.05, small-volume FWE corrected). The scatter plot is not Percentage signal change in rRLPFC used for statistical inference (which was carried out in the SPM framework); it is shown solely for illustrative purposes. Error bars represent s.e.m.
of DV and confidence in vmPFC but no interaction between them (2 × 2 ANOVA with factors value, confidence: main effect of value F1,19 = 5.1, P < 0.05; main effect of confidence F1,19 = 7.6, P < 0.05; interaction F1,19 = 0.7, P > 0.5)). The absence of an inter- action at the neural level is consistent with a theoretical independence between value and noise in the choice process, such that one can have high confidence in a low-value choice and vice versa. Furthermore, Parameter estimates in rRLPFC (high confidence–low confidence) the pattern across conditions closely resembles that seen for RT Choice accuracy (high confidence–low confidence) values providing convergent evidence that vmPFC activity is tightly linked to behavior. We also confirmed that the response to confidence was not driven by a categorical response to error in value-based decision-making should also represent subjective con- (Supplementary Fig. 3).
fidence in a value estimate. In other words, if a brain region involved in value comparison is implementing a process akin to a race modelConfidence in right rostrolateral prefrontal cortex All rights reserved.
then activity in that region should be modulated by both initial DV A key question is how confidence-related information represented and noise (confidence) on that trial. To test this hypothesis, we con- in vmPFC becomes available for self-report. One computationally structed a general linear model (GLM) of our fMRI data in which plausible hypothesis is a hierarchical model wherein confidence in a each trial was modulated by two parametric regressors: DV and comparison process is ‘read out' by an anatomically distinct second- confidence orthogonalized with respect to DV . We show that activ- order networ Right rostrolateral prefrontal cortex (rRLPFC) is America, Inc.
ity in vmPFC was indeed modulated by both value and confidence a likely candidate, as this region is implicated in metacognitive assess- (nd Supplementary Table 3; P < 0.05 family-wise error ments of perceptual decision Consequently, we tested whether
(FWE) corrected at cluster level). This pattern is consistent with the this region acts more general y in metacognitive appraisal by enabling established function of this region in encoding goal-valand explicit report of confidence in a value comparison.
with our hypothesis that this region also represents the confidence We first established that rRLPFC tracked changes in reported con- 2013 Nature
associated with a value comparison.
fidence but did not code for DV Supplementary Fig. 4 and
We next investigated whether DV and confidence interacted in Supplementary Table 3; P < 0.005, small-volume corrected (SVC)),
vmPFC by splitting the model into high- and low-confidence trials, as expected for a region providing a readout of decision confidence. both parametrically modulated by DV (. This analysis showed main effects Figure 5 Connectivity analysis. (a) PPI analysis. Left: three-dimensional rendering, radiological orientation. Right: sagittal section. vmPFC (circled in black in right panel) shows increases in connectivity with a region of rRLPFC (6-mm sphere (x, y, z) = (39, 41, 16); blue, left panel) previously Parameter estimates in vmPFC PPI analysis with rRLPFC seed identified as being modulated by confidence (vmPFC peak (x, y, z) = (9, 50, −11); R, right; A, anterior. P < 0.05, small-volume FWE corrected). 3D rendering made in mricron (radiological orientation). R, right. A, anterior. (b) Between-subjects regression analysis considering the increase in choice accuracy (see red double- headed arrow between high-confidence and low-confidence conditions as a covariate for the modulation of connectivity (vmPFC peak (x, y, z) = (15, 56, −5); P < 0.05, small-volume FWE corrected). Coronal section. The scatter plot was Parameter estimates in vmPFC PPI connectivity with rRLPFC –4 –3 –2 –1 0 not used for statistical inference (which was carried (high confidence–low confidence) out in the SPM framework); it is shown solely for Choice accuracy (high confidence–low confidence) illustrative purposes. Error bars represent s.e.m.
VOLUME 16 NUMBER 1 JANUARY 2013 NATURE NEUROSCIENCE

Figure 6 Schematic of network relating confidence to subjective report. Summary of the relationship between our computational model and neuroimaging analyses. (a) Confidence in the decision ($e) emerges from the value comparison process instantiated in vmPFC. (b) To reach metacognitive awareness (and be reported by the participant), this information is transferred to rRLPFC. The parameter S conf governs the noise in the readout of $e (that is, decision confidence). If Sconf is zero, the information about confidence ($e) is uncorrupted, resulting in a pronounced shift in the choice accuracy between high-confidence and low-confidence trials (red double-headed arrows). As the level of metacognitive noise increases (higher values Sconf) the shift between the two curves (low and high confidence) diminishes. Differences in Sconf account for the inter-subject variability in metacognitive reportability we observed behaviorally. a.u., arbitrary units.
(chose right item)P 0 Value right item– Value right item– Value right item– We next harnessed individual differences in metacognition to provide a more stringent test for the role of rRLPFC. We defined an indivi- dual's metacognitive accuracy as the change in choice accuracy (slope DISCUSSION of the logistic fit) between low- and high-confidence trials ). Here we show that decision confidence emerges from a value com- We reasoned that if rRLPFC acts in the metacognitive appraisal of parison process in vmPFC and that this region is in turn accessed confidence, activity in this region and/or its coupling with vmPFC by rRLPFC to enable a subjective assessment of confidence. Our All rights reserved.
should predict this change in slope across individuals. To test our neural findings are consistent with previous evidence showing that first prediction, we entered change in slope as a between-subjects choice difficulty is coded by vmPFC in humans and analogous OFC covariate in the whole-brain analysis of confidence-related activity, neurons in rodents. There is also an established body of work finding that this parameter significantly modulated the response to showing that this brain area represents the expected value of an out- confidence in rRLPFC (P < 0.05; SVC for multiple comparisons). come. However, as previous studies defined confidence in terms America, Inc.
In other words, participants manifested a neurometric-psychometric of factors governing choice, they were unable to tease apart the match between their behavioral and neural responses to change in relationship between value and confidence. Our results go beyond confidence level ().
these studies by dissociating subjective confidence from DV. In so doing, we demonstrate that neural activity in the same anatomical Metacognitive access: interaction between vmPFC and rRLPFC region represents both variables, suggesting that confidence and 2013 Nature
To test our second prediction, that these two regions are part of the DV are separate behavioral manifestations of the same underlying same functional network (in the context of our task), we performed decision variable.
a psychophysiological interaction (PPI) analysis using rRLPFC Choice confidence can be seen to emerge from the dynamics of as a seed . This analysis revealed a robust modulation of noisy accumulators in the race model leading to dual effects connectivity between rRLPFC and vmPFC (P < 0.05 small-volume of DV and RT on confidence The race model has previously been FWE corrected) by confidence level (). Furthermore, the proposed to account for decision confidence in perceptual decision- strength of connectivity between these two regions also predicted making. In keeping with recent research efforts that have incorpo- metacognitive accuracy across subjects (vmPFC; P < 0.05; SVC for rated dynamic models into the field of economic decision-makin, multiple comparisons) (. Thus, both activity in rRLPFC itself we find that this model captures several features of the relationship and its coupling strength with vmPFC influenced the degree to which between choice, RT and confidence in a value-based choice model. confidence was effectively read out for metacognitive report.
The separation between confidence and BDM values in the present How might this readout process relate to our computational model study provides a new perspective on how an underlying decision vari- of confidence? Intuitively, if reported confidence is a noisy facsimile able can be fractionated into distinct behavioral components. Given of the confidence inherent in a decision process, the relationship that DV and confidence had independent effects on vmPFC activity, between confidence and behavior will weaken and metacognitive this result provides convergent support for the idea that vmPFC acts accuracy will decreas. We were able to modify the race model, as a dynamic accumulator of choice values Our findings also accord introduced previously, to account for the inter-subject variability in with a theoretical Bayesian scheme in which uncertainty, or precision, metacognitive reports observed experimentally. We introduced an is an inherent property of the neural code additional parameter (Sconf) governing the noise in the read-out of $e A central problem for computational models of metacognition is (that is, decision confidence) computed during the value comparison. how confidence information is read out for appraisal and communica- Variation in this parameter captured variability in the change in slope tion to others. It has been proposed that such a computation can between high- and low-confidence conditions, despite overal choice be achieved by a two-layer neural network architecture, in which the accuracy remaining equal (). Together with our imaging results, second-order network receives information about the performance of this analysis suggests that rRLPFC may indeed mediate variability in the first-order network and uses this information to generate reports reported confidence (see and Discussion).
of confidence. Our fMRI data can be interpreted in this framework NATURE NEUROSCIENCE VOLUME 16 NUMBER 1 JANUARY 2013 and suggests that rRLPFC is a plausible locus for this second-order COMPETING FINANCIAL INTERESTS
network. First, rRLPFC represented confidence but not DV, as pre- The authors declare no competing financial interests.
dicted for a brain region that has access to information about confi- dence but is not directly involved in value comparison. Second, both Published online at . Reprints and permissions information is available online at confidence-related activity in rRLPFC and coupling between rRLPFC and vmPFC predicted the relationship between confidence and accu- racy across individuals. This result can be explained if the coupling 1. Rangel, A. & Hare, T. Neural computations associated with goal-directed choice. between vmPFC and rRLPFC reflects the fidelity with which reported Curr. Opin. Neurobiol. 20, 262–270 (2010).
confidence tracks the evolution of a putative accumulator process 2. FitzGerald, T.H.B., Seymour, B. & Dolan, R.J. The role of human orbitofrontal cortex in value comparison for incommensurable objects. J. Neurosci. 29, 8388–8395 in vmPFC (. Notably, confidence-related activity in rRLPFC is also seen in perceptual decision-makin, together with 3. Kable, J.W. & Glimcher, P.W. The neural correlates of subjective value during a modulation of connectivity with visual cortex. This pattern of find- intertemporal choice. Nat. Neurosci. 10, 1625–1633 (2007).
4. Basten, U., Biele, G., Heekeren, H.R. & Fiebach, C.J. How the brain integrates ings suggests that rRLPFC might have a domain-general role in meta- costs and benefits during decision making. Proc. Natl. Acad. Sci. USA 107, cognitive evaluation of decision-making, supporting the notion of a segregated neural process governing metacognitive access.
et al. Distinct value signals in anterior and posterior ventromedial prefrontal cortex. J. Neurosci. 30, 2490–2495 (2010).
An alternative interpretation of our data is that information about 6. Hunt, L.T. et al. Mechanisms underlying cortical activity during value-guided choice. choice confidence is coded elsewhere, perhaps in parallel to the con- Nat. Neurosci. 15, 470–476 (2012).
7. Kepecs, A., Uchida, N., Zariwala, H.A. & Mainen, Z.F. Neural correlates, computation struction of choice values, and is then communicated to vmPFC (pos- and behavioural impact of decision confidence. Nature 455, 227–231 (2008).
sibly via rRLPFC), where it is incorporated into the choice process. This 8. Rolls, E.T., Grabenhorst, F. & Deco, G. Choice, difficulty, and confidence in the mechanism would be analogous to a modulation of the vmPFC value brain. Neuroimage 53, 694–706 (2010).
9. Fleming, S.M., Weil, R.S., Nagy, Z., Dolan, R.J. & Rees, G. Relating introspective accuracy signal during self-control by dorsolateral PFC Resolving this possibil- to individual differences in brain structure. Science 329, 1541–1543 (2010).
ity is beyond the design of the current study and will require techniques 10. Pleskac, T.J. & Busemeyer, J.R. Two-stage dynamic signal detection: A theory of choice, decision time, and confidence. with high temporal resolution, such as magnetoencephalography, that Psychol. Rev. 117, 864–901 (2010).
11. Kepecs, A. & Mainen, Z. A computational framework for the study of confidence in can track the evolution of confidence and valuation in the brain.
humans and animals. Philos. Trans. R. Soc. Lond. B Biol. Sci. 367, 1322–1337 Our data show that humans have metacognitive access to noise in 12. Vickers, D. Decision Processes in Visual Perception (Academic Press, 1979).
a value comparison and that increased choice accuracy is associated 13. Bogacz, R., Brown, E., Moehlis, J., Holmes, P. & Cohen, J.D. The physics of optimal All rights reserved.
with high subjective confidence. In other words, although choices decision making: a formal analysis of models of performance in two-alternative often appear noisy from the point of view of the experimente forced-choice tasks. Psychol. Rev. 113, 700–765 (2006).
14. Fleming, S.M., Huijgen, J. & Dolan, R.J. Prefrontal contributions to metacognition subjective confidence ratings reveal systematic changes in this noise, in perceptual decision making. J. Neurosci. 32, 6117–6125 (2012).
reflected by changes in choice accuracy. Metacognitive access to con- 15. Becker, G.M., DeGroot, M.H. & Marschak, J. Measuring utility by a single-response sequential method. fidence in a value comparison is likely to be useful for revisiting a Behav. Sci. 9, 226–232 (1964).
16. Plassmann, H., O'Doherty, J. & Rangel, A. Orbitofrontal cortex encodes willingness America, Inc.
choice that did not turn out as expected. Alternatively, but not mutu- to pay in everyday economic transactions. J. Neurosci. 27, 9984–9988 (2007).
ally exclusively, metacognitive access may facilitate communication 17. De Martino, B., Kumaran, D., Holt, B. & Dolan, R.J. The neurobiology of reference- dependent value computation. J. Neurosci. 29, 3833–3842 (2009).
of confidence to other, as when a financial advisor directs a client 18. Hare, T.A., O'Doherty, J., Camerer, C.F., Schultz, W. & Rangel, A. Dissociating the toward one stock option over another.
role of the orbitofrontal cortex and the striatum in the computation of goal values By integrating computational modeling with neural analysis, we pro- and prediction errors. J. Neurosci. 28, 5623–5630 (2008).
19. Boorman, E.D., Behrens, T.E.J., Woolrich, M.W. & Rushworth, M.F.S. How green is 2013 Nature
vide evidence that subjective confidence is integral to the brain's repre- the grass on the other side? Frontopolar cortex and the evidence in favor of sentation of value in the vmPFC. Our work outlines a neural schema alternative courses of action. Neuron 62, 733–743 (2009).
20. Sugrue, L.P., Corrado, G.S. & Newsome, W.T. Choosing the greater of two goods: neural for how confidence-related information is computed and transferred currencies for valuation and decision making. Nat. Rev. Neurosci. 6, 363–375 to a distinct brain region (rRLPFC), supporting metacognitive report. Far from being a blind process of selection corrupted by noise, it would 21. Vickers, D. Evidence for an accumulator model of psychophysical discrimination. Ergonomics 13, 37–58 (1970).
appear that value-based choices are accompanied by fluctuations in 22. Insabato, A., Pannunzi, M., Rolls, E.T. & Deco, G. Confidence-related decision subjective confidence. A metacognitive access to value computation making. J. Neurophysiol. 104, 539–547 (2010).
enables us not only to want but also to know what we want.
23. Lau, H. & Rosenthal, D. Empirical support for higher-order theories of conscious awareness. Trends Cogn. Sci. 15, 365–373 (2011).
24. Pasquali, A., Timmermans, B. & Cleeremans, A. Know thyself: metacognitive networks and measures of consciousness. Cognition 117, 182–190 (2010).
25. Yokoyama, O. Methods and any associated references are available in the et al. Right frontopolar cortex activity correlates with reliability of retrospective rating of confidence in short-term recognition memory performance. Neurosci. Res. 68, 199–206 (2010).
26. Maniscalco, B. & Lau, H. Comparing signal detection models of perceptual decision confidence. J. Vis. 10, 213 (2010).
Note: Supplementary information is available in the 27. Kiani, R. & Shadlen, M.N. Representation of confidence associated with a decision by neurons in the parietal cortex. Science 324, 759–764 (2009).
28. Summerfield, C. & Tsetsos, K. Building bridges between perceptual and economic We thank T. Fitzgerald, D. Kumaran and T. Sharot for comments on a previous decision-making: neural and computational mechanisms. Front. Neurosci. 6, 70 draft of this manuscript, and T. Behrens and N. Daw for discussions. This work was 29. Feldman, H. & Friston, K.J. Attention, uncertainty, and free-energy. Front. Hum. supported by a Wellcome Trust Senior Investigator Award, 098362/Z/12/Z to R.J.D.; Neurosci. 4, 215 (2010).
S.M.F. and B.D.M. are supported by Sir Henry Wellcome Fellowships (B.D.M., 30. Knill, D.C. & Pouget, A. The Bayesian brain: the role of uncertainty in neural coding 082674/Z/07/Z; S.M.F., WT096185). The Wellcome Trust Centre for Neuroimaging and computation. Trends Neurosci. 27, 712–719 (2004).
is supported by core funding from the Wellcome Trust, 091593/Z/10/Z.
31. Friston, K. The free-energy principle: a unified brain theory?Nat. Rev. Neurosci. 11, 127–138 (2010).
32. Hare, T.A., Camerer, C.F. & Rangel, A. Self-control in decision-making involves modulation of the vmPFC valuation system. Science 324, 646–648 (2009).
B.D.M. and S.M.F. conceived and designed the study. B.D.M., N.G. and S.M.F. 33. Glimcher, P.W. Indeterminacy in brain and behavior. Annu. Rev. Psychol. 56, 25–56 developed stimuli and gathered and analyzed behavioral and fMRI data. B.D.M., S.M.F. and R.J.D. interpreted the data and wrote the paper.
34. Bahrami, B. et al. Optimally interacting minds. Science 329, 1081–1085 (2010).
VOLUME 16 NUMBER 1 JANUARY 2013 NATURE NEUROSCIENCE A second set of models examined whether confidence in the item price (bid Participants. Twenty-eight participants (mean age 24.24) took part in the study. confidence) modulates the link between DV and choice. On each trial there were
Four participants were excluded because of excessive head motion. Three partici- two bid confidences (one for each item). Model 3 modulated DV by the mean pants were excluded for erratic choice patterns that prevented reliable estimation bid confidence to enable direct comparison with model 2; model 4 split the DV of a logistic fit (an inverse temperature parameter five or more times larger than predictor by bid confidence (low, high and mixed low/high, based on a subject- the average of the group). Participants were only included if they used a sufficient specific median split); model 5 extended model 4 by including additional regres- range of confidence ratings (s.d. > 0.8) to allow estimation of metacognitive abil- sors for the modulation of choice confidence (that is, model 2 split by different ity. This criterion led to the exclusion of one more subject. Twenty participants bid confidences): X [DV DV rconf were included in the final analysis.
Model 3: X = [ DV (DV r conf BDM)] Scanning task. Participants were required to fast for 4 h before the study. During
scanning they were required to make a series of binary choices between 19 com- Model 4: X = [DVlow DVmixed DVhigh] mon snack items (2,000 ms) to consume later (see Supplementary Table 4 for
a list of items). Participants were asked to choose between each combination of ( Vlow rconfchoice) items (n = 170) twice, counterbalanced across left-right spatial configurations (total number of choices = 340) and divided into four sessions. After each choice, ( Vmixed rconfchoice) participants were asked to indicate their confidence in their decision (that is, "How confident are you that the choice you made was the right one for you?") on a continuous sliding scale between 1 (low confidence) and 6 (high confidence). Models were compared via BIC scores using a fixed-effects analysis, where a Participants had 3,500 ms to move the pointer to the position that accurately difference in BIC of 5 indicates strong evidence for one model over another. reflected their confidence in the previous decision.
Two subjects were excluded from the analysis in models 4 and model 5 owing to a low variability in item confidence, precluding a median split. In addition, we Post-scanning BDM task. Participants were presented each item on a computer assessed the improvement in model fit obtained for model 2 over nested model
screen and asked to submit a bid (from £0 to £3, using a sliding scale) to buy the 1 for each subject individual y using a likelihood ratio test (C, 1 d.f.).
item (unlimited time). After each bid, participants were asked to indicate their confidence in the bid they had just submitted (that is, "How confident are you Dynamic model of value comparison (race model). To predict how value, confi-
that the bid you made was the right one for you?"; bid confidence) on a continu- dence and reaction time interact during decision-making, we harnessed a dynamic ous sliding scale between 1 (low confidence) and 6 (high confidence). At the model of the value comparison processIn the race model, separate decision var- end of the experiment, one choice from the scanning phase was played out and iables accumulate evidence for distinct options, with the final decision determined All rights reserved.
the subject had the opportunity to buy the chosen item by means of an auction by which accumulator reaches threshold first. On each time step during accumula- administered according to the Becker-DeGroot-Marschak (BDM) procedure. tion, a new evidence sample is drawn from a normal y distributed random variable More specifical y, the experimenter randomly extracted a price from a uniform st = N(ustim,Sstim). ustim is positive if the correct choice (higher value item) is the distribution (£0 to £3)—the ‘market price' of that item. If the participant's bidding righthand item, negative if the correct choice is the left item. Because st is drawn price (wil ingness-to-pay) was above the market price, no transaction occurred. from a normal distribution, the actual value of st at each time step may be positive America, Inc.
If the subject's bidding price was below the market price, the participant bought the or negative. The accumulators evolve according to the fol owing equations: snack item at the market price. At the end of the experiment, participants had to remain in the lab for an additional hour. During this hour, the only food they were ªRt  St ,St  0 al owed to consume was the item purchased in the auction, if any. This procedure encouraged subjects to choose preferred snacks during the scanning phas. 2013 Nature
Participants were compensated £40 for participation in the study. The price of any item purchased by a subject was deducted from this £40 participation fee.
Behavioral analysis and model. To examine the effect of value and confidence on The race terminates when either Rt or Lt reach a predetermined threshold, Q,
choice we compared five candidate logistic regression models. All had the form with the decision being determined by which accumulator reaches threshold first. Therefore at decision time, t(Q), either R t or Lt = Q. The finishing point of the losing accumulator depends on the values of ustim and Sstim.
where L(x) is the logistic cumulative distribution function: An estimate of decision confidence, $e, can be recovered from the race model as the distance between the two accumulators Rt and Lt at the time the race is terminated r7,12).
We simulated the model using the same parameters as in ref. 7. We simu- lated 1,000 trials at each level of ustim and recorded mean choice, confidence and reaction time. We display the simulation output in an identical manner to the Even for identical levels of decision performance, it is known that the rela- tionship between subjective confidence and decision-making varies between tasks and individuals. We sought to account for this variability by introducing an additional parameter relating model confidence to subjective confidence, The simplest candidate model predicts the probability of choosing the rightmost Sconf. On each trial, reported confidence was drawn from a Gaussian distribu- option from the signed difference in value (DV), defined as VR−VL: tion centered on $e: conf = N($e,Sconf). This feature of our model is consistent with the notion that reported confidence is derived from a higher-order stage of decision-making corrupted by nois We note that other functional forms for The slope of this function is assumed to result from randomness in choice. the link between model confidence and reported confidence are possible, but If, on the other hand, subjects have metacognitive access to the noise in their we do not investigate these here. We repeated the simulation three times with decision process, we might expect choice confidence to modulate the impact three levels of Sconf, for a fixed Sstim. Examination of psychometric function plots ) shows that Sconf can account for the variability in change in slope Model 2: X = [DV (DV × confchoice)]
observed across individuals.
NATURE NEUROSCIENCE Image acquisition and analysis. Scanning acquisition was performed using a were reported as significant if they survived family-wise error correction (FWE)
Siemens 3.0 T Al egra MRI Scanner (Erlangen, Germany). Gradient echo T2*- for multiple comparisons across the whole brain at the cluster level. For rRLPFC, weighted EPI (echoplanar) functional images with BOLD-sensitive contrast were we employed smal -volume correction using an 8-mm sphere centered on the acquired (imaging parameters: 48 transverse slices; TR, 2.88 s; TE, 30 ms; 3 × coordinates (36, 44, 28) taken from ref.14. For GLM 2, rfxplo 3 in-plane resolution; 2-mm slice thickness; 1-mm gap between adjacent slices; was used to extract percentage signal change at each region of z-shim −0.4 mT/m; positive phase encoding direction; slice tilt −30°), optimized interest defined by 6-mm spheres around the vmPFC and rRLPFC peak voxels to detect changes in orbitofrontal cortex. For each subject, 228 volumes per ses- from GLM 1. These values were entered into 2 × 2 ANOVAs (factors value, sion were col ected (total number of volumes over four sessions = 912), fol owed confidence) to further clarify the pattern of activity seen in GLM 1. In analy- by a whole brain high-resolution T1-weighted anatomical structural scan and sis of individual differences we employed small-volume correction (SVC) using local field maps. Image analysis was performed using SPM8 8-mm spheres centered on the peak activations in vmPFC and rRLPFC taken ). The first five volumes from each session were discarded to from GLM 1.
al ow for T1 equilibration. Raw functional, structural and field map files were reconstructed using TBR. Field maps were reconstructed into a single phase file. Psychophysiological interaction analysis. To assess changes in connectivity
This field map file was then used to realign and unwarp EPI functional images. between rRLPFC and vmPFC as a function of confidence, we carried out a psy- Structural images were reregistered to mean EPI images and segmented into gray chophysiological interaction (PPI) analysis. PPI is a measure of context-dependent and white matter. These segmentation parameters were then used to normalize connectivity, explaining the regional activity of other brain regions (here, vmPFC) and bias correct the functional images. Normalized images were smoothed using in terms of the interaction between responses in a seed region (here, rRLPFC) a Gaussian kernel of 8 mm ful -width at half-maximum.
and a cognitive or sensory process. We used the second GLM (factorial) to run our PPI analysis (for details, see paragraph above). We carried out PPI analysis General linear model 1 (parametric). Onset regressors beginning at the presen-
using the Generalised PPI toolbox for SPM (gPPI; tation of the two items were modulated by two parametric regressors: (i) unsigned . gPPI creates a new GLM in which the deconvolved activity of the seed difference in value ( DV ), defined as the absolute difference in value between region is assigned to separate regressors dependent on the status of the origi- the item presented on the right (VR) and the item presented on the left (VL), with nal psychological variable (high or low confidence), and reconvolved with the values VR and VL ascertained from subjects' bids in the post-scanning phase ( DV hemodynamic response function. Average time courses were extracted from all = VR − VL ); and (i ) post-choice confidence ratings, which ranged from 0 to 500 voxels in a 6-mm sphere surrounding the rRLPFC peak coordinate (39, 41, 16). on an arbitrary scale. In this model, confidence is orthogonalized with respect to The main effects of high and low confidence, the seed-region time course and DV by the SPM8 software. Second, for each subject we constructed a separate motion parameters were included as regressors of no interest. The PPI contrast GLM using a factorial design.
compares high_conf * rRLPFC (+1) with low_conf * rRLPFC (−1). This analysis showed a significant activation in vmPFC (9, 50, −11) that reflects the increased All rights reserved.
General linear model 2 (factorial). Events were split into regressors based on connectivity between vmPFC and rRLPFC during high compared to low
confidence level (that is, low and high confidence) using a median split for each confidence trials.
individual subject. Each of these regressors was modulated by a DV parametric regressor (defined above).
35. Kass, R. Bayes factor. J. Am. Stat. Assoc. 430, 773–795 (1995).
36. Song, C. et al. Relating inter-individual differences in metacognitive performance on different perceptual tasks. Conscious. Cogn. 20, 1787–1792 (2011).
America, Inc.
Statistical inference. Second-level group contrasts from GLM 1 were calculated 37. Gläscher, J. Visualization of group inference data in functional neuroimaging.
as one-sample t-tests against zero for each first-level linear contrast. Activations Neuroinformatics 7, 73–82 (2009).
2013 Nature
NATURE NEUROSCIENCE

Source: http://www.cresa.eu/wp-content/uploads/2013/09/De-Martino-et-al.-2013-Confidence-in-value-based-choice.pdf