pubmed.ncbi.nlm.nih.gov

Neuronal encoding of subjective value in dorsal and ventral anterior cingulate cortex - PubMed

  • ️Sun Jan 01 2012

Comparative Study

Neuronal encoding of subjective value in dorsal and ventral anterior cingulate cortex

Xinying Cai et al. J Neurosci. 2012.

Abstract

We examined the activity of individual cells in the primate anterior cingulate cortex during an economic choice task. In the experiments, monkeys chose between different juices offered in variables amounts and subjective values were inferred from the animals' choices. We analyzed neuronal firing rates in relation to a large number of behaviorally relevant variables. We report three main results. First, there were robust differences between the dorsal bank (ACCd) and the ventral bank (ACCv) of the cingulate sulcus. Specifically, neurons in ACCd but not in ACCv were modulated by the movement direction. Furthermore, neurons in ACCd were most active before movement initiation, whereas neurons in ACCv were most active after juice delivery. Second, neurons in both areas encoded the identity and the subjective value of the juice chosen by the animal. In contrast, neither region encoded the value of individual offers. Third, the population of value-encoding neurons in both ACCd and ACCv underwent range adaptation. With respect to economic choice, it is interesting to compare these areas with the orbitofrontal cortex (OFC), previously examined. While neurons in OFC encoded both pre-decision and post-decision variables, neurons in ACCd and ACCv only encoded post-decision variables. Moreover, the encoding of the choice outcome (chosen value and chosen juice) in ACCd and ACCv trailed that found in OFC. These observations indicate that economic decisions (i.e., value comparisons) take place upstream of ACCd and ACCv. The coexistence of choice outcome and movement signals in ACCd suggests that this area constitutes a gateway through which the choice system informs motor systems.

PubMed Disclaimer

Figures

Figure 1.
Figure 1.

Behavioral task and recording locations. a, Experimental design. Offers were presented on the two sides of the fixation point. The color indicated the juice type the number of squares indicated the juice amount. Monkeys indicated their choice with a saccade. Different offer types and different left/right configurations were pseudorandomly interleaved. b, Time windows. All time windows lasted 0.5 s except the Reaction Time, which lasted from the go signal to the initiation of the saccade. c, Recording locations. Our recordings extended for 12–15 mm anterior–posterior. Data from the dorsal bank and fundus (ACCd) and data from the ventral bank (ACCv) were analyzed separately. d, Reconstruction of recording sites. Anterior–posterior (AP) and mediolateral (ML) coordinates are represented by the y-axis and x-axis, respectively. The recording system provided a spatial resolution of 1 mm. The gray scale indicates for each location the number of cells recorded from that location (scale indicated in the legend). For each location, we computed the proportion of cells modulated by the trial type and/or by the movement direction in at least one time window (two-way ANOVA). These proportions are represented by green and purple circles, respectively, for trial type and movement direction (scale indicated in the legend). For this analysis, we pooled data from the two monkeys and we displayed the results as though all the cells were recorded from the right hemisphere. Interestingly, the distribution of neurons modulated by the movement direction within ACCd was not even. In particular, we found more spatially selective cells in the anterior part than in the posterior part (limit set at AP ≤ A30, p < 10−6, Pearson's χ2), and more spatially selective cells in the medial part than in the lateral part (limit set at ML ≤ L4, p < 10−4, Pearson's χ2).

Figure 2.
Figure 2.

ACCd, activity of single neurons. The figure represents the activity of five neuronal responses. a, Response encoding the Chosen Value. In the left panel, the x-axis represents different offer types, ranked from left to right by the ratio #B:#A. The black dots represent the behavioral choice pattern and the relative value inferred from the sigmoid fit is indicated in the top left (1A = 3.2B). The color symbols indicate the neuronal firing rate recorded in the Prejuice time window. Each symbol represents one trial typeLR. The red and green symbols represent, respectively, trials in which the animal executed a leftward and rightward saccade. The diamonds and circles represent, respectively, trials in which the animal chose juice A and juice B. In the right panel, the same data points (y-axis) are plotted against the variable Chosen Value (expressed in units of juice B). The two lines represent the results of the ANCOVA (parallel model). It can be observed that the activity of this cell encodes the variable Chosen Value (as confirmed by the variable selection analysis) and is not directionally selective. b, Another response encoding the Chosen Value. This response was recorded in the Pre-go time window. c, Response encoding the Chosen Value and in a directionally selective way. This response, recorded in the Postoffer time window, encoded the Chosen Value. In addition, the firing rate was higher when the animal executed a leftward saccade compared with a rightward saccade. Cells such as this one, modulated both by the Chosen Value and the saccade direction, were present in the population. However, their incidence was not higher than would be expected by chance (Table 1). d, Response encoding the Mov Dir Only. This neuronal response, recorded in the Late Delay time window, was spatially selective (higher for rightward saccades) but did not depend on the offer type. In the right panel, the firing rate is plotted against the Chosen Value, even though this variable did not explain the response. The two lines represent the results of the ANCOVA (horizontal lines model). e, Response encoding the Chosen Juice. This neuronal response, recorded in the Postjuice time window, was binary depending on the chosen juice and was not spatially selective. All conventions in b–e are as in a. Regression lines in a–c and e are all from the ANCOVA (parallel model).

Figure 3.
Figure 3.

ACCd, population results of ANCOVA. a, Explained responses. The number indicated in each bin represents the number of responses explained by each variable in each time window. For example, in the Postoffer time window, the variable Total Value explains 83 responses. Notably, because total value is highly correlated with Chosen Value, many of these 83 responses are also explained by Chosen Value and thus appear in both bins. The same results indicated numerically are also represented as images by different shades of gray. b, Best fit. The number indicated in each bin represents the number of responses for which the corresponding variable provides the best explanation (highest R2). Thus, in this panel, each response appears in at most one bin. The best fitting variable was identified separately for each response (i.e., for each time window). It can be noted that the redundancy between Total Value and Chosen Value appears clearly resolved in favor of the latter variable. In later time windows, many responses are best explained by the Chosen Juice. Analogous plots for the OFC can be found in the study by Padoa-Schioppa and Assad (2006), their Figure S6.

Figure 4.
Figure 4.

ACCd, variable selection analysis. a, Stepwise selection. The figure illustrates the results of six iterations of the procedure (from top to bottom). The first panel is the same as in Figure 3b. In the first iteration, we select the variable that provides the highest number of best fits in any time window, namely the Chosen Value (highlighted with a star symbol immediately below the panel). Across time windows, this variable explains 556 (66%) responses. The second panel illustrates the residual population of neuronal responses that were not explained by Chosen Value. In the second iteration, we select the variable that provides the highest number of best fits for this residual population, namely the Mov Dir Only. Across time windows, this variable explains an additional 146 (17%) responses. In the third iteration, we select the Chosen Juice, which explains an additional 61 (7%) responses. In all subsequent iterations, the marginal explanatory power of selected variables (i.e., the number of additional responses explained) failed to reach the threshold of 5% of the total. These variables (highlighted with a dot symbol immediately below the panel) were thus discarded. b, Stepwise selection, percentage of explained responses. The figure illustrates the percentage of responses explained at subsequent iterations of the stepwise procedure. In this plot, “100” on the y-axis represents the total number of responses that passed the ANOVA criterion. The dotted line (y = 95%) indicates the number of responses explained overall by the 24 variables examined in the analysis. The three variables Chosen Value, Mov Dir Only, and Chosen Juice collectively explained 95% of the responses explained by the 24 variables, corresponding to 90% of the total responses. c, d, Best-subset selection. c illustrates the percentage of responses (y-axis) explained by the best subset as a function of the number of variables (x-axis). d, The table indicates the best subset for n = 1, 2, and 3 variables. In essence, variables Chosen Value, Mov Dir Only, and Chosen Juice provide indeed the best subset of three variables. The fact that Chosen Value and Mov Dir Only also provide the best subsets of one and two variables can be seen as a sign of robustness of this result. Most importantly, the results obtained with the best-subset method are essentially identical to those obtained with the stepwise selection. e, Distribution of R2. Each response was classified as encoding one of the three selected variables based on the R2. Notably, many more responses encoded the Chosen Value (N = 521) compared with the Mov Dir Only (N = 137) and the Chosen Juice (N = 105). The three histograms here represent the distribution of R2 for each of the three variables. The mean of the distribution was equal to 0.55, 0.52, and 0.46 for Chosen Value, Mov Dir Only, and Chosen Juice, respectively.

Figure 5.
Figure 5.

ACCv, activity of single neurons. The figure represents the activity of four neuronal responses. All conventions are as in Figure 2. a, Response encoding the Chosen Value. In the left panel, the x-axis represents different offer types, the black dots represent the behavioral choice pattern, and the color symbols indicate the neuronal firing rate. Each symbol represents one trial typeLR. The diamonds and circles represent, respectively, trials in which the animal chose juice A and juice B. The red and green symbols represent, respectively, trials in which the animal executed a leftward and rightward saccade. In the right panel, the same data points (y-axis) are plotted against the variable Chosen Value (expressed in units of juice B). The two lines represent the results of the ANCOVA (parallel model). As typically the case in ACCv, the activity of this cell is not directionally selective. This response was recorded in the Late Delay time window. b, Another response encoding the Chosen Value. This response was recorded in the Prejuice time window. c, Response encoding the Chosen Value with a negative slope. This response, recorded in the Postjuice time window, was higher for lower values. d, Response encoding the Chosen Juice. This neuronal response, recorded in the Postjuice2 time window, was binary depending on the chosen juice. All conventions in b–d are as in a.

Figure 6.
Figure 6.

ACCv, population results of ANCOVA. a, Explained responses. The number indicated in each bin represents the number of responses explained by each variable in each time window. b, Best fit. The number indicated in each bin represents the number of responses for which the corresponding variable provides the best explanation (highest R2). The best fitting variable was identified separately for each response (i.e., for each time window). Notably, ACCv neurons were most frequently modulated after juice delivery (Table 2). In terms of encoded variables, Chosen Value appears dominant throughout the delay, whereas Chosen Juice is most prominent after juice delivery. All conventions are as in Figure 3.

Figure 7.
Figure 7.

ACCv, variable selection analysis. All conventions are as in Figure 4. a, Stepwise selection. The first panel is the same as in Figure 6b. In the first two iterations, the algorithm selected variables Chosen Juice and Chosen Value. All variables selected in subsequent iterations fail to meet the 5% threshold and were thus discarded. The variable selected (discarded) at each iteration is highlighted with a star (dot) symbol immediately below the panel. b, Stepwise selection, percentage of explained responses. The figure illustrates the percentage of responses explained at subsequent iterations of the stepwise procedure. In this plot, “100” on the y-axis represents the total number of responses that passed the ANOVA criterion. The dotted line (y = 94%) indicates the number of responses explain overall by the 24 variables examined in the analysis. The two variables Chosen Value and Chosen Juice collectively explained 304 responses, corresponding to 90% of the responses explained by the 24 variables and to 85% of the total responses. c, d, Best-subset selection. c indicates the percentage of responses (y-axis) explained by the best subset as a function of the number of variables (x-axis). d, The table indicates the best subset for n = 1, 2. Variables Chosen Value and Chosen Juice provide the best subset of two variables. (See main text for details on post hoc analysis.) e, Distribution of R2. Each response was classified as encoding either the Chosen Value or the Chosen Juice based on the R2. The two histograms here represent the distribution of R2 for the two variables. The distribution mean was equal to 0.53 for the Chosen Value (N = 221) and to 0.51 for the Chosen Juice (N = 83).

Figure 8.
Figure 8.

U-shaped responses reflect the subjective nature of value. a, b, One response. This neuron was recorded in ACCv and trials for different movement directions were pooled (see Materials and Methods). For this analysis (b), the firing rate was regressed on the amount of juice chosen and received by the animal, separately for trials in which the animal chose juice A and juice B. We thus obtained slopes α and β. If the response indeed encodes the Chosen Value, the slope ratio α/β should provide a neuronal measure for the relative value and thus equal the behavioral measure obtained from the choice pattern—a prediction met by this response (slope ratio α/β = 2.2 ± 0.6; indifference point A/B = 2.6). c, Match between neuronal and behavioral measures of relative value, one juice pair. The scatterplot (log–log scale) includes all U-shaped responses recorded from ACCv while monkeys chose between 2/3 fruit punch and 1/3 cranberry juice (55 responses). The line, obtained from a linear regression, is statistically indistinguishable from identity (y = x). The fact that the neuronal measure of relative value (y-axis) matched the variability observed in the behavioral measure (x-axis) demonstrates that these neurons indeed reflect the subjective nature of value. Indeed, if U-shaped responses encoded a physical property of the juice, the variability in the slope ratio should be independent of that observed in the choice pattern and the regression line should be horizontal. d, e, Neuronal versus behavioral measure of relative value, population. In the scatterplot, each symbol represented one response, different symbols and colors indicate different juice pairs (see legend), and regression lines represent the results of the ANCOVA (full model). This analysis included only juice pairs for which we had at least 20 responses. For both areas, we found a significant effect of the main factor and no effect of either the group or the interaction. In both areas, the relationship between neuronal measure (slope ratio) and behavioral measure (indifference point) was statistically indistinguishable from identity.

Figure 9.
Figure 9.

Range adaptation in ACCd. a, Distribution of regression slopes. Responses with positive/negative slopes were in ratio 60/40 (p < 10−5, binomial test). Apart from the sign, the distributions obtained for positive and negative slopes had equal median (p = 0.84, Wilcoxon's rank-sum test). We thus rectified responses with negative slopes and we pooled all responses for this analysis. b, Distribution of unsigned slopes. c, Regression slope versus value range. In the scatterplot, each cross represents one response, the y-axis represents the regression slope (c1), and the x-axis represents the range of values (ΔV) available to the monkey in the corresponding session. Responses were divided in two groups depending on whether ΔV ≤ 6 uB or ΔV > 6 uB. The two color diamonds represent the two groups (blue for ΔV ≤ 6 uB; yellow for ΔV > 6 uB). Their x- and y-coordinates are in the center of mass of the corresponding group. On average, regression slopes measured in sessions with a small value range (blue) were larger than regression slopes measured in sessions with a large value range (yellow) (p < 10−10, ANOVA). d, Population firing rates, individual responses. The plot shows the entire population of 521 responses Chosen Value responses recorded in ACCd. Each response was normalized, rectified, and color coded depending on the value range. e, Average neuronal responses. The blue and yellow lines represent, respectively, the average neuronal response obtained for ΔV ≤ 6 uB and for ΔV > 6 uB. Notably, the two average responses are well separated throughout the value spectrum, and both are close to linear. f, Distribution of activity ranges. Each histogram illustrates the distribution of activity ranges (Δφ) obtained for the corresponding group. The small triangles indicate the medians. The two distributions are statistically indistinguishable (p = 0.10, Kruskal–Wallis test). Color codes in d–f are the same as in c.

Figure 10.
Figure 10.

Range adaptation in ACCv. a, Distribution of regression slopes. Responses with positive/negative slopes were in ratio 67/33 (p < 10−8, binomial test). Apart from the sign, the distributions obtained for positive and negative slopes had equal median (p = 0.69, Wilcoxon's rank-sum test). We thus rectified responses with negative slopes, and we pooled all responses for this analysis. b, Distribution of unsigned slopes. c, Regression slope versus value range. In the scatterplot, each cross represents one response, the y-axis represents the regression slope (c1), and the x-axis represents the range of values (ΔV). Responses were divided in two groups depending on whether ΔV ≤ 6 uB or ΔV > 6 uB. The blue and yellow diamonds represent responses with ΔV ≤ 6 uB and responses with ΔV > 6 uB, respectively. Each diamond is located in the center of mass of the corresponding group. On average, regression slopes measured in sessions with a small value range were larger than regression slopes measured in sessions with a large value range (p < 10−10, ANOVA). d, Population firing rates, individual responses. The plot shows the entire population of 256 Chosen Value responses recorded in ACCv. Each response was normalized, rectified, and color coded depending on the value range. e, Average neuronal responses. The blue and yellow lines represent, respectively, the average neuronal response obtained for ΔV ≤ 6 uB and for ΔV > 6 uB. Notably, the two average responses are well separated throughout the value spectrum, and both are close to linear. f, Distribution of activity ranges. Each histogram illustrates the distribution of activity ranges (Δφ) obtained for the corresponding group, and small triangles indicate the medians. The two distributions are statistically indistinguishable (p = 0.65, Kruskal–Wallis test). Color codes in d–f are the same as in c.

Figure 11.
Figure 11.

Partial adaptation on the timescale of individual trials in ACCd and ACCv. a, One response. The red symbols represent the response computed pooling all trials. Trials were then separated in two groups, with V(n) > V(n − 1) (yellow symbols) and V(n) < V(n − 1) (blue symbols). For each trial type, trials with V(n) = V(n − 1) were assigned to the group with fewer trials. For most trial types, the “yellow firing rate” was slightly higher than the “blue firing rate,” consistent with neuronal adaptation. The difference between the yellow firing rate and the blue firing rate was computed for each trial type and averaged across trial types. We thus obtained the normalized difference δ = 0.13. This measure represents the mean percentage modulation of trial n − 1 on the activity recorded on trial n. The response shown here is the same as in Figure 2b. However, for this analysis, we pooled together trials with different movement directions (i.e., we analyzed data based on the trial type, not the trial typeLR). b, Population analysis, distribution of δ for n − 1. The two histograms refer to ACCd and ACCv, respectively. In each histogram, the x-axis represents δ and the y-axis represents the number of responses. In both areas, δ varied substantially across the population. However, in both areas δ was above zero in a significant majority of cases (binomial test) and mean(δ) was significantly greater than zero (t test). Mean, SD, and p values are indicated in each histogram. c, Population analysis, mean(δ) over trials. The mean (δ) (±SEM) (y-axis) is plotted against the trial number (x-axis), separately for ACCd (red) and for ACCv (blue). The filled squares indicate data points statistically different from zero (binomial test, p < 0.01; t test, p < 0.01). In the two areas, mean(δ) equal 5–7% for n − 1, equal 1–2% for n − 2, and are indistinguishable from zero for earlier trials. As expected, mean(δ) are also indistinguishable from zero for n + 1 in both areas.

Similar articles

Cited by

References

    1. Abe H, Lee D. Distributed coding of actual and hypothetical outcomes in the orbital and dorsolateral prefrontal cortex. Neuron. 2011;70:731–741. - PMC - PubMed
    1. Alexander WH, Brown JW. Medial prefrontal cortex as an action-outcome predictor. Nat Neurosci. 2011;14:1338–1344. - PMC - PubMed
    1. Amiez C, Joseph JP, Procyk E. Reward encoding in the monkey anterior cingulate cortex. Cereb Cortex. 2006;16:1040–1055. - PMC - PubMed
    1. Bermudez MA, Schultz W. Reward magnitude coding in primate amygdala neurons. J Neurophysiol. 2010;104:3424–3432. - PMC - PubMed
    1. Botvinick MM. Conflict monitoring and decision making: reconciling two perspectives on anterior cingulate function. Cogn Affect Behav Neurosci. 2007;7:356–366. - PubMed

Publication types

MeSH terms