Neuronal Encoding of Subjective Value in Dorsal and Ventral Anterior Cingulate Cortex

Identification of task-related responses

A first set of analyses were conducted to identify task-related neuronal responses, which would be subject to further examination. First, we submitted each neuron in each time window to a three-way ANOVA with factors (offer type by offer position by movement direction). In this analysis, the factor “offer position” captured the spatial left/right configuration of the offers on the monitor, whereas “movement direction” captured the left/right direction of the eye movement. Our results for ACCd and ACCv are summarized in Tables 1 and 2, respectively. In essence, neurons modulated by the offer type were present in both regions. In contrast, neurons modulated by the spatial configuration of the offers were very infrequent in either region (<2% of cells, pooling time windows). Finally, neurons modulated by the direction of the eye movement were fairly frequent in ACCd (20% cells, pooling time windows) but rather infrequent in ACCv (5% of cells, pooling time windows).

Starting with data from ACCd, we submitted each neuron to a two-way ANOVA with factors (trial type by movement direction) including the interaction term (results not included in Table 1). This analysis confirmed that both main factors modulated the activity of neurons in this area: pooling time windows, cells modulated by factors trial type and movement direction were 38 and 20%, respectively. In contrast, cells presenting a significant interaction term were very infrequent (<3% of cells, pooling time windows). In other words, the good-related modulation (captured by the factor trial type) and the action-related modulation (captured by the factor movement direction) were essentially independent from each other for any given cell. We thus repeated the two-way ANOVA including only the main factors, and we quantified how many neurons in ACCd were modulated by both factors (Table 1, sixth column). We found that the size of this subpopulation was very close from that expected by chance if the two modulations were completely independent (6% of cells modulated by both factors vs 7% expected by chance). In other words, the good-related modulation and the action-related modulation were independent across the neuronal population. For data from ACCv, the two-way ANOVAs confirmed that the neuronal activity could be modulated by the trial type, but was independent of the visuomotor contingencies of choice.

Finally, we identified for each area task-related responses using a one-way ANOVA (factor “trial type_LR”). Across time windows, task-related responses were 845 for ACCd and 359 for ACCv. Pooling time windows, neurons that passed the ANOVA criterion in at least one time windows were 406 of 1125 (36%) in ACCd and 194 of 779 (25%) in ACCv. These preliminary analyses also revealed that the two brain regions differed for the activation timing. Indeed, neurons in ACCd were modulated most prominently during the delay (time windows Late Delay and Pre-go). In contrast, in ACCv the modulation was most prominent immediately after juice delivery (time windows Postjuice and Postjuice2). This difference provided further evidence that ACCd and ACCv are physiologically distinct areas.

Choice-related variables encoded in ACCd

What choice-related variables are encoded in ACCd? A qualitative inspection revealed that neurons in ACCd often presented U-shaped responses, suggesting that they encoded the variable Chosen Value. As revealed by the ANOVAs (Table 1), many neurons were also modulated by the movement direction. However, these two factors appeared to be largely independent, in the sense that some neurons encoded only the Chosen Value (Fig. 2a,b), other neurons encoded both the Chosen Value and the movement direction (Fig. 2c), and yet other neurons encoded only the movement direction (variable Mov Dir Only; Fig. 2d). In addition, some neurons presented a binary response seemingly encoding the Chosen Juice (Fig. 2e). Of course, Chosen Value and Chosen Juice are only two of the many variables potentially encoded by this population. To identify the variables that best explained our data set, we adopted the same basic approach used in a previous study (Padoa-Schioppa and Assad, 2006).

Casting a wide net, we defined 24 variables that neurons in ACCd might potentially encode (Table 3). Specifically, we defined several good-based value-related variables (Offer Value, Chosen Value, Other Value, etc.), several quantity-related variables (Max #, Chosen #, Total #, etc.), a binary variable capturing the outcome of the choice process (Chosen Juice), and four spatial value-related variables (Offer Value L, Offer Value R, Chosen Value L, Chosen Value R). We also defined the variable Mov Dir Only, a purely spatial variable that captured responses modulated only by the movement direction (see Materials and Methods).

For each response and for each variable, we ran an ANCOVA, using the variable as a predictor and grouping data by the movement direction. A given variable was said to “explain” the neuronal response if the corresponding factor in the ANCOVA was significant (p < 0.05). As previously noted (Padoa-Schioppa and Assad, 2006), different variables included in this analysis could be highly correlated with one another (e.g., Chosen Value and Total Value were highly correlated in our experiments). Hence, any given response might be explained by more than one variable. In the light of this consideration, we generated two qualitative descriptions of our neuronal data set. Figure 3a illustrates for each time window the number of responses explained by each variable. Notably, any given response may appear in more than one bin in this plot. Conversely, Figure 3b illustrates for each time window and for each variable the number of responses best explained by that variable (highest R²). In this plot, each response appears in at most one bin. A qualitative inspection of Figure 3b revealed that the best explanation was often provided by Chosen Value. However, a sizable number of responses are best explained by Chosen Juice and/or by Mov Dir Only.

Overall, 804 of 845 (85%) task-related responses were explained by at least 1 of the 24 variables defined here. For a quantitative assessment of the variables that best explained the population, we used the stepwise selection method. As illustrated in Figure 4, a and b, the first three iterations of this procedure selected variables Chosen Value, Mov Dir Only, and Chosen Juice. Collectively, these three variables explained 763 responses. All variables selected in subsequent iterations provided a marginal explanatory power of <5%. The best subset method confirmed this result (Fig. 4c,d). As a single variable, Chosen Value explained more responses than any other variable. As a pair of variables, Chosen Value and Mov Dir Only explained more responses that any other pair of variables. Finally, the three variables Chosen Value, Mov Dir Only, and Chosen Juice explained more responses than any other subset of three variables. Again, these three variables collectively explained 763 responses, corresponding to 95% of the responses explained by the whole 24 variables, and to 90% of responses that passed the ANOVA criterion.

In summary, the explanatory power of variables Chosen Value, Chosen Juice, and Mov Dir Only was higher than that of any other subset of three variables. To examine whether this statement was true in a statistical sense, we ran a post hoc analysis in which the marginal explanatory power of each selected variable was tested against that of a discarded and “challenging” variable. As illustrated in Table 4, we found that the explanatory power of each selected variable was significantly higher than that of all challenging alternatives (all p < 0.05).

In conclusion, neurons in the ACCd appear to encode variables Chosen Value, Chosen Juice, and Mov Dir Only. On the basis of this result, we classified task-related responses from this area as encoding one of these three variables. Responses explained by more than one variable were assigned to the variable with highest R². Responses not explained by any of these variables were labeled “unexplained.” In general, the encoded variable provided a good account of the responses [across variables, mean(R²) > 0.5; Fig. 4e].

Choice-related variables encoded in ACCv

A qualitative inspection revealed that neurons in ACCv often presented U-shaped responses, suggesting that they encoded the variable Chosen Value (Fig. 5a–c). Other cells presented a binary response suggesting that they encoded the Chosen Juice (Fig. 5d). These latter responses were most frequent in late time windows, after juice delivery. For a population analysis, we examined the same variables examined in the analysis of ACCd. For each response, we performed an ANCOVA on each variable (group: movement direction). We generated two descriptions of this data set. Figure 6a illustrates for each time window the number of responses explained by each variable. Figure 6b illustrates for each time window and for each variable the number of responses best explained by that variable (highest R²). A qualitative inspection reveals that the best explanation was often provided by the variable Chosen Value. However, especially in late time windows, a sizable number of responses were best explained by the variable Chosen Juice.

Overall, the 24 variables explained 337 of 359 (94%) responses that passed the ANOVA criterion. For a quantitative assessment of the variables that best explain the population, we first used the stepwise selection method. As illustrated in Figure 7, a and b, the first two iterations selected variables Chosen Value and Chosen Juice. All variables selected in subsequent iterations provided a marginal explanatory power of <5%. This result was confirmed by the best-subset procedure (Fig. 7c,d). As a single variable, Chosen Value explained more responses than any other variable. As a pair of variables, Chosen Value and Chosen Juice explained more responses than any other pair of variables. Collectively, Chosen Value and Chosen Juice explained 304 responses, corresponding to 90% of responses explained by the whole 24 variables and to 85% of responses that passed the ANOVA criterion.

We thus proceeded with a post hoc analysis, in which we compared the marginal explanatory power of each of these two variables pairwise against that of other, highly correlated variables (Table 5). In this analysis, the explanatory power of Chosen Value was significantly higher than that of both Value Difference and Chosen #. However, the explanatory power of Chosen Value was not statistically higher than that of Total Value, and the explanatory power of Chosen Juice was not statistically higher than that of either Chosen Value A or Chosen Value B. This analysis would thus leave us with some degree of ambiguity.

To gain further insight, we repeated the variable selection analysis on this data set based on spatially invariant responses. In this case, each data point represented a trial type (not a trial type_LR) and linear fits were performed using linear regressions, from which we obtained a slope and the R². Notably, this procedure is in principle preferable for ACCv, where responses are not spatially selective, because the statistical power is entirely “spent” to identify encoded variables. We obtained results similar to those described above. Across time windows, the one-way ANOVA (factor: trial type) identified 473 task-related responses. Overall, the 19 nonspatial variables included in the analysis explained 433 of 473 (92%) responses. The stepwise method identified Chosen Value, Chosen Juice, and Total Value in the first three iterations, and no other variable reached the 5% criterion in subsequent iterations. The best-subset method revealed that the highest explanatory power was provided by Chosen Value (as a single variable), by Chosen Value and Chosen Juice (as a pair of variables), and by Chosen Value, Chosen Juice, and Max # (as a set of three variables). The partial discrepancy between the results obtained with the two methods was clarified by the post hoc analysis, which revealed the following points. First, the explanatory power of both Chosen Value and Chosen Juice was significantly higher than that of any other challenging variable (all p < 0.005). In particular, the explanatory power of Chosen Value was significantly higher than that of Total Value (p < 10⁻⁵). Second, the explanatory power of Max # was statistically indistinguishable from that of Total Value and Total #. In summary, these analyses indicated that the population of neurons in ACCv certainly encoded variables Chosen Value and Chosen Juice. Together, these two variables explained 367 responses, corresponding to 85% of the responses explained by the whole 19 variables, and to 78% of the responses that passed the ANOVA criterion. In addition, a smaller population of neurons in ACCv might encode the Max #, or the Total # or the Total Value. However, we could not disambiguate between these possibilities based on our current data.

Based on these results, we concluded that the population of ACCv indeed encoded variables Chosen Value and Chosen Juice. We thus classified task-related responses from ACCv as encoding one of these two variables. In general, the two variables provided a good account of the population [across variables, mean(R²) > 0.5; Fig. 7e]. In all subsequent analyses of ACCv, we pooled trials for different movement directions.

Subjective value versus physical properties: analysis of U-shaped responses

One major conclusion of the analyses presented above is that we found in both ACCd and ACCv neurons that encode the Chosen Value. Apart from the dissociation between Chosen Value and other value variables (Total Value, etc.), perhaps the most interesting aspect of this result is that neurons in both ACCd and ACCv appear to encode economic value—a subjective quantity that integrates physically distinct dimensions (juice type and juice amount). In this respect, however, one caveat is in order. Although our analysis included many physical variables, it forcedly did not include all possible physical variables. For example, we did not examine the variable Sugar Amount. Thus, to test whether U-shaped responses indeed encode the subjective value as opposed to any physical property of the juices, we conducted the following analysis.

The hypothesis that U-shaped responses encode the Chosen Value leads to a simple prediction. Separating trials in which the animal chose juice A and juice B, we can regress separately the neuronal firing rate against the amounts of juice A and juice B chosen by the animal. The two regressions provide slopes α and β (Fig. 8a,b). If the neuronal response indeed encodes the Chosen Value, then slope α should be proportional to the subjective value of juice A, slope β should be proportional to the subjective value of juice B, and the ratio α/β should be equal to the value ratio. In other words, the slope ratio α/β provides an independent and neuronal measure of the relative value of the two juices, which should be equal to the relative value measured behaviorally from the choice pattern. For the response illustrated in Figure 8a,b, this prediction is met: the relative value measured neuronally (α/β = 2.2 ± 0.6, SD computed with error propagation) is statistically indistinguishable from that measured behaviorally at the indifference point (A/B = 2.6).

The evidence that U-shaped responses indeed encode subjective value as opposed to any physical property of the juices emerges from the following observation. For any given juices A and B, the relative value measured behaviorally may vary to some extent from day to day. For example, on some days, the animal may be very thirsty and may not forego large quantities of juice B to obtain 1A, resulting in a lower relative value. On other days, the animal may be less thirsty, resulting in a higher relative value. If U-shaped responses encode a physical property of the juices, they should be unaffected by this day to day variability. Conversely, if U-shaped responses encode the subjective value, the slope ratio α/β should also vary from day to day matching the variability in relative value measured behaviorally. Our data were consistent with the latter prediction. In Figure 8c, we plotted the neuronal measure of relative value (slope ratio, y-axis) against the behavioral measure of relative value (indifference point, x-axis). Each circle represents one response, and the plot includes all and only U-shaped responses recorded in ACCv with 2/3 fruit punch and 1/3 cranberry juice. A linear regression y = a₀ + a₁x provided the following estimates (±95% confidence interval): a₀ = 0.09 (±0.45) and a₁ = 0.99 (±0.66). In other words, the two measures of relative value were statistically indistinguishable.

To test the identity between neuronal and behavioral measures of relative values across the population, we performed an ANCOVA (full model) in which we regressed the slope ratio against the behavioral indifference point and we grouped responses by the juice pair. This analysis was done separately for the two brain areas. For ACCd (six juice pairs; Fig. 8d), we found that the slope ratio depended significantly on the indifference point (p < 10⁻¹⁰) but did not depend on either the juice pair or the interaction (indifference point by juice pair) (both p > 0.07). Specifically, for the relationship between slope ratio and indifference point (main term), we obtained slope = 1.18 (±0.24) and intercept = −0.19 (±0.22) (estimate ± SEM). In other words, the neuronal measure and the behavioral measure of relative value were statistically identical with one another.

For ACCv (seven juice pairs; Fig. 8e), we obtained very similar results. The full model ANCOVA indicated that the slope ratio depended significantly on the indifference point (p < 10⁻¹⁰) but did not depend on either the juice pair or the interaction (indifference point by juice pair) (both p > 0.5). For the relationship between slope ratio and indifference point (main term), we obtained slope = 0.83 (±0.16) and intercept = 0.20 (±0.12) (estimate ± SEM). (Note that the intercept did not differ significantly from zero, p > 0.09.) In other words, the two measures of relative value were statistically identical to one another.

In conclusion, this analysis demonstrates that U-shaped responses indeed encode a subjective quantity–namely, the Chosen Value–as opposed to any physical property of the juices.

Range adaptation in ACCd and ACCv

In a previous study, we found that value-encoding neurons in the OFC adapt to the range of values available in any given session (or trial block). In other words, the same range of neuronal firing rates describes different ranges of values in different behavioral conditions (Padoa-Schioppa, 2009). This adaptation phenomenon ensures in principle a computationally efficient encoding of subjective value. We thus examined whether the representations of subjective value in ACCd and ACCv also undergo range adaptation.

We first examined the population of 521 Chosen Value responses recorded in ACCd. Consistent with previous reports (Seo and Lee, 2007; Kennerley et al., 2009), their activity could either increase or decrease as a function of the encoded value. Specifically, responses with positive and negative slopes were 310 of 521 (60%) and 211 of 521 (40%), respectively (Fig. 9a). Apart from the sign, the two distributions obtained for positive and negative slopes had equal median (p = 0.84, Wilcoxon's rank-sum test). For all the analyses of range adaptation, we thus rectified negative-slope responses and pooled them with positive-slope responses (Fig. 9b).

The variable selection analyses presented above indicated that the encoding of the variable Chosen Value was roughly linear. In other words, indicating with φ the firing rate, with V the Chosen Value, and with c₀ and c₁ the intercept and slope of the encoding, respectively, the following held true: φ = c₀ + c₁V. Having defined the value range ΔV as the range of values chosen by the animal within a session and the minimum value V₀ as the lowest value chosen in that session, this relationship can be written as follows: φ = φ₀ + Δφ * (V − V₀)/ΔV, where φ₀ = c₀ + c₁V₀ is the baseline activity and Δφ = c₁ * ΔV is the activity range. In this formalism, the hypothesis that neuronal firing rates adapt to the range of values available in any given session corresponds to the hypothesis that neuronal parameters φ₀ and Δφ are independent of the value range ΔV.

In our experiments, each neuron was recorded in one session with one value range (ΔV). However, the range of chosen values varied from session to session, roughly twofold between 4 and 8 uB. We thus divided neuronal responses in two groups depending on whether the range values chosen in that session was ΔV ≤ 6 uB or ΔV > 6 uB. If neuronal firing rates indeed adapt to the value range, then the slope of the encoding c₁ should be inversely proportional to the value range ΔV. Consistent with this prediction, we found that the average slope was significantly higher when neurons were recorded with ΔV ≤ 6 uB compared with when neurons were recorded with ΔV > 6 uB (p < 10⁻¹⁰, ANOVA; Fig. 9c).

The most direct evidence for neuronal adaptation came from a population analysis of the firing rate as a function of the Chosen Value. For this analysis, we normalized each response by subtracting the baseline activity φ₀ corresponding to the minimum chosen value V₀. The entire population of 521 responses is illustrated in Figure 9d, where the two colors label responses recorded with ΔV ≤ 6 uB and ΔV > 6 uB, respectively. In general, activity ranges (y-axis) varied substantially across the population. However, when we averaged the firing rates separately for the two groups of responses neuronal adaptation appeared evident (Fig. 9e).

We also examined the distributions of activity ranges (Δφ) measured for the two groups of responses. Consistent with range adaptation, the two distributions were statistically indistinguishable (p = 0.10, Kruskal–Wallis test; Fig. 9f). One possible concern is that this last result may be due to saturation or ceiling effects, as opposed to adaptation. However, it can be noted that the two activity traces in Figure 9e are well separated throughout the spectrum of chosen values, and both traces are close to linear. This observation indicates that the fact that activity ranges are independent of the value range (Fig. 9f) is genuinely due to range adaptation.

We repeated these same analyses for the population of 256 Chosen Value responses recorded in ACCv, and we obtained very similar results (Fig. 10). In ACCv, responses with positive and negative slopes were 172 of 256 (67%) and 84 of 256 (33%), respectively. The two distributions had equal median (p = 0.69, Wilcoxon's rank-sum test; Fig. 10a). Thus, we rectified negative-slope responses and pooled them with positive-slope responses (Fig. 10b). We separated neuronal responses in two groups depending on whether they had been recorded in sessions with value range ΔV ≤ 6 uB or ΔV > 6 uB, and we examined the slope of the encoding as a function of the value range. Consistent with range adaptation, the average slope was significantly higher when neurons were recorded with ΔV ≤ 6 uB compared with when neurons were recorded with ΔV > 6 uB (p < 10⁻¹⁰, ANOVA; Fig. 10c). We then examined the neuronal firing rate directly as a function of the Chosen Value. We normalized responses by subtracting the baseline activity, and we plotted the entire population of 256 responses (Fig. 10d). In general, activity ranges (y-axis) varied substantially across the population. However, when we averaged the firing rates separately for the two groups of responses, neuronal adaptation appeared evident (Fig. 10e). The distributions of activity ranges measured for the two groups of responses were statistically indistinguishable (p = 0.65, Kruskal–Wallis test; Fig. 10f). Importantly, the activity traces for the two groups of responses (Fig. 10e) were well separated throughout the spectrum of chosen values, indicating that the result in Figure 10f was indeed due to neuronal adaptation as opposed to saturation or ceiling effects.

In conclusion, our analyses indicated that value-encoding neurons in both ACCd and ACCv undergo range adaptation. In both areas, neuronal responses encoding the Chosen Value adapted in such a way that a given range of firing rates represented different ranges of values in different behavioral conditions.

Partial adaptation on the timescale of individual trials

The results illustrated in Figures 9 and 10 demonstrated range adaptation on the timescale of behavioral sessions, which in this study typically included 200–400 trials. One interesting question concerns the time course of the adaptation process. In our experiments, monkeys learned the range of values included in each session by performing in the task. Thus, adaptation presumably took place over the course of multiple trials at the beginning of each session. In the following analysis, we examined whether and to what extent adaptation could be observed on a trial-by-trial basis.

We used the same approach previously undertaken for the analysis of OFC. In our experiments, different trial types were randomly interleaved so that the Chosen Value varied from trial to trial. The hypothesis that neurons undergo range adaptation leads to a simple prediction. Consider for example the cell in Figure 11a, recorded in a session in which the Chosen Value varied between 2 and 6 uB. Consider now only trials in which the monkey chose 3B (Chosen Value = 3 uB). These trials could follow trials in which the Chosen Value was <3 uB; alternatively, they could follow trials in which the Chosen Value was >3 uB. If the neuron indeed undergoes adaptation, then its activity should depend at least slightly on the value chosen in the previous trial. Specifically, its activity should be slightly elevated when the value chosen in the previous trial was <3 uB; it should be slightly depressed when the value chosen in the previous trial was >3 uB.

To test this prediction more generally, we divided trials depending on whether the Chosen Value in the current trial (trial n) was higher or lower than the Chosen Value in the previous trial (trial n − 1). Indicating with V(k) the value chosen in trial k, the two groups of trials were thus defined as V(n) > V(n − 1) and V(n) < V(n − 1). Consistent with the prediction, the activity for trials V(n) > V(n − 1) was slightly higher than that for trials V(n) < V(n − 1) (Fig. 11a).

To quantify this effect, we computed for each trial type the “normalized difference,” defined as the difference between the firing rate obtained for trials V(n) > V(n − 1) and the firing rate obtained for trials V(n) < V(n − 1), normalized by the firing rate obtained pooling all trials. We then defined δ as the average normalized difference (average across trial types). Thus, δ represented the average percentage modulation of trial n − 1 on trial n. For the response in Figure 11a, we obtained δ = 13%. Two panels in Figure 11b illustrate the distributions of δ obtained, respectively, for the population of 521 Chosen Value responses recorded in ACCd and for the population of 256 Chosen Value responses recorded in ACCv. Although both areas present a substantial variability, responses with δ > 0 were a significant majority in both areas (both p < 10⁻⁴, binomial test). Furthermore, in both areas, mean(δ) was significantly greater than zero (both p < 10⁻⁶, t test). Quantitatively, we measured mean(δ) = 5% and mean(δ) = 7% in ACCd and ACCv, respectively. Interestingly, these measures are similar to the one previously obtained for OFC [mean(δ) = 6% (Padoa-Schioppa, 2009)].

To test the reliability of this result, we repeated the analysis for n − 2, n − 3, etc., dividing trials depending on whether the Chosen Value on trial n was higher or lower than the Chosen Value on trial n − 2, n − 3, etc. As expected, the adaptation effect faded rapidly with the trial distance in both areas (Fig. 11c). For a control, we repeated the analysis for n + 1 dividing trials depending on whether the Chosen Value on trial n was higher or lower than the Chosen Value on trial n + 1. We did not expect any effect in this case because neurons should not adapt to future events. Indeed, mean(δ) was statistically indistinguishable from zero.

In conclusion, a small but significant neuronal adaptation could be observed at the timescale of individual trials in both ACCd and ACCv, with an incidence and a time course very similar to those previously measured for OFC (Padoa-Schioppa, 2009).

A role of ACCd and ACCv in decision making?

From the point of view of economic choice behavior, it is particularly interesting to compare ACCd and ACCv, on the one hand, and OFC, on the other hand. In this respect, we observed three key differences. First and perhaps most important, neurons in OFC encoded both pre-decision variables (Offer Value) and postdecision variables (Chosen Value, Chosen Juice). In contrast, neurons in ACCd and ACCv encoded only post-decision variables. Second, the engagement of OFC in the trial preceded that of ACCd and ACCv. Specifically, responses peaked in the Postoffer time window for OFC, in the Late Delay time window for ACCd, and in the Postjuice time window for ACCv. [Similar findings were previously reported by Kennerley and Wallis (2009b).] Third, the direction of the saccade executed by the animal modulated the activity of neurons in ACCd. In contrast, the activity of neurons in OFC and ACCv did not depend on the visuomotor contingencies of the task. Together, our results indicate that information about the choice outcome (Chosen Value and Chosen Juice) reached ACCd and ACCv after the comparison between values (i.e., the decision) took place in other brain regions. In this respect, it is also interesting to note that, although the activity of neurons in ACCd was spatially selective, we did not find evidence for the encoding of “action value” variables Offer Value L and Offer Value R. In other words, at least from the limited vantage point provided by ACCd, it appears that decisions in our choice task did not involve the comparison of action values. Conversely, the activation timing and the coexistence in ACCd of choice outcome signals and movement direction signals raises the possibility that this area may contribute to the processes through which choice outcomes and subjective values inform the motor systems.

Prima facie, the conclusion that ACCd/v are computationally downstream of economic decisions may appear at odds with previous studies (Walton et al., 2002; Williams et al., 2004; Kennerley et al., 2006; Rudebeck et al., 2008; Camille et al., 2011). Upon closer examination, however, the present results complement well recently emerging notions on the functional role of ACC. In broad strokes, decision tasks can be distinguished depending (1) on whether the task includes or does not include a learning component and (2) on whether options are defined by stimuli and/or by actions. For example, Rudebeck et al. (2008) tested monkeys in two tasks that included a learning component and such that options were defined either only by stimuli or only by actions. A recently emerging notion is that ACC lesions seem to impair performance particularly in tasks that include a learning component, and especially if options are defined exclusively by actions. In contrast, performance is essentially spared in tasks that do not include learning and/or when options are defined by stimuli (Kennerley et al., 2006; Rudebeck et al., 2008). Notably, our economic choice task did not include a learning component. Furthermore, in any given trial, each option was associated with both a stimulus and an action. Thus, the conclusion that ACCd/v does not contribute to decision making in our task is consistent with previous results. In fact, our present observations seem to resonate with recent reports indicating a specific role of ACC in associative learning (Alexander and Brown, 2011; Kennerley et al., 2011; Wallis and Rich, 2011). It can be noted that variables Chosen Value and Chosen Juice encoded in ACCd/v, if measured in a learning context, could provide the quantities necessary to evaluate performance and thus compute a teaching signal. In this view, the role of ACC in decision making would be indirect: by influencing learning, the activity of this areas in any given trial would affect decisions in subsequent trials.

Possible roles of ACCd and ACCv in other cognitive processes

Our results are also relevant to two hypotheses previously put forth on the functional role of ACC. First, based mostly on imaging data, several authors proposed that neural activity in ACC generally reflects the cognitive conflict or the difficulty of the task (Botvinick et al., 2001; Paus, 2001; Ridderinkhof et al., 2004; Rushworth et al., 2004). In our experimental paradigm, the decision difficulty can be operationally identified with the variable Value Ratio (i.e., Oth/Ch Value). This variable has its maximum at the indifference point (Value Ratio = 1), where the two values are very similar and thus the decision is presumably difficult. As one “moves away” from the indifference point, the Value Ratio becomes smaller and the decision becomes presumably easier. In the extreme cases of forced choices (Value Ratio = 0), the decision is trivial. Consistent with this operational definition, we previously observed in a version of the task that did not impose a delay between offer and movement initiation that reaction times were highly correlated with the Value Ratio (Padoa-Schioppa et al., 2006). With this premise, current results appear in contrast with the cognitive conflict hypothesis because vanishingly few neurons encoded the Value Ratio in either area. Notably, our recordings sampled densely and extensively both banks of the cingulate sulcus. Thus, the negative result vis-à-vis decision difficulty appears credible. Interestingly, other studies searched and failed to find individual cells encoding cognitive conflict or task difficulty in the primate ACC (Ito et al., 2003; Nakamura et al., 2005). The discrepancy between single-cell and imaging studies may possibly be due to anatomical mismatches between species (e.g., human ACC include cell types not found in macaques) or perhaps to the imperfect correspondence between hemodynamic responses and neuronal spiking activity (Botvinick, 2007). In any case, the cognitive conflict hypothesis did not find support in our data.

Second, in a recent study, Hayden et al. (2009) reported that neurons in the ACC encode fictive (counterfactual) reward signals. In their experiment, monkeys selected one among eight possible targets, seven of which delivered a small amount of juice and one of which (the optimal target) delivered an amount of juice randomly variable but on average much higher than that of the other targets. On any given trial, the position of the optimal target was most likely (60% of trials) the same as in the previous trial. Thus, the optimal strategy was to select on each trial the target located in the same position occupied by the optimal target in the previous trial. If on trial n the monkey selected the optimal target, the optimal strategy on trial n + 1 was to select the same target; if on trial n the monkey selected a different target, the optimal strategy on trial n + 1 was to switch and select the position occupied by the optimal target on trial n. Since the amount of juice delivered by the optimal target varied randomly from trial to trial, switching in the latter case was optimal independently of the amount “missed” on trial n (the fictive reward). However, curiously, the fictive reward significantly affected switching in both animals. Remarkably, Hayden et al. (2009) found in ACC neurons encoding the fictive reward size as well as the actual reward size. In our experiments, fictive rewards are measured by the variable Other Value, which accounted for very few responses in our data sets. So how can our results be reconciled with those of Hayden et al. (2009)? In this respect, it can be noted that, in the study by Hayden et al. (2009), fictive rewards were behaviorally relevant because they guided target selection in the next trial. In other words, the task included a learning component that was based on both the location and the amount of the fictive reward. Thus, one possibility is that neurons recorded by Hayden et al. (2009) did not encode fictive reward per se (i.e., Other Value), but rather encoded the value of stimuli that guided the learning process. This hypothesis predicts that fictive reward signals in ACC should disappear if they do not guide learning. In this view, our conclusion that neurons in ACCd and ACCv encode the subjective value of chosen goods might extend to the subjective value of other behaviorally relevant goods. Similar considerations apply to OFC, where very few responses encoded the Other Value in our experiments (Padoa-Schioppa and Assad, 2006, 2008), whereas a recent study found fictive reward signals in a learning context (Abe and Lee, 2011) [see discussion in the study by Padoa-Schioppa and Cai (2011)].

In conclusion, we found two independent representations of subjective value in ACCd and ACCv. The general roles of these representations in various cognitive functions remain somewhat unclear. Our results suggest that these two areas do not contribute to economic decisions per se. At the same time, our results complement well recent work indicating a specific role of ACC in associative learning. More generally, the coexistence in ACCd of choice outcome signals and movement direction signals suggests that this area may represent a gateway through which the choice system informs motor systems.

• Abe and Lee, 2011.Abe H, Lee D. Distributed coding of actual and hypothetical outcomes in the orbital and dorsolateral prefrontal cortex. Neuron. 2011;70:731–741. doi: 10.1016/j.neuron.2011.03.026. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Alexander and Brown, 2011.Alexander WH, Brown JW. Medial prefrontal cortex as an action-outcome predictor. Nat Neurosci. 2011;14:1338–1344. doi: 10.1038/nn.2921. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Amiez et al., 2006.Amiez C, Joseph JP, Procyk E. Reward encoding in the monkey anterior cingulate cortex. Cereb Cortex. 2006;16:1040–1055. doi: 10.1093/cercor/bhj046. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Bermudez and Schultz, 2010.Bermudez MA, Schultz W. Reward magnitude coding in primate amygdala neurons. J Neurophysiol. 2010;104:3424–3432. doi: 10.1152/jn.00540.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Botvinick, 2007.Botvinick MM. Conflict monitoring and decision making: reconciling two perspectives on anterior cingulate function. Cogn Affect Behav Neurosci. 2007;7:356–366. doi: 10.3758/cabn.7.4.356. [DOI] [PubMed] [Google Scholar]
• Botvinick et al., 2001.Botvinick MM, Braver TS, Barch DM, Carter CS, Cohen JD. Conflict monitoring and cognitive control. Psychol Rev. 2001;108:624–652. doi: 10.1037/0033-295x.108.3.624. [DOI] [PubMed] [Google Scholar]
• Camille et al., 2011.Camille N, Tsuchida A, Fellows LK. Double dissociation of stimulus-value and action-value learning in humans with orbitofrontal or anterior cingulate cortex damage. J Neurosci. 2011;31:15048–15052. doi: 10.1523/JNEUROSCI.3164-11.2011. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Clifford et al., 2007.Clifford CW, Webster MA, Stanley GB, Stocker AA, Kohn A, Sharpee TO, Schwartz O. Visual adaptation: neural, psychological and computational aspects. Vision Res. 2007;47:3125–3131. doi: 10.1016/j.visres.2007.08.023. [DOI] [PubMed] [Google Scholar]
• Dean et al., 2008.Dean I, Robinson BL, Harper NS, McAlpine D. Rapid neural adaptation to sound level statistics. J Neurosci. 2008;28:6430–6438. doi: 10.1523/JNEUROSCI.0470-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Dunn and Clark, 1987.Dunn OJ, Clark V. Applied statistics: analysis of variance and regression. Ed 2. New York: Wiley; 1987. [Google Scholar]
• Glantz and Slinker, 2001.Glantz SA, Slinker BK. Primer of applied regression and analysis of variance. Ed 2. New York: McGraw-Hill, Medical Publishing Division; 2001. [Google Scholar]
• Griffin et al., 2005.Griffin SJ, Bernstein LR, Ingham NJ, McAlpine D. Neural sensitivity to interaural envelope delays in the inferior colliculus of the guinea pig. J Neurophysiol. 2005;93:3463–3478. doi: 10.1152/jn.00794.2004. [DOI] [PubMed] [Google Scholar]
• Hayden and Platt, 2010.Hayden BY, Platt ML. Neurons in anterior cingulate cortex multiplex information about reward and action. J Neurosci. 2010;30:3339–3346. doi: 10.1523/JNEUROSCI.4874-09.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Hayden et al., 2009.Hayden BY, Pearson JM, Platt ML. Fictive reward signals in the anterior cingulate cortex. Science. 2009;324:948–950. doi: 10.1126/science.1168488. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Hillman and Bilkey, 2010.Hillman KL, Bilkey DK. Neurons in the rat anterior cingulate cortex dynamically encode cost-benefit in a spatial decision-making task. J Neurosci. 2010;30:7705–7713. doi: 10.1523/JNEUROSCI.1273-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Ito et al., 2003.Ito S, Stuphorn V, Brown JW, Schall JD. Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science. 2003;302:120–122. doi: 10.1126/science.1087847. [DOI] [PubMed] [Google Scholar]
• Kable and Glimcher, 2009.Kable JW, Glimcher PW. The neurobiology of decision: consensus and controversy. Neuron. 2009;63:733–745. doi: 10.1016/j.neuron.2009.09.003. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Kennerley and Wallis, 2009a.Kennerley SW, Wallis JD. Evaluating choices by single neurons in the frontal lobe: outcome value encoded across multiple decision variables. Eur J Neurosci. 2009a;29:2061–2073. doi: 10.1111/j.1460-9568.2009.06743.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Kennerley and Wallis, 2009b.Kennerley SW, Wallis JD. Encoding of reward and space during a working memory task in the orbitofrontal cortex and anterior cingulate sulcus. J Neurophysiol. 2009b;102:3352–3364. doi: 10.1152/jn.00273.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Kennerley et al., 2006.Kennerley SW, Walton ME, Behrens TE, Buckley MJ, Rushworth MF. Optimal decision making and the anterior cingulate cortex. Nat Neurosci. 2006;9:940–947. doi: 10.1038/nn1724. [DOI] [PubMed] [Google Scholar]
• Kennerley et al., 2009.Kennerley SW, Dahmubed AF, Lara AH, Wallis JD. Neurons in the frontal lobe encode the value of multiple decision variables. J Cogn Neurosci. 2009;21:1162–1178. doi: 10.1162/jocn.2009.21100. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Kennerley et al., 2011.Kennerley SW, Behrens TE, Wallis JD. Double dissociation of value computations in orbitofrontal and anterior cingulate neurons. Nat Neurosci. 2011;14:1581–1589. doi: 10.1038/nn.2961. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Kobayashi et al., 2010.Kobayashi S, Pinto de Carvalho O, Schultz W. Adaptation of reward sensitivity in orbitofrontal neurons. J Neurosci. 2010;30:534–544. doi: 10.1523/JNEUROSCI.4009-09.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Koyama et al., 2001.Koyama T, Kato K, Tanaka YZ, Mikami A. Anterior cingulate activity during pain-avoidance and reward tasks in monkeys. Neurosci Res. 2001;39:421–430. doi: 10.1016/s0168-0102(01)00197-3. [DOI] [PubMed] [Google Scholar]
• Laughlin, 1989.Laughlin SB. The role of sensory adaptation in the retina. J Exp Biol. 1989;146:39–62. doi: 10.1242/jeb.146.1.39. [DOI] [PubMed] [Google Scholar]
• Lee et al., 2007.Lee D, Rushworth MF, Walton ME, Watanabe M, Sakagami M. Functional specialization of the primate frontal cortex during decision making. J Neurosci. 2007;27:8170–8173. doi: 10.1523/JNEUROSCI.1561-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Matsumoto et al., 2003.Matsumoto K, Suzuki W, Tanaka K. Neuronal correlates of goal-based motor selection in the prefrontal cortex. Science. 2003;301:229–232. doi: 10.1126/science.1084204. [DOI] [PubMed] [Google Scholar]
• Matsumoto et al., 2007.Matsumoto M, Matsumoto K, Abe H, Tanaka K. Medial prefrontal cell activity signaling prediction errors of action values. Nat Neurosci. 2007;10:647–656. doi: 10.1038/nn1890. [DOI] [PubMed] [Google Scholar]
• Nakamura et al., 2005.Nakamura K, Roesch MR, Olson CR. Neuronal activity in macaque SEF and ACC during performance of tasks involving conflict. J Neurophysiol. 2005;93:884–908. doi: 10.1152/jn.00305.2004. [DOI] [PubMed] [Google Scholar]
• Niki and Watanabe, 1979.Niki H, Watanabe M. Prefrontal and cingulate unit activity during timing behavior in the monkey. Brain Res. 1979;171:213–224. doi: 10.1016/0006-8993(79)90328-7. [DOI] [PubMed] [Google Scholar]
• Padoa-Schioppa, 2009.Padoa-Schioppa C. Range-adapting representation of economic value in the orbitofrontal cortex. J Neurosci. 2009;29:14004–14014. doi: 10.1523/JNEUROSCI.3751-09.2009. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Padoa-Schioppa, 2011.Padoa-Schioppa C. Neurobiology of economic choice: a good-based model. Annu Rev Neurosci. 2011;34:333–359. doi: 10.1146/annurev-neuro-061010-113648. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Padoa-Schioppa and Assad, 2006.Padoa-Schioppa C, Assad JA. Neurons in orbitofrontal cortex encode economic value. Nature. 2006;441:223–226. doi: 10.1038/nature04676. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Padoa-Schioppa and Assad, 2008.Padoa-Schioppa C, Assad JA. The representation of economic value in the orbitofrontal cortex is invariant for changes of menu. Nat Neurosci. 2008;11:95–102. doi: 10.1038/nn2020. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Padoa-Schioppa and Cai, 2011.Padoa-Schioppa C, Cai X. The orbitofrontal cortex and the computation of subjective value: consolidated concepts and new perspectives. Ann N Y Acad Sci. 2011;1239:130–137. doi: 10.1111/j.1749-6632.2011.06262.x. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Padoa-Schioppa et al., 2006.Padoa-Schioppa C, Jandolo L, Visalberghi E. Multi-stage mental process for economic choice in capuchins. Cognition. 2006;99:B1–B13. doi: 10.1016/j.cognition.2005.04.008. [DOI] [PubMed] [Google Scholar]
• Paus, 2001.Paus T. Primate anterior cingulate cortex: where motor control, drive and cognition interface. Nat Rev Neurosci. 2001;2:417–424. doi: 10.1038/35077500. [DOI] [PubMed] [Google Scholar]
• Rangel and Hare, 2010.Rangel A, Hare T. Neural computations associated with goal-directed choice. Curr Opin Neurobiol. 2010;20:262–270. doi: 10.1016/j.conb.2010.03.001. [DOI] [PubMed] [Google Scholar]
• Ridderinkhof et al., 2004.Ridderinkhof KR, Ullsperger M, Crone EA, Nieuwenhuis S. The role of the medial frontal cortex in cognitive control. Science. 2004;306:443–447. doi: 10.1126/science.1100301. [DOI] [PubMed] [Google Scholar]
• Rudebeck et al., 2008.Rudebeck PH, Behrens TE, Kennerley SW, Baxter MG, Buckley MJ, Walton ME, Rushworth MF. Frontal cortex subregions play distinct roles in choices between actions and stimuli. J Neurosci. 2008;28:13775–13785. doi: 10.1523/JNEUROSCI.3541-08.2008. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Rushworth and Behrens, 2008.Rushworth MF, Behrens TE. Choice, uncertainty and value in prefrontal and cingulate cortex. Nat Neurosci. 2008;11:389–397. doi: 10.1038/nn2066. [DOI] [PubMed] [Google Scholar]
• Rushworth et al., 2004.Rushworth MF, Walton ME, Kennerley SW, Bannerman DM. Action sets and decisions in the medial frontal cortex. Trends Cogn Sci. 2004;8:410–417. doi: 10.1016/j.tics.2004.07.009. [DOI] [PubMed] [Google Scholar]
• Rushworth et al., 2007.Rushworth MF, Behrens TE, Rudebeck PH, Walton ME. Contrasting roles for cingulate and orbitofrontal cortex in decisions and social behaviour. Trends Cogn Sci. 2007;11:168–176. doi: 10.1016/j.tics.2007.01.004. [DOI] [PubMed] [Google Scholar]
• Sallet et al., 2007.Sallet J, Quilodran R, Rothé M, Vezoli J, Joseph JP, Procyk E. Expectations, gains, and losses in the anterior cingulate cortex. Cogn Affect Behav Neurosci. 2007;7:327–336. doi: 10.3758/cabn.7.4.327. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Seo and Lee, 2007.Seo H, Lee D. Temporal filtering of reward signals in the dorsal anterior cingulate cortex during a mixed-strategy game. J Neurosci. 2007;27:8366–8377. doi: 10.1523/JNEUROSCI.2369-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Shidara and Richmond, 2002.Shidara M, Richmond BJ. Anterior cingulate: single neuronal signals related to degree of reward expectancy. Science. 2002;296:1709–1711. doi: 10.1126/science.1069504. [DOI] [PubMed] [Google Scholar]
• Tobler et al., 2005.Tobler PN, Fiorillo CD, Schultz W. Adaptive coding of reward value by dopamine neurons. Science. 2005;307:1642–1645. doi: 10.1126/science.1105370. [DOI] [PubMed] [Google Scholar]
• Wallis and Kennerley, 2010.Wallis JD, Kennerley SW. Heterogeneous reward signals in prefrontal cortex. Curr Opin Neurobiol. 2010;20:191–198. doi: 10.1016/j.conb.2010.02.009. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Wallis and Rich, 2011.Wallis JD, Rich EL. Challenges of interpreting frontal neurons during value-based decision-making. Front Neurosci. 2011;5:124. doi: 10.3389/fnins.2011.00124. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Walton et al., 2002.Walton ME, Bannerman DM, Rushworth MF. The role of rat medial frontal cortex in effort-based decision making. J Neurosci. 2002;22:10996–11003. doi: 10.1523/JNEUROSCI.22-24-10996.2002. [DOI] [PMC free article] [PubMed] [Google Scholar]
• Williams et al., 2004.Williams ZM, Bush G, Rauch SL, Cosgrove GR, Eskandar EN. Human anterior cingulate neurons and the integration of monetary reward with motor responses. Nat Neurosci. 2004;7:1370–1375. doi: 10.1038/nn1354. [DOI] [PubMed] [Google Scholar]