pmc.ncbi.nlm.nih.gov

Optogenetic Interrogation of Dopaminergic Modulation of the Multiple Phases of Reward-Seeking Behavior

  • ️Invalid Date

Abstract

Phasic activation of dopaminergic neurons is associated with reward-predicting cues and supports learning during behavioral adaptation. While noncontingent activation of dopaminergic neurons in the ventral tegmental are (VTA) is sufficient for passive behavioral conditioning, it remains unknown whether the phasic dopaminergic signal is truly reinforcing. In this study, we first targeted the expression of channelrhodopsin-2 to dopaminergic neurons of the VTA and optimized optogenetically evoked dopamine transients. Second, we showed that phasic activation of dopaminergic neurons in freely moving mice causally enhances positive reinforcing actions in a food-seeking operant task. Interestingly, such effect was not found in the absence of food reward. We further found that phasic activation of dopaminergic neurons is sufficient to reactivate previously extinguished food-seeking behavior in the absence of external cues. This was also confirmed using a single-session reversal paradigm. Collectively, these data suggest that activation of dopaminergic neurons facilitates the development of positive reinforcement during reward-seeking and behavioral flexibility.

Introduction

A critical function of the CNS is to integrate endogenous and environmental stimuli into motivated behaviors. The neuroanatomical substrates of motivated behaviors are composed of diffuse multisynaptic systems distributed throughout the brain and include the mesocorticolimbic pathway, the medial forebrain bundle (MFB), and lateral hypothalamus, habenula, raphe nuclei, prefrontal cortex, amygdala, and striatal regions (Wise, 2004). These brain regions have been originally identified as “brain stimulation reward” sites using intracranial self-stimulation (ICSS) procedures, since animals would repeatedly press a lever to self-stimulate electrically unidentified fibers and cells, presumably neurons, within these target sites (Olds and Milner, 1954; Gallistel and Karras, 1984; Fiorino et al., 1993; Wise, 2004; Cheer et al., 2007). Among these multisynaptic neural circuits, dopaminergic neurons in the midbrain encode the association between sensory and environmental modalities that signal the availability of rewards (Wise, 2004; Berridge, 2007; Schultz, 2007). The switch from tonic to phasic activation of midbrain dopaminergic neurons correlates with unexpected sensory stimuli and reward-predicting cues and results in dopamine release in forebrain structures (Schultz et al., 1997; Phillips et al., 2003; Cheer et al., 2007). Those signals update the predictive value of stimuli during associative learning for future reinforcement of behavioral actions (Fiorillo et al., 2003; Bayer and Glimcher, 2005; Tobler et al., 2005; Day et al., 2007; Schultz, 2007). Although such experimental evidence supports a role for dopamine in reward-seeking behaviors, dopamine may not be required for hedonic responses per se (Wise, 2004; Berridge, 2007) and reward learning or preference (Cannon and Palmiter, 2003; Robinson et al., 2005; Flagel et al., 2011).

To further investigate the role of dopaminergic signal in reinforcement and motivation, we combined in vivo optogenetic stimulation of dopaminergic neurons in the VTA during the successive phases of a food-seeking operant task. We studied the consequence of selective activation of dopaminergic neurons on lever discrimination during the acquisition phase of food-seeking behavior. We further tested whether optogenetic activation of dopaminergic neurons was sufficient to reactivate and reverse previously extinguished food-seeking behaviors.

Materials and Methods

Animals.

Male tyrosine hydroxylase (Th)::IRES-Cre knock-in mice (EM:00254; B6.129X1-Thtm1(cre)Te/Kieg; European Mouse Mutant Archive) were housed in a temperature- and humidity-controlled (40–60%) room under a reverse 12 h light/dark cycle (light onset at 9:00 A.M.). Unless specified otherwise, mice were given food and water ad libitum. All the experiments meet guidelines of the National Institutes of Health Guide for the Care and Use of Laboratory Animals and were approved by Stanford University.

Virus preparation.

Cre-inducible recombinant adeno-associated virus (AAV) vector was used to genetically target channelrhodopsin-2 (ChR2) expression to dopaminergic neurons in the VTA as described previously (Tsai et al., 2009). Briefly, double-floxed reverse EF-1 α::ChR2-eYFP and EF-1 α::eYFP cassette were packaged in AAV vectors and serotyped with AAV5 coat proteins, and packaged (2 × 1012 genome copies/ml) by the viral vector core at the University of North Carolina.

Stereotactic AAV injection and cannula implantation.

Eight- to 10-week-old male (Th)::IRES-Cre mice were anesthetized using a ketamine/xylazine mixture (ketamine, 80 mg/kg; xylazine, 15–20 mg/kg) diluted in sterile saline solution (NaCl, 0.9%). Stereotactic AAV virus delivery and cannula surgery procedures were conducted as described previously (Tsai et al., 2009). Briefly, cannula guides for optic fiber insertion were implanted over the right VTA [anteroposterior (AP), −3.44 mm; mediolateral (ML), 0.48 mm; dorsoventral (DV), 4.4 mm] and secured to the skull with metabond and dental cement. One microliter of purified double-floxed AAV:ChR2-eYFP or AAV:eYFP virus was injected unilaterally through the cannula guide. All mice were singly housed after surgery and recovered for at least 3 weeks before electrophysiological and behavioral experiments. VTA coronal section maps were made according to the mouse brain atlas (Paxinos and Franklin, 2001) to represent the placement of the cannula guide in each animal (brains of four animals were lost during tissue processing) (see Fig. 2B).

Figure 2.

Figure 2.

Experimental strategy and verification of cannula placement. A, Timeline of the experimental strategy and instrumental conditions during the acquisition, extinction, reactivation, and reversal phases of the operant behavioral task. B, Representation of the optical–neural interface for deep brain light delivery (left). The optical fiber is shown in blue. The right panel represents the placement of the cannula guides in the brains of ChR2 (blue circles) and control mice (red circles; n = 10 in each group; some brains were lost during tissue processing) that were included in this study [drawings were generated according to the mouse brain atlas of Paxinos and Franklin (2001)]. Scale bar, 500 μm. cp, Cerebral peduncle; ml, medial lemnicus; SNC, substantia nigra pars compacta; SNL, substantia nigra pars lateralis; SNR, substantia nigra pars reticularis.

In vivo optrode recording.

Simultaneous optical stimulation and electrical recording of VTA neurons from (Th)::IRES-Cre mice transduced with double-floxed AAV:ChR2-eYFP were performed using optrode recording as described previously (Tsai et al., 2009). Briefly, recordings were conducted with the optrode initially placed at the boundary of VTA and gradually lowered in 0.1 mm increments. The optical fiber was coupled to a 473 nm solid-state laser diode with ∼30 mW of output from the 200 μm fiber. Single-unit recordings were done in mice anesthetized with a ketamine/xylazine mixture (as described above). Signals were recorded and bandpass filtered at 300 Hz (low)/5 kHz (high) using an 1800 Microelectrode AC amplifier.

In vivo fast-scan cyclic voltammetry.

Fast-scan cyclic voltammetry (FSCV) experiments were conducted using methods described previously (Tsai et al., 2009). Briefly, mice were anesthetized with a ketamine/xylazine mixture and placed in a stereotaxic frame. A craniotomy was done above the nucleus accumbens (NAcc; AP, +1.4 mm; ML, 0.1 mm) and the VTA (AP, −3.44 mm; ML, 0.48 mm; DV, 4.0 mm) (Zhang et al., 2010). An Ag/AgCl reference electrode was also implanted in the contralateral forebrain. An optical fiber was coupled to an extracellular electrophysiological electrode and positioned just dorsal to the VTA. Unit activity was then monitored during the FSCV to confirm that the optical fiber was in the VTA and capable of activating dopaminergic neurons. A carbon fiber electrode (∼100 μm in length) for voltammetric recordings was then lowered (DV, −3.5 mm; lowered in 0.25 mm intervals) into the NAcc. Voltammetric measurements were made every 100 ms by application of a triangle waveform (−0.4 to +1.3 to −0.4 V vs Ag/AgCl at 400 V/s) to the carbon fiber electrode. Data were acquired and analyzed using software written in LabVIEW. The light-evoked dopamine transient was identified by the background-subtracted cyclic voltammogram. Following experiments in anesthetized mice, electrodes were calibrated in vitro with 1 μm dopamine yielding an average calibration factor of 26 nA/μm, which was used to estimate the concentration of light-evoked dopamine transients in vivo. The mean peak of the dopamine transient in the NAcc induced by 20 blue light pulses (5 ms width; 473 nm) delivered at 25 Hz was measured at 255.38 ± 55.74 nm.

In vivo electrical stimulations.

In the separate set of experiments, a bipolar stimulating electrode was lowered to the VTA using the same coordinates as for the optical fiber. The reference and carbon fiber electrodes were connected to a voltammetric amplifier (Electronics Design Facility, University of North Carolina, Chapel Hill, NC), and voltammetric recordings were made as described above. Dopamine release was evoked using stimulating parameters, which were applied for the optical stimulation, including altering the number of pulses (1–20), frequency (1, 2, 5, 10, 25, and 50 Hz) and single-pulse duration (5 ms). The stimulating current was maintained at 300 μA. Recorded signals showed an oxidation peak at +0.65 V and a reduction peak at −0.2 V (vs Ag/AgCl reference), ensuring that the released chemical was dopamine.

Operant task procedures.

Food-restricted (85% of original weight) (Th)::IRES-Cre male mice transduced with AAV:ChR2-eYFP (ChR2; n = 12) or AAV:eYFP (control; n = 12) were trained for the food-seeking operant task and optogenetic stimulation during 20 min daily sessions. Before each session, animals were connected to an opaque optical fiber for optical stimulation in the VTA through previously implanted cannula guide (Zhang et al., 2010). Optical fibers were removed after each session. A single press on the active lever was paired with the delivery of one 20 mg food pellet (BioServ) and optical stimulations [20 light pulses (473 nm; 5 ms) delivered at 25 Hz every second over 5 s] under a fixed ratio 1 (FR1), 5 s time out (TO) schedule of reinforcement. Both ChR2 and control mice received this optical stimulation when food rewards were earned unless stated otherwise. A single press on the inactive lever was paired only with the delivery of one 20 mg food pellet (i.e., without optical stimulation) under the same schedule of reinforcement (FR1, 5 s TO). During the acquisition session, cue lights above each lever were on; responses on the levers turned the corresponding cue lights off for 5 s (i.e., during the TO period). Responses on the active and inactive levers during TO periods were recorded but had no scheduled consequences (i.e., no optical stimulation and no food pellet delivery). After the acquisition of stable lever-pressing behavior (i.e., <30% variation in lever-press activity over three consecutive sessions), mice underwent a minimum of six consecutive 30 min daily extinction sessions, during which both optical stimulation and food pairing were absent. Cue lights were turned off during the extinction sessions. To keep experimental conditions constant during the operant task, a sham fiber was connected to the animal before (and removed after) each extinction session. Once the behavioral responses were extinguished (<30% variation in lever-press activity over three consecutive sessions), animals underwent a 1 d reactivation procedure (30 min), during which the conditions were identical to those in the extinction phase (see above), but single responses on the active lever were paired to phasic optical stimulations identical to those delivered during the acquisition phases of the food-seeking behavior. As for the extinction phase, cue lights were turned off, and food delivery was not paired to lever presses. Following the reactivation procedure, animals were tested for a 1 d (30 min) reversal procedure, during which the active and inactive levers were switched. In this paradigm, a single response on the previously inactive lever (left lever) was paired to the delivery of phasic optical stimulations (FR1, 5 s TO). Note that performances of the mice showed interindividual variability as revealed by the variable duration of the acquisition and the extinction of the operant task.

For each step of the operant task, the total number of responses on both active and inactive levers were recorded over the entire session. Food consumption during each session was quantified, and animal weight was monitored daily over the entire experiment.

Statistical analysis.

Total responses for acquisition, extinction, reactivation, and reversal phases were analyzed by using a one-way mixed-design ANOVA. Statistically significant effects in the ANOVAs were followed by pairwise comparisons between means of active and inactive lever presses using Tukey's post hoc test in ChR2 versus control mice. Quantification of the number of head entry detections, food intakes, and rewards earned during the reversal were analyzed using a two-tailed unpaired Student's t test between ChR2 and control mice. Correlations of behavioral responses across the operant task were analyzed using a linear regression. Values of p < 0.05 were considered as significant for all analyses.

Results

Optimization of optogenetic control of dopaminergic neuron firing and dopamine release in vivo

We first genetically targeted the expression of the light-driven cation channel ChR2 (Nagel et al., 2003; Boyden et al., 2005) to VTA dopaminergic neurons using a Cre-inducible AAV vector (Tsai et al., 2009). This strategy leads to selective and long-lasting ChR2 expression (up to 6 months) in dopaminergic neurons in the VTA of transduced knock-in (Th)::IRES-Cre mice, as described previously (Tsai et al., 2009).

To optimize optogenetic control of dopamine release in vivo, we recorded dopamine transients in the NAcc using FSCV, while optically stimulating VTA dopaminergic neurons in anesthetized ChR2 mice (Phillips et al., 2003; Tsai et al., 2009). We found that 5 ms light-pulse trains delivered at 25 Hz were the most efficient at inducing dopamine release (Fig. 1A), and robust release of dopamine in the NAcc was obtained with 20 consecutive light pulses (Fig. 1B). Those are consistent with natural reward-triggered dopamine transients (Roitman et al., 2004; Parker et al., 2010). Moreover, we found that dopamine release in the NAcc upon optical stimulation at 25 Hz (5 ms pulse width; 20 pulses) was similar to electrical stimulation at 50 Hz (5 ms pulse width; 20 pulses) (Fig. 1A,B). Using optrode recording in anesthetized mice, we confirmed that 25 Hz (5 ms pulse width; 20 pulses) optical stimulation reliably evoked spikes in ChR2-expressing VTA dopaminergic neurons in vivo (Fig. 1C). Note that the efficiency of light-evoked action potentials decreases at frequencies higher than 25 Hz (Fig. 1D).

Figure 1.

Figure 1.

Optimization of optogenetic control of dopaminergic neurons in vivo. A, In vivo FSCV measurements of optically (white) and electrically induced (black) dopamine transients in the NAcc from anesthetized ChR2-expressing mice over a range of stimulation frequencies (1 to 50 Hz; n = 5). B, Comparison of dopamine transients upon increasing the number (1 to 20) of optical or electrical stimulation (5 ms; n = 3). C, In vivo optrode recording of ChR2-expressing dopaminergic neurons in the VTA showing phasic firing evoked by 25 Hz light pulse trains (20 flashes; 5 ms). D, Percentage of action potentials evoked by 20 light flashes at different frequencies (1 to 50 Hz; n = 3). Error bars indicate SEM.

Optogenetic activation of dopaminergic neurons has positive reinforcement properties

To test whether activation of dopaminergic neurons modulates reward and motivation, we used optimized parameters to unilaterally activate dopaminergic neurons in the VTA of freely moving mice during the multiple phases of a food-seeking operant task (Fig. 2A). Knock-in (Th)::IRES-Cre mice were transduced with Cre-inducible ChR2-enhanced yellow fluorescent protein (eYFP) (ChR2) or eYFP (control) AAV through the cannula guide 3 weeks before the onset of the operant training (see Materials and Methods). Cannula placement was confirmed at the end of the experiment, and mice with a cannula guide outside the target area were removed from the study (Fig. 2B). We food restricted animals to 85% of their original body weight (Fig. 3A) and trained them for food-seeking behavior in a two-lever operant task. Both levers delivered 20 mg food pellets in a fixed ratio schedule of reinforcement (FR1), but only one (hereafter named “active lever”) was paired with optical stimulation of VTA dopaminergic neurons through a unilateral opaque optical fiber (Fig. 2A, timeline). As shown in Figure 3, B and C, ChR2 mice progressively discriminated the active lever (i.e., above 60% discrimination threshold). At the end of the acquisition phase, we found that only ChR2 mice exhibited significantly higher responses on the active lever compared to control mice (89.82 ± 28.74 vs 32.13 ± 4.49, respectively; p < 0.05; n = 12 in each group), whereas both groups showed similar number of responses on the inactive lever (28.74 ± 7.18 vs 26.87 ± 4.64, respectively; p > 0.05; n = 12 in each group) (Fig. 3D). Importantly, despite the fact that ChR2 mice received a greater number of optical stimulations compared to control mice (38.82 ± 8.73 vs 21.38 ± 2.56, respectively; p < 0.05; n = 12 in each group), both groups exhibited similar food consumption (number of food pellets eaten, 45.23 ± 9.20 vs 37.58 ± 5.75, respectively; p > 0.05; n = 12 in each group) (Fig. 3E) and numbers of head entries into the food receptacle (55.98 ± 5.92 vs 68.35 ± 8.43, respectively; p > 0.05; n = 12 in each group) (Fig. 3E, inset). Despite the fact that ChR2 mice earned more total food pellets than control mice, their food consumption (i.e., number of food pellets consumed per session) was similar to control mice, suggesting that food consumption was not affected by the optogenetic stimulation.

Figure 3.

Figure 3.

Optogenetic activation of dopaminergic neurons in vivo facilitates positive reinforcement during food-seeking behavior. A, Body weight throughout the behavioral training. ChR2 and control (Ctrl) mice (n = 12 in each group) were food restricted, and their body weight was maintained at 85% of baseline value. Data are expressed as mean ± SEM. B, Representative patterns of lever presses responses (vertical lines) from ChR2 (blue) and control mice (red) during the acquisition phase. C, Time course representation of the behavioral responses during acquisition of food-seeking behavior in ChR2 (blue; n = 12) and control (red; n = 12) mice. Note that the x-axis is interrupted because the duration of the acquisition phase varies among animals due to interindividual variability. D, Quantification of responses at the end of the acquisition phase (n = 12 in each group). E, Food intake (number of food pellets consumed per session) of ChR2 and control mice (n = 12 in each group) at the end of the acquisition phase. Inset, Number of head entries detected (HED) in the food receptacle during the last three days of the acquisition phase in ChR2 and control mice (n = 12 in each group). F, Quantification of the number of responses on the active (i.e., paired with optical stimulation) and the inactive lever from ChR2 and control mice in the absence of food reward (n = 6 in each group). G, Quantification of the number of responses of ChR2 and control mice (n = 6 in each group) during the acquisition of lever-pressing behavior for food reward in the absence of optical stimulation. Data are expressed as the mean ± SEM. *p < 0.05 using a one-way ANOVA analysis followed by Tukey's post hoc test.

In a separate set of experiments, we tested whether optogenetic activation of dopaminergic neurons could facilitate optical “self-stimulation”-like behavior in the absence of food reward, similar to electrical self-stimulation (i.e., ICSS) behavior (Olds and Milner, 1954; Carlezon and Chartoff, 2007; Cheer et al., 2007). To assess this, one of the levers (i.e., active lever) was paired with the phasic optical stimulation of dopaminergic neurons, although no food rewards were delivered upon lever presses. In this paradigm, we found that ChR2 mice did not develop a discrimination for the active lever (Fig. 3F). Indeed, both ChR2 mice and control mice displayed low, but stable, lever presses (fewer than five lever presses per session) and did not discriminate the active lever, since there was no significant difference between active and inactive lever pressing (Fig. 3F). This was not due to cognitive or motor disabilities since both ChR2 and control mice exhibited a clear lever-pressing behavior for food in the absence of optical stimulation (in that experiment, only food delivery was paired with lever presses) (Fig. 3G). Collectively, these results suggest that activation of dopaminergic neurons is sufficient to induce positive reinforcement and assign preference to the lever that is effectively associated with phasic activation of dopaminergic neuron activation during the food-seeking operant behavior but not during the optical self-stimulation-like behavior.

Optogenetic activation of dopaminergic neurons is sufficient to reactivate previously extinguished food-seeking behavior

Upon acquisition of the operant behavior, ChR2 and control mice underwent a full extinction procedure, during which cue lights were turned off, and neither food pellets nor optical stimulations were delivered (Fig. 2A, timeline). At the end of the extinction phase (6 to 29 d) (Fig. 4A), both ChR2 and control animals exhibited similar lever presses on the active (27.38 ± 4.93 and 26.81 ± 6.10, respectively; p > 0.05) and inactive levers (21.03 ± 3.37 and 16.64 ± 2.37, respectively; p > 0.05) (Fig. 4A). Note that the duration of the extinction procedure shows interindividual variability.

Figure 4.

Figure 4.

Optogenetic activation of dopaminergic neurons reactivates previously extinguished reward-seeking behavior. A, Time course representation of the responses during the extinction procedure in ChR2 (blue) and control (red) mice (n = 12 in each group). The x-axis is interrupted since animals showed interindividual variability in the duration of the extinction (6 to 29 d). Note that the total duration required for extinction of self-stimulation behavior was not significantly different between ChR2 and control mice. B, Behavioral responses of ChR2 and control mice during the reactivation session (n = 12 in each group). Data are expressed as the mean ± SEM. *p < 0.05 using a one-way ANOVA followed by Tukey's post hoc test. C, D, Linear regression of the number of responses on the active lever during the last day of the acquisition phase on the reactivation day for ChR2 (C) and control (D) mice. Dashed lines represent the 95% confidence interval. *p < 0.05.

To test whether phasic activation of dopaminergic neurons could reactivate previously extinguished lever-pressing behavior, we used a single-session reactivation procedure (Fig. 2A, timeline). We found that in the absence of cue lights and food reward, only ChR2 mice showed a reactivation of previously extinguished operant behavior toward the active lever compared to control mice (134.10 ± 34.86 vs 38.58 ± 6.15, respectively; p < 0.05; n = 12 in each group; Fig. 4B), whereas responses on the inactive lever remained unchanged in both groups (20.15 ± 3.04 vs 24.67 ± 6.33, in ChR2 vs control mice, respectively; p > 0.05; n = 12 in each group) (Fig. 4B). The number of responses on the active lever during the acquisition phase significantly correlated with the number of lever presses observed during the reactivation phase in ChR2, but not in control mice (p < 0.001 and p > 0.05, respectively) (Fig. 4C,D). The absence of food reward during the reactivation phase rules out possible residual food-mediated reinforcing actions and suggests that phasic activation of dopaminergic neurons could have positive reinforcing properties.

To further rule out possible confounding factors such as reactivation of habits toward the active lever, we evaluated the behavioral flexibility of the ChR2 animals using a single-session reversal paradigm during which active and inactive levers were switched. We found that 75% of ChR2 mice (9 of 12) responded to this challenge by switching their responses toward the previously inactive (left) lever (Fig. 5A), whereas control mice did not, as assessed by the discrimination index (difference of responses between the active and the inactive levers during the reactivation and reversal procedures; p < 0.01) (Fig. 5B). As a consequence, ChR2 mice obtained more optical stimulations than control animals during the reversal phase (37.92 ± 7.23 vs 14.79 ± 2.85, respectively; p < 0.05; n = 12 in each group) (Fig. 5C). This behavioral adaptation significantly correlated with the animals' performance during the acquisition of operant behavior in ChR2, but not in control mice (p < 0.0001 and p > 0.05, respectively) (Fig. 5D,E). Consistent with the reactivation phase of the operant task, the behavioral adaptation of ChR2 animals during the reversal phase further supports a role for dopaminergic neurons in positive reinforcement.

Figure 5.

Figure 5.

Optogenetic activation of dopaminergic neurons participates to behavioral flexibility. A, Individual responses of ChR2 mice on the previously inactive (left) lever during the reactivation and reversal single sessions. B, Representation of the discrimination index (difference of responses on the active and inactive levers during the reactivation and the reversal procedures). Data are expressed as mean ± SEM. **p < 0.01 using a one-way ANOVA followed by Tukey's post hoc test. C, Quantification of the optical stimulations earned by ChR2 and control mice (n = 12 in each group) during the reversal learning procedure. *p < 0.05 using a two-tailed unpaired Student's t test. D, E, Linear regression of the number of responses on the active lever during the last day of the acquisition phase on the reversal day for ChR2 (D) and control (E) mice. Dashed lines represent the 95% confidence interval. *p < 0.05.

Discussion

In an effort to understand the reinforcing properties of dopaminergic neurons, we optogenetically activated dopaminergic neurons in freely moving mice during the multiple phases of a novel optical self-stimulation-like behavior as it was observed in the classical electrical self-stimulation (i.e., ICSS) studies in rodents (Olds and Milner, 1954; Carlezon and Chartoff, 2007; Cheer et al., 2007). ICSS has been used extensively to identify brain stimulation reward sites in rodents. Among those, behaviorally relevant fibers in the MFB were found to play an important role in the development of ICSS behavior. Fibers activated by electrical stimulation (so-called “first stage” fibers) are in fact rostrocaudal glutamatergic fibers targeting the VTA that eventually activate the NAcc-projecting dopaminergic neurons (Wise and Bozarth, 1984; Bielajew and Shizgal, 1986; Gallistel, 1986). Electrical stimulations along the MFB are associated with variable dopamine release in the NAcc (Millar et al., 1985; Gratton et al., 1988; May and Wightman, 1989; Fiorino et al., 1993), whereas reinforcing actions are attenuated by dopamine receptor antagonists and potentiated by drugs that increase the dopaminergic tone (Fouriezos and Wise, 1976; Franklin, 1978; Gallistel and Karras, 1984; Wise, 2004). Although they showed more variable effects than MFB stimulation, electrical stimulations of the VTA also increase dopamine release in the NAcc and are facilitated by drugs enhancing the dopaminergic tone as well (Fibiger et al., 1987; Phillips et al., 1989; Blaha and Phillips, 1990). Thus, neural substrates of ICSS behavior directly or indirectly converge to result in activation of midbrain dopaminergic neurons.

In this study we used an optogenetic strategy to selectively activate dopaminergic neurons during a self-optical-stimulation-like operant task. Consistent with our previous study (Tsai et al., 2009), we showed that food-restricted ChR2 animals progressively developed a preference for a lever associated with phasic optical stimulation of dopaminergic neurons during the acquisition phase of a food-seeking operant conditioning. Interestingly, we found that optogenetic activation of dopaminergic neurons alone (i.e., in the absence of food rewards) was not effective in driving self-optical-stimulation-like behavior, as suggested by ICSS (Olds and Milner, 1954; Carlezon and Chartoff, 2007; Cheer et al., 2007). There are several possible explanations for this. First, self-stimulation-like behavior refers to ICSS experiments (Olds and Milner, 1954; Gallistel et al., 1981) that identified “brain stimulation reward” sites, including the MFB, lateral hypothalamus, VTA, and several brainstem nuclei (the locus ceruleus, raphe, lateral–dorsal tegmental area/pontine tegmentum), since animals would repeatedly press a lever to self-stimulate electrically fibers and cells, presumably neurons, within these brain areas (Wise, 2004; Carlezon and Chartoff, 2007). Electrical stimulation of these sites activates a heterogenic population of cells and fibers of passage, including glutamatergic fibers (Wise and Bozarth, 1984; Bielajew and Shizgal, 1986; Gallistel, 1986; Geisler et al., 2007) and cholinergic fibers (Blaha et al., 1996; Omelchenko and Sesack, 2005) that eventually excite NAcc-projecting dopaminergic neurons and induce dopamine release in the NAcc (Gratton et al., 1988; Fiorino et al., 1993; Wise, 2004). Although the consequences of electrical stimulation during ICSS converge toward the activation of dopaminergic neurons and dopamine release, coactivation of parallel excitatory and inhibitory pathways is likely to contribute to the strength of the ICSS behavior. Thus, it is likely that concomitant activation of additional nondopaminergic circuits (e.g., related to food-seeking behavior) are required for the acquisition of a self-stimulation-like behavior using optogenetic control of midbrain dopaminergic neurons, as suggested by our results. Food reward is known to activate both dopaminergic and nondopaminergic neurons in multiple brain areas, including the lateral hypothalamus, that may indirectly potentiate the acquisition of the operant behavior and the discrimination of the active lever, as suggested previously (Gallistel, 1986; Berridge, 2009). In addition, it remains to be determined whether optogenetically induced dopamine release in naive animals is similar to that found in animals trained for food-seeking, and whether other nondopaminergic neuromodulators, such as opiates, may account for the development of food-seeking behaviors (Berridge, 2009). Second, in our experiments, optical stimulations were time-locked to the lever presses, which, in regard to the prediction error hypothesis (Schultz et al., 1997; Bayer and Glimcher, 2005), may have resulted in a nonoptimal dopamine encoding of reward and a weaker behavioral performance in the operant task. In addition to this temporal encoding constraint, the requirement of additional circuits for optimal operant performances may also explain why longer training procedures were required for ChR2 animals in this study, compared to the rapid acquisition of ICSS in mice (Carlezon and Chartoff, 2007). Third, we cannot rule out a possible effect due to the use of mechanical levers compared to alternative operant instruments. Indeed, it has been shown that the use of wheels or nose-poke instruments facilitates the acquisition of operant behavior in mice (Carlezon and Chartoff, 2007).

Furthermore, we tested whether activation of dopaminergic neurons alone may control reinstatement of reward-seeking behavior using a modified paradigm of drug reinstatement (Shaham et al., 2003). We found that optogenetic activation of dopaminergic neurons was sufficient to reactivate previously extinguished reward-seeking behaviors in the absence of cues, stress, or primary reinforcers such as food. This was further confirmed in the present study by using a 1 d reversal procedure. These results are consistent with previous pharmacological studies in which reinstatement for natural or drug rewards was induced by nonselective activation of dopaminergic neurons (Shaham et al., 2003). Although we cannot rule out a possible role of food in the initial shaping of the acquisition of the operant behavior, it is important to emphasize that the reactivation and reversal procedures were conducted in the absence of cue lights and food reward delivery, suggesting that the activation of dopaminergic neurons was sufficient for both the reactivation and the reversal phases of the operant behavior. Accordingly, we found a strong individual correlation between the number of active lever presses during the acquisition and the reactivation or the reversal procedure, suggesting that phasic firing of dopaminergic neurons by itself has positive reinforcement properties during the reactivation and reversal phases.

The present results confirm that phasic activation of dopaminergic neurons has positive reinforcement properties (Wise, 2004; Berridge, 2007; Schultz, 2007; Koob and Volkow, 2010) and further support the “incentive salience” hypothesis (Berridge, 2007; Flagel et al., 2011). Although, our experiments do not address a role for dopamine in the hedonic impact of self-stimulation behaviors, they suggest that, in the paradigm reported here, activation of dopaminergic neurons alone may not be sufficient for initiating a “liking” or a “wanting” signal (Berridge, 2007). Recent studies have shown that dopaminergic neurons can release glutamate upon activation (Hnasko et al., 2010; Stuber et al., 2010; Tecuapetla et al., 2010) and that subpopulations of dopaminergic neurons may process nonrewarding or aversive cues (Ungless et al., 2004; Brischoux et al., 2009; Matsumoto and Hikosaka, 2009), suggesting complex reward-encoding modes (Bromberg-Martin et al., 2010). Future investigations should focus on the precise function of possible dopaminergic neuron subpopulations within the VTA (Margolis et al., 2006) that may mediate positive reinforcement in an active operant behavior. Such approaches will undoubtedly extend our understanding of dopaminergic modulation of motivation, addiction, and consumption-related disorders.

Footnotes

This work was supported by the F.R.S.-FNRS (Chargé de Recherche), NIH Award K99, and the Fondation Leon Fredericq (A.A.); the National Alliance for Research on Schizophrenia and Depression (NARSAD) (A.A., L.d.L.); a Stanford Graduate Fellowship (H.-C.T.); NIH and National Research Service Award grants (F.Z., G.D.S.); the NSF, the National Institute of Mental Health, the National Institute on Drug Abuse, and the McKnight, Coulter, Snyder, Albert Yu and Mary Bechmann, and Keck foundations (K.D.); and National Institute of Mental Health Grants MH83702 and MH87592 and National Institute of Drug Abuse Grant DA21880 (L.d.L). We thank the de Lecea and Deisseroth labs for their support.

The authors declare no no competing financial interests.

References

  1. Bayer HM, Glimcher PW. Midbrain dopamine neurons encode a quantitative reward prediction error signal. Neuron. 2005;47:129–141. doi: 10.1016/j.neuron.2005.05.020. [DOI] [PMC free article] [PubMed] [Google Scholar]
  2. Berridge KC. The debate over dopamine's role in reward: the case for incentive salience. Psychopharmacology (Berl) 2007;191:391–431. doi: 10.1007/s00213-006-0578-x. [DOI] [PubMed] [Google Scholar]
  3. Berridge KC. ”Liking” and “wanting” food rewards: brain substrates and roles in eating disorders. Physiol Behav. 2009;97:537–550. doi: 10.1016/j.physbeh.2009.02.044. [DOI] [PMC free article] [PubMed] [Google Scholar]
  4. Bielajew C, Shizgal P. Evidence implicating descending fibers in self-stimulation of the medial forebrain bundle. J Neurosci. 1986;6:919–929. doi: 10.1523/JNEUROSCI.06-04-00919.1986. [DOI] [PMC free article] [PubMed] [Google Scholar]
  5. Blaha CD, Phillips AG. Application of in vivo electrochemistry to the measurement of changes in dopamine release during intracranial self-stimulation. J Neurosci Methods. 1990;34:125–133. doi: 10.1016/0165-0270(90)90050-p. [DOI] [PubMed] [Google Scholar]
  6. Blaha CD, Allen LF, Das S, Inglis WL, Latimer MP, Vincent SR, Winn P. Modulation of dopamine efflux in the nucleus accumbens after cholinergic stimulation of the ventral tegmental area in intact, pedunculopontine tegmental nucleus-lesioned, and laterodorsal tegmental nucleus-lesioned rats. J Neurosci. 1996;16:714–722. doi: 10.1523/JNEUROSCI.16-02-00714.1996. [DOI] [PMC free article] [PubMed] [Google Scholar]
  7. Boyden ES, Zhang F, Bamberg E, Nagel G, Deisseroth K. Millisecond-timescale, genetically targeted optical control of neural activity. Nat Neurosci. 2005;8:1263–1268. doi: 10.1038/nn1525. [DOI] [PubMed] [Google Scholar]
  8. Brischoux F, Chakraborty S, Brierley DI, Ungless MA. Phasic excitation of dopamine neurons in ventral VTA by noxious stimuli. Proc Natl Acad Sci U S A. 2009;106:4894–4899. doi: 10.1073/pnas.0811507106. [DOI] [PMC free article] [PubMed] [Google Scholar]
  9. Bromberg-Martin ES, Matsumoto M, Hikosaka O. Dopamine in motivational control: rewarding, aversive, and alerting. Neuron. 2010;68:815–834. doi: 10.1016/j.neuron.2010.11.022. [DOI] [PMC free article] [PubMed] [Google Scholar]
  10. Cannon CM, Palmiter RD. Reward without dopamine. J Neurosci. 2003;23:10827–10831. doi: 10.1523/JNEUROSCI.23-34-10827.2003. [DOI] [PMC free article] [PubMed] [Google Scholar]
  11. Carlezon WA, Jr, Chartoff EH. Intracranial self-stimulation (ICSS) in rodents to study the neurobiology of motivation. Nat Protoc. 2007;2:2987–2995. doi: 10.1038/nprot.2007.441. [DOI] [PubMed] [Google Scholar]
  12. Cheer JF, Aragona BJ, Heien ML, Seipel AT, Carelli RM, Wightman RM. Coordinated accumbal dopamine release and neural activity drive goal-directed behavior. Neuron. 2007;54:237–244. doi: 10.1016/j.neuron.2007.03.021. [DOI] [PubMed] [Google Scholar]
  13. Day JJ, Roitman MF, Wightman RM, Carelli RM. Associative learning mediates dynamic shifts in dopamine signaling in the nucleus accumbens. Nat Neurosci. 2007;10:1020–1028. doi: 10.1038/nn1923. [DOI] [PubMed] [Google Scholar]
  14. Fibiger HC, LePiane FG, Jakubovic A, Phillips AG. The role of dopamine in intracranial self-stimulation of the ventral tegmental area. J Neurosci. 1987;7:3888–3896. doi: 10.1523/JNEUROSCI.07-12-03888.1987. [DOI] [PMC free article] [PubMed] [Google Scholar]
  15. Fiorillo CD, Tobler PN, Schultz W. Discrete coding of reward probability and uncertainty by dopamine neurons. Science. 2003;299:1898–1902. doi: 10.1126/science.1077349. [DOI] [PubMed] [Google Scholar]
  16. Fiorino DF, Coury A, Fibiger HC, Phillips AG. Electrical stimulation of reward sites in the ventral tegmental area increases dopamine transmission in the nucleus accumbens of the rat. Behav Brain Res. 1993;55:131–141. doi: 10.1016/0166-4328(93)90109-4. [DOI] [PubMed] [Google Scholar]
  17. Flagel SB, Clark JJ, Robinson TE, Mayo L, Czuj A, Willuhn I, Akers CA, Clinton SM, Phillips PE, Akil H. A selective role for dopamine in stimulus-reward learning. Nature. 2011;469:53–57. doi: 10.1038/nature09588. [DOI] [PMC free article] [PubMed] [Google Scholar]
  18. Fouriezos G, Wise RA. Pimozide-induced extinction of intracranial self-stimulation: response patterns rule out motor or performance deficits. Brain Res. 1976;103:377–380. doi: 10.1016/0006-8993(76)90809-x. [DOI] [PubMed] [Google Scholar]
  19. Franklin KB. Catecholamines and self-stimulation: reward and performances effects dissociated. Pharm Biochem Behav. 1978;9:813–820. doi: 10.1016/0091-3057(78)90361-1. [DOI] [PubMed] [Google Scholar]
  20. Gallistel CR. The role of the dopaminergic projections in MFB self-stimulation. Behav Brain Res. 1986;22:97–105. doi: 10.1016/0166-4328(86)90031-8. [DOI] [PubMed] [Google Scholar]
  21. Gallistel CR, Karras D. Pimozide and amphetamine have opposing effects on the reward summation function. Pharm Biochem Behav. 1984;20:73–77. doi: 10.1016/0091-3057(84)90104-7. [DOI] [PubMed] [Google Scholar]
  22. Gallistel CR, Shizgal P, Yeomans JS. A portrait of the substrate for self-stimulation. Psychol Rev. 1981;88:228–273. [PubMed] [Google Scholar]
  23. Geisler S, Derst C, Veh RW, Zahm DS. Glutamatergic afferents of the ventral tegmental area in the rat. J Neurosci. 2007;27:5730–5743. doi: 10.1523/JNEUROSCI.0012-07.2007. [DOI] [PMC free article] [PubMed] [Google Scholar]
  24. Gratton A, Hoffer BJ, Gerhardt GA. Effects of electrical stimulation of brain reward sites on release of dopamine in rat: an in vivo electrochemical study. Brain Res Bull. 1988;21:319–324. doi: 10.1016/0361-9230(88)90247-x. [DOI] [PubMed] [Google Scholar]
  25. Hnasko TS, Chuhma N, Zhang H, Goh GY, Sulzer D, Palmiter RD, Rayport S, Edwards RH. Vesicular glutamate transport promotes dopamine storage and glutamate corelease in vivo. Neuron. 2010;65:643–656. doi: 10.1016/j.neuron.2010.02.012. [DOI] [PMC free article] [PubMed] [Google Scholar]
  26. Koob GF, Volkow ND. Neurocircuitry of addiction. Neuropsychopharmacology. 2010;35:217–238. doi: 10.1038/npp.2009.110. [DOI] [PMC free article] [PubMed] [Google Scholar]
  27. Margolis EB, Lock H, Hjelmstad GO, Fields HL. The ventral tegmental area revisited: is there an electrophysiological marker for dopaminergic neurons? J Physiol. 2006;577:907–924. doi: 10.1113/jphysiol.2006.117069. [DOI] [PMC free article] [PubMed] [Google Scholar]
  28. Matsumoto M, Hikosaka O. Two types of dopamine neuron distinctly convey positive and negative motivational signals. Nature. 2009;459:837–841. doi: 10.1038/nature08028. [DOI] [PMC free article] [PubMed] [Google Scholar]
  29. May LJ, Wightman RM. Effects of D-2 antagonists on frequency-dependent stimulated dopamine overflow in nucleus accumbens and caudate-putamen. J Neurochem. 1989;53:898–906. doi: 10.1111/j.1471-4159.1989.tb11789.x. [DOI] [PubMed] [Google Scholar]
  30. Millar J, Stamford JA, Kruk ZL, Wightman RM. Electrochemical, pharmacological and electrophysiological evidence of rapid dopamine release and removal in the rat caudate nucleus following electrical stimulation of the median forebrain bundle. Eur J Pharmacol. 1985;109:341–348. doi: 10.1016/0014-2999(85)90394-2. [DOI] [PubMed] [Google Scholar]
  31. Nagel G, Szellas T, Huhn W, Kateriya S, Adeishvili N, Berthold P, Ollig D, Hegemann P, Bamberg E. Channelrhodopsin-2, a directly light-gated cation-selective membrane channel. Proc Natl Acad Sci U S A. 2003;100:13940–13945. doi: 10.1073/pnas.1936192100. [DOI] [PMC free article] [PubMed] [Google Scholar]
  32. Olds J, Milner P. Positive reinforcement produced by electrical stimulation of septal area and other regions of rat brain. J Comp Physiol Psychol. 1954;47:419–427. doi: 10.1037/h0058775. [DOI] [PubMed] [Google Scholar]
  33. Omelchenko N, Sesack SR. Laterodorsal tegmental projections to identified cell populations in the rat ventral tegmental area. J Comp Neurol. 2005;483:217–235. doi: 10.1002/cne.20417. [DOI] [PubMed] [Google Scholar]
  34. Parker JG, Zweifel LS, Clark JJ, Evans SB, Phillips PE, Palmiter RD. Absence of NMDA receptors in dopamine neurons attenuates dopamine release but not conditioned approach during pavlovian conditioning. Proc Natl Acad Sci U S A. 2010;107:13491–13496. doi: 10.1073/pnas.1007827107. [DOI] [PMC free article] [PubMed] [Google Scholar]
  35. Paxinos G, Franklin K. The mouse brain in stereotaxic coordinates. Ed 2. New York: Academic; 2001. [Google Scholar]
  36. Phillips AG, Blaha CD, Fibiger HC. Neurochemical correlates of brain-stimulation reward measured by ex vivo and in vivo analyses. Neurosci Biobehav Rev. 1989;13:99–104. doi: 10.1016/s0149-7634(89)80017-x. [DOI] [PubMed] [Google Scholar]
  37. Phillips PE, Stuber GD, Heien ML, Wightman RM, Carelli RM. Subsecond dopamine release promotes cocaine seeking. Nature. 2003;422:614–618. doi: 10.1038/nature01476. [DOI] [PubMed] [Google Scholar]
  38. Robinson S, Sandstrom SM, Denenberg VH, Palmiter RD. Distinguishing whether dopamine regulates liking, wanting, and/or learning about rewards. Behav Neurosci. 2005;119:5–15. doi: 10.1037/0735-7044.119.1.5. [DOI] [PubMed] [Google Scholar]
  39. Roitman MF, Stuber GD, Phillips PE, Wightman RM, Carelli RM. Dopamine operates as a subsecond modulator of food seeking. J Neurosci. 2004;24:1265–1271. doi: 10.1523/JNEUROSCI.3823-03.2004. [DOI] [PMC free article] [PubMed] [Google Scholar]
  40. Schultz W. Multiple dopamine functions at different time courses. Annu Rev Neurosci. 2007;30:259–288. doi: 10.1146/annurev.neuro.28.061604.135722. [DOI] [PubMed] [Google Scholar]
  41. Schultz W, Dayan P, Montague PR. A neural substrate of prediction and reward. Science. 1997;275:1593–1599. doi: 10.1126/science.275.5306.1593. [DOI] [PubMed] [Google Scholar]
  42. Shaham Y, Shalev U, Lu L, De Wit H, Stewart J. The reinstatement model of drug relapse: history, methodology and major findings. Psychopharmacology (Berl) 2003;168:3–20. doi: 10.1007/s00213-002-1224-x. [DOI] [PubMed] [Google Scholar]
  43. Stuber GD, Hnasko TS, Britt JP, Edwards RH, Bonci A. Dopaminergic terminals in the nucleus accumbens but not the dorsal striatum corelease glutamate. J Neurosci. 2010;30:8229–8233. doi: 10.1523/JNEUROSCI.1754-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  44. Tecuapetla F, Patel JC, Xenias H, English D, Tadros I, Shah F, Berlin J, Deisseroth K, Rice ME, Tepper JM, Koos T. Glutamatergic signaling by mesolimbic dopamine neurons in the nucleus accumbens. J Neurosci. 2010;30:7105–7110. doi: 10.1523/JNEUROSCI.0265-10.2010. [DOI] [PMC free article] [PubMed] [Google Scholar]
  45. Tobler PN, Fiorillo CD, Schultz W. Adaptive coding of reward value by dopamine neurons. Science. 2005;307:1642–1645. doi: 10.1126/science.1105370. [DOI] [PubMed] [Google Scholar]
  46. Tsai HC, Zhang F, Adamantidis A, Stuber GD, Bonci A, de Lecea L, Deisseroth K. Phasic firing in dopaminergic neurons is sufficient for behavioral conditioning. Science. 2009;324:1080–1084. doi: 10.1126/science.1168878. [DOI] [PMC free article] [PubMed] [Google Scholar]
  47. Ungless MA, Magill PJ, Bolam JP. Uniform inhibition of dopamine neurons in the ventral tegmental area by aversive stimuli. Science. 2004;303:2040–2042. doi: 10.1126/science.1093360. [DOI] [PubMed] [Google Scholar]
  48. Wise RA. Dopamine, learning and motivation. Nat Rev Neurosci. 2004;5:483–494. doi: 10.1038/nrn1406. [DOI] [PubMed] [Google Scholar]
  49. Wise RA, Bozarth MA. Brain reward circuitry: four circuit elements “wired” in apparent series. Brain Res Bull. 1984;12:203–208. doi: 10.1016/0361-9230(84)90190-4. [DOI] [PubMed] [Google Scholar]
  50. Zhang F, Gradinaru V, Adamantidis AR, Durand R, Airan RD, de Lecea L, Deisseroth K. Optogenetic interrogation of neural circuits: technology for probing mammalian brain structures. Nat Protoc. 2010;5:439–456. doi: 10.1038/nprot.2009.226. [DOI] [PMC free article] [PubMed] [Google Scholar]