Behavioral Reactivity and Addiction: The Adaptation of Behavioral Response to Reward Opportunities
These studies are also beginning to explain the neurochemical processes that are important for cessation of drug use behaviors in addicted individuals. The implications of new research on reinforcement learning and other behavioral principles that are utilized by many clinical addiction treatments 4 , 5 are far-reaching and will improve our understanding of how patients reduce substance use during clinical treatments for addiction. Understanding the mechanisms by which clinical addiction treatments promote recovery should allow us to improve the effectiveness of treatments and better facilitate behavior change.
In this review, we integrate research findings on the behavioral contexts that modify signaling in the nucleus accumbens (NAc) 4 with findings on the behavioral consequences of neuroplastic changes that occur downstream of dopamine signaling in the nucleus accumbens. 6 Integrating these findings allows us to identify one mechanism by which substance use disorder treatment may exert its effects. Specifically, we discuss how changes in availability of reinforcement in the environment alter local neuronal excitability in the nucleus accumbens and thus influence addictive behavior.
Most addiction treatments, including cognitive behavior therapies such as relapse prevention, 7 contingency management, 8 the community reinforcement approach, 9 and behavioral couples therapy for alcoholism and drug abuse, 10 encourage patients to engage in constructive and rewarding activities that lead to an enriched personal and social environment. This article describes a neurobiological process by which increasing overall availability of reinforcers, via enriching the social and personal context of an individual, may decrease habitual response behaviors for substances or other basic rewards. This neurobiological process may be one mechanism by which this element of treatment reduces drug-seeking.
Notably, development and maintenance of addiction involves coordinated processing among many brain regions, and many of these processes may be useful targets for psychological and psychiatric treatments. Here we describe one population of neurons in this greater network and discuss how alteration of processing in these cells may contribute to the effectiveness of certain medications and components of psychosocial treatments. This synthesis provides only one example of how psychosocial or medication-based treatments may produce positive changes in behavior by modifying the signaling properties of specific neuron populations within this greater network. Other neurotransmitters or signaling systems may drive similar changes in excitability of the described neuron population, and other brain regions and neuron populations are certainly involved in producing treatment effects.
Nevertheless, linking concepts from neurobiology and clinical psychology/sociology research generates novel cross-disciplinary hypotheses and facilitates understanding concepts that may be well-developed in one field for researchers in other fields. 11 – 15 We expect that the concepts discussed in this manuscript will be familiar to researchers across a variety of disciplines and acknowledge that at each level, and within each discipline, there is a rich literature describing the details of these concepts. While it is beyond the scope of a brief review to present this full literature, we do anticipate that readers from a wide range of fields will find that considering this cross-disciplinary research synthesis elicits interesting and testable questions within their own fields of expertise.
A Shared Behavioral Concept: Behavioral Reactivity
Our ability to link neurobiological, behavioral, and clinical research findings centers upon a shared concept that has been described in various terms by different fields. To allow discussion across fields, we have chosen a neutral term to refer to this concept, namely behavioral reactivity, which is defined as the intensity of an automatized/habitual behavioral response when an opportunity to obtain positive or negative reinforcement is identified. For example, behavioral reactivity describes how vigorously one will pursue available food, drink, or drugs; solicit social approval from local peers; or pull away when experiencing pain or fear. Stated simply, it is the tendency to automatically seek positive experiences or escape aversive experiences when a known opportunity to do so is present. This definition assumes that the relevance or value of the reward/reinforcement has already been established. Thus, behavioral reactivity does not refer to the conditions that establish which reinforcer becomes contingent on a response (i.e., operant conditioning); nor which conditioned stimulus becomes associated with an unconditioned stimulus (i.e., classical conditioning); nor which reinforcer out of many becomes more or less salient (i.e., deprivation/satiation or establishing operations). Behavioral reactivity simply involves the intensity of overlearned automatic behavioral responses to previously established reward/reinforcers. The behavioral reactivity concept supports and expands upon the behavioral processes known to be crucial to the development and maintenance of addiction.
Behavioral Reactivity Operationalized
Behavioral tests have been used to study behavioral reactivity in basic and clinical models. In basic science literature, the concept of behavioral reactivity has been described as “gating of behavioral responses to emotional stimuli.” 16 In these studies, a series of behavioral tests have been used to probe either the intensity of, or latency to, performing a standard behavioral response during a reward opportunity. These studies have examined response to both positive and negative reinforcers. An example of a positive reinforcement test is conditioned place-preference (for morphine or cocaine), which examines the amount of time an animal will spend in a context previously paired with a drug reward. 17 An example of a negative reinforcement test is the tail flick test, which measures the latency at which an animal moves its tail to avoid continued exposure to a noxious heat stimulus. 18
Similar tests have been conducted in human trials, although these have been limited to testing behavioral response to negative reinforcers and have thus referred to the concept as “task persistence,” “distress tolerance,” or “experiential avoidance.” 19 Examples of tests that have been used in this research include the cold-pressor test, 20 which measures the latency until an individual removes his hand to escape exposure to circulating ice water, and the breath-holding task, 21 which measures the latency until an individual breathes when attempting to hold his breath. Both animal and human studies have suggested that behavioral reactivity is not reward-specific; 15 , 16 , 21 , 22 behavioral response is regulated by the expected value of the reward, but not the specific type of reward (e.g., food versus pain relief). Therefore, a person who shows high behavioral response in one test will also show high behavioral response in other behavioral reactivity tests.
A Putative Neurobiological Substrate for Behavioral Reactivity
Although decisions about whether to express reward-seeking behaviors involve coordinated processing in a network of brain regions (e.g., prefrontal cortex, amygdala, ventral tegmental area, and hippocampus), a subpopulation of neurons in the nucleus accumbens appears to be particularly important in determining levels of behavioral reactivity. Neurons of the nucleus accumbens are well-placed to regulate reward learning and seeking; the nucleus accumbens receives information on reward opportunities and reward value (among other things) from the ventral tegmental area, amygdala, hippocampus and prefrontal cortex, and provides output to circuits that direct expression of trained behavioral sequences. 23 , 24 Nucleus accumbens neurons have been implicated in many reward-related processes other than gating behavioral response to reward opportunities, such as providing an error signal to improve predictions regarding reward value. 4 , 5 Nevertheless, studies suggest that a subpopulation of nucleus accumbens neurons have response properties consistent with a role in gating behavioral response to reward opportunities and that neuroplastic changes in these neurons modulate behavioral reactivity (described in detail below). Based upon extracellular electrophysiological recordings of rats previously trained on a delayed-reponse task, Taha and Fields 25 describe a subpopulation of medium spiny neurons in the rat nucleus accumbens with firing patterns consistent with a role in gating reward-directed behavior patterns. These neurons reduced their firing rate immediately before a rat began trained or spontaneous reward-seeking behaviors and remained inhibited until the rat completed reward-seeking behaviors. Moreover, inhibition of these neurons often occurred following presentation of a cue indicating that a reward was available. Based on the firing patterns of these neurons, Taha and Fields hypothesized that sustained inhibition of these nucleus accumbens neurons permits and maintains reward-directed behaviors but does not provide instruction on which behavior to perform. If these neurons must stop firing to allow reward-seeking behaviors to occur, then the ease with which they can be turned off (i.e., their excitability) should at least partially control behavioral reactivity.
Behavioral Reactivity and Addiction Recovery
Early data on the importance of behavioral reactivity for substance use disorder treatment outcomes suggest that low behavioral reactivity to negative reinforcers is associated with greater success during attempts to quit drug use. 19 Although the association between behavioral reactivity to positive reinforcers and addiction treatment outcomes has not been directly assessed, the associations between nucleus accumbens responses to opportunities for monetary rewards and alcohol use disorder treatment outcomes do support the relevance of the behavioral reactivity process for addiction recovery. 26 Specifically, these data indicate that behavioral reactivity to negative reinforcers and nucleus accumbens response to the availability of positive reinforcers are directly related to substance use disorder outcomes. That is, individuals who escape quickly when negative reinforcers are available, or have larger neuronal responses in the nucleus accumbens when positive reinforcers are available, are less likely to stop substance use. We discuss these studies in greater detail below.
Brown et al. 21 exposed 16 current smokers who failed to sustain any previous quit attempt for more than 24 hours (immediate relapsers) and 16 current smokers with at least one prior quit success of 3 months duration (delayed relapsers) to psychological (mental arithmetic) and physical (CO 2 inhalation and breath holding) stressors. Immediate relapsers showed shorter durations on a breath holding task, greater likelihood of terminating a mental challenge task (mental arithmetic), and greater likelihood of terminating a physical challenge task (CO 2 inhalation). These results suggest that subjects who immediately relapse during smoking cessation attempts show greater behavioral reactivity to stressful stimuli (i.e., greater tendency to escape).
These initial results were replicated in a second study. 19 In this study, 77 smokers were followed for 28 days as they made an unaided quit attempt. Fifty-seven of these subjects (74%) had a lapse to smoking. It was recognized that lower behavioral reactivity to stressful stimuli, as demonstrated in this study through longer latency to terminating a mathematical challenge task, longer latency to terminating administration of carbon dioxide (CO 2 ), and longer duration of breath holding, was associated with a relatively lower risk of smoking lapse. Persistence times were standardized within each task and then summed to provide a composite measure of behavioral reactivity (these measures were all positively correlated). A decrease in behavioral reactivity to distress of one SD was associated with reduction of risk of relapse of 44%, a highly significant decrease. Results indicate that laboratory measures of general behavioral reactivity to available rewards in humans are predictive of an ability to abstain from use of an addictive substance.
While the relationship between behavioral reactivity to positive reinforcers and substance use behaviors has not been specifically addressed with behavioral tests, other data speak to this relationship. For example, a recent functional magnetic resonance imaging (fMRI) study 26 examined the relationship between neuronal activity in the nucleus accumbens in response to the availability of positive rewards (in the form of monetary incentives) and alcohol use outcomes (using the Monetary Incentive Delay task). 27 – 29 Previous studies have identified that opportunities to obtain larger rewards induce both larger nucleus accumbens responses, as measured by fMRI, and more vigorous behavioral responses to attain the rewards. 30 Thus, the magnitude of nucleus accumbens response to availability of a reward as measured by fMRI is a reasonable correlate of current behavioral reactivity to available rewards.
In the Fong et al. 26 study, 15 alcoholic patients underwent fMRI while performing the Monetary Incentive Delay task after 3 weeks of abstinence in an inpatient alcohol treatment program. Three months after treatment, 8 of the patients (53%) had relapsed. Alcoholic patients who relapsed within the 3-month period showed greater increases in nucleus accumbens activation when rewards were available as compared to those who did not relapse in the same timeframe. In other words, patients with greater nucleus accumbens brain responses to reward availability were more likely to relapse to substance use in the subsequent 3 months.
Responsivity to reward availability, in this case positive incentives or positively reinforced behavior, predicts the outcome of addiction treatment attempts. Taken together, the above studies indicate that general responsivity to opportunities to obtain a positive or negative reinforcer is an important factor in efforts to abstain from substance use.
Behavioral Reactivity Modification
Behavioral reactivity modification speaks to the utility of the behavioral reactivity construct for addiction treatment and recovery. Modulation of the excitability of the neuronal population described by Taha and Fields 25 could, theoretically, modify behavioral reactivity ( Figure 1 ). Specifically, increasing the excitability of these medium spiny neurons would make it more difficult to inhibit their firing; if such inhibition is necessary to allow the expression of reward-directed behavior sequences, then overall response to positive and negative reinforcers (behavioral reactivity) would be reduced. Based on the research described below, we suggest that 1) cues that predict reward availability induce firing of midbrain dopamine neurons that terminate in the nucleus accumbens, 2) dopamine signaling in the nucleus accumbens activates transcription factors, including cAMP response element binding protein (CREB), which alter the responsivity of nucleus accumbens neurons, 3) these changes in gene expression alter the excitability of medium spiny neurons in the nucleus accumbens and reduce behavioral reactivity to future reward opportunities, and 4) CREB expression levels decay over time, ensuring that behavioral reactivity increases if opportunities for reward are encountered less frequently.
Immediately, inputs signaling the presence of a reward inhibit firing of a subpopulation of medium spiny neurons (MSNs) in the NAc to disinhibit reward seeking behavior patterns. Exposure to a reward opportunity also leads to the release of neuromodulators such as dopamine in the NAc. These neuromodulators increase the excitability of MSNs by altering gene expression via transcription factors such as CREB. More excitable MSNs are harder to inhibit, and thus when reward opportunities occur in the future, it is less likely that MSNs are inhibited sufficiently to allow reward-seeking behaviors. It is less likely that a behavioral response is initiated when a reward is available (i.e., behavioral reactivity is decreased).
The above research synthesis describes a process where reward opportunities increase dopamine release in the nucleus accumbens. This dopamine release increases activation and expression of CREB and thus CRE-mediated gene transcription in the nucleus accumbens. Increases in CREB activity in the nucleus accumbens alter the excitability of neurons in the nucleus accumbens such that available rewards produce fewer or less intense behaviors to obtain rewards. Thus, as rewards are made available in the present, this system produces general reductions in behavioral reactivity to future available rewards ( Figure 2 ). Conversely, lack of exposure to available rewards leads to a progressive increase in behavioral reactivity over time.
This figure depicts the relationship between 1) the density of reward opportunities in the environment, 2) the excitability of the subpopulation of medium spiny neurons (MSN) described by Taha and Fields, or, more proximally, the amount of dopamine signaling (DA) that occurs in response to a reward opportunity, or the amount of CREB expressed in these neurons, and 3) behavioral reactivity (BRXT). In a nondopamine deficient person, when rewards are available at moderately high levels, MSN excitability is efficiently increased and behavioral reactivity is low. Conversely, when rewards are more difficult to find, behavioral reactivity is high.
In this way, we believe that these changes in gene expression provide a critical mechanism for adaptation to current levels of reward availability in the environment. As rewards become more frequently available, or as available rewards increase in value, dopamine signaling will increase, producing increases in CREB activity and decreases in behavioral reactivity to future available rewards. Dopamine-mediated changes in gene transcription provide a dynamic adaptational feedback system in which the tendency to perform habitual behaviors to obtain reinforcers changes as a function of the perceived availability of reward opportunities in the environment. In other words, this system allows organisms to behaviorally adapt to different levels of reward availability.
Such a system makes sense from an evolutionary perspective. When resources and thus reinforcers are scarce, behavioral reactivity to reward opportunities would be high. Under these conditions one would readily seek any and all available rewards. When rewards are plentiful, behavioral reactivity to reward opportunities would decrease. Under these conditions, one would only seek available rewards on an occasional basis, preventing overconsumption and allowing one to focus on other goals and expand behavioral repertoires.
The more available reinforcers are in any given environment, the less important it becomes to respond to every opportunity. For example, if food is scarce, it is in an individual’s best interest to eat whenever food is available. But if food becomes plentiful, then it is important that an individual does not eat at every available opportunity. Individuals must reduce behavioral reactivity to food availability or risk becoming obese. Conversely, if an individual is in a dangerous area with only one safe hiding place, it is important that he or she stays close enough to that hiding place to allow escape if necessary. If there are many safe hiding places, an individual can roam freely, knowing that there will be an escape available if needed. When safe hiding places are highly available, individuals do not need to perform escape behaviors at every hint of a threat. It is then adaptive to reduce behavioral reactivity to escape opportunities.
Studies Demonstrating the Biological Plausibility of this Process for Modification of Behavioral Reactivity
Following is a review of basic science literature identifying the biological components of this process. For each step of the adaptation process, we briefly review key studies that have described these processes.
When Opportunities for Reward are Present, Dopamine is Released in the Nucleus Accumbens
Historically, dopamine release in the nucleus accumbens was thought to signal receipt of a reward, perhaps providing a pleasure signal to encourage further behavior. 31 Recent research has expanded our understanding of dopamine’s function in the nucleus accumbens and demonstrated that this hypothesis is overly simplistic.
In a series of studies, Shultz 32 studied signaling of dopamine neurons projecting to the nucleus accumbens in awake, behaving monkeys, trained to respond to cues signaling the upcoming availability of a juice reward. These studies demonstrate that dopamine neuron firing occurs upon exposure to the first cue predicting the upcoming availability of reward. If a reward has been accurately predicted, then no further dopamine neuron firing occurs upon receipt of a reward. Thus, dopamine release in the nucleus accumbens signals availability of reward in the current environment, rather than the attainment of a reward. 4 , 5 , 32 , 33 Frequency of midbrain dopamine neuron firing encodes the density of reward opportunities in the environment.
Human imaging studies support that dopamine is released when cues predicting reward availability are presented, regardless of whether the person is allowed to obtain the reward. Using positron emission tomography (PET) imaging with [carbon-11] raclopride to examine changes in dopamine D2 receptor binding, Volkow et al. 34 presented pictures of food to subjects who were pretreated with methylphenidate to amplify the effects of dopamine release on raclopride binding. Presentation of the food images induced striatal dopamine release, even though subjects were not able to consume the food reward. These studies demonstrate that cues indicating the availability of a reward activate neurons that release dopamine in the nucleus accumbens.
Dopamine Signaling Increases Activation of CREB in the Nucleus Accumbens
Activation of both D1 and D2 subtypes of the dopamine receptor leads to CREB activation in striatal neurons, linking stimuli that release dopamine to increases in nucleus accumbens CREB activation. A series of studies of dopamine receptor signaling cell biology have shown that activation of either D1 or D2 dopamine receptors leads to phosphorylation of CREB in striatal neurons, 35 – 37 although the signaling pathways leading to CREB activation differ depending on the receptor type stimulated and possibly the type of neuron in the striatum. 38 Studies in which dopamine is pharmacologically increased confirm the link between dopamine receptor activation and CREB activation. In rodent models, administration of amphetamine, which acts upon the dopamine transporter to increase synaptic dopamine levels, activates CREB and increases CRE-mediated gene expression in the nucleus accumbens, amygdala, dorsal striatum, lateral septum, dorsomedial hypothalamus, and ventral tegmental area. 35 , 39 Amphetamine-induced CRE-mediated gene expression is found in both the nucleus accumbens core and shell.
CREB is phosphorylated and activated within 30 minutes of dopamine release and activation begins to decay within 2 hours. 40 CREB activation in the nucleus accumbens leads to changes in expression of a host of other genes, 41 which alter excitability of local neurons 42 and produce the behavioral changes we describe in the next section. 43 , 44 , 45 These studies demonstrate that release of dopamine in the nucleus accumbens leads to transient activation of the transcription factor CREB in local neurons.
Increases in CREB expression in the nucleus accumbens lead to increased excitability of medium spiny neurons in the nucleus accumbens and reduced behavioral reactivity to both positive and negative reinforcers.
Nestler et al. 16 , 22 , 42 – 46 have investigated the consequences of changes in CREB expression in the nucleus accumbens. Using viral mediated gene transfer techniques, Nestler and colleagues overexpressed CREB or a dominant negative mutant version of CREB in the nucleus accumbens.
Overexpression of CREB increased the excitability of medium spiny neurons both in a nucleus accumbens slice culture model and in nucleus accumbens slices of rats microinjected with the CREB-expressing viruses 1 day previous. 42 Increasing CREB expression in medium spiny neurons increased the number of spikes elicited by depolarizing current injections and reduced action potential threshold; in short, increased CREB expression made medium spiny neurons fire more for a given amount of input.
Behaviorally, rats who overexpressed CREB in the nucleus accumbens showed reduced responsiveness to both positive and negative reinforcers. Specifically, they showed less place-preference to morphine and cocaine, consumed less sugar solution, pulled away more slowly from a noxious heat stimulus, and spent more time in the open arm of an elevated plus maze. 16 , 44 , 45 This suggests that they were both less likely to seek drug or food rewards (positive reinforcers) and less likely to avoid noxious or anxiety producing stimuli (negative reinforcers). Moreover, animals that expressed the dominant negative mutant form of CREB (which binds CRE sites but does not trigger gene transcription, and thus acts as a competitive antagonist) or expressed Inducible cAMP Early Repressor (ICER), an endogenous repressor of CRE-mediated transcription, showed more behavioral response to positive and negative reinforcers (including opposite response in the above mentioned tests, plus evidence of increased grooming of peers in social interaction tests and increased avoidance of novel tastes, a measure of anxiety). 16 , 22 , 44 – 46 This demonstrates that reductions in CRE-mediated transcription, such as would occur over time in the absence of exposure to reward opportunities, increase behavioral reactivity.
Other studies from this group elaborate on these behavioral effects. Using the same overexpression techniques, Pliakas et al. 44 and Newton et al. 43 showed that increasing CREB expression in the nucleus accumbens decreased the amount of time that rats spent making escape attempts and increased the latency to escape in the forced swim test and the inescapable electric footshock test. This reduction in escape attempts may also be conceptualized as a reduction in behavioral response to negative reinforcers, similar to the reduction in response to noxious or anxiogenic stimuli described above.
Expanding on the role of CREB in behavioral reactivity to anxiogenic stimuli, Barrot et al. 22 demonstrated that sexually naive male rats that expressed the dominant negative mutant form of CREB in their nucleus accumbens showed deficits in initiation of sexual behavior. If one focuses only on the reinforcing function of sexual behavior, this result appears discrepant from the results of the previous experiments. However, this deficit was seen only in sexually naive rats; sexually experienced male rats (including sexually naive rats after initiation of sexual behavior) with the same CREB manipulation did not show delays in initiation of sexual behaviors when placed with a receptive female. Importantly, the deficit in initiation of sexual behavior was eliminated when the rats were treated with the anxiolytic diazepam, suggesting that the deficit resulted from anxiety surrounding performance of a novel behavior. Thus, high behavioral reactivity produced avoidance of the unknown and potentially threatening interaction with a receptive female, preventing interaction with what could have become a strong positive reinforcer.
Extending this study, Barrot et al. 22 demonstrated that social isolation mimicked the effects of gene transfer of dominant negative CREB in the nucleus accumbens, reducing CREB expression in the nucleus accumbens and producing a deficit in initiation of sexual behavior in sexually naive male rats. Using gene transfer of CREB to increase CREB expression in the nucleus accumbens of socially isolated rats eliminated the deficit in initiation of sexual behavior, demonstrating that this effect of social isolation was mediated by nucleus accumbens CREB activity.
This study provides new data on the environmental regulation of nucleus accumbens CREB levels. The previous studies cited above demonstrated that a large number of behavioral paradigms increase CREB in the nucleus accumbens. Specifically, injection of drugs of abuse, forced-swim stress, foot-shock exposure, restraint stress, social stress, or unpredictable stress increase CREB activity in the nucleus accumbens 16 and thus would be expected to decrease behavioral reactivity. This study shows that social isolation decreases CREB activity in the nucleus accumbens (and thus would be expected to increase behavioral reactivity). As all of these behavioral paradigms are thought to be stressful, what is the difference between social isolation and those behavioral paradigms that increase, rather than decrease, CREB?
A fundamental difference in these experimental preparations is that they differ in exposure to reward opportunities. The behavioral tests that increase CREB also increase opportunities to either perform a seeking behavior to attain a positive reinforcer (e.g., learn how to get injected with drugs) or to perform an escape behavior to reduce an aversive stimulus, attaining a negative reinforcer (e.g., learn how to avoid foot shock, avoid being harassed by peers, or avoid drowning). Social isolation decreases reward opportunities by preventing interactions that could lead to improved social status, grooming, or other social reinforcers. Thus, this study is consistent with the idea that CREB expression in the nucleus accumbens is modified by the density of reward opportunities in the environment. Animals with little exposure to reward opportunities and low CREB respond rapidly and vigorously to attain positive or negative reinforcers—they show high behavioral reactivity to available reward. Those with greater exposure to reward opportunities and high CREB respond less rapidly or only to greater reward opportunities—they show low behavioral reactivity to reward opportunities. This adaptation to available rewards holds great promise for our understanding of clinical interventions and recovery.
Current Clinical Interventions and Behavioral Reactivity
In this adaptational process, reward availability leads to increases in dopamine signaling that modulate behavioral reactivity. This research synthesis suggests two ways to decrease behavioral reactivity. First, one could increase the amount of adaptation that occurs at each presentation of an available reward. This could theoretically be accomplished either by increasing the perceived value of environmentally available rewards 5 , 47 – 49 or pharmacologically increasing the amount of dopamine signaling or CREB expression that occurs when a reward cue is perceived. Behavioral treatments such as contingency management offer an example of the former; bupropion provides an example of the latter therapy. Both should increase the amount of dopamine signaling per stimulus. Contingency management increases the value of rewards immediately available for sustaining abstinence; 8 bupropion blocks the dopamine transporter. 50 – 52 Based upon the process described in this paper, this increased dopamine will increase the amount the individual adapts with each reward opportunity.
Second, one can increase the perceived density of available rewards and thus the frequency of dopamine signaling . This could be achieved by increasing contact with potential reinforcers using behavioral treatment. Heightening awareness of existing opportunities for reinforcement, or teaching a person to engage in behaviors resulting in contact with additional reinforcers (e.g., reinforcers that were previously unavailable due to skill deficits), could increase reward density. 53
Research on behavioral reactivity modification also predicts that combining interventions which increase the frequency and magnitude of available rewards or dopamine signaling should reduce behavioral reactivity to a greater extent. For example, combining buproprion and treatment to increase the availability of alternative reinforcers should reduce behavioral reactivity more effectively. Combining behavioral interventions that increase the value and the frequency of perceived rewards should have similar effects.
Basic and Clinic Research Supporting This Treatment Process
Animal studies indicate that pharmacological manipulations to increase the amount of dopamine released per stimulus decrease self-administration of drugs of abuse. Although it is not a selective inhibitor of the dopamine transporter, bupropion inhibits the reuptake of dopamine and thus should increase the amount and duration of dopamine present in the synapse following dopamine release. 50 – 52 Several studies examined the effects of bupropion on self-administration of nicotine and food in rats. 54 , 55 Bupropion reduced nicotine self-administration by about 50% and food consumption by about 15% under a fixed ratio schedule but did not alter the break-point for nicotine self-administration and actually increased the break point for food under a progressive ratio schedule. 54 This suggests that buproprion decreases the frequency at which rats will behaviorally react to consume available nicotine without changing the estimated value or amount of work worth doing to gain a single nicotine infusion. In other words, bupropion decreased behavioral reactivity to available nicotine and food rewards without changing the perception of value of a nicotine reward and even increasing the value of a food reward.
In a second study using a fixed ratio self-administration paradigm, high dose bupropion was found to decrease self-administration of nicotine, sucrose, and amphetamine. 55 Moreover, high dose methamphetamine, which releases dopamine via actions on the dopamine transporter, 56 and apomorphine, which directly activates D1 and D2 dopamine receptors, 57 , 58 tended to decrease self-administration of nicotine, suggesting that activation of dopamine receptor signaling pathways using a variety of pharmacological tools can reduce self-administration of an addictive substance. Thus, pharmacologically increasing dopamine signaling in the nucleus accumbens leads to reductions in behavioral reactivity and substance or food consumption.
Animal studies have also demonstrated that altering environmental availability of reward opportunities can alter drug self-administration. In a series of experiments, Bardo et al. 59 investigated the effects of environmental enrichment on amphetamine self-administration in rats. Environmental enrichment consisted of increasing reward density by adding toys or peers to the cage. Rats raised in an enriched versus an isolated environment did not maintain their response in a fixed-ratio paradigm. Whereas rats reared in isolated conditions continued to self-administer low-dose amphetamine, rats reared in an enriched environment decreased self-administration over time. A second study confirmed this effect and demonstrated a “dose”-dependent decrease in amphetamine self-administration associated with the extent of environment enrichment. 60 Rats reared in isolation self-administered more amphetamine on a fixed-ratio schedule than did socially reared rats. Rats reared socially with novel objects added to the cage showed less amphetamine self-administration than all other groups.
Several other studies further examine the effects of altering reward availability on drug self-administration and, more proximally, CREB expression in the nucleus accumbens. Morgan et al. 61 demonstrated that in groups of socially housed cynomolgus macaques, dominant macaques were more likely to be offered a social reward (e.g., were more likely to be groomed by peers) and self-administered less cocaine than submissive macaques. As discussed above, Barrot et al. 16 , 22 found that social housing increases and social isolation decreases expression of CREB in the nucleus accumbens of rats. Moreover, social isolation increased avoidance behaviors to anxiogenic stimuli that were reversed by increasing CREB expression in the nucleus accumbens. Thus, a decrease in reward opportunities in the environment is associated with increased behavioral reactivity to negative reinforcers, an effect that is mediated by decreased expression of CREB in the nucleus accumbens. Together these studies suggest that environmentally altering reward availability or pharmacologically altering dopamine release in the nucleus accumbens leads to reductions in behavioral reactivity to drug and other reinforcers.
Clinical trials provide mixed support for the effects of pharmacologically increasing nucleus accumbens dopamine on consumption of substances in humans. Bupropion has been shown to assist with smoking cessation in multiple clinical trials. Across three randomized controlled trials of bupropion for smoking cessation, point prevalence abstinence rates at 12 months ranged from 23%–25% for bupropion-treated smokers and 12%–16% for placebo-treated smokers. 62 A Cochrane Library Systematic Review found that across 19 placebo-controlled trials of bupropion for smoking cessation, the estimated odds ratio for abstinence from smoking after at least 6 months follow up was 2.06 (95% confidence interval=1.77–2.40), demonstrating that bupropion treatment nearly doubles the chance of smoking cessation success. 63 The clinical efficacy of bupropion for the treatment of other substance use disorders has been less thoroughly investigated and results are inconclusive. 64
Other medications that increase nucleus accumbens dopamine signaling have been less well-characterized for the treatment of substance use disorders. Nevertheless, a few studies suggest that the D2 dopamine receptor agonist bromocriptine can reduce smoking rates over a 5-hr period, 65 and use of this medication for infertility problems was retrospectively associated with increased smoking cessation rates in pregnant women. 66 Trials of dopaminergic agents for the treatment of cocaine dependence have not found them to be effective, 67 raising the possibility that the therapeutic benefit of targeting this system may vary depending on the patients’ drug of choice. More investigation of the effects of dopaminergic agents on substance use in humans is needed to clarify these findings.
Lastly, psychosocial treatments for addiction may increase perceived reward availability. Increasing contact with rewards linked to adaptive behavior is a central tenet of behavior therapy. Virtually every empirically supported treatment for addictive disorders attempts to increase patients’ engagement in rewarding nondrug behaviors, including contingency management, 8 12 step facilitation, 68 cognitive behavior therapy, 7 motivational enhancement therapy 69 and acceptance-based therapies. 70 In line with this effort, all of the above treatments place patients in a more densely, and frequently more intensely, socially reinforcing environment (treatment settings, self-help groups) and encourage patients to increase contact with novel or previously unaccessed reinforcers by engaging in healthy alternatives to using, such as exercising, working, or engaging in other social and recreational activities. 71 , 72 By teaching new emotional and behavioral skills and enriching the social setting, these therapies increase the number of opportunities available to receive reinforcement. 72 , 73
Like bupropion, psychosocial therapies for addiction reduce substance use. For example, a Cochrane Library Systematic Review of 55 trials examining the effectiveness of group therapy for smoking cessation found that the estimated odds ratio for abstinence from smoking after at least 6 months follow up was 2.17 (95% CI 1.37–3.45) when compared to no intervention controls and 2.04 (95% CI 1.60–2.60) when compared to a self-help control. Thus, group psychosocial therapies roughly double the chance of smoking cessation success. 74 These clinical trials support the findings of animal studies, showing that manipulating reward availability or dopamine signaling leads to predictable reductions in drug use.
Individual Differences in the Ability to Modify Behavioral Reactivity: Contribution to Addiction Vulnerability?
Our research synthesis predicts that reduced dopamine responsivity to opportunities for reward would limit the organism’s ability to adapt to reward availability. If less dopamine is released or fewer dopamine receptors are activated each time an organism is presented with a reward opportunity, less CREB is activated in the nucleus accumbens and the organism is less likely to reduce its tendency to respond to future reward opportunities. Because dopamine signaling drives changes in gene transcription that lead to decreases in behavioral reactivity in response to increases in reward availability, 16 , 22 , 43 – 46 persons who have insufficient dopamine signaling in response to available rewards should show less adaptation as rewards become more available (i.e., less reduction in behavioral reactivity). Thus, a “dopamine deficiency” 75 , 76 results in overseeking of rewards in environments with higher reward density. When resources are plentiful, individuals with high dopamine responsivity reduce their consumption such that they seek rewards less frequently (i.e., reduce behavioral reactivity). However, those with low dopamine responsivity underadapt and are therefore prone to overconsumption and overavoidance.
This prediction is consistent with recent imaging and genetic research that is the basis for the “dopamine deficiency” theory of addiction vulnerability. 75 , 76 PET imaging studies have shown that persons with substance use disorders have fewer dopamine receptors or release less dopamine in response to both drugs of abuse and natural reinforcers. 77 Individuals with substance use disorders are also more likely to carry a polymorphism in the dopamine D2 receptor gene that results in lower dopamine D2 receptor expression in the striatum. 78 Because this research has been thoroughly reviewed elsewhere, we have refrained from reviewing these studies here. However, this research would suggest that persons with “dopamine deficiency” would require greater exposure to reward opportunities or ongoing pharmacological enhancement of dopamine signaling to maintain a given level of behavioral reactivity ( Figure 3 ). Clinically, such patients might require ongoing contact with services that supplement reward opportunities in the general environment (e.g., additional social support from mutual help meetings or a contingency management plan to enhance rewards for desired behaviors) to prevent problems with overconsumption or overavoidance in other contexts.
In a dopamine deficient person, the same moderately high exposure to reward opportunities produces less dopamine signaling, less CREB expression and thus relatively lower MSN excitability. Thus, behavioral reactivity remains at intermediate levels. In order to reduce behavioral reactivity to low levels in a dopamine deficient person, reward opportunities must be provided at extremely high levels.
CONCLUSIONS
Changes in reward availability in the environment produce changes in patterns of neuronal firing in the nucleus accumbens that alter the excitability of these neuronal circuits. This modulation of neuronal excitability has important implications for habitual behavioral responding. In summary, we suggest that modulation of general behavioral reactivity to known reinforcers is an adaptational process that may play an important role in addiction recovery. Exposure to available rewards decreases behavioral reactivity via changes in dopamine signaling and CREB expression in the nucleus accumbens. Behavioral reactivity may be clinically modified by altering the prevalence or value of available rewards perceived in the environment or by manipulating dopamine signaling. Additionally, individual variability in dopamine signaling in response to available rewards determines the rate at which behavioral reactivity adapts in response to changing environments. Insufficient adaptation may increase risk of diseases of overconsumption or overavoidance.
1 . Kreek MJ, Nielsen DA, Butelman ER, et al: Genetic influences on impulsivity, risk taking, stress responsivity and vulnerability to drug abuse and addiction. Nat Neurosci 2005; 8:1450–1457Google Scholar
2 . Weiss F: Neurobiology of craving, conditioned reward and relapse. Curr Opin Pharmacol 2005; 5:9–19Google Scholar
3 . Koob GF, Ahmed SH, Boutrel B, et al: Neurobiological mechanisms in the transition from drug use to drug dependence. Neurosci Biobehav Rev 2004; 27:739–749Google Scholar
4 . Schultz W: Behavioral theories and the neurophysiology of reward. Annu Rev Psychol 2006; 57:87–115Google Scholar
5 . Tobler PN, Fiorillo CD, Schultz W: Adaptive coding of reward value by dopamine neurons. Science 2005; 307:1642–1645Google Scholar
6 . Hyman SE, Malenka RC, Nestler EJ: Neural mechanisms of addiction: the role of reward-related learning and memory. Annu Rev Neurosci 2006; 29:565–598Google Scholar
7 . Marlatt GA, Gordon JR: Relapse Prevention: Maintenance Strategies in the Treatment of Addictive Behaviors. New York, Guilford, 1985Google Scholar
8 . Higgins ST, Petry NM: Contingency management: incentives for sobriety. Alcohol Res Health 1999; 23:122–127Google Scholar
9 . Meyers RJ, Miller WR: A Community Reinforcement Approach to Addiction Treatment. Cambridge, MA, Cambridge University Press, 2001Google Scholar
10 . O’Farrell T, Fals-Stewart W: Behavioral couples therapy for alcoholism and drug abuse. J Substance Abuse Treatment 2000; 18:51–54Google Scholar
11 . Rosenfield PL: The potential of transdisiciplinary research for sustaining and extending linkages between the health and social sciences. Soc Sci Med 1992; 35:1343–1357Google Scholar
12 . Abrams DB: Transdisciplinary paradigms for tobacco prevention research. Nicotine Tob Res 1999; 1:S15–23Google Scholar
13 . Abrams DB, Leslie F, Mermelstein R, et al: Transdisciplinary tobacco use research. Nicotine Tob Res 2003; 5(S1):S5–10Google Scholar
14 . Tiffany ST, Conklin CA, Shiffman S, et al: What can dependence theories tell us about assessing the emergence of tobacco dependence? Addiction 2004; 99(S1):78–86Google Scholar
15 . West R: Theory of Addiction. Oxford, UK, Blackwell Scientific Publishing, 2006Google Scholar
16 . Barrot M, Olivier JD, Perrotti LI, et al: CREB activity in the nucleus accumbens shell controls gating of behavioral responses to emotional stimuli. Proc Natl Acad Sci USA 2002; 99:11435–11440Google Scholar
17 . Sanchis-Seguar C, Spanagel R: Behavioural assessment of drug reinforcement and addictive features in rodents: an overview. Addict Biol 2006; 11:2–38Google Scholar
18 . Hole K, Tjolsen A: The tail-flick and formalin tests in rodents: changes in skin temperature as a confounding factor. Pain 1993; 53:247–254Google Scholar
19 . Brown RA, Lejuez CW, Kahler CW: Distress tolerance and early smoking lapse. Clin Psychol Rev 2005; 25:713–733Google Scholar
20 . Hayes SC, Bissett RT, Korn Z, et al: The impact of acceptance versus control rationales on pain tolerance. Psychol Rec 1999; 49:33–47Google Scholar
21 . Brown RA, Lejuez CW, Kahler CW, et al: Distress tolerance and duration of past smoking cessation attempts. J Abnorm Psychol 2002; 111:180–185Google Scholar
22 . Barrot M, Wallace DL, Bolanos CA, et al: Regulation of anxiety and initiation of sexual behavior by CREB in the nucleus accumbens. Proc Natl Acad Sci USA 2005; 102:8357–8362Google Scholar
23 . Nicola SM, Yun IA, Wakabayshi KT, et al: Cue-evoked firing of nucleus accumbens neurons encodes motivational significance during a discriminative stimulus task. J Neurophysiol 2004; 91:1840–1865Google Scholar
24 . Mogenson GJ, Jones DL, Yim CY: From motivation to action: functional interface between the limbic system and the motor system. Prog Neurobiol 1980; 14:69–97Google Scholar
25 . Taha SA, Fields HL: Inhibitions of nucleus accumbens neurons encode a gating-signal for reward directed behavior. J Neurosci 2006; 26:217–222Google Scholar
26 . Fong GW, Knutson B, Risinger RC: Hyperactivation of reward-related mesolimbic areas in alcoholics who subsequently relapse. Human Brain Mapping Conference, Toronto, Canada, June 12–16, 2005Google Scholar
27 . Knutson B, Westdorp A, Kaiser E: fMRI visualization of brain activity during a monetary incentive delay task. Neuroimage 2000; 12:20–27Google Scholar
28 . Knutson B, Adams CS, Fong GW: Anticipation of monetary reward selectively recruits nucleus accumbens. J Neurosci 2001; 21:RC159Google Scholar
29 . Knutson B, Fong GW, Adams CS: Dissociation of reward anticipation versus outcome with event-related fMRI. Neuroreport 2001; 12:3683–3687Google Scholar
30 . Knutson B, Taylor J, Kaufman M: Distributed neural representation of expected value. J Neurosci 2005; 25:4806–4812Google Scholar
31 . Bressan RA, Crippa JA: The role of dopamine in reward and pleasure behaviour: review of data from preclinical research. Acta Psychiatr Scand 2005; 427(Suppl):14–21Google Scholar
32 . Schultz W: Predictive reward signal of dopamine neurons. J Neurophysiol 1998; 80:1–27Google Scholar
33 . Waelti P, Dickenson A, Schultz W: Dopamine responses comply with basic assumptions of formal learning theory. Nature 2001; 412:43–48Google Scholar
34 . Volkow ND, Wang GJ, Fowler JS, et al: “Nonhedonic” food motivation in humans involves dopamine in the dorsal striatum and methylphenidate amplifies this effect. Synapse 2002; 44:175–180Google Scholar
35 . Konradi C, Cole RL, Heckers S, et al: Amphetamine regulates gene expression in rat striatum via transcription factor CREB. J Neurosci 1994; 14:5623–5634Google Scholar
36 . Dudman JT, Eaton ME, Rajadhyaksha A, et al: Dopamine D1 receptors mediate CREB phosphorylation via phosphorylation of the NMDA receptor at Ser897-NR1. J Neurochem 2003; 87:922–934Google Scholar
37 . Yan Z, Feng J, Fienberg AA, et al: D2 dopamine receptors induce mitogen-activated protein kinase and camp response element-binding protein phosphorylation in neurons. Proc Natl Acad Sci USA 1999; 96:11607–11612Google Scholar
38 . Brami-Cherrier K, Valjent E, Garcia M, et al: Dopamine induces a PI3-kinase-independent activation of AKT in striatal neurons: a new route to camp response element-binding protein phosphorylation. J Neurosci 2002; 22:8911–8921Google Scholar
39 . Shaw-Lutchman TZ, Impey S, Storm D, et al: Regulation of CRE-mediated transcription in mouse brain by amphetamine. Synapse 2003; 48:10–17Google Scholar
40 . Jang CG, Lee SY, Lee HK, et al: Time courses of pCREB expression after dopaminergic stimulation by apomorphine in mouse brain. Arch Pharm Res 2002; 25:370–374Google Scholar
41 . McClung CA, Nestler EJ: Regulation of gene expression and cocaine reward by CREB and deltafosb. Nat Neurosci 2003; 6:1208–1215Google Scholar
42 . Dong Y, Green T, Saal D, et al: CREB modulates excitability of nucleus accumbens neurons. Nat Neurosci 2006; 9:475–477Google Scholar
43 . Newton SS, Thome J, Wallace TL, et al: Inhibition of camp response element-binding protein or dynorphin in the nucleus accumbens produces an antidepressant-like effect. J Neurosci 2002; 22:10883–10890Google Scholar
44 . Pliakas AM, Carlson RR, Neve RL, et al: Altered responsiveness to cocaine and increased immobility in the forced swim test associated with elevated camp response element-binding protein expression in nucleus accumbens. J Neurosci 2001; 21:7397–7403Google Scholar
45 . Carlezon WA Jr, Thome J, Olson VG, et al: Regulation of cocaine reward by CREB. Science 1998; 282:2272–2275Google Scholar
46 . Green TA, Alibhai IN, Hommel JD, et al: Induction of inducible camp early repressor expression in nucleus accumbens by stress or amphetamine increases behavioral responses to emotional stimuli. J Neurosci 2006; 26:8235–8242Google Scholar
47 . McClure SM, Daw ND, Montague PR: A computational substrate for incentive salience. Trends Neurosci 2003; 26:423–428Google Scholar
48 . Morris G, Nevet A, Arkadir D, et al: Midbrain dopamine neurons encode decisions for future action. Nat Neurosci 2006; 9:1057–1063Google Scholar
49 . Pessiglione M, Seymour B, Flandin G, et al: Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 2006; 442:1042–1045Google Scholar
50 . Argyelan M, Szabo Z, Kanyo B, et al: Dopamine transporter availability in medication free and in bupropion treated depression: a 99mTc-TRODAT-1 SPECT study. J Affect Disord 2005; 89:115–123Google Scholar
51 . Stahl SM, Pradko JF, Haight BR, et al: A review of the neuropharmacology of bupropion, a dual norepinephrine and dopamine reuptake inhibitor. Prim Care Companion J Clin Psychiatry 2004; 6:159–166Google Scholar
52 . Learned-Coughlin SM, Bergstrom M, Savitcheva I, et al: In vivo activity of bupropion at the human dopamine transporter as measured by positron emission tomography. Biol Psychiatry 2003; 54:800–805Google Scholar
53 . Van Etten ML, Higgins ST, Budney AJ, et al: Comparison of the frequency and enjoyability of pleasant events in cocaine abusers vs. non-abusers using a standardized behavioral inventory. Addiction 1998; 93:1669–1680Google Scholar
54 . Bruijnzeel AW, Markou A: Characterization of the effects of bupropion on the reinforcing properties of nicotine and food in rats. Synapse 2003; 50:20–28Google Scholar
55 . Rauhut AS, Neugebauer N, Dwoskin LP, et al: Effect of bupropion on nicotine self-administration in rats. Psychopharmacology 2003; 169:1–9Google Scholar
56 . Eshleman AJ, Henningsen RA, Neve KA, et al: Release of dopamine via the human transporter. Mol Pharmacol 1994; 45:312–316Google Scholar
57 . Anden NE, Rubenson A, Fuxe K, et al: Evidence for dopamine receptor stimulation by apomorphine. J Pharm Pharmacol 1967; 19:627–629Google Scholar
58 . Colpaert FC, Van Bever WF, Leysen JE: Apomorphine: chemistry, pharmacology, biochemistry. Int Rev Neurobiol 1976; 19:225–268Google Scholar
59 . Bardo MT, Klebaur JE, Valone JM, et al: Environmental enrichment decreases intravenous self-administration of amphetamine in female and male rats. Psychopharmacol 2001; 155:278–284Google Scholar
60 . Green TA, Gehrke BJ, Bardo MT: Environmental enrichment decreases intravenous amphetamine self-administration in rats: dose-response functions for fixed- and progressive-ratio schedules. Psychopharmacology 2002; 162:373–378Google Scholar
61 . Morgan D, Grant KA, Gage HD, et al: Social dominance in monkeys: dopamine D2 receptors and cocaine self-administration. Nat Neurosci 2002; 5:169–174Google Scholar
62 . Richmond R, Zwar N: Review of bupropion for smoking cessation. Drug Alcohol Rev 2003; 22:203–220Google Scholar
63 . Hughes J, Stead L, Lancaster T: Antidepressants for smoking cessation. Cochrane Database Syst Rev 2004; 4:CD000031Google Scholar
64 . Torrens M, Fonseca F, Mateu G, et al: Efficacy of antidepressants in substance use disorders with and without comorbid depression: a systematic review and meta-analysis. Drug Alcohol Depend 2005; 78:1–22Google Scholar
65 . Caskey NH, Jarvik ME, Wirshing WC, et al: Modulating tobacco smoking rates by dopaminergic stimulation and blockade. Nicotine Tob Res 2002; 4:259–266Google Scholar
66 . Murphy MF, Hey K, Johnstone E, et al: Bromocriptine use is associated with decreased smoking rates. Addict Biol 2002; 7:325–328Google Scholar
67 . Soares BG, Lima MS, Reisser AA, et al: Dopamine agonists for cocaine dependence. Cochrane Database Syst Rev 2003; 2:CD003352Google Scholar
68 . Nowinski J, Baker S, Carroll KM: Twelve Step Facilitation Therapy Manual (NIH Publication No. 94–3722). Rockville, MD, National Institutes of Health, 1995Google Scholar
69 . Miller WR, Rollnick S: Motivational Interviewing: Preparing People for Change, 2nd ed. New York, Guilford Press, 2002Google Scholar
70 . Hayes SC, Wilson KG, Gifford EV: The use of twelve-step facilitation and acceptance and commitment therapy with polysubstance abusing methadone maintained opiate addicts: a randomized controlled trial. Behav Ther 2004; 35:679–688Google Scholar
71 . McCrady BS, Irvine S: Self-help groups, in Handbook of Alcoholism Treatment Approaches. Edited by Hester RK, Miller W. New York, Pergamon Press, 1989, pp 153–169Google Scholar
72 . Gifford EV, Ritsher JB, McKellar JD, et al: Acceptance and relationship context: a model of substance use disorder treatment outcome. Addiction 2006; 101:1167–1177Google Scholar
73 . Gifford EV, Humphreys KN: The psychological science of addiction. Addiction 2007; 102:352–361Google Scholar
74 . Stead LF, Lancaster T: Group behavior therapy programmes for smoking cessation. Cochrane Database Syst Rev 2005; 2:CD001007Google Scholar
75 . Blum K, Braverman ER, Holder JM, et al: Reward deficiency syndrome: a biogenetic model for the diagnosis and treatment of impulsive, addictive, and compulsive behaviors. J Psychoactive Drugs 2000; 32(S1–4):1–112Google Scholar
76 . Comings DE, Blum K: Reward deficiency syndrome: genetic aspects of behavioral disorders. Prog Brain Res 2000; 126:325–341Google Scholar
77 . Volkow ND, Fowler JS, Wang GJ, et al: Dopamine in drug abuse and addiction: results from imaging studies and treatment implications. Mol Psychiatry 2004; 9:557–569Google Scholar
78 . Noble EP: The DRD2 gene in psychiatric and neurological disorders and its phenotypes. Pharmacogenomics 2000; 1:309–333Google Scholar
