In-Vitro Reinforcement (IVR) is an experimental procedure (Stein, 1997; Stein, Xue, & Belluzzi, 1993; 1994; Stein & Belluzzi, 1989) that demonstrates a form of neural plasticity apparently distinct from Long-Term Potentiation (LTP). The basic result is changes in neural activity due to the infusion of the neuromodulator, dopamine. A neuron that exhibits a characteristic multi-spike burst is monitored in-vitro. Whenever a burst is detected, dopamine is injected around the cell via pipette. The burst rate is observed to increase. The basic notion is that initially random activity, when regularly followed by a biologically important event, will come to occur more often. This idea is credited to Skinner's (1938; 1974) interpretation of Thorndike's (1898) Law of Effect.

There are important distinctions between the theoretical model of IVR and that of LTP. The most important of these is that while LTP involves variable sensitivity of synaptic connections between two neurons, IVR appears to involve variable sensitivity of a single neuron to its overall activity due to various causes. In LTP, correlated activity by both a pre-synaptic and post-synaptic neuron facilitates later transmission of signals from the former to the latter. In short, after both neurons are activated simultaneously, the conditional likelihood that the post-synaptic neuron fires given that the pre-synaptic neuron fires is raised. With IVR, only one neuron is monitored. The unconditional burst rate of that neuron rises after diffuse intercellular dopamine injections are made conditional upon previous bursting.

IVR has been demonstrated in pyramidal cells from the CA1 layer of the rat hippocampus. There is no reason to assume that other types of cells cannot exhibit IVR. (Stein, 1997, suggests that reinforcable neurons "have a wide distribution in the brain.") CA1 pyramidal cells are relatively easily isolated in vitro and exhibit multi-spike calcium-bursts. Individual spikes recorded from neurons are not conditionable using the IVR procedure. Calcium bursts are. This makes CA1 pyramidal cells ideal for initial investigations of IVR.

Normally, hippocampal bursts are self-limiting due to the fact that the introduction of calcium into the cell causes the dephosphorylation of the L-type Ca²⁺ channels critical to calcium bursts. Stein (1997) proposes that diffuse dopamine around the dendritic spines prevents the dephosphorylation, but only if dopamine delivery follows within 200 milliseconds of a calcium burst (Stein & Belluzzi, 1989). In proposing distinct sequences of events for reinforced and non-reinforced bursting, Stein (1997) offers the following molecular hypothesis of in-vitro reinforcement (IVR):

Molecular Hypothesis of In-Vitro Reinforcement

Hippocampal bursts are made up of a few initial sodium spikes followed by a succession of calcium spikes (Schwartzkroin & Slawsky, 1977; Wong & Prince, 1978; Jensen, Azouz, & Yarri, 1994). The latter are mediated by voltage-gated L-type channels, which open in response to membrane depolarization (especially that produced by the Na⁺ spikes) if the Ca²⁺ channel protein is phosphorylated. The enzymatic addition or removal of phosphate esters alters the conformation and thus changes the activity state of many nerve cell proteins (Nestler & Greengard, 1984).

Nonreinforced bursting (Figure B-1): Following a burst of nonreinforced calcium spikes, a "protective" intracellular cascade is activated to reduce the probability of further bursting (Armstrong, 1989). This arrangement is thought to be self-protective, because each burst of calcium spikes introduces Ca²⁺ into the cell, and because high levels of intracellular Ca²⁺ are toxic. The burst-induced rise in intracellular Ca²⁺ activates the calcium-dependent enzyme calcineurin, which rapidly dephosphorylates the recently-active Ca²⁺ channels that participated in the burst. Calcineurin can itself dephosphorylate the channel protein, but it acts mainly indirectly via inactivation of a key inhibitor (DARPP-32) of the principal dephosphorylating enzyme (phosphatase-l) of L-type channels. Glutamate also elevates intracellular Ca²⁺ by stimulation of NMDA receptors. Effective doses of this transmitter would therefore activate the calcineurin pathway, and the resulting dephosphorylation of Ca²⁺ channels should reduce hippocampal bursting rates -- as we in fact have found (Stein, Xue, & Belluzzi, 1993). The glutamate-induced increase in the frequency of single spikes (i.e., Na⁺ spikes), which we also observed, may be explained by the simultaneous stimulation of non-NMDA (AMPA) receptors that control sodium channels.

Reinforced bursting: According to my hypothesis, if bursting responses are closely followed by stimulation of dopamine Dl/D5 receptors, the protective calcineurin pathway will be overridden and dephosphorylation of the recently-activated Ca²⁺ channels thereby prevented. The central feature of any useful hypothesis of IVR must be a plausible molecular explanation of the burst-dependent nature of dopamine's reinforcing action. As in the case of behavioral reinforcers, response-independent applications of dopamine are not reinforcing; hence, in our account, they should not prevent the dephosphorylation of Ca²⁺ channels. What is needed most critically to complete the hypothesis is a molecular coincidence detector, able to respond selectively to the conjunction of bursting activity and dopamine receptor activation. I propose that the enzyme type VIII adenylyl cyclase performs this function.

Adenylyl cyclase, the enzyme that synthesizes cyclic AMP (cAMP), is the prototypical second messenger generator, indeed, the concept of signalling by second messengers originated with the discovery of the role of cAMP (for reviews, see Nestler & Greengard, 1984 and Cooper, Mons & Karpen, 1995). The ubiquitous cAMP-dependent protein kinase (cAMP kinase) pathway accounts for the hormonal control of many cellular events, included among these is the phosphorylation of L-type Ca²⁺ channels. Eight types of adenylyl cyclases have been cloned to date and each has been shown to be regulated by a variety of influences. Surprisingly, one of these adenylyl cyclases, type VIII, exhibits the precise biochemical properties and regional and subcellular localization required for our molecular coincidence detector. The highest concentrations of type VIII immunoreactivity are found postsynaptically in hippocampal CAl dendritic spines "in intimate association with sites of calcium ion entry into the cell" (Cooper, Mons & Karpen, 1995, p. 421); furthermore, type VIII is the only member of the adenylyl cyclase family that responds synergistically to Ca²⁺ and dopamine (via the stimulatory G protein, Gs) (Cali, Zwaagstra, Mons, Cooper & Krupinski, 1994). Thus, it can be anticipated that the conjunction of burst induced Ca²⁺ elevation and dopamine Dl/D5 receptor stimulation would readily and selectively activate type VIII adenylate cyclase; the cAMP generated would need to diffuse only a short distance before activating its enzymatic target (cAMP kinase), known to be anchored in high concentrations in the same dendritic spines. Such activation would override the calcineurin cascade and prevent the dephosphorylation of the Ca²⁺ channels in those spines. Finally, the important negative action of burst-independent dopamine is nicely explained by the fact that, in the absence of elevated Ca²⁺, the response of type VIII adenylyl cyclase to Dl/D5 activation will be inadequate.

Differences between IVR and LTP

Donahoe, Palmer, & Burgos (1997) have suggested that IVR may not be distinct from LTP. Stein (1997) replies as follows:

In my opinion, IVR and LTP are not mere variations of a common mechanism of synaptic plasticity; they are separate processes. If so, it is possible that their dissimilar neurophysiological properties, when expressed in behavior, may underlie in part the operant-respondent distinction. Although calcium-dependent signaling mechanisms are involved in both processes, IVR and LTP appear to depend on different types of Ca²⁺ channels. The relevant Ca²⁺ channel (NMDA channel) for LTP in the CAl area is well-established (Bliss & Collingridge, 1993); this receptor-operated Ca²⁺ channel is activated by glutamate in conjunction with membrane depolarization. On the other hand, the L-type Ca²⁺ channel seems to be critical for IVR (Stein, Xue, & Belluzzi, 1994); this voltage-gated Ca²⁺ channel is activated by membrane depolarization, but only if the channel protein is phosphorylated.

Evidence that implicates the L-type channel in IVR includes the following: (a) L-type channels control the generation of calcium spikes in hippocampal CAl and CA3 neurons (Kostyuk, 1989), (b) L-type channels are located in hippocampal CAl cell bodies and cluster in high density at the base of major dendrites (Westenbroek, Ahlijanian, & Catterall, 1990), (c) influx of Ca²⁺ through hippocampal L-type channels regulates gene transcription through a distinct signalling pathway (Bading, Guity, & Greenberg, 1993), thus providing a possible mechanism for long-term reinforcement effects, and (d) L-type channels must be phosphorylated in order to open when the cell membrane is depolarized (Armstrong, 1989); this property could reasonably provide the hippocampal pyramidal cell with a mechanism for modulating calcium fluxes in response to external (reinforcing) signals. The tentative identification of the L-type channel as a significant protein target of the cellular reinforcement process has suggested a plausible and testable molecular hypothesis of IVR and (by extrapolation) operant conditioning (Stein, 1994).

In other words, it might be the case that dopamine has its effect in IVR conditioning due to increased sensitivity by the pyramidal cell to whatever neurons are transmitting signals to it. This would make IVR the result of synaptic plasticity like LTP and possibly even due to the same underlying mechanism. In addition to the above molecular evidence, there are reasons to doubt this. First and foremost, even if IVR involves synaptic sensitivity, dopamine is not necessary for LTP to occur, so IVR involves, at a very minimum, more than just LTP. Second, glutamate, the neuromodulator involved in LTP, has no effect on IVR. Evidence suggests that, while sodium spikes may initiate a calcium burst when a sufficient number of phosphorylated Ca²⁺ channels are available, the requisite calcium spikes are governed by post-synaptic activity. Both the logic and the biology of IVR appear distinct from those of LTP.

Evidence for the Molecular Hypothesis

Stein (1997) reports the following ongoing research:

The model is being tested at various steps of the proposed cascade. Three such IVR tests are in progress. In the first test, microinjections of forskolin (a Gs-mimicking activator of adenylyl cyclase) are substituted for dopamine as reinforcement for CAl bursting. It is anticipated that forskolin will function as a typical in vitro reinforcer and produce burst-contingent, but not burst-independent, increases in hippocampal bursting. Similar IVR experiments also are being performed with a membrane-permeant cAMP analogue (8-Br-cAMP, test 2) and a phosphastase-l inhibitor (okadaic acid, test 3). Both 8-Br-cAMP and okadaic acid exert their effects at late stages of the phosphorylation cascade and thus by-pass the coincidence-detecting action of type VIII adenylyl cyclase, which occurs at an earlier step. Hence, according to the model, these agents should not require a contemporaneous Ca²⁺ signal to be effective, and both should facilitate bursting whether administered on a burst contingent or burst-independent schedule. So far, all three tests have yielded promising results.

Armstrong, D. L. (1989). Calcium channel regulation by calcineurin, a Ca2+-activated phosphatase in mammalian brain. Trends in Neuroscience, 12, 117-122.

Bading, H., Guity, D. D., & Greenberg, M. E. (1993). Regulation of gene expression in hippocampal neurons by distinct calcium signalling pathways. Science, 260, 181-186.

Bliss, T. V. P. & Colingridge, G. L. (1993). A synaptic model of memory: Long-term potentiation in the hippocampus. Nature, 361, 31-39.

Bliss, T. V. P. & Lømø, T. (1973). Long-lasting potentiation of synaptic transmission in the dentate area of the anaesthetized rabbit following stimulation of the perforant path. Journal of Physiology, 232, 331-356.

Cali, J. J., Zwaagstra, J. C., Mons, N., Cooper, D. M. F., & Krupinski, J. (1994). Type VIII adenylyl cyclases and the interaction between calcium and cAMP signalling. Journal of Biological Chemistry, 269, 12190-12195.

Cooper, T. F., Mons, T. S., & Karpen, T. T. (1995). Adenylyl cyclases and the interaction between calcium and cAMP signalling. Nature, 374, 421-424.

Donahoe, J. W., Palmer, D. C., & Burgos, J. E. (1997). The S-R issue: Its status in behavior analysis and in Donahoe and Palmer's Learning and Complex Behavior. Journal of the Experimental Analysis of Behavior, 67, 193-211.

Jensen, M. S., Azouz, R., & Yarri, Y. (1994). Variant firing patterns in rat hippocampal pyramidal cells modulated by extracellular potassium. Journal of Neurophysiology, 71, 831-839.

Kostyuk, P. G. (1989). Diversity of calcium ion channels in cellular membranes. Neuroscience, 28, 253-261.

Nestler, T. S. & Greengard, P. (1984). Protein phosphorylation in the nervous system. New York: Wiley.

Schwartzkroin, P. A., & Slawsky, M. (1977). Probable calcium spikes in hippocampal neurons. Brain Research, 135, 157-161.

Skinner, B. F. (1938). The Behavior of Organisms: An experimental analysis. New York: Appleton-Century-Crofts.

Skinner, B. F. (1974). About Behaviorism. New York: Alfred A. Knopf.

Stein, L. (1994). In-vitro reinforcement of hippocampal bursting: Possible cellular and molecular mechanism of drug reward. Regulatory Peptides, 54, 285-286.

Stein, L. (1995, May). Skinner's Behavioral Atom: A cellular analogue of operant conditioning and its implications. Paper presented at the 21st annual meeting of the Association for Behavior Analysis, Washington, DC.

Stein, L. (1997). Biological substrates of operant conditioning and the operant-respondent distinction. Journal of the Experimental Analysis of Behavior, 67, 246-253.

Stein, L. & Belluzzi, J. D. (1982). Beyond the reflex arc: A neuronal model of operant conditioning. In A. R. Morrison & P. L. Strick (Eds.), Changing concepts of the nervous system (pp. 651-665). New York: Academic Press.

Stein, L. & Belluzzi, J. D. (1988). Operant conditioning of individual neurons. In M. L. Commons, R. M. Church, J. R. Stellar, & A. R. Wagner (Eds.), Quantitative Analyses of Behavior, Vol. VII (pp. 249-264). Hillsdale, NJ: Lawrence Erlbaum Associates.

Stein, L. & Belluzzi, J. D. (1989). Cellular investigations of behavioral reinforcement. Neuroscience and Biobehavioral Reviews, 13, 69-80.

Stein, L., Xue, B.G., & Belluzzi, J. D. (1993). A cellular analogue of operant conditioning. Journal of the Experimental Analysis of Behavior, 60, 41-53.

Stein, L., Xue, B.G., & Belluzzi, J. D. (1994). In vitro reinforcement of hippocampal bursting: A search for Skinner's atoms of behavior. Journal of the Experimental Analysis of Behavior, 61, 155-168.

Thorndike, E. L. (1898). Animal intelligence: An experimental study of the associative processes in animals. Psychological Review, Monograph Supplement 8, 1-109.

Westenbroek, R. E., Ahlijanian, M. K., & Catterall, W. A. (1990). Clustering of L-type Ca²⁺ channels at the base of major dendrites in hippocampal pyramidal neurons. Nature, 374, 281-284.

Wong, R. K. S. & Prince, D. A. (1978). Participation of calcium spikes during intrinsic burst firing in hippocampal neurons. Brain Research, 159, 385-390.