In the autumn of 1930, a young Harvard graduate student named Burrhus Frederic Skinner placed a hungry rat inside a small wooden box equipped with a lever and a food dispenser. The rat explored its enclosure at random. At some point — through accident, not intent — it pressed the lever. A pellet dropped. The rat ate. It pressed the lever again. Another pellet. Within minutes, the rat was pressing the lever with methodical persistence. Nothing in the rat's anatomy had changed. Nothing in its environment had changed. Only the relationship between one behavior and its consequence had been established, and that relationship, once formed, governed everything the animal did inside that box.
Skinner had not invented the chamber; he had refined a design he called the operant conditioning apparatus, which journalists would later call the Skinner box despite his documented irritation with the name. But what he had done was more important than engineering. He had isolated, in a closed system, the elementary unit of what he believed was the governing principle of all learned behavior: the operant — a class of behaviors defined not by the stimulus that precedes them but by the consequence that follows. His 1938 monograph, The Behavior of Organisms, laid out this framework with a precision that was almost confrontational in its ambition. Psychology, Skinner argued, did not need inner states, mental representations, or cognitive processes. It needed careful observation of environmental contingencies and the behaviors they produced or extinguished. The rats in his boxes were not, in his framework, thinking. They were being shaped.
The experiments that followed in the ensuing decade elaborated this framework into a complete science of behavior. Skinner moved from rats to pigeons, in part because pigeons proved more tractable subjects for demonstrating complex behavioral sequences. He taught pigeons to play table tennis, to guide missiles in wartime research, and to peck in precise sequences that required chains of operant behavior. In 1953, he published Science and Human Behavior, which extended the operant framework far beyond the laboratory: to education, to government, to psychotherapy, to the design of entire societies. The book was a manifesto as much as a textbook. By mid-century, operant conditioning had become not merely a theory of rat behavior but a candidate theory of human nature.
The Four Contingencies: A Comparative Framework
Operant conditioning is organized around four fundamental relationships between behavior and consequence. The terms are precise: "positive" means the addition of a stimulus; "negative" means the removal of a stimulus; "reinforcement" means the behavior becomes more likely; "punishment" means the behavior becomes less likely. Each of the four combinations has distinct properties, distinct effects, and distinct ethical implications.
| Contingency | Definition | Laboratory Example | Applied Example | Effect on Behavior | Key Consideration |
|---|---|---|---|---|---|
| Positive Reinforcement | A behavior is followed by the addition of an appetitive stimulus | Rat receives food pellet after lever press | Employee receives bonus after hitting sales target | Increases frequency and strength of behavior | Most robust and generalizable effect; least likely to produce emotional side effects |
| Negative Reinforcement | A behavior is followed by the removal of an aversive stimulus | Rat presses lever to terminate electric shock | Taking aspirin removes headache pain, increasing aspirin-taking | Increases frequency and strength of behavior | Commonly confused with punishment; both reinforcement types increase behavior |
| Positive Punishment | A behavior is followed by the addition of an aversive stimulus | Rat receives brief shock after pressing wrong lever | Child receives reprimand after hitting sibling | Decreases frequency of behavior | Produces suppression but not unlearning; can generate fear, aggression, avoidance |
| Negative Punishment | A behavior is followed by the removal of an appetitive stimulus | Rat loses access to food after pressing wrong lever | Teenager loses driving privileges after curfew violation | Decreases frequency of behavior | Produces fewer aversive side effects than positive punishment; requires valued resource |
The table clarifies a confusion that persists even among educated readers: negative reinforcement is not a form of punishment. Both negative reinforcement and positive reinforcement increase the probability of the behavior they follow. What distinguishes them is whether behavior is strengthened by gaining something desirable or by escaping something undesirable. The avoidance behaviors that sustain anxiety disorders — leaving a social situation to escape the discomfort of social anxiety, which briefly relieves the anxiety and thereby reinforces the avoidance — are negative reinforcement in its most clinically significant form.
Intellectual Lineage: From Thorndike to Skinner
Skinner was not working in an intellectual vacuum. The empirical foundation for operant conditioning had been laid three decades earlier by Edward Lee Thorndike, a Columbia University psychologist whose 1898 doctoral dissertation contained one of the most important experiments in the history of psychology. Thorndike placed cats in wooden "puzzle boxes" — slatted enclosures from which escape was possible by pressing a lever, pulling a loop, or striking a latch — and recorded how long each escape took on successive trials. The cats' behavior was initially random and exploratory. Over trials, the behaviors that led to escape occurred sooner and sooner. Thorndike called this the Law of Effect: responses followed by satisfying consequences become more strongly associated with the situation in which they occurred; responses followed by annoying consequences become more weakly associated. The law was stated in terms of association, not reinforcement, and Thorndike's framework remained tied to the stimulus-response vocabulary of his era.
Skinner acknowledged Thorndike's Law of Effect as the empirical precursor to his own work but argued that Thorndike had framed it incorrectly. The law, Skinner insisted, was not about associations between stimuli and responses. It was about the selection of responses by their consequences — a process he explicitly compared to Darwinian natural selection, with consequences playing the role of environmental selection pressure. This was not a minor terminological dispute. It reflected a fundamental reorientation: Thorndike had looked backward from the response to the stimulus; Skinner looked forward from the response to the consequence. The operant was defined by what came after.
Ivan Pavlov's work on classical conditioning, conducted in St. Petersburg between roughly 1890 and 1930, provided a contrasting and complementary framework. Pavlov's dogs salivated to a bell that had been paired with food — a reflexive response elicited by a conditioned stimulus. Skinner distinguished this sharply from operant conditioning: Pavlovian or "respondent" conditioning concerned reflexes elicited by antecedent stimuli; operant conditioning concerned voluntary behaviors emitted by organisms and selected by consequences. The distinction has since been complicated — it is now known that the two systems interact in complex ways — but it organized the field productively for decades.
John B. Watson's radical behaviorism, which dominated American psychology from roughly 1913 to the 1930s, had insisted that psychology must abandon introspection and study only observable behavior. Skinner inherited this methodological commitment but rejected Watson's reliance on physiological explanations and stimulus-response reflexes. Skinner's behaviorism was more radical, in a specific sense: it did not require even physiological events as explanatory entities. Behavior was to be explained by its environmental history alone.
Schedules of Reinforcement: The Mathematics of Habit
Among Skinner's most productive empirical contributions was his systematic investigation of how the schedule by which reinforcement is delivered — rather than simply whether reinforcement is delivered — shapes the rate, pattern, and persistence of behavior. Working with his graduate student Charles Ferster, Skinner published Schedules of Reinforcement in 1957, a 741-page analysis of behavioral patterns produced by different reinforcement arrangements. The findings were striking in their regularity and their implications.
Four basic schedules were identified and extensively characterized. A fixed ratio schedule delivers reinforcement after every nth response — a factory worker paid per unit produced. The behavioral pattern is characteristic: rapid responding punctuated by a brief pause after each reinforcement, producing a "post-reinforcement pause" that grows with the ratio requirement. A fixed interval schedule delivers reinforcement for the first response after a specified time has elapsed — a student studying before a scheduled exam. The pattern here is a "scallop": minimal responding early in the interval, with accelerating response rates as the time of reinforcement approaches. A variable ratio schedule delivers reinforcement after an unpredictable number of responses, with that number varying around an average — a slot machine paying out after an average of every 45 pulls, though the actual number on any given occasion is random. A variable interval schedule delivers reinforcement for the first response after a variable amount of time has elapsed — checking a social media feed for new notifications.
The variable ratio schedule produces behavior that is most resistant to extinction — that is, behavior that continues longest in the absence of reinforcement once it has been established. This property, which Skinner documented rigorously, has since become one of the most practically significant findings in the operant tradition. When reinforcement has been variable and unpredictable, the organism cannot distinguish between a run of non-reinforcement within the normal schedule and the onset of extinction. The gambling industry grasped this logic earlier than most: the slot machine is a variable ratio schedule, engineered to maintain high response rates and extreme resistance to extinction. Every unpaid pull is, within the logic of the variable ratio schedule, merely a not-yet-reinforced trial. The dopaminergic prediction error signal that fires in anticipation of potential reinforcement — to be described in the neuroscience section below — does not require that reinforcement arrive on every occasion to sustain the behavior. Uncertainty itself is sufficient.
Shaping and the Premack Principle
Operant conditioning as Skinner described it in the 1930s could account for the strengthening or weakening of behaviors that the organism already performed. But how could radically new behaviors — behaviors with no initial probability in the organism's repertoire — be brought into existence? The answer Skinner developed was shaping through successive approximation: the differential reinforcement of behaviors that increasingly resemble the target behavior. If a pigeon is to be trained to peck a specific disc, the trainer begins by reinforcing any head movement in the direction of the disc, then movement toward the disc, then proximity to the disc, then contact with the disc. Each reinforced response constitutes a closer approximation to the target, which is then required before the next reinforcement is delivered. The unwanted behaviors drop out through extinction; the successive approximations are maintained and extended.
Shaping is now the foundational technique of applied behavior analysis and animal training alike. Its importance extends beyond practical application: it demonstrated that operant conditioning could produce not just the strengthening of existing behaviors but the construction of genuinely novel behavioral sequences. Skinner argued that language acquisition itself was a product of shaping — parents differentially reinforce successive approximations to adult speech, strengthening phonemes, then words, then grammatical sentences. This argument would attract the most consequential critique in the history of the operant tradition.
The Premack Principle, proposed by David Premack in a 1965 paper in the Psychological Review, extended the operant framework in a direction that had practical implications for behavior modification. Premack observed that a high-probability behavior — one that an organism freely chooses to engage in frequently — could serve as a reinforcer for a low-probability behavior. Put differently: any behavior that is more probable than the target behavior can be used to reinforce the target behavior. A child who runs constantly but rarely reads can have running made contingent on reading. An adult who checks email compulsively but avoids exercise can make email contingent on completing a workout. The Premack Principle replaced the earlier notion of reinforcement as necessarily involving identifiable "drives" or biological needs; it reconceptualized reinforcement as a relative frequency relationship between behaviors. This had the practical consequence of enormously expanding the range of potential reinforcers in applied settings, since any preferred activity could be identified through naturalistic observation and then deployed as a contingent consequence.
Cognitive Science Enters the Picture
The dominance of the operant framework began to erode in the 1950s and 1960s, partly through internal complications and partly through external challenges from the emerging cognitive tradition. Edward Tolman had been a persistent dissenter throughout the Skinnerian ascendancy. His 1948 paper "Cognitive Maps in Rats," published in Psychological Review, reported a series of experiments in which rats that had explored a maze without reinforcement nonetheless showed rapid learning when reinforcement was later introduced — performing almost as well as rats that had been reinforced throughout. Tolman called this latent learning, and it challenged the operant principle that reinforcement was necessary for learning. Something was being learned during unreinforced exploration, and Tolman argued it was a cognitive map — a spatial representation of the environment — rather than a sequence of reinforced responses.
In 1959, Noam Chomsky published a devastating review of Skinner's 1957 book Verbal Behavior in Language, the journal of the Linguistic Society of America. Chomsky's critique, which ran to 33 pages in the journal, systematically argued that Skinner's operant account of language acquisition was either vacuous or demonstrably false. Human children acquire grammatical rules, Chomsky argued, that permit them to produce and understand sentences they have never heard before — a generativity that could not, in principle, be the product of differential reinforcement of specific utterances. The structural complexity of language — its recursion, its abstract hierarchical organization — was incompatible with a model built from discrete response-consequence pairings. Whatever children were acquiring when they learned language, it was not an enormous repertoire of reinforced verbal operants. Chomsky proposed instead that the human capacity for language was underwritten by an innate Language Acquisition Device — a species-specific cognitive endowment that no amount of operant conditioning could have created.
The Rescorla-Wagner model, published in 1972 in a volume edited by A.H. Black and W.F. Prokasy, approached the same challenge from within the conditioning tradition itself. Robert Rescorla and Allan Wagner proposed a mathematical model of classical conditioning in which learning was governed by the discrepancy between predicted and actual outcomes — the prediction error. The model formalized a fundamentally cognitive insight: animals do not simply associate stimuli that co-occur; they learn about the predictive relationships among stimuli, and they update their associations in proportion to the degree to which outcomes surprise them. When a stimulus reliably predicts reinforcement, additional pairings add little new association. When a stimulus fails to predict reinforcement that actually occurs, the error drives rapid learning. The model captured data that purely behavioral accounts could not — including phenomena like blocking, in which prior learning about one predictor retards learning about a new predictor introduced simultaneously.
Rescorla extended these arguments in a 1988 paper, "Pavlovian Conditioning: It's Not What You Think It Is," published in American Psychologist. The title was deliberately provocative. Animals, Rescorla argued, do not learn mere stimulus-response associations; they learn about the structure of the world — which events predict which other events, which causes produce which effects. Conditioning is, in this account, a form of causal inference, not a passive accrual of associations through temporal contiguity.
Four Case Studies
Case 1: The Token Economy in Inpatient Psychiatry (Ayllon and Azrin, 1968)
Teodoro Ayllon and Nathan Azrin developed the token economy system at Anna State Hospital in Illinois during the 1960s and published their comprehensive account in The Token Economy: A Motivational System for Therapy and Rehabilitation (1968). Ayllon and Azrin identified specific target behaviors for chronically hospitalized psychiatric patients — self-care tasks, social interaction, participation in work programs — and established a currency of tokens that patients received contingent on performing those behaviors. Tokens could be exchanged for a menu of backup reinforcers: preferred foods, access to private rooms, recreational activities, off-ward privileges.
The results were substantial. Ayllon and Azrin reported dramatic increases in target behaviors compared to baseline and to control conditions. Patients who had been unresponsive or deteriorated under conventional custodial care showed measurable functional improvements. The token economy was subsequently implemented in hundreds of settings — schools, correctional facilities, substance abuse treatment programs, rehabilitation units — and became one of the most extensively studied applications of operant principles to real-world behavior change. The ethical questions the system raised — about who determines the target behaviors, who controls the backup reinforcers, and what distinguishes treatment from coercion — have remained active concerns in the behavior analysis literature.
Case 2: Applied Behavior Analysis for Autism (Lovaas, 1987)
O. Ivar Lovaas, working at the University of California Los Angeles, published a study in 1987 in the Journal of Consulting and Clinical Psychology that would prove the most influential — and contested — paper in the applied behavior analysis literature. Lovaas assigned young children with autism to an intensive operant treatment program — 40 hours per week of one-to-one behavioral intervention using discrete trial training, a structured technique based on shaping and differential reinforcement — or to a control group receiving minimal behavioral treatment.
After two years, 47 percent of the intensive treatment group had achieved IQ scores in the normal range and had been placed in regular education classrooms, compared with two percent of the control group. Lovaas interpreted these results as demonstrating that intensive early behavioral intervention could produce "normal intellectual and educational functioning" in some children with autism.
The study generated enormous clinical impact and equally enormous methodological scrutiny. Critics noted that the diagnostic categories were not equivalent across conditions, that the sample was not randomly assigned, that the treatment was implemented by Lovaas's own laboratory rather than independent practitioners, and that follow-up assessment was incomplete. The concept of "recovery from autism" raised questions about what had actually changed. A subsequent replication by McEachin, Smith, and Lovaas (1993), published in American Journal on Mental Retardation, maintained that gains had been sustained at a long-term follow-up. The broader evidence base for early intensive behavioral intervention has continued to grow, with consistent but more modest effect sizes than Lovaas's original data suggested. The debate over its mechanisms — whether gains reflect operant learning, social learning, neuroplasticity, or some combination — remains unresolved.
Case 3: The Misbehavior of Organisms (Breland and Breland, 1961)
Keller Breland and Marian Breland had been graduate students of Skinner's at the University of Minnesota. They left academia to found Animal Behavior Enterprises, a commercial animal training company that produced trained animal displays for television programs, theme parks, and department store promotions. With their technical expertise in operant conditioning and the resources of a commercial operation, they trained thousands of animals of dozens of species — raccoons to deposit coins in a piggy bank, chickens to play miniature pianos, pigs to carry wooden discs to a large piggy bank.
In 1961, they published a paper in American Psychologist titled "The Misbehavior of Organisms" — a deliberate echo of Skinner's 1938 title — that reported a systematic pattern they had discovered through this work. Animals trained with food reinforcement to perform behaviors that conflicted with their species-typical food-procurement behaviors showed progressive deterioration of the trained behavior in favor of instinctive behaviors. The raccoon, trained to deposit coins in a piggy bank, began to rub the coins together and dip them in and out of the bank — behaviors resembling the washing and manipulation of food items that raccoons perform with prey in the wild. The pig, trained to carry discs, began to root and toss and nose them along the ground, in patterns resembling pig foraging behavior. The trained behaviors "drifted" toward species-typical patterns despite their being incompatible with the reinforcement contingency.
Breland and Breland called this instinctive drift and argued it revealed a fundamental limit of operant conditioning: the behavior of organisms could not be understood solely in terms of reinforcement history. Biological constraints — the species-typical behavioral repertoires that evolution had built in — interacted with operant contingencies, and under certain conditions overrode them. The animals were not being shaped by consequences alone; their evolutionary heritage constrained the space of behaviors that conditioning could produce. This paper, written from within the operant tradition by students of its founder, was among the earliest serious empirical challenges to the sufficiency of the operant framework as a complete account of behavior.
Case 4: Insight Learning and the Limits of Trial and Error (Kohler, 1917)
Wolfgang Kohler's observations of chimpanzee problem-solving, conducted at the Prussian Academy of Sciences research station in the Canary Islands and published in German in 1917 as Intelligenzprufungen an Anthropoiden (translated as The Mentality of Apes), presented a different kind of challenge. Kohler placed his chimpanzees — particularly a male named Sultan — in situations where food was out of reach and could be obtained only by using tools or constructing novel solutions. Sultan learned to stack boxes to reach food hanging from the ceiling, to fit bamboo sticks together to form a longer tool to rake in fruit placed outside his enclosure, and to use a stick to knock another stick — which was too short to reach the food — toward himself so that the longer stick could then be retrieved and used.
What struck Kohler was not the eventual solution but the manner of its achievement. Sultan did not show the progressive, gradual improvement characteristic of Thorndikean trial-and-error learning. Instead, after a period of apparent inactivity or irrelevant behavior, the solution appeared suddenly and completely — Kohler described it as an "aha" experience, or insight. The solution, once achieved, was reproduced with minimal error on subsequent trials. This pattern — the sudden appearance of a complete solution following apparent contemplation, with immediate and reliable retention — was difficult to account for in terms of the gradual strengthening of responses through reinforcement. Kohler argued it required positing a cognitive process: the perceptual reorganization of the problem space. The animal was not being shaped; it was, in some sense, thinking.
The Neuroscience of Reinforcement
The most consequential convergence between the operant tradition and contemporary neuroscience arrived in 1997, when Wolfram Schultz, Peter Dayan, and P. Read Montague published "A Neural Substrate of Prediction and Reward" in Science. Schultz had spent years recording from dopaminergic neurons in the ventral tegmental area and substantia nigra of awake, behaving monkeys. The pattern he found in those neurons bore a striking formal resemblance to the Rescorla-Wagner prediction error signal.
At the outset of training, dopaminergic neurons fired in response to the delivery of an unexpected reward — juice or food. As training progressed and a conditioned stimulus began reliably predicting the reward, neuronal firing shifted: the neurons now fired in response to the conditioned stimulus, not in response to the reward itself. When reward was expected but failed to arrive, dopamine neurons showed a suppression of firing below baseline — a negative prediction error signal. When reward arrived unexpectedly, the neurons fired strongly — a positive prediction error signal. The neurons, in other words, were encoding the discrepancy between predicted and actual outcomes, not the outcomes themselves.
This finding unified, for the first time, the behavioral mathematical models of conditioning with a neural substrate. Reinforcement learning — the computational framework derived from operant principles — could now be mapped onto a specific neurochemical system. The dopamine neuron was, in effect, computing the Rescorla-Wagner delta rule in biological substrate. The variable ratio schedule's extraordinary resistance to extinction was explicable in terms of dopaminergic prediction error: unpredictable reinforcement sustains dopaminergic firing to conditioned stimuli precisely because the outcome is never fully predicted, so there is always a prediction error component maintaining the associative strength of the cue. The slot machine and the Skinner box operate through the same neural circuitry; they merely do so at different magnitudes and with different consequences for the organism's welfare.
Limits, Critiques, and Enduring Nuances
The operant framework has generated more productive research, more applied technology, and more sustained criticism than almost any other theoretical proposal in twentieth-century psychology. The criticisms are worth taking seriously, not because they demolish the framework, but because they delimit it.
The biological constraints literature, inaugurated by Breland and Breland (1961) and extended by John Garcia and Robert Koelling's 1966 paper in Psychonomic Science, demonstrated that conditioning is not uniform across all stimulus-response-outcome combinations. Garcia and Koelling showed that rats readily associate tastes with gastric illness even when the illness is delayed by hours — but do not readily associate lights or sounds with gastric illness, even when the delay is shorter. Conversely, rats readily associate exteroceptive cues (lights, sounds) with external shock but do not readily associate taste with shock. Organisms come to learning situations with preparedness — some associations are easy to form, others are difficult or impossible, and the pattern of ease and difficulty follows the organism's ecological history. The operant framework as Skinner presented it assumed an essentially arbitrary relationship between any stimulus, response, and reinforcer; the biological constraints research showed this assumption to be false.
The cognitive critique represented by Tolman, Rescorla, and the broader cognitive revolution has been substantially vindicated. Learning is not merely the strengthening of response tendencies by consequences; it involves the formation of representations of environmental structure. Animals and humans learn what predicts what, what causes what, which situations call for which behavioral strategies. The Rescorla-Wagner model, despite being derived from within conditioning research, is fundamentally a model of cognitive error-correction learning, not of response strengthening. The behavioral technology of operant conditioning retains its utility; the theoretical claim that consequences alone explain learning has not survived.
The language acquisition debate, set in motion by Chomsky's 1959 review, has not been fully resolved but has clearly shifted against Skinner's position. The existence of a critical period for language acquisition, the universality of grammatical structure across cultures, the inability of intensive reinforcement procedures to teach grammar to non-human primates beyond very narrow limits, and the generativity of human language all suggest that the operant framework is insufficient as an account of language development. This does not preclude operant contributions to vocabulary acquisition or pragmatic competence; it means that the structural core of language requires an account that conditioning cannot provide.
The pure behavioral model also neglects the role of observational learning, demonstrated by Albert Bandura in a series of studies beginning in 1961. Children who watched an adult model behave aggressively toward a Bobo doll subsequently reproduced those behaviors — including novel behaviors they had never themselves performed and had never been reinforced for performing. Social learning, in Bandura's account, proceeded through observation and cognitive encoding rather than through direct reinforcement of the learner's own responses. Bandura's broader social cognitive theory incorporated operant principles but situated them within a cognitive architecture that Skinner would not have accepted.
The motivational consequences of reinforcement have also proven more complex than the simple strengthening model suggests. Research by Mark Lepper, David Greene, and Richard Nisbett, published in the Journal of Personality and Social Psychology in 1973, demonstrated an "overjustification effect": children who were rewarded for engaging in an activity they had initially found intrinsically interesting showed reduced subsequent interest in the activity when rewards were withdrawn, compared to children who had received no reward. Extrinsic reinforcement, under some conditions, undermined intrinsic motivation — a finding that a simple operant model could not predict and that generated a large literature on the relationship between reward and motivation. The effect is real and has been meta-analytically confirmed, though its magnitude varies substantially with the type and salience of the reward.
The Empirical Research Foundation
Despite these critiques, the empirical foundation of operant conditioning is among the most solid in behavioral science. The basic phenomena — the acquisition, maintenance, extinction, and recovery of behavior as a function of reinforcement contingencies — are extraordinarily reliable, replicated across species ranging from pigeons to primates, and observable in human behavior in both laboratory and naturalistic settings. The four reinforcement schedules produce their characteristic behavioral signatures with remarkable consistency across organisms and environments. Shaping procedures that produce novel behavioral sequences work in training laboratories, therapeutic settings, and classrooms. The resistance-to-extinction gradient across schedules has been confirmed in hundreds of studies.
The applied behavior analysis literature, reviewed in journals including the Journal of Applied Behavior Analysis (founded 1968), contains a large evidence base for the use of operant procedures in treating problem behaviors, developing adaptive skills in individuals with intellectual disabilities, managing classroom behavior, and rehabilitating individuals with acquired brain injury. The evidence is not uniformly strong — effect sizes vary, generalization from trained to untrained settings is inconsistent, and maintenance of gains over time requires continued attention to contingencies — but the core procedures are well-validated.
The computational reinforcement learning tradition, which treats the Rescorla-Wagner and related algorithms as mathematical theories of learning, has generated an extensive literature in both neuroscience and artificial intelligence. Reinforcement learning algorithms underlie the most significant recent advances in machine learning, including the systems that defeated world champions at chess and Go. The operant principle — that behavior is shaped by its consequences, and that agents learn to maximize cumulative reinforcement through experience — has proven generative far beyond the boundaries of behavioral psychology.
Conclusion: An Incomplete but Irreplaceable Science
Skinner's rat pressing its lever in 1930 was not thinking, or not obviously thinking, in any way that required Skinner to invoke mental representations to explain what happened next. The rat pressed the lever; a pellet dropped; pressing became more frequent. The simplicity of this unit — the contingency, the consequence, the changed probability — turns out to explain a great deal about how organisms, including human organisms, acquire and maintain behavior. The four contingencies, the schedules of reinforcement, the principle of successive approximation, the Premack Principle: these are genuine contributions to our understanding of how behavior works, and they retain their explanatory power despite the theoretical disputes that have surrounded them.
What the operant framework cannot do, or cannot do alone, is account for the full complexity of learning. Biological constraints mean that organisms are not blank slates on which any contingency can write any behavior. Cognitive processes — representations, predictions, error-correction, insight — are necessary to explain the learning phenomena that Tolman, Rescorla, Kohler, and Chomsky identified. Social learning, observed and imitated without direct reinforcement, operates outside the simple operant loop. Intrinsic motivation can be disrupted by extrinsic reinforcement in ways the model did not predict.
The legacy of operant conditioning is perhaps best understood as a productive tension. Skinner's radical simplification — reduce everything to behavior, consequences, and contingencies — generated research that disclosed both the extraordinary scope and the definite limits of the principle he was exploring. The principle is real. It is not the whole story. The mind that Skinner refused to look inside turned out to be necessary for explaining what happened in his own boxes.
References
Skinner, B. F. (1938). The Behavior of Organisms: An Experimental Analysis. New York: Appleton-Century-Crofts.
Skinner, B. F. (1953). Science and Human Behavior. New York: Macmillan.
Thorndike, E. L. (1898). Animal intelligence: An experimental study of the associative processes in animals. Psychological Review Monograph Supplements, 2(4), 1-109.
Ferster, C. B., & Skinner, B. F. (1957). Schedules of Reinforcement. New York: Appleton-Century-Crofts.
Premack, D. (1965). Reinforcement theory. In D. Levine (Ed.), Nebraska Symposium on Motivation (Vol. 13, pp. 123-180). Lincoln: University of Nebraska Press.
Breland, K., & Breland, M. (1961). The misbehavior of organisms. American Psychologist, 16(11), 681-684.
Tolman, E. C. (1948). Cognitive maps in rats and men. Psychological Review, 55(4), 189-208.
Rescorla, R. A., & Wagner, A. R. (1972). A theory of Pavlovian conditioning: Variations in the effectiveness of reinforcement and nonreinforcement. In A. H. Black & W. F. Prokasy (Eds.), Classical Conditioning II: Current Research and Theory (pp. 64-99). New York: Appleton-Century-Crofts.
Rescorla, R. A. (1988). Pavlovian conditioning: It's not what you think it is. American Psychologist, 43(3), 151-160.
Schultz, W., Dayan, P., & Montague, P. R. (1997). A neural substrate of prediction and reward. Science, 275(5306), 1593-1599.
Lovaas, O. I. (1987). Behavioral treatment and normal educational and intellectual functioning in young autistic children. Journal of Consulting and Clinical Psychology, 55(1), 3-9.
Chomsky, N. (1959). Review of Verbal Behavior by B. F. Skinner. Language, 35(1), 26-58.
Frequently Asked Questions
What is operant conditioning?
Operant conditioning is a learning process in which the frequency of a voluntary behavior is modified by its consequences. B.F. Skinner developed the systematic study of operant conditioning in his 1938 book 'The Behavior of Organisms' and the broader theoretical framework in 'Science and Human Behavior' (1953). The core principle, inherited from Edward Thorndike's Law of Effect (1898), is that behaviors followed by satisfying consequences are strengthened (more likely to occur) and behaviors followed by unsatisfying or aversive consequences are weakened (less likely to occur). Skinner distinguished four contingency types: positive reinforcement (adding a rewarding stimulus after a behavior, increasing its frequency), negative reinforcement (removing an aversive stimulus after a behavior, increasing its frequency), positive punishment (adding an aversive stimulus after a behavior, decreasing its frequency), and negative punishment (removing a rewarding stimulus after a behavior, decreasing its frequency). The most common misconception is that 'negative reinforcement' is a form of punishment — in fact, both positive and negative reinforcement increase behavior; they differ only in whether they work by adding or removing stimuli.
How do schedules of reinforcement shape behavior?
Skinner and Charles Ferster's 1957 book 'Schedules of Reinforcement' documented the behavioral signatures of four reinforcement schedules. Fixed ratio schedules (reinforcement after every nth response) produce high response rates with post-reinforcement pauses — factory piecework pay produces this pattern. Variable ratio schedules (reinforcement after an unpredictable average number of responses) produce the highest and most persistent response rates, with no post-reinforcement pauses — slot machines, social media notifications, and fishing exploit this schedule. Fixed interval schedules (reinforcement after a fixed time period if any response occurs) produce scalloping patterns: low responding after reinforcement, escalating as the interval elapses — checking the oven on a 30-minute timer produces this. Variable interval schedules (reinforcement after unpredictable time intervals) produce steady, moderate response rates — checking email or refreshing a feed for expected but unpredictable content follows this schedule. Variable ratio schedules are most resistant to extinction: because reinforcement is unpredictably intermittent, the absence of reinforcement provides no information that reinforcement will never come.
What is shaping and the Premack Principle?
Shaping is the procedure of reinforcing successive approximations to a target behavior that the organism has never previously performed. Because reinforcement can only follow behavior that already occurs, complex behaviors cannot be trained by simply waiting for the final form to appear. Skinner used shaping to train pigeons to play ping-pong: first reinforcing any movement toward the paddle, then reinforcing contact with the paddle, then reinforcing contact that directed the ball, and so on. David Premack's 1965 Psychological Review paper introduced the Premack Principle — a reformulation of reinforcement that dispenses with the concept of reward: any high-frequency (preferred) behavior can reinforce a low-frequency behavior. If a child spends more time playing video games than doing homework, playing video games can reinforce homework completion ('first homework, then video games'). The principle correctly predicts that the reinforcement relationship is reversible: if circumstances change such that homework becomes the higher-frequency activity, homework could theoretically reinforce game-playing — a result the conventional reward concept cannot easily explain.
What does neuroscience say about how reinforcement works in the brain?
Wolfram Schultz, Peter Dayan, and P. Read Montague's 1997 Science paper provided the neurobiological basis for operant conditioning: dopaminergic neurons in the ventral tegmental area of monkeys fire when an unexpected reward occurs, stop firing when an expected reward fails to occur (negative prediction error), and shift their response to the predictive cue once learning is established. This dopamine prediction error signal implements the Rescorla-Wagner learning rule (1972) — the fundamental equation of associative learning — in neural hardware. The discovery explained why dopamine is rewarding: not because it signals pleasure directly, but because it encodes the discrepancy between expected and received outcomes, driving learning to reduce that discrepancy. It also explained the addictive power of variable ratio schedules: because reinforcement is unpredictable, each reward is an unexpected positive prediction error, producing a dopamine surge even after extended experience. The neural machinery that evolved to learn from consequences in natural environments is precisely what slot machines, social media algorithms, and addictive drugs exploit.
What are the limits of operant conditioning?
Keller and Marian Breland's 1961 American Psychologist paper 'The Misbehavior of Organisms' — the title an ironic nod to Skinner's 1938 book — documented a fundamental limit of pure operant conditioning: animals trained using food reinforcement for arbitrary behaviors that conflicted with their species-typical food-related behaviors would eventually drift back toward instinctive patterns despite consistent reinforcement of the trained behavior. Raccoons trained to drop coins into a piggy bank for food reinforcement began rubbing the coins together and dipping them in and out of the slot — behaviors resembling their natural food-washing and food-manipulation. This 'instinctive drift' demonstrated that biological constraints on learning override operant contingencies when motivated behaviors compete. John Garcia and Robert Koelling's 1966 Psychonomic Science paper showed a complementary point: rats could learn taste-illness associations (learned taste aversion) after a single trial with a long delay, but could not learn tone-illness associations with the same parameters — biology prepares some associations to be learned easily and others to be learned barely at all.