Training and maintenance of keypecking in the pigeon by negative reinforcement.

Pigeons were trained to peck a key to escape a pulsing shock of linearly increasing intensity. As the rate of increase was varied from 0.0374 milliamperes per minute to 37.4 milliamperes per minute, the intensity at which most pecking occurred varied from 2.2 to 5.0 milliamperes.

Syntheses, analgetic activity, and physical dependence capacity of 5-phenyl-6,7-benzomorphan derivatives.

The synthesis, analgetic activity, and physical dependence capacity of a large number of 5-phenyl-6,7-benzomorphan derivatives are described. Observations made during the Stevens' rearrangement of 1-benzyl-1-methyl-delta 3-piperidinium salt derivatives (V) under various conditions are discussed. The absolute configuration of the 9-demethyl series and the 2'-deoxy series is established by comparison of their ORD and CD spectra with those of 49, whose absolute configuration was previously established by X-ray crystallography. A convenient synthesis of 3H-labeled phenols using 3H3PO4 is described, as well as the preparation of 14C-labeled compounds by conventional methods.

Teleological behaviorism.

A psychological science of efficient causes, using internal mechanisms to explain overt behavior, is distinguished from another psychological science, based on Aristotelian final causes, using external objects and goals to explain overt behavior. Efficient-cause psychology is designed to answer the question of how a particular act is emitted; final-cause psychology is designed to answer the question of why a particular act is emitted. Physiological psychology, modern cognitive psychology, and some parts of behaviorism including Skinnerian behaviorism are efficient-cause psychologies; final-cause psychology, a development of Skinnerian behaviorism, is here called teleological behaviorism. Each of these two conceptions of causality in psychology implies a different view of the mind, hence a different meaning of mental terms.

Self-control and social cooperation.

Participants repeatedly played a self-control game in which choice of the higher of two monetary rewards on the present trial reduced the overall reward ('alone condition'). Other participants played a prisoner's dilemma (social cooperation) game in which choices alternated so that overall reward-reducing consequences of choosing the higher current amount were experienced by the other player ('together condition'). Participants playing the self-control game chose the lower current amount (and higher overall reward) significantly more frequently than did those playing the social cooperation game. In a second phase, half of the subjects who had played the self-control game played the social cooperation game and vice-versa. Little or no transfer was observed between conditions. In a second experiment, raising the amount of the next-trial reward increased self-control but not social cooperation. Some transfer between self-control and social cooperation was observed. The crucial variable responsible for participants' better performance (closer to optimization) in the self-control game compared to the social cooperation game may have been the higher probability in the former that choice of the lower reward on the present trial would be repeated on subsequent trials.

Diminishing marginal value as delay discounting.

The fundamental law underlying economic demand and exchange is the tendency for value of marginal units to diminish with increasing amounts of a commodity. The present paper demonstrates that this law follows from three still-more-basic psychological assumptions: (a) limited consumption rate, (b) delay discounting, and (c) choice of highest valued alternative. Cases of diminishing marginal value apparently due to pure intensity of reward may plausibly be attributed to the above three factors. The further assumption that maximum consumption rate may vary within and across individuals implies that some substances may be unusually addictive and that some individual animals may be unusually susceptible to addiction.

Behavioral economics without anomalies.

Behavioral economics is often conceived as the study of anomalies superimposed on a rational system. As research has progressed, anomalies have multiplied until little is left of rationality. Another conception of behavioral economics is based on the axiom that value is always maximized. It incorporates so-called anomalies either as conflicts between temporal patterns of behavior and the individual acts comprising those patterns or as outcomes of nonexponential time discounting. This second conception of behavioral economics is both empirically based and internally consistent.

A molar theory of reinforcement schedules.

Behavior of subjects exposed to concurrent and individual interval and ratio schedules of reinforcement may be described in terms of a set of expressions relating the value of responses to their durations, a feedback equation relating reinforcement to response duration, and the assumption that subjects allocate their time among various responses so as to maximize value.

Philosophical behaviorism: a review of things that happen because they should: a teleological approach to action, by Rowland Stout.

Mentalistic terms such as belief and desire have been rejected by behavior analysts because they are traditionally held to refer to unobservable events inside the organism. Behavior analysis has consequently been viewed by philosophers to be at best irrelevant to psychology, understood as a science of the mind. In this book, the philosopher Rowland Stout argues cogently that beliefs and desires (like operants such as rats' lever presses) are best understood in terms of an interaction over time between overt behavior and its overt consequences (a viewpoint called teleological behaviorism). This book is important because it identifies the science of the mind with the science of overt behavior and implies that the psychologists best equipped to study mental life are not those who purport to do so but those who focus on the experimental analysis of behavior.

The effect of shock intensity on concurrent and single-key responding in concurrent-chain schedules.

Pigeons were trained to respond in a two-link, concurrent-chain schedule. Pecks on each key during the concurrent initial links occasionally produced a 5-min terminal link, during which only that key was operative. Food reinforcement and various intensities of shock were scheduled during the terminal links. When shock was contingent on response, the effect of shock was greater on terminal-link responding than on initial-link responding. When shock was independent of response, the effect was reversed, with larger changes in initial-link responding than terminal-link responding. In general, shock was found to affect behavior most drastically when behavior could, in turn, affect the rate of shock.

Recovery of responses during mild punishment.

Pigeons were punished with mild shock for pecking during one of two components of a multiple schedule. They eventually recovered so that they pecked at the same rate during both components. In one experiment they were extinguished after recovering. When punishment was maintained during extinction, they extinguished faster during the punished, than during the unpunished component. When punishment was stopped during extinction, they extinguished faster during the unpunished than during the previously punished component. In another experiment, punishment was programmed first during neither of the two components, then during one, then during both, and finally during the other component. The extent of recovery decreased with each successive cycle. It is concluded that, if transient emotional states are ignored, reward and punishment are symmetrical in their effects.

Autoshaping of key pecking in pigeons with negative reinforcement.

Pigeons exposed to gradually increasing intensities of pulsing electric shock pecked a key and thereby reduced the intensity of shock to zero for 2 min. Acquisition of key pecking was brought about through an autoshaping process in which periodic brief keylight presentations immediately preceded automatic reduction of the shock. On the occasions of such automatic reduction of shock preceding the first measured key peck, little or no orientation to the key was observed. Observations of pigeons with autoshaping of positive reinforcement also revealed little evidence of orientation toward the key.

On the tautology of the matching law.

The generalized matching law, that organisms divide their time between alternatives in proportion to the value of the reinforcement consequent on the choice, is derivable from the assumption that an organism choosing between alternatives is under no constraints except those the contingencies of reinforcement impose. Hence, the law is not subject to empirical test. Its value lies in its simple codification of assumptions underlying choice experiments.

Response control with titration of punishment.

Pigeons and rats were exposed to multiple schedules with different schedules of electric shock superimposed on identical schedules of food reinforcement during each of two components. During one component, (adjusting-intensity) the intensity of electric shock depended on responding. Each response increased the intensity while intensity decreased between responses. During the other component (constant-intensity) the intensity was fixed at the value at which it had been adjusted at the end of the immediately preceeding adjusting-intensity component. In one experiment, shock was continuous during both components. In another experiment, instead of continuous shock, a brief pulse was delivered immediately after each response. During the adjusting-intensity component of both experiments, pigeons and rats responded at a rate just sufficient to keep the shock constant (critical rate). During the constant-intensity component, responding depended on whether shock was delivered continuously or in pulses. When shock was continuous, response rate during the constant-intensity component was higher than the critical rate. When shock was pulsed, response rate during the constant-intensity component was equal to the critical rate.

Economic substitutability of electrical brain stimulation, food, and water.

Concurrent variable-ratio schedules of electrical brain stimulation, food, and water were paired in various combinations as reinforcement of rats' lever presses. Relative prices of the concurrent reinforcers were varied by changing the ratio of the response requirements on the two levers. Economic substitutability, measured by the sensitivity of response ratio to changes in relative price, was highest with brain stimulation reinforcement of presses on both levers and lowest with food reinforcement of presses on one lever and water reinforcement of presses on the other. Substitutability with brain stimulation reinforcement of presses on one lever and either food or water reinforcement for presses on the other was about as high as with brain stimulation for presses on both levers. Electrical brain stimulation for rats may thus serve as an economic substitute for two reinforcers, neither of which is substitutable for the other.

Soft commitment: self-control achieved by response persistence.

With reinforcement contingent on a single peck on either of two available keys (concurrent continuous reinforcement schedules) 4 pigeons, at 80% of free-feeding weights, preferred a smaller-sooner reinforcer (2.5 s of mixed grain preceded by a 0.5-s delay) to a larger-later reinforcer (4.5 s of mixed grain preceded by a 3.5-s delay). However, when the smaller-sooner and larger-later reinforcers were contingent on a concurrent fixed-ratio 31 schedule (the first 30 pecks distributed in any way on the two keys), all pigeons obtained the larger-later reinforcer much more often than they did when only a single peck was required. This "self-control" was achieved by beginning to peck the key leading to the larger-later reinforcer and persisting on that key until reinforcement occurred. We call this persistence "soft commitment" to distinguish it from strict commitment, in which self-control is achieved by preventing changeovers. Soft commitment also effectively achieved self-control when a brief (1-s) signal was inserted between the 30th and 31st response of the ratio and with concurrent fixed-interval 30-s schedules (rather than ratio schedules) of reinforcement. In a second experiment with the same subjects, the fixed ratio was interrupted by darkening both keys and lighting a third (center) key on which pecking was required for various fractions of the fixed-ratio count. The interruption significantly reduced self-control. When interruption was complete (30 responses on the center key followed by a single choice response), pigeons chose the smaller-sooner reinforcer as frequently as they did when only a single choice response was required.

Eating and drinking: An economic analysis.

Food-deprived rats were exposed to various schedules of food delivery; water-deprived rats were exposed to various schedules of water delivery. Eating and drinking were measured over sessions and at points throughout sessions. The symmetries and asymmetries of food and water consumption were explored in terms of: (1) substitutability of food versus water, and of food and water on the one hand versus leisure on the other, (2) constraints imposed by various schedules of food and water, and (3) the tendency of rats to maximize utility within the imposed constraints.

Indifference between punishment and free shock: evidence for the negative law of effect.

Pigeons were trained to respond under two conditions with two identical variable-interval schedules of positive reinforcement. While the schedules operated for separate response keys, they were not available concurrently. During one condition, each response was punished with electric shock. During the other condition, shocks were delivered independently of responding. The punishment suppressed responding but the free shocks did not. However, when allowed to choose, the pigeons preferred the condition associated with the lowest rate of shock regardless of whether or not the shock was dependent on responding. In general, shocks exerted their greatest effect on whichever response had the greatest influence on shocks. In this respect, punishment is instrumental in suppressing behavior and the properties of punishment are symmetrical to those of reinforcement. This empirical symmetry dictates a corresponding conceptual symmetry in terms of a positive law of effect accounting for response increments and a negative law accounting for response decrements.

Commitment using punishment.

Experimental parameters were adjusted so that pigeons' pairwise choices among three alternatives reflected the following order of preference: (a) a smaller-sooner reinforcer, (b) a larger-later reinforcer, and (c) the smaller-sooner reinforcer followed by a punishment (consisting of an extended blackout period). After this order of preference was established, the pigeons were exposed to a two-link, concurrent-chain-like choice procedure. One terminal link consisted of a choice between the smaller-sooner and the larger-later reinforcer; the other terminal link was identical to the first except that the smaller-sooner reinforcer was followed by blackout punishment. The pigeons' preference (in their initial-link choice) for the terminal link with the punished smaller-sooner alternative increased as the delay between the initial and terminal links increased. By choosing this terminal link, the pigeons are said to have "committed" themselves to obtaining the larger-later reinforcer. However, unlike prior studies of commitment (e.g., Rachlin & Green, 1972), it was still possible after making the commitment for the pigeons to choose the smaller-sooner reinforcer and undergo the punishment. The pigeons did in fact occasionally make this highly deleterious choice.

Economic and biological influences on a pigeon's key peck.

Pigeons were studied in a two-component multiple schedule. In the first phase of the experiment, key pecks were reinforced on a variable-interval 2-min schedule in both components and free food was delivered additionally during one component. When components alternated every 8 sec, all pigeons pecked at a much higher rate during the component with free food than during the other component. At a component duration of 16 min, the reverse was true: all pigeons pecked at a higher rate during the component without free food. In the second phase, the additional food during one component was made contingent on pecking. Responding during the component without the extra food remained essentially unchanged, as expected, since rate of reinforcement remained identical to that in the previous phase. However, rate of responding during the component with the extra food (now contingent on pecking) was elevated, compared to the rate in the first phase, and did not show the marked decline as component duration was increased.

Commitment, choice and self-control.

When offered a choice (Choice Y) between a small immediate reward (2-sec exposure to grain) and a large reward (4-sec exposure to grain) delayed by 4 sec, pigeons invariably preferred the small, immediate reward. However, when offered a choice (Choice X) between a delay of T seconds followed by Choice Y and a delay of T seconds followed by restriction to the large delayed reward only, the pigeon's choice depended on T. When T was small, the pigeons chose the alternative leading to Choice Y (and then chose the small, immediate reward). When T was large, the pigeons chose the alternative leading to the large delayed reward only. The reversal of preference as T increases is predicted by several recent models for choice between various amounts and delays of reward. The preference for the large delayed alternative with long durations of T parallels everyday instances of advance commitment to a given course of action. Such commitment may be seen as a prototype for self-control.