Human Behavior in Suboptimal Choice Tasks: Defining Optimality.

Suboptimal choice behavior, or behavior that leads to a loss of resources over time, has been observed in a laboratory setting from multiple species. A procedure commonly used to capture this effect involves presenting two alternatives during choice trials, one of which is optimal whereas the other is suboptimal. The optimal alternative yields reinforcement more often than the suboptimal alternative, but often does not produce signals that indicate whether reinforcement will occur. The suboptimal alternative produces less reinforcement than the optimal alternative but may include reinforcement-predictive stimuli that indicate to the organism whether reinforcement will occur. This procedural framework has consistently produced a preference for the suboptimal alternative in pigeons and, to a lesser extent, rats. However, human participants have demonstrated preference for the optimal alternative. Following a review of past suboptimal choice research, we applied the reinforcement-derived definition of optimality to two sets of our previously published human data. We found that under multiple conditions, human choice behavior was consistent with what was predicted by the proportion of obtained reinforcement, thus supporting that the behavior was optimal. However, we found that participants in two conditions chose the suboptimal alternative more than expected. This finding could be considered as a demonstration of suboptimal choice in humans. We propose that comparing choice behavior to what past obtained reinforcement outcomes would predict might be a more accurate view of whether patterns of choice are within the parameters of this task.

Treatments for undefined log ratios in matching analyses.

A challenge in carrying out matching analyses is to deal with undefined log ratios. If any reinforcer or response rate equals zero, the logarithm of the ratio is undefined: data are unsuitable for analyses. There have been some tentative solutions, but they had not been thoroughly investigated. The purpose of this article is to assess the adequacy of five treatments: omit undefined ratios, use full information maximum likelihood, replace undefined ratios by the mean divided by 100, replace them by a constant 1/10, and add the constant .50 to ratios. Based on simulations, the treatments are compared on their estimations of variance accounted for, sensitivity, and bias. The results show that full information maximum likelihood and omiting undefined ratios had the best overall performance, with negligibly biased and more accurate estimates than mean divided by 100, constant 1/10, and constant .50. The study suggests that mean divided by 100, constant 1/10, and constant .50 should be avoided and recommends full information maximum likelihood to deal with undefined log ratios in matching analyses.

Multilevel analysis of matching behavior: A comparison of maximum likelihood and Bayesian estimation.

While trying to infer laws of behavior, accounting for both within-subjects and between-subjects variance is often overlooked. It has been advocated recently to use multilevel modeling to analyze matching behavior. Using multilevel modeling within behavior analysis has its own challenges though. Adequate sample sizes are required (at both levels) for unbiased parameter estimates. The purpose of the current study is to compare parameter recovery and hypothesis rejection rates of maximum likelihood (ML) estimation and Bayesian estimation (BE) of multilevel models for matching behavior studies. Four factors were investigated through simulations: number of subjects, number of measurements by subject, sensitivity (slope), and variance of the random effect. Results showed that both ML estimation and BE with flat priors yielded acceptable statistical properties for intercept and slope fixed effects. The ML estimation procedure generally had less bias, lower RMSE, more power, and false-positive rates closer to the nominal rate. Thus, we recommend ML estimation over BE with uninformative priors, considering our results. The BE procedure requires more informative priors to be used in multilevel modeling of matching behavior, which will require further studies.

An evaluation of methods for studying the effects of conditioned reinforcement on human choice.

Shahan et al. (2006) found that the relative rate of pigeons' pecking on two observing responses (i.e., responses that only produced an S+ or stimulus correlated with primary reinforcement) was well described by the relative rate of S+ delivery. Researchers have not evaluated the effects of S+ delivery rate in a concurrent observing response procedure with human subjects, so the necessary procedural modifications for studying the effects of conditioned reinforcement on human choice remain unclear. The purpose of the current study was to conduct an additive component analysis of modifications to the procedures of Shahan et al. (2006). We evaluated the additive effects of introducing response cost, a changeover response, and ordinal discriminative stimuli on correspondence with the results of Shahan et al. and the quality of fits of the generalized matching equation. When our procedures were most similar to those of Shahan et al., we observed low rates of observing and indifference between the two observing responses. For the group of subjects with whom all three additive components were included, we obtained the highest level of sensitivity to relative rate of S+ delivery, but the slope and R2 of our fits of the generalized matching equation were still much lower than those obtained by Shahan et al. Potential reasons for these discrepancies, methods of resolving them, and implications for future research are discussed.

Multiple preferred escape trajectories are explained by a geometric model incorporating prey's turn and predator attack endpoint.

The escape trajectory (ET) of prey - measured as the angle relative to the predator's approach path - plays a major role in avoiding predation. Previous geometric models predict a single ET; however, many species show highly variable ETs with multiple preferred directions. Although such a high ET variability may confer unpredictability to avoid predation, the reasons why animals prefer specific multiple ETs remain unclear. Here, we constructed a novel geometric model that incorporates the time required for prey to turn and the predator's position at the end of its attack. The optimal ET was determined by maximizing the time difference of arrival at the edge of the safety zone between the prey and predator. By fitting the model to the experimental data of fish Pagrus major, we show that the model can clearly explain the observed multiple preferred ETs. By changing the parameters of the same model within a realistic range, we were able to produce various patterns of ETs empirically observed in other species (e.g., insects and frogs): a single preferred ET and multiple preferred ETs at small (20-50°) and large (150-180°) angles from the predator. Our results open new avenues of investigation for understanding how animals choose their ETs from behavioral and neurosensory perspectives.

When a prey spots a predator about to pounce, it turns swiftly and accelerates away to avoid being captured. The initial direction the prey chooses to take ­ known as its escape trajectory ­ can greatly impact their chance of survival. Previous models were able to predict the optimal direction an animal should take to maximize its chances of evading the predator. However, experimental data suggest that prey actually tend to escape via multiple specific directions, although why animals use this approach has not been clarified. To investigate this puzzle, Kawabata et al. built a new mathematical model that better represents how prey and predators interact with one another in the real world. Unlike past models, Kawabata et al. incorporated the time required for prey to change direction and only allowed the predators to move toward the prey for a limited distance. By including these two factors, they were able to reproduce the escape trajectories of real animals, including a species of fish, as well as species from other taxa such as frogs and insects. The new model suggests that prey escape along one of two directions: either by moving directly away from the predator in order to outrun its attack, or by dodging sideways to avoid being captured. Which strategy the prey chooses has some elements of unpredictability, which makes it more difficult for predators to adjust their capturing method. These findings shed light on why escaping in multiple specific directions makes prey harder to catch. The model could also be extended to test the escape trajectories of a wider variety of predator and prey species, which may avoid capture via different routes. This could help researchers better understand how predators and prey interact with one another. The findings could also reveal how sensory information (such as sound and sight) associated with the threat of an approaching predator is processed and stimulates the muscle activity required to escape in multiple different directions.

Basic and applied research on extinction bursts.

Discontinuation of the contingency between a response and its reinforcer sometimes produces a temporary increase in the response before its rate decreases, a phenomenon called the extinction burst. Prior clinical and basic studies on the prevalence of the extinction burst provide highly disparate estimates. Existing theories on the extinction burst fail to account for the dynamic nature of this phenomenon, and the basic behavioral processes that control response bursting remain poorly understood. In this paper, we first review the basic and applied literature on the extinction burst. We then describe a recent refinement of the concatenated matching law called the temporally weighted matching law that appears to resolve the above-mentioned issues regarding the extinction burst. We present illustrative translational data based conceptually on the model. Finally, we discuss specific recommendations derived from the temporally weighted matching law regarding procedures clinicians could implement to potentially mitigate or prevent extinction bursts.

Howard Rachlin: An extended scientist.

During the latter half of his career, Rachlin's work increasingly focused on integrating the study of temporal discounting and social cooperation-choices for an extended self. His notion of a self that is extended across time and social space is a useful framework within which to consider Rachlin's impact as a philosopher, scientist, and mentor over the course of his 56-year career in behavior science.

Overconsumption as a function of how individuals make choices: A paper in honor of Howard Rachlin's contributions to psychology.

Howard Rachlin's widely influential behavioral economic approach to self-control and related issues provides the model for this submission. The topic is overconsumption. Current human consumption levels are unsustainable. Explanations typically focus on societal factors, such as the seductive power of advertising and/or misguided tax policies. However, the effectiveness of these factors depends on the degree to which individuals are susceptible to the message: "consume more." Humans are not blank slates. This paper argues that how individuals frame their choices establishes the susceptibility to overconsume. According to economic theory, consumers frame their options as bundles, composed of different combinations of the available items and activities. This leads to maximizing. In experiments, participants tend to frame their options as "either-or" choices. This leads to the matching law. Mathematical models of concurrent schedule choice procedures show that (1) the matching law implies overconsumption of the most preferred option and (2) that individuals will persist in preferring their favorite option even when doing so reduces overall reward rates. Given that the matching law better describes how individuals choose than does maximizing, the mathematical models of widely used choice procedures help explain why efforts to increase consumption have been more influential than efforts to control consumption.

Igualación entre Actividades Hechas por y Recibidas de la Pareja / Matching among Activities Done for and Received from the Spouse

Resumen En el presente estudio se investigó si la frecuencia de las actividades hechas por la pareja tiende a igualar la frecuencia de las actividades recibidas del cónyuge; conforme a la ley de igualación. Participaron 120 hombres y 120 mujeres divididos en seis bloques de cinco años de matrimonio cada uno. Se pidió a los participantes que, de una lista de 63 actividades divididas en 9 áreas de interacción marital, indicaran cuáles realizaban por y cuáles recibían de su pareja. Posteriormente, se estimó si la proporción de actividades que los miembros de la pareja reportaron que hacían por su pareja tendía a igualar la proporción de actividades que decían recibir de ella. Los participantes reportaron que la proporción de actividades que hacen por su pareja es similar a la proporción de actividades que reciben de ella. Estos hallazgos sugieren que la relación entre las actividades del matrimonio y los reforzadores que mantienen unida a una pareja puede explicarse cuantitativamente con la ley de igualación; esto es, en términos de la relación entre los patrones de conducta que los miembros de la pareja hacen por y reciben de su pareja.

Abstract In the present study, it was explored if the frequency of the activities done for the spouse tends to match the frequency of the activities received from the partner, according to the matching law. Participants were 120 men and 120 women divided into six blocks of five years of marriage each one; that is, from 1 to 5, 6 to 10, 11 to 15, 16 to 20, 21 to 25, and more than 26 years of marriage. They were asked to choose, from a list of 63 activities divided into 9 areas of marital interaction (household responsibilities, raising children, social activities, finance, communication, sexual interaction, occupational or academic progress, personal independence, and spouse independence), which activities they performed for their partner and which ones they received from them. Based on the importance scores reported in a previous study, the proportion of the reinforcing value of the 63 marital activities was calculated. Subsequently, it was estimated whether the proportion of activities done for the spouse tends to equate the proportion of activities received from the partner. It was found that the proportion of activities the participants do for their partners is similar to the proportion of activities they received from them. Linear regressions were calculated for each block of years of marriage with the proportion of activities done predicted by the proportion of activities received. With these regressions, the deviation of the matching between the proportions was estimated. Slopes greater than 1.0 would show that participants reported to do more activities than they reported to receive; slopes lower than 1.0 would show that participants did fewer activities than those received; and if the value is close to 0.5 it would indicate indifference; that is, participants would do the same number of activities, regardless of those received from their partner. It was found that for men the slope varied between 0.71 and 1.02, and for women the slope varied between 0.908 and 1.035. These findings suggest that the relationship between marital activities and the reinforcers that hold a couple together can be explained quantitatively by the matching law; this is, by the correlation between the activities done for and receive from the spouse.

A Theory of the Extinction Burst.

A preliminary theory of a temporary increase in the rate of an operant response with the transition to extinction (i.e., the extinction burst) is proposed. The theory assumes reinforcers are events permitting access to some valuable activity, and that such activity can compete for allocation with the target response under some conditions (e.g., very high reinforcement rates). With the transition to extinction, elimination of this competition for allocation can produce an increase in the the target response, but the increase is transient because the value of the target response decreases with exposure to extinction. The theory provides a way to understand why the extinction burst is not ubiquitous, seems more common following very small ratio schedules, occurs for a short period of time following the transition to extinction, and may be eliminated with the availability of alternative reinforcement. It appears to provide a reasonable starting point for a theory of the extinction burst that does not necessarily require inclusion of invigorating effects of frustration, and it is closely aligned with Resurgence as Choice theory. Additional research on factors modulating reinforcement-related activities and how they affect the extinction burst could help to further evaluate the theory.

Instructional control in choice tasks: The relation between type of schedule and relative expected values.

The present work aims improve our understanding of the boundaries of instructional control. It does so by solving contradictory results obtained on two different fields: Three studies conducted on the description-experience gap field, showing that instructions are neglected when personal experience is available, and several others conducted on the experimental analysis of behavior paradigm getting to the opposite conclusion. Two factors were studied: the type of schedule, and the relative expected values between options. The present work showed that (1) positive evidence of instructional control was found in a choice task with probability schedules and different expected values between options; (2) negative evidence of instructional control was found in a choice task with VI schedules and similar expected values between options; and (3) these results, together with previous research, suggest that relative expected values are a fundamental factor on understanding the presence of instructional control in choice tasks. We conclude that the relevance of this factor relies on its capacity to make participants' decisions easier: all else being equal, adding descriptions enables participants to better discriminate optimal behavior in choice tasks.

A method for studying reinforcement factors controlling impulsive choice for use in behavioral neuroscience.

Although procedures originating within the experimental analysis of behavior commonly are used in behavioral neuroscience to produce behavioral endpoints, they are used less often to analyze the behavioral processes involved, particularly at the level of individual organisms (see Soto, 2020). Concurrent-chains procedures have been used extensively to study choice and to quantify relations between various dimensions of reinforcement and preference. Unfortunately, parametric analysis of those relations using traditional steady-state, single-subject experimental designs can be time-consuming, often rendering these procedures impractical for use in behavioral neuroscience. The purpose of this paper is to describe how concurrent-chains procedures can be adapted to allow for parametric examination of effects of the reinforcement dimensions involved in impulsive choice (magnitude and delay) within experimental sessions in rats. Data are presented indicating that this procedure can produce relatively consistent within-session estimates of sensitivity to reinforcement in individual subjects, and that these estimates can be modified by neurobiological manipulation (drug administration). These data suggest that this type of procedure offers a promising approach to the study of neurobiological mechanisms of complex behavior in individual organisms, which could facilitate a more fruitful relationship between behavior analysis and behavioral neuroscience.

The matching law provides a quantitative description of social time allocation in children with autism.

Recent research has developed and evaluated assessments of sociability in which time allocation near or away from an adult who initiates social interactions is used to characterize the participant as social, indifferent, or avoidant of social interaction. Though these qualitative outcomes have been useful, no studies have evaluated methods of obtaining more quantitative measures of sociability. The matching law has been demonstrated to describe a wide range of human behavior and may also be useful in describing social time allocation. We adapted the matching law and assessment of sociability procedures with the aim of providing a more precise, quantitative measure of sociability. We fitted the matching equation to the social time allocation data of 8 children with autism spectrum disorder. The equation was effective in quantifying sociability, accounted for a large proportion of variance in participants' behavior, did so equally well for participants who were social and avoidant, and provided a more sensitive measure relative to those used in previous research. The implications of this methodology, its potential utility, and directions for future research are discussed.

A quantitative analysis of the effects of alternative reinforcement rate and magnitude on resurgence.

Resurgence occurs when a previously reinforced and then extinguished target response increases due to a worsening of reinforcement conditions for an alternative response. We conducted four crowdsourcing experiments to evaluate effects of alternative-reinforcer rate and magnitude on resurgence with humans. Contingent on an alternative response, we manipulated across groups either the rate of point delivery (Experiment 1) or number of points delivered per reinforcer (Experiments 2-3). Experiment 4 arranged combinations of high- or low-rate and high- or low-magnitude alternative reinforcement across four groups. When extinguishing alternative responding across experiments, we observed resurgence of target responding with relatively high rates of alternative reinforcement but differences in reinforcer magnitude did not influence resurgence. A quantitative model based on the concatenated matching law, Resurgence as Choice in Context (RaC2), provided a poor fit to the data, generally underpredicted target responding, and could not account for data from control groups experiencing extinction of target responding in the absence of alternative reinforcement. We then fit a modified version of RaC2 borrowing an assumption from theories of choice that suggest some proportion of reinforcers are misallocated between responses - this modified version of RaC2 provided a better account of these findings.

Translating quantitative theories of behavior into improved clinical treatments for problem behavior.

The most important advancement in the treatment of destructive behavior has been the development of the functional analysis, which is used to prescribe effective treatments like functional communication training. Although this approach can be highly effective, extinction bursts and forms of treatment relapse commonly occur when function-based treatments are implemented by caregivers in natural community settings. In recent years, researchers have increasingly applied quantitative theories of behavior like behavioral momentum theory (BMT) and the temporally weighted matching law (TWML) to understand, prevent, or mitigate extinction bursts and treatment relapse. In this paper, we describe BMT and TWML and selectively review the basic, translational, and applied research supporting and opposing each theory. Then, we describe how function-based treatments may be refined based on these theories to improve the effectiveness, generality, and durability of function-based treatments for individuals with autism spectrum and related disorders who display problem behavior.

An introduction to "discrete choice experiments" for behavior analysts.

In this paper, we introduce discrete choice experiments (DCEs) and provide foundational knowledge on the topic. DCEs are one of the most popular methods within econometrics to study the distribution of choices within a population. DCEs are particularly useful when studying the effects of categorical variables on choice. Procedurally, a DCE involves recruiting a large sample of individuals exposed to a set of choice arrays. The factors that are suspected to affect choice are varied systematically across the choice arrays. Most commonly, DCE data are analyzed with a multinomial logit statistical model with a goal of determining the relative utility of each relevant factor. We also discuss DCEs in comparison with behavioral choice models, such as those based on the matching law, and we show an example of a DCE to illustrate how a DCE can be used to understand choice with behavioral, social, and organizational factors.

Evolutionary theory prediction: Response rate as a joint function of reinforcement rate and reinforcer magnitude.

Artificial organisms (AOs) animated by an evolutionary theory of behavior dynamics (ETBD) worked on concurrent interval schedules with a standard reinforcer magnitude on 1 alternative and a range of reinforcer magnitudes on the other. The reinforcer magnitudes on the second alternative were hedonically scaled using the generalized matching law. The AOs then worked on single interval schedules that arranged various combinations of the scaled reinforcer magnitudes and a range of nominal schedule values. This produced bivariate response rate data to which 5 candidate equations were fitted. One equation was found to provide the best description of the bivariate data in terms of percentage of variance accounted for, information criterion value, and residual profile. This equation consisted of 2 factors, 1 entailing the scaled magnitude, 1 entailing the obtained reinforcement rate, and both expressed in the form of exponentiated hyperbolas. The theory's prediction of the bivariate equation, along with additional predictions of the theory, were tested on data from an experiment in which rats pressed levers for various concentrations of sucrose pellets. The bivariate equation predicted by the theory was confirmed, as were all the additional predictions of the theory that could be tested on this data set.

Zebrafish choice behavior is sensitive to reinforcer rate, immediacy, and magnitude ratios.

Behavioral flexibility has, in part, been defined by choice behavior changing as a function of changes in reinforcer payoffs. We examined whether the generalized matching law quantitatively described changes in choice behavior in zebrafish when relative reinforcer rates, delays/immediacy, and magnitudes changed between two alternatives across conditions. Choice was sensitive to each of the three reinforcer properties. Sensitivity estimates to changes in relative reinforcer rates were greater when 2 variable-interval schedules were arranged independently between alternatives (Experiment 1a) than when a single schedule pseudorandomly arranged reinforcers between alternatives (Experiment 1b). Sensitivity estimates for changes in relative reinforcer immediacy (Experiment 2) and magnitude (Experiment 3) were similar but lower than estimates for reinforcer rates. These differences in sensitivity estimates are consistent with studies examining other species, suggesting flexibility in zebrafish choice behavior in the face of changes in payoff as described by the generalized matching law.

Dinámica de elección en humanos: efectos de la modalidad de respuesta / Dynamics of Choice in Humans: Effects of Response Modality

The main aim of the study was to assess the effects of varying response modality in the human allocation of operant responses to different reinforcement sources using a rapidly changing environment procedure. A total of nine undergraduate students participated. In each session, participants were presented with seven independent concurrent variable interval schedules of reinforcement in an irregular order. Individual schedules lasted for the delivery of five reinforcers. In Experiment 1, six students were randomly assigned to two conditions (BO and MT). In the BO condition, participants pressed buttons on the screen to earn points. In the MT condition, participants tracked and pressed moving targets that changed position every 2000 ms. Results showed that sensitivity to reinforcement developed just for the BO condition. A second Experiment was conducted, having three participants, and using the MT condition only. The rate of target position change parameter was modified to 300 milliseconds. Under those conditions, the development of sensitivity to reinforcement was observed. However, variable performances were found. It is concluded that response modality can affect the development of sensitivity if response requisites to obtain reinforcement depend on the way the response modality is arranged (AU)

Matching, induction, and covariance with mixed response-contingent food and noncontingent food.

The multiscale molar view of behavior is based on three basic laws of behavior: the Law of Allocation, the Law of Induction, and the Law of Covariance. Experiments that mix response-contingent food with noncontingent food shed light on these three laws. Food, like other phylogenetically important events, induces various activities that compete in allocation. Quantitative accounts represent induction with power functions. These power functions define activities' competitive weights, and relative time allocation among activities matches relative competitive weight. Behavior-food covariance determines which activities are induced. Phylogenetic (behavior-fitness) covariance determines which adjunctive activities are induced. Ontogenetic covariance may be represented in feedback functions. Feedback functions for variable-interval schedules may be observed even when overlaid by noncontingent food deliveries. Equations derived from the three laws describe responding in experiments with mixed response-contingent and noncontingent food. Equations derived here accounted for responding in three data sets: (a) from Rachlin and Baum (1972); (b) a new data set in which overall food rate was fixed while the proportion of response-contingent and noncontingent food varied; and (c) a new data set in which food occurred according to various variable-interval schedules. The same pigeons served throughout. All results were accommodated by the derived equations.