A complete method for assessing the effectiveness of eyewitness identification procedures: Expected information gain.

We present a method for measuring the efficacy of eyewitness identification procedures by applying fundamental principles of information theory. The resulting measure evaluates the expected information gain (EIG) for an identification attempt, a single value that summarizes an identification procedure's overall potential for reducing uncertainty about guilt or innocence across all possible witness responses. In a series of demonstrations, we show that EIG often disagrees with existing measures (e.g., diagnosticity ratios or area under the receiver operating characteristic) about the relative effectiveness of different identification procedures. Each demonstration is designed to highlight key distinctions between existing measures and EIG. An overarching theme is that EIG provides a complete measure of evidentiary value, in the sense that it factors in all aspects of identification performance. Collectively, these demonstrations show that EIG has substantial potential to inspire new discoveries in eyewitness research and provide a new perspective on policy recommendations for the use of identifications in real investigations. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

A 'compensatory selection' effect with standardized tests: Lack of correlation between test scores and success is evidence that test scores are predictive of success.

We introduce the statistical concept of 'compensatory selection', which arises when selecting a subset of applicants based on multiple predictors, such as when standardized test scores are used in combination with other predictors required in a school application (e.g., previous grades, references letters, and personal statements). Post-hoc analyses often fail to find a positive correlation between test scores and subsequent success, and this failure is sometimes taken as evidence against the predictive validity of the standardized test. The present analysis reveals that the failure to find a negative correlation indicates that the standardized test is in fact a valid predictor of success. This is due to compensation between predictors during selection: Some students are admitted despite a low test score because their application is exceptional in other respects, while other students are admitted primarily based on a high test score despite weakness in the rest of their application. This compensatory selection process introduces a negative correlation between test scores and other predictors among those admitted (a 'collider bias' or 'Berkson's paradox' effect). If test scores are valid predictors of success, this negative correlation between the predictors counteracts the positive correlation between test scores and success that would have been observed if all applicants were admitted. If test scores are not predictive of success, but were nevertheless used in a compensatory selection process, there would be a spurious negative correlation between test scores and success (i.e., an admitted student with a weak application except for a high test score would be unlikely to succeed). The selection effect that is described here is fundamentally different from the well-known 'restricted range' problem and can powerfully alter results even in situations that accept most applicants.

Learning from gain and loss: Links to suicide risk.

Despite preliminary evidence that people with suicide attempt histories demonstrate deficits in processing feedback, no studies have examined the interrelations of learning from feedback and emotional state on suicide risk. This study examined the influence of suicide risk and negative emotions on learning accuracy and rates among individuals with a range of borderline personality features (N = 145). Participants completed a reinforcement learning task after neutral and negative emotion inductions. Results revealed interactions between suicide risk and emotion condition, with elevated risk linked to greater increases in loss learning rate (training phase models) and gain learning rate (test phase models) post-negative emotion induction. Emotion-dependent fluctuations in learning performance may be markers of decision-making that are associated with greater suicide risk. This line of work has the potential to identify the contexts that confer greater risk for suicidal behaviors.

sdtlu: An R package for the signal detection analysis of eyewitness lineup data.

In a standard eyewitness lineup scenario, a witness observes a culprit commit a crime and is later asked to identify the culprit from a set of faces, the lineup. Signal detection theory (SDT), a powerful modeling framework for analyzing data, has recently become a common way to analyze lineup data. The goal of this paper is to introduce a new R package, sdtlu (Signal Detection Theory - LineUp), that streamlines and automates the SDT analysis of lineup data. sdtlu provides functions to process lineup data, determine the best-fitting SDT parameters, compute model-based performance measures such as area under the curve (AUC) and diagnosticity, use bootstrapping to determine uncertainty intervals around these parameters and measures, and compare parameters across two different data sets. The package incorporates closed-form solutions for both simultaneous and sequential lineups that allow for model-based analyses without Monte Carlo simulation. Show-ups are also supported. The package can estimate the base-rate of lineups that include a guilty suspect when the guilt or innocence of each suspect in the data set is unknown, as in "real-world" lineups. The package can also produce a full set of graphs, including data and model-based ROC curves and the underlying SDT model.

Framing context effects with reference points.

Research on reference points highlights how alternatives outside the choice set can alter the perceived value of available alternatives, arguably framing the choice scenario. The present work utilizes reference points to study the effects of framing in preferential choice, using the similarity and attraction context effects as performance measures. We specifically test the predictions of Multialternative Decision by Sampling (MDbS; Noguchi & Stewart, 2018), a recent preferential choice model that can account for both reference points and context effects. In Experiment 1, consistent with predictions by MDbS, we find a standard similarity effect when no reference point is given that increases when both dimensions are framed negatively and decreases when both dimensions are framed positively. Contrary to predictions by MDbS, when the two dimensions are framed as tradeoffs, participants prefer whichever alternative performs best in the negatively framed dimension. Performance of MDbS was improved by the addition of a frame-based global attention allocation mechanism. Experiment 2 extends these results to a "by-dimension" presentation format in an attempt to bring participant behavior in line with MDbS assumptions. The empirical and modeling results replicated those of Experiment 1. Experiment 3 used the attraction effect to test the effects of framing when the best-performing alternative on each dimension was identical across target conditions, therefore reducing the potential effects of a global attention allocation mechanism. The effects of framing were indeed greatly reduced, and the performance of MDbS was markedly improved. The results extend framing to the context effects literature, provide new benchmarks for models and theories of context effects, and point to the need for a global attention mechanism.

Estimating the proportion of guilty suspects and posterior probability of guilt in lineups using signal-detection models.

BACKGROUND: The majority of eyewitness lineup studies are laboratory-based. How well the conclusions of these studies, including the relationship between confidence and accuracy, generalize to real-world police lineups is an open question. Signal detection theory (SDT) has emerged as a powerful framework for analyzing lineups that allows comparison of witnesses' memory accuracy under different types of identification procedures. Because the guilt or innocence of a real-world suspect is generally not known, however, it is further unknown precisely how the identification of a suspect should change our belief in their guilt. The probability of guilt after the suspect has been identified, the posterior probability of guilt (PPG), can only be meaningfully estimated if we know the proportion of lineups that include a guilty suspect, P(guilty). Recent work used SDT to estimate P(guilty) on a single empirical data set that shared an important property with real-world data; that is, no information about the guilt or innocence of the suspects was provided. Here we test the ability of the SDT model to recover P(guilty) on a wide range of pre-existing empirical data from more than 10,000 identification decisions. We then use simulations of the SDT model to determine the conditions under which the model succeeds and, where applicable, why it fails. RESULTS: For both empirical and simulated studies, the model was able to accurately estimate P(guilty) when the lineups were fair (the guilty and innocent suspects did not stand out) and identifications of both suspects and fillers occurred with a range of confidence levels. Simulations showed that the model can accurately recover P(guilty) given data that matches the model assumptions. The model failed to accurately estimate P(guilty) under conditions that violated its assumptions; for example, when the effective size of the lineup was reduced, either because the fillers were selected to be poor matches to the suspect or because the innocent suspect was more familiar than the guilty suspect. The model also underestimated P(guilty) when a weapon was shown. CONCLUSIONS: Depending on lineup quality, estimation of P(guilty) and, relatedly, PPG, from the SDT model can range from poor to excellent. These results highlight the need to carefully consider how the similarity relations between fillers and suspects influence identifications.

Failure to detect function word repetitions and omissions in reading: Are eye movements to blame?

We tested whether failure to notice repetitions of function words during reading (e.g., Amanda jumped off the the swing and landed on her feet.) is due to the eyes' tendency to skip one of the instances of the word. Eye movements were recorded during reading of sentences with repetitions of the word the or repetitions of a noun, after which readers were asked whether an error was present. A repeated the was detected on 46% of trials overall. On trials on which both instances of the were fixated, detection was still only 66%. A repeated noun was detected on 90% of trials, with no significant effect of eye movement patterns. Detecting an omitted the also proved difficult, with eye movement patterns having only a small effect. Readers frequently overlook function word errors even when their eye movements provide maximal opportunity for noticing such errors, but they notice content word repetitions regardless of eye movement patterns. We propose that readers overlook function word errors because they attribute the apparent error to noise in the eye movement control system.

The comparison process as an account of variation in the attraction, compromise, and similarity effects.

Context effects are changes in preference that occur when alternatives are added to a choice set. Models that account for context effects typically assume a within-dimension comparison process; however, the presentation format of a choice set can influence comparison strategies. The present study jointly tests the influence of presentation format on the attraction, compromise, and similarity effects in a within-subjects design. Participants completed a series of choices designed to elicit each of the three context effects, with either a by-alternative or by-dimension format. Whereas the by-alternative format elicited a standard similarity effect, but null attraction and reverse compromise effects, the by-dimension format elicited standard attraction and compromise effects, but a reverse similarity effect. These novel results are supported by a re-analysis of the eye-tracking data collected by Noguchi and Stewart (Cognition, 132(1), 44-56, 2014) and demonstrate that flexibility in the comparison process should be incorporated into theories of preferential choice.

Reversing the similarity effect: The effect of presentation format.

A context effect is a change in preference that occurs when alternatives are added to a choice set. Models of preferential choice that account for context effects largely assume a within-dimension comparison process. It has been shown, however, that the format in which a choice set is presented can influence comparison strategies. That is, a by-alternative or by-dimension grouping of the dimension values encourage within-alternative or within-dimension comparisons, respectively. For example, one classic context effect, the compromise effect, is strengthened by a by-dimension presentation format. Extrapolation from this result suggests that a second context effect, the similarity effect, will actually reverse when stimuli are presented in a by-dimension format. In the current study, we presented participants with a series of apartment choice sets designed to elicit the similarity effect, with either a by-alternative or by-dimension presentation format. Participants in the by-alternative condition demonstrated a standard similarity effect; however, participants in the by-dimension condition demonstrated a strong reverse similarity effect. The present data can be accounted for by Multialternative Decision Field Theory (MDFT) and the Multiattribute Linear Ballistic Accumulator (MLBA), but not Elimination by Aspects (EBA). Indeed, when some weak assumptions of within-dimension processes are met, MDFT and the MLBA predict the reverse similarity effect. These modeling results suggest that the similarity effect is governed by either forgetting and inhibition (MDFT), or attention to positive or negative differences (MLBA). These results demonstrate that flexibility in the comparison process needs to be incorporated into theories of preferential choice.

Impaired reward prediction error encoding and striatal-midbrain connectivity in depression.

Anhedonia (hyposensitivity to rewards) and negative bias (hypersensitivity to punishments) are core features of major depressive disorder (MDD), which could stem from abnormal reinforcement learning. Emerging evidence highlights blunted reward learning and reward prediction error (RPE) signaling in the striatum in MDD, although inconsistencies exist. Preclinical studies have clarified that ventral tegmental area (VTA) neurons encode RPE and habenular neurons encode punishment prediction error (PPE), which are then transmitted to the striatum and cortex to guide goal-directed behavior. However, few studies have probed striatal activation, and functional connectivity between VTA-striatum and VTA-habenula during reward and punishment learning respectively, in unmedicated MDD. To fill this gap, we acquired fMRI data from 25 unmedicated MDD and 26 healthy individuals during a monetary instrumental learning task and utilized a computational modeling approach to characterize underlying neural correlates of RPE and PPE. Relative to controls, MDD individuals showed impaired reward learning, blunted RPE signal in the striatum and overall reduced VTA-striatal connectivity to feedback. Critically, striatal RPE signal was increasingly blunted with more major depressive episodes (MDEs). No group differences emerged in PPE signals in the habenula and VTA or in connectivity between these regions. However, PPE signals in the habenula correlated positively with number of MDEs. These results highlight impaired reward learning, disrupted RPE signaling in the striatum (particularly among individuals with more lifetime MDEs) as well as reduced VTA-striatal connectivity in MDD. Collectively, these findings highlight reward-related learning deficits in MDD and their underlying pathophysiology.

Frontostriatal and Dopamine Markers of Individual Differences in Reinforcement Learning: A Multi-modal Investigation.

Prior studies have shown that dopamine (DA) functioning in frontostriatal circuits supports reinforcement learning (RL), as phasic DA activity in ventral striatum signals unexpected reward and may drive coordinated activity of striatal and orbitofrontal regions that support updating of action plans. However, the nature of DA functioning in RL is complex, in particular regarding the role of DA clearance in RL behavior. Here, in a multi-modal neuroimaging study with healthy adults, we took an individual differences approach to the examination of RL behavior and DA clearance mechanisms in frontostriatal learning networks. We predicted that better RL would be associated with decreased striatal DA transporter (DAT) availability and increased intrinsic functional connectivity among DA-rich frontostriatal regions. In support of these predictions, individual differences in RL behavior were related to DAT binding potential in ventral striatum and resting-state functional connectivity between ventral striatum and orbitofrontal cortex. Critically, DAT binding potential had an indirect effect on reinforcement learning behavior through frontostriatal connectivity, suggesting potential causal relationships across levels of neurocognitive functioning. These data suggest that individual differences in DA clearance and frontostriatal coordination may serve as markers for RL, and suggest directions for research on psychopathologies characterized by altered RL.

Multi-attribute, multi-alternative models of choice: Choice, reaction time, and process tracing.

The first aim of this research is to compare computational models of multi-alternative, multi-attribute choice when attribute values are explicit. The choice predictions of utility (standard random utility & weighted valuation), heuristic (elimination-by-aspects, lexicographic, & maximum attribute value), and dynamic (multi-alternative decision field theory, MDFT, & a version of the multi-attribute linear ballistic accumulator, MLBA) models are contrasted on both preferential and risky choice data. Using both maximum likelihood and cross-validation fit measures on choice data, the utility and dynamic models are preferred over the heuristic models for risky choice, with a slight overall advantage for the MLBA for preferential choice. The response time predictions of these models (except the MDFT) are then tested. Although the MLBA accurately predicts response time distributions, it only weakly accounts for stimulus-level differences. The other models completely fail to account for stimulus-level differences. Process tracing measures, i.e., eye and mouse tracking, were also collected. None of the qualitative predictions of the models are completely supported by that data. These results suggest that the models may not appropriately represent the interaction of attention and preference formation. To overcome this potential shortcoming, the second aim of this research is to test preference-formation assumptions, independently of attention, by developing the models of attentional sampling (MAS) model family which incorporates the empirical gaze patterns into a sequential sampling framework. An MAS variant that includes attribute values, but only updates the currently viewed alternative and does not contrast values across alternatives, performs well in both experiments. Overall, the results support the dynamic models, but point to the need to incorporate a framework that more accurately reflects the relationship between attention and the preference-formation process.

Beliefs and Bayesian reasoning.

We examine whether judgments of posterior probabilities in Bayesian reasoning problems are affected by reasoners' beliefs about corresponding real-world probabilities. In an internet-based task, participants were asked to determine the probability that a hypothesis is true (posterior probability, e.g., a person has a disease, given a positive medical test) based on relevant probabilities (e.g., that any person has the disease and the true and false positive rates of the test). We varied whether the correct posterior probability was close to, or far from, independent intuitive estimates of the corresponding 'real-world' probability. Responses were substantially closer to the correct posterior when this value was close to the intuitive estimate. A model in which the response is a weighted sum of the intuitive estimate and an additive combination of the probabilities provides an excellent account of the results.

Punishment Learning in U.S. Veterans With Posttraumatic Stress Disorder.

Learning processes have been implicated in the development and course of posttraumatic stress disorder (PTSD); however, little is currently known about punishment-based learning in PTSD. The current study investigated impairments in punishment-based learning in U.S. veterans. We expected that veterans with PTSD would demonstrate greater punishment-based learning compared to a non-PTSD control group. We compared a PTSD group with and without co-occurring depression (n = 27) to a control group (with and without trauma exposure) without PTSD or depression (n = 29). Participants completed a computerized probabilistic punishment-based learning task. Compared to the non-PTSD control group, veterans with PTSD showed significantly greater punishment-based learning. Specifically, there was a significant Block × Group interaction, F(1, 54) = 4.12, p = .047, η(2) = .07. Veterans with PTSD demonstrated greater change in response bias for responding toward a less frequently punished stimulus across blocks. The observed hypersensitivity to punishment in individuals with PTSD may contribute to avoidant responses that are not specific to trauma cues.

The Rational Adolescent: Strategic Information Processing during Decision Making Revealed by Eye Tracking.

Adolescence is often viewed as a time of irrational, risky decision-making - despite adolescents' competence in other cognitive domains. In this study, we examined the strategies used by adolescents (N=30) and young adults (N=47) to resolve complex, multi-outcome economic gambles. Compared to adults, adolescents were more likely to make conservative, loss-minimizing choices consistent with economic models. Eye-tracking data showed that prior to decisions, adolescents acquired more information in a more thorough manner; that is, they engaged in a more analytic processing strategy indicative of trade-offs between decision variables. In contrast, young adults' decisions were more consistent with heuristics that simplified the decision problem, at the expense of analytic precision. Collectively, these results demonstrate a counter-intuitive developmental transition in economic decision making: adolescents' decisions are more consistent with rational-choice models, while young adults more readily engage task-appropriate heuristics.

The Dilution Effect and Information Integration in Perceptual Decision Making.

In cognitive science there is a seeming paradox: On the one hand, studies of human judgment and decision making have repeatedly shown that people systematically violate optimal behavior when integrating information from multiple sources. On the other hand, optimal models, often Bayesian, have been successful at accounting for information integration in fields such as categorization, memory, and perception. This apparent conflict could be due, in part, to different materials and designs that lead to differences in the nature of processing. Stimuli that require controlled integration of information, such as the quantitative or linguistic information (commonly found in judgment studies), may lead to suboptimal performance. In contrast, perceptual stimuli may lend themselves to automatic processing, resulting in integration that is closer to optimal. We tested this hypothesis with an experiment in which participants categorized faces based on resemblance to a family patriarch. The amount of evidence contained in the top and bottom halves of each test face was independently manipulated. These data allow us to investigate a canonical example of sub-optimal information integration from the judgment and decision making literature, the dilution effect. Splitting the top and bottom halves of a face, a manipulation meant to encourage controlled integration of information, produced farther from optimal behavior and larger dilution effects. The Multi-component Information Accumulation model, a hybrid optimal/averaging model of information integration, successfully accounts for key accuracy, response time, and dilution effects.

Within-subject consistency and between-subject variability in Bayesian reasoning strategies.

It is well known that people tend to perform poorly when asked to determine a posterior probability on the basis of a base rate, true positive rate, and false positive rate. The present experiments assessed the extent to which individual participants nevertheless adopt consistent strategies in these Bayesian reasoning problems, and investigated the nature of these strategies. In two experiments, one laboratory-based and one internet-based, each participant completed 36 problems with factorially manipulated probabilities. Many participants applied consistent strategies involving use of only one of the three probabilities provided in the problem, or additive combination of two of the probabilities. There was, however, substantial variability across participants in which probabilities were taken into account. In the laboratory experiment, participants' eye movements were tracked as they read the problems. There was evidence of a relationship between information use and attention to a source of information. Participants' self-assessments of their performance, however, revealed little confidence that the strategies they applied were actually correct. These results suggest that the hypothesis of base rate neglect actually underestimates people's difficulty with Bayesian reasoning, but also suggest that participants are aware of their ignorance.

The effect of emotional state on visual detection: A signal detection analysis.

This research explores the effect of emotional states on visual detection. Previous research has shown that emotional states characterized by an intolerance of uncertainty, such as anxiety, can affect performance on visual detection tasks. It is unclear, however, to what extent these effects are a result of increased perceptual ability, a decisional bias, or both. The present study used signal detection theory to determine whether uncertain emotional states affect perceptual discriminability and/or decisional bias. In 2 experiments, an anxious, angry, or calm emotional state was induced, and participants were asked to identify which of a series of noisy images contained an embedded target image. The target images were either faces or houses. Emotional state had no effect on decisional bias for either target, but the ability to detect a face was higher for anxious participants. No effect on discriminability was found for houses. These results suggest that emotional state can change perceptual discriminability, but that this change may be limited to certain stimulus classes.

Accidental scratch--or a sign of self-cutting?

Having a high index of suspicion is key to identifying adolescents who cut themselves. The approach described here can help you properly evaluate these patients and get them the help they need.