Comparing human evaluations of eyewitness statements to a machine learning classifier under pristine and suboptimal lineup administration procedures.

Recent work highlights the ability of verbal machine learning classifiers to distinguish between accurate and inaccurate recognition memory decisions (Dobbins, 2022; Dobbins & Kantner, 2019; Seale-Carlisle, Grabman, & Dodson, 2022). Given the surge of interest in these modeling techniques, there is an urgent need to investigate verbal classifiers' limitations - particularly in applied contexts such as when police collect eyewitness's confidence statements. We find that confirmatory feedback (e.g., "This study now has a total of 87 participants, 84 of them made the same decision as you!") weakens the relationship between identification accuracy and verbal classifier scores to a similar degree as mock witnesses' numeric confidence judgments (Experiment 1). Crucially, for the first time, we compare the discriminative value of verbal classifier scores to the ratings of human evaluators who assessed the identical verbal confidence statements (Experiment 2). Our results suggest that human evaluators outperform the classifier when mock witnesses received no feedback; however, the classifier matches (or exceeds) the performance of human evaluators when mock witnesses received confirmatory feedback. Providing lineup information to human evaluators resulted in a worse ability to distinguish between correct and filler identifications, suggesting that this particular information may encourage the use of inappropriate heuristics when rendering accuracy judgments. Overall, these results suggest that the utility of verbal classifiers may be enhanced when contextual effects (e.g., lineup presence) impair human estimates of others' performance, but that translating witnesses' statements into classifier scores will not fix the problems of an improperly conducted lineup procedure.

Detecting recollection: Human evaluators can successfully assess the veracity of others' memories.

Humans have the highly adaptive ability to learn from others' memories. However, because memories are prone to errors, in order for others' memories to be a valuable source of information, we need to assess their veracity. Previous studies have shown that linguistic information conveyed in self-reported justifications can be used to train a machine-learner to distinguish true from false memories. But can humans also perform this task, and if so, do they do so in the same way the machine-learner does? Participants were presented with justifications corresponding to Hits and False Alarms and were asked to directly assess whether the witness's recognition was correct or incorrect. In addition, participants assessed justifications' recollective qualities: their vividness, specificity, and the degree of confidence they conveyed. Results show that human evaluators can discriminate Hits from False Alarms above chance levels, based on the justifications provided per item. Their performance was on par with the machine learner. Furthermore, through assessment of the perceived recollective qualities of justifications, participants were able to glean more information from the justifications than they used in their own direct decisions and than the machine learner did.

The limited memory of value following value directed encoding.

Items associated with higher values during encoding are later recognized and recalled better than are lower valued items. During recall paradigms, these value directed encoding (VDE) effects heavily depend upon learned strategies acquired during repeated testing with earnings feedback. However, because VDE effects also occur in single test recognition designs, precluding such learning, it has been suggested that high value may automatically induce good encoding. We tested this by manipulating encoding instructions (Experiments 1a and 1b) and manipulating concurrent levels of processing (LOP) requirements during encoding (Experiment 2a and 2b). Two main findings emerged. First, subject initiated strategies played a dominant role in VDE effects with little evidence for automaticity. This was demonstrated in Experiment 1 by a more than three-fold increase in the VDE recognition effect when instructions specifically encouraged selective elaboration of high-value items. It was also shown by the complete elimination of VDE recognition effects in Experiment 2 when LOP tasks were concurrently performed during encoding. Critically, the blocking of VDE effects occurred even though a catch trial procedure verified that value was being processed during encoding and remained even when subjects had unlimited time to process the materials during encoding. Second, the data showed, for the first time, that when subjects attempted to specify the value of recognized items, they heavily depended upon a recognition heuristic in which increases in recognition strength, even when nondiagnostic, were inferred to reflect high encoding value. The tendency for subjects to conflate recognition strength and value may have important implications for behavioral economics.

Recognition receiver operating characteristic asymmetry: Increased noise or information?

The recognition memory receiver operating characteristic (ROC) is typically asymmetric with a characteristic elevation of the left-hand portion. Whereas the unequal variance signal detection model (uvsd) assumes the asymmetry results because old item evidence is noisier than new item evidence, the dual process signal detection model (dpsd) assumes it results because old items convey more useful information than new items. To test these assumptions, the models were fit to old/new recognition data and their evidence parameters were used to predict performance on a novelty, three-alternative forced-choice (N3AFC) recognition task. Critically, under the uvsd model, increased old item variance (sigma) predicts poorer N3AFC performance, whereas under the dpsd model, increased recollection rates (Ro) predict better N3AFC performance. Hence, the asymmetry parameters of the two models make divergent predictions. In two experiments the dpsd model's predictions were supported, whereas the uvsd model yielded unpredicted (from that model's perspective) patterns. Through simulation it was also shown that the dpsd model predicted the uvsd model's mispredictions, which resulted because increases in old item noise markedly depress the upper portion of the ROC. Overall, the data demonstrate that increasing ROC asymmetry is not a function of increasingly noisier target evidence, but instead increasingly informative target evidence. These findings invalidate the uvsd model, which heretofore has been primarily supported by its post-hoc fitting ability, not its construct validity. (PsycInfo Database Record (c) 2023 APA, all rights reserved).

Recognition language classifiers demonstrate far transfer of learning.

Machine learners trained on verbal justifications of recognition decisions reliably predict recognition accuracy. If these recognition language classifiers are recollection sensitive, they should generalize beyond the single-item, verbal recognition paradigms upon which they were trained. To test this, three classifiers were trained to distinguish justification language in three different single-item verbal recognition paradigms, learning to distinguish the language justifying hits from false alarms, high from medium confidence hits, and remember from know judgements. The resulting classifiers were then used to predictively score language justifying correct versus incorrect eyewitness lineup selections constituting a test of far transfer because of the differences in materials (faces vs. words), subject populations (undergraduate vs. online), testing procedures (single vs. multiple items), and test lengths (12 vs. hundreds of targets per subject) among others. All three classifiers reliably predicted eyewitness accuracy despite these differences. Additionally, mixed modeling demonstrated that the classifiers demonstrated both convergent and divergent validity with respect to the recollection sensitivity hypothesis. That is, they strongly predicted the accuracy of eyewitness selections (i.e., hits vs. false alarms) but failed to predict the accuracy of eyewitness rejections (i.e., correct rejections vs. misses). Moreover, one classifier was shown to predict eyewitness confidence despite being trained on a design devoid of all metacognitive judgments. These findings support the hypothesis that recognition language classifiers detect recollection conveyed in the language subjects use to justify their memory decisions.

Critical tests of the continuous dual-process model of recognition.

Dual process recognition models assume recognition depends upon context recollection and/or item familiarity. While most assume recollection is more highly valued or weighted than familiarity during judgment, we tested a continuous dual process (CDP) model that instead assumes recollection and familiarity are equally weighted during recognition judgment. Experiments 1a and 1b used a joint rating scale in which each probe was rated for recollection and familiarity strength, which were then used to predict overall recognition confidence. In both, recollection dominated familiarity such that familiarity ratings were only predictive of confidence when recollection ratings were relatively weaker. In contrast, when recollection ratings were stronger, familiarity made no contribution to recognition confidence. Experiment 2 used a different, bifurcated rating scale previously demonstrating that strong ratings of familiarity can lead to better recognition yet worse contextual source memory than weak ratings of recollection. However, the current study failed to find this dissociation, instead demonstrating that weak recollection ratings were as or more accurate than the strongest familiarity ratings in both recognition and source memory. Replacing the CDP model equal weighting decision rule with one incorporating a strong relative preference for recollection over familiarity yielded simulation data more consistent with the empirical data and is more optimal if recollection is in fact more diagnostic of recognition than familiarity. Overall, these findings suggest that observers have a strong preference for relying on recollection over familiarity during recognition, presumably because it better situates the probe within a specific episode.

Attempted recall of biographical information influences face attractiveness.

We examined affective consequences arising from the kinds of memory retrieval failures that often accompany social interaction. To do so, we measured the influence of cued-recall outcomes for biographical information on the rated attractiveness of faces. The data demonstrate that retrieval of names (Experiment 1a) and professions (Experiment 1b) increases the rated attractiveness of target faces relative to faces that failed to produce recall of associative information. This was predicted by a confirmation of search (COS) model originally developed on verbal memoranda, which assumes that confirmation bias during memory search leads to affective consequences depending upon retrieval's success or failure. The current study extends this model, showing that evaluative judgments of individuals are in part contingent upon the memory retrieval skills of their assessors. We conclude by discussing potential extensions of the COS paradigm to the measurement of implicit attitudes and special populations.

Pupil dilation signals recognition salience.

Stimuli that are recognized from a prior encounter elicit larger pupil dilations than those that are not. This study tests an account of this recognition dilation response (RDR) that assumes newly encountered recognition signals, like new percepts, elicit attentional orienting responses. Because orienting is moderated by motivational significance and expectation, the RDR was tested for these properties; manipulating incentives for "old" versus "new" judgments, and isolating the effects of runs of "old" versus "new" decisions on the subsequent RDR, in two experiments. Whereas incentivizing "new" decisions largely eliminated the RDR, incentivizing "old" decisions amplified it. Moreover, the RDR was prominent following runs of "new" decisions, yet minimal following runs of "old" decisions. Thus, the pupil dilates more as recognition memory becomes more valuable and/or unexpected. This recognition-orienting response was functionally separate from an additional, late dilation linked to feedback expectancy. Thus, the pupil separately signals the salience of recognition evidence, and the expectation of post-decision feedback.

Pupil dilation during memory encoding reflects time pressure rather than depth of processing.

Under cognitive load theory, time pressure/urgency-induced arousal is a major contributor to pupil dilation during cognition. However, pupillometric encoding studies have failed to consider the possible role of time pressure/urgency effects, instead often assuming that encoding dilations directly reflect encoding strength. To isolate possible encoding strength and time pressure effects, we manipulated levels of processing (deep vs. shallow) and response deadlines (speeded vs. unspeeded) during verbal recognition memory encoding. Rather than reflecting encoding strength, pupil dilation signaled time pressure and decision urgency, as indicated by four findings. First, dilation was greater for speeded than unspeeded trials, yet later recognition was similar. Second, within every combination of levels of processing and response deadline, slower individual decisions yielded increased dilations compared to quicker decisions. Third, even when encoding dilations during deep and shallow tasks were closely matched, later recognition remained markedly higher for the deep trials. Finally, within every combination of levels of processing and response deadline, dilation levels were similar for items subsequently recognized (hits) versus subsequently forgotten (misses). Taken together, our results support a time pressure/decision urgency account: instead of directly reflecting encoding efficacy, pupillary dilation mainly reflects the arousal induced by an increasingly urgent demand to process information. In the discussion section, we consider other possible paradigms during which arousal-based dilations may forecast subsequent memory outcomes, unlike here. Nonetheless, we emphasize that even in these situations, the proximal cause of dilation would be the time pressure or urgency of information processing. (PsycInfo Database Record (c) 2021 APA, all rights reserved).

Source retrieval under cueing: Dissociated effects on accuracy versus confidence.

During recognition memory decisions, external hints or cues alter the accuracy and confidence of correct rejections (valid > uncued > invalid). In contrast, although hits show analogous accuracy effects, hit confidence remains largely unaffected by cue validity. Prior research suggested this confidence validity dissociation (CVD) may depend upon the presence of recollection during hits. If so, confidence during other recollection dependent tasks such as source memory should show the same insensitivity to cue validity, despite clear changes in accuracy. We tested this in 5 source-memory experiments manipulating encoding location (left or right, Experiments 1, 2, and 5) or study list (first or second, Experiments 3 and 4). At test, memoranda were preceded by predictive arrow cues (75% valid/25% invalid) indicating the likely prior location or list of the source memory probe. Cue validity affected accuracy in all 5 Experiments. Nonetheless, mean confidence for both correct and incorrect source judgments was unaffected by cue validity. These data demonstrate that the subjective confidence of source attributions can become untethered from accuracy when external influences are present. Analyses of previously published recognition data elucidated this finding by showing that confidence is not affected by cue validity for items recognized as "old" regardless of accuracy (i.e., hits and false alarms). However, confidence is affected by cue validity for items judged "new" regardless of accuracy (i.e., correct rejections and misses). We suggest this dissociation depends upon the retrieval schemas and decision heuristics that observers use when concluding items arise from candidate experiences held in mind. (PsycInfo Database Record (c) 2020 APA, all rights reserved).

The language of accurate recognition memory.

The natural language accompanying recognition judgments is a largely untapped though potentially rich source of information about the kinds of processing that may support recognition memory. The current report illustrates a series of methods using machine learning and receiver operating characteristics (ROCs) to examine whether the language participants use to justify their 'old' and 'new' recognition decisions (viz., memory justifications) predicts accuracy. The findings demonstrate that the natural language of observers conveys the accuracy of 'old' (hits versus false alarms) but not 'new' (misses versus correct rejections) decisions. The classifier trained on this language was considerably more predictive of accuracy than the initial speed of the decisions, generalized to the justification language of two independent experiments using different procedures, and appeared sensitive to the presence versus absence of recollective experiences in the observer's reports. We conclude by considering extensions of the approach to several basic and applied areas, and, more broadly, to identifying the explicit bases (if any) of classification decisions in general.

Mere exposure effect(s) in the context of explicit memory search.

Prior stimulus exposure often increases later ratings of positive affect (e.g., pleasantness ratings). This phenomenon - the mere exposure effect (MEE) - appears robust following subliminal and incidental exposures. However, its expression in the context of explicit memory judgment remains unclear. In four studies, memory and pleasantness ratings were combined to investigate how memory conclusions (e.g., "studied" or "unstudied") might moderate exposure effects. Experiment 1 examined basic recognition, Experiment 2 manipulated incentives for recognition decisions, and Experiments 3 and 4 examined source memory and paired-associate recall respectively. In general, items endorsed as recognized, attributed to the queried source, or accompanied by successful recall of a paired associate (i.e., confirmations) were rated as more pleasant than baseline norms. As important, items endorsed as unstudied, rejected as originating from a queried source, or failing to yield successful recall of a paired associated were rated as less pleasant than baseline norms. This suggests that it is the outcome of memory search that alters pleasantness ratings in the context of retrieval demands, and we discuss how this confirmation of search (COS) hypothesis accounts for current and prior findings.

Partitioning the sources of recognition confidence: The role of individual differences.

Confidence ratings during old-new recognition memory tasks are thought to index the strength of memory evidence elicited by test probes. However, various subject-specific factors may also influence reported confidence, including perceived self-efficacy and idiosyncratic interpretation of the confidence scale. To measure the contribution of subject-specific variables to confidence ratings, we performed regression analyses on extant data from three recognition experiments encompassing procedural variations in encoding and stimuli, testing the degree to which the person making the judgment (the "subject" factor), versus whether or not the judgment is accurate, influences reported confidence. Overall, confidence was less linked to changes in accuracy, for "new" than for "old" judgments. Critically, the subject factor was at least as predictive of rated confidence as accuracy for "old" judgments, whereas for "new" judgments the subject factor was substantially more predictive of confidence than accuracy. These results suggest that measured confidence is largely a function of who is making the rating, especially when items are identified as "new." This suggests that the utility of confidence in predicting memory accuracy will be limited when stable estimates of subject contributions are unavailable, such as when each subject provides one or a few responses.

Confidence carryover during interleaved memory and perception judgments.

Recognition memory tests typically consist of randomly intermixed studied and nonstudied items that subjects classify as old or new, often while indicating their confidence in these classifications. Under most decision theories, confidence ratings index an item's memory strength-the extent to which it elicits evidence of prior occurrence. Because the test probes are randomly ordered, these theories predict that confidence judgments should be sequentially independent: confidence on trial n should not predict confidence on n + 1. However, analysis of two extant data sets demonstrated reliable serial correlations in recognition memory confidence (confidence carryover). In a new experiment, we examined the domain specificity of confidence carryover by serially interleaving recognition and perceptual classification judgments. Analysis revealed domain-general and domain-specific confidence carryover effects: The confidence of a current recognition judgment was shown to reflect both the confidence of an immediately preceding perceptual gender judgment (domain-general carryover at Lag 1) and also the confidence of the recognition judgment prior to that (domain-specific carryover at Lag 2). Moreover, the domain-specific effect was sensitive to response consistency: Confidence carryover was highest when old-new classifications repeated across trials. Whereas the domain-general effect may reflect metacognitive monitoring of internal factors such as alertness, the domain-specific effect was easily simulated by assuming that evidence within domains is "sticky," such that current memory or perceptual evidence is pulled toward prior evidence representations.

Revising recognition judgments during noisy recognition evidence accumulation: The dynamics of losses versus gains.

Outside the laboratory, we sometimes revise our recognition judgments of others-realizing, for example, that we have accidentally failed to greet an acquaintance we just passed in the hallway. These recognition reversals have rarely been studied. Here, using a basic noisy-accumulation framework, we simulated recognition response reversals in which initial speeded recognition judgments were followed by an opportunity to revise the initial judgment. The simulation predictions were compared to empirical data from two experiments in which we gave participants the opportunity to revise each of their initial speeded recognition judgments. The speeded old-new responses were restricted to either 300-800 ms (Exp. 1) or 200-600 ms (Exp. 2) after each probe's onset, and the second response was self-paced in both experiments. The noisy-accumulation framework correctly anticipated three findings. First, gain rates (incorrect followed by correct responses) always exceeded loss rates (correct followed by incorrect responses). Second, despite being corrective, the raw gain rates exhibited a modest negative correlation with overall recognition skill. Third, when gain rates were conditioned on the opportunity to correct an initial error (conditional gain rate), they were then positively correlated with recognition skill but were less diagnostic than the conditional loss rates. Thus, the mechanics of noisy accumulation naturally predict that skilled recognizers will demonstrate infrequent corrective behavior but a high probability of correction, should an initial error occur.

Ignoring memory hints: The stubborn influence of environmental cues on recognition memory.

Recognition judgments can benefit from the use of environmental cues that signal the general likelihood of encountering familiar versus unfamiliar stimuli. While incorporating such cues is often adaptive, there are circumstances (e.g., eyewitness testimony) in which observers should fully ignore environmental cues in order to preserve memory report fidelity. The current studies used the explicit memory cueing paradigm to examine whether participants could intentionally ignore reliable environmental cues when instructed. Three experiments demonstrated that participants could volitionally dampen the directional influence of environmental cues on their recognition judgments (i.e., whether influenced to respond "old" or "new") but did not fully eliminate their influence. Although monetary incentives diminished the mean influence of cues on responses rates, finer grained individual differences analysis, as well as confidence and RTs analyses, demonstrated that participants were still systematically influenced. These results demonstrate that environmental cues presented at test remain a potent influence on recognition decisions and subjective confidence even when ostensibly ignored. (PsycINFO Database Record

Characterizing adult age differences in the initiation and organization of retrieval: A further investigation of retrieval dynamics in dual-list free recall.

In a recent experiment using dual-list free recall of unrelated word lists, C. N. Wahlheim and M. J. Huff (2015) found that relative to younger adults, older adults showed: (a) impaired recollection of temporal context, (b) a broader pattern of retrieval initiation when recalling from 2 lists, and (c) more intrusions when selectively recalling from 1 of 2 lists. These findings showed older adults' impaired ability to use controlled retrieval to avoid proactive and retroactive interference. In the present investigation, 3 studies examined whether differences in retrieval initiation patterns were unique to aging and whether they were governed by the control mechanisms that underlie individuals' susceptibility to intrusions. In Study 1, we conducted additional analyses of Wahlheim and Huff's data and found that older adults' broader retrieval initiation when recalling 2 lists was a unique effect of age that was not redundant with intrusions made when recalling from individual lists. In Study 2, we replicated these age differences in a dual-list paradigm with semantically associated lists. In Study 3, we found that older adults' broader retrieval initiation generalized when they were given twice the encoding time compared with Study 2. Analyses of transitions between recalls in Studies 2 and 3 showed that older adults used temporal associations less than younger adults, but both groups made similar use of semantic associations. Overall, these findings demonstrate adult age differences in the controlled retrieval of temporal context in hierarchically structured events. (PsycINFO Database Record