A guide to measuring expert performance in forensic pattern matching.

Decisions in forensic science are often binary. A firearms expert must decide whether a bullet was fired from a particular gun or not. A face comparison expert must decide whether a photograph matches a suspect or not. A fingerprint examiner must decide whether a crime scene fingerprint belongs to a suspect or not. Researchers who study these decisions have therefore quantified expert performance using measurement models derived largely from signal detection theory. Here we demonstrate that the design and measurement choices researchers make can have a dramatic effect on the conclusions drawn about the performance of forensic examiners. We introduce several performance models - proportion correct, diagnosticity ratio, and parametric and non-parametric signal detection measures - and apply them to forensic decisions. We use data from expert and novice fingerprint comparison decisions along with a resampling method to demonstrate how experimental results can change as a function of the task, case materials, and measurement model chosen. We also graphically show how response bias, prevalence, inconclusive responses, floor and ceiling effects, case sampling, and number of trials might affect one's interpretation of expert performance in forensics. Finally, we discuss several considerations for experimental and diagnostic accuracy studies: (1) include an equal number of same-source and different-source trials; (2) record inconclusive responses separately from forced choices; (3) include a control comparison group; (4) counterbalance or randomly sample trials for each participant; and (5) present as many trials to participants as is practical.

The illusion of insight: detailed warnings reduce but do not prevent false "Aha!" moments.

False "Aha!" moments can be elicited experimentally using the False Insight Anagram Task (FIAT), which combines semantic priming and visual similarity manipulations to lead participants into having "Aha!" moments for incorrect anagram solutions. In a preregistered experiment (N = 255), we tested whether warning participants and explaining to them exactly how they were being deceived, would reduce their susceptibility to false insights. We found that simple warnings did not reduce the incidence of false insights. On the other hand, participants who were given a detailed explanation of the methods used to deceive them experienced a small reduction in false insights compared to participants given no warning at all. Our findings suggest that the FIAT elicits a robust false insight effect that is hard to overcome, demonstrating the persuasive nature of false insights when the conditions are ripe for them.

Thinking style and psychosis proneness do not predict false insights.

The FIAT paradigm (Grimmer et al., 2021) is a novel method of eliciting 'Aha' moments for incorrect solutions to anagrams in the laboratory, i.e. false insights. There exist many documented reports of psychotic symptoms accompanying strong feelings of 'Aha!' (Feyaerts, Henriksen, Vanheule, Myin-Germeys, & Sass, 2021; Mishara, 2010; Tulver, Kaup, Laukkonen, & Aru, 2021), suggesting that the newly developed FIAT could reveal whether people who have more false insights are more prone to psychosis and delusional belief. To test this possibility, we recruited 200 participants to take an adapted version of the FIAT and complete measures of thinking style and psychosis proneness. We found no association between experimentally induced false insights and measures of Schizotypy, Need for Cognition, Jumping to Conclusions, Aberrant Salience, Faith in Intuition, or the Cognitive Reflection Task. We conclude that experiencing false insights might not be constrained to any particular type of person, but rather, may arise for anyone under the right circumstances.

Getting a grip on insight: real-time and embodied Aha experiences predict correct solutions.

Insight experiences are sudden, persuasive, and can accompany valuable new ideas in science and art. In this preregistered experiment, we aim to validate a novel visceral and continuous measure of insight problem solving and to test whether real-time and embodied feelings of insight can predict correct solutions. We report several findings. Consistent with recent work, we find a strong positive relationship between Aha moments and accuracy for problems that demand implicit processing. We also found that the intensity of the insight experience further predicted the accuracy of solutions and participants naturally embodied the intensity of their insight experiences by squeezing the dynamometer more tightly. Intriguingly, this unintentional embodiment further predicted the accuracy of solutions. We suggest that the dynamometer complements previous measures by (1) simultaneously capturing both process and feeling in real-time, (2) highlights the value of measuring Aha moments on a continuum of intensity, and (3) firmly establishes that the impulsive feeling of Aha can carry information about the veracity of an idea. We discuss the findings in light of a recent theoretical account of how feelings of insight may act as a heuristic to select ideas from the stream of consciousness.

A visual familiarity account of evidence for orthographic processing in pigeons (Columbia livia): a reply to Scarf, Corballis, Güntürkün, and Colombo (2017).

Scarf et al. (Proc Natl Acad Sci 113(40):11272-11276, 2016) demonstrated that pigeons, as with baboons (Grainger et al. in Science 336(6078):245-248, 2012; Ziegler in Psychol Sci. https://doi.org/10.1177/0956797612474322 , 2013), can be trained to display several behavioural hallmarks of human orthographic processing. But, Vokey and Jamieson (Psychol Sci 25(4):991-996, 2014) demonstrated that a standard, autoassociative neural network model of memory applied to pixel maps of the words and nonwords reproduces all of those results. In a subsequent report, Scarf et al. (Anim Cognit 20(5):999-1002, 2017) demonstrated that pigeons can reproduce one more marker of human orthographic processing: the ability to discriminate visually presented four-letter words from their mirror-reversed counterparts (e.g. "LEFT" vs. " "). The current report shows that the model of Vokey and Jamieson (2014) reproduces the results of Scarf et al. (2017) and reinforces the original argument: the recent results thought to support a conclusion of orthographic processing in pigeons and baboons are consistent with but do not force that conclusion.

Can observing a Necker cube make you more insightful?

It is a compelling idea that an image as simple as a Necker cube, or a duck-rabbit illusion, can reveal something about a person's creativity. Surprisingly, there are now multiple examples showing that people who are better at discovering 'hidden' images in a picture, are also better at solving some creative problems. Although this idea goes back at least a century, little is known about how these two tasks-that seem so different on the surface-are related to each other. At least some forms of creativity (and indeed scientific discoveries) may require that we change our perspectives in order to discover a novel solution to a problem. It's possible that such problems involve a similar cognitive process, and perhaps the same cognitive capacities, as switching perspectives in an ambiguous image. We begin by replicating previous work, and also show metacognitive similarities between the sudden appearance of hidden images in consciousness, and the sudden appearance of solutions to verbal insight problems. We then show that simply observing a Necker cube can improve subsequent creative problem-solving and lead to more self-reported insights. We speculate that these results may in part be explained by Conflict Monitoring Theory.

Thinking forensics: Cognitive science for forensic practitioners.

Human factors and their implications for forensic science have attracted increasing levels of interest across criminal justice communities in recent years. Initial interest centred on cognitive biases, but has since expanded such that knowledge from psychology and cognitive science is slowly infiltrating forensic practices more broadly. This article highlights a series of important findings and insights of relevance to forensic practitioners. These include research on human perception, memory, context information, expertise, decision-making, communication, experience, verification, confidence, and feedback. The aim of this article is to sensitise forensic practitioners (and lawyers and judges) to a range of potentially significant issues, and encourage them to engage with research in these domains so that they may adapt procedures to improve performance, mitigate risks and reduce errors. Doing so will reduce the divide between forensic practitioners and research scientists as well as improve the value and utility of forensic science evidence.

Putting bias into context: The role of familiarity in identification.

Previous demonstrations of context effects in the forensic comparison sciences have shown that the number of "match" responses a person makes can be swayed by case information. Less clear is whether these effects are a result of changes in accuracy (e.g., discrimination ability), a shift in response bias (e.g., tendency to say "match" or "no match") or a mix of the 2. We present a series of experiments where we use a signal detection framework to examine the effects of case information (separately) on forensic comparison accuracy and response bias. We also explore the role of familiarity as 1 potential mechanism for case information to sway accuracy. In Experiment 1, case information about crimes perceived to be more severe swayed people to say "match" more, but had little bearing on their ability to discriminate matching and nonmatching fingerprint pairs. In Experiment 2, case information did affect accuracy when it was familiar (i.e., if a previous similar case was associated with a "match" then people were more likely to also rate the current case as a "match," even though it was not). Even when we blinded people to all extrinsic case information in Experiment 3, accuracy was significantly affected by the familiarity of the fingerprints. These results demonstrate that contextual factors can have different (and independent) influences on accuracy and response bias and that even subtle information can affect accuracy if it is sufficiently similar to the case or trace at hand.

Human matching performance of genuine crime scene latent fingerprints.

There has been very little research into the nature and development of fingerprint matching expertise. Here we present the results of an experiment testing the claimed matching expertise of fingerprint examiners. Expert (n = 37), intermediate trainee (n = 8), new trainee (n = 9), and novice (n = 37) participants performed a fingerprint discrimination task involving genuine crime scene latent fingerprints, their matches, and highly similar distractors, in a signal detection paradigm. Results show that qualified, court-practicing fingerprint experts were exceedingly accurate compared with novices. Experts showed a conservative response bias, tending to err on the side of caution by making more errors of the sort that could allow a guilty person to escape detection than errors of the sort that could falsely incriminate an innocent person. The superior performance of experts was not simply a function of their ability to match prints, per se, but a result of their ability to identify the highly similar, but nonmatching fingerprints as such. Comparing these results with previous experiments, experts were even more conservative in their decision making when dealing with these genuine crime scene prints than when dealing with simulated crime scene prints, and this conservatism made them relatively less accurate overall. Intermediate trainees-despite their lack of qualification and average 3.5 years experience-performed about as accurately as qualified experts who had an average 17.5 years experience. New trainees-despite their 5-week, full-time training course or their 6 months experience-were not any better than novices at discriminating matching and similar nonmatching prints, they were just more conservative. Further research is required to determine the precise nature of fingerprint matching expertise and the factors that influence performance. The findings of this representative, lab-based experiment may have implications for the way fingerprint examiners testify in court, but what the findings mean for reasoning about expert performance in the wild is an open, empirical, and epistemological question.

Modelling the impact of single vs. dual presentation on visual discrimination across resolutions.

Visual categorisation relies on our ability to extract useful diagnostic information from complex stimuli. To do this, we can utilise both the "high-level" and "low-level" information in a stimulus; however, the extent to which changes in these properties impact the decision-making process is less clear. We manipulated participants' access to high-level category features via gradated reductions to image resolution while exploring the impact of access to additional category features through a dual-stimulus presentation when compared with single stimulus presentation. Results showed that while increasing image resolution consistently resulted in better choice performance, no benefit was found for dual presentation over single presentation, despite responses for dual presentation being slower compared with single presentation. Applying the diffusion decision model revealed increases in drift rate as a function of resolution, but no change in drift rate for single versus dual presentation. The increase in response time for dual presentation was instead accounted for by an increase in response caution for dual presentations. These findings suggest that while increasing access to high-level features (via increased resolution) can improve participants' categorisation performance, increasing access to both high- and low-level features (via an additional stimulus) does not.

The effect of fingerprint expertise on visual short-term memory.

Expert fingerprint examiners demonstrate impressive feats of memory that may support their accuracy when making high-stakes identification decisions. Understanding the interplay between expertise and memory is therefore critical. Across two experiments, we tested fingerprint examiners and novices on their visual short-term memory for fingerprints. In Experiment 1, experts showed substantially higher memory performance compared to novices for fingerprints from their domain of expertise. In Experiment 2, we manipulated print distinctiveness and found that while both groups benefited from distinctive prints, experts still outperformed novices. This indicates that beyond stimulus qualities, expertise itself enhances short-term memory, likely through more effective organisational processing and sensitivity to meaningful patterns. Taken together, these findings shed light on the cognitive mechanisms that may explain fingerprint examiners' superior memory performance within their domain of expertise. They further suggest that training to improve memory for diverse fingerprints could practically boost examiner performance. Given the high-stakes nature of forensic identification, characterising psychological processes like memory that potentially contribute to examiner accuracy has important theoretical and practical implications.

Honeybees can discriminate between Monet and Picasso paintings.

Honeybees (Apis mellifera) have remarkable visual learning and discrimination abilities that extend beyond learning simple colours, shapes or patterns. They can discriminate landscape scenes, types of flowers, and even human faces. This suggests that in spite of their small brain, honeybees have a highly developed capacity for processing complex visual information, comparable in many respects to vertebrates. Here, we investigated whether this capacity extends to complex images that humans distinguish on the basis of artistic style: Impressionist paintings by Monet and Cubist paintings by Picasso. We show that honeybees learned to simultaneously discriminate between five different Monet and Picasso paintings, and that they do not rely on luminance, colour, or spatial frequency information for discrimination. When presented with novel paintings of the same style, the bees even demonstrated some ability to generalize. This suggests that honeybees are able to discriminate Monet paintings from Picasso ones by extracting and learning the characteristic visual information inherent in each painting style. Our study further suggests that discrimination of artistic styles is not a higher cognitive function that is unique to humans, but simply due to the capacity of animals-from insects to humans-to extract and categorize the visual characteristics of complex images.

The invisible 800-pound gorilla: expertise can increase inattentional blindness.

People can fail to notice objects and events in their visual environment when their attention is engaged elsewhere. This phenomenon is known as inattentional blindness, and its consequences can be costly for important real-world decisions. However, not noticing certain visual information could also signal expertise in a domain. In this study, we compared professional fingerprint analysts and novices on a fingerprint matching task in which we covertly placed an image of a gorilla into one of the prints. This gorilla was either small, or large, but always embedded in a way that made it largely irrelevant to the primary task. We found that analysts were more likely than the novices to miss the large gorilla. We interpret this finding not as a flaw in how these experts make decisions, but most likely an expression of their expertise; instead of processing more information they filter out irrelevant information and constrain their attention to what is important.

Insight and the selection of ideas.

Perhaps it is no accident that insight moments accompany some of humanity's most important discoveries in science, medicine, and art. Here we propose that feelings of insight play a central role in (heuristically) selecting an idea from the stream of consciousness by capturing attention and eliciting a sense of intuitive confidence permitting fast action under uncertainty. The mechanisms underlying this Eureka heuristic are explained within an active inference framework. First, implicit restructuring via Bayesian reduction leads to a higher-order prediction error (i.e., the content of insight). Second, dopaminergic precision-weighting of the prediction error accounts for the intuitive confidence, pleasure, and attentional capture (i.e., the feeling of insight). This insight as precision account is consistent with the phenomenology, accuracy, and neural unfolding of insight, as well as its effects on belief and decision-making. We conclude by reflecting on dangers of the Eureka Heuristic, including the arising and entrenchment of false beliefs and the vulnerability of insights under psychoactive substances and misinformation.

Specific versus varied practice in perceptual expertise training.

We used a longitudinal randomized control experiment to compare the effect of specific practice (training on one form of a task) and varied practice (training on various forms of a task) on perceptual learning and transfer. Participants practiced a visual search task for 10 hours over 2 to 4 weeks. The specific practice group searched for features only in fingerprints during each session, whereas the varied practice group searched for features in five different image categories. Both groups were tested on a series of tasks at four time points: before training, midway through training, immediately after training ended, and 6 to 8 weeks later. The specific group improved more during training and demonstrated greater pre-post performance gains than the varied group on a visual search task with untrained fingerprint images. Both groups improved equally on a visual search task with an untrained image category, but only the specific group's performance dropped significantly when tested several weeks later. Finally, both groups improved equally on a series of untrained fingerprint tasks. Practice with respect to a single category (versus many) instills better near transfer, but category-specific and category-general visual search training appear equally effective for developing task-general expertise. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

Irrelevant insights make worldviews ring true.

Our basic beliefs about reality can be impossible to prove and yet we can feel a strong intuitive conviction about them, as exemplified by insights that imbue an idea with immediate certainty. Here we presented participants with worldview beliefs such as "people's core qualities are fixed" and simultaneously elicited an aha moment. In the first experiment (N = 3000, which included a direct replication), participants rated worldview beliefs as truer when they solved anagrams and also experienced aha moments. A second experiment (N = 1564) showed that the worldview statement and the aha moment must be perceived simultaneously for this 'insight misattribution' effect to occur. These results demonstrate that artificially induced aha moments can make worldview beliefs seem truer, possibly because humans partially rely on feelings of insight to appraise an idea's veracity. Feelings of insight are therefore not epiphenomenal and should be investigated for their effects on decisions, beliefs, and delusions.

The effect of expertise, target usefulness and image structure on visual search.

Experts outperform novices on many cognitive and perceptual tasks. Extensive training has tuned experts to the most relevant information in their specific domain, allowing them to make decisions quickly and accurately. We compared a group of fingerprint examiners to a group of novices on their ability to search for information in fingerprints across two experiments-one where participants searched for target features within a single fingerprint and another where they searched for points of difference between two fingerprints. In both experiments, we also varied how useful the target feature was and whether participants searched for these targets in a typical fingerprint or one that had been scrambled. Experts more efficiently located targets when searching for them in intact but not scrambled fingerprints. In Experiment 1, we also found that experts more efficiently located target features classified as more useful compared to novices, but this expert-novice difference was not present when the target feature was classified as less useful. The usefulness of the target may therefore have influenced the search strategies that participants used, and the visual search advantages that experts display appear to depend on their vast experience with visual regularity in fingerprints. These results align with a domain-specific account of expertise and suggest that perceptual training ought to involve learning to attend to task-critical features.

Collective intelligence in fingerprint analysis.

When a fingerprint is located at a crime scene, a human examiner is counted upon to manually compare this print to those stored in a database. Several experiments have now shown that these professional analysts are highly accurate, but not infallible, much like other fields that involve high-stakes decision-making. One method to offset mistakes in these safety-critical domains is to distribute these important decisions to groups of raters who independently assess the same information. This redundancy in the system allows it to continue operating effectively even in the face of rare and random errors. Here, we extend this "wisdom of crowds" approach to fingerprint analysis by comparing the performance of individuals to crowds of professional analysts. We replicate the previous findings that individual experts greatly outperform individual novices, particularly in their false-positive rate, but they do make mistakes. When we pool the decisions of small groups of experts by selecting the decision of the majority, however, their false-positive rate decreases by up to 8% and their false-negative rate decreases by up to 12%. Pooling the decisions of novices results in a similar drop in false negatives, but increases their false-positive rate by up to 11%. Aggregating people's judgements by selecting the majority decision performs better than selecting the decision of the most confident or the most experienced rater. Our results show that combining independent judgements from small groups of fingerprint analysts can improve their performance and prevent these mistakes from entering courts.

The dark side of Eureka: Artificially induced Aha moments make facts feel true.

Some ideas that we have feel mundane, but others are imbued with a sense of profundity. We propose that Aha! moments make an idea feel more true or valuable in order to aid quick and efficient decision-making, akin to a heuristic. To demonstrate where the heuristic may incur errors, we hypothesized that facts would appear more true if they were artificially accompanied by an Aha! moment elicited using an anagram task. In a preregistered experiment, we found that participants (n=300) provided higher truth ratings for statements accompanied by solved anagrams even if the facts were false, and the effect was particularly pronounced when participants reported an Aha! experience (d = .629). Recent work suggests that feelings of insight usually accompany correct ideas. However, here we show that feelings of insight can be overgeneralized and bias how true an idea or fact appears, simply if it occurs in the temporal 'neighbourhood' of an Aha! moment. We raise the possibility that feelings of insight, epiphanies, and Aha! moments have a dark side, and discuss some circumstances where they may even inspire false beliefs and delusions, with potential clinical importance.

Is Perruchet's dissociation between eyeblink conditioned responding and outcome expectancy evidence for two learning systems?

P. Perruchet (1985b) showed a double dissociation of conditioned responses (CRs) and expectancy for an airpuff unconditioned stimulus (US) in a 50% partial reinforcement schedule in human eyeblink conditioning. In the Perruchet effect, participants show an increase in CRs and a concurrent decrease in expectancy for the airpuff across runs of reinforced trials; conversely, participants show a decrease in CRs and a concurrent increase in expectancy for the airpuff across runs of nonreinforced trials. Three eyeblink conditioning experiments investigated whether the linear trend in eyeblink CRs in the Perruchet effect is a result of changes in associative strength of the conditioned stimulus (CS), US sensitization, or learning the precise timing of the US. Experiments 1 and 2 demonstrated that the linear trend in eyeblink CRs is not the result of US sensitization. Experiment 3 showed that the linear trend in eyeblink CRs is present with both a fixed and a variable CS-US interval and so is not the result of learning the precise timing of the US. The results are difficult to reconcile with a single learning process model of associative learning in which expectancy mediates CRs.