A guide to measuring expert performance in forensic pattern matching.

Decisions in forensic science are often binary. A firearms expert must decide whether a bullet was fired from a particular gun or not. A face comparison expert must decide whether a photograph matches a suspect or not. A fingerprint examiner must decide whether a crime scene fingerprint belongs to a suspect or not. Researchers who study these decisions have therefore quantified expert performance using measurement models derived largely from signal detection theory. Here we demonstrate that the design and measurement choices researchers make can have a dramatic effect on the conclusions drawn about the performance of forensic examiners. We introduce several performance models - proportion correct, diagnosticity ratio, and parametric and non-parametric signal detection measures - and apply them to forensic decisions. We use data from expert and novice fingerprint comparison decisions along with a resampling method to demonstrate how experimental results can change as a function of the task, case materials, and measurement model chosen. We also graphically show how response bias, prevalence, inconclusive responses, floor and ceiling effects, case sampling, and number of trials might affect one's interpretation of expert performance in forensics. Finally, we discuss several considerations for experimental and diagnostic accuracy studies: (1) include an equal number of same-source and different-source trials; (2) record inconclusive responses separately from forced choices; (3) include a control comparison group; (4) counterbalance or randomly sample trials for each participant; and (5) present as many trials to participants as is practical.

Thinking false and slow: Implausible beliefs and the Cognitive Reflection Test.

Why do people believe implausible claims like conspiracy theories, pseudoscience, and fake news? Past studies using the Cognitive Reflection Test (CRT) suggest that implausible beliefs may result from an unwillingness to effortfully process information (i.e., cognitive miserliness). Our analysis (N = 664) tests this account by comparing CRT performance (total score, number and proportion of incorrect intuitive responses, and completion time) for endorsers and non-endorsers of implausible claims. Our results show that endorsers performed worse than non-endorsers on the CRT, but they took significantly longer to answer the questions and did not make proportionally more intuitive mistakes. Endorsers therefore appear to process information effortfully but nonetheless score lower on the CRT. Poorer overall CRT performance may not necessarily indicate that those who endorse implausible beliefs have a more reflexive, intuitive, or non-analytical cognitive style than non-endorsers.

The invisible 800-pound gorilla: expertise can increase inattentional blindness.

People can fail to notice objects and events in their visual environment when their attention is engaged elsewhere. This phenomenon is known as inattentional blindness, and its consequences can be costly for important real-world decisions. However, not noticing certain visual information could also signal expertise in a domain. In this study, we compared professional fingerprint analysts and novices on a fingerprint matching task in which we covertly placed an image of a gorilla into one of the prints. This gorilla was either small, or large, but always embedded in a way that made it largely irrelevant to the primary task. We found that analysts were more likely than the novices to miss the large gorilla. We interpret this finding not as a flaw in how these experts make decisions, but most likely an expression of their expertise; instead of processing more information they filter out irrelevant information and constrain their attention to what is important.

Specific versus varied practice in perceptual expertise training.

We used a longitudinal randomized control experiment to compare the effect of specific practice (training on one form of a task) and varied practice (training on various forms of a task) on perceptual learning and transfer. Participants practiced a visual search task for 10 hours over 2 to 4 weeks. The specific practice group searched for features only in fingerprints during each session, whereas the varied practice group searched for features in five different image categories. Both groups were tested on a series of tasks at four time points: before training, midway through training, immediately after training ended, and 6 to 8 weeks later. The specific group improved more during training and demonstrated greater pre-post performance gains than the varied group on a visual search task with untrained fingerprint images. Both groups improved equally on a visual search task with an untrained image category, but only the specific group's performance dropped significantly when tested several weeks later. Finally, both groups improved equally on a series of untrained fingerprint tasks. Practice with respect to a single category (versus many) instills better near transfer, but category-specific and category-general visual search training appear equally effective for developing task-general expertise. (PsycInfo Database Record (c) 2022 APA, all rights reserved).

The effect of expertise, target usefulness and image structure on visual search.

Experts outperform novices on many cognitive and perceptual tasks. Extensive training has tuned experts to the most relevant information in their specific domain, allowing them to make decisions quickly and accurately. We compared a group of fingerprint examiners to a group of novices on their ability to search for information in fingerprints across two experiments-one where participants searched for target features within a single fingerprint and another where they searched for points of difference between two fingerprints. In both experiments, we also varied how useful the target feature was and whether participants searched for these targets in a typical fingerprint or one that had been scrambled. Experts more efficiently located targets when searching for them in intact but not scrambled fingerprints. In Experiment 1, we also found that experts more efficiently located target features classified as more useful compared to novices, but this expert-novice difference was not present when the target feature was classified as less useful. The usefulness of the target may therefore have influenced the search strategies that participants used, and the visual search advantages that experts display appear to depend on their vast experience with visual regularity in fingerprints. These results align with a domain-specific account of expertise and suggest that perceptual training ought to involve learning to attend to task-critical features.