A call for open science in forensics.

The modern canon of open science consists of five "schools of thought" that justify unfettered access to the fruits of scientific research: i) public engagement, ii) democratic right of access, iii) efficiency of knowledge gain, iv) shared technology, and v) better assessment of impact. Here, we introduce a sixth school: due process. Due process under the law includes a right to "discovery" by a defendant of potentially exculpatory evidence held by the prosecution. When such evidence is scientific, due process becomes a Constitutional mandate for open science. To illustrate the significance of this new school, we present a case study from forensics, which centers on a federally funded investigation that reports summary statistics indicating that identification decisions made by forensic firearms examiners are highly accurate. Because of growing concern about validity of forensic methods, the larger scientific community called for public release of the complete analyzable dataset for independent audit and verification. Those in possession of the data opposed release for three years while summary statistics were used by prosecutors to gain admissibility of evidence in criminal trials. Those statistics paint an incomplete picture and hint at flaws in experimental design and analysis. Under the circumstances, withholding the underlying data in a criminal proceeding violates due process. Following the successful open-science model of drug validity testing through "clinical trials," which place strict requirements on experimental design and timing of data release, we argue for registered and open "forensic trials" to ensure transparency and accountability.

A scientist's take on scientific evidence in the courtroom.

Scientific evidence is frequently offered to answer questions of fact in a court of law. DNA genotyping may link a suspect to a homicide. Receptor binding assays and behavioral toxicology may testify to the teratogenic effects of bug repellant. As for any use of science to inform fateful decisions, the immediate question raised is one of credibility: Is the evidence a product of valid methods? Are results accurate and reproducible? While the rigorous criteria of modern science seem a natural model for this evaluation, there are features unique to the courtroom that make the decision process scarcely recognizable by normal standards of scientific investigation. First, much science lies beyond the ken of those who must decide; outside "experts" must be called upon to advise. Second, questions of fact demand immediate resolution; decisions must be based on the science of the day. Third, in contrast to the generative adversarial process of scientific investigation, which yields successive approximations to the truth, the truth-seeking strategy of American courts is terminally adversarial, which risks fracturing knowledge along lines of discord. Wary of threats to credibility, courts have adopted formal rules for determining whether scientific testimony is trustworthy. Here, I consider the effectiveness of these rules and explore tension between the scientists' ideal that momentous decisions should be based upon the highest standards of evidence and the practical reality that those standards are difficult to meet. Justice lies in carefully crafted compromise that benefits from robust bonds between science and law.

Scientific guidelines for evaluating the validity of forensic feature-comparison methods.

When it comes to questions of fact in a legal context-particularly questions about measurement, association, and causality-courts should employ ordinary standards of applied science. Applied sciences generally develop along a path that proceeds from a basic scientific discovery about some natural process to the formation of a theory of how the process works and what causes it to fail, to the development of an invention intended to assess, repair, or improve the process, to the specification of predictions of the instrument's actions and, finally, empirical validation to determine that the instrument achieves the intended effect. These elements are salient and deeply embedded in the cultures of the applied sciences of medicine and engineering, both of which primarily grew from basic sciences. However, the inventions that underlie most forensic science disciplines have few roots in basic science, and they do not have sound theories to justify their predicted actions or results of empirical tests to prove that they work as advertised. Inspired by the "Bradford Hill Guidelines"-the dominant framework for causal inference in epidemiology-we set forth four guidelines that can be used to establish the validity of forensic comparison methods generally. This framework is not intended as a checklist establishing a threshold of minimum validity, as no magic formula determines when particular disciplines or hypotheses have passed a necessary threshold. We illustrate how these guidelines can be applied by considering the discipline of firearm and tool mark examination.

Science, evidence, law, and justice.

For nearly 25 y, the Committee on Science, Technology, and Law (CSTL), of the National Academies of Sciences, Engineering, and Medicine, has brought together distinguished members of the science and law communities to stimulate discussions that would lead to a better understanding of the role of science in legal decisions and government policies and to a better understanding of the legal and regulatory frameworks that govern the conduct of science. Under the leadership of recent CSTL co-chairs David Baltimore and David Tatel, and CSTL director Anne-Marie Mazza, the committee has overseen many interdisciplinary discussions and workshops, such as the international summits on human genome editing and the science of implicit bias, and has delivered advisory consensus reports focusing on topics of broad societal importance, such as dual use research in the life sciences, voting systems, and advances in neural science research using organoids and chimeras. One of the most influential CSTL activities concerns the use of forensic evidence by law enforcement and the courts, with emphasis on the scientific validity of forensic methods and the role of forensic testimony in bringing about justice. As coeditors of this Special Feature, CSTL alumni Tom Albright and Jennifer Mnookin have recruited articles at the intersection of science and law that reveal an emerging scientific revolution of forensic practice, which we hope will engage a broad community of scientists, legal scholars, and members of the public with interest in science-based legal policy and justice reform.

How to make better forensic decisions.

Much of forensic practice today involves human decisions about the origins of patterned sensory evidence, such as tool marks and fingerprints discovered at a crime scene. These decisions are made by trained observers who compare the evidential pattern to an exemplar pattern produced by the suspected source of the evidence. The decision consists of a determination as to whether the two patterns are similar enough to have come from the same source. Although forensic pattern comparison disciplines have for decades played a valued role in criminal investigation and prosecution, the extremely high personal and societal costs of failure-the conviction of innocent people-has elicited calls for caution and for the development of better practices. These calls have been heard by the scientific community involved in the study of human information processing, which has begun to offer much-needed perspectives on sensory measurement, discrimination, and classification in a forensic context. Here I draw from a well-established theoretical and empirical approach in sensory science to illustrate the vulnerabilities of contemporary pattern comparison disciplines and to suggest specific strategies for improvement.

The US Department of Justice stumbles on visual perception.

A large and highly valuable category of forensic evidence consists of patterned impressions created during the perpetration of a crime. These crime scene artifacts, such as fingerprints or tire tracks, offer visual sensory information that is assessed by trained human observers and compared to sensory experiences elicited by model patterns that would have been produced under a hypothesized set of conditions. By means of this "forensic feature comparison," the observer makes a judgment about whether the evidence and the model are sufficiently similar to support common origin. In light of documented failures of this approach, significant concerns have been raised about its scientific validity. In response to these concerns, the US Department of Justice has made assertions about how forensic examiners perform feature comparison tasks that are not consistent with modern scientific understanding of the processes of sensation and perception. Clarification of these processes highlights new ways of thinking about and improving the accuracy of forensic feature comparison and underscores the vital role of science in achieving justice.

A call for more science in forensic science.

Forensic science is critical to the administration of justice. The discipline of forensic science is remarkably complex and includes methodologies ranging from DNA analysis to chemical composition to pattern recognition. Many forensic practices developed under the auspices of law enforcement and were vetted primarily by the legal system rather than being subjected to scientific scrutiny and empirical testing. Beginning in the 1990s, exonerations based on DNA-related methods revealed problems with some forensic disciplines, leading to calls for major reforms. This process generated a National Academy of Science report in 2009 that was highly critical of many forensic practices and eventually led to the establishment of the National Commission for Forensic Science (NCFS) in 2013. The NCFS was a deliberative body that catalyzed communication between nonforensic scientists, forensic scientists, and other stakeholders in the legal community. In 2017, despite continuing problems with forensic science, the Department of Justice terminated the NCFS. Just when forensic science needs the most support, it is getting the least. We urge the larger scientific community to come to the aid of our forensic colleagues by advocating for urgently needed research, testing, and financial support.

Why eyewitnesses fail.

Eyewitness identifications play an important role in the investigation and prosecution of crimes, but it is well known that eyewitnesses make mistakes, often with serious consequences. In light of these concerns, the National Academy of Sciences recently convened a panel of experts to undertake a comprehensive study of current practice and use of eyewitness testimony, with an eye toward understanding why identification errors occur and what can be done to prevent them. The work of this committee led to key findings and recommendations for reform, detailed in a consensus report entitled Identifying the Culprit: Assessing Eyewitness Identification In this review, I focus on the scientific issues that emerged from this study, along with brief discussions of how these issues led to specific recommendations for additional research, best practices for law enforcement, and use of eyewitness evidence by the courts.

Auditory modulation of spiking activity and local field potentials in area MT does not appear to underlie an audiovisual temporal illusion.

The timing of brief stationary sounds has been shown to alter the perceived speed of visual apparent motion (AM), presumably by altering the perceived timing of the individual frames of the AM stimuli and/or the duration of the interstimulus intervals (ISIs) between those frames. To investigate the neural correlates of this "temporal ventriloquism" illusion, we recorded spiking and local field potential (LFP) activity from the middle temporal area (area MT) in awake, fixating macaques. We found that the spiking activity of most MT neurons (but not the LFP) was tuned for the ISI/speed (these parameters covaried) of our AM stimuli but that auditory timing had no effect on that tuning. We next asked whether the predicted changes in perceived timing were reflected in the timing of neuronal responses to the individual frames of the AM stimuli. Although spiking dynamics were significantly, if weakly, affected by auditory timing in a minority of neurons, the timing of spike responses did not systematically mirror the predicted perception of stimuli. Conversely, the duration of LFP responses in ß- and γ-frequency bands was qualitatively consistent with human perceptual reports. We discovered, however, that LFP responses to auditory stimuli presented alone were robust and that responses to audiovisual stimuli were predicted by the linear sum of responses to auditory and visual stimuli presented individually. In conclusion, we find evidence of auditory input into area MT but not of the nonlinear audiovisual interactions we had hypothesized to underlie the illusion. NEW & NOTEWORTHY We utilized a set of audiovisual stimuli that elicit an illusion demonstrating "temporal ventriloquism" in visual motion and that have spatiotemporal intervals for which neurons within the middle temporal area are selective. We found evidence of auditory input into the middle temporal area but not of the nonlinear audiovisual interactions underlying this illusion. Our findings suggest that either the illusion was absent in our nonhuman primate subjects or the neuronal correlates of this illusion lie within other areas.

Sensory adaptation as optimal resource allocation.

Visual adaptation is expected to improve visual performance in the new environment. This expectation has been contradicted by evidence that adaptation sometimes decreases sensitivity for the adapting stimuli, and sometimes it changes sensitivity for stimuli very different from the adapting ones. We hypothesize that this pattern of results can be explained by a process that optimizes sensitivity for many stimuli, rather than changing sensitivity only for those stimuli whose statistics have changed. To test this hypothesis, we measured visual sensitivity across a broad range of spatiotemporal modulations of luminance, while varying the distribution of stimulus speeds. The manipulation of stimulus statistics caused a large-scale reorganization of visual sensitivity, forming the orderly pattern of sensitivity gains and losses. This pattern is predicted by a theory of distribution of receptive field characteristics in the visual system.

Stimulus-dependent variability and noise correlations in cortical MT neurons.

Population codes assume that neural systems represent sensory inputs through the firing rates of populations of differently tuned neurons. However, trial-by-trial variability and noise correlations are known to affect the information capacity of neural codes. Although recent studies have shown that stimulus presentation reduces both variability and rate correlations with respect to their spontaneous level, possibly improving the encoding accuracy, whether these second order statistics are tuned is unknown. If so, second-order statistics could themselves carry information, rather than being invariably detrimental. Here we show that rate variability and noise correlation vary systematically with stimulus direction in directionally selective middle temporal (MT) neurons, leading to characteristic tuning curves. We show that such tuning emerges in a stochastic recurrent network, for a set of connectivity parameters that overlaps with a single-state scenario and multistability. Information theoretic analysis shows that second-order statistics carry information that can improve the accuracy of the population code.

Nonhuman primate model of schizophrenia using a noninvasive EEG method.

There is growing evidence that impaired sensory-processing significantly contributes to the cognitive deficits found in schizophrenia. For example, the mismatch negativity (MMN) and P3a event-related potentials (ERPs), neurophysiological indices of sensory and cognitive function, are reduced in schizophrenia patients and may be used as biomarkers of the disease. In agreement with glutamatergic theories of schizophrenia, NMDA antagonists, such as ketamine, elicit many symptoms of schizophrenia when administered to normal subjects, including reductions in the MMN and the P3a. We sought to develop a nonhuman primate (NHP) model of schizophrenia based on NMDA-receptor blockade using subanesthetic administration of ketamine. This provided neurophysiological measures of sensory and cognitive function that were directly comparable to those recorded from humans. We first developed methods that allowed recording of ERPs from humans and rhesus macaques and found homologous MMN and P3a ERPs during an auditory oddball paradigm. We then investigated the effect of ketamine on these ERPs in macaques. As found in humans with schizophrenia, as well as in normal subjects given ketamine, we observed a significant decrease in amplitude of both ERPs. Our findings suggest the potential of a pharmacologically induced model of schizophrenia in NHPs that can pave the way for EEG-guided investigations into cellular mechanisms and therapies. Furthermore, given the established link between these ERPs, the glutamatergic system, and deficits in other neuropsychiatric disorders, our model can be used to investigate a wide range of pathologies.

Optogenetics through windows on the brain in the nonhuman primate.

Optogenetics combines optics and genetics to control neuronal activity with cell-type specificity and millisecond temporal precision. Its use in model organisms such as rodents, Drosophila, and Caenorhabditis elegans is now well-established. However, application of this technology in nonhuman primates (NHPs) has been slow to develop. One key challenge has been the delivery of viruses and light to the brain through the thick dura mater of NHPs, which can only be penetrated with large-diameter devices that damage the brain. The opacity of the NHP dura prevents visualization of the underlying cortex, limiting the spatial precision of virus injections, electrophysiological recordings, and photostimulation. Here, we describe a new optogenetics approach in which the native dura is replaced with an optically transparent artificial dura. This artificial dura can be penetrated with fine glass micropipettes, enabling precisely targeted injections of virus into brain tissue with minimal damage to cortex. The expression of optogenetic agents can be monitored visually over time. Most critically, this optical window permits targeted, noninvasive photostimulation and concomitant measurements of neuronal activity via intrinsic signal imaging and electrophysiological recordings. We present results from both anesthetized-paralyzed (optical imaging) and awake-behaving NHPs (electrophysiology). The improvements over current methods made possible by the artificial dura should enable the widespread use of optogenetic tools in NHP research, a key step toward the development of therapies for neuropsychiatric and neurological diseases in humans.

Receptive field positions in area MT during slow eye movements.

Perceptual stability requires the integration of information across eye movements. We first tested the hypothesis that motion signals are integrated by neurons whose receptive fields (RFs) do not move with the eye but stay fixed in the world. Specifically, we measured the RF properties of neurons in the middle temporal area (MT) of macaques (Macaca mulatta) during the slow phase of optokinetic nystagmus. Using a novel method to estimate RF locations for both spikes and local field potentials, we found that the location on the retina that changed spike rates or local field potentials did not change with eye position; RFs moved with the eye. Second, we tested the hypothesis that neurons link information across eye positions by remapping the retinal location of their RFs to future locations. To test this, we compared RF locations during leftward and rightward slow phases of optokinetic nystagmus. We found no evidence for remapping during slow eye movements; the RF location was not affected by eye-movement direction. Together, our results show that RFs of MT neurons and the aggregate activity reflected in local field potentials are yoked to the eye during slow eye movements. This implies that individual MT neurons do not integrate sensory information from a single position in the world across eye movements. Future research will have to determine whether such integration, and the construction of perceptual stability, takes place in the form of a distributed population code in eye-centered visual cortex or is deferred to downstream areas.

Modulation of neuronal responses during covert search for visual feature conjunctions.

While searching for an object in a visual scene, an observer's attentional focus and eye movements are often guided by information about object features and spatial locations. Both spatial and feature-specific attention are known to modulate neuronal responses in visual cortex, but little is known of the dynamics and interplay of these mechanisms as visual search progresses. To address this issue, we recorded from directionally selective cells in visual area MT of monkeys trained to covertly search for targets defined by a unique conjunction of color and motion features and to signal target detection with an eye movement to the putative target. Two patterns of response modulation were observed. One pattern consisted of enhanced responses to targets presented in the receptive field (RF). These modulations occurred at the end-stage of search and were more potent during correct target identification than during erroneous saccades to a distractor in RF, thus suggesting that this modulation is not a mere presaccadic enhancement. A second pattern of modulation was observed when RF stimuli were nontargets that shared a feature with the target. The latter effect was observed during early stages of search and is consistent with a global feature-specific mechanism. This effect often terminated before target identification, thus suggesting that it interacts with spatial attention. This modulation was exhibited not only for motion but also for color cue, although MT neurons are known to be insensitive to color. Such cue-invariant attentional effects may contribute to a feature binding mechanism acting across visual dimensions.

Spatially distributed computation in cortical circuits.

The traditional view of neural computation in the cerebral cortex holds that sensory neurons are specialized, i.e., selective for certain dimensions of sensory stimuli. This view was challenged by evidence of contextual interactions between stimulus dimensions in which a neuron's response to one dimension strongly depends on other dimensions. Here, we use methods of mathematical modeling, psychophysics, and electrophysiology to address shortcomings of the traditional view. Using a model of a generic cortical circuit, we begin with the simple demonstration that cortical responses are always distributed among neurons, forming characteristic waveforms, which we call neural waves. When stimulated by patterned stimuli, circuit responses arise by interference of neural waves. Results of this process depend on interaction between stimulus dimensions. Comparison of modeled responses with responses of biological vision makes it clear that the framework of neural wave interference provides a useful alternative to the standard concept of neural computation.

Remembering visual motion: neural correlates of associative plasticity and motion recall in cortical area MT.

The pictorial content of visual memories recalled by association is embodied by neuronal activity at the highest processing stages of primate visual cortex. This activity is elicited by top-down signals from the frontal lobe and recapitulates the bottom-up pattern normally obtained by the recalled stimulus. To explore the generality and mechanisms of this phenomenon, we recorded motion-sensitive neurons at an early stage of cortical processing. After monkeys learned to associate directions of motion with static shapes, these neurons exhibited unprecedented selectivity for the shapes. This emergent shape selectivity reflects activation of neurons representing the motion stimuli recalled by association, and it suggests that recall-related activity may be a general feature of neurons in visual cortex.

Adaptive surround modulation in cortical area MT.

Visual motion perception relies on two opposing operations: integration and segmentation. Integration overcomes motion ambiguity in the visual image by spatial pooling of motion signals, whereas segmentation identifies differences between adjacent moving objects. For visual motion area MT, previous investigations have reported that stimuli in the receptive field surround, which do not elicit a response when presented alone, can nevertheless modulate responses to stimuli in the receptive field center. The directional tuning of this "surround modulation" has been found to be mainly antagonistic and hence consistent with segmentation. Here, we report that surround modulation in area MT can be either antagonistic or integrative depending upon the visual stimulus. Both types of modulation were delayed relative to response onset. Our results suggest that the dominance of antagonistic modulation in previous MT studies was due to stimulus choice and that segmentation and integration are achieved, in part, via adaptive surround modulation.

Selective and quickly reversible inactivation of mammalian neurons in vivo using the Drosophila allatostatin receptor.

Genetic strategies for perturbing activity of selected neurons hold great promise for understanding circuitry and behavior. Several such strategies exist, but there has been no direct demonstration of reversible inactivation of mammalian neurons in vivo. We previously reported quickly reversible inactivation of neurons in vitro using expression of the Drosophila allatostatin receptor (AlstR). Here, adeno-associated viral vectors are used to express AlstR in vivo in cortical and thalamic neurons of rats, ferrets, and monkeys. Application of the receptor's ligand, allatostatin (AL), leads to a dramatic reduction in neural activity, including responses of visual neurons to optimized visual stimuli. Additionally, AL eliminates activity in spinal cords of transgenic mice conditionally expressing AlstR. This reduction occurs selectively in AlstR-expressing neurons. Inactivation can be reversed within minutes upon washout of the ligand and is repeatable, demonstrating that the AlstR/AL system is effective for selective, quick, and reversible silencing of mammalian neurons in vivo.

Bayesian adaptive estimation of the contrast sensitivity function: the quick CSF method.

The contrast sensitivity function (CSF) predicts functional vision better than acuity, but long testing times prevent its psychophysical assessment in clinical and practical applications. This study presents the quick CSF (qCSF) method, a Bayesian adaptive procedure that applies a strategy developed to estimate multiple parameters of the psychometric function (A. B. Cobo-Lewis, 1996; L. L. Kontsevich & C. W. Tyler, 1999). Before each trial, a one-step-ahead search finds the grating stimulus (defined by frequency and contrast) that maximizes the expected information gain (J. V. Kujala & T. J. Lukka, 2006; L. A. Lesmes et al., 2006), about four CSF parameters. By directly estimating CSF parameters, data collected at one spatial frequency improves sensitivity estimates across all frequencies. A psychophysical study validated that CSFs obtained with 100 qCSF trials ( approximately 10 min) exhibited good precision across spatial frequencies (SD < 2-3 dB) and excellent agreement with CSFs obtained independently (mean RMSE = 0.86 dB). To estimate the broad sensitivity metric provided by the area under the log CSF (AULCSF), only 25 trials were needed to achieve a coefficient of variation of 15-20%. The current study demonstrates the method's value for basic and clinical investigations. Further studies, applying the qCSF to measure wider ranges of normal and abnormal vision, will determine how its efficiency translates to clinical assessment.