The logic of forensic pathology opinion.

Evaluating evidence and providing opinions are at the heart of forensic science, and forensic experts are expected to provide opinions that are based on logically sound and transparent scientific reasoning, and that honour the boundaries of their area of expertise. In order to meet these objectives, many fields of science explicitly apply Bayes' theorem, which describes the logically correct way to update probabilities on the basis of observations. Making a distinction between 'investigative' and evaluative' modes of operating helps to implement the theorem into daily casework. Use of these principles promotes the logic and transparency of the reasoning that leads to expert's opinion and helps the expert to stay within her remit. Despite these important benefits, forensic pathology seems slow to adopt these principles. In this article, we explore this issue and suggest a way forward. We start with a short introduction to Bayes' theorem and its benefits, followed by a discussion of why its application is actually second nature to medical practitioners. We then discuss the difference between investigative and evaluative opinions, and how they enable the forensic pathologist to reconcile Bayes' theorem with the different phases of a forensic investigation. Throughout the text, practical examples illustrate the various ways in which the logically correct way of evidence interpretation can be implemented, and how it may help the forensic pathologist to provide an appropriate and relevant opinion.

Targeting relevant sampling areas for human biological traces: Where to sample displaced bodies for offender DNA?

Sampling strategy is one of the deciding factors in DNA typing success rates. Small amounts of bodily fluid traces and (skin) contact traces are currently not visualized in standard forensic practice. Trace recovery is usually based on the information available in a particular case and on the experience and 'forensic common sense' applied by the trace recovery expert. Interactions between an offender and a victim may have characteristic features, resulting in specific trace patterns. Understanding these interactions, and their resulting trace patterns, might improve crime related trace recovery as well as DNA typing success rates. In this study, we examined the interactions between offender and victim when a body has been relocated from one position/location to another. The contact between the hands of the offender and the body of the victim was visualized using a fluorescent dye in a lotion that was applied to the hands of the individual undertaking the relocation. The contact locations were scored and patterns were analyzed based on both victim and offender characteristics (height, weight, age, gender). The resulting patterns were compared to current trace recovery practices in the Netherlands. The results of this large-scale study facilitate evidence-based sampling supporting both investigative and evaluative forensic examinations.

Evaluating evidence in linked crimes with multiple offenders.

In de Zoete et al. (2015) a framework for the evaluation of evidence when an individual is a suspect of two separate offenses (based on Evett et al., 2006) is implemented using a Bayesian network. Here, we extend this to situations with multiple offenders. When we have multiple offenders, new questions arise: (1) Can we distinguish between the offenders, even if we do not know their identity? (2) Do we know that certain pieces of evidence originate from the same person? (3) Do we know the number of offenders? With the aid of a mock case example, we show that such subtle differences between situations can lead to substantially different conclusions in terms of posterior probabilities of a certain suspect being one of the offenders in a particular crime. We reach our conclusions by constructing appropriate Bayesian networks for each situation. Although we find it undesirable that Bayesian networks are demonstrated in court, they can be very helpful in guiding expert and legal reasoning, identifying pitfalls and assist in preventing them. Bayesian networks can be used as a tool to understand how the different pieces of evidence influence each others evidential value, and the probabilities of the hypotheses of interest.

The combined evidential value of autosomal and Y-chromosomal DNA profiles obtained from the same sample.

When a Y-chromosomal and a (partial) autosomal DNA profile are obtained from one crime sample, and both profiles match the suspect's profiles, we would like to know the combined evidential value. To calculate the likelihood ratio of observing the autosomal and Y-chromosomal DNA profiles combined, we need to know the conditional random match probability of the observed autosomal DNA profile, given the Y-chromosomal match. We examine this conditional probability in two ways: (1) with a database containing data of 2,085 men and (2) using a simulation model. We conclude that if the Y-chromosomal DNA profiles match, we can still regard the autosomal DNA profile as independent from the Y-chromosomal DNA profile if the matching person is not a descendant of the father of the donor of the (crime) sample. The evidential value can, in that case, be computed by multiplying the random match probabilities of the individual profiles.

An overview of log likelihood ratio cost in forensic science - Where is it used and what values can we expect?

There is increasing support for reporting evidential strength as a likelihood ratio (LR) and increasing interest in (semi-)automated LR systems. The log-likelihood ratio cost (Cllr) is a popular metric for such systems, penalizing misleading LRs further from 1 more. Cllr = 0 indicates perfection while Cllr = 1 indicates an uninformative system. However, beyond this, what constitutes a "good" Cllr is unclear. Aiming to provide handles on when a Cllr is "good", we studied 136 publications on (semi-)automated LR systems. Results show Cllr use heavily depends on the field, e.g., being absent in DNA analysis. Despite more publications on automated LR systems over time, the proportion reporting Cllr remains stable. Noticeably, Cllr values lack clear patterns and depend on the area, analysis and dataset. As LR systems become more prevalent, comparing them becomes crucial. This is hampered by different studies using different datasets. We advocate using public benchmark datasets to advance the field.

From data to a validated score-based LR system: A practitioner's guide.

Likelihood ratios (LRs) are a useful measure of evidential strength. In forensic casework consisting of a flow of cases with essentially the same question and the same analysis method, it is feasible to construct an 'LR system', that is, an automated procedure that has the observations as input and an LR as output. This paper is aimed at practitioners interested in building their own LR systems. It gives an overview of the different steps needed to get to a validated LR system from data. The paper is accompanied by a notebook that illustrates each step with an example using glass data. The notebook introduces open-source software in Python constructed by NFI (Netherlands Forensic Institute) data scientists and statisticians.

The evidential strength of a combination of corresponding class features in tire examination.

Tire marks are an important type of forensic evidence as they are frequently encountered at crime scenes. When the tires of a suspect's car are compared, the evidence can be very strong if so-called 'acquired features' are observed to correspond. When only 'class characteristics' such as parts of the tire pattern are observed to correspond, it is obvious that many other tires will exist that also correspond, and so this evidence is usually considered very weak or is simply ignored. Like Benedict et al. (2014) we argue that such evidence can still be strong and should be taken into account. We describe a method for assessing the evidential strength of a set of corresponding class characteristics by presenting a case example from the Netherlands in which tire marks were obtained. Only part of two different tire patterns were visible, in combination with measurements on the axes width. Suitable databases were found already existing and accessible to forensic experts. We show how such data can be used to quantify the strength of evidence and how it can be reported. We also show how the risk of bias due to information surrounding the case may be minimized in cases like this. Our 'blind' procedure enables the expert to report a correspondence between class features in a more convincing way than standard procedures allow. In the particular exemplar case quite strong evidence was obtained, which was accepted and used by the Dutch court. We generalize this procedure for quantifying the evidential value of an expert's opinion of a correspondence. This examination procedure can be applied directly to other types of pattern evidence such as shoeprints, fingerprints, or images. Furthermore, it is 'blind' in the sense that the risk of contextual bias is minimized.

Evidential value of duct tape comparison using loopbreaking patterns.

A new method for the evaluation of duct tape ends is proposed. This method is based on the breaks of the loops in the warp yarns, when duct tape with a scrim of chain-stitched warp yarns and weft-insertion is torn. After tearing, the loop at the end of each warp yarn can be in one of four states: open, closed, complex or missing. Additionally, the horizontal position of each warp yarn can be expressed in terms of weft yarns. The evidential strength of these loopbreaking patterns is expressed in terms of likelihood ratios. We construct a likelihood ratio system to determine these likelihood ratios. This consists of three dynamic Bayesian networks, which are based on the main assumption that the loopbreaking patterns are a stochastic process which comply to the Markov property. A dataset is used to train and test the LR-system. Based on these results, it is found that the loopbreaking patterns contain very strong evidence. More data, especially on observation errors, is needed to evaluate the system further.

Measuring calibration of likelihood-ratio systems: A comparison of four metrics, including a new metric devPAV.

Numerical likelihood-ratio (LR) systems aim to calculate evidential strength for forensic evidence evaluation. Calibration of such LR-systems is essential: one does not want to over- or understate the strength of the evidence. Metrics that measure calibration differ in sensitivity to errors in calibration of such systems. In this paper we compare four calibration metrics by a simulation study based on Gaussian Log LR-distributions. Three calibration metrics are taken from the literature (Good, 1985; Royall, 1997; Ramos and Gonzalez-Rodriguez, 2013) [1-3], and a fourth metric is proposed by us. We evaluated these metrics by two performance criteria: differentiation (between well- and ill-calibrated LR-systems) and stability (of the value of the metric for a variety of well-calibrated LR-systems). Two metrics from the literature (the expected values of LR and of 1/LR, and the rate of misleading evidence stronger than 2) do not behave as desired in many simulated conditions. The third one (Cllrcal) performs better, but our newly proposed method (which we coin devPAV) is shown to behave equally well to clearly better under almost all simulated conditions. On the basis of this work, we recommend to use both devPAV and Cllrcal to measure calibration of LR-systems, where the current results indicate that devPAV is the preferred metric. In the future external validity of this comparison study can be extended by simulating non-Gaussian LR-distributions.

Why calibrating LR-systems is best practice. A reaction to "The evaluation of evidence for microspectrophotometry data using functional data analysis", in FSI 305.

In their paper "The evaluation of evidence for microspectrophotometry data using functional data analysis", in FSI 305, Aitken et al. present a likelihood-ratio (LR) system for their data. We show the values generated by this system cannot be interpreted as LRs: they are ill-calibrated and should be interpreted as discriminating scores. We demonstrate how to transform the scores to well-calibrated LRs using a post-hoc calibrating step. Also, we address criticisms of calibration posited by Aitken et al. We conclude by noting that ill-calibrated LR-values are misleadingly small or large. Therefore calibration should be measured and, if necessary, corrected for. The corrected LR-values (instead of the discriminating scores) can be used to update the prior odds in Bayes rule.

Combining evidence in complex cases - a practical approach to interdisciplinary casework.

Activity level evaluations, although still a major challenge for many disciplines, bring a wealth of possibilities for a more formal approach to the evaluation of interdisciplinary forensic evidence. This paper proposes a practical methodology for combining evidence from different disciplines within the likelihood ratio framework. Evidence schemes introduced in this paper make the process of combining evidence more insightful and intuitive thereby assisting experts in their interdisciplinairy evaluation and in explaining this process to the courts. When confronted with two opposing scenarios and multiple types of evidence, the likelihood ratio approach allows experts to combine this evidence in a probabilistic manner. Parts of the prosecution and defence scenarios for which forensic science is expected to be informative are identified. For these so called core elements, activity level propositions are formulated. Afterwards evidence schemes are introduced to assist the expert in combining the evidence in a logical manner. Two types of evidence relations are identified: serial and parallel evidence. Practical guidelines are given on how to deal with both types of evidence relations when combining the evidence.

Anti-doping researchers should conform to certain statistical standards from forensic science.

Anti-doping is currently viewed as a forensic science. However, close examination shows that the statistical treatment of evidence is inconsistent with that view. Here it is insisted that anti-doping researchers should conform to certain statistical standards from forensic science.

Shoe print examinations: effects of expectation, complexity and experience.

Even though trace evidence is becoming more and more important in legal cases, only little is known about the influence of task and context factors on comparative judgments. In the present study we investigated how expectations and complexity affect shoe print examinations and to what extent differences exist between beginners and experienced examiners. Twelve examiners assessed similarity between a shoe print and a shoe for eight different cases. For half the cases expectation was induced by providing additional incriminating evidence. A complex case meant that the print was relatively noisy, for example because the perpetrator rotated his foot. A simple case meant that the print was clear. The results showed that there was no effect of expectation and no effect of experience. Only complexity affected the examiners' assessments: when the background was noisy, the acquired features received a lower evidential value than when the background was clear. Apparently, examiners compensated for the quality of the print and were more cautious in drawing conclusions when prints were less clear. Even though the results allow for some optimism with regard to the influence of expectations on shoe print examinations, it has to be taken into account that the Dutch procedure is supported by a formal guideline, which may (partly) explain the present findings.

Performance Study of a Score-based Likelihood Ratio System for Forensic Fingermark Comparison.

In this article, the performance of a score-based likelihood ratio (LR) system for comparisons of fingerprints with fingermarks is studied. The system is based on an automated fingerprint identification system (AFIS) comparison algorithm and focuses on fingerprint comparisons where the fingermarks contain 6-11 minutiae. The hypotheses under consideration are evaluated at the level of the person, not the finger. The LRs are presented with bootstrap intervals indicating the sampling uncertainty involved. Several aspects of the performance are measured: leave-one-out cross-validation is applied, and rates of misleading evidence are studied in two ways. A simulation study is performed to study the coverage of the bootstrap intervals. The results indicate that the evidential strength for same source comparisons that do not meet the Dutch twelve-point standard may be substantial. The methods used can be generalized to measure the performance of score-based LR systems in other fields of forensic science.

A probabilistic approach for the interpretation of RNA profiles as cell type evidence.

DNA profiles can be used as evidence to distinguish between possible donors of a crime stain. In some cases, both the prosecution and the defence claim that the cell material was left by the suspect but they dispute which cell type was left behind. For example, in sexual offense cases the prosecution could claim that the sample contains semen cells where the defence argues that the sample contains skin cells. In these cases, traditional methods (e.g. a phosphatase test) can be used to examine the cell type contained in the sample. However, there are some drawbacks when using these methods. For instance, many of these techniques need to be carried out separately for each cell type and each of them requires part of the available sample, which reduces the amount that can be used for DNA analysis. Another option is messenger RNA (mRNA) evidence. mRNA expression levels vary among cell types and can be used to make (probability) statements about the cell type(s) present in a sample. Existing methods for the interpretation of RNA profiles as evidence for the presence of certain cell types aim at making categorical statements. Such statements limit the possibility to report the associated uncertainty. Some of these existing methods will be discussed. Most notably, a method based on a 'n/2' scoring rule (Lindenbergh et al.) and a method using marker values and cell type scoring thresholds (Roeder et al.). From a statistical point of view, a probabilistic approach is the most obvious choice. Two approaches (multinomial logistic regression and naïve Bayes') are suggested. All methods are compared, using two different datasets and several criteria regarding their ability to assess the evidential value of RNA profiles. We conclude that both the naïve Bayes' method and a method based on multinomial logistic regression, that produces a probabilistic statement as measure of the evidential value, are an important improvement of the existing methods. Besides a better performance, they are flexible and can be adapted to other situations. For example, they could potentially assist in the combination of RNA with DNA evidence.

Cell type determination and association with the DNA donor.

In forensic casework, evidence regarding the type of cell material contained in a stain can be crucial in determining what happened. For example, a DNA match in a sexual offense can become substantially more incriminating when there is evidence supporting that semen cells are present. Besides the question which cell types are present in a sample, also the question who donated what (association) is very relevant. This question is surprisingly difficult, even for stains with a single donor. The evidential value of a DNA profile needs to be combined with knowledge regarding the specificity and sensitivity of cell type tests. This, together with prior probabilities for the different donor-cell type combinations, determines the most likely combination. We present a Bayesian network that can assist in associating donors and cell types. A literature overview on the sensitivity and specificity of three cell type tests (PSA test for seminal fluid, RSID saliva and RSID semen) is helpful in assigning conditional probabilities. The Bayesian network is linked with a software package for interpreting mixed DNA profiles. This allows for a sensitivity analysis that shows to what extent the conclusion depends on the quantity of available research data. This can aid in making decisions regarding further research. It is shown that the common assumption that an individual (e.g. the victim) is one of the donors in a mixed DNA profile can have unwanted consequences for the association between donors and cell types.

Leveraging probabilistic peak detection to estimate baseline drift in complex chromatographic samples.

Accurate analysis of chromatographic data often requires the removal of baseline drift. A frequently employed strategy strives to determine asymmetric weights in order to fit a baseline model by regression. Unfortunately, chromatograms characterized by a very high peak saturation pose a significant challenge to such algorithms. In addition, a low signal-to-noise ratio (i.e. s/n<40) also adversely affects accurate baseline correction by asymmetrically weighted regression. We present a baseline estimation method that leverages a probabilistic peak detection algorithm. A posterior probability of being affected by a peak is computed for each point in the chromatogram, leading to a set of weights that allow non-iterative calculation of a baseline estimate. For extremely saturated chromatograms, the peak weighted (PW) method demonstrates notable improvement compared to the other methods examined. However, in chromatograms characterized by low-noise and well-resolved peaks, the asymmetric least squares (ALS) and the more sophisticated Mixture Model (MM) approaches achieve superior results in significantly less time. We evaluate the performance of these three baseline correction methods over a range of chromatographic conditions to demonstrate the cases in which each method is most appropriate.

Modelling crime linkage with Bayesian networks.

When two or more crimes show specific similarities, such as a very distinct modus operandi, the probability that they were committed by the same offender becomes of interest. This probability depends on the degree of similarity and distinctiveness. We show how Bayesian networks can be used to model different evidential structures that can occur when linking crimes, and how they assist in understanding the complex underlying dependencies. That is, how evidence that is obtained in one case can be used in another and vice versa. The flip side of this is that the intuitive decision to "unlink" a case in which exculpatory evidence is obtained leads to serious overestimation of the strength of the remaining cases.

Class-conditional feature modeling for ignitable liquid classification with substantial substrate contribution in fire debris analysis.

Forensic chemical analysis of fire debris addresses the question of whether ignitable liquid residue is present in a sample and, if so, what type. Evidence evaluation regarding this question is complicated by interference from pyrolysis products of the substrate materials present in a fire. A method is developed to derive a set of class-conditional features for the evaluation of such complex samples. The use of a forensic reference collection allows characterization of the variation in complex mixtures of substrate materials and ignitable liquids even when the dominant feature is not specific to an ignitable liquid. Making use of a novel method for data imputation under complex mixing conditions, a distribution is modeled for the variation between pairs of samples containing similar ignitable liquid residues. Examining the covariance of variables within the different classes allows different weights to be placed on features more important in discerning the presence of a particular ignitable liquid residue. Performance of the method is evaluated using a database of total ion spectrum (TIS) measurements of ignitable liquid and fire debris samples. These measurements include 119 nominal masses measured by GC-MS and averaged across a chromatographic profile. Ignitable liquids are labeled using the American Society for Testing and Materials (ASTM) E1618 standard class definitions. Statistical analysis is performed in the class-conditional feature space wherein new forensic traces are represented based on their likeness to known samples contained in a forensic reference collection. The demonstrated method uses forensic reference data as the basis of probabilistic statements concerning the likelihood of the obtained analytical results given the presence of ignitable liquid residue of each of the ASTM classes (including a substrate only class). When prior probabilities of these classes can be assumed, these likelihoods can be connected to class probabilities. In order to compare the performance of this method to previous work, a uniform prior was assumed, resulting in an 81% accuracy for an independent test of 129 real burn samples.

Error rates in forensic DNA analysis: definition, numbers, impact and communication.

Forensic DNA casework is currently regarded as one of the most important types of forensic evidence, and important decisions in intelligence and justice are based on it. However, errors occasionally occur and may have very serious consequences. In other domains, error rates have been defined and published. The forensic domain is lagging behind concerning this transparency for various reasons. In this paper we provide definitions and observed frequencies for different types of errors at the Human Biological Traces Department of the Netherlands Forensic Institute (NFI) over the years 2008-2012. Furthermore, we assess their actual and potential impact and describe how the NFI deals with the communication of these numbers to the legal justice system. We conclude that the observed relative frequency of quality failures is comparable to studies from clinical laboratories and genetic testing centres. Furthermore, this frequency is constant over the five-year study period. The most common causes of failures related to the laboratory process were contamination and human error. Most human errors could be corrected, whereas gross contamination in crime samples often resulted in irreversible consequences. Hence this type of contamination is identified as the most significant source of error. Of the known contamination incidents, most were detected by the NFI quality control system before the report was issued to the authorities, and thus did not lead to flawed decisions like false convictions. However in a very limited number of cases crucial errors were detected after the report was issued, sometimes with severe consequences. Many of these errors were made in the post-analytical phase. The error rates reported in this paper are useful for quality improvement and benchmarking, and contribute to an open research culture that promotes public trust. However, they are irrelevant in the context of a particular case. Here case-specific probabilities of undetected errors are needed. These should be reported, separately from the match probability, when requested by the court or when there are internal or external indications for error. It should also be made clear that there are various other issues to consider, like DNA transfer. Forensic statistical models, in particular Bayesian networks, may be useful to take the various uncertainties into account and demonstrate their effects on the evidential value of the forensic DNA results.