The logic of forensic pathology opinion.

Evaluating evidence and providing opinions are at the heart of forensic science, and forensic experts are expected to provide opinions that are based on logically sound and transparent scientific reasoning, and that honour the boundaries of their area of expertise. In order to meet these objectives, many fields of science explicitly apply Bayes' theorem, which describes the logically correct way to update probabilities on the basis of observations. Making a distinction between 'investigative' and evaluative' modes of operating helps to implement the theorem into daily casework. Use of these principles promotes the logic and transparency of the reasoning that leads to expert's opinion and helps the expert to stay within her remit. Despite these important benefits, forensic pathology seems slow to adopt these principles. In this article, we explore this issue and suggest a way forward. We start with a short introduction to Bayes' theorem and its benefits, followed by a discussion of why its application is actually second nature to medical practitioners. We then discuss the difference between investigative and evaluative opinions, and how they enable the forensic pathologist to reconcile Bayes' theorem with the different phases of a forensic investigation. Throughout the text, practical examples illustrate the various ways in which the logically correct way of evidence interpretation can be implemented, and how it may help the forensic pathologist to provide an appropriate and relevant opinion.

Providing a Forensic Expert Opinion on the "Degree of Force": Evidentiary Considerations.

Forensic pathologists and anthropologists are often asked in court for an opinion about the degree of force required to cause a specific injury. This paper examines and discusses the concept of 'degree of force' and why it is considered a pertinent issue in legal proceedings. This discussion identifies the implicit assumptions that often underpin questions about the 'degree of force'. The current knowledge base for opinions on the degree of force is then provided by means of a literature review. A critical appraisal of this literature shows that much of the results from experimental research is of limited value in routine casework. An alternative approach to addressing the issue is provided through a discussion of the application of Bayes' theorem, also called the likelihood ratio framework. It is argued that the use of this framework makes it possible for an expert to provide relevant and specific evidence, whilst maintaining the boundaries of their field of expertise.

Firearm examination: Examiner judgments and computer-based comparisons.

Forensic firearm examination provides the court of law with information about the source of fired cartridge cases. We assessed the validity of source decisions of a computer-based method and of 73 firearm examiners who compared breechface and firing pin impressions of 48 comparison sets. We also compared the computer-based method's comparison scores with the examiners' degree-of-support judgments and assessed the validity of the latter. The true-positive rate (sensitivity) and true-negative rate (specificity) of the computer-based method (for the comparison of both the breechface and firing pin impressions) were 94.4% and at least 91.7%, respectively. For the examiners, the true-positive rate was at least 95.3% and the true-negative rate was at least 86.2%. The validity of the source decisions improved when the evaluations of breechface and firing pin impressions were combined and for the examiners also when the perceived difficulty of the comparison decreased. The examiners were reluctant to provide source decisions for "difficult" comparisons even though their source decisions were mostly correct. The correlation between the computer-based method's comparison scores and the examiners' degree-of-support judgments was low for the same-source comparisons to negligible for the different-source comparisons. Combining the outcomes of computer-based methods with the judgments of examiners could increase the validity of firearm examinations. The examiners' numerical degree-of-support judgments for their source decisions were not well-calibrated and showed clear signs of overconfidence. We suggest studying the merits of performance feedback to calibrate these judgments.

Cognitive biases in the peer review of bullet and cartridge case comparison casework: A field study.

OBJECTIVE: Forensic judgments and their peer review are often the result of human assessment and are thus subjective and prone to bias. This study examined whether bias affects forensic peer review. HYPOTHESES: We hypothesized that the probability of disagreement between two forensic examiners about the proposed conclusion would be higher with "blind" peer review (reviewer saw only the first examiner's comparison photos) than with "non-blind" peer review (reviewer also saw the first examiner's interpretation and proposed conclusion). We also hypothesized that examiners with a higher perceived professional status would have a larger effect on the reported conclusion than examiners with a lower status. METHOD: We acquired data during a non-blind and a blind peer review procedure in a naturalistic, covert study with eight examiners (3-26 years of experience). We acquired 97 conclusions of bullet and cartridge case comparisons in the blind and 471 in the non-blind peer review procedure. RESULTS: The odds of disagreement between examiners about the evidential strength of a comparison were approximately five times larger (95%-CI [3.06, 8.50]) in the blind than in the non-blind procedure, with disagreement about 42.3% and 12.5% of the proposed conclusions, respectively. Also, the odds that their proposed conclusion was reported as the final conclusion were approximately 2.5 higher for the higher-status examiners than for lower-status examiners. CONCLUSIONS: Our results support both the hypothesis that bias occurs during non-blind forensic peer review and the hypothesis that higher-status examiners determine the outcome of a discussion more than lower-status examiners. We conclude that blind peer review may reduce the probability of bias and that status effects have an impact on the peer reviewing process.

Assessing the frequency of general fingerprint patterns by fingerprint examiners and novices.

The rarity of general fingerprint patterns should be taken into account in the assessment of fingerprint evidence to provide a more complete assessment of fingerprint evidence than when only considering the minutiae. This should be done because, the rarer the corresponding pattern, the stronger the support for the hypothesis that the fingermark stems from the same source as the reference fingerprint. Fingerprint examiners' experience should enable them to provide meaningful assessments of the frequencies of these general patterns according to the theories of perceptual learning, exemplar theory of categorization and visual statistical learning. In this study we examined the accuracy of fingerprint examiners' and novices' judgments on the rarity of general fingerprint patterns. We found that fingerprint examiners seem to have acquired some knowledge about the rarity of general patterns, but had difficulty expressing this knowledge quantitatively using a novel sub-classification of general patterns. As a consequence, their judgments were not accurate and they did not perform better on this task than novices. For both participant groups judgments of more common patterns were more accurate. However, examiners did outperform novices in rank ordering general patterns from common to rare. We conclude that our study does not show that fingerprint examiners have expertise in explicitly judging frequencies of novel sub-classifications of general fingerprint patterns, but our results do indicate that the examiners have acquired knowledge about the rarity of patterns that novices do not possess.

Combining evidence in complex cases - a practical approach to interdisciplinary casework.

Activity level evaluations, although still a major challenge for many disciplines, bring a wealth of possibilities for a more formal approach to the evaluation of interdisciplinary forensic evidence. This paper proposes a practical methodology for combining evidence from different disciplines within the likelihood ratio framework. Evidence schemes introduced in this paper make the process of combining evidence more insightful and intuitive thereby assisting experts in their interdisciplinairy evaluation and in explaining this process to the courts. When confronted with two opposing scenarios and multiple types of evidence, the likelihood ratio approach allows experts to combine this evidence in a probabilistic manner. Parts of the prosecution and defence scenarios for which forensic science is expected to be informative are identified. For these so called core elements, activity level propositions are formulated. Afterwards evidence schemes are introduced to assist the expert in combining the evidence in a logical manner. Two types of evidence relations are identified: serial and parallel evidence. Practical guidelines are given on how to deal with both types of evidence relations when combining the evidence.

Validity and reliability of forensic firearm examiners.

Forensic firearm examiners compare the features in cartridge cases to provide a judgment addressing the question about their source: do they originate from one and the same or from two different firearms? In this article, the validity and reliability of these judgments is studied and compared to the outcomes of a computer-based method. The features we looked at were the striation patterns of the firing pin aperture shear marks of four hundred test shots from two hundred Glock pistols, which were compared by a computer-based method. Sixty of the resulting 79,800 comparisons were shown to 77 firearm examiners. They were asked to judge whether the cartridge cases had the same source or a different source, and to indicate the degree of support the evidence provided for those judgments. The results show that the true positive rates (sensitivity) and the true negative rates (specificity) of firearm examiners are quite high. The examiners seem to be slightly less proficient at identifying same-source comparisons correctly, while they outperform the used computer-based method at identifying different-source comparisons. The judged degrees of support by examiners who report likelihood ratios are not well-calibrated. The examiners are overconfident, giving judgments of evidential strength that are too high. The judgments of the examiners and the outcomes of the computer-based method are only moderately correlated. We suggest to implement performance feedback to reduce overconfidence, to improve the calibration of degree of support judgments, and to study the possibility of combining the judgments of examiners and the outcomes of computer-based methods to increase the overall validity.

A response to "Likelihood ratio as weight of evidence: A closer look" by Lund and Iyer.

Recently, Lund and Iyer (L&I) raised an argument regarding the use of likelihood ratios in court. In our view, their argument is based on a lack of understanding of the paradigm. L&I argue that the decision maker should not accept the expert's likelihood ratio without further consideration. This is agreed by all parties. In normal practice, there is often considerable and proper exploration in court of the basis for any probabilistic statement. We conclude that L&I argue against a practice that does not exist and which no one advocates. Further we conclude that the most informative summary of evidential weight is the likelihood ratio. We state that this is the summary that should be presented to a court in every scientific assessment of evidential weight with supporting information about how it was constructed and on what it was based.

Response paper to "The likelihood of encapsulating all uncertainty": The relevance of additional information for the LR.

In this response paper, part of the Virtual Special Issue on "Measuring and Reporting the Precision of Forensic Likelihood Ratios", we further develop our position on likelihood ratios which we described previously in Berger et al. (2016) "The LR does not exist". Our exposition is inspired by an example given in Martire et al. (2016) "On the likelihood of encapsulating all uncertainty", where the consequences of obtaining additional information on the LR were discussed. In their example, two experts use the same data in a different way, and the LRs of these experts change differently when new data are taken into account. Using this example as a starting point we will demonstrate that the probability distribution for the frequency of the characteristic observed in trace and reference material can be used to predict how much an LR will change when new data become available. This distribution can thus be useful for such a sensitivity analysis, and address the question of whether to obtain additional data or not. But it does not change the answer to the original question of how to update one's prior odds based on the evidence, and it does not represent an uncertainty on the likelihood ratio based on the current data.

Activity level DNA evidence evaluation: On propositions addressing the actor or the activity.

More often than not, the source of DNA traces found at a crime scene is not disputed, but the activity or timing of events that resulted in their transfer is. As a consequence, practitioners are increasingly asked to assign a value to DNA evidence given propositions about activities provided by prosecution and defense counsel. Given that the dispute concerns the nature of the activity that took place or the identity of the actor that carried out the activity, several factors will determine how to formulate the propositions. Determining factors are (1) whether defense claims the crime never took place, (2) whether defense claims someone other than the accused (either an unknown individual or a known person) performed the criminal activity, and (3) whether it is claimed and disputed that the suspect performed an alternative, legitimate activity or has a relation to the victim, the object, or the scene of crime that implies a legitimate interaction. Addressing such propositions using Bayesian networks, we demonstrate the effects of the various proposition sets on the evaluation of the evidence.

The LR does not exist.

More than 40years ago, De Finetti warned that probability is a misleading misconception when regarded as objectively existing exterior to the mind. According to De Finetti, probabilities are necessarily subjective, and quantify our belief in the truth of events in the real world. Given evidence of a shared feature of a trace and an accused, we apply this framework to assign an evidential value to this correspondence. Dividing 1 by the objectively existing proportion of the population sharing that feature would give that evidential value - expressed as a likelihood ratio (LR) - only if that proportion were known. As in practice the proportion can only be estimated, this leads some to project their sampling uncertainty - or precision - associated with the estimated proportion onto the likelihood ratio, and to report an interval. Limited data should limit our LR however, because as we will demonstrate the LR is given by what we know about the proportion rather than by the unknown proportion itself. Encapsulating all uncertainty - including sampling uncertainty of the proportion - our LR reflects how much information we have retrieved from the feature regarding the trace's origin, based on our present knowledge. Not an interval but a number represents this amount of information, equal to the logarithm of the LR. As long as we know how to interpret the evidence with a well-defined probabilistic model, we know what our evidence is worth.

Measuring coherence of computer-assisted likelihood ratio methods.

Measuring the performance of forensic evaluation methods that compute likelihood ratios (LRs) is relevant for both the development and the validation of such methods. A framework of performance characteristics categorized as primary and secondary is introduced in this study to help achieve such development and validation. Ground-truth labelled fingerprint data is used to assess the performance of an example likelihood ratio method in terms of those performance characteristics. Discrimination, calibration, and especially the coherence of this LR method are assessed as a function of the quantity and quality of the trace fingerprint specimen. Assessment of the coherence revealed a weakness of the comparison algorithm in the computer-assisted likelihood ratio method used.

A more straightforward derivation of the LR for a database search.

Matching DNA profiles of an accused person and a crime scene trace are one of the most common forms of forensic evidence. A number of years ago the so-called 'DNA controversy' was concerned with how to quantify the value of such evidence. Given its importance, the lack of understanding of such a basic issue was quite surprising and concerning. Deriving the equation for the likelihood ratio of a DNA database match in a much more direct and simple way is the topic of this paper. As it is much easier to follow it is hoped that this derivation will contribute to the understanding.

Objective ink color comparison through image processing and machine learning.

Making changes or additions to written entries in a document can be profitable and illegal at the same time. A simple univariate approach is first used in this paper to quantify the evidential value in color measurements for inks on a document coming from a different or the same source. Graphic, qualitative discrimination is then obtained independently by applying color deconvolution image processing to document images, with parameters optionally optimized by support vector machines (SVM), a machine learning method. Discrimination based on qualitative results from image processing is finally compared to the quantitative results of the statistical approach. As color differences increase, optimized color deconvolution achieves qualitative discrimination when the statistical approach indicates evidence for the different source hypothesis.

Objective paper structure comparison: assessing comparison algorithms.

More than just being a substrate, paper can also provide evidence for the provenance of documents. An earlier paper described a method to compare paper structure, based on the Fourier power spectra of light transmission images. Good results were obtained by using the 2D correlation of images derived from the power spectra as a similarity score, but the method was very computationally intensive. Different comparison algorithms are evaluated in this paper, using information theoretical criteria. An angular invariant algorithm turned out to be as effective as the original one but 4 orders of magnitude faster, making the use of much larger databases possible.

Evidence evaluation: a response to the court of appeal judgment in R v T.

This is a discussion of a number of issues that arise from the recent judgment in R v T [1]. Although the judgment concerned with footwear evidence, more general remarks have implications for all disciplines within forensic science. Our concern is that the judgment will be interpreted as being in opposition to the principles of logical interpretation of evidence. We reiterate those principles and then discuss several extracts from the judgment that may be potentially harmful to the future of forensic science. A position statement with regard to evidence evaluation, signed by many forensic scientists, statisticians and lawyers, has appeared in this journal [2] and the present paper expands on the points made in that statement.