A Logical Framework for Forensic DNA Interpretation.

The forensic community has devoted much effort over the last decades to the development of a logical framework for forensic interpretation, which is essential for the safe administration of justice. We review the research and guidelines that have been published and provide examples of how to implement them in casework. After a discussion on uncertainty in the criminal trial and the roles that the DNA scientist may take, we present the principles of interpretation for evaluative reporting. We show how their application helps to avoid a common fallacy and present strategies that DNA scientists can apply so that they do not transpose the conditional. We then discuss the hierarchy of propositions and explain why it is considered a fundamental concept for the evaluation of biological results and the differences between assessing results given propositions that are at the source level or the activity level. We show the importance of pre-assessment, especially when the questions relate to the alleged activities, and when transfer and persistence need to be considered by the scientists to guide the court. We conclude with a discussion on statement writing and testimony. This provides guidance on how DNA scientists can report in a balanced, transparent, and logical way.

Use of Bayesian Networks for the investigation of the nature of biological material in casework.

Chemical and staining methods, immunochromatography, spectroscopy, RNA expression or methylation patterns, do not allow to determine the nature of the biological material with certainty. However, to our knowledge, there are few forensic scientists that assess the value of such test results using a probabilistic approach. This is surprising as it would allow account for false positives and false negatives and avoid misleading conclusions. In this paper, we developed three Bayesian Networks (BNs) to assess the presence of blood, saliva and sperm in the recovered material and combine potentially contradictory observations. The approach was successfully tested using 188 traces from proficiency tests. We have implemented an online user-friendly application (https://forensic-genetic.shinyapps.io/BodyFluidsApp/) that allows forensic scientists to assess the value of their results without having to build Bayesian Networks themselves. They can also input their own data, use the application to identify a potential lack of knowledge and report their conclusions regarding the presence of sperm, blood or/and saliva considering uncertainty.

A mixed DNA profile controversy revisited.

Semaan et al. (J Forensic Res, 2020, 11, 453) discuss a mock case "where eight different individuals [P1 through P8 ] could not be excluded in a mixed DNA analysis. Even though … expert DNA mixture analysis software was used." Two of these are the true donors. The LRs reported are incorrect due to the incorrect entry of propositions into LRmix Studio. This forced the software to account for most of the alleles as drop-in, resulting in LRs 60-70 orders of magnitude larger than expected. P1 , P2 , P4 , P5 , and P8 can be manually excluded using peak heights. This has relevance when using LRmix which does not use peak heights. We extend the work using the same two reference genotypes who were the true contributors as Semaan et al. (J Forensic Res, 2020, 11, 453). We simulate three two-donor mixtures with peak heights using these two genotypes and analyze using STRmix™. For the simulated 1:1 mixture, one of the non-donors' LRs supported him being a contributor when no conditioning was used. When considered in combination with any other potential donors (i.e., with conditioning), this non-donor was correctly eliminated. For the 3:1 mixture, all results correctly supported that the non-donors were not contributors. The low-template 4:1 mixture LRs with no conditioning showed support for all eight profiles as donors. However, the results from pair-wise conditioning showed that only the two ground truth donors had LRs supporting that they were contributors to the mixture. We recommend the use of peak heights and conditioning profiles, as this allows better sensitivity and specificity even when the persons share many alleles.

A Swiss collaborative exercise for Disaster Victim Identification (DVI): Covering situations with different levels of complexity.

The identification of victims of a disaster (DVI) requires the collaboration of different specialists. Within a DVI context, DNA analyses often play an important role. Consequently, forensic genetic laboratories should be prepared to cope with DVI situations, as this can involve large-scale DNA profile comparisons. Six forensic genetic laboratories from Switzerland participated in an exercise where supposedly a plane had crashed. The goal of the exercise was to monitor participants use of dedicated software with ground truth cases and to make them aware of the existence of particular situations that may occur in real cases. For assigning the value of the comparison of the DNA profiles, all participating laboratories used the DVI module of Familias v3.2.1 In addition, one of the 6 laboratories used the Pedigree Searcher from CODIS v7.0. The data (AmpFlSTR® NGM SElect™ profiles) were generated to challenge the participating laboratories: cases with first, second degree biological parents, mutation events, as well as non-paternity cases were included. This study shows that the majority of the participants used the software in an appropriate way. However, a few misleading conclusions were detected for the most challenging situations. These errors belonged to one of the following categories: false pedigree, false association using the higher LR, misleading contextual information (false paternity) and not clustering family members. Specific recommendations are provided in order to reduce misuse of the software and the risk of misinterpretations by using all the relevant information.

Comparing multiple POI to DNA mixtures.

In casework, laboratories may be asked to compare DNA mixtures to multiple persons of interest (POI). Guidelines on forensic DNA mixture interpretation recommend that analysts consider several pairs of propositions; however, it is unclear if several likelihood ratios (LRs) per person should be reported or not. The propositions communicated to the court should not depend on the value of the LR. As such, we suggest that the propositions should be functionally exhaustive. This implies that all propositions with a non-zero prior probability need to be considered, at least initially. Those that have a significant posterior probability need to be used in the final evaluation. Using standard probability theory we combine various propositions so that collectively they are exhaustive. This involves a prior probability that the sub-proposition is true, given that the primary proposition is true. Imagine a case in which there are two possible donors: i and j. We focus our analysis first on donor i so that the primary proposition is that i is one of the sources of the DNA. In this example, given that i is a donor, we would further consider that j is either a donor or not. In practice, the prior weights for these sub-propositions may be difficult to assign. However, the LR is often linearly related to these priors and its behaviour is predictable. We also believe that these priors are unavoidable and are hidden in alternative methods. We term the likelihood ratio formed from these context-exhaustive propositions LRi/i¯. LRi/i¯ is trialed in a set of two- and three-person mixtures. For two-person mixtures, LRi/i¯ is often well approximated by LRij/ja, where the subscript ij describes the proposition that i and j are the donors and ja describes the proposition that j and an alternate, unknown individual (a), who is unrelated to both i and j, are the donors. For three-person mixtures, LRi/i¯ is often well approximated by LRijk/jka where the subscript ijk describes the proposition that i, j, and k are the donors and jka describes the proposition that j, k, and an unknown, unrelated (to i, j, and k) individual (a) are the donors. In our simulations, LRij/ja had fewer inclusionary LRs for non-contributors than the unconditioned LR (LRia/aa).

When evaluating DNA evidence within a likelihood ratio framework, should the propositions be exhaustive?

We seek to develop a rational approach to forming propositions when little information is available from the outset, as this often happens in casework. If propositions used when evaluating evidence are not exhaustive (in the context of the case), then there is a theoretical risk that an LR greater than one may be associated with a proposition in the numerator that - if all meaningful propositions had been considered - would in fact have a lower posterior probability after consideration of the evidence. Ideally, all propositions should be considered. However, with multiple propositions, some terms will be larger than others and for simplification very small terms can be neglected without changing the order of magnitude of the value of the evidence (i.e. LR). Our analysis shows that mathematically a contributor's DNA can be assumed to be present under both prosecution and alternative propositions (Hp and Ha) if there is a reasonable prior probability of their DNA being present and their inclusion is supported by the profile. This is because the terms associated to these sub-propositions will dominate our LR. For example, in the absence of specific information, when considering two persons of interest (POI) as potential contributors to a mixed DNA profile we suggest the assumption of one when examining the presence of the other, after checking that both collectively explain the profile well. This represents more meaningful propositions and allows better discrimination. Slooten and Caliebe have shown that the overall LR is the weighted average of LRs with the same number of contributors (NoC) under both propositions. The weights involve both an assessment of the probability of the crime scene DNA profile and the probability of this NoC given the background information.

A review of likelihood ratios in forensic science based on a critique of Stiffelman "No longer the Gold standard: Probabilistic genotyping is changing the nature of DNA evidence in criminal trials".

Stiffelman [1] gives a broad critique of the application of likelihood ratios (LRs) in forensic science, in particular their use in probabilistic genotyping (PG) software. These are discussed in this review. LRs do not infringe on the ultimate issue. The Bayesian paradigm clearly separates the role of the scientist from that of the decision makers and distances the scientist from comment on the ultimate and subsidiary issues. LRs do not affect the reasonable doubt standard. Fact finders must still make decisions based on all the evidence and they must do this considering all evidence, not just that given probabilistically. LRs do not infringe on the presumption of innocence. The presumption of innocence does not equate with a prior probability of zero but simply that the person of interest (POI) is no more likely than anyone else to be the donor. Propositions need to be exhaustive within the context of the case. That is, propositions deemed relevant by either defense or prosecution which are not fanciful must not be omitted from consideration.

DNA commission of the International society for forensic genetics: Assessing the value of forensic biological evidence - Guidelines highlighting the importance of propositions. Part II: Evaluation of biological traces considering activity level propositions.

The value of the evidence depends critically on propositions. In the second of two papers intended to provide advice to the community on difficult aspects of evaluation and the formulation of propositions, we focus primarily on activity level propositions. This helps the court address the question of "How did an individual's cell material get there?". In order to do this, we expand the framework outlined in the first companion paper. First, it is important not to conflate results and propositions. Statements given activity level propositions aim to help address issues of indirect vs direct transfer, and the time of the activity, but it is important to avoid use of the word 'transfer' in propositions. This is because propositions are assessed by the Court, but DNA transfer is a factor that scientists need to take into account for the interpretation of their results. Suitable activity level propositions are ideally set before knowledge of the results and address issues like: X stabbed Y vs. an unknown person stabbed Y but X met Y the day before. The scientist assigns the probability of the evidence, if each of the alternate propositions is true, to derive a likelihood ratio. To do this, the scientist asks: a) "what are the expectations if each of the propositions is true?" b) "What data are available to assist in the evaluation of the results given the propositions?" When presenting evidence, scientists work within the hierarchy of propositions framework. The value of evidence calculated for a DNA profile cannot be carried over to higher levels in the hierarchy - the calculations given sub-source, source and activity level propositions are all separate. A number of examples are provided to illustrate the principles espoused, and the criteria that such assessments should meet. Ideally in order to assign probabilities, the analyst should have/collect data that are relevant to the case in question. These data must be relevant to the case at hand and we encourage further research and collection of data to form knowledge bases. Bayesian Networks are extremely useful to help us think about a problem, because they force us to consider all relevant possibilities in a logical way. An example is provided.

Structuring cases into propositions, assumptions, and undisputed case information.

In this short publication we address the topic of evaluation of biological results given activity level propositions, particularly when the source of the biological material is not in question. When the issue regards the mechanisms or actions that led to the deposition of the biological material concerned, there is a need for more case information than when the issue pertains to the source of the DNA. It is up to the scientist to structure the provided case information into propositions, assumptions and undisputed case information. In our statements, deciding what goes in the propositions and what will be part of the paragraph dedicated to case information is partly personal. It may however affect the way we think about a case and thereby have consequences for the evaluation of the results as well as for the communication of the information to the recipient. In particular we highlight the importance of considering and communicating all DNA transfer mechanisms that are relevant to the evaluation under each proposition, and how the way the propositions are thought about (and expressed) can assist in this endeavour.

STRmix™ collaborative exercise on DNA mixture interpretation.

An intra and inter-laboratory study using the probabilistic genotyping (PG) software STRmix™ is reported. Two complex mixtures from the PROVEDIt set, analysed on an Applied Biosystems™ 3500 Series Genetic Analyzer, were selected. 174 participants responded. For Sample 1 (low template, in the order of 200 rfu for major contributors) five participants described the comparison as inconclusive with respect to the POI or excluded him. Where LRs were assigned, the point estimates ranging from 2 × 104 to 8 × 106. For Sample 2 (in the order of 2000 rfu for major contributors), LRs ranged from 2 × 1028 to 2 × 1029. Where LRs were calculated, the differences between participants can be attributed to (from largest to smallest impact): This study demonstrates a high level of repeatability and reproducibility among the participants. For those results that differed from the mode, the differences in LR were almost always minor or conservative.

DNA commission of the International society for forensic genetics: Assessing the value of forensic biological evidence - Guidelines highlighting the importance of propositions: Part I: evaluation of DNA profiling comparisons given (sub-) source propositions.

The interpretation of evidence continues to be one of the biggest challenges facing the forensic community. This is the first of two papers intended to provide advice on difficult aspects of evaluation and in particular on the formulation of propositions. The scientist has a dual role: investigator (crime-focused), where often there is no suspect available and a database search may be required; evaluator (suspect-focused), where the strength of evidence is assessed in the context of the case. In investigative mode, generally the aim is to produce leads regarding the source of the DNA. Sub-source level propositions will be adequate to help identify potential suspects who can be further investigated by the authorities. Once in evaluative mode, given the defence version of events of the person of interest, it may become necessary to consider alternatives that go beyond the source of the DNA (i.e., to consider activity level propositions). In the evaluation phase, it is crucial that formulation of propositions allows the assessment of all the results that will help with the issue at hand. Propositions should therefore be precise (indication of the number of contributors, information on the relevant population etc.), be about causes, not effects (e.g. a 'matching' DNA profile) and to avoid bias, must not be findings-led. This means that ideally, propositions should be decided based on the case information and before the results of the comparisons are known. This paper primarily reflects upon what has been coined as "sub-source level propositions". These are restricted to the evaluation of the DNA profiles themselves, and help answer the issue regarding the source of the DNA. It is to be emphasised that likelihood ratios given sub-source level propositions cannot be carried over to a different level - for example, activity level propositions, where the DNA evidence is put into the context of the alleged activities. This would be highly misleading and could give rise to miscarriages of justice; this will be discussed in a second paper. The value of forensic results depends not only on propositions, but also on the type of results (e.g. allelic designations, peak heights, replicates) and upon the model used: it is therefore important to discuss these aspects. Finally, since communication is key to help understanding by courts, we will explore how to convey the value of the results and explain the importance of avoiding the practice of transposing the conditional.

A template for constructing Bayesian networks in forensic biology cases when considering activity level propositions.

The hierarchy of propositions has been accepted amongst the forensic science community for some time. It is also accepted that the higher up the hierarchy the propositions are, against which the scientist are competent to evaluate their results, the more directly useful the testimony will be to the court. Because each case represents a unique set of circumstances and findings, it is difficult to come up with a standard structure for evaluation. One common tool that assists in this task is Bayesian networks (BNs). There is much diversity in the way that BN can be constructed. In this work, we develop a template for BN construction that allows sufficient flexibility to address most cases, but enough commonality and structure that the flow of information in the BN is readily recognised at a glance. We provide seven steps that can be used to construct BNs within this structure and demonstrate how they can be applied, using a case example.

What is the error margin of your signature analysis?

In our experience, it seems to become more and more common for mandating authorities or parties to ask forensic signature examiners to quantify the degree of certainty of their conclusion regarding a signature analysis. This paper reports the likelihood ratio approach followed by examiners to answer such a question, in a case where the Court asked whether a questioned signature was written, or not, by Mr Jones. The Court also required an assessment of the error margin of the signature analysis. This question was answered using Bayes' theorem (i.e., a full Bayesian approach) and this paper seeks to show that such an approach can be used despite the popular belief that Bayes' theorem is beyond what courts may accept. Using a practical example, we present advantages of the approach we have chosen to assess our results and show that a logical approach for evidence evaluation can be followed even in a forensic discipline where no tabulated data are available. This example also illustrates a practical way of addressing the error margin question, which helps the Court understand what can be the risk of being wrong in this particular case (and not in cases in general). We further present the way these results were communicated to the fact finders in the case at hand and provide guidance as how forensic observations can logically be combined with the other elements of the case.

Helping to distinguish primary from secondary transfer events for trace DNA.

DNA is routinely recovered in criminal investigations. The sensitivity of laboratory equipment and DNA profiling kits means that it is possible to generate DNA profiles from very small amounts of cellular material. As a consequence, it has been shown that DNA we detect may not have arisen from a direct contact with an item, but rather through one or more intermediaries. Naturally the questions arising in court, particularly when considering trace DNA, are of how DNA may have come to be on an item. While scientists cannot directly answer this question, forensic biological results can help in discriminating between alleged activities. Much experimental research has been published showing the transfer and persistence of DNA under varying conditions, but as of yet the results of these studies have not been combined to deal with broad questions about transfer mechanisms. In this work we use published data and Bayesian networks to develop a statistical logical framework by which questions of transfer mechanism can be approached probabilistically. We also identify a number of areas where further work could be carried out in order to improve our knowledge base when helping to address questions about transfer mechanisms. Finally, we apply the constructed Bayesian network to ground truth known data to determine if, with current knowledge, there is any power in DNA quantities to distinguish primary and secondary transfer events.

Evaluation of Forensic DNA Traces When Propositions of Interest Relate to Activities: Analysis and Discussion of Recurrent Concerns.

When forensic scientists evaluate and report on the probative strength of single DNA traces, they commonly rely on only one number, expressing the rarity of the DNA profile in the population of interest. This is so because the focus is on propositions regarding the source of the recovered trace material, such as "the person of interest is the source of the crime stain." In particular, when the alternative proposition is "an unknown person is the source of the crime stain," one is directed to think about the rarity of the profile. However, in the era of DNA profiling technology capable of producing results from small quantities of trace material (i.e., non-visible staining) that is subject to easy and ubiquitous modes of transfer, the issue of source is becoming less central, to the point that it is often not contested. There is now a shift from the question "whose DNA is this?" to the question "how did it get there?" As a consequence, recipients of expert information are now very much in need of assistance with the evaluation of the meaning and probative strength of DNA profiling results when the competing propositions of interest refer to different activities. This need is widely demonstrated in day-to-day forensic practice and is also voiced in specialized literature. Yet many forensic scientists remain reluctant to assess their results given propositions that relate to different activities. Some scientists consider evaluations beyond the issue of source as being overly speculative, because of the lack of relevant data and knowledge regarding phenomena and mechanisms of transfer, persistence and background of DNA. Similarly, encouragements to deal with these activity issues, expressed in a recently released European guideline on evaluative reporting (Willis et al., 2015), which highlights the need for rethinking current practice, are sometimes viewed skeptically or are not considered feasible. In this discussion paper, we select and discuss recurrent skeptical views brought to our attention, as well as some of the alternative solutions that have been suggested. We will argue that the way forward is to address now, rather than later, the challenges associated with the evaluation of DNA results (from small quantities of trace material) in light of different activities to prevent them being misrepresented in court.

Discussion on how to implement a verbal scale in a forensic laboratory: Benefits, pitfalls and suggestions to avoid misunderstandings.

In a recently published guideline for evaluative reporting in forensic science, the European Network of Forensic Science Institutes (ENFSI) recommended the use of the likelihood ratio for the measurement of the value of forensic results. As a device to communicate the probative value of the results, the ENFSI guideline mentions the possibility to define and use a verbal scale, which should be unified within a forensic institution. This paper summarizes discussions held between scientists of our institution to develop and implement such a verbal scale. It intends to contribute to general discussions likely to be faced by any forensic institution that engages in continuous monitoring and improving of their evaluation and reporting format. We first present published arguments in favour of the use of such verbal qualifiers. We emphasise that verbal qualifiers do not replace the use of numbers to evaluate forensic findings, but are useful to communicate the probative value, since the weight of evidence in terms of likelihood ratio are still apprehended with difficulty by both the forensic scientists, especially in the absence of hard data, and the recipient of information. We further present arguments that support the development of the verbal scale that we propose. Recognising the limits of the use of such a verbal scale, we then discuss its disadvantages: it may lead to the spurious view according to which the value of the observations made in a given case is relative to other cases. Verbal qualifiers are also prone to misunderstandings and cannot be coherently combined with other evidence. We therefore recommend not using the verbal qualifier alone in a written statement. While scientists should only report on the probability of the findings - and not on the probability of the propositions, which are the duty of the Court - we suggest showing examples to let the recipient of information understand how the scientific evidence affects the probabilities of the propositions. To avoid misunderstandings, we also advise to mention in the statement what the results do not mean. Finally, we are of the opinion that if experts were able to coherently articulate numbers, and if recipients of information could properly handle such numbers, then verbal qualifiers could be abandoned completely. At that time, numerical expressions of probative value will be appropriately understood, as other numerical measures that most of us understand without the need of any further explanation, such as expressions for length or temperature.

Using sensitivity analyses in Bayesian Networks to highlight the impact of data paucity and direct future analyses: a contribution to the debate on measuring and reporting the precision of likelihood ratios.

Bayesian networks are being increasingly used to address complex questions of forensic interest. Like all probabilities, those that underlie the nodes within a network rely on structured data and knowledge. Obviously, the more structured data we have, the better. But, in real life, the numbers of experiments that can be carried out are limited. It is thus important to know if/when our knowledge is sufficient and when one needs to perform further experiments to be in a position to report the value of the observations made. To explore the impact of the amount of data that are available for assessing results, we have constructed Bayesian Networks and explored the sensitivity of the likelihood ratios to changes to the data that underlie each node. Bayesian networks are constructed and sensitivity analyses performed using freely available R libraries (gRain and BNlearn). We demonstrate how the analyses can be used to yield information about the robustness provided by the data used to inform the conditional probability table, and also how they can be used to direct further research for maximum effect. By maximum effect, we mean to contribute with the least investment to an increased robustness. In addition, the paper investigates the consequences of the sensitivity analysis to the discussion on how the evidence shall be reported for a given state of knowledge in terms of underpinning data.

Evaluation of infrared spectra analyses using a likelihood ratio approach: A practical example of spray paint examination.

Depending on the forensic disciplines and on the analytical techniques used, Bayesian methods of evaluation have been applied both as a two-step approach (first comparison, then evaluation) and as a continuous approach (comparison and evaluation in one step). However in order to use the continuous approach, the measurements have to be reliably summarized as a numerical value linked to the property of interest, which occurrence can be determined (e.g., refractive index measurement of glass samples). For paint traces analyzed by Fourier transform infrared spectroscopy (FTIR) however, the statistical comparison of the spectra is generally done by a similarity measure (e.g., Pearson correlation, Euclidean distance). Although useful, these measures cannot be directly associated to frequencies of occurrence of the chemical composition (binders, extenders, pigments). The continuous approach as described above is not possible, and a two-step evaluation, 1) comparison of the spectra and 2) evaluation of the results, is therefore the common practice reported in most of the laboratories. Derived from a practical question that arose during casework, a way of integrating the similarity measure between spectra into a continuous likelihood ratio formula was explored. This article proposes the use of a likelihood ratio approach with the similarity measure of infrared spectra of spray paints based on distributions of sub-populations given by the color and composition of spray paint cans. Taking into account not only the rarity of paint composition, but also the "quality" of the analytical match provides a more balanced evaluation given source or activity level propositions. We will demonstrate also that a joint statistical-expertal methodology allows for a more transparent evaluation of the results and makes a better use of current knowledge.