Use of automated quality assessment algorithms in fingermark detection research - Application to IND/Zn vs DFO.

When developing detection techniques for fingermarks, the detected fingermarks must be evaluated for their quality to assess the effectiveness of the new method. It is a common practice to compare the performance of the new (optimized) technique with the traditional or well-established ones. In current practice, this evaluation step is carried out by a group of human assessors. A new approach is applied in this paper and consists of using algorithms to perform this task. To implement this approach, the comparison between IND/Zn and DFO has been chosen because it has already been the subject of many articles published in recent years and a consensus exists on the superiority of IND/Zn over DFO. The quality of 3'600 fingermarks developed using both detection techniques was assessed automatically using two algorithms: LQM (Latent Quality Metric) and ILFQM (Improved Latent Fingerprint Quality Metric). The distribution of quality scores was studied for both detection techniques. The results showed that fingermarks detected with IND/Zn received higher scores on average than fingermarks detected with DFO, which is in line with the consensus in the literature based on human assessment. The results of this research are promising and shows that automated fingermark quality assessment is an efficient and viable way to comparatively assess fingermark detection techniques.

Striated toolmarks comparison and reporting methods: Review and perspectives.

Forensic toolmark examiners have been comparing features observed in toolmarks to help determine their source for over a century. However, in the past decade, the holistic process of comparing toolmarks and presenting findings in court have faced intense scrutiny. This paper provides a summary of the voiced criticisms, primarily concerning the scientific reliability and validity of the comparison methods employed by examiners and the conclusions they testify to. The focus of this review is specifically on the examination of striated toolmarks. We assess the comparison methods and reporting practices currently in use, while also delving deeper into research aligned with current recommendations, such as PCAST (The President's Council of Advisors on Science and Technology). Throughout the review, we examine both the strengths and weaknesses of existing practices, aiming to assist practitioners in identifying key research needs and addressing the concerns raised by critics. By doing so, we seek to enhance the credibility and effectiveness of toolmark analysis in the field of forensic science.

A practical treatment of sensitivity analyses in activity level evaluations.

Evaluations of forensic observations considering activity level propositions are becoming more common place in forensic institutions. A measure that can be taken to interrogate the evaluation for robustness is called sensitivity analysis. A sensitivity analysis explores the sensitivity of the evaluation to the data used when assigning probabilities, or to the level of uncertainty surrounding a probability assignment, or to the choice of various assumptions within the model. There have been a number of publications that describe sensitivity analysis in technical terms, and demonstrate their use, but limited literature on how that theory can be applied in practice. In this work we provide some simplified examples of how sensitivity analyses can be carried out, when they are likely to show that the evaluation is sensitive to underlying data, knowledge or assumptions, how to interpret the results of sensitivity analysis, and how the outcome can be reported. We also provide access to an application to conduct sensitivity analysis.

Use of Bayesian Networks for the investigation of the nature of biological material in casework.

Chemical and staining methods, immunochromatography, spectroscopy, RNA expression or methylation patterns, do not allow to determine the nature of the biological material with certainty. However, to our knowledge, there are few forensic scientists that assess the value of such test results using a probabilistic approach. This is surprising as it would allow account for false positives and false negatives and avoid misleading conclusions. In this paper, we developed three Bayesian Networks (BNs) to assess the presence of blood, saliva and sperm in the recovered material and combine potentially contradictory observations. The approach was successfully tested using 188 traces from proficiency tests. We have implemented an online user-friendly application (https://forensic-genetic.shinyapps.io/BodyFluidsApp/) that allows forensic scientists to assess the value of their results without having to build Bayesian Networks themselves. They can also input their own data, use the application to identify a potential lack of knowledge and report their conclusions regarding the presence of sperm, blood or/and saliva considering uncertainty.

Proactive forensic science in biometrics: Novel materials for fingerprint spoofing.

Motivated by the need to prepare for the next generation of fingerprint spoofing, we applied the "proactive forensic science" strategy to the biometric field. The working concept, already successful in a few fields, aimed at adopting the sophisticated criminals' way of thinking, predicting their next move so that the crime-fighting authorities can be one step ahead of them and take preventive measures, against biometric spoofing in this instance. This strategy involved the design, production, and characterization of innovative polymeric materials that could possibly serve in advanced fingerprint spoofs. Special attention was given to materials capable of fooling fingerprint readers equipped with spoof-detecting abilities, known as "Presentation Attack Detection" (PAD) systems and often referred to as liveness detection. A series of direct cast fake fingerprints was produced from known commercially available spoofing materials, and was functionally tested to compare their performance with that of spoofs produced from the new polymers. The novel materials thus prepared were hydrogels based on polyethylene glycols (PEGs) that were chain-extended. They showed good performance in deceiving security systems, considerably better than that of spoofs produced from commercial materials, and are, therefore, good spoofing candidates that law-enforcement authorities should be aware of.

Using the likelihood ratio in bloodstain pattern analysis.

There is an apparent paradox that the likelihood ratio (LR) approach is an appropriate measure of the weight of evidence when forensic findings have to be evaluated in court, while it is typically not used by bloodstain pattern analysis (BPA) experts. This commentary evaluates how the scope and methods of BPA relate to several types of evaluative propositions and methods to which LRs are applicable. As a result of this evaluation, we show how specificities in scope (BPA being about activities rather than source identification), gaps in the underlying science base, and the reliance on a wide range of methods render the use of LRs in BPA more complex than in some other forensic disciplines. Three directions are identified for BPA research and training, which would facilitate and widen the use of LRs: research in the underlying physics; the development of a culture of data sharing; and the development of training material on the required statistical background. An example of how recent fluid dynamics research in BPA can lead to the use of LR is provided. We conclude that an LR framework is fully applicable to BPA, provided methodic efforts and significant developments occur along the three outlined directions.

The importance of considering common sources of unknown DNA when evaluating findings given activity level propositions.

Evaluating forensic biological evidence considering activity level propositions is becoming more prominent around the world. In such evaluations it is common to combine results from multiple items associated with the alleged activities. The results from these items may not be conditionally independent, depending on the mechanism of cell/DNA transfer being considered and it is important that the evaluation takes these dependencies into account. Part of this consideration is to incorporate our understanding of prevalent DNA and of background DNA on objects and people, and how activities can lead to common sources of unknown DNA being deposited on items. We demonstrate a framework for evaluation of DNA evidence in such a scenario using Object-Oriented Bayesian Networks and apply it to a motivating case from South Australia.

Predicting suitability of finger marks using machine learning techniques and examiner annotations.

Previous research has established the variability of examiners in reaching suitability determinations for friction ridge comparisons. Attempts to create predictive models to assist in this determination have been made, but have been largely confined to fully automated processes that focus on suitability for AFIS entry. This work develops, optimizes, and validates a hybrid predictive model that utilizes both examiner-observed variables and automated measures of quality and rarity to arrive at suitability classifications along four scales that have been proposed in our previous research: Value, Complexity, AFIS, and Difficulty. We show that a model based only on automatically extracted quality or selectivity measures does not perform as well as when used in conjunction with a limited set of user inputs. The model is then based on a limited set of input from the users while taking advantage of automatic measures with a view to limit the user encoding effort while maintaining accuracy. The developed model is able to make predictions at up to 83.13% accuracy when using full study data and maintains similar levels of accuracy in an external validation study. The model achieved accuracy at a similar level to that of examiners asked to make the same suitability determinations across all scales. The model can easily be introduced into an operational laboratory with very little additional operational burden to provide guidance on suitability, complexity, AFIS, and quality assurance decisions; to assist in designing testing and training exercises of progressive difficulty; to describe the difficulty of a mark in testimony; and to provide a consensus-based opinion in laboratories where a second opinion is desired but the laboratory lacks sufficient personnel to form a consensus panel.

Mind-set - How bias leads to errors in friction ridge comparisons.

Mind-set is a term used in the friction ridge discipline to describe a confirmation bias in which an examiner makes early decisions about their interpretation of a mark but fails to update or reconsider those decisions in light of additional information. This most often occurs during the analysis of a mark when an examiner makes decisions (such as orientation or anatomical source of a mark) to help expedite a manual search or set parameters for an automated search, but fails to re-evaluate these decisions if the initial screening of available exemplars does not yield a comparable area, potentially leading to a miss or an erroneous exclusion. Mind-set can also occur when an examiner believes a comparison may be an identification early in the comparison process and employs poor comparison habits to convince themselves it is true, often creating or adapting comparison notes after seeing the exemplar, straining logic to justify their decision, and potentially leading to an erroneous identification. A recent black box study on palmar comparison accuracy and reliability noted both behaviors in the annotations and notes provided by some study participants. Examples are provided in this paper to serve as a reminder to examiners to not allow mind-set to lead them into errors. Particularly given the high false negative error rates reported throughout the literature, examiners need to make re-considering their initial analysis before rendering an exclusion decision part of their comparison routine.

Testing the accuracy and reliability of palmar friction ridge comparisons - A black box study.

Critics and commentators have been calling for some time for black box studies in the forensic science disciplines to establish the foundational validity of those fields-that is, to establish a discipline-wide, base-rate estimate of the error rates that may be expected in each field. While the well-known FBI/Noblis black box study has answered that call for fingerprints, no research to establish similar error rates for palmar impressions has been previously undertaken. We report the results of the first large-scale black box study to establish a discipline-wide error rate estimate for palmar comparisons. The 226 latent print examiner participants returned 12,279 decisions over a dataset of 526 known ground-truth pairings. There were 12 false identification decisions made yielding a false positive error rate of 0.7%. There were also 552 false exclusion decisions made yielding a false negative error rate of 9.5%. Given their larger number, false negative error rates were further stratified by size, comparison difficulty, and area of the palm from which the mark originated. The notion of "questionable conclusions," in which the ground truth response may not be the most appropriate, is introduced and discussed in light of the data obtained in the study. Measures of examiner consistency in analysis and comparison decisions are presented along with statistical analysis of the ability of many variables, such as demographics or image quality, to predict outcomes. Two online apps are introduced that will allow the reader to fully explore the results on their own, or to explore the notions of frequentist confidence intervals and Bayesian credible intervals.

Examining and expanding the friction ridge value decision.

The first step of a friction ridge examination involves determining the suitability-or value-of an impression. Often, this is interpreted as whether the impression is suitable for comparison. However, examiners tend to be variable in their suitability determinations, and suitability itself can be a multi-faceted decision, comprising suitability for comparison, suitability for exclusion, suitability for identification, suitability for AFIS entry, complexity, and others. We undertook a white box study to explore the different facets of suitability determinations and to measure the specific categories of information upon which examiners most heavily rely when reaching these decisions. Although minutiae count was the best indicator of a value determination, clarity and distortion were better predictors of complexity determinations. Examiners were found to be highly variable in their determinations, as well as in their annotations of what information they relied upon. Some unanimous decisions were reached for only high-quality impressions; there was never unanimity on "no value" determinations. Examiners tended to use high-confidence minutiae markers, even when there was connective ambiguity or low clarity. Several new suitability categorizations were introduced and had good usage from study participants, indicating that they might have some value for inclusion in routine casework.

Comparison and interpretation of impressed marks left by a firearm on cartridge cases - Towards an operational implementation of a likelihood ratio based technique.

Firearm examination is subject to increased scrutiny regarding its foundational validity and inherent subjective nature. The increased use of automatic comparison systems may help to reduce subjectivity. In this paper, we present the performance and limits of an automatic comparison system that assigns a weight to the forensic findings for the comparisons between firing pin marks, breechface marks, or a combination of the two. This weight is expressed by a likelihood ratio (LR) based on 3D topographical measurements coupled with a bi-dimensional statistical model. As the performance of such systems may depend on the reference databases used to inform the model, we investigated the impact of the brand of ammunition and the number of samples. We show that reference databases used to calculate LRs should ideally consist of the same type of ammunition as is seen in the case under investigation and that 7 specimens fired by the same firearm are enough to obtain rates of misleading evidence of a similar magnitude compared to those obtained when far more specimens (60) are used. Additionally, the automatic system was used to assess the outcomes of 7 cases with known same-source or different-source ground truths. These cases were also examined by 8 qualified firearm examiners. In all cases, the experts' appraisals were in line with the ground truth. The automatic system showed some limitations in cases were the data were not sufficient to calculate a robust LR, but also that it can assist and enhance the examiners in their decision process.

The use of Bayesian Networks and simulation methods to identify the variables impacting the value of evidence assessed under activity level propositions in stabbing cases.

This paper presents a methodology allowing identification of the variables associated with transfer, persistence or recovery of DNA traces that have the most significant impact on the result of an evaluation measured through a likelihood ratio (LR). It builds on a case scenario involving trace DNA recovered from knife handles where the prosecution alleges that the person of interest (POI) stabbed a victim whereas the defence claims that the POI has nothing to do with the incident and the victim was stabbed by an alternative offender (AO). The defence proposition will also be refined to account for the possibility of secondary transfer. The variables having a significant impact on the LR are identified taking advantage of a graphical probabilistic environment (using Bayesian Networks, BN), coupled with simulation techniques. The paper presents (a) a BN, based on previous work Taylor et al. [5]; (b) its parametrization based on the current literature that represents the current state of knowledge used to inform the conditional probability tables of the BN and; (c) the implementation of the simulation methods. Results show that, regardless of the DNA outcome obtained, the most impacting variable is the "DNA match probability" when the defence alleged that the POI has nothing to do with the incident. It means that, given the current state of knowledge, such cases can easily be interpreted considering activity level propositions as they would not require any further data acquisition. When secondary transfer is alleged under the defence's perspective, the LRs are generally much lower than for the previous case. The DNA match probability has less impact and variables associated with the donor will take the lead on the ranges observed on the LRs. Overall, once extraction and sampling efficiency have been set, the remaining variables that have an impact on the value of the evidence are the DNA quantity on hands and the background. With the most impacting variables so identified, it becomes manageable to direct further data acquisition if so required. Generally, the background that could be present on the knife handle, the environmental conditions are not critical due to their limited impact on the LR value. We note, however, that this identification of the significant variables depends on the obtained DNA results and this selection may be refined on a case by case basis. To allow one to explore all possibilities a dedicated Shiny application has been designed (https://lydie-samie.shinyapps.io/DNA_Activity/).

Occurrence and associative value of non-identifiable fingermarks.

Fingermarks that have insufficient characteristics for identification often have discernible characteristics that could form the basis for lesser degrees of correspondence or probability of occurrence within a population. Currently, those latent prints that experts judge to be insufficient for identification are not used as associative evidence. How often do such prints occur and what is their potential value for association? The answers are important. We could be routinely setting aside a very important source of associative evidence, with high potential impact, in many cases; or such prints might be of very low utility, adding very little, or only very rarely contributing to cases in a meaningful way. The first step is to better understand the occurrence and range of associative value of these fingermarks. The project goal was to explore and test a theory that in large numbers of cases fingermarks of no value for identification purposes occur and are readily available, though not used, and yet have associative value that could provide useful information. Latent fingermarks were collected from nine state and local jurisdictions. Fingermarks included were those (1) collected in the course of investigations using existing jurisdictional procedures, (2) originally assessed by the laboratory as of no value for identification (NVID), (3) re-assessed by expert review as NVID, but with least three clear and reliable minutiae in relationship to one another, and (4) determined to show at least three auto-encoded minutiae. An expected associative value (ESLR) for each mark was measured, without reference to a putative source, based on modeling within-variability and between-variability of AFIS scores. This method incorporated (1) latest generation feature extraction, (2) a (minutiae-only) matcher, (3) a validated distortion model, and (4) NIST SD27 database calibration. Observed associative value distributions were determined for violent crimes, property crimes, and for existing objective measurements of latent print quality. 750 Non Identifiable Fingermarks (NIFMs) showed values of Log10 ESLR ranging from 1.05-10.88, with a mean value of 5.56 (s.d. 2.29), corresponding to an ESLR of approximately 380,000. It is clear that there are large numbers of cases where NIFMs occur that have high potential associative value as indicated by the ESLR. These NIFMs are readily available, but not used, yet have associative value that could provide useful information. These findings lead to the follow-on questions, "How useful would NIFM evidence be in actual practice?" and, "What developments or improvements are needed to maximize this contribution?"

Automated face recognition in forensic science: Review and perspectives.

With recent technological innovations, the multiplication of captured images of criminal events has brought the comparison of faces to the forefront of the judicial scene. Forensic face recognition has become a ubiquitous tool to guide investigations, gather intelligence and provide evidence in court. However, its reliability in court still suffers from the lack of methodological standardization and empirical validation, notably when using automatic systems, which compare images and generate a matching score. Although the use of such systems increases drastically, it still requires more empirical studies based on adequate forensic data (surveillance footage and identity documents) to become a reliable method to present evidence in court. In this paper, we propose a review of the literature leading to the establishment of a methodological workflow to develop a score-based likelihood-ratio computation model using a Bayesian framework. Different approaches are proposed in the literature regarding the within-source and between-sources variability distributions modelling. Depending on the data available, the modelling approach can be specific to the case or generic. Generic approaches allow interpreting the score without any available images of the suspect. Such model is henceforth harder to defend in court because the results are not anchored to the suspect. To make sure the computed score-based LR is robust, we must assess the performance of the model with two main characteristics: the discriminating power and the calibration state of the model. We hence describe the main metrics (Equal Error Rate and Cost of log likelihood-ratio), and graphical representations (Tippett plots, Detection Error Trade-off plot and Empirical Cross-Entropy plot) used to quantify and visualize the performance characteristics.

Interpol review of fingermarks and other body impressions 2016-2019.

This review paper covers the forensic-relevant literature in fingerprint and bodily impression sciences from 2016 to 2019 as a part of the 19th Interpol International Forensic Science Managers Symposium. The review papers are also available at the Interpol website at: https://www.interpol.int/content/download/14458/file/Interpol%20 Review%20 Papers%202019. pdf.