From data to a validated score-based LR system: A practitioner's guide.

Likelihood ratios (LRs) are a useful measure of evidential strength. In forensic casework consisting of a flow of cases with essentially the same question and the same analysis method, it is feasible to construct an 'LR system', that is, an automated procedure that has the observations as input and an LR as output. This paper is aimed at practitioners interested in building their own LR systems. It gives an overview of the different steps needed to get to a validated LR system from data. The paper is accompanied by a notebook that illustrates each step with an example using glass data. The notebook introduces open-source software in Python constructed by NFI (Netherlands Forensic Institute) data scientists and statisticians.

Evaluation of glass evidence at activity level: A new distribution for the background population.

For evidence evaluation of the physicochemical properties of glass at activity level a well-known formula introduced by Evett & Buckleton [1,2] is commonly used. Parameters in this formula are, amongst others, the probability in a background population to find on somebody's clothing the observed number of glass sources and the probability in a background population to find on somebody's clothing a group of fragments with the same size as the observed matching group. Recently, for efficiency reasons, the Netherlands Forensic Institute changed its methodology to measure not all the glass fragments but a subset of glass fragments found on clothing. Due to the measurement of subsets, it is difficult to get accurate estimates for these parameters in this formula. We offer a solution to this problem. The heart of the solution consists of relaxing the assumption of conditional independence of group sizes of background fragments, and modelling the probability of an allocation of background fragments into groups given a total number of background fragments by a two-parameter Chinese restaurant process (CRP) [3]. Under the assumption of random sampling of fragments to be measured from recovered fragments in the laboratory, parameter values for the Chinese restaurant process may be estimated from a relatively small dataset of glass in other relevant cases. We demonstrate this for a dataset of glass fragments collected from upper garments in casework, show model fit and provide a prototypical calculation of an LR at activity level accompanied with a parameter sensitivity analysis for reasonable ranges of the CRP parameter values. Considering that other laboratories may want to measure subsets as well, we believe this is an important alternative approach to the evaluation of numerical LRs for glass analyses at activity level.

Overall mean estimation of trace evidence in a two-level normal-normal model.

In the evaluation of measurements on characteristics of forensic trace evidence, Aitken and Lucy (2004) model the data as a two-level model using assumptions of normality where likelihood ratios are used as a measure for the strength of evidence. A two-level model assumes two sources of variation: the variation within measurements in a group (first level) and the variation between different groups (second level). Estimates of the variation within groups, the variation between groups and the overall mean are required in this approach. This paper discusses three estimators for the overall mean. In forensic science, two of these estimators are known as the weighted and unweighted mean. For an optimal choice between these estimators, the within- and between-group covariance matrices are required. In this paper a generalization to the latter two mean estimators is suggested, which is referred to as the generalized weighted mean. The weights of this estimator can be chosen such that they minimize the variance of the generalized weighted mean. These optimal weights lead to a "toy estimator", because they depend on the unknown within- and between-group covariance matrices. Using these optimal weights with estimates for the within- and between-group covariance matrices leads to the third estimator, the optimal "plug-in" generalized weighted mean estimator. The three estimators and the toy estimator are compared through a simulation study. Under conditions generally encountered in practice, we show that the unweighted mean can be preferred over the weighted mean. Moreover, in these situations the unweighted mean and the optimal generalized weighted mean behave similarly. An artificial choice of parameters is used to provide an example where the optimal generalized weighted mean outperforms both the weighted and unweighted mean. Finally, the three mean estimators are applied to real XTC data to illustrate the impact of the choice of overall mean estimator.

Performance Study of a Score-based Likelihood Ratio System for Forensic Fingermark Comparison.

In this article, the performance of a score-based likelihood ratio (LR) system for comparisons of fingerprints with fingermarks is studied. The system is based on an automated fingerprint identification system (AFIS) comparison algorithm and focuses on fingerprint comparisons where the fingermarks contain 6-11 minutiae. The hypotheses under consideration are evaluated at the level of the person, not the finger. The LRs are presented with bootstrap intervals indicating the sampling uncertainty involved. Several aspects of the performance are measured: leave-one-out cross-validation is applied, and rates of misleading evidence are studied in two ways. A simulation study is performed to study the coverage of the bootstrap intervals. The results indicate that the evidential strength for same source comparisons that do not meet the Dutch twelve-point standard may be substantial. The methods used can be generalized to measure the performance of score-based LR systems in other fields of forensic science.