Reliability of ordinal outcomes in forensic black-box studies.

Forensic science disciplines such as latent print examination, bullet and cartridge case comparisons, and shoeprint analysis, involve subjective decisions by forensic experts throughout the examination process. Most of the decisions involve ordinal categories. Examples include a three-category outcome for latent print comparisons (exclusion, inconclusive, identification) and a seven-category outcome for footwear comparisons (exclusion, indications of non-association, inconclusive, limited association of class characteristics, association of class characteristics, high degree of association, identification). As the results of the forensic examinations of evidence can heavily influence the outcomes of court proceedings, it is important to assess the reliability and accuracy of the underlying decisions. "Black box" studies are the most common approach for assessing the reliability and accuracy of subjective decisions. In these studies, researchers produce evidence samples consisting of a sample of questioned source and a sample of known source where the ground truth (same source or different source) is known. Examiners provide assessments for selected samples using the same approach they would use in actual casework. These studies often have two phases; the first phase comprises of decisions on samples of varying complexities by different examiners, and the second phase involves repeated decisions by the same examiner on a (usually) small subset of samples that were encountered by examiners in the first phase. We provide a statistical method to analyze ordinal decisions from black-box trials with the objective of obtaining inferences for the reliability of these decisions and quantifying the variation in decisions attributable to the examiners, the samples, and statistical interaction effects between examiners and samples. We present simulation studies to judge the performance of the model on data with known parameter values and apply the model to data from a handwritten signature complexity study, a latent fingerprint examination black-box study, and a handwriting comparisons black-box study.

Location distribution of randomly acquired characteristics on a shoe sole.

Footwear comparison is used to link between a suspect's shoe and a shoeprint found at a crime scene. Forensic examiners compare the two items, and the conclusion reached is based on class characteristics and randomly acquired characteristics (RACs), such as scratches or holes. An important question concerns the distribution of the location of RACs on shoe soles, which can serve as a benchmark for comparison. This study examines the probability of observing RACs in different areas of a shoe sole using a database of approximately 13,000 RACs observed on 386 outsoles. The analysis is somewhat complicated as the shoes are differentiated by shape and contact surface, and the RACs' locations are subject to measurement errors. A method that takes into account these challenges is presented. All impressions are normalized to a standardized axis to allow for inter-comparison of RACs on outsoles of different sizes and contact areas, and RACs are localized to one of 14 subareas of the shoe sole. Expected frequencies in each region are assumed to be Poisson distributed with rate parameters that depend on the subarea and the contact surface. Three different estimation approaches are studied: a naive crude approach, a shoe-specific random effects model, and an estimate that is based on conditional maximum likelihood. It is shown that the rate is not uniform across the shoe sole and that RACs are approximately twice as likely to appear at certain locations, corresponding to the foot's morphology. The results can guide investigators in determining a shoeprint's evidential value.

Dataset of Digitized RACs and Their Rarity Score Analysis for Strengthening Shoeprint Evidence.

In recent years, there is a growing demand to fortify the scientific basis of forensic methodology. During 2016, the President's Council of Advisors on Science and Technology (PCAST) published a report that states there are no appropriate empirical studies that support the foundational validity of footwear analysis to associate shoeprints with particular shoes based on specific identifying marks, which is a basic scientific demand from the field. Furthermore, meaningful databases that can support such studies do not exist. Without such databases, statistical presentation of the comparison results cannot be fulfilled either. In this study, a database of over 13,000 randomly acquired characteristics (RACs) such as scratches, nicks, tears, and holes, as they appear on shoe sole test impressions, from nearly 400 shoe soles was collected semi-automatically. The location, orientation, and the contour of each RAC were determined for all the RACs on each test impression. The statistical algorithm Statistic Evaluation of Shoeprint Accidentals (SESA) was developed to calculate a score for finding another feature similar to a particular scanned and digitized RAC in the same shape, location, and orientation as the examined one. A correlation was found between the results of SESA and the results of real casework, strengthening our belief in the ability of SESA to assist the expert in reaching a conclusion while performing casework. The score received at the end of the process serves the expert as a guiding number, allowing more objective and accurate results and conclusions.

Dependence among randomly acquired characteristics on shoeprints and their features.

Randomly acquired characteristics (RACs), also known as accidental marks, are random markings on a shoe sole, such as scratches or holes, that are used by forensic experts to compare a suspect's shoe with a print found at the crime scene. This article investigates the relationships among three features of a RAC: its location, shape type and orientation. If these features, as well as the RACs, are independent of each other, a simple probabilistic calculation could be used to evaluate the rarity of a RAC and hence the evidential value of the shoe and print comparison, whereas a correlation among the features would complicate the analysis. Using a data set of about 380 shoes, it is found that RACs and their features are not independent, and moreover, are not independent of the shoe sole pattern. It is argued that some of the dependencies found are caused by the elements of the sole. The results have important implications for the way forensic experts should evaluate the degree of rarity of a combination of RACs.