Repeatability and reproducibility of comparison decisions by firearms examiners.

In a comprehensive study to assess various aspects of the performance of qualified forensic firearms examiners, volunteer examiners compared both bullets and cartridge cases fired from three different types of firearms. They rendered opinions on each comparison according to the Association of Firearm & Tool Mark Examiners (AFTE) Range of Conclusions, as Identification, Inconclusive (A, B, or C), Elimination, or Unsuitable. In this part of the study, comparison sets used previously to characterize the overall accuracy of examiners were blindly resubmitted to examiners to assess the repeatability (105 examiners; 5700 comparisons of bullets and cartridge cases) and reproducibility (191 examiners of bullets, 193 of cartridge cases; 5790 comparisons) of firearms examinations. Data gathered using the prevailing AFTE Range were also recategorized into two hypothetical scoring systems. Consistently positive differences between observed agreement and expected agreement indicate that the repeatability and reproducibility of examiners exceed chance agreement. When averaged over bullets and cartridge cases, the repeatability of comparison decisions (involving all five levels of the AFTE Range) was 78.3% for known matches and 64.5% for known nonmatches. Similarly averaged reproducibility was 67.3%% for known matches and 36.5% for known nonmatches. For both repeatability and reproducibility, many of the observed disagreements were between a definitive and inconclusive category. Examiner decisions are reliable and trustworthy in the sense that identifications are unlikely when examiners are comparing non-matching items, and eliminations are unlikely when they are comparing matching items.

Accuracy of comparison decisions by forensic firearms examiners.

This black box study assessed the performance of forensic firearms examiners in the United States. It involved three different types of firearms and 173 volunteers who performed a total of 8640 comparisons of both bullets and cartridge cases. The overall false-positive error rate was estimated as 0.656% and 0.933% for bullets and cartridge cases, respectively, while the rate of false negatives was estimated as 2.87% and 1.87% for bullets and cartridge cases, respectively. The majority of errors were made by a limited number of examiners. Because chi-square tests of independence strongly suggest that error probabilities are not the same for each examiner, these are maximum-likelihood estimates based on the beta-binomial probability model and do not depend on an assumption of equal examiner-specific error rates. Corresponding 95% confidence intervals are (0.305%, 1.42%) and (0.548%, 1.57%) for false positives for bullets and cartridge cases, respectively, and (1.89%, 4.26%) and (1.16%, 2.99%) for false negatives for bullets and cartridge cases, respectively. The results of this study are consistent with prior studies, despite its comprehensive design and challenging specimens.

Accuracy, reproducibility, and repeatability of forensic footwear examiner decisions.

The interpretation of footwear evidence relies on the expertise of forensic footwear examiners. Here we report on the largest study to date of the accuracy, reproducibility (inter-examiner variation), and repeatability (intra-examiner variation) of footwear examiners' decisions. In this study, 84 practicing footwear examiners each conducted up to 100 comparisons between questioned footwear impressions (provided as photographs and digital images) and known footwear (provided as photographs, transparent test impressions, and digital images), resulting in a total of 6610 comparisons. The quality and characteristics of the impressions were selected to be broadly representative of those encountered in casework. A multilevel conclusion scale was used: 40% of responses were definitive conclusions (identification or exclusion), 14% probable conclusions (high degree of association or indications of non-association), 40% class associations (association of class characteristics or limited association of class characteristics), and 6% neutral conclusions (inconclusive or not suitable). On nonmated comparisons, 0.2% of conclusions were erroneous identifications (false positives), and 1.4% were incorrect responses of "high degree of association." The majority of erroneous identifications were made by a single participant. On mated comparisons, 6.0% of conclusions were erroneous exclusions (false negatives), and 1.8% were incorrect responses of "indications of non-association." Erroneous conclusions were sometimes reproduced by different examiners, but rarely repeated by the same examiner-1.1% of erroneous identifications were reproduced (none were repeated) and 19.9% of erroneous exclusions were reproduced (just one was repeated). Examiners' assessments of whether a questioned impression was suitable for comparison were notably inconsistent and may benefit from standardization. Rates of correct definitive conclusions are directly associated with the quality of the questioned impression and the extent of class similarities/differences between the questioned impression and known footwear.

Accuracy and reliability of forensic handwriting comparisons.

Forensic handwriting examination involves the comparison of writing samples by forensic document examiners (FDEs) to determine whether or not they were written by the same person. Here we report the results of a large-scale study conducted to assess the accuracy and reliability of handwriting comparison conclusions. Eighty-six practicing FDEs each conducted up to 100 handwriting comparisons, resulting in 7,196 conclusions on 180 distinct comparison sets, using a five-level conclusion scale. Erroneous "written by" conclusions (false positives) were reached in 3.1% of the nonmated comparisons, while 1.1% of the mated comparisons yielded erroneous "not written by" conclusions (false negatives). False positive rates were markedly higher for nonmated samples written by twins (8.7%) compared to nontwins (2.5%). Notable associations between training and performance were observed: FDEs with less than 2 y of formal training generally had higher error rates, but they also had higher true positive and true negative rates because they tended to provide more definitive conclusions; FDEs with at least 2 y of formal training were less likely to make definitive conclusions, but those definitive conclusions they made were more likely to be correct (higher positive predictive and negative predictive values). We did not observe any association between writing style (cursive vs. printing) and rates of errors or incorrect conclusions. This report also provides details on the repeatability and reproducibility of conclusions, and reports how conclusions are affected by the quantity of writing and the similarity of content.

Endogenous GHB in Segmented Hair Part II: Intra-individual Variation for Exogenous Discrimination.

The endogenous presence of gamma-hydroxybutyric acid (GHB) complicates the interpretation of results in cases where an exogenous dosing is suspected. Due to GHB's rapid metabolism and clearance following exogenous doses, hair has become a preferential matrix for confirmation of GHB exposure in drug-facilitated crimes. However, unlike blood and urine where an agreed-upon cut-off concentration for differentiation between endogenous and exogenous GHB has been made, there has been no consensus on a cut-off concentration for hair. This is due in part to the wide inter- and intra-individual variation that has been observed in endogenous GHB hair studies. A large (>50) population study of 214 donors was conducted to better understand these variations and to evaluate whether a cut-off concentration could be established for endogenous GHB in human hair. As seen in our previous study, the inter-individual variation was large, with concentrations ranging from <0.40 to 5.47 ng/mg. This range made an absolute cut-off concentration recommendation inappropriate, so an alternative approach for GHB discrimination was investigated utilizing the intra-individual variation. Male donors appeared to have greater intra-individual variation than female donors, yet it was noted that segment-to-segment variation along the length of hair had minimal change between individual donor's adjacent segments. Overall, 97.1% of the adjacent segment differences were within ±0.5 ng/mg. Therefore, instead of a recommended cut-off concentration, it appears that using adjacent segment concentration differences could be a strategy to assist in differentiating endogenous from single exogenous GHB exposure. In the absence of controlled dosing data, previously published segmented results from controlled and suspected dosing donors are examined using the adjacent segmental difference approach and the results compared to currently used ratio-based calculations.

Endogenous GHB in Segmented Hair Part I: Inter-individual Variation for Group Comparisons.

While earlier studies have attempted to resolve the challenges encountered when interpreting gamma-hydroxybutyric acid (GHB) concentrations in hair (primarily due to its endogenous presence), few have had large sample sizes. The first objective of this study was to evaluate the inter-individual variation of endogenous GHB concentrations. The second objective, to be detailed in another report, was to assess intra-individual variation and the impact on exogenous GHB discrimination. Over 2,000 hair segments from 141 women and 73 men (all processed hair 3-12 cm long) were analyzed in this study. The raw calculated range of endogenous GHB concentrations was <0.40-5.47 ng/mg with 97.5% of the segmental results calculated less than 2.00 ng/mg. Imputation, assuming a lognormal distribution, was applied to the data to include non-detect (ND) data (