Beta-Binomial Statistical Model for Validation Studies of Analytes with a Binary Response.

BACKGROUND: The probability of detection (POD) model has had widespread application for statistically analyzing single and multiple collaborator validations studies with binary outcome data for a wide range of analytes over the last decade. OBJECTIVE: The POD model is placed on a firm theoretical foundation, and extended to a more generalized beta-binomial model. METHODS: The POD model is revisited and embedded in the beta-binomial model. This generalization includes collaborator reproducibility as a specific parameter. The new model includes only two distributional parameters: the overall across-collaborator probability of detection (LPOD) and the intraclass correlation of collaborators (ICC), measuring irreproducibility. Differences between methods are measured by the difference in LPOD values, denoted dLPOD. RESULTS: Accurate statistical estimators and confidence intervals are provided with validation by simulation. This new beta-binomial model will be applicable to a full range of candidate methods giving binary qualitative results, including microbiological, toxin, allergen, biothreat, and botanical analytes. CONCLUSIONS: The new beta-binomial model provides easy equivalence tests to show the study clearly demonstrates (with 95% confidence) that the method differences and collaborator reproducibility are acceptable. HIGHLIGHTS: The validation system for qualitative binary methods using probability of detection (POD) of an analyte as the parameter of interest has been modified and further validated.

Determination and Identification of Nicarbazin, Measured as 4,4'-Dinitrocarbanilide (DNC), in Chicken Tissues by Liquid Chromatography With Tandem Mass Spectrometry: Final Action 2013.07.

BACKGROUND: AOAC Method 2013.07 was adopted as First Action in 2013. Since then, the method has been used in numerous residue depletion studies with favorable comments from analysts. OBJECTIVE: To analyze data from residue depletion studies to support Final Action status. METHOD: Ten residue depletion studies were conducted during May 2014 through May 2019. For each study, harvested incurred tissues were analyzed for nicarbazin using AOAC Method 2013.07 in 1 of 4 laboratories. Each analytical run included one or more fortified quality control test portions. The data from these known fortified matrix test portions were analyzed for reproducibility and repeatability. RESULTS: For muscle tissues, relative recovery was 90.4% (95% CI 83.8 to 97.5); RSDr was 5.4% (95% CI 3.8 to 9.2); and RSDR was 7.9%. In the liver, values were 94.5% (95% CI 91.1 to 98.0), 5.8% (95% CI 4.1 to 9.9), and 6.8%, respectively. In the kidney, values were 91.5% (95% CI 85.3 to 98.1), 5.2% (95% CI 3.7 to 8.8), and 9.0%, respectively. In skin with adhering fat, values were 94.5% (95% CI 89.2 to 100.1), 8.9% (95% CI 6.3 to 15.1), and 8.9%, respectively. In all cases, repeatability and reproducibility were within acceptable limits. CONCLUSIONS: The data and positive feedback support the transition of AOAC Method 2013.07 from First Action to Final Action. HIGHLIGHTS: Final action status is supported by data collected during routine use of the method rather than a traditional multi-laboratory collaborative study. Data were subjected to statistical analysis using the pC-metamer, and then transformed back to the traditional C-metamer.

Comparison of 3M Petrifilm Aerobic Count Plates to standard plating methodology for use with AOAC antimicrobial efficacy methods 955.14, 955.15, 964.02, and 966.04 as an alternative enumeration procedure: collaborative study.

A multilaboratory study was conducted to determine the equivalence of the 3M Petrifilm Aerobic Count Plate and standard plating methodology for measuring viable bacteria and spores recovered from hard-surface carriers (stainless steel and porcelain), also known as "control carrier counts," used in AOAC antimicrobial efficacy test methods. Six laboratories participated in the study in which carriers inoculated with Staphylococcus aureus, Pseudomonas aeruginosa, Salmonella enterica, and spores of Bacillus subtilis were evaluated using 3M Petrifilm Aerobic Count (AC) plates and standard plating side-by-side. The data were analyzed using a matched-pair t-test to determine the between-method effect with confidence intervals. For all test organisms pooled across all laboratories, the mean difference in log10 concentration between the standard plate count method and 3M Petrifilm AC Plates was -0.012, with a 95% confidence interval of (-0.090, +0.066), which was well within the -0.5, +0.5 interval established as the acceptance criterion. The between-carrier SD averaged 0.139; the between-replicate SD was 0.050. The carrier reproducibility, given that a single replicate per carrier is done, was estimated to be 0.148. Although differences were seen in the final concentrations of the test organisms among laboratories, there were no statistical differences between the enumeration methods. Based on the results from this study, 3M Petrifilm AC Plates are equivalent to standard plating methodology and can be used as an alternative procedure for the enumeration of test organisms used in AOAC Methods 955.14, 955.15, 964.02, and 966.04.

Probability of identification: a statistical model for the validation of qualitative botanical identification methods.

A qualitative botanical identification method (BIM) is an analytical procedure that returns a binary result (1 = Identified, 0 = Not Identified). A BIM may be used by a buyer, manufacturer, or regulator to determine whether a botanical material being tested is the same as the target (desired) material, or whether it contains excessive nontarget (undesirable) material. The report describes the development and validation of studies for a BIM based on the proportion of replicates identified, or probability of identification (POI), as the basic observed statistic. The statistical procedures proposed for data analysis follow closely those of the probability of detection, and harmonize the statistical concepts and parameters between quantitative and qualitative method validation. Use of POI statistics also harmonizes statistical concepts for botanical, microbiological, toxin, and other analyte identification methods that produce binary results. The POI statistical model provides a tool for graphical representation of response curves for qualitative methods, reporting of descriptive statistics, and application of performance requirements. Single collaborator and multicollaborative study examples are given.

A bootstrap test for comparing two variances: simulation of size and power in small samples.

An F statistic was proposed by Good and Chernick ( 1993 ) in an unpublished paper, to test the hypothesis of the equality of variances from two independent groups using the bootstrap; see Hall and Padmanabhan ( 1997 ), for a published reference where Good and Chernick ( 1993 ) is discussed. We look at various forms of bootstrap tests that use the F statistic to see whether any or all of them maintain the nominal size of the test over a variety of population distributions when the sample size is small. Chernick and LaBudde ( 2010 ) and Schenker ( 1985 ) showed that bootstrap confidence intervals for variances tend to provide considerably less coverage than their theoretical asymptotic coverage for skewed population distributions such as a chi-squared with 10 degrees of freedom or less or a log-normal distribution. The same difficulties may be also be expected when looking at the ratio of two variances. Since bootstrap tests are related to constructing confidence intervals for the ratio of variances, we simulated the performance of these tests when the population distributions are gamma(2,3), uniform(0,1), Student's t distribution with 10 degrees of freedom (df), normal(0,1), and log-normal(0,1) similar to those used in Chernick and LaBudde ( 2010 ). We find, surprisingly, that the results for the size of the tests are valid (reasonably close to the asymptotic value) for all the various bootstrap tests. Hence we also conducted a power comparison, and we find that bootstrap tests appear to have reasonable power for testing equivalence of variances.

Probability of Detection (POD) as a statistical model for the validation of qualitative methods.

A statistical model is presented for use in validation of qualitative methods. This model, termed Probability of Detection (POD), harmonizes the statistical concepts and parameters between quantitative and qualitative method validation. POD characterizes method response with respect to concentration as a continuous variable. The POD model provides a tool for graphical representation of response curves for qualitative methods. In addition, the model allows comparisons between candidate and reference methods, and provides calculations of repeatability, reproducibility, and laboratory effects from collaborative study data. Single laboratory study and collaborative study examples are given.