Uncertainty--a chemist's view.

Complete characterization of the performance of analytical methods requires an evaluation of the halo of uncertainty bracketing the reported result. Achieving a satisfactory estimate of this uncertainty is more important than how this estimate is produced. Enumeration of all conceivable error components--the so-called error budget approach--is one way to estimate the uncertainty, but it is not the only way. In fact, when applied to analytical chemistry this approach is likely to (1) overlook important variables and double count others, (2) avoid considering unknown and unknowable interactions and interferences, and (3) adjust for missing variables with an uncontrollable "Type B" component. The problem is one of experimental design. Alternative and more efficient ways of estimating uncertainty in analytical chemistry include the Youden ruggedness procedure, accompanied by a bonus of optimization, and the all-encompassing interlaboratory method-performance trial.

Relationship of (known) control values to (unknown) test values in proficiency studies of pesticide residues.

Proficiency studies have been suggested as an alternative source of information for evaluating method performance characteristics when results from interlaboratory method performance studies conforming to internationally recognized protocols are not available. To explore this possibility, results were examined from ongoing proficiency studies of pesticide residue analyses in celery, carrot, and grape purees, and in wine. Statistical performance parameters were calculated from 18 data sets analyzed as unknowns by about 60 analysts for 12 analytes in the 25-1,000 microg/kg range, and from presumably parallel control (spike) analyses conducted by about half of the participants. A surprising finding was that recovery of known, independent control additions by the participant did not correlate with the recoveries determined as unknowns in the exercise. The data suggest that censoring or truncating of control data has occurred. The question of substitution of proficiency data for method performance data cannot be answered until the problem of unbiased reporting of control data is resolved.

Reliability of the determinations of polychlorinated contaminants (biphenyls, dioxins, furans).

Precision performance parameters from results of 34 interlaboratory performance studies of polychlorinated aromatic ring compounds (biphenyls, dioxins, and furans) (PCCs) have been recalculated by using the international Union of Pure and Applied Chemistry-1987 harmonized protocol. Most studies of 1052 test samples, 56 analytes, 19 matrixes, and 2 types of detectors (electron capture and mass spectrometers) provide among-laboratories relative standard deviations (RSDRS), that are considerably better than those predicted from the Horwitz equation at fractional concentrations of 10(-5) down to 10(-15). The explanation suggested is that supplying common reference calibration solutions, as was done in many of these studies, does not reflect realistic operating conditions. Furthermore, the ability to repeat, discuss, and reassess aberrant reported values results in underestimating the true RSDR. The commonly reported problems of preparation of standard calibrating solutions, instability of the detection system, and failure to follow quality control instructions and good laboratory practices may be important sources of interlaboratory variability in PCC determinations.

Reliability of mycotoxin assays--an update.

The precision parameters of the method-performance (collaborative) studies for mycotoxins published in the literature through 1991 have been recalculated on a uniform basis by following the International Union of Pure and Applied Chemistry protocol. About 80% of the 793 accepted assays for mycotoxins, almost all of which have been conducted by thin-layer chromatography (TLC), liquid chromatography (LC), and enzyme-linked immunosorbent assays (ELISA), exhibit relative standard deviations among laboratories (RSDR) that are less than 2 times the values predicted from the Horwitz equation: RSDR, % = 2(1-0.5log10C) where C is the concentration expressed as a decimal fraction. The precision of TLC and LC methods is about the same, but that of ELISA is somewhat poorer. For those commodities for which sufficient data exist to provide a meaningful comparison, the methods applied to cottonseed products have the best precision and corn the worst, with peanuts intermediate. Overall, however, the primary factor affecting RSDR is concentration, more or less independent of analyte, method, matrix, and age of the study. If it is assumed that the test results are normally distributed and that an RSDR of 50% is the point where effective control of the results begins to be lost (a value equivalent to the production of 2% false-negative values), then relying on the Horwitz curve, the limit of quantitative measurement is the single digit, i.e., 5, micrograms/kg (10(-9); ppb) concentration for solid food commodities. Such a value must be considered as a limit applicable to a single analyte, aflatoxin B1, and not as a mean, and not applicable to the sum of the individual components, each of whose associated standard deviation would lie in the unacceptable region. Enforcement of a 5 micrograms aflatoxin B1/kg limit, under the assumptions made, requires that a responsible manufacturer and a prudent regulator operate at opposite extremes of tolerance limits: e.g., the producer at 2 micrograms/kg and the consumer at 10. A proposed Codex "maximum level" of 0.05 micrograms aflatoxin M1/kg milk cannot be supported by the available data applied in an interlaboratory enforcement environment. These conclusions are also supported by an examination of the reported data from the ongoing, large-scale proficiency studies routinely performed by the American Oil Chemists' Society and the International Agency for Research on Cancer.

Incomplete data sets: coping with inadequate databases.

Three problems arise in handling numerical values in databases: bad data, missing data, and sloppy data. The effects of bad data are mitigated by using statistical subterfuges such as robust statistics or outlier removal. Missing data are replaced by creating a substitute through interpolation or by using statistics appropriate to unbalanced designs. Sloppy, semiquantitative data are relegated to innocuous positions by using nonparametric, rank, or attribute statistics. These techniques are illustrated by the telephone directory, a database of carcinogenicity test results, and a database of precision parameters derived from method performance (collaborative) studies.

Variability associated with analytical methods used to measure aflatoxin in agricultural commodities.

A total of 1019 analytical precision estimates obtained from method-performance (collaborative) studies for mycotoxins published through 1991 were sorted by type of variance measurement, type of analytical method, and type of agricultural commodity. Precision estimates for total aflatoxin were sorted into 2 precision measurements (among-laboratories and within-laboratory), 3 analytical methods (thin-layer chromatography [TLC], liquid chromatography [LC], and enzyme-linked immunosorbent assay [ELISA]), and 11 agricultural commodities. Sufficient data existed to study the analytical variability (precision) associated with 36 sorted combinations (of a possible 66). In all but one combination (within-laboratory, barley, and TLC), the variance (V) was a function of total aflatoxin concentration (C). A power function of the form V = aCb, where a and b are constants, describes the relationship between variance and aflatoxin concentration. The coefficients a and b were determined from regression analysis. When results were pooled across all agricultural commodities, LC had the lowest analytical variability while ELISA had the highest. For a given method, among-laboratories variability was approximately double the within-laboratory variability. These analytical variability estimates can be coupled with previously determined variability estimates of sampling and sample preparation to determine the performance associated with specific test procedures used to inspect agricultural commodities for aflatoxin.

A simple method for evaluating data from an interlaboratory study.

Large-scale laboratory- and method-performance studies involving more than about 30 laboratories may be evaluated by calculating the HORRAT ratio for each test sample (HORRAT = [experimentally found among-laboratories relative standard deviation] divided by [relative standard deviation calculated from the Horwitz formula]). The chemical analytical method is deemed acceptable per se if HORRAT approximately 1.0 (+/- 0.5). If HORRAT is > or approximately 2.0, the most extreme values are removed successively until an "acceptable" ratio is obtained. The laboratories responsible for the extreme values that are removed should examine their technique and procedures. If > or approximately 15% of the values have to be removed, the instructions and the methods should be examined. This suggested computation procedure is simple and does not require statistical outlier tables. Proposed action limits may be adjusted according to experience. Data supporting U.S. Environmental Protection Agency method 245.1 for mercury in waters (manual cold-vapor atomic absorption spectrometry), supplemented by subsequent laboratory-performance data, were reexamined in this manner. Method-performance parameters (means and among-laboratories relative standard deviations) were comparable with results from the original statistical analysis that used a robust biweight procedure for outlier removal. The precision of the current controlled performance is better by a factor of 4 than that of estimates resulting from the original method-performance study, at the expense of rejecting more experimental values as outliers.

Examination of proficiency and control recovery data from analyses for pesticide residues in food: sources of variability.

We examined a number of large proficiency and control databases supporting the values reported for pesticide residues in agricultural commodities at fractions of a part per million (mg/kg). The average recovery from >100,000 recovery records in 13 databases was 94%. The overall average single-value relative standard deviation (RSD) of the reported recoveries was 17% at a mean concentration (C) of about 10(-7) (0.1 mg/kg). The average apparent HORRAT value (RSD found/RSDR predicted from the Horwitz formula [2*C(-0.1505)]) was 0.8. Analysis of variance indicated that about 60-70% of the variance could not be associated with any particular factor or combination of factors-analyte, commodity, method, laboratory, concentration, database, or their interactions. The most predominant factor, analyte, and its third-order interaction with laboratory and concentration contributed most of the assignable variance. These findings suggested that most of the variability of trace analysis for pesticide residues is "random" in the sense of being inherent and not assignable to specific factor fluctuations.

Laboratory proficiency testing of aflatoxins in corn and peanuts--a cooperative project between Thailand and the United States.

The objective of this project was to conduct an aflatoxin proficiency test program in government, academia, and industry laboratories in Thailand. Aflatoxin-free corn and peanuts and corn and peanuts naturally contaminated with aflatoxins diluted to approximately 25 micrograms/kg were analyzed. Homogeneity of prepared, naturally contaminated test samples was checked on multiple replicates. The test was conducted according to the ISO/IUPAC/AOAC INTERNATIONAL Harmonized Protocol with z scores indicating laboratory performance. The participants used 3 methods: enzyme-linked immunosorbent assay, thin-layer chromatography, and the minicolumn. Of 19 laboratories that reported results for aflatoxins in naturally contaminated corn, 13 (68%) performed satisfactorily, on the basis of the mean obtained by an expert laboratory, a calculated target value for standard deviation, and the z score. Of 21 laboratories that reported results for aflatoxins in naturally contaminated peanuts, 10 (48%) performed satisfactorily. For aflatoxin-free corn, 6 laboratories reported finding aflatoxins at > or = 10 ng/g, chiefly by the minicolumn method; for aflatoxin-free peanuts, 1 laboratory reported finding aflatoxins at > 10 ng/g. Subsequently, a workshop of lectures and laboratory sessions was conducted to improve performance. A new and simple successive outlier removal procedure applied to the same data removed the same laboratories as did the use of z scores.

Sampling and preparation of sample for chemical examination.

Sampling and methods for reducing a laboratory sample to a test sample are discussed, with particular emphasis on sampling peanuts for aflatoxin analysis as a practical example. The only way to control the total error in the analysis of this heterogeneous product is to take and to analyze many and large samples.

Effects of scientific advances on the decision-making process: analytical chemistry.

The analytical chemist has taken the science of identification and quantitation of chemical compounds far beyond the capability of the toxicologist to correlate with the results of animal experiments. In pursuing the vanishing zero, however, the analytical chemist has failed to acknowledge the inherent variability of chemical measurements at very low concentrations that manifest itself not only as discrepant values but as an appreciable fraction of false positive and false negative results and as real negative values (the blank or control is greater than the determination). These are the symptoms of a system not in statistical control. The same type of variable results, evident in biological systems as "biological variability," are then manipulated by statisticians as if they were reproducible measurements. Progress in characterizing biological uncertainty cannot be made until the invisible systematic error of individual laboratories is transformed into random error, amenable to the application of statistical principles. In measurement theory, an examination and correction of systematic error requires knowledge or assignment of a "true value," a concept that does not appear to exist in many biological systems.

The variability of AOAC methods of analysis as used in analytical pharmaceutical chemistry.

Pharmaceutical analytical chemistry, which ordinarily deals with the analysis of formulations containing from 0.1 to 100% of active ingredient, uses methods with a reproducibility (between-laboratory variability) of about 2.5% and a repeatability (within-laboratory variability) of about half that amount. The best between-laboratory precision attainable appears to be about 1.0% and within-laboratory precision, about 0.5%. On the basis of the results available, automated methods do not appear to be any more precise than manual methods, although the studies show fewer outlying data points. Replicates (preferably blind ones) should always be conducted in a collaborative interlaboratory study in order to obtain the important information as to whether efforts should be concentrated on improving the method itself or on the performance of laboratories and analysis in applying it.

Analytical methods for sulfonamides in foods and feeds. II. Performance characteristics of sulfonamide methods.

Important factors in interpretation of methods for sulfonamides in tissues are value of the blank, use or omission of recovery factors, and precision of the methods. For determining sulfonamide in tissues, no interlaboratory collaborative studies have been performed to provide reproducibility parameters. By assuming comparability with other tissue residue methods at equivalent concentrations, it may be anticipated that the coefficient of variation within-laboratories of the Bratton-Marshall method is about 15% at concentrations of a fraction of a part per million. It is estimated that the limit of reliable measurement of the Bratton-Marshall method is about 0.2 ppm, varying with the individual laboratory. This value is higher than the tolerance it is intended to enforce. Obviously, the method in this case has been stretched beyond its original claimed capabilities. This method also has high blanks and low recoveries. Assignment of sufficient resources to the solution of the problem by regulatory agencies has resulted in methods capable of handling the sulfonamide residue problem at 0.1 ppm.

Harmonization of collaborative study protocols.

The 2 major protocols for the design, conduct, and interpretation of collaborative analytical studies--those from AOAC and the International Organization for Standardization--are already fairly well harmonized. The statistical models are identical and the outlier tests are essentially the same. The major differences are in symbols and terminology and in the specification of the minimum number of laboratories and replicates.

International coordination and validation of analytical methods.

Most experimentation dealing with analytical methodology in the physical and biological sciences has been conducted within a single laboratory. Method validation by other laboratories was considered not only unnecessary but also detrimental because, in the words of one commentator, 'the results are too variable'. Within the last two decades, however, largely as a result of the requirements of international environmental and food standards programmes, it has become increasingly apparent that a collaborative interlaboratory study is the only way to estimate the variability characteristics of methods of analysis as performed by the typical population of laboratories using the methods. To obtain a common basis for measuring the statistical characteristics of analytical methods, representatives of almost two dozen international organizations meeting in Geneva, in May 1987, approved by consensus a protocol for the design and interpretation of data from collaborative studies of chemical methods of analysis.

Problems of sampling and analytical methods.

The need for acceptable, reliable, and practical methods of analysis, chemical, physical, and biological, by the Food Standards Program of the Food and Agriculture Organization/World Health Organization is leading to cooperation by those organizations with the resources and experience to supply them. Methods should be shown to be workable and practicable and then validated in a properly designed international collaborative study for the efficient utilization of the time and effort of participating organizations and laboratories. The methods to be subjected to interlaboratory collaborative study should be clearly written so that the method itself is being tested without unauthorized variations. Satisfactory reference standard materials are often an essential part of the method. Uniform, international methods which have been developed through interlaboratory collaborative studies are applicable to the areas of microbiology and toxicology, as well as chemistry.

Determination of nitrogen content in milk by the Kjeldahl method using copper sulfate: interlaboratory study.

Copper sulfate was substituted for mercury as the catalyst in the International Dairy Federation (IDF) Standard 20A:1986 method for the determination of nitrogen content in milk. The substitution was supported by results obtained in an interlaboratory study by 24 laboratories in 12 countries. Each laboratory analyzed 12 test samples of milk as blind duplicates in a double split level design with high, medium, and low nitrogen concentrations. The method protocol requires the concurrent analyses of an ammonium salt solution and a tryptophan solution as internal quality control standards with a minimum nitrogen recovery between 99 and 100% for the former and at least 98% for the latter. The repeatability and reproducibility relative standard deviations are 0.5 and 1%, respectively, for the range 0.35-0.70 g N/100 g. The performance of the laboratories that did not meet the required quality control specifications was clearly poorer than that of those that did meet the specifications.