Improvement in physician's office laboratory practices, 1989-1994.

BACKGROUND: In 1986 and 1989, the Centers for Disease Control and Prevention sponsored institutes on Critical Issues in Health Laboratory Practice. It was noted during the institutes that physician's office laboratories were a rapidly emerging site for clinical laboratory testing, yet no comprehensive data were available regarding the practice of clinical laboratory medicine in physician's office laboratories. As a mechanism to begin addressing this void, the Centers for Disease Control and Prevention added questions on clinical laboratory practice to the National Ambulatory Medical Care Survey, a national probability sample of ambulatory care provided by office-based physicians. Data were collected for survey years 1989, 1991, 1993, and 1994. METHODS: Each survey was conducted among a nationally representative, random sample of office-based physicians who provide ambulatory patient care. Sample physicians were enlisted using both mail and telephone contacts. Clinical laboratory data were obtained via telephone by trained field representatives. Weighted univariate and multivariate analyses were performed on responses from each of the 4 survey years. Analyses were repeated after combining survey responses from years 1989 and 1991 and 1993 and 1994 as representative of physician's office laboratory practices before and after implementation of the Clinical Laboratory Improvement Amendments of 1988 (CLIA '88) final rule in 1992. RESULTS: Quality laboratory practice indicators showed significant increases during the study interval, with implementation of the CLIA '88 final rule in 1992 playing a pivotal role. Relative to 1992, enrollment in proficiency testing programs increased from 32.4% to 52.7% (P<.001), use of daily quality control samples increased from 79.2% to 89.0% (P<.001), and use of daily quality control with written instructions for action following a questionable quality control result (quality control with action step documentation) increased from 62.6% to 77.2% (P<.001). The presence of a medical technologist or technician in the office laboratory was also significantly and independently associated with each of the quality indicators. Although the percentage of physician's offices performing on-site testing decreased from 56% to 45% during the survey interval, overall testing volume appeared unchanged. CONCLUSIONS: The quality of clinical laboratory practice in physician's office laboratories improved during the study interval (1989-1994) as measured by the quality indicators used in the study. The association of this improvement with implementation of the CLIA '88 final rule and the presence of a trained laboratory professional in the testing site indicate the importance of minimum practice standards and professional expertise in ensuring use of quality laboratory practices. Overall test volume appeared to be stable despite a decreased proportion of physician's offices at which on-site testing was performed.

Laboratory performance in HTLV-I/II analysis.

BACKGROUND: Since 1989, the CDC's Model Performance Evaluation Program has shipped samples to voluntary participant laboratories that test for HTLV antibodies. Each laboratory tests the well-characterized samples, reports the results, and provides information about its testing practices. The data from 15 performance survey periods are reported here. STUDY DESIGN AND METHODS: Multiple logistic regression was used to analyze all data from 15 survey periods from 1989 through 1996. RESULTS: The mean analytic sensitivity for EIA was 99.2 percent per survey period (range, 96-100%), the mean analytic specificity was 97.8 percent (75.6-100%), and the overall accuracy was 88.8 percent (63.8-100%). The mean analytic sensitivity for Western blot was 88.8 percent (75.6-100%); the mean analytic specificity was 95.7 percent (86.7-100%), and the overall accuracy was 91.1 percent (78.1-100%). CONCLUSIONS: Statistical analyses suggested associations between performance and both the retroviral serologic status of the sample and the analytical testing method. Western blot accuracy was associated with weekly testing volume. In early survey periods, performance problems were noted in the analysis of samples from donors with concomitant HTLV and HIV infections and those from donors who were positive for HTLV-II. Technological developments in test methods, such as the addition of recombinant antigens, appeared to have improved the laboratory performance of specific testing methods.

Do proficiency test results correlate with the work performance of screeners who screen Papanicolaou smears?

We rescreened Papanicolaou smear slides from 40,245 women, which had been examined by 81 cytology screeners, scored the screeners' work performance, and compared these scores with the results of the screeners' performance on glass slide and computer-based proficiency tests. All diagnoses (i.e., from the proficiency tests, the original slides, and the rescreened slides) were classified in the 4 diagnostic categories specified in the Clinical Laboratory Improvement Amendments. The rescreening scores were standardized to account for different distributions of abnormalities in the proficiency tests and rescreened slides. We compared a standardized score with the proficiency test scores. Of the cases, 91% were categorized as normal, benign, or reactive changes when rescreened, and 98% of these agreed with the original diagnosis. Sixteen percent of low-grade and 15% of high-grade intraepithelial lesions were classified as normal. The rank correlation between the rescreening scores and both proficiency tests was 0.24 using a scoring scheme for cytotechnologists. The correlation between the rescreening and proficiency testing scores indicates that performance on a 10-slide test gives some indication of the true performance of screeners. The computer-based test shows promise as an alternative to the glass slide test but needs further development and validation.

Impact of quality control on accuracy in enzyme immunoassay testing for human immunodeficiency virus type 1 antibodies.

OBJECTIVE: To assess use of quality control (QC) material, supplemental to internal kit controls (calibrators), as protection against errors in enzyme immunoassay testing for human immunodeficiency virus type 1 antibodies. DESIGN: From August 1994 to January 1996, enzyme immunoassay testing accuracy was assessed for laboratories participating in the Centers for Disease Control and Prevention Model Performance Evaluation Program that provided information regarding their use of QC material. Error rates were examined for human immunodeficiency virus type 1 antibody-negative, strongly positive, and weakly positive samples. RESULTS: The overall error rate with QC (2.20%) was significantly (P = .0023) lower than the error rate without QC (2.90%). With QC use there was a significant reduction in the relative risk of error for negative (P = .014) and weakly positive (P = .0067) samples. After multivariate analysis, use of QC lowered overall error rate by 29% (P = .0009). Laboratories not using QC were at increased risk of systematic error. Following the Clinical Laboratory Improvement Amendments of 1988 guidelines for QC material was relatively more protective against error than lower frequencies/number of levels. CONCLUSIONS: Using QC protected against errors in enzyme immunoassay testing for human immunodeficiency virus type 1 antibodies. Two levels of QC should be used with each run as mandated by the Clinical Laboratory Improvement Amendments of 1988.

Variation in proficiency testing performance by testing site.

CONTEXT: Congress enacted the Clinical Laboratory Improvement Amendments of 1988 (CLIA) to promote uniform quality and standards among all testing sites in the United States. The performance indicators specified in the legislation are proficiency testing (PT) performance and periodic inspections. OBJECTIVE: To evaluate variation in PT performance by type of testing facility during the first year of compulsory participation under CLIA. DESIGN: All 1994 PT score data electronically reported to the Health Care Financing Administration as a component of compliance with the CLIA regulations were obtained. Over 1.2 million PT event scores from 17058 unique testing sites were sorted into 2 groups based on the type of testing facility: hospitals and independent laboratories (HI) and all other testing sites (AOT). MAIN OUTCOME MEASURES: Satisfactory and unsatisfactory performance rates for HI and AOT for each analyte and/or test, according to the criteria specified by the CLIA regulations. RESULTS: The aggregate rates of satisfactory event performance for all regulated analytes, tests, and specialties were 97% and 91% for the HI and AOTgroups, respectively. The aggregate odds ratio for unsatisfactory PT event performance for the AOT group compared with the HI group was 2.89, with a range of 2.19 to 7.51 for the individual analytes. CONCLUSION: There was a consistent difference in PT performance during the first full year of compulsory PT under the CLIA regulations based on the type of testing facility performing the analysis. Traditional testing sites achieved higher rates of satisfactory performance than newly regulated, alternative testing sites.

Analytic sensitivity and specificity of enzyme immunoassay results in testing for human immunodeficiency virus type 1 antibody.

In 1986, a performance evaluation program at the Centers for Disease Control was implemented to assess the quality of performance of laboratories testing for human immunodeficiency virus type 1 antibody and to identify problems that occur during the testing process. Laboratories participating in the Centers for Disease Control Model Performance Evaluation Program for human immunodeficiency virus type 1 antibody testing furnished enzyme immunoassay results after they tested performance evaluation panels that were sent to them in August and November 1989. The panels consisted of 10 individual samples containing antibody-negative and antibody-positive samples, some of which were duplicates. Not all laboratories received the same panel of samples. Low false-negative and false-positive rates, as well as high intrashipment and intershipment reproducibility, indicate that most laboratories did not experience difficulty in testing performance evaluation samples sent to them in August and November 1989.

Human T-lymphotropic virus type I/II. Status of enzyme immunoassay and western blot testing in the United States in 1989 and 1990.

In three performance evaluation surveys, panels that consisted of human T-lymphotropic virus type I or type II (HTLV-I/II) antibody-positive and -negative plasma samples were mailed to laboratories that voluntarily participated in the Centers for Disease Control Model Performance Evaluation Program. Donor samples were identical among surveys. In each survey, more than 98% of the laboratories reported enzyme immunoassay (EIA) test results; about 11% also reported results of Western blot (WB) testing. Variation in analytic sensitivity (96.7% to 99.4%) and specificity (98.3% to 99.5%) of EIA tests was noted in the three surveys. For WB testing, no nonreactive interpretations were reported for HTLV-I/II antibody-positive samples in any survey; however, indeterminate interpretations were reported for 35.2% to 40.7% of the WB tests that were performed on HTLV-I/II antibody-positive samples. More than 95% of these indeterminate WB test interpretations were reported for HTLV-II antibody-positive samples. Although HTLV-I/II antibody tests are generally sensitive and specific, their accuracy could be further improved by increasing the specificity of EIA tests and the sensitivity of WB tests.

Identification of nonanalytic issues in HIV-1 antibody testing using blind proficiency testing.

Blind proficiency testing was used to examine nonanalytic performance indicators for human immunodeficiency virus type 1 (HIV-1) antibody testing. Physician offices, clinics, and hospitals located throughout Southern California submitted simulated patient specimens to laboratories as routine test requests. A total of 32 laboratories were involved during five blind proficiency testing surveys. Turnaround time for a reactive specimen ranged from three to 17 days. Laboratory charges for evaluating a reactive specimen varied depending on the volume of testing, prevalence of reactive specimens, and whether screening and confirmatory tests were billed separately or as a package price. Charges for an enzyme immunoassay screening test plus supplemental tests ranged from $11.75 to $114.50, with a median of $31.00 for 24 laboratories that participated in one of the five surveys. Evaluation of laboratory report content revealed that 37% of the 16 screening reports and 71% of the 14 supplemental reports contained information that was unrelated to the patient results. Evaluation of the testing system documents the need to monitor multiple outcomes of the total laboratory testing process, not just the analytic testing phase.

Quality of laboratory performance in testing for human immunodeficiency virus type 1 antibody. Variables associated in multivariate analyses.

In May 1988, the Centers for Disease Control's Model Performance Evaluation Program (Atlanta, Ga) surveyed 1092 laboratories that performed enzyme immunoassays and Western blot tests for human immunodeficiency virus type 1 antibody on mailed plasma samples of known human immunodeficiency virus type 1 antibody reactivity and that described their laboratory characteristics and testing practices. The study objective was to evaluate the quality of laboratory performance in testing for human immunodeficiency virus type 1 antibody. After identifying relevant variables in univariate analyses, multivariate analyses were performed using stepwise logistic models. Human immunodeficiency virus type 1 antibody test performance was independently associated with analytic variables such as commercial test kit used and with nonanalytic variables such as experience, training, and degree requirements of laboratory personnel. These results validate the importance of nonanalytic variables to the quality of outcomes in laboratory testing.

Capacity of US labs to provide TLI in support of early HIV-1 intervention.

We surveyed laboratories to assess their capacity to perform T-lymphocyte immunophenotyping. Of the 1026 respondents, 279 located in 41 states and the District of Columbia performed this type of testing. Most laboratories were located in hospitals, reported a low weekly test volume, and indicated that it took 6-24 weeks for flow cytometer operators to become proficient. Many laboratories appear to have the capacity to perform additional CD4+ cell testing, but training additional operators may be necessary. The paucity of laboratories performing T-lymphocyte immunophenotyping in the public sector may affect referral patterns from that setting.

Indirect immunofluorescence test performance and questionnaire results from the Centers for Disease Control Model Performance Evaluation Program for human immunodeficiency virus type 1 testing.

Results from laboratories performing indirect immunofluorescence (IIF) testing for human immunodeficiency virus type 1 antibody and participating in the Centers for Disease Control Model Performance Evaluation Program in 1988 are presented. Approximately 90% of all laboratories receiving specimen panels or questionnaires furnished results to the Centers for Disease Control. In September 1988, 111 reports were received from IIF laboratories from 34 states and nine countries; most of these laboratories did IIF testing in conjunction with other antibody tests. Hospital laboratories were the most common type of laboratory participating in the program. Laboratories that performed IIF employed fewer personnel and performed testing less frequently than did laboratories that performed enzyme immunoassays or Western blot (immunoblot) tests and were likely to use a commercial test kit. Most of the laboratories that referred specimens for IIF testing sent them to the state laboratory. The analytic specificity for the Model Performance Evaluation Program specimens was 98.5% when indeterminate results on a negative specimen were considered correct (negative) and 89.6% when indeterminate results on a negative specimen were considered incorrect; analytic sensitivity was 94.8% when indeterminate results on a positive specimen were correct (positive) and 91.4% when indeterminate results on a positive specimen were considered incorrect. When indeterminate results were considered correct, all types of laboratories (blood bank, state, hospital, independent, and other) had analytic specificities over 96%, and all manufacturers had analytic specificities above 95%. All types of laboratories had analytic sensitivities over 92%, and analytic sensitivities were above 94% for all manufacturers and reagent sources except Cellular Products. Comparison of percentages of correct responses between IIF and Western blot assays on those samples for which there was good agreement on the target interpretation revealed no significant differences. Both individual donor and diluted materials were included in the evaluations; the diluted donor material presented the greatest testing difficulty. Within-survey reproducibility was about 93% overall and by specimen type. Between-survey reproducibility was about 81% for negative and indeterminate specimens and 88.5% for positive specimens, for an overall between-survey reproducibility of 84.3%. Differences in performance were noted when results were compared by type of laboratory and test manufacturer.

Centers for Disease Control perspective on quality assurance for human immunodeficiency virus type 1 antibody testing. Model Performance Evaluation Program.

Ensuring high quality in human immunodeficiency virus type 1 (HIV-1) antibody testing is an essential component of the organized public health response to epidemic HIV-1 infection. In 1986, the Centers for Disease Control designed the Model Performance Evaluation Program to assess and improve the analytic quality of HIV-1 antibody testing. In addition, the program was designed to gather information about HIV-1 antibody testing practices. The utility of this information is in identifying potential barriers to quality throughout the total testing process. Currently, 1405 laboratories participate in the program. Participating laboratories are located both within and outside the United States and consist primarily of hospitals, blood banks, health departments, and independent laboratories. The responses to a questionnaire completed by 1050 program-participant laboratories in September 1988 suggest that at several stages in the HIV-1 antibody total testing process, laboratory practices (including the interpretation of Western blot patterns) are variable and that standardization of these practices would improve quality.

CDC's Model Performance Evaluation Program: assessment of the quality of laboratory performance for HIV-1 antibody testing.

In 1986, the Centers for Disease Control (CDC) implemented the Model Performance Evaluation Program (MPEP) to evaluate the performance of laboratories that test for antibody directed against human immunodeficiency virus type 1 (HIV-1). The impetus for developing this program came from the recognition of a need to assess the quality of existing and changing laboratory technology and to ensure that the quality of testing was sufficient to meet medical and public health needs. To develop the program, CDC chose HIV-1 antibody testing as the first specific application for assessing the quality of laboratory performance because (a) of the importance of accurate and reproducible test results for acquired immunodeficiency syndrome (AIDS) surveillance, prevention, and treatment programs; (b) HIV-1 testing technology is new to many laboratories; and (c) HIV-1 testing practices and applications continue to evolve. Unlike proficiency testing programs, the MPEP is not limited to assessing quality in the analytical step, alone. It will also assess quality in the preanalytical and postanalytical steps of the testing process, that is, from the time a test is requested until the clinician who ordered the test takes an action based on the test result. The participating laboratories furnish the information needed for the performance evaluation program by (a) completing questionnaires designed to describe HIV-1 testing laboratories and their testing practices, (b) analyzing specially prepared sample panels for HIV-1 antibody reactivity, and (c) reporting results to CDC.

Reporting the results of human immunodeficiency virus testing.

We conducted a pilot study of potential sources of incorrect laboratory reports of human immunodeficiency virus type 1 testing using blind proficiency testing. Sets of three serum samples, including one serum sample with negative reactions in antibody tests, one serum sample with positive reactions, and one that gave false-positive results with certain testing kits, were sent as routine patient specimens to testing laboratories. Half the laboratories reported the serum sample positive for human immunodeficiency virus antibodies as "indeterminate"; one laboratory rendered a final positive report without supplemental testing. On the report forms, the actual laboratory results were often obscured and intermingled with information, sometimes incorrect, such as identifying the agent as "HTLV-III" (human T-cell lymphotropic virus type III) and advising that a test with positive results is evidence of exposure to the virus. Many of these reports have the potential to confuse, rather than to enlighten, the requesting physician.

Phenylalanine analyses of blood-spot control materials: preparation of samples and evaluation of interlaboratory performance.

Aliquots (0.1 mL) of whole-blood pools prepared to contain various concentrations of phenylalanine were applied to filter-paper collection cards, dried, and stored in sealed bags. We measured the phenylalanine content of the dried blood spots by bioassay, fluorometry, and "high-performance" liquid chromatography, and found that the concentrations remained constant for two years when samples were kept at -20 degrees C or lower. Intra- and interlaboratory studies showed that results for phenylalanine were greater for laboratories using bioassay procedures than for those using fluorometric procedures. Further, CVs (both among- and within-laboratory) obtained with fluorometric procedures were nearly half as great as the CVs obtained by laboratories using bioassay techniques.

Results of a nationwide survey of analyses for creatine kinase and creatine kinase isoenzymes.

More than 300 laboratories participated in an interlaboratory survey of creatine kinase (CK, EC 2.7.3.2) determinations in which they analyzed seven lyophilized samples for total CK and CK isoenzymes and furnished information about their methodology. The samples were not necessarily intended to mimic typical patients' specimens but rather to determine the analytical ability of the laboratories to distinguish isoenzyme fraction CK-MB from CK-BB and to detect small but abnormal amounts of CK-BB. For total CK measurement, most laboratories used an NADP+ reduction method monitored at 340 nm (89%), and reported results in units per liter (U/L) (99%) at either 30 degrees C (34%) or 37 degrees C (60%). Despite the variety of analytical conditions, most laboratories (89%) correctly reported results within their normal range for all samples. The 287 laboratories that reported isoenzyme distributions in the samples used either cellulose acetate (37%) or agarose (44%) electrophoresis, ion-exchange chromatography (9%), or immunoinhibition (7%). Results from laboratories that used nonspecific CK-MB immunoinhibition techniques were biased when a significant amount of CK-BB isoenzyme was present.

Laboratory evaluation and assistance efforts: mailed, on-site and blind proficiency testing surveys conducted by the Centers for Disease Control.

During the last three years, the Centers for Disease Control (CDC) has conducted: 1) on-site surveys in which trained personnel visited laboratories that had experienced performance problems in the quarterly mailed proficiency testing (PT) program, reviewing the laboratories' analytical procedures by using carefully referenced samples to determine sources of errors and providing assistance in correcting them; 2) special assistance surveys in which carefully referenced samples were mailed to laboratories that had performed unsatisfactorily in routine mailed PT surveys and then telephone consultations were conducted to correct the problems; and 3) blind surveys in which carefully referenced samples were sent through normal patient sample acquisition routes to assess the actual day-to-day performance capability of the laboratories. Results suggest that on-site surveys by trained laboratory surveyors and special mailed assistance surveys can be very effective in identifying the source of analytical errors in laboratories previously found, through mailed PT surveys, to have performance problems. Blind-survey results indicate that good performance in mailed PT does not necessarily imply good laboratory performance with routine patient specimens. Although difficult to conduct, blind surveys should be conducted whenever the logistics can be worked out by contractors for laboratory services, clinicians using laboratory services, and the laboratories themselves to assure the continuation of quality service.

Interlaboratory surveys of the quantitation of thyroxin and thyrotropin (thyroid-stimulating hormone) in dried blood spot specimens.

Screening of newborns for hypothyroidism is mandated by law in most states. The screening usually consists of measuring thyroxin in dried blood spot specimens followed by measurement of thyrotropin if the results for thyroxin are suggestive of hypothyroidism. The first nationwide interlaboratory surveys designed to assess the proficiency of screening laboratories in the identification of euthyroid and hypothyroid specimens are reported here. Each survey consisted of three specimens mailed quarterly from late 1979 through 1980. A total of 88 laboratories participated in at least one of the four surveys. More than 17 different methods or diagnostic kits were used by participants in the surveys. Coefficients of variation ranged from 20% to 41% for thyroxin data and from 23% to 63% for thyrotropin. Despite the large analytical interlaboratory variation, most participants assigned clinical classifications consistent with the survey design. The incidence of mis-classifications was low.

Smoking and disease: effect on serum antitrypsin in hospitalized patients.

The mean serum antitrypsin (AT) activity for 1,829 patients hospitalized with medical problems exceeded the mean for a group of blood bank donors by 51%. Analysis of data from individual patients in terms of the general category (there were 16) of their disease and smoking status (smoker, nonsmoker, former smoker) indicated significant differences due to disease status and smoking status. The highest mean AT levels were associated with infectious, respiratory, and neoplastic diseases. Smokers had significantly higher mean levels than nonsmokers, and lung cancer patients had significantly higher mean levels than those with other malignant neoplasms. Among smokers (as a group) mean AT levels. through elevated relative to nonsmokers were not significantly related to duration or intensity of smoking; but former smokers showed a decline (with time following cessation of smoking) in their mean AT level to the mean level for nonsmokers. These findings provide further evidence of the sensitivity of serum antitrypsin activity to environmental influences.