Evaluation of the extent of nonlinearity in reportable range studies.

OBJECTIVES: To extend the polynomial method for evaluating linearity in 2 ways. First, we developed a screen to ascertain whether the data were precise enough to permit a reliable evaluation of linearity and therefore eliminate findings of linearity due to low statistical power. Second, we assessed whether the degree of nonlinearity detected by the polynomial method was clinically relevant using a statistically rigorous method. METHODS: Because we assessed linearity relative to a clinically determined level of importance instead of the default value of zero, we used sampling theory based on the noncentral chi(2) distribution. Using statistical power calculations, we incorporated a screen for imprecision that guarantees that the probability of correctly identifying nonlinear methods is at least 80%. RESULTS: With the described methods, we achieved a sensitivity of at least 80% and a specificity of at least 95%. When the data were too imprecise to achieve a sensitivity of 80%, no determination of linearity was made. This procedure mimics the practice in manual inspection of flagging data that appear imprecise by visual inspection and halting the evaluation. CONCLUSIONS: Formal statistical tests for precision and amount of nonlinearity are advantageous because they allow us to quantify and limit classification errors. By formalizing these various aspects of linearity assessment, we maintain some of the complex features of manual methods while making the linearity assessment feasible to apply to a high volume of assessments and removing the between-analyst variability.

Reparative changes and the false-positive/false-negative Papanicolaou test: a study from the College of American Pathologists Interlaboratory Comparison Program in Cervicovaginal Cytology .

CONTEXT: Reparative changes can be a diagnostic challenge on Papanicolaou tests and must be distinguished from epithelial cell abnormalities. Both squamous intraepithelial lesions (SIL) and carcinoma may be underdiagnosed as repair. This study examines laboratory and cytologist performance in the diagnosis of repair. OBJECTIVES: To determine if laboratories and cytologists can consistently distinguish reparative changes from SIL and carcinoma and to document how often SIL and carcinoma are mistaken for repair in a standardized educational slide program. DESIGN: Results for reparative changes, SIL, and carcinoma slides from the College of American Pathologists Interlaboratory Comparison Program in Cervicovaginal Cytology for 1998 were analyzed. Only results from validated referenced slide responses were used. RESULTS: The concordancy rate for reparative-change slides, that is, any response of within normal limits or benign cellular changes, ranged from 91% to 94% for cytotechnologist, pathologist, and laboratory responses. False-negative rates for squamous carcinoma and adenocarcinoma, high-grade SIL, and low-grade SIL ranged from 0.47% to 5.41%; the proportion of false-negative diagnoses of reparative changes ranged from 24% to 62% of all discordant responses. CONCLUSIONS: Of all benign cellular changes and within normal limits categories in the Interlaboratory Comparison Program in Cervicovaginal Cytology, repair most often elicits a false-positive laboratory response. Underdiagnosing epithelial abnormalities as repair is also a source of false-negative Papanicolaou test results.

Interobserver variability in subclassification of squamous intraepithelial lesions: Results of the College of American Pathologists Interlaboratory Comparison Program in Cervicovaginal Cytology.

OBJECTIVE: To determine whether, on a national cytology proficiency test, a competent cytologist can consistently distinguish grades of squamous intraepithelial lesions. DESIGN: Results for low- and high-grade squamous intraepithelial lesion referenced slides from the College of American Pathologists Interlaboratory Comparison Program in Cervicovaginal Cytology for 1996 and 1997 were analyzed including educational, nongraded vs graded validated slides. RESULTS: The discrepant rate between low- and high- grade lesions ranged from 9.8% to 15% for cytotechnologist, pathologist, laboratory, and all responses. There was a statistically significant difference in performance on graded, validated slides vs educational slides with better performance on validated slides. CONCLUSION: This significant interobserver variability in subclassification of squamous lesions should be considered in management guidelines for abnormal Papanicolaou test results and implementation of national cytology proficiency testing.

A method for proficiency testing of small peer groups in the College of American Pathologists Coagulation Surveys.

OBJECTIVE: To develop a grading scheme for the proficiency testing of small peer groups of fewer than 10 members for the prothrombin time (PT) and activated partial thromboplastin time (APTT). METHODS: A modified target value for small peer groups was derived based on the assumption that measurement variability in the PT and APTT is more greatly influenced by variations in reagents than in instruments. Criteria for grading were established by statistical simulation to achieve misclassification errors of less than 5% for both incorrectly passing and failing participants. College of American Pathologists Coagulation Survey data were analyzed to determine the number of additional laboratories graded using the proposed scheme, as well as the failure rates among participants in the small peer groups. RESULTS: The modified target value for small peer groups is a weighted average between the mean of the peer group and the mean of all participants using the same reagent (reagent group). Peer groups with as few as 4 members can be graded provided that specific criteria are satisfied: there must be at least 5 peer groups for the same reagent, at least 3 of these 5 peer groups must have more than 3 members, and the coefficient of variation for the reagent group must be less than 10%. This proposed grading scheme decreased the number of ungraded laboratories by 44% to 46% for the PT and 42% to 55% for the APTT. The percentage of failing grades among participants in the small peer groups ranged from 1.3% to 4.1% for the PT and 1.4% to 7.2% for the APTT. These failure rates were 2.8- to 13.0-fold higher than the failure rates in large peer groups (P < or = .05). CONCLUSIONS: The proposed small peer group grading scheme can improve the effectiveness of College of American Pathologists proficiency testing for the PT and APTT and may also be generally applicable to other test methods and analytes.