Papnet-assisted cytological diagnosis intensifies the already marked variability among cytological laboratories.

ABSTRACT

OBJECTIVE: The main objective was to assess the sensitivity, specificity and reliability of PAPNET-assisted diagnosis in comparison with conventional screening. SETTING: Seven Italian and one English University or Research Institutes, and a random sample of an other 20 Italian Laboratories of the Italian National Health Service (INHS) provided the cervical smears. METHODS: During the training phase every center examined in rotation four sets of slides for a total of 300 representative slides. Afterwards, 900 "positive" slides were added to the 3,100 slides which were collected consecutively without any selection or exclusion. The eight main centers were divided into four couples and each couple of centers examined 775 slides with the PAPNET system, "blindly" to the original diagnosis. An expert cytopathologist (M.A.) of the National Institute of Health (NIH) reassessed 40% of the slides with an original negative diagnosis to evaluate the false negative rate. Two expert NIH cytopathologists (M.A., G.M.) re-examined all slides where a disagreement had been observed between the original and one or both of the study diagnoses. The main analyses concerned the following three main categories WNL and unsatisfactory for evaluation; ASCUS, AGUS and LSIL; HSIL and carcinoma. A special algorithm was devised to define the reference diagnosis for sensitivity and specificity assessment. RESULTS: Laboratories, even belonging to the same couple, classified as "no review" a very different proportion of slides ranging from 35% to 74%. The index of kappa agreement between the members of couples examining the same sets of slides was low or very low, ranging from 0.30 to 0.03. The sensitivity of the review classification was particularly low in some laboratories. Surprisingly, only a small correlation was observed between the sensitivity of the review classification and the proportion of slides classified as "review". The "tentative" diagnosis on PAPNET tiles of the "review" slides was almost as reliable as the microscopic diagnosis. In the overall performance, there were many significant differences among the eight laboratories. The best laboratory had a sensitivity of 95% and a specificity of 96%. At least three laboratories displayed unacceptably low sensitivity and one a very low specificity. CONCLUSION: Altogether these results seem to confirm that there are wide differences among cytological laboratories per se, and that these differences are intensified by the use of an instrument like PAPNET. The huge variation in performance may be explained by differences in basic skills and by different training, but it is difficult to understand exactly what could have been done to reduce it.