Thyroid Cytopathology Cancer Diagnosis from Smartphone Images Using Machine Learning.

We examined the performance of deep learning models on the classification of thyroid fine-needle aspiration biopsies using microscope images captured in 2 ways: with a high-resolution scanner and with a mobile phone camera. Our training set consisted of images from 964 whole-slide images captured with a high-resolution scanner. Our test set consisted of 100 slides; 20 manually selected regions of interest (ROIs) from each slide were captured in 2 ways as mentioned above. Applying a baseline machine learning algorithm trained on scanner ROIs resulted in performance deterioration when applied to the smartphone ROIs (97.8% area under the receiver operating characteristic curve [AUC], CI = [95.4%, 100.0%] for scanner images vs 89.5% AUC, CI = [82.3%, 96.6%] for mobile images, P = .019). Preliminary analysis via histogram matching showed that the baseline model was overly sensitive to slight color variations in the images (specifically, to color differences between mobile and scanner images). Adding color augmentation during training reduces this sensitivity and narrows the performance gap between mobile and scanner images (97.6% AUC, CI = [95.0%, 100.0%] for scanner images vs 96.0% AUC, CI = [91.8%, 100.0%] for mobile images, P = .309), with both modalities on par with human pathologist performance (95.6% AUC, CI = [91.6%, 99.5%]) for malignancy prediction (P = .398 for pathologist vs scanner and P = .875 for pathologist vs mobile). For indeterminate cases (pathologist-assigned Bethesda category of 3, 4, or 5), color augmentations confer some improvement (88.3% AUC, CI = [73.7%, 100.0%] for the baseline model vs 96.2% AUC, CI = [90.9%, 100.0%] with color augmentations, P = .158). In addition, we found that our model's performance levels off after 15 ROIs, a promising indication that ROI data collection would not be time-consuming for our diagnostic system. Finally, we showed that the model has sensible Bethesda category (TBS) predictions (increasing risk malignancy rate with predicted TBS category, with 0% malignancy for predicted TBS 2 and 100% malignancy for TBS 6).

Use of Machine Learning-Based Software for the Screening of Thyroid Cytopathology Whole Slide Images.

CONTEXT.­: The use of whole slide images (WSIs) in diagnostic pathology presents special challenges for the cytopathologist. Informative areas on a direct smear from a thyroid fine-needle aspiration biopsy (FNAB) smear may be spread across a large area comprising blood and dead space. Manually navigating through these areas makes screening and evaluation of FNA smears on a digital platform time-consuming and laborious. We designed a machine learning algorithm that can identify regions of interest (ROIs) on thyroid fine-needle aspiration biopsy WSIs. OBJECTIVE.­: To evaluate the ability of the machine learning algorithm and screening software to identify and screen for a subset of informative ROIs on a thyroid FNA WSI that can be used for final diagnosis. DESIGN.­: A representative slide from each of 109 consecutive thyroid fine-needle aspiration biopsies was scanned. A cytopathologist reviewed each WSI and recorded a diagnosis. The machine learning algorithm screened and selected a subset of 100 ROIs from each WSI to present as an image gallery to the same cytopathologist after a washout period of 117 days. RESULTS.­: Concordance between the diagnoses using WSIs and those using the machine learning algorithm-generated ROI image gallery was evaluated using pairwise weighted κ statistics. Almost perfect concordance was seen between the 2 methods with a κ score of 0.924. CONCLUSIONS.­: Our results show the potential of the screening software as an effective screening tool with the potential to reduce cytopathologist workloads.

A Hybrid Human-Machine Learning Approach for Screening Prostate Biopsies Can Improve Clinical Efficiency Without Compromising Diagnostic Accuracy.

CONTEXT.­: Prostate cancer is a common malignancy, and accurate diagnosis typically requires histologic review of multiple prostate core biopsies per patient. As pathology volumes and complexity increase, new tools to improve the efficiency of everyday practice are keenly needed. Deep learning has shown promise in pathology diagnostics, but most studies silo the efforts of pathologists from the application of deep learning algorithms. Very few hybrid pathologist-deep learning approaches have been explored, and these typically require complete review of histologic slides by both the pathologist and the deep learning system. OBJECTIVE.­: To develop a novel and efficient hybrid human-machine learning approach to screen prostate biopsies. DESIGN.­: We developed an algorithm to determine the 20 regions of interest with the highest probability of malignancy for each prostate biopsy; presenting these regions to a pathologist for manual screening limited the initial review by a pathologist to approximately 2% of the tissue area of each sample. We evaluated this approach by using 100 biopsies (29 malignant, 60 benign, 11 other) that were reviewed by 4 pathologists (3 urologic pathologists, 1 general pathologist) using a custom-designed graphical user interface. RESULTS.­: Malignant biopsies were correctly identified as needing comprehensive review with high sensitivity (mean, 99.2% among all pathologists); conversely, most benign prostate biopsies (mean, 72.1%) were correctly identified as needing no further review. CONCLUSIONS.­: This novel hybrid system has the potential to efficiently triage out most benign prostate core biopsies, conserving time for the pathologist to dedicate to detailed evaluation of malignant biopsies.

Weakly supervised instance learning for thyroid malignancy prediction from whole slide cytopathology images.

We consider machine-learning-based thyroid-malignancy prediction from cytopathology whole-slide images (WSI). Multiple instance learning (MIL) approaches, typically used for the analysis of WSIs, divide the image (bag) into patches (instances), which are used to predict a single bag-level label. These approaches perform poorly in cytopathology slides due to a unique bag structure: sparsely located informative instances with varying characteristics of abnormality. We address these challenges by considering multiple types of labels: bag-level malignancy and ordered diagnostic scores, as well as instance-level informativeness and abnormality labels. We study their contribution beyond the MIL setting by proposing a maximum likelihood estimation (MLE) framework, from which we derive a two-stage deep-learning-based algorithm. The algorithm identifies informative instances and assigns them local malignancy scores that are incorporated into a global malignancy prediction. We derive a lower bound of the MLE, leading to an improved training strategy based on weak supervision, that we motivate through statistical analysis. The lower bound further allows us to extend the proposed algorithm to simultaneously predict multiple bag and instance-level labels from a single output of a neural network. Experimental results demonstrate that the proposed algorithm provides competitive performance compared to several competing methods, achieves (expert) human-level performance, and allows augmentation of human decisions.