Overcoming an Annotation Hurdle: Digitizing Pen Annotations from Whole Slide Images.

BACKGROUND: The development of artificial intelligence (AI) in pathology frequently relies on digitally annotated whole slide images (WSI). The creation of these annotations - manually drawn by pathologists in digital slide viewers - is time consuming and expensive. At the same time, pathologists routinely annotate glass slides with a pen to outline cancerous regions, for example, for molecular assessment of the tissue. These pen annotations are currently considered artifacts and excluded from computational modeling. METHODS: We propose a novel method to segment and fill hand-drawn pen annotations and convert them into a digital format to make them accessible for computational models. Our method is implemented in Python as an open source, publicly available software tool. RESULTS: Our method is able to extract pen annotations from WSI and save them as annotation masks. On a data set of 319 WSI with pen markers, we validate our algorithm segmenting the annotations with an overall Dice metric of 0.942, Precision of 0.955, and Recall of 0.943. Processing all images takes 15 min in contrast to 5 h manual digital annotation time. Further, the approach is robust against text annotations. CONCLUSIONS: We envision that our method can take advantage of already pen-annotated slides in scenarios in which the annotations would be helpful for training computational models. We conclude that, considering the large archives of many pathology departments that are currently being digitized, our method will help to collect large numbers of training samples from those data.

Multi-magnification-based machine learning as an ancillary tool for the pathologic assessment of shaved margins for breast carcinoma lumpectomy specimens.

The surgical margin status of breast lumpectomy specimens for invasive carcinoma and ductal carcinoma in situ (DCIS) guides clinical decisions, as positive margins are associated with higher rates of local recurrence. The "cavity shave" method of margin assessment has the benefits of allowing the surgeon to orient shaved margins intraoperatively and the pathologist to assess one inked margin per specimen. We studied whether a deep convolutional neural network, a deep multi-magnification network (DMMN), could accurately segment carcinoma from benign tissue in whole slide images (WSIs) of shave margin slides, and therefore serve as a potential screening tool to improve the efficiency of microscopic evaluation of these specimens. Applying the pretrained DMMN model, or the initial model, to a validation set of 408 WSIs (348 benign, 60 with carcinoma) achieved an area under the curve (AUC) of 0.941. After additional manual annotations and fine-tuning of the model, the updated model achieved an AUC of 0.968 with sensitivity set at 100% and corresponding specificity of 78%. We applied the initial model and updated model to a testing set of 427 WSIs (374 benign, 53 with carcinoma) which showed AUC values of 0.900 and 0.927, respectively. Using the pixel classification threshold selected from the validation set, the model achieved a sensitivity of 92% and specificity of 78%. The four false-negative classifications resulted from two small foci of DCIS (1 mm, 0.5 mm) and two foci of well-differentiated invasive carcinoma (3 mm, 1.5 mm). This proof-of-principle study demonstrates that a DMMN machine learning model can segment invasive carcinoma and DCIS in surgical margin specimens with high accuracy and has the potential to be used as a screening tool for pathologic assessment of these specimens.

(Re) Defining the High-Power Field for Digital Pathology.

BACKGROUND: The microscope high-power field (HPF) is the cornerstone for histopathology diagnostic evaluation such as the quantification of mitotic figures, lymphocytes, and tumor grading. With traditional light microscopy, HPFs are typically evaluated by quantifying histologic events in 10 fields of view at × 400 magnification. In the era of digital pathology, new variables are introduced that may affect HPF evaluation. The aim of this study was to determine the parameters that influence HPF in whole slide images (WSIs). MATERIALS AND METHODS: Glass slides scanned on various devices (Leica's Aperio GT450, AT2, and ScanScope XT; Philips UltraFast Scanner; Hamamatsu's Nanozoomer 2.0HT; and 3DHistech's P1000) were compared to acquired digital slides reviewed on each vendor's respective WSI viewer software (e.g., Aperio ImageScope, ImageScope DX, Philips IMS, 3DHistech CaseViewer, and Hamamatsu NDP.view) and an in-house developed vendor-agnostic viewer. WSIs were reviewed at "×40" equivalent HPF on different sized monitors with varying display resolutions (1900 × 1080-4500 × 3000) and aspect ratios (e.g., Food and Drug Administration [FDA]-cleared 27" Philips PS27QHDCR, FDA-cleared 24" Dell MR2416, 24" Hewlett Packard Z24n G2, and 28" Microsoft Surface Studio). Digital and microscopic HPF areas were calculated and compared. RESULTS: A significant variation of HPF area occurred between differing monitor size and display resolutions with minor differences between WSI viewers. No differences were identified by scanner or WSIs scanned at different resolutions (e.g., 0.5, 0.25, 0.24, and 0.12 µm/pixel). CONCLUSION: Glass slide HPF at × 400 magnification with conventional light microscopy was not equivalent to "×40" digital HPF areas. Digital HPF quantification may vary due to differences in the tissue area displayed by monitor sizes, display resolutions, and WSI viewers but not by scanner or scanning resolution. These findings will need to be further clinically validated with potentially new digital metrics for evaluation.