A Question-and-Answer System to Extract Data From Free-Text Oncological Pathology Reports (CancerBERT Network): Development Study.

BACKGROUND: Information in pathology reports is critical for cancer care. Natural language processing (NLP) systems used to extract information from pathology reports are often narrow in scope or require extensive tuning. Consequently, there is growing interest in automated deep learning approaches. A powerful new NLP algorithm, bidirectional encoder representations from transformers (BERT), was published in late 2018. BERT set new performance standards on tasks as diverse as question answering, named entity recognition, speech recognition, and more. OBJECTIVE: The aim of this study is to develop a BERT-based system to automatically extract detailed tumor site and histology information from free-text oncological pathology reports. METHODS: We pursued three specific aims: extract accurate tumor site and histology descriptions from free-text pathology reports, accommodate the diverse terminology used to indicate the same pathology, and provide accurate standardized tumor site and histology codes for use by downstream applications. We first trained a base language model to comprehend the technical language in pathology reports. This involved unsupervised learning on a training corpus of 275,605 electronic pathology reports from 164,531 unique patients that included 121 million words. Next, we trained a question-and-answer (Q&A) model that connects a Q&A layer to the base pathology language model to answer pathology questions. Our Q&A system was designed to search for the answers to two predefined questions in each pathology report: What organ contains the tumor? and What is the kind of tumor or carcinoma? This involved supervised training on 8197 pathology reports, each with ground truth answers to these 2 questions determined by certified tumor registrars. The data set included 214 tumor sites and 193 histologies. The tumor site and histology phrases extracted by the Q&A model were used to predict International Classification of Diseases for Oncology, Third Edition (ICD-O-3), site and histology codes. This involved fine-tuning two additional BERT models: one to predict site codes and another to predict histology codes. Our final system includes a network of 3 BERT-based models. We call this CancerBERT network (caBERTnet). We evaluated caBERTnet using a sequestered test data set of 2050 pathology reports with ground truth answers determined by certified tumor registrars. RESULTS: caBERTnet's accuracies for predicting group-level site and histology codes were 93.53% (1895/2026) and 97.6% (1993/2042), respectively. The top 5 accuracies for predicting fine-grained ICD-O-3 site and histology codes with 5 or more samples each in the training data set were 92.95% (1794/1930) and 96.01% (1853/1930), respectively. CONCLUSIONS: We have developed an NLP system that outperforms existing algorithms at predicting ICD-O-3 codes across an extensive range of tumor sites and histologies. Our new system could help reduce treatment delays, increase enrollment in clinical trials of new therapies, and improve patient outcomes.

Weakly Supervised Skull Stripping of Magnetic Resonance Imaging of Brain Tumor Patients.

Automatic brain tumor segmentation is particularly challenging on magnetic resonance imaging (MRI) with marked pathologies, such as brain tumors, which usually cause large displacement, abnormal appearance, and deformation of brain tissue. Despite an abundance of previous literature on learning-based methodologies for MRI segmentation, few works have focused on tackling MRI skull stripping of brain tumor patient data. This gap in literature can be associated with the lack of publicly available data (due to concerns about patient identification) and the labor-intensive nature of generating ground truth labels for model training. In this retrospective study, we assessed the performance of Dense-Vnet in skull stripping brain tumor patient MRI trained on our large multi-institutional brain tumor patient dataset. Our data included pretreatment MRI of 668 patients from our in-house institutional review board-approved multi-institutional brain tumor repository. Because of the absence of ground truth, we used imperfect automatically generated training labels using SPM12 software. We trained the network using common MRI sequences in oncology: T1-weighted with gadolinium contrast, T2-weighted fluid-attenuated inversion recovery, or both. We measured model performance against 30 independent brain tumor test cases with available manual brain masks. All images were harmonized for voxel spacing and volumetric dimensions before model training. Model training was performed using the modularly structured deep learning platform NiftyNet that is tailored toward simplifying medical image analysis. Our proposed approach showed the success of a weakly supervised deep learning approach in MRI brain extraction even in the presence of pathology. Our best model achieved an average Dice score, sensitivity, and specificity of, respectively, 94.5, 96.4, and 98.5% on the multi-institutional independent brain tumor test set. To further contextualize our results within existing literature on healthy brain segmentation, we tested the model against healthy subjects from the benchmark LBPA40 dataset. For this dataset, the model achieved an average Dice score, sensitivity, and specificity of 96.2, 96.6, and 99.2%, which are, although comparable to other publications, slightly lower than the performance of models trained on healthy patients. We associate this drop in performance with the use of brain tumor data for model training and its influence on brain appearance.

Deep neural network to locate and segment brain tumors outperformed the expert technicians who created the training data.

Purpose: Deep learning (DL) algorithms have shown promising results for brain tumor segmentation in MRI. However, validation is required prior to routine clinical use. We report the first randomized and blinded comparison of DL and trained technician segmentations. Approach: We compiled a multi-institutional database of 741 pretreatment MRI exams. Each contained a postcontrast T1-weighted exam, a T2-weighted fluid-attenuated inversion recovery exam, and at least one technician-derived tumor segmentation. The database included 729 unique patients (470 males and 259 females). Of these exams, 641 were used for training the DL system, and 100 were reserved for testing. We developed a platform to enable qualitative, blinded, controlled assessment of lesion segmentations made by technicians and the DL method. On this platform, 20 neuroradiologists performed 400 side-by-side comparisons of segmentations on 100 test cases. They scored each segmentation between 0 (poor) and 10 (perfect). Agreement between segmentations from technicians and the DL method was also evaluated quantitatively using the Dice coefficient, which produces values between 0 (no overlap) and 1 (perfect overlap). Results: The neuroradiologists gave technician and DL segmentations mean scores of 6.97 and 7.31, respectively ( p < 0.00007 ). The DL method achieved a mean Dice coefficient of 0.87 on the test cases. Conclusions: This was the first objective comparison of automated and human segmentation using a blinded controlled assessment study. Our DL system learned to outperform its "human teachers" and produced output that was better, on average, than its training data.

QuantitativeT2: interactive quantitative T2 MRI witnessed in mouse glioblastoma.

The aim of this study was to establish an advanced analytical platform for complex in vivo pathologies. We have developed a software program, QuantitativeT2, for voxel-based real-time quantitative T2 magnetic resonance imaging. We analyzed murine brain tumors to confirm feasibility of our method for neurological conditions. Anesthetized mice (with invasive gliomas, and controls) were imaged on a 9.4 Tesla scanner using a Carr-Purcell-Meiboom-Gill sequence. The multiecho T2 decays from axial brain slices were analyzed using QuantitativeT2. T2 distribution histograms demonstrated substantial characteristic differences between normal and pathological brain tissues. Voxel-based quantitative maps of tissue water fraction (WF) and geometric mean T2 (gmT2) revealed the heterogeneous alterations to water compartmentalization caused by pathology. The numeric distribution of WF and gmT2 indicated the extent of tumor infiltration. Relative evaluations between in vivo scans and ex vivo histology indicated that the T2s between 30 and 150 ms were related to cellular density and the integrity of the extracellular matrix. Overall, QuantitativeT2 has demonstrated significant advancements in qT2 analysis with real-time operation. It is interactive with an intuitive workflow; can analyze data from many MR manufacturers; and is released as open-source code to encourage examination, improvement, and expansion of this method.

Texture analysis of images using a two-dimensional fast time-frequency transform.

The two-dimensional S-transform (ST-2D) is a time-frequency representation that is widely used in medical image processing but prohibitive in both storage and computation time. The high computation time required for generating local spectrum discourages the use of ST-2D for analyzing textures in medical images. A two-dimensional fast time-frequency transform (FTFT-2D) for computing the local spectrum instantaneously and accurately is proposed. It can also be used to compute the complete redundant discrete ST-2D output, if needed. It reduces the storage requirement by generating a compressed form of the ST-2D. In addition, the memory efficient and adaptive nature of the FTFT-2D make it suitable for user-specific requirements.

Selection and pilot implementation of a mobile image viewer: a case study.

BACKGROUND: For health care providers, mobile image viewing increases image accessibility, which could lead to faster interpretation/consultations and improved patient outcomes. OBJECTIVE: We explored the technical requirements and challenges associated with implementing a commercial mobile image viewer and conducted a small study testing the hypothesis that the mobile image viewer would provide faster image access. METHODS: A total of 19 clinicians (9 radiologists, 3 surgeons, 4 neurologists, and 3 physician assistants) evaluated (1) a desktop commercial picture archiving and communication system (PACS) viewer, (2) a desktop viewer developed internally over 20 years and deployed throughout the enterprise (ENTERPRISE viewer) and (3) a commercial Food and Drug Administration class II-cleared mobile viewer compatible with Web browsers, tablets, and mobile phones. Data were collected during two separate 7-day periods, before and after mobile image viewer deployment. Data included image viewer chosen, time to view first image, technical issues, diagnostic confidence, and ease of use. RESULTS: For 565 image-viewing events, ease of use was identical for PACS and mobile viewers (mean 3.6 for all scores of a possible 4.0), and significantly worse for the enterprise viewer (mean 2.9, P=.001). Technical issues were highest with the enterprise viewer (26%, 56/215) compared with the mobile (7%,19/259, P=.001) and PACS (8%, 7/91, P=.003) viewers. Mean time to first image for the mobile viewer (2.4 minutes) was significantly faster than PACS (12.5 minutes, P=.001) and the enterprise viewer (4.5 minutes, P=.001). Diagnostic confidence was similar for PACS and mobile viewers and worst for enterprise viewer. Mobile image viewing increased by sixfold, from 14% (37/269, before the deployment) to 88.9% (263/296, after the deployment). CONCLUSIONS: A mobile viewer provided faster time to first image, improved technical performance, ease of use, and diagnostic confidence, compared with desktop image viewers.