Leveraging Serial Low-Dose CT Scans in Radiomics-based Reinforcement Learning to Improve Early Diagnosis of Lung Cancer at Baseline Screening.

Purpose To evaluate the feasibility of leveraging serial low-dose CT (LDCT) scans to develop a radiomics-based reinforcement learning (RRL) model for improving early diagnosis of lung cancer at baseline screening. Materials and Methods In this retrospective study, 1951 participants (female patients, 822; median age, 61 years [range, 55-74 years]) (male patients, 1129; median age, 62 years [range, 55-74 years]) were randomly selected from the National Lung Screening Trial between August 2002 and April 2004. An RRL model using serial LDCT scans (S-RRL) was trained and validated using data from 1404 participants (372 with lung cancer) containing 2525 available serial LDCT scans up to 3 years. A baseline RRL (B-RRL) model was trained with only LDCT scans acquired at baseline screening for comparison. The 547 held-out individuals (150 with lung cancer) were used as an independent test set for performance evaluation. The area under the receiver operating characteristic curve (AUC) and the net reclassification index (NRI) were used to assess the performances of the models in the classification of screen-detected nodules. Results Deployment to the held-out baseline scans showed that the S-RRL model achieved a significantly higher test AUC (0.88 [95% CI: 0.85, 0.91]) than both the Brock model (AUC, 0.84 [95% CI: 0.81, 0.88]; P = .02) and the B-RRL model (AUC, 0.86 [95% CI: 0.83, 0.90]; P = .02). Lung cancer risk stratification was significantly improved by the S-RRL model as compared with Lung CT Screening Reporting and Data System (NRI, 0.29; P < .001) and the Brock model (NRI, 0.12; P = .008). Conclusion The S-RRL model demonstrated the potential to improve early diagnosis and risk stratification for lung cancer at baseline screening as compared with the B-RRL model and clinical models. Keywords: Radiomics-based Reinforcement Learning, Lung Cancer Screening, Low-Dose CT, Machine Learning © RSNA, 2024 Supplemental material is available for this article.

Detection of Severe Lung Infection on Chest Radiographs of COVID-19 Patients: Robustness of AI Models across Multi-Institutional Data.

The diagnosis of severe COVID-19 lung infection is important because it carries a higher risk for the patient and requires prompt treatment with oxygen therapy and hospitalization while those with less severe lung infection often stay on observation. Also, severe infections are more likely to have long-standing residual changes in their lungs and may need follow-up imaging. We have developed deep learning neural network models for classifying severe vs. non-severe lung infections in COVID-19 patients on chest radiographs (CXR). A deep learning U-Net model was developed to segment the lungs. Inception-v1 and Inception-v4 models were trained for the classification of severe vs. non-severe COVID-19 infection. Four CXR datasets from multi-country and multi-institutional sources were used to develop and evaluate the models. The combined dataset consisted of 5748 cases and 6193 CXR images with physicians' severity ratings as reference standard. The area under the receiver operating characteristic curve (AUC) was used to evaluate model performance. We studied the reproducibility of classification performance using the different combinations of training and validation data sets. We also evaluated the generalizability of the trained deep learning models using both independent internal and external test sets. The Inception-v1 based models achieved AUC ranging between 0.81 ± 0.02 and 0.84 ± 0.0, while the Inception-v4 models achieved AUC in the range of 0.85 ± 0.06 and 0.89 ± 0.01, on the independent test sets, respectively. These results demonstrate the promise of using deep learning models in differentiating COVID-19 patients with severe from non-severe lung infection on chest radiographs.

Improved Repeatability of Mouse Tibia Volume Segmentation in Murine Myelofibrosis Model Using Deep Learning.

A murine model of myelofibrosis in tibia was used in a co-clinical trial to evaluate segmentation methods for application of image-based biomarkers to assess disease status. The dataset (32 mice with 157 3D MRI scans including 49 test-retest pairs scanned on consecutive days) was split into approximately 70% training, 10% validation, and 20% test subsets. Two expert annotators (EA1 and EA2) performed manual segmentations of the mouse tibia (EA1: all data; EA2: test and validation). Attention U-net (A-U-net) model performance was assessed for accuracy with respect to EA1 reference using the average Jaccard index (AJI), volume intersection ratio (AVI), volume error (AVE), and Hausdorff distance (AHD) for four training scenarios: full training, two half-splits, and a single-mouse subsets. The repeatability of computer versus expert segmentations for tibia volume of test-retest pairs was assessed by within-subject coefficient of variance (%wCV). A-U-net models trained on full and half-split training sets achieved similar average accuracy (with respect to EA1 annotations) for test set: AJI = 83-84%, AVI = 89-90%, AVE = 2-3%, and AHD = 0.5 mm-0.7 mm, exceeding EA2 accuracy: AJ = 81%, AVI = 83%, AVE = 14%, and AHD = 0.3 mm. The A-U-net model repeatability wCV [95% CI]: 3 [2, 5]% was notably better than that of expert annotators EA1: 5 [4, 9]% and EA2: 8 [6, 13]%. The developed deep learning model effectively automates murine bone marrow segmentation with accuracy comparable to human annotators and substantially improved repeatability.

Clinical MRI morphological analysis of functional seizures compared to seizure-naïve and psychiatric controls.

PURPOSE: Functional seizures (FS), also known as psychogenic nonepileptic seizures (PNES), are physical manifestations of acute or chronic psychological distress. Functional and structural neuroimaging have identified objective signs of this disorder. We evaluated whether magnetic resonance imaging (MRI) morphometry differed between patients with FS and clinically relevant comparison populations. METHODS: Quality-screened clinical-grade MRIs were acquired from 666 patients from 2006 to 2020. Morphometric features were quantified with FreeSurfer v6. Mixed-effects linear regression compared the volume, thickness, and surface area within 201 regions-of-interest for 90 patients with FS, compared to seizure-naïve patients with depression (n = 243), anxiety (n = 68), and obsessive-compulsive disorder (OCD, n = 41), respectively, and to other seizure-naïve controls with similar quality MRIs, accounting for the influence of multiple confounds including depression and anxiety based on chart review. These comparison populations were obtained through review of clinical records plus research studies obtained on similar scanners. RESULTS: After Bonferroni-Holm correction, patients with FS compared with seizure-naïve controls exhibited thinner bilateral superior temporal cortex (left 0.053 mm, p = 0.014; right 0.071 mm, p = 0.00006), thicker left lateral occipital cortex (0.052 mm, p = 0.0035), and greater left cerebellar white-matter volume (1085 mm3, p = 0.0065). These findings were not accounted for by lower MRI quality in patients with FS. CONCLUSIONS: These results reinforce prior indications of structural neuroimaging correlates of FS and, in particular, distinguish brain morphology in FS from that in depression, anxiety, and OCD. Future work may entail comparisons with other psychiatric disorders including bipolar and schizophrenia, as well as exploration of brain structural heterogeneity within FS.

Recursive Training Strategy for a Deep Learning Network for Segmentation of Pathology Nuclei With Incomplete Annotation.

This study developed a recursive training strategy to train a deep learning model for nuclei detection and segmentation using incomplete annotation. A dataset of 141 H&E stained breast cancer pathologic images with incomplete annotation was randomly split into training/validation set and test set of 89 and 52 images, respectively. The positive training samples were extracted at each annotated cell and augmented with affine translation. The negative training samples were selected from the non-cellular regions free of nuclei using a histogram-based semi-automatic method. A U-Net model was initially trained by minimizing a custom loss function. After the first stage of training, the trained U-Net model was applied to the images in the training set in an inference mode. The U-Net segmented objects with high quality were selected by a semi-automated method. Combining the newly selected high quality objects with the annotated nuclei and the previously generated negative samples, the U-Net model was retrained recursively until the stopping criteria were satisfied. For the 52 test images, the U-Net trained with and without using our recursive training method achieved a sensitivity of 90.3% and 85.3% for nuclei detection, respectively. For nuclei segmentation, the average Dice coefficient and average Jaccard index were 0.831±0.213 and 0.750±0.217, 0.780±0.270 and 0.697±0.264, for U-Net with and without recursive training, respectively. The improvement achieved by our proposed method was statistically significant (P < 0.05). In conclusion, our recursive training method effectively enlarged the set of annotated objects for training the deep learning model and further improved the detection and segmentation performance.

Hybrid U-Net-based deep learning model for volume segmentation of lung nodules in CT images.

OBJECTIVE: Accurate segmentation of the lung nodule in computed tomography images is a critical component of a computer-assisted lung cancer detection/diagnosis system. However, lung nodule segmentation is a challenging task due to the heterogeneity of nodules. This study is to develop a hybrid deep learning (H-DL) model for the segmentation of lung nodules with a wide variety of sizes, shapes, margins, and opacities. MATERIALS AND METHODS: A dataset collected from Lung Image Database Consortium image collection containing 847 cases with lung nodules manually annotated by at least two radiologists with nodule diameters greater than 7 mm and less than 45 mm was randomly split into 683 training/validation and 164 independent test cases. The 50% consensus consolidation of radiologists' annotation was used as the reference standard for each nodule. We designed a new H-DL model combining two deep convolutional neural networks (DCNNs) with different structures as encoders to increase the learning capabilities for the segmentation of complex lung nodules. Leveraging the basic symmetric U-shaped architecture of U-Net, we redesigned two new U-shaped deep learning (U-DL) models that were expanded to six levels of convolutional layers. One U-DL model used a shallow DCNN structure containing 16 convolutional layers adapted from the VGG-19 as the encoder, and the other used a deep DCNN structure containing 200 layers adapted from DenseNet-201 as the encoder, while the same decoder with only one convolutional layer at each level was used in both U-DL models, and we referred to them as the shallow and deep U-DL models. Finally, an ensemble layer was used to combine the two U-DL models into the H-DL model. We compared the effectiveness of the H-DL, the shallow U-DL and the deep U-DL models by deploying them separately to the test set. The accuracy of volume segmentation for each nodule was evaluated by the 3D Dice coefficient and Jaccard index (JI) relative to the reference standard. For comparison, we calculated the median and minimum of the 3D Dice and JI over the individual radiologists who segmented each nodule, referred to as M-Dice, min-Dice, M-JI, and min-JI. RESULTS: For the 164 test cases with 327 nodules, our H-DL model achieved an average 3D Dice coefficient of 0.750 ± 0.135 and an average JI of 0.617 ± 0.159. The radiologists' average M-Dice was 0.778 ± 0.102, and the average M-JI was 0.651 ± 0.127; both were significantly higher than those achieved by the H-DL model (p < 0.05). The radiologists' average min-Dice (0.685 ± 0.139) and the average min-JI (0.537 ± 0.153) were significantly lower than those achieved by the H-DL model (p < 0.05). The results indicated that the H-DL model approached the average performance of radiologists and was superior to the radiologist whose manual segmentation had the min-Dice and min-JI. Moreover, the average Dice and average JI achieved by the H-DL model were significantly higher than those achieved by the individual shallow U-DL model (Dice of 0.745 ± 0.139, JI of 0.611 ± 0.161; p < 0.05) or the individual deep U-DL model alone (Dice of 0.739 ± 0.145, JI of 0.604 ± 0.163; p < 0.05). CONCLUSION: Our newly developed H-DL model outperformed the individual shallow or deep U-DL models. The H-DL method combining multilevel features learned by both the shallow and deep DCNNs could achieve segmentation accuracy comparable to radiologists' segmentation for nodules with wide ranges of image characteristics.

Effect of Dose Level on Radiologists' Detection of Microcalcifications in Digital Breast Tomosynthesis: An Observer Study with Breast Phantoms.

OBJECTIVES: To compare radiologists' sensitivity, confidence level, and reading efficiency of detecting microcalcifications in digital breast tomosynthesis (DBT) at two clinically relevant dose levels. MATERIALS AND METHODS: Six 5-cm-thick heterogeneous breast phantoms embedded with a total of 144 simulated microcalcification clusters of four speck sizes were imaged at two dose modes by a clinical DBT system. The DBT volumes at the two dose levels were read independently by six MQSA radiologists and one fellow with 1-33 years (median 12 years) of experience in a fully-crossed counter-balanced manner. The radiologist located each potential cluster and rated its conspicuity and his/her confidence that the marked location contained a cluster. The differences in the results between the two dose modes were analyzed by two-tailed paired t-test. RESULTS: Compared to the lower-dose mode, the average glandular dose in the higher-dose mode for the 5-cm phantoms increased from 1.34 to 2.07 mGy. The detection sensitivity increased for all speck sizes and significantly for the two smaller sizes (p <0.05). An average of 13.8% fewer false positive clusters was marked. The average conspicuity rating and the radiologists' confidence level were higher for all speck sizes and reached significance (p <0.05) for the three larger sizes. The average reading time per detected cluster reduced significantly (p <0.05) by an average of 13.2%. CONCLUSION: For a 5-cm-thick breast, an increase in average glandular dose from 1.34 to 2.07 mGy for DBT imaging increased the conspicuity of microcalcifications, improved the detection sensitivity by radiologists, increased their confidence levels, reduced false positive detections, and increased the reading efficiency.

Multi-class deep learning segmentation and automated measurements in periodontal sonograms of a porcine model.

OBJECTIVES: Ultrasound emerges as a complement to cone-beam computed tomography in dentistry, but struggles with artifacts like reverberation and shadowing. This study seeks to help novice users recognize soft tissue, bone, and crown of a dental sonogram, and automate soft tissue height (STH) measurement using deep learning. METHODS: In this retrospective study, 627 frames from 111 independent cine loops of mandibular and maxillary premolar and incisors collected from our porcine model (N = 8) were labeled by a reader. 274 premolar sonograms, including data augmentation, were used to train a multi class segmentation model. The model was evaluated against several test sets, including premolar of the same breed (n = 74, Yucatan) and premolar of a different breed (n = 120, Sinclair). We further proposed a rule-based algorithm to automate STH measurements using predicted segmentation masks. RESULTS: The model reached a Dice similarity coefficient of 90.7±4.39%, 89.4±4.63%, and 83.7±10.5% for soft tissue, bone, and crown segmentation, respectively on the first test set (n = 74), and 90.0±7.16%, 78.6±13.2%, and 62.6±17.7% on the second test set (n = 120). The automated STH measurements have a mean difference (95% confidence interval) of -0.22 mm (-1.4, 0.95), a limit of agreement of 1.2 mm, and a minimum ICC of 0.915 (0.857, 0.948) when compared to expert annotation. CONCLUSION: This work demonstrates the potential use of deep learning in identifying periodontal structures on sonograms and obtaining diagnostic periodontal dimensions.

A minority of patients with functional seizures have abnormalities on neuroimaging.

OBJECTIVE: Functional seizures often are managed incorrectly as a diagnosis of exclusion. However, a significant minority of patients with functional seizures may have abnormalities on neuroimaging that typically are associated with epilepsy, leading to diagnostic confusion. We evaluated the rate of epilepsy-associated findings on MRI, FDG-PET, and CT in patients with functional seizures. METHODS: We studied radiologists' reports from neuroimages at our comprehensive epilepsy center from a consecutive series of patients diagnosed with functional seizures without comorbid epilepsy from 2006 to 2019. We summarized the MRI, FDG-PET, and CT results as follows: within normal limits, incidental findings, unrelated findings, non-specific abnormalities, post-operative study, epilepsy risk factors (ERF), borderline epilepsy-associated findings (EAF), and definitive EAF. RESULTS: Of the 256 MRIs, 23% demonstrated ERF (5%), borderline EAF (8%), or definitive EAF (10%). The most common EAF was hippocampal sclerosis, with the majority of borderline EAF comprising hippocampal atrophy without T2 hyperintensity or vice versa. Of the 87 FDG-PETs, 26% demonstrated borderline EAF (17%) or definitive EAF (8%). Epilepsy-associated findings primarily included focal hypometabolism, especially of the temporal lobes, with borderline findings including subtle or questionable hypometabolism. Of the 51 CTs, only 2% had definitive EAF. SIGNIFICANCE: This large case series provides further evidence that, while uncommon, EAF are seen in patients with functional seizures. A significant portion of these abnormal findings are borderline. The moderately high rate of these abnormalities may represent framing bias from the indication of the study being "seizures," the relative subtlety of EAF, or effects of antiseizure medications.

Intraobserver Variability in Bladder Cancer Treatment Response Assessment With and Without Computerized Decision Support.

We evaluated the intraobserver variability of physicians aided by a computerized decision-support system for treatment response assessment (CDSS-T) to identify patients who show complete response to neoadjuvant chemotherapy for bladder cancer, and the effects of the intraobserver variability on physicians' assessment accuracy. A CDSS-T tool was developed that uses a combination of deep learning neural network and radiomic features from computed tomography (CT) scans to detect bladder cancers that have fully responded to neoadjuvant treatment. Pre- and postchemotherapy CT scans of 157 bladder cancers from 123 patients were collected. In a multireader, multicase observer study, physician-observers estimated the likelihood of pathologic T0 disease by viewing paired pre/posttreatment CT scans placed side by side on an in-house-developed graphical user interface. Five abdominal radiologists, 4 diagnostic radiology residents, 2 oncologists, and 1 urologist participated as observers. They first provided an estimate without CDSS-T and then with CDSS-T. A subset of cases was evaluated twice to study the intraobserver variability and its effects on observer consistency. The mean areas under the curves for assessment of pathologic T0 disease were 0.85 for CDSS-T alone, 0.76 for physicians without CDSS-T and improved to 0.80 for physicians with CDSS-T (P = .001) in the original evaluation, and 0.78 for physicians without CDSS-T and improved to 0.81 for physicians with CDSS-T (P = .010) in the repeated evaluation. The intraobserver variability was significantly reduced with CDSS-T (P < .0001). The CDSS-T can significantly reduce physicians' variability and improve their accuracy for identifying complete response of muscle-invasive bladder cancer to neoadjuvant chemotherapy.

Pathologic categorization of lung nodules: Radiomic descriptors of CT attenuation distribution patterns of solid and subsolid nodules in low-dose CT.

PURPOSE: Develop a quantitative image analysis method to characterize the heterogeneous patterns of nodule components for the classification of pathological categories of nodules. MATERIALS AND METHODS: With IRB approval and permission of the National Lung Screening Trial (NLST) project, 103 subjects with low dose CT (LDCT) were used in this study. We developed a radiomic quantitative CT attenuation distribution descriptor (qADD) to characterize the heterogeneous patterns of nodule components and a hybrid model (qADD+) that combined qADD with subject demographic data and radiologist-provided nodule descriptors to differentiate aggressive tumors from indolent tumors or benign nodules with pathological categorization as reference standard. The classification performances of qADD and qADD + were evaluated and compared to the Brock and the Mayo Clinic models by analysis of the area under the receiver operating characteristic curve (AUC). RESULTS: The radiomic features were consistently selected into qADDs to differentiate pathological invasive nodules from (1) preinvasive nodules, (2) benign nodules, and (3) the group of preinvasive and benign nodules, achieving test AUCs of 0.847 ±â€¯0.002, 0.842 ±â€¯0.002 and 0.810 ±â€¯0.001, respectively. The qADD + obtained test AUCs of 0.867 ±â€¯0.002, 0.888 ±â€¯0.001 and 0.852 ±â€¯0.001, respectively, which were higher than both the Brock and the Mayo Clinic models. CONCLUSION: The pathologic invasiveness of lung tumors could be categorized according to the CT attenuation distribution patterns of the nodule components manifested on LDCT images, and the majority of invasive lung cancers could be identified at baseline LDCT scans.

Computer-aided diagnosis in the era of deep learning.

Computer-aided diagnosis (CAD) has been a major field of research for the past few decades. CAD uses machine learning methods to analyze imaging and/or nonimaging patient data and makes assessment of the patient's condition, which can then be used to assist clinicians in their decision-making process. The recent success of the deep learning technology in machine learning spurs new research and development efforts to improve CAD performance and to develop CAD for many other complex clinical tasks. In this paper, we discuss the potential and challenges in developing CAD tools using deep learning technology or artificial intelligence (AI) in general, the pitfalls and lessons learned from CAD in screening mammography and considerations needed for future implementation of CAD or AI in clinical use. It is hoped that the past experiences and the deep learning technology will lead to successful advancement and lasting growth in this new era of CAD, thereby enabling CAD to deliver intelligent aids to improve health care.

Generalization error analysis for deep convolutional neural network with transfer learning in breast cancer diagnosis.

Deep convolutional neural network (DCNN), now popularly called artificial intelligence (AI), has shown the potential to improve over previous computer-assisted tools in medical imaging developed in the past decades. A DCNN has millions of free parameters that need to be trained, but the training sample set is limited in size for most medical imaging tasks so that transfer learning is typically used. Automatic data mining may be an efficient way to enlarge the collected data set but the data can be noisy such as incorrect labels or even a wrong type of image. In this work we studied the generalization error of DCNN with transfer learning in medical imaging for the task of classifying malignant and benign masses on mammograms. With a finite available data set, we simulated a training set containing corrupted data or noisy labels. The balance between learning and memorization of the DCNN was manipulated by varying the proportion of corrupted data in the training set. The generalization error of DCNN was analyzed by the area under the receiver operating characteristic curve for the training and test sets and the weight changes after transfer learning. The study demonstrates that the transfer learning strategy of DCNN for such tasks needs to be designed properly, taking into consideration the constraints of the available training set having limited size and quality for the classification task at hand, to minimize memorization and improve generalizability.

Deep Learning in Medical Image Analysis.

Deep learning is the state-of-the-art machine learning approach. The success of deep learning in many pattern recognition applications has brought excitement and high expectations that deep learning, or artificial intelligence (AI), can bring revolutionary changes in health care. Early studies of deep learning applied to lesion detection or classification have reported superior performance compared to those by conventional techniques or even better than radiologists in some tasks. The potential of applying deep-learning-based medical image analysis to computer-aided diagnosis (CAD), thus providing decision support to clinicians and improving the accuracy and efficiency of various diagnostic and treatment processes, has spurred new research and development efforts in CAD. Despite the optimism in this new era of machine learning, the development and implementation of CAD or AI tools in clinical practice face many challenges. In this chapter, we will discuss some of these issues and efforts needed to develop robust deep-learning-based CAD tools and integrate these tools into the clinical workflow, thereby advancing towards the goal of providing reliable intelligent aids for patient care.

CAD and AI for breast cancer-recent development and challenges.

Computer-aided diagnosis (CAD) has been a popular area of research and development in the past few decades. In CAD, machine learning methods and multidisciplinary knowledge and techniques are used to analyze the patient information and the results can be used to assist clinicians in their decision making process. CAD may analyze imaging information alone or in combination with other clinical data. It may provide the analyzed information directly to the clinician or correlate the analyzed results with the likelihood of certain diseases based on statistical modeling of the past cases in the population. CAD systems can be developed to provide decision support for many applications in the patient care processes, such as lesion detection, characterization, cancer staging, treatment planning and response assessment, recurrence and prognosis prediction. The new state-of-the-art machine learning technique, known as deep learning (DL), has revolutionized speech and text recognition as well as computer vision. The potential of major breakthrough by DL in medical image analysis and other CAD applications for patient care has brought about unprecedented excitement of applying CAD, or artificial intelligence (AI), to medicine in general and to radiology in particular. In this paper, we will provide an overview of the recent developments of CAD using DL in breast imaging and discuss some challenges and practical issues that may impact the advancement of artificial intelligence and its integration into clinical workflow.

Variabilities in Reference Standard by Radiologists and Performance Assessment in Detection of Pulmonary Embolism in CT Pulmonary Angiography.

Annotating lesion locations by radiologists' manual marking is a key step to provide reference standard for the training and testing of a computer-aided detection system by supervised machine learning. Inter-reader variability is not uncommon in readings even by expert radiologists. This study evaluated the variability of the radiologist-identified pulmonary emboli (PEs) to demonstrate the importance of improving the reliability of the reference standard by a multi-step process for performance evaluation. In an initial reading of 40 CTPA PE cases, two experienced thoracic radiologists independently marked the PE locations. For markings from the two radiologists that did not agree, each radiologist re-read the cases independently to assess the discordant markings. Finally, for markings that still disagreed after the second reading, the two radiologists read together to reach a consensus. The variability of radiologists was evaluated by analyzing the agreement between two radiologists. For the 40 cases, 475 and 514 PEs were identified by radiologists R1 and R2 in the initial independent readings, respectively. For a total of 545 marks by the two radiologists, 81.5% (444/545) of the marks agreed but 101 marks in 36 cases differed. After consensus, 65 (64.4%) and 36 (35.6%) of the 101 marks were determined to be true PEs and false positives (FPs), respectively. Of these, 48 and 17 were false negatives (FNs) and 14 and 22 were FPs by R1 and R2, respectively. Our study demonstrated that there is substantial variability in reference standards provided by radiologists, which impacts the performance assessment of a lesion detection system. Combination of multiple radiologists' readings and consensus is needed to improve the reliability of a reference standard.

Deep Learning Approach for Assessment of Bladder Cancer Treatment Response.

We compared the performance of different Deep learning-convolutional neural network (DL-CNN) models for bladder cancer treatment response assessment based on transfer learning by freezing different DL-CNN layers and varying the DL-CNN structure. Pre- and posttreatment computed tomography scans of 123 patients (cancers, 129; pre- and posttreatment cancer pairs, 158) undergoing chemotherapy were collected. After chemotherapy 33% of patients had T0 stage cancer (complete response). Regions of interest in pre- and posttreatment scans were extracted from the segmented lesions and combined into hybrid pre -post image pairs (h-ROIs). Training (pairs, 94; h-ROIs, 6209), validation (10 pairs) and test sets (54 pairs) were obtained. The DL-CNN consisted of 2 convolution (C1-C2), 2 locally connected (L3-L4), and 1 fully connected layers. The DL-CNN was trained with h-ROIs to classify cancers as fully responding (stage T0) or not fully responding to chemotherapy. Two radiologists provided lesion likelihood of being stage T0 posttreatment. The test area under the ROC curve (AUC) was 0.73 for T0 prediction by the base DL-CNN structure with randomly initialized weights. The base DL-CNN structure with pretrained weights and transfer learning (no frozen layers) achieved test AUC of 0.79. The test AUCs for 3 modified DL-CNN structures (different C1-C2 max pooling filter sizes, strides, and padding, with transfer learning) were 0.72, 0.86, and 0.69. For the base DL-CNN with (C1) frozen, (C1-C2) frozen, and (C1-C2-L3) frozen, the test AUCs were 0.81, 0.78, and 0.71, respectively. The radiologists' AUCs were 0.76 and 0.77. DL-CNN performed better with pretrained than randomly initialized weights.

U-Net based deep learning bladder segmentation in CT urography.

OBJECTIVES: To develop a U-Net-based deep learning approach (U-DL) for bladder segmentation in computed tomography urography (CTU) as a part of a computer-assisted bladder cancer detection and treatment response assessment pipeline. MATERIALS AND METHODS: A dataset of 173 cases including 81 cases in the training/validation set (42 masses, 21 with wall thickening, 18 normal bladders), and 92 cases in the test set (43 masses, 36 with wall thickening, 13 normal bladders) were used with Institutional Review Board approval. An experienced radiologist provided three-dimensional (3D) hand outlines for all cases as the reference standard. We previously developed a bladder segmentation method that used a deep learning convolution neural network and level sets (DCNN-LS) within a user-input bounding box. However, some cases with poor image quality or with advanced bladder cancer spreading into the neighboring organs caused inaccurate segmentation. We have newly developed an automated U-DL method to estimate a likelihood map of the bladder in CTU. The U-DL did not require a user-input box and the level sets for postprocessing. To identify the best model for this task, we compared the following models: (a) two-dimensional (2D) U-DL and 3D U-DL using 2D CT slices and 3D CT volumes, respectively, as input, (b) U-DLs using CT images of different resolutions as input, and (c) U-DLs with and without automated cropping of the bladder as an image preprocessing step. The segmentation accuracy relative to the reference standard was quantified by six measures: average volume intersection ratio (AVI), average percent volume error (AVE), average absolute volume error (AAVE), average minimum distance (AMD), average Hausdorff distance (AHD), and the average Jaccard index (AJI). As a baseline, the results from our previous DCNN-LS method were used. RESULTS: In the test set, the best 2D U-DL model achieved AVI, AVE, AAVE, AMD, AHD, and AJI values of 93.4 ± 9.5%, -4.2 ± 14.2%, 9.2 ± 11.5%, 2.7 ± 2.5 mm, 9.7 ± 7.6 mm, 85.0 ± 11.3%, respectively, while the corresponding measures by the best 3D U-DL were 90.6 ± 11.9%, -2.3 ± 21.7%, 11.5 ± 18.5%, 3.1 ± 3.2 mm, 11.4 ± 10.0 mm, and 82.6 ± 14.2%, respectively. For comparison, the corresponding values obtained with the baseline method were 81.9 ± 12.1%, 10.2 ± 16.2%, 14.0 ± 13.0%, 3.6 ± 2.0 mm, 12.8 ± 6.1 mm, and 76.2 ± 11.8%, respectively, for the same test set. The improvement for all measures between the best U-DL and the DCNN-LS were statistically significant (P < 0.001). CONCLUSION: Compared to a previous DCNN-LS method, which depended on a user-input bounding box, the U-DL provided more accurate bladder segmentation and was more automated than the previous approach.

Automated pectoral muscle identification on MLO-view mammograms: Comparison of deep neural network to conventional computer vision.

OBJECTIVES: The aim of this study was to develop a fully automated deep learning approach for identification of the pectoral muscle on mediolateral oblique (MLO) view mammograms and evaluate its performance in comparison to our previously developed texture-field orientation (TFO) method using conventional image feature analysis. Pectoral muscle segmentation is an important step for automated image analyses such as breast density or parenchymal pattern classification, lesion detection, and multiview correlation. MATERIALS AND METHODS: Institutional Review Board (IRB) approval was obtained before data collection. A dataset of 729 MLO-view mammograms including 637 digitized film mammograms (DFM) and 92 digital mammograms (DM) from our previous study was used for the training and validation of our deep convolutional neural network (DCNN) segmentation method. In addition, we collected an independent set of 203 DMs from 131 patients for testing. The film mammograms were digitized at a pixel size of 50 µm × 50 µm with a Lumiscan digitizer. All DMs were acquired with GE systems at a pixel size of 100 µm × 100 µm. An experienced MQSA radiologist manually drew the pectoral muscle boundary on each mammogram as the reference standard. We trained the DCNN to estimate a probability map of the pectoral muscle region on mammograms. The DCNN consisted of a contracting path to capture multiresolution image context and a symmetric expanding path for prediction of the pectoral muscle region. Three DCNN structures were compared for automated identification of pectoral muscles. Tenfold cross-validation was used in training of the DCNNs. After training, we applied the ten trained models during cross-validation to the independent DM test set. The predicted pectoral muscle region of each test DM was obtained as the mean probability map by averaging the ensemble of probability maps from the ten models. The DCNN-segmented pectoral muscle was evaluated by three performance measures relative to the reference standard: (a) the percent overlap area (POA) of the pectoral muscle regions, (b) the Hausdorff distance (Hdist), and (c) the average Euclidean distance (AvgDist) between the boundaries. The results were compared to those obtained with the TFO method, used as our baseline. A two-tailed paired t test was performed to examine the significance in the differences between the DCNN and the baseline. RESULTS: In the ten test partitions of the cross-validation set, the DCNN achieved a mean POA of 96.5 ± 2.9%, a mean Hdist of 2.26 ± 1.31 mm, and a mean AvgDist of 0.78 ± 0.58 mm, while the corresponding measures by the baseline method were 94.2 ± 4.8%, 3.69 ± 2.48 mm, and 1.30 ± 1.22 mm, respectively. For the independent DM test set, the DCNN achieved a mean POA of 93.7% ± 6.9%, a mean Hdist of 3.80 ± 3.21 mm, and a mean AvgDist of 1.49 ± 1.62 mm comparing to 86.9% ± 16.0%, 7.18 ± 14.22 mm, and 3.98 ± 14.13 mm, respectively, by the baseline method. CONCLUSION: In comparison to the TFO method, DCNN significantly improved the accuracy of pectoral muscle identification on mammograms (P < 0.05).