Effects of a comprehensive brain computed tomography deep learning model on radiologist detection accuracy.

OBJECTIVES: Non-contrast computed tomography of the brain (NCCTB) is commonly used to detect intracranial pathology but is subject to interpretation errors. Machine learning can augment clinical decision-making and improve NCCTB scan interpretation. This retrospective detection accuracy study assessed the performance of radiologists assisted by a deep learning model and compared the standalone performance of the model with that of unassisted radiologists. METHODS: A deep learning model was trained on 212,484 NCCTB scans drawn from a private radiology group in Australia. Scans from inpatient, outpatient, and emergency settings were included. Scan inclusion criteria were age ≥ 18 years and series slice thickness ≤ 1.5 mm. Thirty-two radiologists reviewed 2848 scans with and without the assistance of the deep learning system and rated their confidence in the presence of each finding using a 7-point scale. Differences in AUC and Matthews correlation coefficient (MCC) were calculated using a ground-truth gold standard. RESULTS: The model demonstrated an average area under the receiver operating characteristic curve (AUC) of 0.93 across 144 NCCTB findings and significantly improved radiologist interpretation performance. Assisted and unassisted radiologists demonstrated an average AUC of 0.79 and 0.73 across 22 grouped parent findings and 0.72 and 0.68 across 189 child findings, respectively. When assisted by the model, radiologist AUC was significantly improved for 91 findings (158 findings were non-inferior), and reading time was significantly reduced. CONCLUSIONS: The assistance of a comprehensive deep learning model significantly improved radiologist detection accuracy across a wide range of clinical findings and demonstrated the potential to improve NCCTB interpretation. CLINICAL RELEVANCE STATEMENT: This study evaluated a comprehensive CT brain deep learning model, which performed strongly, improved the performance of radiologists, and reduced interpretation time. The model may reduce errors, improve efficiency, facilitate triage, and better enable the delivery of timely patient care. KEY POINTS: • This study demonstrated that the use of a comprehensive deep learning system assisted radiologists in the detection of a wide range of abnormalities on non-contrast brain computed tomography scans. • The deep learning model demonstrated an average area under the receiver operating characteristic curve of 0.93 across 144 findings and significantly improved radiologist interpretation performance. • The assistance of the comprehensive deep learning model significantly reduced the time required for radiologists to interpret computed tomography scans of the brain.

Translating preclinical MRI methods to clinical oncology.

The complexity of modern in vivo magnetic resonance imaging (MRI) methods in oncology has dramatically changed in the last 10 years. The field has long since moved passed its (unparalleled) ability to form images with exquisite soft-tissue contrast and morphology, allowing for the enhanced identification of primary tumors and metastatic disease. Currently, it is not uncommon to acquire images related to blood flow, cellularity, and macromolecular content in the clinical setting. The acquisition of images related to metabolism, hypoxia, pH, and tissue stiffness are also becoming common. All of these techniques have had some component of their invention, development, refinement, validation, and initial applications in the preclinical setting using in vivo animal models of cancer. In this review, we discuss the genesis of quantitative MRI methods that have been successfully translated from preclinical research and developed into clinical applications. These include methods that interrogate perfusion, diffusion, pH, hypoxia, macromolecular content, and tissue mechanical properties for improving detection, staging, and response monitoring of cancer. For each of these techniques, we summarize the 1) underlying biological mechanism(s); 2) preclinical applications; 3) available repeatability and reproducibility data; 4) clinical applications; and 5) limitations of the technique. We conclude with a discussion of lessons learned from translating MRI methods from the preclinical to clinical setting, and a presentation of four fundamental problems in cancer imaging that, if solved, would result in a profound improvement in the lives of oncology patients. Level of Evidence: 5 Technical Efficacy: Stage 3 J. Magn. Reson. Imaging 2019;50:1377-1392.

Creating Value through Incremental Innovation: Managing Culture, Structure, and Process.

While the looming threat of large-scale disruptive innovation consumes disproportionate attention, incremental innovation remains an important tool for preserving and growing radiology practices within a dynamic marketplace. Incremental innovation, defined as the process of making improvements or additions to an organization while maintaining the organization's core product or service model, is accessible to practices of all sizes and must not be overlooked if practices are to maintain their competitive advantage. This article explores cultural, structural, and process enablers for incremental innovation. Successful innovation cultures foster the ability to import and exploit external knowledge (adaptive capacity), encourage creative thought from all levels of the organization, display sensitivity toward the competency-destroying potential of certain changes, cultivate a positive perceptual bias toward organizational threats, and build tolerance for risk and uncertainty when prototyping new ideas. Structural elements promoting incremental innovation include dedicated resources for innovation planning, flexible and organic team structures, strong centralized governance models, robust communication systems, and organizational incentives encouraging exploration of new concepts. Processes important to innovation include periodic environmental scanning, strategic and scenario planning, use of an objectively gated system for testing and filtering new ideas, and use of an approach to implementation that emphasizes empowerment of project managers, removal of barriers, and proactive communication around change.

Repeatability, reproducibility, and accuracy of quantitative mri of the breast in the community radiology setting.

BACKGROUND: Quantitative diffusion-weighted MRI (DW-MRI) and dynamic contrast-enhanced MRI (DCE-MRI) have the potential to impact patient care by providing noninvasive biological information in breast cancer. PURPOSE/HYPOTHESIS: To quantify the repeatability, reproducibility, and accuracy of apparent diffusion coefficient (ADC) and T1 -mapping of the breast in community radiology practices. STUDY TYPE: Prospective. SUBJECTS/PHANTOM: Ice-water DW-MRI and T1 gel phantoms were used to assess accuracy. Normal subjects (n = 3) and phantoms across three sites (one academic, two community) were used to assess reproducibility. Test-retest analysis at one site in normal subjects (n = 12) was used to assess repeatability. FIELD STRENGTH/SEQUENCE: 3T Siemens Skyra MRI quantitative DW-MRI and T1 -mapping. ASSESSMENT: Quantitative DW-MRI and T1 -mapping parametric maps of phantoms and fibroglandular and adipose tissue of the breast. STATISTICAL TESTS: Average values of breast tissue were quantified and Bland-Altman analysis was performed to assess the repeatability of the MRI techniques, while the Friedman test assessed reproducibility. RESULTS: ADC measurements were reproducible across sites, with an average difference of 1.6% in an ice-water phantom and 7.0% in breast fibroglandular tissue. T1 measurements in gel phantoms had an average difference of 2.8% across three sites, whereas breast fibroglandular and adipose tissue had 8.4% and 7.5% average differences, respectively. In the repeatability study, we found no bias between first and second scanning sessions (P = 0.1). The difference between repeated measurements was independent of the mean for each MRI metric (P = 0.156, P = 0.862, P = 0.197 for ADC, T1 of fibroglandular tissue, and T1 of adipose tissue, respectively). DATA CONCLUSION: Community radiology practices can perform repeatable, reproducible, and accurate quantitative T1 -mapping and DW-MRI. This has the potential to dramatically expand the number of sites that can participate in multisite clinical trials and increase clinical translation of quantitative MRI techniques for cancer response assessment. LEVEL OF EVIDENCE: 1 Technical Efficacy: Stage 2 J. Magn. Reson. Imaging 2018.

The Attenuation Distribution Across the Long Axis of Breast Cancer Liver Metastases at CT: A Quantitative Biomarker for Predicting Overall Survival.

OBJECTIVE: The objective of our study was to compare attenuation distribution across the long-axis (ADLA) measurements, Response Evaluation Criteria in Solid Tumors (RECIST) version 1.1, and Choi criteria for predicting overall survival (OS) in patients with metastatic breast cancer treated with bevacizumab. MATERIALS AND METHODS: We obtained HIPAA-compliant data from a prospective, multisite, phase 3 trial of bevacizumab for the treatment of metastatic breast cancer. For patients with one or more liver metastases measuring 15 mm or larger at baseline, we evaluated up to two target liver lesions using RECIST, Choi criteria, and ADLA measurements, with the latter defined as the SD of the CT attenuation values of each pixel along the tumor long-axis diameter. The optimal percentage change threshold for defining an ADLA response was computed by cross-validation analysis in a Cox model. The log-rank test was applied to evaluate RECIST, Choi criteria, and ADLA for discriminating patients with superior OS. The predictive accuracies of all three techniques were compared using Brier scores and areas under the ROC curve (AUC). All analyses were performed separately using best overall response (BOR) and response at the first follow-up time point (FU1). RESULTS: One hundred sixty-four patients met the inclusion criteria. A 25% decrease in the ADLA measurement from baseline was the optimal ADLA response threshold for BOR and FU1. RECIST, Choi criteria, and ADLA successfully identified patients with superior OS when using BOR (RECIST, p = 0.02; Choi and ADLA, p < 0.001), but only Choi criteria and ADLA measurements were successful when using FU1 (RECIST, p = 0.43; Choi and ADLA, p < 0.001). In a direct comparison, ADLA measurements outperformed both RECIST and Choi criteria using BOR (95% CI for Brier score differences, ADLA-RECIST [-0.58 to -0.08] and ADLA-Choi [-0.55 to -0.06]; 95% CI for AUC differences, ADLA-RECIST [0.16-0.33] and ADLA-Choi [0.17-0.36]) as well as using FU1 (95% CI for Brier score differences, ADLA-RECIST [-0.77 to -0.08] and ADLA-Choi [-0.58 to -0.03]; 95% CI for AUC differences, ADLA-RECIST [0.22-0.39] and ADLA-Choi [0.01-0.22]). CONCLUSION: ADLA measurements may be a useful noninvasive indicator of cancer treatment response. Because ADLA measurements may be extracted relatively easily using existing radiologist workflows, further investigation of the ADLA technique is warranted.

Towards real-time topical detection and characterization of FDG dose infiltration prior to PET imaging.

PURPOSE: To dynamically detect and characterize 18F-fluorodeoxyglucose (FDG) dose infiltrations and evaluate their effects on positron emission tomography (PET) standardized uptake values (SUV) at the injection site and in control tissue. METHODS: Investigational gamma scintillation sensors were topically applied to patients with locally advanced breast cancer scheduled to undergo limited whole-body FDG-PET as part of an ongoing clinical study. Relative to the affected breast, sensors were placed on the contralateral injection arm and ipsilateral control arm during the resting uptake phase prior to each patient's PET scan. Time-activity curves (TACs) from the sensors were integrated at varying intervals (0-10, 0-20, 0-30, 0-40, and 30-40 min) post-FDG and the resulting areas under the curve (AUCs) were compared to SUVs obtained from PET. RESULTS: In cases of infiltration, observed in three sensor recordings (30 %), the injection arm TAC shape varied depending on the extent and severity of infiltration. In two of these cases, TAC characteristics suggested the infiltration was partially resolving prior to image acquisition, although it was still apparent on subsequent PET. Areas under the TAC 0-10 and 0-20 min post-FDG were significantly different in infiltrated versus non-infiltrated cases (Mann-Whitney, p < 0.05). When normalized to control, all TAC integration intervals from the injection arm were significantly correlated with SUVpeak and SUVmax measured over the infiltration site (Spearman ρ ≥ 0.77, p < 0.05). Receiver operating characteristic (ROC) analyses, testing the ability of the first 10 min of post-FDG sensor data to predict infiltration visibility on the ensuing PET, yielded an area under the ROC curve of 0.92. CONCLUSIONS: Topical sensors applied near the injection site provide dynamic information from the time of FDG administration through the uptake period and may be useful in detecting infiltrations regardless of PET image field of view. This dynamic information may also complement the static PET image to better characterize the true extent of infiltrations.

Clinical Activity of Ipilimumab in Acral Melanoma: A Retrospective Review.

BACKGROUND: Ipilimumab improves overall survival (OS) in advanced melanoma. Acral melanoma is an uncommon clinical subtype of this disease associated with poor prognosis. The clinical activity of ipilimumab has not been well-defined in advanced acral melanoma. METHODS: We retrospectively reviewed the demographics, treatment history, and clinical outcomes for all patients with acral melanoma treated with ipilimumab from two academic centers between February 2006 and June 2013. Using Cox proportional hazards models, we assessed for factors that correlated with OS. RESULTS: A total of 35 patients with acral melanoma received ipilimumab. Melanomas arose on volar surfaces (n = 28) and subungual sites (n = 7); stage M1c disease was present in 54%, and 45% had elevated serum lactate dehydrogenase (LDH). Best response by RECIST 1.1 criteria was complete response in 1 patient, partial response in 3, and stable disease (SD) in 4 for an objective response rate (ORR) of 11.4% and a clinical benefit rate (ORR + SD) at 24 weeks of 22.9%. Median progression-free survival was 2.5 months (95% confidence interval [CI]: 2.3-2.7 months); median OS was 16.7 months (95% CI: 10.9-22.5 months). Normal LDH and absolute lymphocyte count ≥1,000 at 7 weeks predicted longer OS. Immune-related adverse events (irAEs) were noted in 16 patients including 7 with grade 3/4 irAEs (20%). CONCLUSION: Ipilimumab is clinically active in acral melanoma with similar ORR and OS compared with unselected melanoma populations. Ipilimumab remains a viable therapeutic option for patients with advanced acral melanoma. IMPLICATIONS FOR PRACTICE: Ipilimumab is a commonly used immune therapy that improves survival in metastatic melanoma. The clinical activity of ipilimumab in certain rare melanoma subtypes, such as uveal or mucosal melanomas, is suboptimal. Acral melanoma is another unusual subtype of this disease that arises on the palms, soles, and nailbeds. In this study of 35 patients with acral melanoma from 2 centers, ipilimumab was found to have activity that appears equivalent to unselected melanoma (response rate of 11.4%, median overall survival of 16.7 months). Ipilimumab remains a viable treatment option for this melanoma subpopulation.

DCE-MRI analysis methods for predicting the response of breast cancer to neoadjuvant chemotherapy: pilot study findings.

PURPOSE: The purpose of this pilot study is to determine (1) if early changes in both semiquantitative and quantitative DCE-MRI parameters, observed after the first cycle of neoadjuvant chemotherapy in breast cancer patients, show significant difference between responders and nonresponders and (2) if these parameters can be used as a prognostic indicator of the eventual response. METHODS: Twenty-eight patients were examined using DCE-MRI pre-, post-one cycle, and just prior to surgery. The semiquantitative parameters included longest dimension, tumor volume, initial area under the curve, and signal enhancement ratio related parameters, while quantitative parameters included K(trans), v(e), k(ep), v(p), and τ(i) estimated using the standard Tofts-Kety, extended Tofts-Kety, and fast exchange regime models. RESULTS: Our preliminary results indicated that the signal enhancement ratio washout volume and k(ep) were significantly different between pathologic complete responders from nonresponders (P < 0.05) after a single cycle of chemotherapy. Receiver operator characteristic analysis showed that the AUC of the signal enhancement ratio washout volume was 0.75, and the AUCs of k(ep) estimated by three models were 0.78, 0.76, and 0.73, respectively. CONCLUSION: In summary, the signal enhancement ratio washout volume and k(ep) appear to predict breast cancer response after one cycle of neoadjuvant chemotherapy. This observation should be confirmed with additional prospective studies.

Atezolizumab in Combination With Carboplatin and Survival Outcomes in Patients With Metastatic Triple-Negative Breast Cancer: The TBCRC 043 Phase 2 Randomized Clinical Trial.

Importance: Agents targeting programmed death ligand 1 (PD-L1) have demonstrated efficacy in triple-negative breast cancer (TNBC) when combined with chemotherapy and are now the standard of care in patients with PD-L1-positive metastatic disease. In contrast to microtubule-targeting agents, the effect of combining platinum compounds with programmed cell death 1 (PD-1)/PD-L1 immunotherapy has not been extensively determined. Objective: To evaluate the efficacy of atezolizumab with carboplatin in patients with metastatic TNBC. Design, Setting, and Participants: This phase 2 randomized clinical trial was conducted in 6 centers from August 2017 to June 2021. Interventions: Patients with metastatic TNBC were randomized to receive carboplatin area under the curve (AUC) 6 alone or with atezolizumab, 1200 mg, every 3 weeks until disease progression or unacceptable toxic effects with a 3-year duration of follow-up. Main Outcome and Measures: The primary end point was investigator-assessed progression-free survival (PFS). Secondary end points included overall response rate (ORR), clinical benefit rate (CBR), and overall survival (OS). Other objectives included correlation of response with tumor PD-L1 levels, tumor-infiltrating lymphocytes (TILs), tumor DNA- and RNA-sequenced biomarkers, TNBC subtyping, and multiplex analyses of immune markers. Results: All 106 patients with metastatic TNBC who were enrolled were female with a mean (range) age of 55 (27-79) years, of which 12 (19%) identified as African American/Black, 1 (1%) as Asian, 73 (69%) as White, and 11 (10%) as unknown. Patients were randomized and received either carboplatin (n = 50) or carboplatin and atezolizumab (n = 56). The combination improved PFS (hazard ratio [HR], 0.66; 95% CI, 0.44-1.01; P = .05) from a median of 2.2 to 4.1 months, increased ORR from 8.0% (95% CI, 3.2%-18.8%) to 30.4% (95% CI, 19.9%-43.3%), increased CBR at 6 months from 18.0% (95% CI, 9.8%-30.1%) to 37.5% (95% CI, 26.0%-50.6%), and improved OS (HR, 0.60; 95% CI, 0.37-0.96; P = .03) from a median of 8.6 to 12.6 months. Subgroup analysis showed PD-L1-positive tumors did not benefit more from adding atezolizumab (HR, 0.62; 95% CI, 0.23-1.65; P = .35). Patients with high TILs (HR, 0.12; 95% CI, 0.30-0.50), high mutation burden (HR, 0.50; 95% CI, 0.23-1.06), and prior chemotherapy (HR, 0.59; 95% CI, 0.36-0.95) received greater benefit on the combination. Patients with obesity and patients with more than 125 mg/dL on-treatment blood glucose levels were associated with better PFS (HR, 0.35; 95% CI, 0.10-1.80) on the combination. TNBC subtypes benefited from adding atezolizumab, except the luminal androgen receptor subtype. Conclusions and Relevance: In this randomized clinical trial, the addition of atezolizumab to carboplatin significantly improved survival of patients with metastatic TNBC regardless of PD-L1 status. Further, lower risk of disease progression was associated with increased TILs, higher mutation burden, obesity, and uncontrolled blood glucose levels. Trial Registration: ClinicalTrials.gov Identifier: NCT03206203.

Complications of targeted drug therapies for solid malignancies: manifestations and mechanisms.

OBJECTIVE: This article reviews important complications of targeted drug therapies for solid malignancies that can be identified on diagnostic imaging. Wherever possible, known or proposed mechanistic explanations for drug complications are emphasized. CONCLUSION: Familiarity with the toxicity profiles of different targeted cancer therapies is important for identifying drug-related complications and for differentiating drug effects from disease progression. A mechanistic understanding may be useful for associating individual drugs with their complications and for predicting the complications of emerging agents.

UNesT: Local spatial representation learning with hierarchical transformer for efficient medical segmentation.

Transformer-based models, capable of learning better global dependencies, have recently demonstrated exceptional representation learning capabilities in computer vision and medical image analysis. Transformer reformats the image into separate patches and realizes global communication via the self-attention mechanism. However, positional information between patches is hard to preserve in such 1D sequences, and loss of it can lead to sub-optimal performance when dealing with large amounts of heterogeneous tissues of various sizes in 3D medical image segmentation. Additionally, current methods are not robust and efficient for heavy-duty medical segmentation tasks such as predicting a large number of tissue classes or modeling globally inter-connected tissue structures. To address such challenges and inspired by the nested hierarchical structures in vision transformer, we proposed a novel 3D medical image segmentation method (UNesT), employing a simplified and faster-converging transformer encoder design that achieves local communication among spatially adjacent patch sequences by aggregating them hierarchically. We extensively validate our method on multiple challenging datasets, consisting of multiple modalities, anatomies, and a wide range of tissue classes, including 133 structures in the brain, 14 organs in the abdomen, 4 hierarchical components in the kidneys, inter-connected kidney tumors and brain tumors. We show that UNesT consistently achieves state-of-the-art performance and evaluate its generalizability and data efficiency. Particularly, the model achieves whole brain segmentation task complete ROI with 133 tissue classes in a single network, outperforming prior state-of-the-art method SLANT27 ensembled with 27 networks. Our model performance increases the mean DSC score of the publicly available Colin and CANDI dataset from 0.7264 to 0.7444 and from 0.6968 to 0.7025, respectively. Code, pre-trained models, and use case pipeline are available at: https://github.com/MASILab/UNesT.

RAP-NET: COARSE-TO-FINE MULTI-ORGAN SEGMENTATION WITH SINGLE RANDOM ANATOMICAL PRIOR.

Performing coarse-to-fine abdominal multi-organ segmentation facilitates extraction of high-resolution segmentation minimizing the loss of spatial contextual information. However, current coarse-to-refine approaches require a significant number of models to perform single organ segmentation. We propose a coarse-to-fine pipeline RAP-Net, which starts from the extraction of the global prior context of multiple organs from 3D volumes using a low-resolution coarse network, followed by a fine phase that uses a single refined model to segment all abdominal organs instead of multiple organ corresponding models. We combine the anatomical prior with corresponding extracted patches to preserve the anatomical locations and boundary information for performing high-resolution segmentation across all organs in a single model. To train and evaluate our method, a clinical research cohort consisting of 100 patient volumes with 13 organs well-annotated is used. We tested our algorithms with 4-fold cross-validation and computed the Dice score for evaluating the segmentation performance of the 13 organs. Our proposed method using single auto-context outperforms the state-of-the-art on 13 models with an average Dice score 84.58% versus 81.69% (p<0.0001).

Validation and estimation of spleen volume via computer-assisted segmentation on clinically acquired CT scans.

Purpose: Deep learning is a promising technique for spleen segmentation. Our study aims to validate the reproducibility of deep learning-based spleen volume estimation by performing spleen segmentation on clinically acquired computed tomography (CT) scans from patients with myeloproliferative neoplasms. Approach: As approved by the institutional review board, we obtained 138 de-identified abdominal CT scans. A sum of voxel volume on an expert annotator's segmentations establishes the ground truth (estimation 1). We used our deep convolutional neural network (estimation 2) alongside traditional linear estimations (estimation 3 and 4) to estimate spleen volumes independently. Dice coefficient, Hausdorff distance, R 2 coefficient, Pearson R coefficient, the absolute difference in volume, and the relative difference in volume were calculated for 2 to 4 against the ground truth to compare and assess methods' performances. We re-labeled on scan-rescan on a subset of 40 studies to evaluate method reproducibility. Results: Calculated against the ground truth, the R 2 coefficients for our method (estimation 2) and linear method (estimation 3 and 4) are 0.998, 0.954, and 0.973, respectively. The Pearson R coefficients for the estimations against the ground truth are 0.999, 0.963, and 0.978, respectively (paired t -tests produced p < 0.05 between 2 and 3, and 2 and 4). Conclusion: The deep convolutional neural network algorithm shows excellent potential in rendering more precise spleen volume estimations. Our computer-aided segmentation exhibits reasonable improvements in splenic volume estimation accuracy.

High-resolution 3D abdominal segmentation with random patch network fusion.

Deep learning for three dimensional (3D) abdominal organ segmentation on high-resolution computed tomography (CT) is a challenging topic, in part due to the limited memory provide by graphics processing units (GPU) and large number of parameters and in 3D fully convolutional networks (FCN). Two prevalent strategies, lower resolution with wider field of view and higher resolution with limited field of view, have been explored but have been presented with varying degrees of success. In this paper, we propose a novel patch-based network with random spatial initialization and statistical fusion on overlapping regions of interest (ROIs). We evaluate the proposed approach using three datasets consisting of 260 subjects with varying numbers of manual labels. Compared with the canonical "coarse-to-fine" baseline methods, the proposed method increases the performance on multi-organ segmentation from 0.799 to 0.856 in terms of mean DSC score (p-value < 0.01 with paired t-test). The effect of different numbers of patches is evaluated by increasing the depth of coverage (expected number of patches evaluated per voxel). In addition, our method outperforms other state-of-the-art methods in abdominal organ segmentation. In conclusion, the approach provides a memory-conservative framework to enable 3D segmentation on high-resolution CT. The approach is compatible with many base network structures, without substantially increasing the complexity during inference. Given a CT scan with at high resolution, a low-res section (left panel) is trained with multi-channel segmentation. The low-res part contains down-sampling and normalization in order to preserve the complete spatial information. Interpolation and random patch sampling (mid panel) is employed to collect patches. The high-dimensional probability maps are acquired (right panel) from integration of all patches on field of views.

Learning from dispersed manual annotations with an optimized data weighting policy.

Purpose: Deep learning methods have become essential tools for quantitative interpretation of medical imaging data, but training these approaches is highly sensitive to biases and class imbalance in the available data. There is an opportunity to increase the available training data by combining across different data sources (e.g., distinct public projects); however, data collected under different scopes tend to have differences in class balance, label availability, and subject demographics. Recent work has shown that importance sampling can be used to guide training selection. To date, these approaches have not considered imbalanced data sources with distinct labeling protocols. Approach: We propose a sampling policy, known as adaptive stochastic policy (ASP), inspired by reinforcement learning to adapt training based on subject, data source, and dynamic use criteria. We apply ASP in the context of multiorgan abdominal computed tomography segmentation. Training was performed with cross validation on 840 subjects from 10 data sources. External validation was performed with 20 subjects from 1 data source. Results: Four alternative strategies were evaluated with the state-of-the-art baseline as upper confident bound (UCB). ASP achieves average Dice of 0.8261 compared to 0.8135 UCB ( p < 0.01 , paired t -test) across fivefold cross validation. On withheld testing datasets, the proposed ASP achieved 0.8265 mean Dice versus 0.8077 UCB ( p < 0.01 , paired t -test). Conclusions: ASP provides a flexible reweighting technique for training deep learning models. We conclude that the proposed method adapts the sample importance, which leverages the performance on a challenging multisite, multiorgan, and multisize segmentation task.

Validation and Optimization of Multi-Organ Segmentation on Clinical Imaging Archives.

Segmentation of abdominal computed tomography (CT) provides spatial context, morphological properties, and a framework for tissue-specific radiomics to guide quantitative Radiological assessment. A 2015 MICCAI challenge spurred substantial innovation in multi-organ abdominal CT segmentation with both traditional and deep learning methods. Recent innovations in deep methods have driven performance toward levels for which clinical translation is appealing. However, continued cross-validation on open datasets presents the risk of indirect knowledge contamination and could result in circular reasoning. Moreover, "real world" segmentations can be challenging due to the wide variability of abdomen physiology within patients. Herein, we perform two data retrievals to capture clinically acquired deidentified abdominal CT cohorts with respect to a recently published variation on 3D U-Net (baseline algorithm). First, we retrieved 2004 deidentified studies on 476 patients with diagnosis codes involving spleen abnormalities (cohort A). Second, we retrieved 4313 deidentified studies on 1754 patients without diagnosis codes involving spleen abnormalities (cohort B). We perform prospective evaluation of the existing algorithm on both cohorts, yielding 13% and 8% failure rate, respectively. Then, we identified 51 subjects in cohort A with segmentation failures and manually corrected the liver and gallbladder labels. We re-trained the model adding the manual labels, resulting in performance improvement of 9% and 6% failure rate for the A and B cohorts, respectively. In summary, the performance of the baseline on the prospective cohorts was similar to that on previously published datasets. Moreover, adding data from the first cohort substantively improved performance when evaluated on the second withheld validation cohort.

Semi-Supervised Multi-Organ Segmentation through Quality Assurance Supervision.

Human in-the-loop quality assurance (QA) is typically performed after medical image segmentation to ensure that the systems are performing as intended, as well as identifying and excluding outliers. By performing QA on large-scale, previously unlabeled testing data, categorical QA scores (e.g. "successful" versus "unsuccessful") can be generated. Unfortunately, the precious use of resources for human in-the-loop QA scores are not typically reused in medical image machine learning, especially to train a deep neural network for image segmentation. Herein, we perform a pilot study to investigate if the QA labels can be used as supplementary supervision to augment the training process in a semi-supervised fashion. In this paper, we propose a semi-supervised multi-organ segmentation deep neural network consisting of a traditional segmentation model generator and a QA involved discriminator. An existing 3-D abdominal segmentation network is employed, while the pre-trained ResNet-18 network is used as discriminator. A large-scale dataset of 2027 volumes are used to train the generator, whose 2-D montage images and segmentation mask with QA scores are used to train the discriminator. To generate the QA scores, the 2-D montage images were reviewed manually and coded 0 (success), 1 (errors consistent with published performance), and 2 (gross failure). Then, the ResNet-18 network was trained with 1623 montage images in equal distribution of all three code labels and achieved an accuracy 94% for classification predictions with 404 montage images withheld for the test cohort. To assess the performance of using the QA supervision, the discriminator was used as a loss function in a multi-organ segmentation pipeline. The inclusion of QA-loss function boosted performance on the unlabeled test dataset from 714 patients to 951 patients over the baseline model. Additionally, the number of failures decreased from 606 (29.90%) to 402 (19.83%). The contributions of the proposed method are three-fold: We show that (1) the QA scores can be used as a loss function to perform semi-supervised learning for unlabeled data, (2) the well trained discriminator is learnt by QA score rather than traditional "true/false", and (3) the performance of multi-organ segmentation on unlabeled datasets can be fine-tuned with more robust and higher accuracy than the original baseline method. The use of QA-inspired loss functions represents a promising area of future research and may permit tighter integration of supervised and semi-supervised learning.

Contrast Phase Classification with a Generative Adversarial Network.

Dynamic contrast enhanced computed tomography (CT) is an imaging technique that provides critical information on the relationship of vascular structure and dynamics in the context of underlying anatomy. A key challenge for image processing with contrast enhanced CT is that phase discrepancies are latent in different tissues due to contrast protocols, vascular dynamics, and metabolism variance. Previous studies with deep learning frameworks have been proposed for classifying contrast enhancement with networks inspired by computer vision. Here, we revisit the challenge in the context of whole abdomen contrast enhanced CTs. To capture and compensate for the complex contrast changes, we propose a novel discriminator in the form of a multi-domain disentangled representation learning network. The goal of this network is to learn an intermediate representation that separates contrast enhancement from anatomy and enables classification of images with varying contrast time. Briefly, our unpaired contrast disentangling GAN(CD-GAN) Discriminator follows the ResNet architecture to classify a CT scan from different enhancement phases. To evaluate the approach, we trained the enhancement phase classifier on 21060 slices from two clinical cohorts of 230 subjects. The scans were manually labeled with three independent enhancement phases (non-contrast, portal venous and delayed). Testing was performed on 9100 slices from 30 independent subjects who had been imaged with CT scans from all contrast phases. Performance was quantified in terms of the multi-class normalized confusion matrix. The proposed network significantly improved correspondence over baseline UNet, ResNet50 and StarGAN's performance of accuracy scores 0.54. 0.55, 0.62 and 0.91, respectively (p-value<0.0001 paired t-test for ResNet versus CD-GAN). The proposed discriminator from the disentangled network presents a promising technique that may allow deeper modeling of dynamic imaging against patient specific anatomies.

Outlier Guided Optimization of Abdominal Segmentation.

Abdominal multi-organ segmentation of computed tomography (CT) images has been the subject of extensive research interest. It presents a substantial challenge in medical image processing, as the shape and distribution of abdominal organs can vary greatly among the population and within an individual over time. While continuous integration of novel datasets into the training set provides potential for better segmentation performance, collection of data at scale is not only costly, but also impractical in some contexts. Moreover, it remains unclear what marginal value additional data have to offer. Herein, we propose a single-pass active learning method through human quality assurance (QA). We built on a pre-trained 3D U-Net model for abdominal multi-organ segmentation and augmented the dataset either with outlier data (e.g., exemplars for which the baseline algorithm failed) or inliers (e.g., exemplars for which the baseline algorithm worked). The new models were trained using the augmented datasets with 5-fold cross-validation (for outlier data) and withheld outlier samples (for inlier data). Manual labeling of outliers increased Dice scores with outliers by 0.130, compared to an increase of 0.067 with inliers (p<0.001, two-tailed paired t-test). By adding 5 to 37 inliers or outliers to training, we find that the marginal value of adding outliers is higher than that of adding inliers. In summary, improvement on single-organ performance was obtained without diminishing multi-organ performance or significantly increasing training time. Hence, identification and correction of baseline failure cases present an effective and efficient method of selecting training data to improve algorithm performance.