Graph-based automatic detection and classification of lesion changes in pairs of CT studies for oncology follow-up.

PURPOSE: Radiological follow-up of oncology patients requires the quantitative analysis of lesion changes in longitudinal imaging studies, which is time-consuming, requires expertise, and is subject to variability. This paper presents a comprehensive graph-based method for the automatic detection and classification of lesion changes in current and prior CT scans. METHODS: The inputs are the current and prior CT scans and their organ and lesion segmentations. Classification of lesion changes is formalized as bipartite graph matching where lesion pairings are computed by adaptive overlap-based lesion matching. Six types of lesion changes are computed by connected components analysis. The method was evaluated on 208 pairs of lung and liver CT scans from 57 patients with 4600 lesions, 1713 lesion matchings and 2887 lesion changes. Ground-truth lesion segmentations, lesion matchings and lesion changes were created by an expert radiologist. RESULTS: Our method yields a lesion matching rate accuracy of 99.7% (394/395) and 95.0% (1252/1318) for the lung and liver datasets. Precision and recall are > 0.99 and 0.94 and 0.95 (respectively) for the detection of lesion changes. The analysis of lesion changes helped the radiologist detect 48 missed lesions and 8 spurious lesions in the input ground-truth lesion datasets. CONCLUSION: The classification of lesion classification provides the clinician with a readily accessible and intuitive identification and classification of the lesion changes and their patterns in support of clinical decision making. Comprehensive automatic computer-aided lesion matching and analysis of lesion changes may improve quantitative follow-up and evaluation of disease status, assessment of treatment efficacy and response to therapy.

Follow-up of liver metastases: a comparison of deep learning and RECIST 1.1.

OBJECTIVES: To compare liver metastases changes in CT assessed by radiologists using RECIST 1.1 and with aided simultaneous deep learning-based volumetric lesion changes analysis. METHODS: A total of 86 abdominal CT studies from 43 patients (prior and current scans) of abdominal CT scans of patients with 1041 liver metastases (mean = 12.1, std = 11.9, range 1-49) were analyzed. Two radiologists performed readings of all pairs; conventional with RECIST 1.1 and with computer-aided assessment. For computer-aided reading, we used a novel simultaneous multi-channel 3D R2U-Net classifier trained and validated on other scans. The reference was established by having an expert radiologist validate the computed lesion detection and segmentation. The results were then verified and modified as needed by another independent radiologist. The primary outcome measure was the disease status assessment with the conventional and the computer-aided readings with respect to the reference. RESULTS: For conventional and computer-aided reading, there was a difference in disease status classification in 40 out of 86 (46.51%) and 10 out of 86 (27.9%) CT studies with respect to the reference, respectively. Computer-aided reading improved conventional reading in 30 CT studies by 34.5% for two readers (23.2% and 46.51%) with respect to the reference standard. The main reason for the difference between the two readings was lesion volume differences (p = 0.01). CONCLUSIONS: AI-based computer-aided analysis of liver metastases may improve the accuracy of the evaluation of neoplastic liver disease status. CLINICAL RELEVANCE STATEMENT: AI may aid radiologists to improve the accuracy of evaluating changes over time in metastasis of the liver. KEY POINTS: • Classification of liver metastasis changes improved significantly in one-third of the cases with an automatically generated comprehensive lesion and lesion changes report. • Simultaneous deep learning changes detection and volumetric assessment may improve the evaluation of liver metastases temporal changes potentially improving disease management.

The Liver Tumor Segmentation Benchmark (LiTS).

In this work, we report the set-up and results of the Liver Tumor Segmentation Benchmark (LiTS), which was organized in conjunction with the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 and the International Conferences on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2017 and 2018. The image dataset is diverse and contains primary and secondary tumors with varied sizes and appearances with various lesion-to-background levels (hyper-/hypo-dense), created in collaboration with seven hospitals and research institutions. Seventy-five submitted liver and liver tumor segmentation algorithms were trained on a set of 131 computed tomography (CT) volumes and were tested on 70 unseen test images acquired from different patients. We found that not a single algorithm performed best for both liver and liver tumors in the three events. The best liver segmentation algorithm achieved a Dice score of 0.963, whereas, for tumor segmentation, the best algorithms achieved Dices scores of 0.674 (ISBI 2017), 0.702 (MICCAI 2017), and 0.739 (MICCAI 2018). Retrospectively, we performed additional analysis on liver tumor detection and revealed that not all top-performing segmentation algorithms worked well for tumor detection. The best liver tumor detection method achieved a lesion-wise recall of 0.458 (ISBI 2017), 0.515 (MICCAI 2017), and 0.554 (MICCAI 2018), indicating the need for further research. LiTS remains an active benchmark and resource for research, e.g., contributing the liver-related segmentation tasks in http://medicaldecathlon.com/. In addition, both data and online evaluation are accessible via https://competitions.codalab.org/competitions/17094.

Liver lesion changes analysis in longitudinal CECT scans by simultaneous deep learning voxel classification with SimU-Net.

The identification and quantification of liver lesions changes in longitudinal contrast enhanced CT (CECT) scans is required to evaluate disease status and to determine treatment efficacy in support of clinical decision-making. This paper describes a fully automatic end-to-end pipeline for liver lesion changes analysis in consecutive (prior and current) abdominal CECT scans of oncology patients. The three key novelties are: (1) SimU-Net, a simultaneous multi-channel 3D R2U-Net model trained on pairs of registered scans of each patient that identifies the liver lesions and their changes based on the lesion and healthy tissue appearance differences; (2) a model-based bipartite graph lesions matching method for the analysis of lesion changes at the lesion level; (3) a method for longitudinal analysis of one or more of consecutive scans of a patient based on SimU-Net that handles major liver deformations and incorporates lesion segmentations from previous analysis. To validate our methods, five experimental studies were conducted on a unique dataset of 3491 liver lesions in 735 pairs from 218 clinical abdominal CECT scans of 71 patients with metastatic disease manually delineated by an expert radiologist. The pipeline with the SimU-Net model, trained and validated on 385 pairs and tested on 249 pairs, yields a mean lesion detection recall of 0.86±0.14, a precision of 0.74±0.23 and a lesion segmentation Dice of 0.82±0.14 for lesions > 5 mm. This outperforms a reference standalone 3D R2-UNet mdel that analyzes each scan individually by ∼50% in precision with similar recall and Dice score on the same training and test datasets. For lesions matching, the precision is 0.86±0.18 and the recall is 0.90±0.15. For lesion classification, the specificity is 0.97±0.07, the precision is 0.85±0.31, and the recall is 0.86±0.23. Our new methods provide accurate and comprehensive results that may help reduce radiologists' time and effort and improve radiological oncology evaluation.

Progression of cRORA (Complete RPE and Outer Retinal Atrophy) in Dry Age-Related Macular Degeneration Measured Using SD-OCT.

Purpose: The purpose of this study was to evaluate the long-term rate of progression and baseline predictors of geographic atrophy (GA) using complete retinal pigment epithelium and outer retinal atrophy (cRORA) annotation criteria. Methods: This is a retrospective study. Columns of GA were manually annotated by two graders using a self-developed software on optical coherence tomography (OCT) B-scans and projected onto the infrared images. The primary outcomes were: (1) rate of area progression, (2) rate of square root area progression, and (3) rate of radial progression towards the fovea. The effects of 11 additional baseline predictors on the primary outcomes were analyzed: total area, focality (defined as the number of lesions whose area is >0.05 mm2), circularity, total lesion perimeter, minimum diameter, maximum diameter, minimum distance from the center, sex, age, presence/absence of hypertension, and lens status. Results: GA was annotated in 33 pairs of baseline and follow-up OCT scans from 33 eyes of 18 patients with dry age-related macular degeneration (AMD) followed for at least 6 months. The mean rate of area progression was 1.49 ± 0.86 mm2/year (P < 0.0001 vs. baseline), and the mean rate of square root area progression was 0.33 ± 0.15 mm/year (P < 0.0001 vs. baseline). The mean rate of radial progression toward the fovea was 0.07 ± 0.11 mm/year. A multiple variable linear regression model (adjusted r2 = 0.522) revealed that baseline focality and female sex were significantly correlated with the rate of GA area progression. Conclusions: GA area progression was quantified using OCT as an alternative to conventional measurements performed on fundus autofluorescence images. Baseline focality correlated with GA area progression rate and lesion's minimal distance from the center correlated with GA radial progression rate toward the center. These may be important markers for the assessment of GA activity. Translational Relevance: Advanced method linking specific retinal micro-anatomy to GA area progression analysis.

A column-based deep learning method for the detection and quantification of atrophy associated with AMD in OCT scans.

The objective quantification of retinal atrophy associated with age-related macular degeneration (AMD) is required for clinical diagnosis, follow-up, treatment efficacy evaluation, and clinical research. Spectral Domain Optical Coherence Tomography (OCT) has become an essential imaging technology to evaluate the macula. This paper describes a novel automatic method for the identification and quantification of atrophy associated with AMD in OCT scans and its visualization in the corresponding infrared imaging (IR) image. The method is based on the classification of light scattering patterns in vertical pixel-wide columns (A-scans) in OCT slices (B-scans) in which atrophy appears with a custom column-based convolutional neural network (CNN). The network classifies individual columns with 3D column patches formed by adjacent neighboring columns from the volumetric OCT scan. Subsequent atrophy columns form atrophy segments which are then projected onto the IR image and are used to identify and segment atrophy lesions in the IR image and to measure their areas and distances from the fovea. Experimental results on 106 clinical OCT scans (5,207 slices) in which cRORA atrophy (the end point of advanced dry AMD) was identified in 2,952 atrophy segments and 1,046 atrophy lesions yield a mean F1 score of 0.78 (std 0.06) and an AUC of 0.937, both close to the observer variability. Automated computer-based detection and quantification of atrophy associated with AMD using a column-based CNN classification in OCT scans can be performed at expert level and may be a useful clinical decision support and research tool for the diagnosis, follow-up and treatment of retinal degenerations and dystrophies.

Automatic detection and diagnosis of sacroiliitis in CT scans as incidental findings.

Early diagnosis of sacroiliitis may lead to preventive treatment which can significantly improve the patient's quality of life in the long run. Oftentimes, a CT scan of the lower back or abdomen is acquired for suspected back pain. However, since the differences between a healthy and an inflamed sacroiliac joint in the early stages are subtle, the condition may be missed. We have developed a new automatic algorithm for the diagnosis and grading of sacroiliitis CT scans as incidental findings, for patients who underwent CT scanning as part of their lower back pain workout. The method is based on supervised machine and deep learning techniques. The input is a CT scan that includes the patient's pelvis. The output is a diagnosis for each sacroiliac joint. The algorithm consists of four steps: (1) computation of an initial region of interest (ROI) that includes the pelvic joints region using heuristics and a U-Net classifier; (2) refinement of the ROI to detect both sacroiliiac joints using a four-tree random forest; (3) individual sacroiliitis grading of each sacroiliiac joint in each CT slice with a custom slice CNN classifier, and; (4) sacroiliitis diagnosis and grading by combining the individual slice grades using a random forest. Experimental results on 484 sacroiliiac joints yield a binary and a 3-class case classification accuracy of 91.9% and 86%, a sensitivity of 95% and 82%, and an Area-Under-the-Curve of 0.97 and 0.57, respectively. Automatic computer-based analysis of CT scans has the potential of being a useful method for the diagnosis and grading of sacroiliitis as an incidental finding.