Improving the Generalizability of Deep Learning for T2-Lesion Segmentation of Gliomas in the Post-Treatment Setting.

Although fully automated volumetric approaches for monitoring brain tumor response have many advantages, most available deep learning models are optimized for highly curated, multi-contrast MRI from newly diagnosed gliomas, which are not representative of post-treatment cases in the clinic. Improving segmentation for treated patients is critical to accurately tracking changes in response to therapy. We investigated mixing data from newly diagnosed (n = 208) and treated (n = 221) gliomas in training, applying transfer learning (TL) from pre- to post-treatment imaging domains, and incorporating spatial regularization for T2-lesion segmentation using only T2 FLAIR images as input to improve generalization post-treatment. These approaches were evaluated on 24 patients suspected of progression who had received prior treatment. Including 26% of treated patients in training improved performance by 13.9%, and including more treated and untreated patients resulted in minimal changes. Fine-tuning with treated glioma improved sensitivity compared to data mixing by 2.5% (p < 0.05), and spatial regularization further improved performance when used with TL by 95th HD, Dice, and sensitivity (6.8%, 0.8%, 2.2%; p < 0.05). While training with ≥60 treated patients yielded the majority of performance gain, TL and spatial regularization further improved T2-lesion segmentation to treated gliomas using a single MR contrast and minimal processing, demonstrating clinical utility in response assessment.

The knee connectome: A novel tool for studying spatiotemporal change in cartilage thickness.

Cartilage thickness change is a well-documented biomarker of osteoarthritis pathogenesis. However, there is still much to learn about the spatial and temporal patterns of cartilage thickness change in health and disease. In this study, we develop a novel analysis method for elucidating such patterns using a functional connectivity approach. Descriptive statistics are reported for 1186 knees that did not develop osteoarthritis during the 8 years of observation, which we present as a model of cartilage thickness change related to healthy aging. Within the control population, patterns vary greatly between male and female subjects, while body mass index (BMI) has a more moderate impact. Finally, several differences are shown between knees that did and did not develop osteoarthritis. Some but not all significance appears to be accounted for by differences in sex, BMI, and knee alignment. With this work, we present the connectome as a novel tool for studying spatiotemporal dynamics of tissue change.

Comparing bone shape models from deep learning processing of magnetic resonance imaging to computed tomography-based models.

Background: The purpose of this study was to develop a deep learning approach to automatically segment the scapular bone on magnetic resonance imaging (MRI) images and to compare the accuracy of these three-dimensional (3D) models with that of 3D computed tomography (CT). Methods: Fifty-five patients with high-resolution 3D fat-saturated T2 MRI were retrospectively identified. The underlying pathology included rotator cuff tendinopathy and tears, shoulder instability, and impingement. Two experienced musculoskeletal researchers manually segmented the scapular bone. Five cross-validation training and validation splits were generated to independently train two-dimensional (2D) and 3D models using a convolutional neural network approach. Model performance was evaluated using the Dice similarity coefficient (DSC). All models with DSC > 0.70 were ensembled and used for the test set, which consisted of four patients with matching high-resolution MRI and CT scans. Clinically relevant glenoid measurements, including glenoid height, width, and retroversion, were calculated for two of the patients. Paired t-tests and Wilcoxon signed-rank tests were used to compare the DSC of the models. Results: The 2D and 3D models achieved a best DSC of 0.86 and 0.82, respectively, with no significant difference observed. Augmentation of imaging data significantly improved 3D but not 2D model performance. In comparing clinical measurements of 3D MRI and CT, there was a mean difference ranging from 1.29 mm to 3.46 mm and 0.05° to 7.47°. Conclusion: We have presented a fully automatic, deep learning-based strategy for extracting scapular shape from a high-resolution MRI scan. Further developments of this technology have the potential to allow for surgeons to obtain all clinically relevant information from MRI scans and reduce the need for multiple imaging studies for patients with shoulder pathology.

Large-Scale Analysis of Meniscus Morphology as Risk Factor for Knee Osteoarthritis.

OBJECTIVE: Although it is established that structural damage of the meniscus is linked to knee osteoarthritis (OA) progression, the predisposition to future development of OA because of geometric meniscal shapes is plausible and unexplored. This study aims to identify common variations in meniscal shape and determine their relationships to tissue morphology, OA onset, and longitudinal changes in cartilage thickness. METHODS: A total of 4,790 participants from the Osteoarthritis Initiative data set were studied. A statistical shape model was developed for the meniscus, and shape scores were evaluated between a control group and an OA incidence group. Shape features were then associated with cartilage thickness changes over 8 years to localize the relationship between meniscus shape and cartilage degeneration. RESULTS: Seven shape features between the medial and lateral menisci were identified to be different between knees that remain normal and those that develop OA. These include length-width ratios, horn lengths, root attachment angles, and concavity. These "at-risk" shapes were linked to unique cartilage thickness changes that suggest a relationship between meniscus geometry and decreased tibial coverage and rotational imbalances. Additionally, strong associations were found between meniscal shape and demographic subpopulations, future tibial extrusion, and meniscal and ligamentous tears. CONCLUSION: This automatic method expanded upon known meniscus characteristics that are associated with the onset of OA and discovered novel shape features that have yet to be investigated in the context of OA risk.

A More Posterior Tibial Tubercle (Decreased Sagittal Tibial Tubercle-Trochlear Groove Distance) Is Significantly Associated With Patellofemoral Joint Degenerative Cartilage Change: A Deep Learning Analysis.

PURPOSE: To perform patellofemoral joint (PFJ) geometric measurements on knee magnetic resonance imaging scans and determine their relations with chondral lesions in a multicenter cohort using deep learning. METHODS: The sagittal tibial tubercle-trochlear groove (sTTTG) distance, tibial tubercle-trochlear groove distance, trochlear sulcus angle, trochlear depth, Caton-Deschamps Index (CDI), and flexion angle were measured by use of deep learning-generated segmentations on a subset of the Osteoarthritis Initiative study with radiologist-graded PFJ cartilage grades (n = 2,461). Kruskal-Wallis H tests were performed to compare differences in PFJ morphology between subjects without PFJ osteoarthritis (OA) and those with PFJ OA. PFJ morphology was correlated with secondary outcomes of mean patellar cartilage thickness and mean patellar cartilage T2 relaxation time using linear regression models controlling for age, sex, and body mass index. RESULTS: A total of 1,626 knees did not have PFJ OA, whereas 835 knees had PFJ OA. Knees without PFJ OA had an increased (anterior) sTTTG distance (mean ± standard deviation, 11.1 ± 12.8 mm) compared with knees with PFJ OA (8.4 ± 12.7 mm) (P < .001), indicating a more posterior tibial tubercle in subjects with PFJ OA. Knees without PFJ OA had a decreased sulcus angle (127.4° ± 7.1° vs 128.0° ± 8.4°, P = .01) and increased trochlear depth (9.1 ± 1.7 mm vs 9.0 ± 2.0 mm, P = .03) compared with knees with PFJ OA. Decreased patellar cartilage thickness was associated with decreased trochlear depth (ß = 0.12, P = .002) and increased CDI (ß = -0.07, P < .001). Increased patellar cartilage T2 relaxation time was correlated with decreased sTTTG distance (ß = -0.08, P = .01), decreased sulcus angle (ß = -0.12, P = .04), and decreased CDI (ß = -0.12, P < .001). CONCLUSIONS: PFJ OA, patellar cartilage thickness, and patellar cartilage T2 relaxation time were shown to be associated with the underlying geometries within the PFJ. This large longitudinal study highlights that a decreased sTTTG distance (i.e., a more posterior tibial tubercle) is significantly associated with PFJ degenerative cartilage change. LEVEL OF EVIDENCE: Level III, retrospective comparative prognostic trial.

Region of interest-specific loss functions improve T2 quantification with ultrafast T2 mapping MRI sequences in knee, hip and lumbar spine.

MRI T2 mapping sequences quantitatively assess tissue health and depict early degenerative changes in musculoskeletal (MSK) tissues like cartilage and intervertebral discs (IVDs) but require long acquisition times. In MSK imaging, small features in cartilage and IVDs are crucial for diagnoses and must be preserved when reconstructing accelerated data. To these ends, we propose region of interest-specific postprocessing of accelerated acquisitions: a recurrent UNet deep learning architecture that provides T2 maps in knee cartilage, hip cartilage, and lumbar spine IVDs from accelerated T2-prepared snapshot gradient-echo acquisitions, optimizing for cartilage and IVD performance with a multi-component loss function that most heavily penalizes errors in those regions. Quantification errors in knee and hip cartilage were under 10% and 9% from acceleration factors R = 2 through 10, respectively, with bias for both under 3 ms for most of R = 2 through 12. In IVDs, mean quantification errors were under 12% from R = 2 through 6. A Gray Level Co-Occurrence Matrix-based scheme showed knee and hip pipelines outperformed state-of-the-art models, retaining smooth textures for most R and sharper ones through moderate R. Our methodology yields robust T2 maps while offering new approaches for optimizing and evaluating reconstruction algorithms to facilitate better preservation of small, clinically relevant features.

Surface spherical encoding and contrastive learning for virtual bone shape aging.

Bone shape changes are considered a relevant biomarker in understanding the onset and progression of knee osteoarthritis (OA). This study used a novel deep learning pipeline to predict longitudinal bone shape changes in the femur four years in advance, using bone surfaces that were extracted in knee MRIs from the OA initiative study, via a segmentation procedure and encoded as shape maps using spherical coordinates. Given a sequence of three consecutive shape maps (collected in a time window of 24 months), a fully convolutional network was trained to predict the whole bone surface 48 months after the last observed time point, and a classifier to diagnose OA in the predicted maps. For this, a novel multi-term loss function, based on contrastive learning was designed. Experimental results show that the model predicted shape changes with an L1 error comparable to the MRI slice thickness (0.7mm). Next, an ablation study demonstrated that the introduction of a contrastive term in the loss improved sensitivity of the OA classifier, increasing sensitivity from 0.537 to 0.709, just shy of the upper bound of 0.740 computed on the ground truth bone shape maps. Our approach provides a promising tool, suitable for patient specific OA trajectory analysis.

Studying osteoarthritis with artificial intelligence applied to magnetic resonance imaging.

The 3D nature and soft-tissue contrast of MRI makes it an invaluable tool for osteoarthritis research, by facilitating the elucidation of disease pathogenesis and progression. The recent increasing employment of MRI has certainly been stimulated by major advances that are due to considerable investment in research, particularly related to artificial intelligence (AI). These AI-related advances are revolutionizing the use of MRI in clinical research by augmenting activities ranging from image acquisition to post-processing. Automation is key to reducing the long acquisition times of MRI, conducting large-scale longitudinal studies and quantitatively defining morphometric and other important clinical features of both soft and hard tissues in various anatomical joints. Deep learning methods have been used recently for multiple applications in the musculoskeletal field to improve understanding of osteoarthritis. Compared with labour-intensive human efforts, AI-based methods have advantages and potential in all stages of imaging, as well as post-processing steps, including aiding diagnosis and prognosis. However, AI-based methods also have limitations, including the arguably limited interpretability of AI models. Given that the AI community is highly invested in uncovering uncertainties associated with model predictions and improving their interpretability, we envision future clinical translation and progressive increase in the use of AI algorithms to support clinicians in optimizing patient care.

Uncovering associations between data-driven learned qMRI biomarkers and chronic pain.

Knee pain is the most common and debilitating symptom of knee osteoarthritis (OA). While there is a perceived association between OA imaging biomarkers and pain, there are weak or conflicting findings for this relationship. This study uses Deep Learning (DL) models to elucidate associations between bone shape, cartilage thickness and T2 relaxation times extracted from Magnetic Resonance Images (MRI) and chronic knee pain. Class Activation Maps (Grad-CAM) applied on the trained chronic pain DL models are used to evaluate the locations of features associated with presence and absence of pain. For the cartilage thickness biomarker, the presence of features sensitive for pain presence were generally located in the medial side, while the features specific for pain absence were generally located in the anterior lateral side. This suggests that the association of cartilage thickness and pain varies, requiring a more personalized averaging strategy. We propose a novel DL-guided definition for cartilage thickness spatial averaging based on Grad-CAM weights. We showed a significant improvement modeling chronic knee pain with the inclusion of the novel biomarker definition: likelihood ratio test p-values of 7.01 × 10-33 and 1.93 × 10-14 for DL-guided cartilage thickness averaging for the femur and tibia, respectively, compared to the cartilage thickness compartment averaging.

Augmenting Osteoporosis Imaging with Machine Learning.

PURPOSE OF REVIEW: In this paper, we discuss how recent advancements in image processing and machine learning (ML) are shaping a new and exciting era for the osteoporosis imaging field. With this paper, we want to give the reader a basic exposure to the ML concepts that are necessary to build effective solutions for image processing and interpretation, while presenting an overview of the state of the art in the application of machine learning techniques for the assessment of bone structure, osteoporosis diagnosis, fracture detection, and risk prediction. RECENT FINDINGS: ML effort in the osteoporosis imaging field is largely characterized by "low-cost" bone quality estimation and osteoporosis diagnosis, fracture detection, and risk prediction, but also automatized and standardized large-scale data analysis and data-driven imaging biomarker discovery. Our effort is not intended to be a systematic review, but an opportunity to review key studies in the recent osteoporosis imaging research landscape with the ultimate goal of discussing specific design choices, giving the reader pointers to possible solutions of regression, segmentation, and classification tasks as well as discussing common mistakes.

The International Workshop on Osteoarthritis Imaging Knee MRI Segmentation Challenge: A Multi-Institute Evaluation and Analysis Framework on a Standardized Dataset.

PURPOSE: To organize a multi-institute knee MRI segmentation challenge for characterizing the semantic and clinical efficacy of automatic segmentation methods relevant for monitoring osteoarthritis progression. MATERIALS AND METHODS: A dataset partition consisting of three-dimensional knee MRI from 88 retrospective patients at two time points (baseline and 1-year follow-up) with ground truth articular (femoral, tibial, and patellar) cartilage and meniscus segmentations was standardized. Challenge submissions and a majority-vote ensemble were evaluated against ground truth segmentations using Dice score, average symmetric surface distance, volumetric overlap error, and coefficient of variation on a holdout test set. Similarities in automated segmentations were measured using pairwise Dice coefficient correlations. Articular cartilage thickness was computed longitudinally and with scans. Correlation between thickness error and segmentation metrics was measured using the Pearson correlation coefficient. Two empirical upper bounds for ensemble performance were computed using combinations of model outputs that consolidated true positives and true negatives. RESULTS: Six teams (T 1-T 6) submitted entries for the challenge. No differences were observed across any segmentation metrics for any tissues (P = .99) among the four top-performing networks (T 2, T 3, T 4, T 6). Dice coefficient correlations between network pairs were high (> 0.85). Per-scan thickness errors were negligible among networks T 1-T 4 (P = .99), and longitudinal changes showed minimal bias (< 0.03 mm). Low correlations (ρ < 0.41) were observed between segmentation metrics and thickness error. The majority-vote ensemble was comparable to top-performing networks (P = .99). Empirical upper-bound performances were similar for both combinations (P = .99). CONCLUSION: Diverse networks learned to segment the knee similarly, where high segmentation accuracy did not correlate with cartilage thickness accuracy and voting ensembles did not exceed individual network performance.See also the commentary by Elhalawani and Mak in this issue.Keywords: Cartilage, Knee, MR-Imaging, Segmentation © RSNA, 2020Supplemental material is available for this article.

T2 analysis of the entire osteoarthritis initiative dataset.

While substantial work has been done to understand the relationships between cartilage T2 relaxation times and osteoarthritis (OA), diagnostic and prognostic abilities of T2 on a large population yet need to be established. Using 3921 manually annotated 2D multi-slice multi-echo spin-echo magnetic resonance imaging volume, a segmentation model for automatic knee cartilage segmentation was built and evaluated. The optimized model was then used to calculate T2 values on the entire osteoarthritis initiative (OAI) dataset composed of longitudinal acquisitions of 4796 unique patients, 25 729 magnetic resonance imaging studies in total. Cross-sectional relationships between T2 values, OA risk factors, radiographic OA, and pain were analyzed in the entire OAI dataset. The performance of T2 values in predicting the future incidence of radiographic OA as well as total knee replacement (TKR) were also explored. Automatic T2 values were comparable with manual ones. Significant associations between T2 relaxation times and demographic and clinical variables were found. Subjects in the highest 25% quartile of tibio-femoral T2 values had a five times higher risk of radiographic OA incidence 2 years later. Elevation of medial femur T2 values was significantly associated with TKR after 5 years (coeff = 0.10; P = .036; CI = [0.01,0.20]). Our investigation reinforces the predictive value of T2 for future incidence OA and TKR. The inclusion of T2 averages from the automatic segmentation model improved several evaluation metrics when compared to only using demographic and clinical variables.

Towards understanding mechanistic subgroups of osteoarthritis: 8-year cartilage thickness trajectory analysis.

Many studies have validated cartilage thickness as a biomarker for knee osteoarthritis (OA); however, few studies investigate beyond cross-sectional observations or comparisons across two timepoints. By characterizing the trajectory of cartilage thickness changes over 8 years in healthy individuals from the OA initiative data set, this study discovers associations between the dynamics of cartilage changes and OA incidence. A fully automated cartilage segmentation and thickness measurement method were developed and validated against manual measurements: mean absolute error = 0.11-0.14 mm (n = 4129 knees) and automatic reproducibility = 0.04-0.07 mm (n = 316 knees). The mean thickness for the medial and lateral tibia (MT, LT), central weight-bearing medial and lateral femur (cMF, cLF), and patella (P) cartilage compartments were quantified for 1453 knees at seven timepoints. Trajectory subgroups were defined per cartilage compartment such as stable, thinning to thickening, accelerated thickening, plateaued thickening, thickening to thinning, accelerated thinning, or plateaued thinning. For tibiofemoral compartments, the stable (22%-36%) and plateaued thinning (22%-37%) trajectories were the most common, with an average initial velocity (µm/month), acceleration (µm/month2 ) for the plateaued thinning trajectories LT: -2.66, 0.0326; MT: -2.49, 0.0365; cMF: -3.51, 0.0509; and cLF: -2.68, 0.041. In the patella compartment, the plateaued thinning (35%) and thickening to thinning (24%) trajectories were the most common, with an average initial velocity, acceleration for each -4.17, 0.0424; 1.95, -0.0835. Knees with nonstable trajectories had higher adjusted odds of OA incidence than stable trajectories: accelerated thickening, accelerated thinning, and plateaued thinning trajectories of the MT had adjusted odds ratio (OR) of 18.9, 5.48, and 1.47, respectively; in the cMF, adjusted OR of 8.55, 10.1, and 2.61, respectively.

Deep Learning for Hierarchical Severity Staging of Anterior Cruciate Ligament Injuries from MRI.

PURPOSE: To evaluate the diagnostic utility of two convolutional neural networks (CNNs) for severity staging of anterior cruciate ligament (ACL) injuries. MATERIALS AND METHODS: In this retrospective study, 1243 knee MR images (1008 intact, 18 partially torn, 77 fully torn, and 140 reconstructed ACLs) from 224 patients (mean age, 47 years ± 14 [standard deviation]; 54% women) were analyzed. The MRI examinations were performed between 2011 and 2014. A modified scoring metric was used. Classification of ACL injuries using deep learning involved use of two types of CNN, one with three-dimensional (3D) and the other with two-dimensional (2D) convolutional kernels. Performance metrics included sensitivity, specificity, weighted Cohen κ, and overall accuracy, and the McNemar test was used to compare the performance of the CNNs. RESULTS: The overall accuracies for ACL injury classification using the 3D CNN and 2D CNN were 89% (225 of 254) and 92% (233 of 254), respectively (P = .27), and both CNNs had a weighted Cohen κ of 0.83. The 2D CNN and 3D CNN performed similarly in classifying intact ACLs (2D CNN, sensitivity of 93% [188 of 203] and specificity of 90% [46 of 51] vs 3D CNN, sensitivity of 89% [180 of 203] and specificity of 88% [45 of 51]). Classification of full tears by both networks was also comparable (2D CNN, sensitivity of 82% [14 of 17] and specificity of 94% [222 of 237] vs 3D CNN, sensitivity of 76% [13 of 17] and specificity of 100% [236 of 237]). The 2D CNN classified all reconstructed ACLs correctly. CONCLUSION: Two-dimensional and 3D CNNs applied to ACL lesion classification had high sensitivity and specificity, suggesting that these networks could be used to help nonexperts grade ACL injuries. Supplemental material is available for this article. © RSNA, 2020.

Learning osteoarthritis imaging biomarkers from bone surface spherical encoding.

PURPOSE: To learn bone shape features from spherical bone map of knee MRI images using established convolutional neural networks (CNN) and use these features to diagnose and predict osteoarthritis (OA). METHODS: A bone segmentation model was trained on 25 manually annotated 3D MRI volumes to segment the femur, tibia, and patella from 47 078 3D MRI volumes. Each bone segmentation was converted to a 3D point cloud and transformed into spherical coordinates. Different fusion strategies were performed to merge spherical maps obtained by each bone. A total of 41 822 merged spherical maps with corresponding Kellgren-Lawrence grades for radiographic OA were used to train a CNN classifier model to diagnose OA using bone shape learned features. Several OA Diagnosis models were tested and the weights for each trained model were transferred to the OA Incidence models. The OA incidence task consisted of predicting OA from a healthy scan within a range of eight time points, from 1 y to 8 y. The validation performance was compared and the test set performance was reported. RESULTS: The OA Diagnosis model had an area-under-the-curve (AUC) of 0.905 on the test set with a sensitivity and specificity of 0.815 and 0.839. The OA Incidence models had an AUC ranging from 0.841 to 0.646 on the test set for the range from 1 y to 8 y. CONCLUSION: Bone shape was successfully used as a predictive imaging biomarker for OA. This approach is novel in the field of deep learning applications for musculoskeletal imaging and can be expanded to other OA biomarkers.

Deep Learning for Lesion Detection, Progression, and Prediction of Musculoskeletal Disease.

Deep learning is one of the most exciting new areas in medical imaging. This review article provides a summary of the current clinical applications of deep learning for lesion detection, progression, and prediction of musculoskeletal disease on radiographs, computed tomography (CT), magnetic resonance imaging (MRI), and nuclear medicine. Deep-learning methods have shown success for estimating pediatric bone age, detecting fractures, and assessing the severity of osteoarthritis on radiographs. In particular, the high diagnostic performance of deep-learning approaches for estimating pediatric bone age and detecting fractures suggests that the new technology may soon become available for use in clinical practice. Recent studies have also documented the feasibility of using deep-learning methods for identifying a wide variety of pathologic abnormalities on CT and MRI including internal derangement, metastatic disease, infection, fractures, and joint degeneration. However, the detection of musculoskeletal disease on CT and especially MRI is challenging, as it often requires analyzing complex abnormalities on multiple slices of image datasets with different tissue contrasts. Thus, additional technical development is needed to create deep-learning methods for reliable and repeatable interpretation of musculoskeletal CT and MRI examinations. Furthermore, the diagnostic performance of all deep-learning methods for detecting and characterizing musculoskeletal disease must be evaluated in prospective studies using large image datasets acquired at different institutions with different imaging parameters and different imaging hardware before they can be implemented in clinical practice. LEVEL OF EVIDENCE: 5 TECHNICAL EFFICACY STAGE: 2 J. MAGN. RESON. IMAGING 2020;52:1607-1619.

Exudates Segmentation using Fully Convolutional Neural Network and Auxiliary Codebook.

Diabetic retinopathy (DR) is an asymptotic complication of diabetes and the leading cause of preventable blindness in the working-age population. Early detection and treatment of DR is critical to avoid vision loss. Exudates are one of the earliest and most prevalent signs of DR. In this work, we propose a novel two-stage method for the detection and segmentation of exudates in fundus photographs. In the first stage, a fully convolutional neural network architecture is trained to segment exudates using small image patches. Next, an auxilary codebook is built from network's intermediate layer output using incremental principal component analysis. Finally, outputs of both systems are combined to produce final result. Compared to other methods, the proposed algorithm does not require computation of candidate regions or removal of other anatomical structures. Furthermore, a transfer learning approach was applied to improve the performance of the system. The proposed method was evaluated using publicly available E-Ophtha datasets. It achieved better results than the state-of-the-art methods in terms of sensitivity and specificity metrics. The proposed method accomplished better results using a diseased//not diseased evaluation scenario which indicates its applicability for screening purposes. Simplicity, performance, efficiency and robustness of the proposed method demonstrate its suitability for diabetic retinopathy screening applications.

DISCERN: Generative Framework for Vessel Segmentation using Convolutional Neural Network and Visual Codebook.

This paper presents a novel two-stage vessel segmentation framework applied to retinal fundus images. In the first stage a convolutional neural network (CNN) is used to correlate an image patch with a corresponding groundtruth reduced using Totally Random Trees Embedding. In the second stage training patches are forward propagated through CNN to create a visual codebook. The codebook is used to build a generative nearest neighbour search space that can be queried by feature vectors created through forward propagating previously-unseen patches through CNN. The proposed framework is able to generate segmentation patches that were not seen during training. Evaluated using publicly available datasets (DRIVE, STARE) demonstrated better performance than state-of-the-art methods in terms of multiple evaluation metrics. The accuracy, robustness, speed and simplicity of the proposed framework demonstrates its suitability for automated vessel segmentation.

Microaneurysm detection using fully convolutional neural networks.

BACKROUND AND OBJECTIVES: Diabetic retinopathy is a microvascular complication of diabetes that can lead to sight loss if treated not early enough. Microaneurysms are the earliest clinical signs of diabetic retinopathy. This paper presents an automatic method for detecting microaneurysms in fundus photographies. METHODS: A novel patch-based fully convolutional neural network with batch normalization layers and Dice loss function is proposed. Compared to other methods that require up to five processing stages, it requires only three. Furthermore, to the best of the authors' knowledge, this is the first paper that shows how to successfully transfer knowledge between datasets in the microaneurysm detection domain. RESULTS: The proposed method was evaluated using three publicly available and widely used datasets: E-Ophtha, DIARETDB1, and ROC. It achieved better results than state-of-the-art methods using the FROC metric. The proposed algorithm accomplished highest sensitivities for low false positive rates, which is particularly important for screening purposes. CONCLUSIONS: Performance, simplicity, and robustness of the proposed method demonstrates its suitability for diabetic retinopathy screening applications.

A fluid-dynamic based approach to reconnect the retinal vessels in fundus photography.

This paper introduces the use of fluid-dynamic modeling to determine the connectivity of overlapping venous and arterial vessels in fundus images. Analysis of the retinal vascular network may provide information related to systemic and local disorders. However, the automated identification of the vascular trees in retinal images is a challenging task due to the low signal-to-noise ratio, nonuniform illumination and the fact that fundus photography is a projection on to the imaging plane of three-dimensional retinal tissue. A zero-dimensional model was created to estimate the hemodynamic status of candidate tree configurations. Simulated annealing was used to search for an optimal configuration. Experimental results indicate that simulated annealing was very efficient on test cases that range from small to medium size networks, while ineffective on large networks. Although for large networks the nonconvexity of the cost function and the large solution space made searching for the optimal solution difficult, the accuracy (average success rate = 98.35%), and simplicity of our novel approach demonstrate its potential effectiveness in segmenting retinal vascular trees.