TIST-Net: style transfer in dynamic contrast enhanced MRI using spatial and temporal information.

Objective.Training deep learning models for image registration or segmentation of dynamic contrast enhanced (DCE) MRI data is challenging. This is mainly due to the wide variations in contrast enhancement within and between patients. To train a model effectively, a large dataset is needed, but acquiring it is expensive and time consuming. Instead, style transfer can be used to generate new images from existing images. In this study, our objective is to develop a style transfer method that incorporates spatio-temporal information to either add or remove contrast enhancement from an existing image.Approach.We propose a temporal image-to-image style transfer network (TIST-Net), consisting of an auto-encoder combined with convolutional long short-term memory networks. This enables disentanglement of the content and style latent spaces of the time series data, using spatio-temporal information to learn and predict key structures. To generate new images, we use deformable and adaptive convolutions which allow fine grained control over the combination of the content and style latent spaces. We evaluate our method, using popular metrics and a previously proposed contrast weighted structural similarity index measure. We also perform a clinical evaluation, where experts are asked to rank images generated by multiple methods.Main Results.Our model achieves state-of-the-art performance on three datasets (kidney, prostate and uterus) achieving an SSIM of 0.91 ± 0.03, 0.73 ± 0.04, 0.88 ± 0.04 respectively when performing style transfer between a non-enhanced image and a contrast-enhanced image. Similarly, SSIM results for style transfer from a contrast-enhanced image to a non-enhanced image were 0.89 ± 0.03, 0.82 ± 0.03, 0.87 ± 0.03. In the clinical evaluation, our method was ranked consistently higher than other approaches.Significance.TIST-Net can be used to generate new DCE-MRI data from existing images. In future, this may improve models for tasks such as image registration or segmentation by allowing small training datasets to be expanded.

Deep learning detection of diabetic retinopathy in Scotland's diabetic eye screening programme.

BACKGROUND/AIMS: Support vector machine-based automated grading (known as iGradingM) has been shown to be safe, cost-effective and robust in the diabetic retinopathy (DR) screening (DES) programme in Scotland. It triages screening episodes as gradable with no DR versus manual grading required. The study aim was to develop a deep learning-based autograder using images and gradings from DES and to compare its performance with that of iGradingM. METHODS: Retinal images, quality assurance (QA) data and routine DR grades were obtained from national datasets in 179 944 patients for years 2006-2016. QA grades were available for 744 images. We developed a deep learning-based algorithm to detect whether either eye contained ungradable images or any DR. The sensitivity and specificity were evaluated against consensus QA grades and routine grades. RESULTS: Images used in QA which were ungradable or with DR were detected by deep learning with better specificity compared with manual graders (p<0.001) and with iGradingM (p<0.001) at the same sensitivities. Any DR according to the DES final grade was detected with 89.19% (270 392/303 154) sensitivity and 77.41% (500 945/647 158) specificity. Observable disease and referable disease were detected with sensitivities of 96.58% (16 613/17 201) and 98.48% (22 600/22 948), respectively. Overall, 43.84% of screening episodes would require manual grading. CONCLUSION: A deep learning-based system for DR grading was evaluated in QA data and images from 11 years in 50% of people attending a national DR screening programme. The system could reduce the manual grading workload at the same sensitivity compared with the current automated grading system.

Fully automated volumetric measurement of malignant pleural mesothelioma by deep learning AI: validation and comparison with modified RECIST response criteria.

BACKGROUND: In malignant pleural mesothelioma (MPM), complex tumour morphology results in inconsistent radiological response assessment. Promising volumetric methods require automation to be practical. We developed a fully automated Convolutional Neural Network (CNN) for this purpose, performed blinded validation and compared CNN and human response classification and survival prediction in patients treated with chemotherapy. METHODS: In a multicentre retrospective cohort study; 183 CT datasets were split into training and internal validation (123 datasets (80 fully annotated); 108 patients; 1 centre) and external validation (60 datasets (all fully annotated); 30 patients; 3 centres). Detailed manual annotations were used to train the CNN, which used two-dimensional U-Net architecture. CNN performance was evaluated using correlation, Bland-Altman and Dice agreement. Volumetric response/progression were defined as ≤30%/≥20% change and compared with modified Response Evaluation Criteria In Solid Tumours (mRECIST) by Cohen's kappa. Survival was assessed using Kaplan-Meier methodology. RESULTS: Human and artificial intelligence (AI) volumes were strongly correlated (validation set r=0.851, p<0.0001). Agreement was strong (validation set mean bias +31 cm3 (p=0.182), 95% limits 345 to +407 cm3). Infrequent AI segmentation errors (4/60 validation cases) were associated with fissural tumour, contralateral pleural thickening and adjacent atelectasis. Human and AI volumetric responses agreed in 20/30 (67%) validation cases κ=0.439 (0.178 to 0.700). AI and mRECIST agreed in 16/30 (55%) validation cases κ=0.284 (0.026 to 0.543). Higher baseline tumour volume was associated with shorter survival. CONCLUSION: We have developed and validated the first fully automated CNN for volumetric MPM segmentation. CNN performance may be further improved by enriching future training sets with morphologically challenging features. Volumetric response thresholds require further calibration in future studies.

Calibration of coronary calcium scores determined using iterative image reconstruction (AIDR 3D) at 120, 100, and 80 kVp.

PURPOSE: Computed tomography (CT) radiation dose reduction is frequently achieved by applying lower tube voltages and using iterative reconstruction (IR). For calcium scoring, the reference protocol at 120 kVp with filtered back projection (FBP) is still used, because kVp and IR may influence the Agatston score (AS) and volume score (VS). The authors present a two-step method to optimize dose: first, to determine the lowest feasible exposure and highest noise thresholds; second, to define a calibration method that ensures that the AS and VS are similar to the reference protocol. METHODS: AS and VS were measured for an anthropomorphic thoracic phantom that includes a calcium calibration module. The phantom was scanned on a 320-row CT scanner, at tube voltages of 120 kVp using FBP, and 120, 100, and 80 kVp using adaptive iterative dose reduction (AIDR 3D) reconstruction. The minimum CTDIs were determined based on three objective quality criteria. Calibration was performed to estimate adjusted CT number thresholds for the lower kVp acquisitions. Finally, the accuracies of the total and individual insert scores at dose level close to the minimum CTDI level were investigated and compared to low (FBPLD - 120) and high (FBPHD - 120) dose reference protocols (based on ten repeated acquisitions for each group). RESULTS: IR allows the exposure to be reduced by 69% at 120 kVp, with no significant effect on the total scores when averaged over all included dose steps and compared to FBP-120 (AS: 693 vs 699, p = 0.182; VS: 588 vs 587 mm(3), p = 0.569). Also when averaged over ten repeated scans and compared to FBPHD - 120 (AS: 709 vs 704, p = 0.435; VS: 604 vs 601 mm(3), p = 0.479), there is no statistical significant effect. Reducing the peak tube voltage allows even greater dose reductions: 73% at 100 kVp and 76% at 80 kVp. The calibrated CT number thresholds for analysis at 120, 100, and 80 kVp were, respectively, 130, 133, and 160 HU for the Agatston score, and 130, 132, and 140 HU for the volume score. Following the calibration, the mean scores of the four groups with dose variation were not significantly different from the reference protocol, at 100 kVp (AS: 698 vs 699, p = 0.818; VS: 584 vs 587 mm(3), p = 0.365) or at 80 kVp (AS: 698 vs 699, p = 0.996; VS: 586 vs 587 mm(3), p = 0.827). Similarly, there was no significant score difference with FBPLD - 120 during repeated scanning: 100 kVp (AS: 690 vs 694, p = 0.394; VS: 579 vs 585 mm(3), p = 0.168) and 80 kVp (AS: 703 vs 694, p = 0.115; VS: 588 vs 585 mm(3), p = 0.613). Compared to FBPHD - 120 group, the relative score deviation for the accuracy of the 400 and 800 mg/cm(3) HA inserts with 3 and 5 mm diameter is less than 7%. However, the relative deviation of the smaller 1 mm inserts is poorer (up to 41% deviations for scores <3). CONCLUSIONS: With iterative reconstruction using AIDR 3D, deviations of the total Agatston and volume scores remain within 4% of the reference protocol. The 1 mm inserts were detected as calcification, but scores less than ten tend to be underestimated. Following the calibration process, the application of IR in combination with reduced tube voltages allows up to 76% lower radiation dose.

Quality assurance in diabetic retinal screening in South Africa.

BACKGROUND: Diabetic retinopathy (DR) is an important biomarker for microvascular disease and blindness. Digital fundus photography is a cost-effective way of screening for DR. Access to DR screening is difficult for many South Africans with diabetes. OBJECTIVE: To perform external quality assurance (EQA) on graders registered in the Ophthalmological Society of South Africa DR screening programme. METHODS: Graders registered on the South African (SA) Diabetic Register website were invited to participate in the study. The Scottish EQA software system was used to enable on-line grading of 100 retinal photographs. Expert National Health Service graders provided the consensus expert grading for the image set. RESULTS: Two hundred and sixty-one participants completed the EQA process, including nine ophthalmologists, 243 optometrists, and nine other graders. A wide range of outcomes were demonstrated, with a mean sensitivity of 0.905 (range 0.286 - 1.000) and mean specificity of 0.507 (0.000 - 0.935). The mean diagnostic odds ratio was calculated to be 12.3 (range 0.147 - 148.2). CONCLUSIONS: This is the first quality assurance study conducted with SA healthcare professionals. The outcomes are of interest to all stakeholders dealing with the diabetes epidemic. The disparity in grader performance indicates room for improvement. The results demonstrate a high referral rate to ophthalmology, suggesting that on average graders are performing safely, but with a high number of inappropriate referrals.

Automated clarity assessment of retinal images using regionally based structural and statistical measures.

An automated image analysis system for application in mass medical screening must assess the clarity of the images before analysing their content. This is the case in grading for diabetic retinopathy screening where the failure to assess clarity could result in retinal images of people with retinopathy being erroneously classed as normal. This paper compares methods of clarity assessment based on the degradation of visible structures and based on the deviation of image properties outside expected norms caused by clarity loss. Vessel visibility measures and statistical measures were determined at locations in the image which have high saliency and these were used to obtain an image clarity assessment using supervised classification. The usefulness of the measures as indicators of image clarity was assessed. Tests were performed on 987 disc-centred and macula-centred retinal photographs (347 with inadequate clarity) obtained from the English National Screening Programme. Images with inadequate clarity were detected with 92.6% sensitivity at 90% specificity. In a set of 2000 macula-centred images (200 with inadequate clarity) from the Scottish Screening Programme, inadequate clarity was detected with 96.7% sensitivity at 90% specificity. This study has shown that structural and statistical measures are equally useful for retinal image clarity assessment.

The evidence for automated grading in diabetic retinopathy screening.

Systematic screening for diabetic retinopathy using retinal photography has been shown to reduce the incidence of blindness among people with diabetes. The implementation of diabetic retinopathy screening programmes faces several challenges. Consequently, methods for improving the efficiency of screening are being sought, one of which is the automation of image grading involving detection of images with either disease or of inadequate quality using computer software. This review aims to bring together the available evidence that is suitable for making a judgement about whether automated grading systems could be used effectively in diabetic retinopathy screening. To do this, it focuses on studies made by the few centres who have presented results tests of automated grading software on large sets of patients or screening episodes. It also considers economic model analyses and papers describing the effectiveness of manual grading in order that the effect of replacing stages of manual grading by automated grading can be judged. In conclusion, the review shows that there is sufficient evidence to suggest that automated grading, operating as a disease / no disease grader, is safe and could reduce the workload of manual grading in diabetic retinopathy screening.

Detection of new vessels on the optic disc using retinal photographs.

Proliferative diabetic retinopathy is a rare condition likely to lead to severe visual impairment. It is characterized by the development of abnormal new retinal vessels. We describe a method for automatically detecting new vessels on the optic disc using retinal photography. Vessel-like candidate segments are first detected using a method based on watershed lines and ridge strength measurement. Fifteen feature parameters, associated with shape, position, orientation, brightness, contrast and line density are calculated for each candidate segment. Based on these features, each segment is categorized as normal or abnormal using a support vector machine (SVM) classifier. The system was trained and tested by cross-validation using 38 images with new vessels and 71 normal images from two diabetic retinal screening centers and one hospital eye clinic. The discrimination performance of the fifteen features was tested against a clinical reference standard. Fourteen features were found to be effective and used in the final test. The area under the receiver operator characteristic curve was 0.911 for detecting images with new vessels on the disc. This accuracy may be sufficient for it to play a useful clinical role in an automated retinopathy analysis system.

Automated grading for diabetic retinopathy: a large-scale audit using arbitration by clinical experts.

BACKGROUND/AIMS: Automated grading software has the potential to reduce the manual grading workload within diabetic retinopathy screening programmes. This audit was undertaken at the request of Scotland's National Diabetic Retinopathy Screening Collaborative to assess whether the introduction of automated grading software into the national screening programme would be safe, robust and effective. METHODS: Automated grading, performed by software for image quality assessment and for microaneurysm/dot haemorrhage detection, was carried out on 78,601 images, obtained from 33,535 consecutive patients, which had been manually graded at one of two regional diabetic retinopathy screening programmes. Cases where the automated grading software assessment indicated gradable images with no disease but the screening programme indicated ungradable images or disease more severe than mild retinopathy were arbitrated by seven senior ophthalmologists. RESULTS: 100% (180/180) of patients with proliferative retinopathy, 100% (324/324) with referable background retinopathy, 100% (193/193) with observable background retinopathy, 97.3% (1099/1130) with referable maculopathy, 99.2% (384/387) with observable maculopathy and 99.8% (1824/1827) with ungradable images were detected by the software. CONCLUSION: The automated grading software operated to previously published results when applied to a large, unselected population attending two regional screening programmes. Manual grading workload reduction would be 36.3%.

The role of haemorrhage and exudate detection in automated grading of diabetic retinopathy.

BACKGROUND/AIMS: Automated grading has the potential to improve the efficiency of diabetic retinopathy screening services. While disease/no disease grading can be performed using only microaneurysm detection and image-quality assessment, automated recognition of other types of lesions may be advantageous. This study investigated whether inclusion of automated recognition of exudates and haemorrhages improves the detection of observable/referable diabetic retinopathy. METHODS: Images from 1253 patients with observable/referable retinopathy and 6333 patients with non-referable retinopathy were obtained from three grading centres. All images were reference-graded, and automated disease/no disease assessments were made based on microaneurysm detection and combined microaneurysm, exudate and haemorrhage detection. RESULTS: Introduction of algorithms for exudates and haemorrhages resulted in a statistically significant increase in the sensitivity for detection of observable/referable retinopathy from 94.9% (95% CI 93.5 to 96.0) to 96.6% (95.4 to 97.4) without affecting manual grading workload. CONCLUSION: Automated detection of exudates and haemorrhages improved the detection of observable/referable retinopathy.

Automated detection of exudates for diabetic retinopathy screening.

Automated image analysis is being widely sought to reduce the workload required for grading images resulting from diabetic retinopathy screening programmes. The recognition of exudates in retinal images is an important goal for automated analysis since these are one of the indicators that the disease has progressed to a stage requiring referral to an ophthalmologist. Candidate exudates were detected using a multi-scale morphological process. Based on local properties, the likelihoods of a candidate being a member of classes exudate, drusen or background were determined. This leads to a likelihood of the image containing exudates which can be thresholded to create a binary decision. Compared to a clinical reference standard, images containing exudates were detected with sensitivity 95.0% and specificity 84.6% in a test set of 13,219 images of which 300 contained exudates. Depending on requirements, this method could form part of an automated system to detect images showing either any diabetic retinopathy or referable diabetic retinopathy.

Automatic detection of retinal anatomy to assist diabetic retinopathy screening.

Screening programmes for diabetic retinopathy are being introduced in the United Kingdom and elsewhere. These require large numbers of retinal images to be manually graded for the presence of disease. Automation of image grading would have a number of benefits. However, an important prerequisite for automation is the accurate location of the main anatomical features in the image, notably the optic disc and the fovea. The locations of these features are necessary so that lesion significance, image field of view and image clarity can be assessed. This paper describes methods for the robust location of the optic disc and fovea. The elliptical form of the major retinal blood vessels is used to obtain approximate locations, which are refined based on the circular edge of the optic disc and the local darkening at the fovea. The methods have been tested on 1056 sequential images from a retinal screening programme. Positional accuracy was better than 0.5 of a disc diameter in 98.4% of cases for optic disc location, and in 96.5% of cases for fovea location. The methods are sufficiently accurate to form an important and effective component of an automated image grading system for diabetic retinopathy screening.

Automated microaneurysm detection using local contrast normalization and local vessel detection.

Screening programs using retinal photography for the detection of diabetic eye disease are being introduced in the UK and elsewhere. Automatic grading of the images is being considered by health boards so that the human grading task is reduced. Microaneurysms (MAs) are the earliest sign of this disease and so are very important for classifying whether images show signs of retinopathy. This paper describes automatic methods for MA detection and shows how image contrast normalization can improve the ability to distinguish between MAs and other dots that occur on the retina. Various methods for contrast normalization are compared. Best results were obtained with a method that uses the watershed transform to derive a region that contains no vessels or other lesions. Dots within vessels are handled successfully using a local vessel detection technique. Results are presented for detection of individual MAs and for detection of images containing MAs. Images containing MAs are detected with sensitivity 85.4% and specificity 83.1%.

Automated assessment of diabetic retinal image quality based on clarity and field definition.

PURPOSE: To evaluate the performance of an automated retinal image quality assessment system for use in automated diabetic retinopathy grading. METHODS: Algorithmic methods have been developed for assessing the quality of 45 degrees single field retinal images for use in diabetic retinopathy screening. For this purpose, image quality was defined by two aspects: image clarity and field definition. An image with adequate clarity was defined as one that shows sufficient detail for automated retinopathy grading. The visibility of the macular vessels was used as an indicator of image clarity, since these vessels are known to be narrow and become less visible with any image degradation. An image with adequate field definition was defined as one that shows the desired field of view for retinopathy grading, including the full 45 degrees field of view, the optic disc, and at least two optic disc diameters of visible retina around the fovea. From 489 patients attending a diabetic retinopathy screening program, 1039 retinal images were obtained. The images were graded by a clinician for image clarity and field definition, with a comprehensive image-quality grading scheme. RESULTS: The sensitivity and specificity were, respectively, 100% and 90.9% for inadequate clarity detection, 95.3% and 96.4% for inadequate field definition detection, and 99.1% and 89.4% for inadequate overall quality detection. CONCLUSIONS: The automated system performs with sufficient accuracy to form part of an automated diabetic retinopathy grading system.

Reduction in shear stress, activation of the endothelium, and leukocyte priming are all required for leukocyte passage across the blood--retina barrier.

The passage of leukocytes across the blood-retina barrier at the early stages of an inflammatory reaction is influenced by a complex series of interactions about which little is known. In particular, the relationship between hydrodynamic factors, such as shear stress and leukocyte velocity, to the adherence and subsequent extravasation of leukocytes into the retina is unclear. We have used a physiological method, scanning laser ophthalmoscopy, to track labeled leukocytes circulating in the retina, followed by confocal microscopy of retinal flatmounts to detect infiltrating cells at the early stage of experimental autoimmune uveitis. This has shown that retinal vessels are subjected to high shear stress under normal circumstances. During the inflammatory reaction, shear stress in retinal veins is reduced 24 h before leukocyte infiltration. This reduction is negatively correlated with leukocyte rolling and sticking in veins and postcapillary venules, the sites of leukocyte extravasation. Activation of vascular endothelial cells is also a prerequisite for leukocyte rolling and infiltration. In addition, antigen priming of leukocytes is influential at the early stage of inflammation, and this is seen clearly in the reduction in rolling velocity and adherence of the primed leukocytes in activated retinal venules, 9 days postimmunization.

Automated measurement of microaneurysm turnover.

PURPOSE: An automated system for the measurement of microaneurysm (MA) turnover was developed and compared with manual measurement. The system analyses serial fluorescein angiogram (FA) or red-free (RF) fundus images; fluorescein angiography was used in this study because it is the more sensitive test for MAs. Previous studies have shown that the absolute number of MAs observed does not reflect the dynamic temporal nature of the MA population. In this study, almost half of the MAs present at baseline had regressed after a year and been replaced by new lesions elsewhere. METHODS: Two clinical datasets were used to evaluate the performance of the automated turnover measurement system. The first consisted of 10 patients who had two fluorescein angiograms acquired a year apart. These data were analyzed, both manually and using the automated system, to investigate the inter- and intraobserver variations associated with manual measurement and to assess the performance of the automated system. The second dataset contained FAs from a further 25 patients. This dataset was analyzed only with the automated system to investigate some properties of microaneurysm turnover, in particular the differing detection sensitivities of new, static and regressed microaneurysms. RESULTS: Manual measurements exhibited large inter- and intraobserver variation. The sensitivity and specificity of the automated system were similar to those of the human observers. However, the automated measurements were more consistent-an important condition for accurate turnover quantification. Regressed MAs were more difficult to detect reliably than new MAs, which were themselves more difficult to detect reliably than static MAs. CONCLUSIONS: The automated system was shown to be fast, reliable, and repeatable, making it suitable for processing large numbers of images. Performance was similar to that of trained manual observers.

Improved leukocyte tracking in mouse retinal and choroidal circulation.

The purpose of this study is to develop a new method with which to visualize leukocyte dynamics in murine choroidal and retinal circulation. Both pigmented (B10.RIII) and non-pigmented (BALB/c) mice were used in this study. One hundred microl of 0.05% sodium fluorescein was injected via the mice tail vein to outline the vessel, followed by 150 microl (10(7) cells) C-AM labelled leukocytes. Fundus images were obtained with a confocal scanning laser ophthalmoscope. The dynamic image sequences were recorded simultaneously on videotape (S-VHS) and digitally at 25 frames per sec. The digital images were later analysed with a custom-made personal computer-based image analysis system. Both the choroidal and retinal circulation can be visualized in non-pigmented mice in the first few seconds of fluorescein angiography. However, the view of the choroidal and the retinal capillary circulation is soon blurred due to the rapid fluorescein leakage in the choroid. In contrast, in pigmented mice, retinal circulation is clear against the dark background of the choroid, while choroidal circulation is masked behind the pigment epithelial layer and cannot be seen at all. C-AM labelled leukocytes were clearly seen in the retinal circulation of all experimental mice and in the choroidal circulation of non-pigmented mice for as long as 30 min. The number of labelled circulating cells decreased as time clasped. Cells moved rapidly in the retinal arteries, slowing down or even stopping for a few seconds in the capillary system, and then moved slightly faster again through the postcapillary venules and veins. In non-pigmented mice, significant number of cells were seen to have arrested in the choroidal circulation. There was no difference between B10.RIII mice and BALB/c mice in vessel diameters, leukocyte velocities and shear stresses. This method allows the visualization of leukocytes and provides data on their behavior as they move through the choroidal and retinal circulation of non-pigmented mice, and in the retinal circulation of pigmented mice. It provides a valuable new tool for the investigation of real time leukocyte dynamics in murine retinal and choroidal microcirculations both under physiological conditions and during the development of ocular disease.