Unsupervised lung CT image registration via stochastic decomposition of deformation fields.

We address the problem of lung CT image registration, which underpins various diagnoses and treatments for lung diseases. The main crux of the problem is the large deformation that the lungs undergo during respiration. This physiological process imposes several challenges from a learning point of view. In this paper, we propose a novel training scheme, called stochastic decomposition, which enables deep networks to effectively learn such a difficult deformation field during lung CT image registration. The key idea is to stochastically decompose the deformation field, and supervise the registration by synthetic data that have the corresponding appearance discrepancy. The stochastic decomposition allows for revealing all possible decompositions of the deformation field. At the learning level, these decompositions can be seen as a prior to reduce the ill-posedness of the registration yielding to boost the performance. We demonstrate the effectiveness of our framework on Lung CT data. We show, through extensive numerical and visual results, that our technique outperforms existing methods.

Deep learning-based diffusion tensor cardiac magnetic resonance reconstruction: a comparison study.

In vivo cardiac diffusion tensor imaging (cDTI) is a promising Magnetic Resonance Imaging (MRI) technique for evaluating the microstructure of myocardial tissue in living hearts, providing insights into cardiac function and enabling the development of innovative therapeutic strategies. However, the integration of cDTI into routine clinical practice poses challenging due to the technical obstacles involved in the acquisition, such as low signal-to-noise ratio and prolonged scanning times. In this study, we investigated and implemented three different types of deep learning-based MRI reconstruction models for cDTI reconstruction. We evaluated the performance of these models based on the reconstruction quality assessment, the diffusion tensor parameter assessment as well as the computational cost assessment. Our results indicate that the models discussed in this study can be applied for clinical use at an acceleration factor (AF) of × 2 and × 4 , with the D5C5 model showing superior fidelity for reconstruction and the SwinMR model providing higher perceptual scores. There is no statistical difference from the reference for all diffusion tensor parameters at AF × 2 or most DT parameters at AF × 4 , and the quality of most diffusion tensor parameter maps is visually acceptable. SwinMR is recommended as the optimal approach for reconstruction at AF × 2 and AF × 4 . However, we believe that the models discussed in this study are not yet ready for clinical use at a higher AF. At AF × 8 , the performance of all models discussed remains limited, with only half of the diffusion tensor parameters being recovered to a level with no statistical difference from the reference. Some diffusion tensor parameter maps even provide wrong and misleading information.

STADNet: Spatial-Temporal Attention-Guided Dual-Path Network for cardiac cine MRI super-resolution.

Cardiac cine magnetic resonance imaging (MRI) is a commonly used clinical tool for evaluating cardiac function and morphology. However, its diagnostic accuracy may be compromised by the low spatial resolution. Current methods for cine MRI super-resolution reconstruction still have limitations. They typically rely on 3D convolutional neural networks or recurrent neural networks, which may not effectively capture long-range or non-local features due to their limited receptive fields. Optical flow estimators are also commonly used to align neighboring frames, which may cause information loss and inaccurate motion estimation. Additionally, pre-warping strategies may involve interpolation, leading to potential loss of texture details and complicated anatomical structures. To overcome these challenges, we propose a novel Spatial-Temporal Attention-Guided Dual-Path Network (STADNet) for cardiac cine MRI super-resolution. We utilize transformers to model long-range dependencies in cardiac cine MR images and design a cross-frame attention module in the location-aware spatial path, which enhances the spatial details of the current frame by using complementary information from neighboring frames. We also introduce a recurrent flow-enhanced attention module in the motion-aware temporal path that exploits the correlation between cine MRI frames and extracts the motion information of the heart. Experimental results demonstrate that STADNet outperforms SOTA approaches and has significant potential for clinical practice.

CoNIC Challenge: Pushing the frontiers of nuclear detection, segmentation, classification and counting.

Nuclear detection, segmentation and morphometric profiling are essential in helping us further understand the relationship between histology and patient outcome. To drive innovation in this area, we setup a community-wide challenge using the largest available dataset of its kind to assess nuclear segmentation and cellular composition. Our challenge, named CoNIC, stimulated the development of reproducible algorithms for cellular recognition with real-time result inspection on public leaderboards. We conducted an extensive post-challenge analysis based on the top-performing models using 1,658 whole-slide images of colon tissue. With around 700 million detected nuclei per model, associated features were used for dysplasia grading and survival analysis, where we demonstrated that the challenge's improvement over the previous state-of-the-art led to significant boosts in downstream performance. Our findings also suggest that eosinophils and neutrophils play an important role in the tumour microevironment. We release challenge models and WSI-level results to foster the development of further methods for biomarker discovery.

Contrastive Registration for Unsupervised Medical Image Segmentation.

Medical image segmentation is an important task in medical imaging, as it serves as the first step for clinical diagnosis and treatment planning. While major success has been reported using deep learning supervised techniques, they assume a large and well-representative labeled set. This is a strong assumption in the medical domain where annotations are expensive, time-consuming, and inherent to human bias. To address this problem, unsupervised segmentation techniques have been proposed in the literature. Yet, none of the existing unsupervised segmentation techniques reach accuracies that come even near to the state-of-the-art of supervised segmentation methods. In this work, we present a novel optimization model framed in a new convolutional neural network (CNN)-based contrastive registration architecture for unsupervised medical image segmentation called CLMorph. The core idea of our approach is to exploit image-level registration and feature-level contrastive learning, to perform registration-based segmentation. First, we propose an architecture to capture the image-to-image transformation mapping via registration for unsupervised medical image segmentation. Second, we embed a contrastive learning mechanism in the registration architecture to enhance the discriminative capacity of the network at the feature level. We show that our proposed CLMorph technique mitigates the major drawbacks of existing unsupervised techniques. We demonstrate, through numerical and visual experiments, that our technique substantially outperforms the current state-of-the-art unsupervised segmentation methods on two major medical image datasets.

Beyond fine-tuning: Classifying high resolution mammograms using function-preserving transformations.

The task of classifying mammograms is very challenging because the lesion is usually small in the high resolution image. The current state-of-the-art approaches for medical image classification rely on using the de-facto method for convolutional neural networks-fine-tuning. However, there are fundamental differences between natural images and medical images, which based on existing evidence from the literature, limits the overall performance gain when designed with algorithmic approaches. In this paper, we propose to go beyond fine-tuning by introducing a novel framework called MorphHR, in which we highlight a new transfer learning scheme. The idea behind the proposed framework is to integrate function-preserving transformations, for any continuous non-linear activation neurons, to internally regularise the network for improving mammograms classification. The proposed solution offers two major advantages over the existing techniques. Firstly and unlike fine-tuning, the proposed approach allows for modifying not only the last few layers but also several of the first ones on a deep ConvNet. By doing this, we can design the network front to be suitable for learning domain specific features. Secondly, the proposed scheme is scalable to hardware. Therefore, one can fit high resolution images on standard GPU memory. We show that by using high resolution images, one prevents losing relevant information. We demonstrate, through numerical and visual experiments, that the proposed approach yields to a significant improvement in the classification performance over state-of-the-art techniques, and is indeed on a par with radiology experts. Moreover and for generalisation purposes, we show the effectiveness of the proposed learning scheme on another large dataset, the ChestX-ray14, surpassing current state-of-the-art techniques.

LaplaceNet: A Hybrid Graph-Energy Neural Network for Deep Semisupervised Classification.

Semisupervised learning (SSL) has received a lot of recent attention as it alleviates the need for large amounts of labeled data which can often be expensive, requires expert knowledge, and be time consuming to collect. Recent developments in deep semisupervised classification have reached unprecedented performance and the gap between supervised and SSL is ever-decreasing. This improvement in performance has been based on the inclusion of numerous technical tricks, strong augmentation techniques, and costly optimization schemes with multiterm loss functions. We propose a new framework, LaplaceNet, for deep semisupervised classification that has a greatly reduced model complexity. We utilize a hybrid approach where pseudolabels are produced by minimizing the Laplacian energy on a graph. These pseudolabels are then used to iteratively train a neural-network backbone. Our model outperforms state-of-the-art methods for deep semisupervised classification, over several benchmark datasets. Furthermore, we consider the application of strong augmentations to neural networks theoretically and justify the use of a multisampling approach for SSL. We demonstrate, through rigorous experimentation, that a multisampling augmentation approach improves generalization and reduces the sensitivity of the network to augmentation.

A Three-Stage Self-Training Framework for Semi-Supervised Semantic Segmentation.

Semantic segmentation has been widely investigated in the community, in which state-of-the-art techniques are based on supervised models. Those models have reported unprecedented performance at the cost of requiring a large set of high quality segmentation masks for training. Obtaining such annotations is highly expensive and time consuming, in particular, in semantic segmentation where pixel-level annotations are required. In this work, we address this problem by proposing a holistic solution framed as a self-training framework for semi-supervised semantic segmentation. The key idea of our technique is the extraction of the pseudo-mask information on unlabelled data whilst enforcing segmentation consistency in a multi-task fashion. We achieve this through a three-stage solution. Firstly, a segmentation network is trained using the labelled data only and rough pseudo-masks are generated for all images. Secondly, we decrease the uncertainty of the pseudo-mask by using a multi-task model that enforces consistency and that exploits the rich statistical information of the data. Finally, the segmentation model is trained by taking into account the information of the higher quality pseudo-masks. We compare our approach against existing semi-supervised semantic segmentation methods and demonstrate state-of-the-art performance with extensive experiments.

Machine Learning for Workflow Applications in Screening Mammography: Systematic Review and Meta-Analysis.

Background Advances in computer processing and improvements in data availability have led to the development of machine learning (ML) techniques for mammographic imaging. Purpose To evaluate the reported performance of stand-alone ML applications for screening mammography workflow. Materials and Methods Ovid Embase, Ovid Medline, Cochrane Central Register of Controlled Trials, Scopus, and Web of Science literature databases were searched for relevant studies published from January 2012 to September 2020. The study was registered with the PROSPERO International Prospective Register of Systematic Reviews (protocol no. CRD42019156016). Stand-alone technology was defined as a ML algorithm that can be used independently of a human reader. Studies were quality assessed using the Quality Assessment of Diagnostic Accuracy Studies 2 and the Prediction Model Risk of Bias Assessment Tool, and reporting was evaluated using the Checklist for Artificial Intelligence in Medical Imaging. A primary meta-analysis included the top-performing algorithm and corresponding reader performance from which pooled summary estimates for the area under the receiver operating characteristic curve (AUC) were calculated using a bivariate model. Results Fourteen articles were included, which detailed 15 studies for stand-alone detection (n = 8) and triage (n = 7). Triage studies reported that 17%-91% of normal mammograms identified could be read by adapted screening, while "missing" an estimated 0%-7% of cancers. In total, an estimated 185 252 cases from three countries with more than 39 readers were included in the primary meta-analysis. The pooled sensitivity, specificity, and AUC was 75.4% (95% CI: 65.6, 83.2; P = .11), 90.6% (95% CI: 82.9, 95.0; P = .40), and 0.89 (95% CI: 0.84, 0.98), respectively, for algorithms, and 73.0% (95% CI: 60.7, 82.6), 88.6% (95% CI: 72.4, 95.8), and 0.85 (95% CI: 0.78, 0.97), respectively, for readers. Conclusion Machine learning (ML) algorithms that demonstrate a stand-alone application in mammographic screening workflows achieve or even exceed human reader detection performance and improve efficiency. However, this evidence is from a small number of retrospective studies. Therefore, further rigorous independent external prospective testing of ML algorithms to assess performance at preassigned thresholds is required to support these claims. ©RSNA, 2021 Online supplemental material is available for this article. See also the editorial by Whitman and Moseley in this issue.

GraphXCOVID: Explainable deep graph diffusion pseudo-Labelling for identifying COVID-19 on chest X-rays.

Can one learn to diagnose COVID-19 under extreme minimal supervision? Since the outbreak of the novel COVID-19 there has been a rush for developing automatic techniques for expert-level disease identification on Chest X-ray data. In particular, the use of deep supervised learning has become the go-to paradigm. However, the performance of such models is heavily dependent on the availability of a large and representative labelled dataset. The creation of which is a heavily expensive and time consuming task, and especially imposes a great challenge for a novel disease. Semi-supervised learning has shown the ability to match the incredible performance of supervised models whilst requiring a small fraction of the labelled examples. This makes the semi supervised paradigm an attractive option for identifying COVID-19. In this work, we introduce a graph based deep semi-supervised framework for classifying COVID-19 from chest X-rays. Our framework introduces an optimisation model for graph diffusion that reinforces the natural relation among the tiny labelled set and the vast unlabelled data. We then connect the diffusion prediction output as pseudo-labels that are used in an iterative scheme in a deep net. We demonstrate, through our experiments, that our model is able to outperform the current leading supervised model with a tiny fraction of the labelled examples. Finally, we provide attention maps to accommodate the radiologist's mental model, better fitting their perceptual and cognitive abilities. These visualisation aims to assist the radiologist in judging whether the diagnostic is correct or not, and in consequence to accelerate the decision.

Compressed sensing plus motion (CS + M): A new perspective for improving undersampled MR image reconstruction.

We address the problem of reconstructing high quality images from undersampled MRI data. This is a challenging task due to the highly ill-posed nature of the problem. In particular, in dynamic MRI scans, the interaction between the target structure and the physical motion affects the acquired measurements leading to blurring artefacts and loss of fine details. In this work, we propose a framework for dynamic MRI reconstruction framed under a new multi-task optimisation model called Compressed Sensing Plus Motion (CS + M). Firstly, we propose a single optimisation problem that simultaneously computes the MRI reconstruction and the physical motion. Secondly, we show our model can be efficiently solved by breaking it up into two computationally tractable problems. The potentials and generalisation capabilities of our approach are demonstrated in different clinical applications including cardiac cine, cardiac perfusion and brain perfusion imaging. We show, through numerical experiments, that the proposed scheme reduces blurring artefacts, and preserves the target shape and fine details in the reconstruction. We also report the highest quality reconstruction under high undersampling rates in comparison to several state of the art techniques.

Rethinking medical image reconstruction via shape prior, going deeper and faster: Deep joint indirect registration and reconstruction.

Indirect image registration is a promising technique to improve image reconstruction quality by providing a shape prior for the reconstruction task. In this paper, we propose a novel hybrid method that seeks to reconstruct high quality images from few measurements whilst requiring low computational cost. With this purpose, our framework intertwines indirect registration and reconstruction tasks is a single functional. It is based on two major novelties. Firstly, we introduce a model based on deep nets to solve the indirect registration problem, in which the inversion and registration mappings are recurrently connected through a fixed-point interaction based sparse optimisation. Secondly, we introduce specific inversion blocks, that use the explicit physical forward operator, to map the acquired measurements to the image reconstruction. We also introduce registration blocks based deep nets to predict the registration parameters and warp transformation accurately and efficiently. We demonstrate, through extensive numerical and visual experiments, that our framework outperforms significantly classic reconstruction schemes and other bi-task method; this in terms of both image quality and computational time. Finally, we show generalisation capabilities of our approach by demonstrating their performance on fast Magnetic Resonance Imaging (MRI), sparse view computed tomography (CT) and low dose CT with measurements much below the Nyquist limit.

Mirror, Mirror, on the Wall, Who's Got the Clearest Image of Them All?-A Tailored Approach to Single Image Reflection Removal.

Removing reflection artefacts from a single image is a problem of both theoretical and practical interest, which still presents challenges because of the massively ill-posed nature of the problem. In this paper, we propose a technique based on a novel optimization problem. First, we introduce a simple user interaction scheme, which helps minimize information loss in the reflection-free regions. Second, we introduce an H2 fidelity term, which preserves fine detail while enforcing the global color similarity. We show that this combination allows us to mitigate the shortcomings in structure and color preservation, which presents some of the most prominent drawbacks in the existing methods for reflection removal. We demonstrate, through numerical and visual experiments, that our method is able to outperform the state-of-the-art model-based methods and compete with recent deep-learning approaches.

ReTouchImg: Fusioning from-local-to-global context detection and graph data structures for fully-automatic specular reflection removal for endoscopic images.

Minimally invasive surgical and diagnostic systems are commonly used in clinical practices. However, the accuracy and robustness of these systems depend heavily on computer based processes such as tracking, detecting or segmenting clinically meaningful regions of interest, which are significantly affected by the inherent specular reflections that appear on the organs' surfaces. Restoration of the acquired data for clinical purposes still presents challenges because of the high texture and color variations across the image. In this work, we propose a novel fully-automated solution for endoscopic image restoration, which we call ReTouchImg. Our approach is designed as a two-step scheme. The first is a detection step that is based on the synergy of a set of color variations and gradient information conditions. For the second step, we introduce an inpainting process which is based on graph data structures for recovering the missing information. We exhaustively evaluate our approach on real endoscopic datasets and compare it against some works from the body of literature. We also demonstrate that our solution deals with complex cases such as strong illumination variation and large affected areas through a careful quantitative evaluation of a range of numerical results.

Sliding to predict: vision-based beating heart motion estimation by modeling temporal interactions.

PURPOSE: Technical advancements have been part of modern medical solutions as they promote better surgical alternatives that serve to the benefit of patients. Particularly with cardiovascular surgeries, robotic surgical systems enable surgeons to perform delicate procedures on a beating heart, avoiding the complications of cardiac arrest. This advantage comes with the price of having to deal with a dynamic target which presents technical challenges for the surgical system. In this work, we propose a solution for cardiac motion estimation. METHODS: Our estimation approach uses a variational framework that guarantees preservation of the complex anatomy of the heart. An advantage of our approach is that it takes into account different disturbances, such as specular reflections and occlusion events. This is achieved by performing a preprocessing step that eliminates the specular highlights and a predicting step, based on a conditional restricted Boltzmann machine, that recovers missing information caused by partial occlusions. RESULTS: We carried out exhaustive experimentations on two datasets, one from a phantom and the other from an in vivo procedure. The results show that our visual approach reaches an average minima in the order of magnitude of [Formula: see text] while preserving the heart's anatomical structure and providing stable values for the Jacobian determinant ranging from 0.917 to 1.015. We also show that our specular elimination approach reaches an accuracy of 99% compared to a ground truth. In terms of prediction, our approach compared favorably against two well-known predictors, NARX and EKF, giving the lowest average RMSE of 0.071. CONCLUSION: Our approach avoids the risks of using mechanical stabilizers and can also be effective for acquiring the motion of organs other than the heart, such as the lung or other deformable objects.