Automated airway quantification associates with mortality in idiopathic pulmonary fibrosis.

OBJECTIVES: The study examined whether quantified airway metrics associate with mortality in idiopathic pulmonary fibrosis (IPF). METHODS: In an observational cohort study (n = 90) of IPF patients from Ege University Hospital, an airway analysis tool AirQuant calculated median airway intersegmental tapering and segmental tortuosity across the 2nd to 6th airway generations. Intersegmental tapering measures the difference in median diameter between adjacent airway segments. Tortuosity evaluates the ratio of measured segmental length against direct end-to-end segmental length. Univariable linear regression analyses examined relationships between AirQuant variables, clinical variables, and lung function tests. Univariable and multivariable Cox proportional hazards models estimated mortality risk with the latter adjusted for patient age, gender, smoking status, antifibrotic use, CT usual interstitial pneumonia (UIP) pattern, and either forced vital capacity (FVC) or diffusion capacity of carbon monoxide (DLco) if obtained within 3 months of the CT. RESULTS: No significant collinearity existed between AirQuant variables and clinical or functional variables. On univariable Cox analyses, male gender, smoking history, no antifibrotic use, reduced DLco, reduced intersegmental tapering, and increased segmental tortuosity associated with increased risk of death. On multivariable Cox analyses (adjusted using FVC), intersegmental tapering (hazard ratio (HR) = 0.75, 95% CI = 0.66-0.85, p < 0.001) and segmental tortuosity (HR = 1.74, 95% CI = 1.22-2.47, p = 0.002) independently associated with mortality. Results were maintained with adjustment using DLco. CONCLUSIONS: AirQuant generated measures of intersegmental tapering and segmental tortuosity independently associate with mortality in IPF patients. Abnormalities in proximal airway generations, which are not typically considered to be abnormal in IPF, have prognostic value. CLINICAL RELEVANCE STATEMENT: Quantitative measurements of intersegmental tapering and segmental tortuosity, in proximal (second to sixth) generation airway segments, independently associate with mortality in IPF. Automated airway analysis can estimate disease severity, which in IPF is not restricted to the distal airway tree. KEY POINTS: • AirQuant generates measures of intersegmental tapering and segmental tortuosity. • Automated airway quantification associates with mortality in IPF independent of established measures of disease severity. • Automated airway analysis could be used to refine patient selection for therapeutic trials in IPF.

Immunohistochemical biomarker validation in highly selective needle biopsy microarrays derived from mpMRI-characterized prostates.

INTRODUCTION: Diagnosing prostate cancer routinely involves tissue biopsy and increasingly image guided biopsy using multiparametric MRI (mpMRI). Excess tissue after diagnosis can be used for research to improve the diagnostic pathway and the vertical assembly of prostate needle biopsy cores into tissue microarrays (TMAs) allows the parallel immunohistochemical (IHC) validation of cancer biomarkers in routine diagnostic specimens. However, tissue within a biopsy core is often heterogeneous and cancer is not uniformly present, resulting in needle biopsy TMAs that suffer from highly variable cancer detection rates that complicate parallel biomarker validation. MATERIALS AND METHODS: The prostate cores with the highest tumor burden (in terms of Gleason score and/or maximum cancer core length) were obtained from 249 patients in the PICTURE trial who underwent transperineal template prostate mapping (TPM) biopsy at 5 mm intervals preceded by mpMRI. From each core, 2 mm segments containing tumor or benign tissue (as assessed on H&E pathology) were selected, excised and embedded vertically into a new TMA block. TMA sections were then IHC-stained for the routinely used prostate cancer biomarkers PSA, PSMA, AMACR, p63, and MSMB and assessed using the h-score method. H-scores in patient matched malignant and benign tissue were correlated with the Gleason grade of the original core and the MRI Likert score for the sampled prostate area. RESULTS: A total of 2240 TMA cores were stained and IHC h-scores were assigned to 1790. There was a statistically significant difference in h-scores between patient matched malignant and adjacent benign tissue that is independent of Likert score. There was no association between the h-scores and Gleason grade or Likert score within each of the benign or malignant groups. CONCLUSION: The construction of highly selective TMAs from prostate needle biopsy cores is possible. IHC data obtained through this method are highly reliable and can be correlated with imaging. IHC expression patterns for PSA, PSMA, AMACR, p63, and MSMB are distinct in malignant and adjacent benign tissue but did not correlate with mpMRI Likert score.

Accuracy of Transperineal Targeted Prostate Biopsies, Visual Estimation and Image Fusion in Men Needing Repeat Biopsy in the PICTURE Trial.

PURPOSE: We evaluated the detection of clinically significant prostate cancer using magnetic resonance imaging targeted biopsies and compared visual estimation to image fusion targeting in patients requiring repeat prostate biopsies. MATERIALS AND METHODS: The prospective, ethics committee approved PICTURE trial (ClinicalTrials.gov NCT01492270) enrolled 249 consecutive patients from January 11, 2012 to January 29, 2014. Men underwent multiparametric magnetic resonance imaging and were blinded to the results. All underwent transperineal template prostate mapping biopsies. In 200 men with a lesion this was preceded by visual estimation and image fusion targeted biopsies. As the primary study end point clinically significant prostate cancer was defined as Gleason 4 + 3 or greater and/or any grade of cancer with a length of 6 mm or greater. Other definitions of clinically significant prostate cancer were also evaluated. RESULTS: Mean ± SD patient age was 62.6 ± 7 years, median prostate specific antigen was 7.17 ng/ml (IQR 5.25-10.09), mean primary lesion size was 0.37 ± 1.52 cc with a mean of 4.3 ± 2.3 targeted cores per lesion on visual estimation and image fusion combined, and a mean of 48.7 ± 12.3 transperineal template prostate mapping biopsy cores. Transperineal template prostate mapping biopsies detected 97 clinically significant prostate cancers (48.5%) and 85 insignificant cancers (42.5%). Overall multiparametric magnetic resonance imaging targeted biopsies detected 81 clinically significant prostate cancers (40.5%) and 63 insignificant cancers (31.5%). In the 18 cases (9%) of clinically significant prostate cancer on magnetic resonance imaging targeted biopsies were benign or clinically insignificant on transperineal template prostate mapping biopsy. Clinically significant prostate cancer was detected in 34 cases (17%) on transperineal template prostate mapping biopsy but not on magnetic resonance imaging targeted biopsies and approximately half was present in nontargeted areas. Clinically significant prostate cancer was found on visual estimation and image fusion in 53 (31.3%) and 48 (28.4%) of the 169 patients (McNemar test p = 0.5322). Visual estimation missed 23 clinically significant prostate cancers (13.6%) detected by image fusion. Image fusion missed 18 clinically significant prostate cancers (10.8%) detected by visual estimation. CONCLUSIONS: Magnetic resonance imaging targeted biopsies are accurate for detecting clinically significant prostate cancer and reducing the over diagnosis of insignificant cancers. To maximize detection visual estimation as well as image fusion targeted biopsies are required.

The PICTURE study: diagnostic accuracy of multiparametric MRI in men requiring a repeat prostate biopsy.

BACKGROUND: Transrectal prostate biopsy has limited diagnostic accuracy. Prostate Imaging Compared to Transperineal Ultrasound-guided biopsy for significant prostate cancer Risk Evaluation (PICTURE) was a paired-cohort confirmatory study designed to assess diagnostic accuracy of multiparametric magnetic resonance imaging (mpMRI) in men requiring a repeat biopsy. METHODS: All underwent 3 T mpMRI and transperineal template prostate mapping biopsies (TTPM biopsies). Multiparametric MRI was reported using Likert scores and radiologists were blinded to initial biopsies. Men were blinded to mpMRI results. Clinically significant prostate cancer was defined as Gleason ⩾4+3 and/or cancer core length ⩾6 mm. RESULTS: Two hundred and forty-nine had both tests with mean (s.d.) age was 62 (7) years, median (IQR) PSA 6.8 ng ml (4.98-9.50), median (IQR) number of previous biopsies 1 (1-2) and mean (s.d.) gland size 37 ml (15.5). On TTPM biopsies, 103 (41%) had clinically significant prostate cancer. Two hundred and fourteen (86%) had a positive prostate mpMRI using Likert score ⩾3; sensitivity was 97.1% (95% confidence interval (CI): 92-99), specificity 21.9% (15.5-29.5), negative predictive value (NPV) 91.4% (76.9-98.1) and positive predictive value (PPV) 46.7% (35.2-47.8). One hundred and twenty-nine (51.8%) had a positive mpMRI using Likert score ⩾4; sensitivity was 80.6% (71.6-87.7), specificity 68.5% (60.3-75.9), NPV 83.3% (75.4-89.5) and PPV 64.3% (55.4-72.6). CONCLUSIONS: In men advised to have a repeat prostate biopsy, prostate mpMRI could be used to safely avoid a repeat biopsy with high sensitivity for clinically significant cancers. However, such a strategy can miss some significant cancers and overdiagnose insignificant cancers depending on the mpMRI score threshold used to define which men should be biopsied.

Identifying the index lesion with template prostate mapping biopsies.

PURPOSE: The natural history of prostate cancer might be driven by the index lesion. We determined the percent of men in whom the index lesion could be defined using transperineal template prostate mapping biopsies. MATERIALS AND METHODS: Included in study were consecutive men undergoing transperineal template prostate mapping biopsies with biopsies grouped into 20 zones. Men with clinically significant disease in only 1 prostate area were considered to have an identifiable index lesion. We evaluated the impact of using 2 definitions of clinically significant disease (Gleason grade pattern 4 and/or lesion volume 0.5 cc or greater) and 2 clustering rules (stringent and tolerant) to define the index lesion. RESULTS: Included in study were 391 men with a median age of 62 years (IQR 58-67) and a median prostate specific antigen of 6.9 ng/ml (IQR 4.8-10.0). Of the men 269 (69%) were previously diagnosed with prostate cancer. By deploying a median of 1.2 cores per ml (IQR 0.9-1.7) cancer was diagnosed in 82.9% of the men (324 of 391) with a median of 6 positive cores (IQR 2-9), a median maximum cancer core length of 5 mm (IQR 3-8) and a total cancer core length per zone of 7 mm (IQR 3-13). Insignificant disease was found in 26.3% to 42.9% of cases. When a stringent spatial relationship was used to define individual lesions, 44.4% to 54.6% of patients had 1 index lesion and 12.7% to 19.1% had more than 1 area with clinically significant disease. These proportions changed to 46.6% to 59.2% and 10.5% to 14.5%, respectively, when less stringent spatial clustering was applied. CONCLUSIONS: Transperineal template prostate mapping biopsies enable the index lesion to be localized in most men with clinically significant disease. This information may be important to select appropriate candidates for targeted therapy and to plan a tailored treatment strategy in men undergoing radical therapy.

Time conditioning for arbitrary contrast phase generation in interventional computed tomography.

Minimally invasive ablation techniques for renal cancer are becoming more popular due to their low complication rate and rapid recovery period. Despite excellent visualisation, one drawback of the use of computed tomography (CT) in these procedures is the requirement for iodine-based contrast agents, which are associated with adverse reactions and require a higher x-ray dose. The purpose of this work is to examine the use of time information to generate synthetic contrast enhanced images at arbitrary points after contrast agent injection from non-contrast CT images acquired during renal cryoablation cases. To achieve this, we propose a new method of conditioning generative adversarial networks with normalised time stamps and demonstrate that the use of a HyperNetwork is feasible for this task, generating images of competitive quality compared to standard generative modelling techniques. We also show that reducing the receptive field can help tackle challenges in interventional CT data, offering significantly better image quality as well as better performance when generating images for a downstream segmentation task. Lastly, we show that all proposed models are robust enough to perform inference on unseen intra-procedural data, while also improving needle artefacts and generalising contrast enhancement to other clinically relevant regions and features.

Effects on Iron Metabolism and System Xc- /GPX4 Pathway from Hydroquinone Suggest Ferroptosis of Jurkat Cells.

Prolonged exposure to hydroquinone (HQ), a metabolite of benzene, can cause severe haematologic disorders in humans. However, the mechanism is still unclear. In the present study, we investigated whether HQ can induce haematological diseases through ferroptosis, which is another form of cell death apart from apoptosis. The results showed that HQ inhibited the viability of Jurkat cells in a dose-dependent and time-dependent manner. The half inhibitory concentrations (IC50s) of HQ-treated Jurkat cells for 12 h, 24 h and 48 h were 107.16 µmol/L, 33.29 µmol/L, and 14.78 µmol/L. The exposure of Jurkat cells to HQ increased intracellular Fe2+, malondialdehyde (MDA) and lipid reactive oxygen species (ROS) levels and down-regulated glutathione (GSH) levels. We used erastin-treated cells as a positive control and cells treated with HQ combined with deferoxamine mesylate (DFO) and ferrostain-1 (Fer-1)-treated cells as the negative controls. DFO and Fer-1 partially restored the degradation of cell viability and GSH content and the accumulation of Fe2+, MDA and lipid ROS caused by HQ. In addition, we found that cellular mitochondria in the HQ-treated group showed a decrease in volume, an increase in the density of the bilayer membrane and a decrease or disappearance of mitochondrial cristae. Changes in the erastin-treated group were similar to those in the HQ-treated group. We inferred that HQ induces ferroptosis in Jurkat cells. Subsequently, we found that HQ up-regulated the levels of transferrin receptor 1 (TFRC) mRNA and protein expression and down-regulated FTH1, SLC7A11 and synthetic substrate of antioxidant enzyme 4 (GPX4) mRNA levels and protein expression levels. However, the exposure of Jurkat cells to HQ with DFO and Fer-1 alleviated these changes. Notably, the activation of TFRC and the inhibition of FTH1 and System Xc- (cystine-glutamate reverse transporter protein) /GPX4 were associated with HQ-induced ferroptosis. These results provide novel insights into how HQ exacerbates haematopoietic cytotoxicity and provide potential targets for the prevention of HQ-induced diseases.

The distinct roles of reinforcement learning between pre-procedure and intra-procedure planning for prostate biopsy.

PURPOSE: Magnetic resonance (MR) imaging targeted prostate cancer (PCa) biopsy enables precise sampling of MR-detected lesions, establishing its importance in recommended clinical practice. Planning for the ultrasound-guided procedure involves pre-selecting needle sampling positions. However, performing this procedure is subject to a number of factors, including MR-to-ultrasound registration, intra-procedure patient movement and soft tissue motions. When a fixed pre-procedure planning is carried out without intra-procedure adaptation, these factors will lead to sampling errors which could cause false positives and false negatives. Reinforcement learning (RL) has been proposed for procedure plannings on similar applications such as this one, because intelligent agents can be trained for both pre-procedure and intra-procedure planning. However, it is not clear if RL is beneficial when it comes to addressing these intra-procedure errors. METHODS: In this work, we develop and compare imitation learning (IL), supervised by demonstrations of predefined sampling strategy, and RL approaches, under varying degrees of intra-procedure motion and registration error, to represent sources of targeting errors likely to occur in an intra-operative procedure. RESULTS: Based on results using imaging data from 567 PCa patients, we demonstrate the efficacy and value in adopting RL algorithms to provide intelligent intra-procedure action suggestions, compared to IL-based planning supervised by commonly adopted policies. CONCLUSIONS: The improvement in biopsy sampling performance for intra-procedure planning has not been observed in experiments with only pre-procedure planning. These findings suggest a strong role for RL in future prospective studies which adopt intra-procedure planning. Our open source code implementation is available here .

Combiner and HyperCombiner networks: Rules to combine multimodality MR images for prostate cancer localisation.

One of the distinct characteristics of radiologists reading multiparametric prostate MR scans, using reporting systems like PI-RADS v2.1, is to score individual types of MR modalities, including T2-weighted, diffusion-weighted, and dynamic contrast-enhanced, and then combine these image-modality-specific scores using standardised decision rules to predict the likelihood of clinically significant cancer. This work aims to demonstrate that it is feasible for low-dimensional parametric models to model such decision rules in the proposed Combiner networks, without compromising the accuracy of predicting radiologic labels. First, we demonstrate that either a linear mixture model or a nonlinear stacking model is sufficient to model PI-RADS decision rules for localising prostate cancer. Second, parameters of these combining models are proposed as hyperparameters, weighing independent representations of individual image modalities in the Combiner network training, as opposed to end-to-end modality ensemble. A HyperCombiner network is developed to train a single image segmentation network that can be conditioned on these hyperparameters during inference for much-improved efficiency. Experimental results based on 751 cases from 651 patients compare the proposed rule-modelling approaches with other commonly-adopted end-to-end networks, in this downstream application of automating radiologist labelling on multiparametric MR. By acquiring and interpreting the modality combining rules, specifically the linear-weights or odds ratios associated with individual image modalities, three clinical applications are quantitatively presented and contextualised in the prostate cancer segmentation application, including modality availability assessment, importance quantification and rule discovery.

CF-Loss: Clinically-relevant feature optimised loss function for retinal multi-class vessel segmentation and vascular feature measurement.

Characterising clinically-relevant vascular features, such as vessel density and fractal dimension, can benefit biomarker discovery and disease diagnosis for both ophthalmic and systemic diseases. In this work, we explicitly encode vascular features into an end-to-end loss function for multi-class vessel segmentation, categorising pixels into artery, vein, uncertain pixels, and background. This clinically-relevant feature optimised loss function (CF-Loss) regulates networks to segment accurate multi-class vessel maps that produce precise vascular features. Our experiments first verify that CF-Loss significantly improves both multi-class vessel segmentation and vascular feature estimation, with two standard segmentation networks, on three publicly available datasets. We reveal that pixel-based segmentation performance is not always positively correlated with accuracy of vascular features, thus highlighting the importance of optimising vascular features directly via CF-Loss. Finally, we show that improved vascular features from CF-Loss, as biomarkers, can yield quantitative improvements in the prediction of ischaemic stroke, a real-world clinical downstream task. The code is available at https://github.com/rmaphoh/feature-loss.

Expectation maximisation pseudo labels.

In this paper, we study pseudo-labelling. Pseudo-labelling employs raw inferences on unlabelled data as pseudo-labels for self-training. We elucidate the empirical successes of pseudo-labelling by establishing a link between this technique and the Expectation Maximisation algorithm. Through this, we realise that the original pseudo-labelling serves as an empirical estimation of its more comprehensive underlying formulation. Following this insight, we present a full generalisation of pseudo-labels under Bayes' theorem, termed Bayesian Pseudo Labels. Subsequently, we introduce a variational approach to generate these Bayesian Pseudo Labels, involving the learning of a threshold to automatically select high-quality pseudo labels. In the remainder of the paper, we showcase the applications of pseudo-labelling and its generalised form, Bayesian Pseudo-Labelling, in the semi-supervised segmentation of medical images. Specifically, we focus on: (1) 3D binary segmentation of lung vessels from CT volumes; (2) 2D multi-class segmentation of brain tumours from MRI volumes; (3) 3D binary segmentation of whole brain tumours from MRI volumes; and (4) 3D binary segmentation of prostate from MRI volumes. We further demonstrate that pseudo-labels can enhance the robustness of the learned representations. The code is released in the following GitHub repository: https://github.com/moucheng2017/EMSSL.

Active learning using adaptable task-based prioritisation.

Supervised machine learning-based medical image computing applications necessitate expert label curation, while unlabelled image data might be relatively abundant. Active learning methods aim to prioritise a subset of available image data for expert annotation, for label-efficient model training. We develop a controller neural network that measures priority of images in a sequence of batches, as in batch-mode active learning, for multi-class segmentation tasks. The controller is optimised by rewarding positive task-specific performance gain, within a Markov decision process (MDP) environment that also optimises the task predictor. In this work, the task predictor is a segmentation network. A meta-reinforcement learning algorithm is proposed with multiple MDPs, such that the pre-trained controller can be adapted to a new MDP that contains data from different institutes and/or requires segmentation of different organs or structures within the abdomen. We present experimental results using multiple CT datasets from more than one thousand patients, with segmentation tasks of nine different abdominal organs, to demonstrate the efficacy of the learnt prioritisation controller function and its cross-institute and cross-organ adaptability. We show that the proposed adaptable prioritisation metric yields converging segmentation accuracy for a new kidney segmentation task, unseen in training, using between approximately 40% to 60% of labels otherwise required with other heuristic or random prioritisation metrics. For clinical datasets of limited size, the proposed adaptable prioritisation offers a performance improvement of 22.6% and 10.2% in Dice score, for tasks of kidney and liver vessel segmentation, respectively, compared to random prioritisation and alternative active sampling strategies.

Interpolation-split: a data-centric deep learning approach with big interpolated data to boost airway segmentation performance.

The morphology and distribution of airway tree abnormalities enable diagnosis and disease characterisation across a variety of chronic respiratory conditions. In this regard, airway segmentation plays a critical role in the production of the outline of the entire airway tree to enable estimation of disease extent and severity. Furthermore, the segmentation of a complete airway tree is challenging as the intensity, scale/size and shape of airway segments and their walls change across generations. The existing classical techniques either provide an undersegmented or oversegmented airway tree, and manual intervention is required for optimal airway tree segmentation. The recent development of deep learning methods provides a fully automatic way of segmenting airway trees; however, these methods usually require high GPU memory usage and are difficult to implement in low computational resource environments. Therefore, in this study, we propose a data-centric deep learning technique with big interpolated data, Interpolation-Split, to boost the segmentation performance of the airway tree. The proposed technique utilises interpolation and image split to improve data usefulness and quality. Then, an ensemble learning strategy is implemented to aggregate the segmented airway segments at different scales. In terms of average segmentation performance (dice similarity coefficient, DSC), our method (A) achieves 90.55%, 89.52%, and 85.80%; (B) outperforms the baseline models by 2.89%, 3.86%, and 3.87% on average; and (C) produces maximum segmentation performance gain by 14.11%, 9.28%, and 12.70% for individual cases when (1) nnU-Net with instant normalisation and leaky ReLU; (2) nnU-Net with batch normalisation and ReLU; and (3) modified dilated U-Net are used respectively. Our proposed method outperformed the state-of-the-art airway segmentation approaches. Furthermore, our proposed technique has low RAM and GPU memory usage, and it is GPU memory-efficient and highly flexible, enabling it to be deployed on any 2D deep learning model.

Competing for Pixels: A Self-play Algorithm for Weakly-supervised Semantic Segmentation.

Weakly-supervised semantic segmentation (WSSS) methods, reliant on image-level labels indicating object presence, lack explicit correspondence between labels and regions of interest (ROIs), posing a significant challenge. Despite this, WSSS methods have attracted attention due to their much lower annotation costs compared to fully-supervised segmentation. Leveraging reinforcement learning (RL) self-play, we propose a novel WSSS method that gamifies image segmentation of a ROI. We formulate segmentation as a competition between two agents that compete to select ROI-containing patches until exhaustion of all such patches. The score at each time-step, used to compute the reward for agent training, represents likelihood of object presence within the selection, determined by an object presence detector pre-trained using only image-level binary classification labels of object presence. Additionally, we propose a game termination condition that can be called by either side upon exhaustion of all ROI-containing patches, followed by the selection of a final patch from each. Upon termination, the agent is incentivised if ROI-containing patches are exhausted or disincentivised if a ROI-containing patch is found by the competitor. This competitive setup ensures minimisation of over- or under-segmentation, a common problem with WSSS methods. Extensive experimentation across four datasets demonstrates significant performance improvements over recent state-of-the-art methods. Code: https://github.com/s-sd/spurl/tree/main/wss.

Image-directed, tissue-preserving focal therapy of prostate cancer: a feasibility study of a novel deformable magnetic resonance-ultrasound (MR-US) registration system.

OBJECTIVE: To evaluate the feasibility of using computer-assisted, deformable image registration software to enable three-dimensional (3D), multi-parametric (mp) magnetic resonance imaging (MRI)-derived information on tumour location and extent, to inform the planning and conduct of focal high-intensity focused ultrasound (HIFU) therapy. PATIENTS AND METHODS: A nested pilot study of 26 consecutive men with a visible discrete focus on mpMRI, correlating with positive histology on transperineal template mapping biopsy, who underwent focal HIFU (Sonablate 500®) within a prospective, Ethics Committee-approved multicentre trial ('INDEX'). Non-rigid image registration software developed in our institution was used to transfer data on the location and limits of the index lesion as defined by mpMRI. Manual contouring of the prostate capsule and histologically confirmed MR-visible lesion was performed preoperatively by a urologist and uro-radiologist. A deformable patient-specific computer model, which captures the location of the target lesion, was automatically generated for each patient and registered to a 3D transrectal ultrasonography (US) volume using a small number (10-20) of manually defined capsule points. During the focal HIFU, the urologist could add additional sonications after image-registration if it was felt that the original treatment plan did not cover the lesion sufficiently with a margin. RESULTS: Prostate capsule and lesion contouring was achieved in <5 min preoperatively. The mean (range) time taken to register images was 6 (3-16) min. Additional treatment sonications were added in 13 of 26 cases leading to a mean (range) additional treatment time of 45 (9-90) s. CONCLUSION: Non-rigid MR-US registration is feasible, efficient and can locate lesions on US. The process has potential for improved accuracy of focal treatments, and improved diagnostic sampling strategies for prostate cancer. Further work on whether deformable MR-US registration impacts on efficacy is required.

Meta-Learning Initializations for Interactive Medical Image Registration.

We present a meta-learning framework for interactive medical image registration. Our proposed framework comprises three components: a learning-based medical image registration algorithm, a form of user interaction that refines registration at inference, and a meta-learning protocol that learns a rapidly adaptable network initialization. This paper describes a specific algorithm that implements the registration, interaction and meta-learning protocol for our exemplar clinical application: registration of magnetic resonance (MR) imaging to interactively acquired, sparsely-sampled transrectal ultrasound (TRUS) images. Our approach obtains comparable registration error (4.26 mm) to the best-performing non-interactive learning-based 3D-to-3D method (3.97 mm) while requiring only a fraction of the data, and occurring in real-time during acquisition. Applying sparsely sampled data to non-interactive methods yields higher registration errors (6.26 mm), demonstrating the effectiveness of interactive MR-TRUS registration, which may be applied intraoperatively given the real-time nature of the adaptation process.

Multi-phase synthetic contrast enhancement in interventional computed tomography for guiding renal cryotherapy.

PURPOSE: Minimally invasive treatments for renal carcinoma offer a low rate of complications and quick recovery. One drawback of the use of computed tomography (CT) for needle guidance is the use of iodinated contrast agents, which require an increased X-ray dose and can potentially cause adverse reactions. The purpose of this work is to generalise the problem of synthetic contrast enhancement to allow the generation of multiple phases on non-contrast CT data from a real-world, clinical dataset without training multiple convolutional neural networks. METHODS: A framework for switching between contrast phases by conditioning the network on the phase information is proposed and compared with separately trained networks. We then examine how the degree of supervision affects the generated contrast by evaluating three established architectures: U-Net (fully supervised), Pix2Pix (adversarial with supervision), and CycleGAN (fully adversarial). RESULTS: We demonstrate that there is no performance loss when testing the proposed method against separately trained networks. Of the training paradigms investigated, the fully adversarial CycleGAN performs the worst, while the fully supervised U-Net generates more realistic voxel intensities and performed better than Pix2Pix in generating contrast images for use in a downstream segmentation task. Lastly, two models are shown to generalise to intra-procedural data not seen during the training process, also enhancing features such as needles and ice balls relevant to interventional radiological procedures. CONCLUSION: The proposed contrast switching framework is a feasible option for generating multiple contrast phases without the overhead of training multiple neural networks, while also being robust towards unseen data and enhancing contrast in features relevant to clinical practice.

Long-term Dependency for 3D Reconstruction of Freehand Ultrasound Without External Tracker.

OBJECTIVE: Reconstructing freehand ultrasound in 3D without any external tracker has been a long-standing challenge in ultrasound-assisted procedures. We aim to define new ways of parameterising long-term dependencies, and evaluate the performance. METHODS: First, long-term dependency is encoded by transformation positions within a frame sequence. This is achieved by combining a sequence model with a multi-transformation prediction. Second, two dependency factors are proposed, anatomical image content and scanning protocol, for contributing towards accurate reconstruction. Each factor is quantified experimentally by reducing respective training variances. RESULTS: 1) The added long-term dependency up to 400 frames at 20 frames per second (fps) indeed improved reconstruction, with an up to 82.4% lowered accumulated error, compared with the baseline performance. The improvement was found to be dependent on sequence length, transformation interval and scanning protocol and, unexpectedly, not on the use of recurrent networks with long-short term modules; 2) Decreasing either anatomical or protocol variance in training led to poorer reconstruction accuracy. Interestingly, greater performance was gained from representative protocol patterns, than from representative anatomical features. CONCLUSION: The proposed algorithm uses hyperparameter tuning to effectively utilise long-term dependency. The proposed dependency factors are of practical significance in collecting diverse training data, regulating scanning protocols and developing efficient networks. SIGNIFICANCE: The proposed new methodology with publicly available volunteer data and code for parametersing the long-term dependency, experimentally shown to be valid sources of performance improvement, which could potentially lead to better model development and practical optimisation of the reconstruction application.

Delineating COVID-19 subgroups using routine clinical data identifies distinct in-hospital outcomes.

The COVID-19 pandemic has been a great challenge to healthcare systems worldwide. It highlighted the need for robust predictive models which can be readily deployed to uncover heterogeneities in disease course, aid decision-making and prioritise treatment. We adapted an unsupervised data-driven model-SuStaIn, to be utilised for short-term infectious disease like COVID-19, based on 11 commonly recorded clinical measures. We used 1344 patients from the National COVID-19 Chest Imaging Database (NCCID), hospitalised for RT-PCR confirmed COVID-19 disease, splitting them equally into a training and an independent validation cohort. We discovered three COVID-19 subtypes (General Haemodynamic, Renal and Immunological) and introduced disease severity stages, both of which were predictive of distinct risks of in-hospital mortality or escalation of treatment, when analysed using Cox Proportional Hazards models. A low-risk Normal-appearing subtype was also discovered. The model and our full pipeline are available online and can be adapted for future outbreaks of COVID-19 or other infectious disease.

Prostate cancer segmentation from MRI by a multistream fusion encoder.

BACKGROUND: Targeted prostate biopsy guided by multiparametric magnetic resonance imaging (mpMRI) detects more clinically significant lesions than conventional systemic biopsy. Lesion segmentation is required for planning MRI-targeted biopsies. The requirement for integrating image features available in T2-weighted and diffusion-weighted images poses a challenge in prostate lesion segmentation from mpMRI. PURPOSE: A flexible and efficient multistream fusion encoder is proposed in this work to facilitate the multiscale fusion of features from multiple imaging streams. A patch-based loss function is introduced to improve the accuracy in segmenting small lesions. METHODS: The proposed multistream encoder fuses features extracted in the three imaging streams at each layer of the network, thereby allowing improved feature maps to propagate downstream and benefit segmentation performance. The fusion is achieved through a spatial attention map generated by optimally weighting the contribution of the convolution outputs from each stream. This design provides flexibility for the network to highlight image modalities according to their relative influence on the segmentation performance. The encoder also performs multiscale integration by highlighting the input feature maps (low-level features) with the spatial attention maps generated from convolution outputs (high-level features). The Dice similarity coefficient (DSC), serving as a cost function, is less sensitive to incorrect segmentation for small lesions. We address this issue by introducing a patch-based loss function that provides an average of the DSCs obtained from local image patches. This local average DSC is equally sensitive to large and small lesions, as the patch-based DSCs associated with small and large lesions have equal weights in this average DSC. RESULTS: The framework was evaluated in 931 sets of images acquired in several clinical studies at two centers in Hong Kong and the United Kingdom. In particular, the training, validation, and test sets contain 615, 144, and 172 sets of images, respectively. The proposed framework outperformed single-stream networks and three recently proposed multistream networks, attaining F1 scores of 82.2 and 87.6% in the lesion and patient levels, respectively. The average inference time for an axial image was 11.8 ms. CONCLUSION: The accuracy and efficiency afforded by the proposed framework would accelerate the MRI interpretation workflow of MRI-targeted biopsy and focal therapies.