Generating multi-pathological and multi-modal images and labels for brain MRI.

The last few years have seen a boom in using generative models to augment real datasets, as synthetic data can effectively model real data distributions and provide privacy-preserving, shareable datasets that can be used to train deep learning models. However, most of these methods are 2D and provide synthetic datasets that come, at most, with categorical annotations. The generation of paired images and segmentation samples that can be used in downstream, supervised segmentation tasks remains fairly uncharted territory. This work proposes a two-stage generative model capable of producing 2D and 3D semantic label maps and corresponding multi-modal images. We use a latent diffusion model for label synthesis and a VAE-GAN for semantic image synthesis. Synthetic datasets provided by this model are shown to work in a wide variety of segmentation tasks, supporting small, real datasets or fully replacing them while maintaining good performance. We also demonstrate its ability to improve downstream performance on out-of-distribution data.

Enhancing surgical instrument segmentation: integrating vision transformer insights with adapter.

PURPOSE: In surgical image segmentation, a major challenge is the extensive time and resources required to gather large-scale annotated datasets. Given the scarcity of annotated data in this field, our work aims to develop a model that achieves competitive performance with training on limited datasets, while also enhancing model robustness in various surgical scenarios. METHODS: We propose a method that harnesses the strengths of pre-trained Vision Transformers (ViTs) and data efficiency of convolutional neural networks (CNNs). Specifically, we demonstrate how a CNN segmentation model can be used as a lightweight adapter for a frozen ViT feature encoder. Our novel feature adapter uses cross-attention modules that merge the multiscale features derived from the CNN encoder with feature embeddings from ViT, ensuring integration of the global insights from ViT along with local information from CNN. RESULTS: Extensive experiments demonstrate our method outperforms current models in surgical instrument segmentation. Specifically, it achieves superior performance in binary segmentation on the Robust-MIS 2019 dataset, as well as in multiclass segmentation tasks on the EndoVis 2017 and EndoVis 2018 datasets. It also showcases remarkable robustness through cross-dataset validation across these 3 datasets, along with the CholecSeg8k and AutoLaparo datasets. Ablation studies based on the datasets prove the efficacy of our novel adapter module. CONCLUSION: In this study, we presented a novel approach integrating ViT and CNN. Our unique feature adapter successfully combines the global insights of ViT with the local, multi-scale spatial capabilities of CNN. This integration effectively overcomes data limitations in surgical instrument segmentation. The source code is available at: https://github.com/weimengmeng1999/AdapterSIS.git .

Exclusion of non-Involved uterus from the target volume (EXIT-trial): An individualized treatment for locally advanced cervical cancer using modern radiotherapy and imaging techniques followed by completion surgery.

Background and purpose: Chemoradiotherapy followed by brachytherapy is the standard of care for locally advanced cervical cancer (LACC). In this study, we postulate that omitting an iconographical unaffected uterus (+12 mm distance from the tumour) from the treatment volume is safe and that no tumour will be found in the non-targeted uterus (NTU) leading to reduction of high-dose volumes of surrounding organs at risk (OARs). Material and Methods: In this single-arm phase 2 study, two sets of target volumes were delineated: one standard-volume (whole uterus) and an EXIT-volume (exclusion of non-tumour-bearing parts of the uterus with a minimum 12 mm margin from the tumour). All patients underwent chemoradiotherapy targeting the EXIT-volume, followed by completion hysterectomy. In 15 patients, a plan comparison between two treatment plans (PTV vs PTV_EXIT) was performed. The primary endpoint was the pathological absence of tumour involvement in the non-targeted uterus (NTU). Secondary endpoints included dosimetric impact of target volume reduction on OARs, acute and chronic toxicity, overall survival (OS), locoregional recurrence-free survival (LRFS), and progression-free survival (PFS). Results: In all 21 (FIGO stage I: 2; II: 14;III: 3; IV: 2) patients the NTU was pathologically negative. Ssignificant reductions in Dmean in bladder, sigmoid and rectum; V15Gy in sigmoid and rectum, V30Gy in bladder, sigmoid and rectum; V40Gy and V45Gy in bladder, bowel bag, sigmoid and rectum; V50Gy in rectum were achieved. Median follow-up was 54 months (range 7-79 months). Acute toxicity was mainly grade 2 and 5 % grade 3 urinary. The 3y- OS, PFS and LRFS were respectively 76,2%, 64,9% and 81 %. Conclusion: MRI-based exclusion of the non-tumour-bearing parts of the uterus at a minimum distance of 12 mm from the tumour out of the target volume in LACC can be done without risk of residual disease in the NTU, leading to a significant reduction of the volume of surrounding OARS treated to high doses.

MONAI Label: A framework for AI-assisted interactive labeling of 3D medical images.

The lack of annotated datasets is a major bottleneck for training new task-specific supervised machine learning models, considering that manual annotation is extremely expensive and time-consuming. To address this problem, we present MONAI Label, a free and open-source framework that facilitates the development of applications based on artificial intelligence (AI) models that aim at reducing the time required to annotate radiology datasets. Through MONAI Label, researchers can develop AI annotation applications focusing on their domain of expertise. It allows researchers to readily deploy their apps as services, which can be made available to clinicians via their preferred user interface. Currently, MONAI Label readily supports locally installed (3D Slicer) and web-based (OHIF) frontends and offers two active learning strategies to facilitate and speed up the training of segmentation algorithms. MONAI Label allows researchers to make incremental improvements to their AI-based annotation application by making them available to other researchers and clinicians alike. Additionally, MONAI Label provides sample AI-based interactive and non-interactive labeling applications, that can be used directly off the shelf, as plug-and-play to any given dataset. Significant reduced annotation times using the interactive model can be observed on two public datasets.

Deep learning for automatic segmentation of vestibular schwannoma: a retrospective study from multi-center routine MRI.

Automatic segmentation of vestibular schwannoma (VS) from routine clinical MRI has potential to improve clinical workflow, facilitate treatment decisions, and assist patient management. Previous work demonstrated reliable automatic segmentation performance on datasets of standardized MRI images acquired for stereotactic surgery planning. However, diagnostic clinical datasets are generally more diverse and pose a larger challenge to automatic segmentation algorithms, especially when post-operative images are included. In this work, we show for the first time that automatic segmentation of VS on routine MRI datasets is also possible with high accuracy. We acquired and publicly release a curated multi-center routine clinical (MC-RC) dataset of 160 patients with a single sporadic VS. For each patient up to three longitudinal MRI exams with contrast-enhanced T1-weighted (ceT1w) (n = 124) and T2-weighted (T2w) (n = 363) images were included and the VS manually annotated. Segmentations were produced and verified in an iterative process: (1) initial segmentations by a specialized company; (2) review by one of three trained radiologists; and (3) validation by an expert team. Inter- and intra-observer reliability experiments were performed on a subset of the dataset. A state-of-the-art deep learning framework was used to train segmentation models for VS. Model performance was evaluated on a MC-RC hold-out testing set, another public VS datasets, and a partially public dataset. The generalizability and robustness of the VS deep learning segmentation models increased significantly when trained on the MC-RC dataset. Dice similarity coefficients (DSC) achieved by our model are comparable to those achieved by trained radiologists in the inter-observer experiment. On the MC-RC testing set, median DSCs were 86.2(9.5) for ceT1w, 89.4(7.0) for T2w, and 86.4(8.6) for combined ceT1w+T2w input images. On another public dataset acquired for Gamma Knife stereotactic radiosurgery our model achieved median DSCs of 95.3(2.9), 92.8(3.8), and 95.5(3.3), respectively. In contrast, models trained on the Gamma Knife dataset did not generalize well as illustrated by significant underperformance on the MC-RC routine MRI dataset, highlighting the importance of data variability in the development of robust VS segmentation models. The MC-RC dataset and all trained deep learning models were made available online.

A comparative study of analytical models of diffuse reflectance in homogeneous biological tissues: Gelatin-based phantoms and Monte Carlo experiments.

Information about tissue oxygen saturation (StO2) and other related important physiological parameters can be extracted from diffuse reflectance spectra measured through non-contact imaging. Three analytical optical reflectance models for homogeneous, semi-infinite, tissue have been proposed (Modified Beer-Lambert, Jacques 1999, Yudovsky 2009) but these have not been directly compared for tissue parameter extraction purposes. We compare these analytical models using Monte Carlo (MC) simulated diffuse reflectance spectra and controlled gelatin-based phantoms with measured diffuse reflectance spectra and known ground truth composition parameters. The Yudovsky model performed best against MC simulations and measured spectra of tissue phantoms in terms of goodness of fit and parameter extraction accuracy followed closely by Jacques' model. In this study, Yudovsky's model appeared most robust; however, our results demonstrated that both Yudovsky and Jacques models are suitable for modeling tissue that can be approximated as a single, homogeneous, semi-infinite slab.

Disease Control and Late Toxicity in Adaptive Dose Painting by Numbers Versus Nonadaptive Radiation Therapy for Head and Neck Cancer: A Randomized Controlled Phase 2 Trial.

PURPOSE: Local recurrence remains the main cause of death in stage III-IV nonmetastatic head and neck cancer (HNC), with relapse-prone regions within high 18F-fluorodeoxyglucose positron emission tomography (18F-FDG-PET)-signal gross tumor volume. We investigated if dose escalation within this subvolume combined with a 3-phase treatment adaptation could increase local (LC) and regional (RC) control at equal or minimized radiation-induced toxicity, by comparing adaptive 18F-FDG-PET voxel intensity-based dose painting by numbers (A-DPBN) with nonadaptive standard intensity modulated radiation therapy (S-IMRT). METHODS AND MATERIALS: This 2-center randomized controlled phase 2 trial assigned (1:1) patients to receive A-DPBN or S-IMRT (+/-chemotherapy). Eligibility: nonmetastatic HNC of oral cavity, oro-/hypopharynx, or larynx, needing radio(chemo)therapy; T1-4N0-3 (exception: T1-2N0 glottic); KPS ≥ 70; ≥18 years; and informed consent. PRIMARY OUTCOMES: 1-year LC and RC. The dose prescription for A-DPBN was intercurrently adapted in 2 steps to an absolute dose-volume limit (≤1.75 cm3 can receive >84 Gy and normalized isoeffective dose >96 Gy) as a safety measure during the study course after 4/7 A-DPBN patients developed ≥G3 mucosal ulcers. RESULTS: Ninety-five patients were randomized (A-DPBN, 47; S-IMRT, 48). Median follow-up was 31 months (IQR, 14-48 months); 29 patients died (17 of cancer progression). A-DPBN resulted in superior LC compared with S-IMRT, with 1- and 2-year LC of 91% and 88% versus 78% and 75%, respectively (hazard ratio, 3.13; 95% CI, 1.13-8.71; P = .021). RC and overall survival were comparable between arms, as was overall grade (G) ≥3 late toxicity (36% vs 20%; P = .1). More ≥G3 late mucosal ulcers were observed in active smokers (29% vs 3%; P = .005) and alcohol users (33% vs 13%; P = .02), independent of treatment arm. Similarly, in the A-DPBN arm, significantly more patients who smoked at diagnosis developed ≥G3 (46% vs 12%; P = .005) and ≥G4 (29% vs 8%; P = .048) mucosal ulcers. One arterial blowout occurred after a G5 mucosal toxicity. CONCLUSIONS: A-DPBN resulted in superior 1- and 2-year LC for HNC compared with S-IMRT. This supports further exploration in multicenter phase 3 trials. It will, however, be challenging to recruit a substantial patient sample for such trials, as concerns have arisen regarding the association of late mucosal ulcers when escalating the dose in continuing smokers.

Unpaired intra-operative OCT (iOCT) video super-resolution with contrastive learning.

Regenerative therapies show promise in reversing sight loss caused by degenerative eye diseases. Their precise subretinal delivery can be facilitated by robotic systems alongside with Intra-operative Optical Coherence Tomography (iOCT). However, iOCT's real-time retinal layer information is compromised by inferior image quality. To address this limitation, we introduce an unpaired video super-resolution methodology for iOCT quality enhancement. A recurrent network is proposed to leverage temporal information from iOCT sequences, and spatial information from pre-operatively acquired OCT images. Additionally, a patchwise contrastive loss enables unpaired super-resolution. Extensive quantitative analysis demonstrates that our approach outperforms existing state-of-the-art iOCT super-resolution models. Furthermore, ablation studies showcase the importance of temporal aggregation and contrastive loss in elevating iOCT quality. A qualitative study involving expert clinicians also confirms this improvement. The comprehensive evaluation demonstrates our method's potential to enhance the iOCT image quality, thereby facilitating successful guidance for regenerative therapies.

A Dempster-Shafer Approach to Trustworthy AI With Application to Fetal Brain MRI Segmentation.

Deep learning models for medical image segmentation can fail unexpectedly and spectacularly for pathological cases and images acquired at different centers than training images, with labeling errors that violate expert knowledge. Such errors undermine the trustworthiness of deep learning models for medical image segmentation. Mechanisms for detecting and correcting such failures are essential for safely translating this technology into clinics and are likely to be a requirement of future regulations on artificial intelligence (AI). In this work, we propose a trustworthy AI theoretical framework and a practical system that can augment any backbone AI system using a fallback method and a fail-safe mechanism based on Dempster-Shafer theory. Our approach relies on an actionable definition of trustworthy AI. Our method automatically discards the voxel-level labeling predicted by the backbone AI that violate expert knowledge and relies on a fallback for those voxels. We demonstrate the effectiveness of the proposed trustworthy AI approach on the largest reported annotated dataset of fetal MRI consisting of 540 manually annotated fetal brain 3D T2w MRIs from 13 centers. Our trustworthy AI method improves the robustness of four backbone AI models for fetal brain MRIs acquired across various centers and for fetuses with various brain abnormalities.

High-fidelity, low-cost synthetic training model for fetoscopic spina bifida repair.

BACKGROUND: Fetoscopic spina bifida repair is increasingly being practiced, but limited skill acquisition poses a barrier to widespread adoption. Extensive training in relevant models, including both ex vivo and in vivo models may help. To address this, a synthetic training model that is affordable, realistic, and that allows skill analysis would be useful. OBJECTIVE: This study aimed to create a high-fidelity model for training in the essential neurosurgical steps of fetoscopic spina bifida repair using synthetic materials. In addition, we aimed to obtain a cheap and easily reproducible model. STUDY DESIGN: We developed a 3-layered, silicon-based model that resemble the anatomic layers of a typical myelomeningocele lesion. It allows for filling of the cyst with fluid and conducting a water tightness test after repair. A compliant silicon ball mimics the uterine cavity and is fixed to a solid 3-dimensional printed base. The fetal back with the lesion (single-use) is placed inside the uterine ball, which is reusable and repairable to allow for practicing port insertion and fixation multiple times. Following cannula insertion, the uterus is insufflated and a clinical fetoscopic or robotic or prototype instruments can be used. Three skilled endoscopic surgeons each did 6 simulated fetoscopic repairs using the surgical steps of an open repair. The primary outcome was surgical success, which was determined by water tightness of the repair, operation time <180 minutes and an Objective Structured Assessment of Technical Skills score of ≥18 of 25. Skill retention was measured using a competence cumulative sum analysis of a composite binary outcome of surgical success. Secondary outcomes were cost and fabrication time of the model. RESULTS: We made a model that can be used to simulate the neurosurgical steps of spina bifida repair, including anatomic details, port insertion, placode release and descent, undermining of skin and muscular layer, and endoscopic suturing. The model was made using reusable 3-dimensional printed molds and easily accessible materials. The 1-time startup cost was €211, and each single-use, simulated myelomeningocele lesion cost €9.5 in materials and 50 minutes of working time. Two skilled endoscopic surgeons performed 6 simulated, 3-port fetoscopic repairs, whereas a third used a Da Vinci surgical robot. Operation times decreased by more than 30% from the first to the last trial. Six experiments per surgeon did not show an obvious Objective Structured Assessment of Technical Skills score improvement. Competence cumulative sum analysis confirmed competency for each surgeon. CONCLUSION: This high-fidelity, low-cost spina bifida model allows simulated dissection and closure of a myelomeningocele lesion. VIDEO ABSTRACT.

Opto-Lipidomics of Tissues.

Lipid metabolism and signaling play pivotal functions in biology and disease development. Despite this, currently available optical techniques are limited in their ability to directly visualize the lipidome in tissues. In this study, opto-lipidomics, a new approach to optical molecular tissue imaging is introduced. The capability of vibrational Raman spectroscopy is expanded to identify individual lipids in complex tissue matrices through correlation with desorption electrospray ionization (DESI) - mass spectrometry (MS) imaging in an integrated instrument. A computational pipeline of inter-modality analysis is established to infer lipidomic information from optical vibrational spectra. Opto-lipidomic imaging of transient cerebral ischemia-reperfusion injury in a murine model of ischemic stroke demonstrates the visualization and identification of lipids in disease with high molecular specificity using Raman scattered light. Furthermore, opto-lipidomics in a handheld fiber-optic Raman probe is deployed and demonstrates real-time classification of bulk brain tissues based on specific lipid abundances. Opto-lipidomics opens a host of new opportunities to study lipid biomarkers for diagnostics, prognostics, and novel therapeutic targets.

Unified Brain MR-Ultrasound Synthesis using Multi-Modal Hierarchical Representations.

We introduce MHVAE, a deep hierarchical variational autoencoder (VAE) that synthesizes missing images from various modalities. Extending multi-modal VAEs with a hierarchical latent structure, we introduce a probabilistic formulation for fusing multi-modal images in a common latent representation while having the flexibility to handle incomplete image sets as input. Moreover, adversarial learning is employed to generate sharper images. Extensive experiments are performed on the challenging problem of joint intra-operative ultrasound (iUS) and Magnetic Resonance (MR) synthesis. Our model outperformed multi-modal VAEs, conditional GANs, and the current state-of-the-art unified method (ResViT) for synthesizing missing images, demonstrating the advantage of using a hierarchical latent representation and a principled probabilistic fusion operation. Our code is publicly available.

Rapid and robust endoscopic content area estimation: A lean GPU-based pipeline and curated benchmark dataset.

Endoscopic content area refers to the informative area enclosed by the dark, non-informative, border regions present in most endoscopic footage. The estimation of the content area is a common task in endoscopic image processing and computer vision pipelines. Despite the apparent simplicity of the problem, several factors make reliable real-time estimation surprisingly challenging. The lack of rigorous investigation into the topic combined with the lack of a common benchmark dataset for this task has been a long-lasting issue in the field. In this paper, we propose two variants of a lean GPU-based computational pipeline combining edge detection and circle fitting. The two variants differ by relying on handcrafted features, and learned features respectively to extract content area edge point candidates. We also present a first-of-its-kind dataset of manually annotated and pseudo-labelled content areas across a range of surgical indications. To encourage further developments, the curated dataset, and an implementation of both algorithms, has been made public (https://doi.org/10.7303/syn32148000, https://github.com/charliebudd/torch-content-area). We compare our proposed algorithm with a state-of-the-art U-Net-based approach and demonstrate significant improvement in terms of both accuracy (Hausdorff distance: 6.3 px versus 118.1 px) and computational time (Average runtime per frame: 0.13 ms versus 11.2 ms).