Multitask Weakly Supervised Generative Network for MR-US Registration.

Registering pre-operative modalities, such as magnetic resonance imaging or computed tomography, to ultrasound images is crucial for guiding clinicians during surgeries and biopsies. Recently, deep-learning approaches have been proposed to increase the speed and accuracy of this registration problem. However, all of these approaches need expensive supervision from the ultrasound domain. In this work, we propose a multitask generative framework that needs weak supervision only from the pre-operative imaging domain during training. To perform a deformable registration, the proposed framework translates a magnetic resonance image to the ultrasound domain while preserving the structural content. To demonstrate the efficacy of the proposed method, we tackle the registration problem of pre-operative 3D MR to transrectal ultrasonography images as necessary for targeted prostate biopsies. We use an in-house dataset of 600 patients, divided into 540 for training, 30 for validation, and the remaining for testing. An expert manually segmented the prostate in both modalities for validation and test sets to assess the performance of our framework. The proposed framework achieves a 3.58 mm target registration error on the expert-selected landmarks, 89.2% in the Dice score, and 1.81 mm 95th percentile Hausdorff distance on the prostate masks in the test set. Our experiments demonstrate that the proposed generative model successfully translates magnetic resonance images into the ultrasound domain. The translated image contains the structural content and fine details due to an ultrasound-specific two-path design of the generative model. The proposed framework enables training learning-based registration methods while only weak supervision from the pre-operative domain is available.

Shape completion in the dark: completing vertebrae morphology from 3D ultrasound.

PURPOSE: Ultrasound (US) imaging, while advantageous for its radiation-free nature, is challenging to interpret due to only partially visible organs and a lack of complete 3D information. While performing US-based diagnosis or investigation, medical professionals therefore create a mental map of the 3D anatomy. In this work, we aim to replicate this process and enhance the visual representation of anatomical structures. METHODS: We introduce a point cloud-based probabilistic deep learning (DL) method to complete occluded anatomical structures through 3D shape completion and choose US-based spine examinations as our application. To enable training, we generate synthetic 3D representations of partially occluded spinal views by mimicking US physics and accounting for inherent artifacts. RESULTS: The proposed model performs consistently on synthetic and patient data, with mean and median differences of 2.02 and 0.03 in Chamfer Distance (CD), respectively. Our ablation study demonstrates the importance of US physics-based data generation, reflected in the large mean and median difference of 11.8 CD and 9.55 CD, respectively. Additionally, we demonstrate that anatomical landmarks, such as the spinous process (with reconstruction CD of 4.73) and the facet joints (mean distance to ground truth (GT) of 4.96 mm), are preserved in the 3D completion. CONCLUSION: Our work establishes the feasibility of 3D shape completion for lumbar vertebrae, ensuring the preservation of level-wise characteristics and successful generalization from synthetic to real data. The incorporation of US physics contributes to more accurate patient data completions. Notably, our method preserves essential anatomical landmarks and reconstructs crucial injections sites at their correct locations.

Radiomics-Based Machine Learning Model for Diagnosis of Acute Pancreatitis Using Computed Tomography.

In the early diagnostic workup of acute pancreatitis (AP), the role of contrast-enhanced CT is to establish the diagnosis in uncertain cases, assess severity, and detect potential complications like necrosis, fluid collections, bleeding or portal vein thrombosis. The value of texture analysis/radiomics of medical images has rapidly increased during the past decade, and the main focus has been on oncological imaging and tumor classification. Previous studies assessed the value of radiomics for differentiating between malignancies and inflammatory diseases of the pancreas as well as for prediction of AP severity. The aim of our study was to evaluate an automatic machine learning model for AP detection using radiomics analysis. Patients with abdominal pain and contrast-enhanced CT of the abdomen in an emergency setting were retrospectively included in this single-center study. The pancreas was automatically segmented using TotalSegmentator and radiomics features were extracted using PyRadiomics. We performed unsupervised hierarchical clustering and applied the random-forest based Boruta model to select the most important radiomics features. Important features and lipase levels were included in a logistic regression model with AP as the dependent variable. The model was established in a training cohort using fivefold cross-validation and applied to the test cohort (80/20 split). From a total of 1012 patients, 137 patients with AP and 138 patients without AP were included in the final study cohort. Feature selection confirmed 28 important features (mainly shape and first-order features) for the differentiation between AP and controls. The logistic regression model showed excellent diagnostic accuracy of radiomics features for the detection of AP, with an area under the curve (AUC) of 0.932. Using lipase levels only, an AUC of 0.946 was observed. Using both radiomics features and lipase levels, we showed an excellent AUC of 0.933 for the detection of AP. Automated segmentation of the pancreas and consecutive radiomics analysis almost achieved the high diagnostic accuracy of lipase levels, a well-established predictor of AP, and might be considered an additional diagnostic tool in unclear cases. This study provides scientific evidence that automated image analysis of the pancreas achieves comparable diagnostic accuracy to lipase levels and might therefore be used in the future in the rapidly growing era of AI-based image analysis.

Diagnostic performance of photon-counting detector CT for differentiation between adrenal adenomas and metastases.

OBJECTIVES: Aim of this study was to assess the value of virtual non-contrast (VNC) reconstructions in differentiating between adrenal adenomas and metastases on a photon-counting detector CT (PCD-CT). MATERIAL AND METHODS: Patients with adrenal masses and contrast-enhanced CT scans in portal venous phase were included. Image reconstructions were performed, including conventional VNC (VNCConv) and PureCalcium VNC (VNCPC), as well as virtual monochromatic images (VMI, 40-90 keV) and iodine maps. We analyzed images using semi-automatic segmentation of adrenal lesions and extracted quantitative data. Logistic regression models, non-parametric tests, Bland-Altman plots, and a random forest classifier were used for statistical analyses. RESULTS: The final study cohort consisted of 90 patients (36 female, mean age 67.8 years [range 39-87]) with adrenal lesions (45 adenomas, 45 metastases). Compared to metastases, adrenal adenomas showed significantly lower CT-values in VNCConv and VNCPC (p = 0.007). Mean difference between VNC and true non-contrast (TNC) was 17.67 for VNCConv and 14.85 for VNCPC. Random forest classifier and logistic regression models both identified VNCConv and VNCPC as the best discriminators. When using 26 HU as the threshold in VNCConv reconstructions, adenomas could be discriminated from metastases with a sensitivity of 86.7% and a specificity of 75.6%. CONCLUSION: VNC algorithms overestimate CT values compared to TNC in the assessment of adrenal lesions. However, they allow a reliable discrimination between adrenal adenomas and metastases and could be used in clinical routine in near future with an increased threshold (e.g., 26 HU). Further (multi-center) studies with larger patient cohorts and standardized protocols are required. CLINICAL RELEVANCE STATEMENT: VNC reconstructions overestimate CT values compared to TNC. Using a different threshold (e.g., 26 HU compared to the established 10 HU), VNC has a high diagnostic accuracy for the discrimination between adrenal adenomas and metastases. KEY POINTS: • Virtual non-contrast reconstructions may be promising tools to differentiate adrenal lesions and might save further diagnostic tests. • The conventional and a new calcium-preserving virtual non-contrast algorithm tend to systematically overestimate CT-values compared to true non-contrast images. • Therefore, increasing the established threshold for true non-contrast images (e.g., 10HU) may help to differentiate between adrenal adenomas and metastases on contrast-enhanced CT.

EANM guidance document: dosimetry for first-in-human studies and early phase clinical trials.

The numbers of diagnostic and therapeutic nuclear medicine agents under investigation are rapidly increasing. Both novel emitters and novel carrier molecules require careful selection of measurement procedures. This document provides guidance relevant to dosimetry for first-in human and early phase clinical trials of such novel agents. The guideline includes a short introduction to different emitters and carrier molecules, followed by recommendations on the methods for activity measurement, pharmacokinetic analyses, as well as absorbed dose calculations and uncertainty analyses. The optimal use of preclinical information and studies involving diagnostic analogues is discussed. Good practice reporting is emphasised, and relevant dosimetry parameters and method descriptions to be included are listed. Three examples of first-in-human dosimetry studies, both for diagnostic tracers and radionuclide therapies, are given.

Molecular image-guided surgery in gynaecological cancer: where do we stand?

PURPOSE: The aim of this review is to give an overview of the current status of molecular image-guided surgery in gynaecological malignancies, from both clinical and technological points of view. METHODS: A narrative approach was taken to describe the relevant literature, focusing on clinical applications of molecular image-guided surgery in gynaecology, preoperative imaging as surgical roadmap, and intraoperative devices. RESULTS: The most common clinical application in gynaecology is sentinel node biopsy (SNB). Other promising approaches are receptor-target modalities and occult lesion localisation. Preoperative SPECT/CT and PET/CT permit a roadmap for adequate surgical planning. Intraoperative detection modalities span from 1D probes to 2D portable cameras and 3D freehand imaging. CONCLUSION: After successful application of radio-guided SNB and SPECT, innovation is leaning towards hybrid modalities, such as hybrid tracer and fusion of imaging approaches including SPECT/CT and PET/CT. Robotic surgery, as well as augmented reality and virtual reality techniques, is leading to application of these innovative technologies to the clinical setting, guiding surgeons towards a precise, personalised, and minimally invasive approach.

Artificial Intelligence-powered automatic volume calculation in medical images - available tools, performance and challenges for nuclear medicine.

Volumetry is crucial in oncology and endocrinology, for diagnosis, treatment planning, and evaluating response to therapy for several diseases. The integration of Artificial Intelligence (AI) and Deep Learning (DL) has significantly accelerated the automatization of volumetric calculations, enhancing accuracy and reducing variability and labor. In this review, we show that a high correlation has been observed between Machine Learning (ML) methods and expert assessments in tumor volumetry; Yet, it is recognized as more challenging than organ volumetry. Liver volumetry has shown progression in accuracy with a decrease in error. If a relative error below 10 % is acceptable, ML-based liver volumetry can be considered reliable for standardized imaging protocols if used in patients without major anomalies. Similarly, ML-supported automatic kidney volumetry has also shown consistency and reliability in volumetric calculations. In contrast, AI-supported thyroid volumetry has not been extensively developed, despite initial works in 3D ultrasound showing promising results in terms of accuracy and reproducibility. Despite the advancements presented in the reviewed literature, the lack of standardization limits the generalizability of ML methods across diverse scenarios. The domain gap, i. e., the difference in probability distribution of training and inference data, is of paramount importance before clinical deployment of AI, to maintain accuracy and reliability in patient care. The increasing availability of improved segmentation tools is expected to further incorporate AI methods into routine workflows where volumetry will play a more prominent role in radionuclide therapy planning and quantitative follow-up of disease evolution.

Multimodal graph attention network for COVID-19 outcome prediction.

When dealing with a newly emerging disease such as COVID-19, the impact of patient- and disease-specific factors (e.g., body weight or known co-morbidities) on the immediate course of the disease is largely unknown. An accurate prediction of the most likely individual disease progression can improve the planning of limited resources and finding the optimal treatment for patients. In the case of COVID-19, the need for intensive care unit (ICU) admission of pneumonia patients can often only be determined on short notice by acute indicators such as vital signs (e.g., breathing rate, blood oxygen levels), whereas statistical analysis and decision support systems that integrate all of the available data could enable an earlier prognosis. To this end, we propose a holistic, multimodal graph-based approach combining imaging and non-imaging information. Specifically, we introduce a multimodal similarity metric to build a population graph that shows a clustering of patients. For each patient in the graph, we extract radiomic features from a segmentation network that also serves as a latent image feature encoder. Together with clinical patient data like vital signs, demographics, and lab results, these modalities are combined into a multimodal representation of each patient. This feature extraction is trained end-to-end with an image-based Graph Attention Network to process the population graph and predict the COVID-19 patient outcomes: admission to ICU, need for ventilation, and mortality. To combine multiple modalities, radiomic features are extracted from chest CTs using a segmentation neural network. Results on a dataset collected in Klinikum rechts der Isar in Munich, Germany and the publicly available iCTCF dataset show that our approach outperforms single modality and non-graph baselines. Moreover, our clustering and graph attention increases understanding of the patient relationships within the population graph and provides insight into the network's decision-making process.

Theoretical error analysis of spotlight-based instrument localization for retinal surgery.

Retinal surgery is widely considered to be a complicated and challenging task even for specialists. Image-guided robot-assisted intervention is among the novel and promising solutions that may enhance human capabilities therein. In this paper, we demonstrate the possibility of using spotlights for 5D guidance of a microsurgical instrument. The theoretical basis of the localization for the instrument based on the projection of a single spotlight is analyzed to deduce the position and orientation of the spotlight source. The usage of multiple spotlights is also proposed to check the possibility of further improvements for the performance boundaries. The proposed method is verified within a high-fidelity simulation environment using the 3D creation suite Blender. Experimental results show that the average positioning error is 0.029 mm using a single spotlight and 0.025 mm with three spotlights, respectively, while the rotational errors are 0.124 and 0.101, which shows the application to be promising in instrument localization for retinal surgery.

Machine learning methods for tracer kinetic modelling.

Tracer kinetic modelling based on dynamic PET is an important field of Nuclear Medicine for quantitative functional imaging. Yet, its implementation in clinical routine has been constrained by its complexity and computational costs. Machine learning poses an opportunity to improve modelling processes in terms of arterial input function prediction, the prediction of kinetic modelling parameters and model selection in both clinical and preclinical studies while reducing processing time. Moreover, it can help improving kinetic modelling data used in downstream tasks such as tumor detection. In this review, we introduce the basics of tracer kinetic modelling and present a literature review of original works and conference papers using machine learning methods in this field.

The Medical Informatics Initiative and the Network University Medicine - Perspectives for Nuclear Medicine.

Digitization in the healthcare sector and the support of clinical workflows with artificial intelligence (AI), including AI-supported image analysis, represent a great challenge and equally a promising perspective for preclinical and clinical nuclear medicine. In Germany, the Medical Informatics Initiative (MII) and the Network University Medicine (NUM) are of central importance for this transformation. This review article outlines these structures and highlights their future role in enabling privacy-preserving federated multi-center analyses with interoperable data structures harmonized between site-specific IT infrastructures. The newly founded working group "Digitization and AI" in the German Society of Nuclear Medicine (DGN) as well as the Fach- und Organspezifische Arbeitsgruppe (FOSA, specialty- and organ-specific working group) founded for the field of nuclear medicine (FOSA Nuklearmedizin) within the NUM aim to initiate and coordinate measures in the context of digital medicine and (image-)data-driven analyses for the DGN.

A survey of catheter tracking concepts and methodologies.

Catheter tracking has become an integral part of interventional radiology. Over the last decades, researchers have significantly contributed to theoretical and technical catheter tracking solutions. However, most of the published work thus far focuses on a single application or a single tracking technology. This paper provides an exhaustive review of the state-of-the-art for catheter tracking in general by analyzing significant contributions in this field. We first present a historical overview that led to catheter tracking and continue with a survey of leading tracking technologies. These include image-based tracking, active and passive tracking, electromagnetic tracking, fiber optic shape sensing, bioelectric navigation, robotic tracking solutions, and hybrid tracking. As for imaging modalities, the focus is on x-ray based modalities, ultrasound, and magnetic resonance imaging. Finally, we review each tracking technology with respect to the imaging modality and establish the relation between the two and the underlying anatomy of interest.

A lightweight neural network with multiscale feature enhancement for liver CT segmentation.

Segmentation of abdominal Computed Tomography (CT) scan is essential for analyzing, diagnosing, and treating visceral organ diseases (e.g., hepatocellular carcinoma). This paper proposes a novel neural network (Res-PAC-UNet) that employs a fixed-width residual UNet backbone and Pyramid Atrous Convolutions, providing a low disk utilization method for precise liver CT segmentation. The proposed network is trained on medical segmentation decathlon dataset using a modified surface loss function. Additionally, we evaluate its quantitative and qualitative performance; the Res16-PAC-UNet achieves a Dice coefficient of 0.950 ± 0.019 with less than half a million parameters. Alternatively, the Res32-PAC-UNet obtains a Dice coefficient of 0.958 ± 0.015 with an acceptable parameter count of approximately 1.2 million.

Non-Invasive Glucose Metabolism Quantification Method Based on Unilateral ICA Image Derived Input Function by Hybrid PET/MR in Ischemic Cerebrovascular Disease.

The non-invasive quantification of the cerebral metabolic rate for glucose (CMRGlc) and the characterization of cerebral metabolism in the cerebrovascular territories are helpful in understanding ischemic cerebrovascular disease (ICVD). Firstly, we investigated a non-invasive quantification approach based on an image-derived input function (IDIF) in ICVD. Second, we studied the metabolic changes in CMRGlc after surgical intervention. We evaluated the hypothesis that the IDIF method based on the unilateral internal carotid artery could address challenges in ICVD quantification. The CMRGlc and standardized uptake value ratio (SUVR) were used to measure glucose metabolism activity. Healthy controls showed no significant differences in CMRGlc values between bilateral and unilateral IDIF measurements (intraclass correlation coefficient [ICC]: 0.91-0.98). Patients with ICVD showed significantly increased CMRGlc values after surgical intervention for all territories (percentage changes: 7.4%-22.5%). In contrast, SUVR showed minor differences between postoperative and preoperative patients, indicating that it was a poor biomarker for the diagnosis of ICVD. A significant association between CMRGlc and the National Institutes of Health Stroke Scale (NIHSS) scores was observed (r=-0.54). Our findings suggested that IDIF could be a valuable tool for CMRGlc quantification in patients with ICVD and may advance personalized precision interventions.

Tracked 3D ultrasound and deep neural network-based thyroid segmentation reduce interobserver variability in thyroid volumetry.

Thyroid volumetry is crucial in the diagnosis, treatment, and monitoring of thyroid diseases. However, conventional thyroid volumetry with 2D ultrasound is highly operator-dependent. This study compares 2D and tracked 3D ultrasound with an automatic thyroid segmentation based on a deep neural network regarding inter- and intraobserver variability, time, and accuracy. Volume reference was MRI. 28 healthy volunteers (24-50 a) were scanned with 2D and 3D ultrasound (and by MRI) by three physicians (MD 1, 2, 3) with different experience levels (6, 4, and 1 a). In the 2D scans, the thyroid lobe volumes were calculated with the ellipsoid formula. A convolutional deep neural network (CNN) automatically segmented the 3D thyroid lobes. 26, 6, and 6 random lobe scans were used for training, validation, and testing, respectively. On MRI (T1 VIBE sequence) the thyroid was manually segmented by an experienced MD. MRI thyroid volumes ranged from 2.8 to 16.7ml (mean 7.4, SD 3.05). The CNN was trained to obtain an average Dice score of 0.94. The interobserver variability comparing two MDs showed mean differences for 2D and 3D respectively of 0.58 to 0.52ml (MD1 vs. 2), -1.33 to -0.17ml (MD1 vs. 3) and -1.89 to -0.70ml (MD2 vs. 3). Paired samples t-tests showed significant differences for 2D (p = .140, p = .002 and p = .002) and none for 3D (p = .176, p = .722 and p = .057). Intraobsever variability was similar for 2D and 3D ultrasound. Comparison of ultrasound volumes and MRI volumes showed a significant difference for the 2D volumetry of all MDs (p = .002, p = .009, p <.001), and no significant difference for 3D ultrasound (p = .292, p = .686, p = 0.091). Acquisition time was significantly shorter for 3D ultrasound. Tracked 3D ultrasound combined with a CNN segmentation significantly reduces interobserver variability in thyroid volumetry and increases the accuracy of the measurements with shorter acquisition times.

Personalized irradiation therapy for NMSC by rhenium-188 skin cancer therapy: a long-term retrospective study.

OBJECTIVES: This study aimed to provide long-term clinical data about an innovative epidermal radioisotope therapy called Rhenium-SCT® (Skin Cancer Therapy) for non-melanoma skin cancer (NMSC), based on the use of the non-sealed beta emitter rhenium-188. MATERIAL AND METHODS: 52 NMSC patients with a mean age of 71.7 years were treated with rhenium-188 skin cancer therapy between the years 2005 and 2014. An acryl matrix containing rhenium-188 was applied on a plastic foil covering the tumor. The treatment time for reaching a radiation dose of 50 Gy was calculated by a software program. Patients' characteristics and clinical follow-up data were collected and retrospectively analyzed. RESULTS: Overall 55 lesions (32 BCC, 19 SCC, 2 M. Bowen and 2 extramammary Paget's disease (EMPD)) mainly in the head and neck region (72.3%) were treated. The average size of the irradiation area was 9.79 cm2 and the mean treatment time 46.35 min. All lesions showed a complete remission after a follow-up period between 3 and more than 12 months. No complications or other post-interventional problems were reported. CONCLUSIONS: Rhenium-SCT® is considered as an effective, rapid, safe, painless treatment mostly performed in a single therapeutic session, regardless of the shape complexity, anatomical site and number of lesions.

How molecular imaging will enable robotic precision surgery : The role of artificial intelligence, augmented reality, and navigation.

Molecular imaging is one of the pillars of precision surgery. Its applications range from early diagnostics to therapy planning, execution, and the accurate assessment of outcomes. In particular, molecular imaging solutions are in high demand in minimally invasive surgical strategies, such as the substantially increasing field of robotic surgery. This review aims at connecting the molecular imaging and nuclear medicine community to the rapidly expanding armory of surgical medical devices. Such devices entail technologies ranging from artificial intelligence and computer-aided visualization technologies (software) to innovative molecular imaging modalities and surgical navigation (hardware). We discuss technologies based on their role at different steps of the surgical workflow, i.e., from surgical decision and planning, over to target localization and excision guidance, all the way to (back table) surgical verification. This provides a glimpse of how innovations from the technology fields can realize an exciting future for the molecular imaging and surgery communities.

Evaluation of camera-based freehand SPECT in preoperative sentinel lymph node mapping for melanoma patients.

BACKGROUND: Assessment of lymphatic status via sentinel lymph node (SLN) biopsy is an integral and crucial part of melanoma surgical oncology. The most common technique for sentinel node mapping is preoperative planar scintigraphy of an injected gamma-emitting lymphatic tracer followed by intraoperative node localization using a non-imaging gamma probe with auditory feedback. In recent years, intraoperative visualization of SLNs in 3D has become possible by coupling the probe to an external system capable of tracking its location and orientation as it is read out, thereby enabling computation of the 3D distribution of the tracer (freehand SPECT). In this project, the non-imaging probe of the fhSPECT system was replaced by a unique handheld gamma camera containing an array of sodium iodide crystals optically coupled to an array of silicon photomultipliers (SiPMs). A feasibility study was performed in which preoperative SLN mapping was performed using camera fhSPECT and the number of detected nodes was compared to that visualized by lymphoscintigraphy, probe fhSPECT, and to the number ultimately excised under non-imaging probe guidance. RESULTS: Among five subjects, SLNs were detected in nine lymphatic basins, with one to five SLNs detected per basin. A basin-by-basin comparison showed that the number of SLNs detected using camera fhSPECT exceeded that using lymphoscintigraphy and probe fhSPECT in seven of nine basins and five of five basins, respectively. (Probe fhSPECT scans were not performed for four basins.) It exceeded the number excised under non-imaging probe guidance for seven of nine basins and equaled the number excised for the other two basins. CONCLUSIONS: Freehand SPECT using a prototype SiPM-based gamma camera demonstrates high sensitivity for detection of SLNs in a preoperative setting. Camera fhSPECT is a potential means for efficiently obtaining real-time 3D activity distribution maps in applications such as image-guided percutaneous biopsy, and surgical SLN biopsy or radioguided tumor excision.