Gamified Crowdsourcing as a Novel Approach to Lung Ultrasound Data Set Labeling: Prospective Analysis.

BACKGROUND: Machine learning (ML) models can yield faster and more accurate medical diagnoses; however, developing ML models is limited by a lack of high-quality labeled training data. Crowdsourced labeling is a potential solution but can be constrained by concerns about label quality. OBJECTIVE: This study aims to examine whether a gamified crowdsourcing platform with continuous performance assessment, user feedback, and performance-based incentives could produce expert-quality labels on medical imaging data. METHODS: In this diagnostic comparison study, 2384 lung ultrasound clips were retrospectively collected from 203 emergency department patients. A total of 6 lung ultrasound experts classified 393 of these clips as having no B-lines, one or more discrete B-lines, or confluent B-lines to create 2 sets of reference standard data sets (195 training clips and 198 test clips). Sets were respectively used to (1) train users on a gamified crowdsourcing platform and (2) compare the concordance of the resulting crowd labels to the concordance of individual experts to reference standards. Crowd opinions were sourced from DiagnosUs (Centaur Labs) iOS app users over 8 days, filtered based on past performance, aggregated using majority rule, and analyzed for label concordance compared with a hold-out test set of expert-labeled clips. The primary outcome was comparing the labeling concordance of collated crowd opinions to trained experts in classifying B-lines on lung ultrasound clips. RESULTS: Our clinical data set included patients with a mean age of 60.0 (SD 19.0) years; 105 (51.7%) patients were female and 114 (56.1%) patients were White. Over the 195 training clips, the expert-consensus label distribution was 114 (58%) no B-lines, 56 (29%) discrete B-lines, and 25 (13%) confluent B-lines. Over the 198 test clips, expert-consensus label distribution was 138 (70%) no B-lines, 36 (18%) discrete B-lines, and 24 (12%) confluent B-lines. In total, 99,238 opinions were collected from 426 unique users. On a test set of 198 clips, the mean labeling concordance of individual experts relative to the reference standard was 85.0% (SE 2.0), compared with 87.9% crowdsourced label concordance (P=.15). When individual experts' opinions were compared with reference standard labels created by majority vote excluding their own opinion, crowd concordance was higher than the mean concordance of individual experts to reference standards (87.4% vs 80.8%, SE 1.6 for expert concordance; P<.001). Clips with discrete B-lines had the most disagreement from both the crowd consensus and individual experts with the expert consensus. Using randomly sampled subsets of crowd opinions, 7 quality-filtered opinions were sufficient to achieve near the maximum crowd concordance. CONCLUSIONS: Crowdsourced labels for B-line classification on lung ultrasound clips via a gamified approach achieved expert-level accuracy. This suggests a strategic role for gamified crowdsourcing in efficiently generating labeled image data sets for training ML systems.

massNet: integrated processing and classification of spatially resolved mass spectrometry data using deep learning for rapid tumor delineation.

MOTIVATION: Mass spectrometry imaging (MSI) provides rich biochemical information in a label-free manner and therefore holds promise to substantially impact current practice in disease diagnosis. However, the complex nature of MSI data poses computational challenges in its analysis. The complexity of the data arises from its large size, high-dimensionality and spectral nonlinearity. Preprocessing, including peak picking, has been used to reduce raw data complexity; however, peak picking is sensitive to parameter selection that, perhaps prematurely, shapes the downstream analysis for tissue classification and ensuing biological interpretation. RESULTS: We propose a deep learning model, massNet, that provides the desired qualities of scalability, nonlinearity and speed in MSI data analysis. This deep learning model was used, without prior preprocessing and peak picking, to classify MSI data from a mouse brain harboring a patient-derived tumor. The massNet architecture established automatically learning of predictive features, and automated methods were incorporated to identify peaks with potential for tumor delineation. The model's performance was assessed using cross-validation, and the results demonstrate higher accuracy and a substantial gain in speed compared to the established classical machine learning method, support vector machine. AVAILABILITY AND IMPLEMENTATION: https://github.com/wabdelmoula/massNet. The data underlying this article are available in the NIH Common Fund's National Metabolomics Data Repository (NMDR) Metabolomics Workbench under project id (PR001292) with http://dx.doi.org/10.21228/M8Q70T. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

PET/CT Fluoroscopy during PET/CT-Guided Interventions: Initial Experience.

This study assessed the feasibility and functionality of the use of a high-speed image fusion technology to generate and display positron emission tomography (PET)/computed tomography (CT) fluoroscopic images during PET/CT-guided tumor ablation procedures. Thirteen patients underwent 14 PET/CT-guided ablations for the treatment of 20 tumors. A Food and Drug Administration-cleared multimodal image fusion platform received images pushed from a scanner, followed by near-real-time, nonrigid image registration. The most recent intraprocedural PET dataset was fused to each single-rotation CT fluoroscopy dataset as it arrived, and the fused images were displayed on an in-room monitor. PET/CT fluoroscopic images were generated and displayed in all procedures and enabled more confident targeting in 3 procedures. The mean lag time from CT fluoroscopic image acquisition to the in-room display of the fused PET/CT fluoroscopic image was 21 seconds ± 8. The registration accuracy was visually satisfactory in 13 of 14 procedures. In conclusion, PET/CT fluoroscopy was feasible and may have the potential to facilitate PET/CT-guided procedures.

Positron Emission Tomography and Computed Tomography Contributions to Patient Dose and Personnel Exposure to Radiation during PET/CT-Guided Tumor Ablations.

This study sought to quantify the positron emission tomography (PET) and computed tomography (CT) components of patient radiation doses and personnel exposure to radiations during PET/CT-guided tumor ablations and assess the utility of a rolling lead shield for operator protection. Two operators performed 21 PET/CT-guided ablations behind a customized, 25-mm-thick lead shield with midchest-to-midthigh coverage. The mean patient radiation dose per procedure was 3.90 mSv ± 1.13 (11.3%) from PET and 30.51 mSv ± 19.05 (88.7%) from CT. The mean primary and secondary operator exposure outside neck-level thyroid shields was 0.05 and 0.02 mSv per procedure, respectively. The radiation exposure levels behind the rolling lead shield, inside the primary operator's thyroid shield, and on the other personnel were below the measurable threshold cumulatively over 21 procedures. The mean PET exposure level at continuous close proximity to patients was 0.02 mSv per procedure. The PET radiation doses to the patients and personnel were small. Thus, the rolling lead shield provided limited benefit.

Automatic 3D Nonlinear Registration of Mass Spectrometry Imaging and Magnetic Resonance Imaging Data.

Multimodal integration between mass spectrometry imaging (MSI) and radiology-established modalities such as magnetic resonance imaging (MRI) would allow the investigations of key questions in complex biological systems such as the central nervous system. Such integration would provide complementary multiscale data to bridge the gap between molecular and anatomical phenotypes, potentially revealing new insights into molecular mechanisms underlying anatomical pathologies presented on MRI. Automatic coregistration between 3D MSI/MRI is a computationally challenging process due to dimensional complexity, MSI data sparsity, lack of direct spatial-correspondences, and nonlinear tissue deformation. Here, we present a new computational approach based on stochastic neighbor embedding to nonlinearly align 3D MSI to MRI data, identify and reconstruct biologically relevant molecular patterns in 3D, and fuse the MSI datacube to the MRI space. We demonstrate our method using multimodal high-spectral resolution matrix-assisted laser desorption ionization (MALDI) 9.4 T MSI and 7 T in vivo MRI data, acquired from a patient-derived, xenograft mouse brain model of glioblastoma following administration of the EGFR inhibitor drug of Erlotinib. Results show the distribution of some identified molecular ions of the EGFR inhibitor erlotinib, a phosphatidylcholine lipid, and cholesterol, which were reconstructed in 3D and mapped to the MRI space. The registration quality was evaluated on two normal mouse brains using the Dice coefficient for the regions of brainstem, hippocampus, and cortex. The method is generic and can therefore be applied to hyperspectral images from different mass spectrometers and integrated with other established in vivo imaging modalities such as computed tomography (CT) and positron emission tomography (PET).

Whole brain white matter connectivity analysis using machine learning: An application to autism.

In this paper, we propose an automated white matter connectivity analysis method for machine learning classification and characterization of white matter abnormality via identification of discriminative fiber tracts. The proposed method uses diffusion MRI tractography and a data-driven approach to find fiber clusters corresponding to subdivisions of the white matter anatomy. Features extracted from each fiber cluster describe its diffusion properties and are used for machine learning. The method is demonstrated by application to a pediatric neuroimaging dataset from 149 individuals, including 70 children with autism spectrum disorder (ASD) and 79 typically developing controls (TDC). A classification accuracy of 78.33% is achieved in this cross-validation study. We investigate the discriminative diffusion features based on a two-tensor fiber tracking model. We observe that the mean fractional anisotropy from the second tensor (associated with crossing fibers) is most affected in ASD. We also find that local along-tract (central cores and endpoint regions) differences between ASD and TDC are helpful in differentiating the two groups. These altered diffusion properties in ASD are associated with multiple robustly discriminative fiber clusters, which belong to several major white matter tracts including the corpus callosum, arcuate fasciculus, uncinate fasciculus and aslant tract; and the white matter structures related to the cerebellum, brain stem, and ventral diencephalon. These discriminative fiber clusters, a small part of the whole brain tractography, represent the white matter connections that could be most affected in ASD. Our results indicate the potential of a machine learning pipeline based on white matter fiber clustering.

Multimodal imaging for improved diagnosis and treatment of cancers.

The authors review methods for image-guided diagnosis and therapy that increase precision in the detection, characterization, and localization of many forms of cancer to achieve optimal target definition and complete resection or ablation. A new model of translational, clinical, image-guided therapy research is presented, and the Advanced Multimodality Image-Guided Operating (AMIGO) suite is described. AMIGO was conceived and designed to allow for the full integration of imaging in cancer diagnosis and treatment. Examples are drawn from over 500 procedures performed on brain, neck, spine, thorax (breast, lung), and pelvis (prostate and gynecologic) areas and are used to describe how they address some of the many challenges of treating brain, prostate, and lung tumors. Cancer 2015;121:817-827. © 2014 American Cancer Society.

Real-time active MR-tracking of metallic stylets in MR-guided radiation therapy.

PURPOSE: To develop an active MR-tracking system to guide placement of metallic devices for radiation therapy. METHODS: An actively tracked metallic stylet for brachytherapy was constructed by adding printed-circuit micro-coils to a commercial stylet. The coil design was optimized by electromagnetic simulation, and has a radio-frequency lobe pattern extending ∼5 mm beyond the strong B0 inhomogeneity region near the metal surface. An MR-tracking sequence with phase-field dithering was used to overcome residual effects of B0 and B1 inhomogeneities caused by the metal, as well as from inductive coupling to surrounding metallic stylets. The tracking system was integrated with a graphical workstation for real-time visualization. The 3 Tesla MRI catheter-insertion procedures were tested in phantoms and ex vivo animal tissue, and then performed in three patients during interstitial brachytherapy. RESULTS: The tracking system provided high-resolution (0.6 × 0.6 × 0.6 mm(3) ) and rapid (16 to 40 frames per second, with three to one phase-field dithering directions) catheter localization in phantoms, animals, and three gynecologic cancer patients. CONCLUSION: This is the first demonstration of active tracking of the shaft of metallic stylet in MR-guided brachytherapy. It holds the promise of assisting physicians to achieve better targeting and improving outcomes in interstitial brachytherapy.

The prognostic value of improving congestion on lung ultrasound during treatment for acute heart failure differs based on patient characteristics at admission.

BACKGROUND: Lung ultrasound congestion scoring (LUS-CS) is a congestion severity biomarker. The BLUSHED-AHF trial demonstrated feasibility for LUS-CS-guided therapy in acute heart failure (AHF). We investigated two questions: 1) does change (∆) in LUS-CS from emergency department (ED) to hospital-discharge predict patient outcomes, and 2) is the relationship between in-hospital decongestion and adverse events moderated by baseline risk-factors at admission? METHODS: We performed a secondary analysis of 933 observations/128 patients from 5 hospitals in the BLUSHED-AHF trial receiving daily LUS. ∆LUS-CS from ED arrival to inpatient discharge (scale -160 to +160, where negative = improving congestion) was compared to a primary outcome of 30-day death/AHF-rehospitalization. Cox regression was used to adjust for mortality risk at admission [Get-With-The-Guidelines HF risk score (GWTG-RS)] and the discharge LUS-CS. An interaction between ∆LUS-CS and GWTG-RS was included, under the hypothesis that the association between decongestion intensity (by ∆LUS-CS) and adverse outcomes would be stronger in admitted patients with low-mortality risk but high baseline congestion. RESULTS: Median age was 65 years, GWTG-RS 36, left ventricular ejection fraction 36 %, and ∆LUS-CS -20. In the multivariable analysis ∆LUS-CS was associated with event-free survival (HR = 0.61; 95 % CI: 0.38-0.97), while discharge LUS-CS (HR = 1.00; 95%CI: 0.54-1.84) did not add incremental prognostic value to ∆LUS-CS alone. As GWTG-RS rose, benefits of LUS-CS reduction attenuated (interaction p < 0.05). ∆LUS-CS and event-free survival were most strongly correlated in patients without tachycardia, tachypnea, hypotension, hyponatremia, uremia, advanced age, or history of myocardial infarction at ED/baseline, and those with low daily loop diuretic requirements. CONCLUSIONS: Reduction in ∆LUS-CS during AHF treatment was most associated with improved readmission-free survival in heavily congested patients with otherwise reassuring features at admission. ∆LUS-CS may be most useful as a measure to ensure adequate decongestion prior to discharge, to prevent early readmission, rather than modify survival.

Feasibility and safety of bipolar radiofrequency track cautery during percutaneous image-guided abdominal biopsy procedures.

PURPOSE: The purpose of this study was to assess the feasibility and safety of using a bipolar radiofrequency track cautery device during percutaneous image-guided abdominal biopsy procedures in at-risk patients. METHODS: Forty-two patients (26-79 years old; female 44%) with at least one bleeding risk factor who underwent an abdominal image-guided (CT or US) biopsy and intended bipolar radiofrequency track cautery (BRTC) were retrospectively studied. An 18G radiofrequency electrode was inserted through a 17G biopsy introducer needle immediately following coaxial 18G core biopsy, to cauterize the biopsy track using temperature control. Bleeding risk factors, technical success, and adverse events were recorded. RESULTS: BRTC was technically successful in 41/42 (98%) of procedures; in one patient, the introducer needle retracted from the liver due to respiratory motion prior to BRTC. BRTC following percutaneous biopsy was applied during 41 abdominal biopsy procedures (renal mass = 12, renal parenchyma = 10, liver mass = 9, liver parenchyma = 5, splenic mass or parenchyma = 4, gastrohepatic mass = 1). All patients had one or more of the following risk factors: high-risk organ (spleen or renal parenchyma), hypervascular mass, elevated prothrombin time, renal insufficiency, thrombocytopenia, recent anticoagulation or anticoagulation not withheld for recommended interval, cirrhosis, intraprocedural hypertension, brisk back bleeding observed from the introducer needle, or subcapsular tumor location. No severe adverse events (grade 3 or higher) occurred. Two (2/41, 5%) mild (grade 1) bleeding events did not cause symptoms or require intervention. CONCLUSION: Bipolar radiofrequency track cautery was feasible and safe during percutaneous image-guided abdominal biopsy procedures. IRB approval: MBG 2022P002277.

ReMIND: The Brain Resection Multimodal Imaging Database.

The standard of care for brain tumors is maximal safe surgical resection. Neuronavigation augments the surgeon's ability to achieve this but loses validity as surgery progresses due to brain shift. Moreover, gliomas are often indistinguishable from surrounding healthy brain tissue. Intraoperative magnetic resonance imaging (iMRI) and ultrasound (iUS) help visualize the tumor and brain shift. iUS is faster and easier to incorporate into surgical workflows but offers a lower contrast between tumorous and healthy tissues than iMRI. With the success of data-hungry Artificial Intelligence algorithms in medical image analysis, the benefits of sharing well-curated data cannot be overstated. To this end, we provide the largest publicly available MRI and iUS database of surgically treated brain tumors, including gliomas (n=92), metastases (n=11), and others (n=11). This collection contains 369 preoperative MRI series, 320 3D iUS series, 301 iMRI series, and 356 segmentations collected from 114 consecutive patients at a single institution. This database is expected to help brain shift and image analysis research and neurosurgical training in interpreting iUS and iMRI.

Evaluation of mediastinal lymph node segmentation of heterogeneous CT data with full and weak supervision.

Accurate lymph node size estimation is critical for staging cancer patients, initial therapeutic management, and assessing response to therapy. Current standard practice for quantifying lymph node size is based on a variety of criteria that use uni-directional or bi-directional measurements. Segmentation in 3D can provide more accurate evaluations of the lymph node size. Fully convolutional neural networks (FCNs) have achieved state-of-the-art results in segmentation for numerous medical imaging applications, including lymph node segmentation. Adoption of deep learning segmentation models in clinical trials often faces numerous challenges. These include lack of pixel-level ground truth annotations for training, generalizability of the models on unseen test domains due to the heterogeneity of test cases and variation of imaging parameters. In this paper, we studied and evaluated the performance of lymph node segmentation models on a dataset that was completely independent of the one used to create the models. We analyzed the generalizability of the models in the face of a heterogeneous dataset and assessed the potential effects of different disease conditions and imaging parameters. Furthermore, we systematically compared fully-supervised and weakly-supervised methods in this context. We evaluated the proposed methods using an independent dataset comprising 806 mediastinal lymph nodes from 540 unique patients. The results show that performance achieved on the independent test set is comparable to that on the training set. Furthermore, neither the underlying disease nor the heterogeneous imaging parameters impacted the performance of the models. Finally, the results indicate that our weakly-supervised method attains 90%- 91% of the performance achieved by the fully supervised training.

ReMIND: The Brain Resection Multimodal Imaging Database.

The standard of care for brain tumors is maximal safe surgical resection. Neuronavigation augments the surgeon's ability to achieve this but loses validity as surgery progresses due to brain shift. Moreover, gliomas are often indistinguishable from surrounding healthy brain tissue. Intraoperative magnetic resonance imaging (iMRI) and ultrasound (iUS) help visualize the tumor and brain shift. iUS is faster and easier to incorporate into surgical workflows but offers a lower contrast between tumorous and healthy tissues than iMRI. With the success of data-hungry Artificial Intelligence algorithms in medical image analysis, the benefits of sharing well-curated data cannot be overstated. To this end, we provide the largest publicly available MRI and iUS database of surgically treated brain tumors, including gliomas (n = 92), metastases (n = 11), and others (n = 11). This collection contains 369 preoperative MRI series, 320 3D iUS series, 301 iMRI series, and 356 segmentations collected from 114 consecutive patients at a single institution. This database is expected to help brain shift and image analysis research and neurosurgical training in interpreting iUS and iMRI.

Development of the PSYCHS: Positive SYmptoms and Diagnostic Criteria for the CAARMS Harmonized with the SIPS.

AIM: To harmonize two ascertainment and severity rating instruments commonly used for the clinical high risk syndrome for psychosis (CHR-P): the Structured Interview for Psychosis-risk Syndromes (SIPS) and the Comprehensive Assessment of At-Risk Mental States (CAARMS). METHODS: The initial workshop is described in the companion report from Addington et al. After the workshop, lead experts for each instrument continued harmonizing attenuated positive symptoms and criteria for psychosis and CHR-P through an intensive series of joint videoconferences. RESULTS: Full harmonization was achieved for attenuated positive symptom ratings and psychosis criteria, and modest harmonization for CHR-P criteria. The semi-structured interview, named Positive SYmptoms and Diagnostic Criteria for the CAARMS Harmonized with the SIPS (PSYCHS), generates CHR-P criteria and severity scores for both CAARMS and SIPS. CONCLUSIONS: Using the PSYCHS for CHR-P ascertainment, conversion determination, and attenuated positive symptom severity rating will help in comparing findings across studies and in meta-analyses.

Accelerating Medicines Partnership® Schizophrenia (AMP® SCZ): Rationale and Study Design of the Largest Global Prospective Cohort Study of Clinical High Risk for Psychosis.

This article describes the rationale, aims, and methodology of the Accelerating Medicines Partnership® Schizophrenia (AMP® SCZ). This is the largest international collaboration to date that will develop algorithms to predict trajectories and outcomes of individuals at clinical high risk (CHR) for psychosis and to advance the development and use of novel pharmacological interventions for CHR individuals. We present a description of the participating research networks and the data processing analysis and coordination center, their processes for data harmonization across 43 sites from 13 participating countries (recruitment across North America, Australia, Europe, Asia, and South America), data flow and quality assessment processes, data analyses, and the transfer of data to the National Institute of Mental Health (NIMH) Data Archive (NDA) for use by the research community. In an expected sample of approximately 2000 CHR individuals and 640 matched healthy controls, AMP SCZ will collect clinical, environmental, and cognitive data along with multimodal biomarkers, including neuroimaging, electrophysiology, fluid biospecimens, speech and facial expression samples, novel measures derived from digital health technologies including smartphone-based daily surveys, and passive sensing as well as actigraphy. The study will investigate a range of clinical outcomes over a 2-year period, including transition to psychosis, remission or persistence of CHR status, attenuated positive symptoms, persistent negative symptoms, mood and anxiety symptoms, and psychosocial functioning. The global reach of AMP SCZ and its harmonized innovative methods promise to catalyze the development of new treatments to address critical unmet clinical and public health needs in CHR individuals.

Can MRI features predict clinically relevant hepatocellular carcinoma genetic subtypes?

PURPOSE: Recent studies in cancer genomics have revealed core drivers for hepatocellular carcinoma (HCC) pathogenesis. We aim to study whether MRI features can serve as non-invasive markers for the prediction of common genetic subtypes of HCC. METHODS: Sequencing of 447 cancer-implicated genes was performed on 43 pathology proven HCC from 42 patients, who underwent contrast-enhanced MRI followed by biopsy or resection. MRI features were retrospectively evaluated including tumor size, infiltrative tumor margin, diffusion restriction, arterial phase hyperenhancement, non-peripheral washout, enhancing capsule, peritumoral enhancement, tumor in vein, fat in mass, blood products in mass, cirrhosis and tumor heterogeneity. Fisher's exact test was used to correlate genetic subtypes with imaging features. Prediction performance using correlated MRI features for genetic subtype and inter-reader agreement were assessed. RESULTS: The two most prevalent genetic mutations were TP53 (13/43, 30%) and CTNNB1 (17/43, 40%). Tumors with TP53 mutation more often demonstrated an infiltrative tumor margin on MRI (p = 0.01); inter-reader agreement was almost perfect (kappa = 0.95). The CTNNB1 mutation was associated with peritumoral enhancement on MRI (p = 0.04), inter-reader agreement was substantial (kappa = 0.74). The MRI feature of an infiltrative tumor margin correlated with the TP53 mutation with accuracy, sensitivity, and specificity of 74.4%, 61.5% and 80.0%, respectively. Peritumoral enhancement correlated with the CTNNB1 mutation with accuracy, sensitivity, and specificity of 69.8%, 47.0% and 84.6%, respectively. CONCLUSION: An infiltrative tumor margin on MRI correlated with TP53 mutation and peritumoral enhancement correlated with CTNNB1 mutation in HCC. Absence of these MRI features are potential negative predictors of the respective HCC genetic subtypes that have implications for prognosis and treatment response.

A Tool-free Neuronavigation Method based on Single-view Hand Tracking.

This work presents a novel tool-free neuronavigation method that can be used with a single RGB commodity camera. Compared with freehand craniotomy placement methods, the proposed system is more intuitive and less error prone. The proposed method also has several advantages over standard neuronavigation platforms. First, it has a much lower cost, since it doesn't require the use of an optical tracking camera or electromagnetic field generator, which are typically the most expensive parts of a neuronavigation system, making it much more accessible. Second, it requires minimal setup, meaning that it can be performed at the bedside and in circumstances where using a standard neuronavigation system is impractical. Our system relies on machine-learning-based hand pose estimation that acts as a proxy for optical tool tracking, enabling a 3D-3D pre-operative to intra-operative registration. Qualitative assessment from clinical users showed that the concept is clinically relevant. Quantitative assessment showed that on average a target registration error (TRE) of 1.3cm can be achieved. Furthermore, the system is framework-agnostic, meaning that future improvements to hand-tracking frameworks would directly translate to a higher accuracy.

Learning Expected Appearances for Intraoperative Registration during Neurosurgery.

We present a novel method for intraoperative patient-to-image registration by learning Expected Appearances. Our method uses preoperative imaging to synthesize patient-specific expected views through a surgical microscope for a predicted range of transformations. Our method estimates the camera pose by minimizing the dissimilarity between the intraoperative 2D view through the optical microscope and the synthesized expected texture. In contrast to conventional methods, our approach transfers the processing tasks to the preoperative stage, reducing thereby the impact of low-resolution, distorted, and noisy intraoperative images, that often degrade the registration accuracy. We applied our method in the context of neuronavigation during brain surgery. We evaluated our approach on synthetic data and on retrospective data from 6 clinical cases. Our method outperformed state-of-the-art methods and achieved accuracies that met current clinical standards.

SlicerCBM: automatic framework for biomechanical analysis of the brain.

PURPOSE: Brain shift that occurs during neurosurgery disturbs the brain's anatomy. Prediction of the brain shift is essential for accurate localisation of the surgical target. Biomechanical models have been envisaged as a possible tool for such predictions. In this study, we created a framework to automate the workflow for predicting intra-operative brain deformations. METHODS: We created our framework by uniquely combining our meshless total Lagrangian explicit dynamics (MTLED) algorithm for computing soft tissue deformations, open-source software libraries and built-in functions within 3D Slicer, an open-source software package widely used for medical research. Our framework generates the biomechanical brain model from the pre-operative MRI, computes brain deformation using MTLED and outputs results in the form of predicted warped intra-operative MRI. RESULTS: Our framework is used to solve three different neurosurgical brain shift scenarios: craniotomy, tumour resection and electrode placement. We evaluated our framework using nine patients. The average time to construct a patient-specific brain biomechanical model was 3 min, and that to compute deformations ranged from 13 to 23 min. We performed a qualitative evaluation by comparing our predicted intra-operative MRI with the actual intra-operative MRI. For quantitative evaluation, we computed Hausdorff distances between predicted and actual intra-operative ventricle surfaces. For patients with craniotomy and tumour resection, approximately 95% of the nodes on the ventricle surfaces are within two times the original in-plane resolution of the actual surface determined from the intra-operative MRI. CONCLUSION: Our framework provides a broader application of existing solution methods not only in research but also in clinics. We successfully demonstrated the application of our framework by predicting intra-operative deformations in nine patients undergoing neurosurgical procedures.

Unified Brain MR-Ultrasound Synthesis using Multi-Modal Hierarchical Representations.

We introduce MHVAE, a deep hierarchical variational autoencoder (VAE) that synthesizes missing images from various modalities. Extending multi-modal VAEs with a hierarchical latent structure, we introduce a probabilistic formulation for fusing multi-modal images in a common latent representation while having the flexibility to handle incomplete image sets as input. Moreover, adversarial learning is employed to generate sharper images. Extensive experiments are performed on the challenging problem of joint intra-operative ultrasound (iUS) and Magnetic Resonance (MR) synthesis. Our model outperformed multi-modal VAEs, conditional GANs, and the current state-of-the-art unified method (ResViT) for synthesizing missing images, demonstrating the advantage of using a hierarchical latent representation and a principled probabilistic fusion operation. Our code is publicly available.