Efficient Photoacoustic Image Synthesis with Deep Learning.

Photoacoustic imaging potentially allows for the real-time visualization of functional human tissue parameters such as oxygenation but is subject to a challenging underlying quantification problem. While in silico studies have revealed the great potential of deep learning (DL) methodology in solving this problem, the inherent lack of an efficient gold standard method for model training and validation remains a grand challenge. This work investigates whether DL can be leveraged to accurately and efficiently simulate photon propagation in biological tissue, enabling photoacoustic image synthesis. Our approach is based on estimating the initial pressure distribution of the photoacoustic waves from the underlying optical properties using a back-propagatable neural network trained on synthetic data. In proof-of-concept studies, we validated the performance of two complementary neural network architectures, namely a conventional U-Net-like model and a Fourier Neural Operator (FNO) network. Our in silico validation on multispectral human forearm images shows that DL methods can speed up image generation by a factor of 100 when compared to Monte Carlo simulations with 5×108 photons. While the FNO is slightly more accurate than the U-Net, when compared to Monte Carlo simulations performed with a reduced number of photons (5×106), both neural network architectures achieve equivalent accuracy. In contrast to Monte Carlo simulations, the proposed DL models can be used as inherently differentiable surrogate models in the photoacoustic image synthesis pipeline, allowing for back-propagation of the synthesis error and gradient-based optimization over the entire pipeline. Due to their efficiency, they have the potential to enable large-scale training data generation that can expedite the clinical application of photoacoustic imaging.

Photoacoustic Quantification of Tissue Oxygenation Using Conditional Invertible Neural Networks.

Intelligent systems in interventional healthcare depend on the reliable perception of the environment. In this context, photoacoustic tomography (PAT) has emerged as a non-invasive, functional imaging modality with great clinical potential. Current research focuses on converting the high-dimensional, not human-interpretable spectral data into the underlying functional information, specifically the blood oxygenation. One of the largely unexplored issues stalling clinical advances is the fact that the quantification problem is ambiguous, i.e. that radically different tissue parameter configurations could lead to almost identical photoacoustic spectra. In the present work, we tackle this problem with conditional Invertible Neural Networks (cINNs). Going beyond traditional point estimates, our network is used to compute an approximation of the conditional posterior density of tissue parameters given the photoacoustic spectrum. To this end, an automatic mode detection algorithm extracts the plausible solution from the sample-based posterior. According to a comprehensive validation study based on both synthetic and real images, our approach is well-suited for exploring ambiguity in quantitative PAT.

The SPECTRAL Perfusion Arm Clamping dAtaset (SPECTRALPACA) for video-rate functional imaging of the skin.

Spectral imaging has the potential to become a key technique in interventional medicine as it unveils much richer optical information compared to conventional RBG (red, green, and blue)-based imaging. Thus allowing for high-resolution functional tissue analysis in real time. Its higher information density particularly shows promise for the development of powerful perfusion monitoring methods for clinical use. However, even though in vivo validation of such methods is crucial for their clinical translation, the biomedical field suffers from a lack of publicly available datasets for this purpose. Closing this gap, we generated the SPECTRAL Perfusion Arm Clamping dAtaset (SPECTRALPACA). It comprises ten spectral videos (∼20 Hz, approx. 20,000 frames each) systematically recorded of the hands of ten healthy human participants in different functional states. We paired each spectral video with concisely tracked regions of interest, and corresponding diffuse reflectance measurements recorded with a spectrometer. Providing the first openly accessible in human spectral video dataset for perfusion monitoring, our work facilitates the development and validation of new functional imaging methods.

Miniaturized electromagnetic tracking enables efficient ultrasound-navigated needle insertions.

Ultrasound (US) has gained popularity as a guidance modality for percutaneous needle insertions because it is widely available and non-ionizing. However, coordinating scanning and needle insertion still requires significant experience. Current assistance solutions utilize optical or electromagnetic tracking (EMT) technology directly integrated into the US device or probe. This results in specialized devices or introduces additional hardware, limiting the ergonomics of both the scanning and insertion process. We developed the first ultrasound (US) navigation solution designed to be used as a non-permanent accessory for existing US devices while maintaining the ergonomics during the scanning process. A miniaturized EMT source is reversibly attached to the US probe, temporarily creating a combined modality that provides real-time anatomical imaging and instrument tracking at the same time. Studies performed with 11 clinical operators show that the proposed navigation solution can guide needle insertions with a targeting accuracy of about 5 mm, which is comparable to existing approaches and unaffected by repeated attachment and detachment of the miniaturized tracking solution. The assistance proved particularly helpful for non-expert users and needle insertions performed outside of the US plane. The small size and reversible attachability of the proposed navigation solution promises streamlined integration into the clinical workflow and widespread access to US navigated punctures.

Standard error estimation in meta-analysis of studies reporting medians.

We consider the setting of an aggregate data meta-analysis of a continuous outcome of interest. When the distribution of the outcome is skewed, it is often the case that some primary studies report the sample mean and standard deviation of the outcome and other studies report the sample median along with the first and third quartiles and/or minimum and maximum values. To perform meta-analysis in this context, a number of approaches have recently been developed to impute the sample mean and standard deviation from studies reporting medians. Then, standard meta-analytic approaches with inverse-variance weighting are applied based on the (imputed) study-specific sample means and standard deviations. In this article, we illustrate how this common practice can severely underestimate the within-study standard errors, which results in poor coverage for the pooled mean in common effect meta-analyses and overestimation of between-study heterogeneity in random effects meta-analyses. We propose a straightforward bootstrap approach to estimate the standard errors of the imputed sample means. Our simulation study illustrates how the proposed approach can improve the estimation of the within-study standard errors and consequently improve coverage for the pooled mean in common effect meta-analyses and estimation of between-study heterogeneity in random effects meta-analyses. Moreover, we apply the proposed approach in a meta-analysis to identify risk factors of a severe course of COVID-19.

Spectral imaging enables contrast agent-free real-time ischemia monitoring in laparoscopic surgery.

Laparoscopic surgery has evolved as a key technique for cancer diagnosis and therapy. While characterization of the tissue perfusion is crucial in various procedures, such as partial nephrectomy, doing so by means of visual inspection remains highly challenging. We developed a laparoscopic real-time multispectral imaging system featuring a compact and lightweight multispectral camera and the possibility to complement the conventional surgical view of the patient with functional information at a video rate of 25 Hz. To enable contrast agent-free ischemia monitoring during laparoscopic partial nephrectomy, we phrase the problem of ischemia detection as an out-of-distribution detection problem that does not rely on data from any other patient and uses an ensemble of invertible neural networks at its core. An in-human trial demonstrates the feasibility of our approach and highlights the potential of spectral imaging combined with advanced deep learning-based analysis tools for fast, efficient, reliable, and safe functional laparoscopic imaging.

Photoacoustic image synthesis with generative adversarial networks.

Photoacoustic tomography (PAT) has the potential to recover morphological and functional tissue properties with high spatial resolution. However, previous attempts to solve the optical inverse problem with supervised machine learning were hampered by the absence of labeled reference data. While this bottleneck has been tackled by simulating training data, the domain gap between real and simulated images remains an unsolved challenge. We propose a novel approach to PAT image synthesis that involves subdividing the challenge of generating plausible simulations into two disjoint problems: (1) Probabilistic generation of realistic tissue morphology, and (2) pixel-wise assignment of corresponding optical and acoustic properties. The former is achieved with Generative Adversarial Networks (GANs) trained on semantically annotated medical imaging data. According to a validation study on a downstream task our approach yields more realistic synthetic images than the traditional model-based approach and could therefore become a fundamental step for deep learning-based quantitative PAT (qPAT).

Semantic segmentation of multispectral photoacoustic images using deep learning.

Photoacoustic (PA) imaging has the potential to revolutionize functional medical imaging in healthcare due to the valuable information on tissue physiology contained in multispectral photoacoustic measurements. Clinical translation of the technology requires conversion of the high-dimensional acquired data into clinically relevant and interpretable information. In this work, we present a deep learning-based approach to semantic segmentation of multispectral photoacoustic images to facilitate image interpretability. Manually annotated photoacoustic and ultrasound imaging data are used as reference and enable the training of a deep learning-based segmentation algorithm in a supervised manner. Based on a validation study with experimentally acquired data from 16 healthy human volunteers, we show that automatic tissue segmentation can be used to create powerful analyses and visualizations of multispectral photoacoustic images. Due to the intuitive representation of high-dimensional information, such a preprocessing algorithm could be a valuable means to facilitate the clinical translation of photoacoustic imaging.

Reattachable fiducial skin marker for automatic multimodality registration.

PURPOSE: Fusing image information has become increasingly important for optimal diagnosis and treatment of the patient. Despite intensive research towards markerless registration approaches, fiducial marker-based methods remain the default choice for a wide range of applications in clinical practice. However, as especially non-invasive markers cannot be positioned reproducibly in the same pose on the patient, pre-interventional imaging has to be performed immediately before the intervention for fiducial marker-based registrations. METHODS: We propose a new non-invasive, reattachable fiducial skin marker concept for multi-modal registration approaches including the use of electromagnetic or optical tracking technologies. We furthermore describe a robust, automatic fiducial marker localization algorithm for computed tomography (CT) and magnetic resonance imaging (MRI) images. Localization of the new fiducial marker has been assessed for different marker configurations using both CT and MRI. Furthermore, we applied the marker in an abdominal phantom study. For this, we attached the marker at three poses to the phantom, registered ten segmented targets of the phantom's CT image to live ultrasound images and determined the target registration error (TRE) for each target and each marker pose. RESULTS: Reattachment of the marker was possible with a mean precision of 0.02 mm ± 0.01 mm. Our algorithm successfully localized the marker automatically in all ([Formula: see text]) evaluated CT/MRI images. Depending on the marker pose, the mean ([Formula: see text]) TRE of the abdominal phantom study ranged from 1.51 ± 0.75 mm to 4.65 ± 1.22 mm. CONCLUSIONS: The non-invasive, reattachable skin marker concept allows reproducible positioning of the marker and automatic localization in different imaging modalities. The low TREs indicate the potential applicability of the marker concept for clinical interventions, such as the puncture of abdominal lesions, where current image-based registration approaches still lack robustness and existing marker-based methods are often impractical.

SIMPA: an open-source toolkit for simulation and image processing for photonics and acoustics.

SIGNIFICANCE: Optical and acoustic imaging techniques enable noninvasive visualisation of structural and functional properties of tissue. The quantification of measurements, however, remains challenging due to the inverse problems that must be solved. Emerging data-driven approaches are promising, but they rely heavily on the presence of high-quality simulations across a range of wavelengths due to the lack of ground truth knowledge of tissue acoustical and optical properties in realistic settings. AIM: To facilitate this process, we present the open-source simulation and image processing for photonics and acoustics (SIMPA) Python toolkit. SIMPA is being developed according to modern software design standards. APPROACH: SIMPA enables the use of computational forward models, data processing algorithms, and digital device twins to simulate realistic images within a single pipeline. SIMPA's module implementations can be seamlessly exchanged as SIMPA abstracts from the concrete implementation of each forward model and builds the simulation pipeline in a modular fashion. Furthermore, SIMPA provides comprehensive libraries of biological structures, such as vessels, as well as optical and acoustic properties and other functionalities for the generation of realistic tissue models. RESULTS: To showcase the capabilities of SIMPA, we show examples in the context of photoacoustic imaging: the diversity of creatable tissue models, the customisability of a simulation pipeline, and the degree of realism of the simulations. CONCLUSIONS: SIMPA is an open-source toolkit that can be used to simulate optical and acoustic imaging modalities. The code is available at: https://github.com/IMSY-DKFZ/simpa, and all of the examples and experiments in this paper can be reproduced using the code available at: https://github.com/IMSY-DKFZ/simpa_paper_experiments.

Robust hand tracking for surgical telestration.

PURPOSE: As human failure has been shown to be one primary cause for post-operative death, surgical training is of the utmost socioeconomic importance. In this context, the concept of surgical telestration has been introduced to enable experienced surgeons to efficiently and effectively mentor trainees in an intuitive way. While previous approaches to telestration have concentrated on overlaying drawings on surgical videos, we explore the augmented reality (AR) visualization of surgical hands to imitate the direct interaction with the situs. METHODS: We present a real-time hand tracking pipeline specifically designed for the application of surgical telestration. It comprises three modules, dedicated to (1) the coarse localization of the expert's hand and the subsequent (2) segmentation of the hand for AR visualization in the field of view of the trainee and (3) regression of keypoints making up the hand's skeleton. The semantic representation is obtained to offer the ability for structured reporting of the motions performed as part of the teaching. RESULTS: According to a comprehensive validation based on a large data set comprising more than 14,000 annotated images with varying application-relevant conditions, our algorithm enables real-time hand tracking and is sufficiently accurate for the task of surgical telestration. In a retrospective validation study, a mean detection accuracy of 98%, a mean keypoint regression accuracy of 10.0 px and a mean Dice Similarity Coefficient of 0.95 were achieved. In a prospective validation study, it showed uncompromised performance when the sensor, operator or gesture varied. CONCLUSION: Due to its high accuracy and fast inference time, our neural network-based approach to hand tracking is well suited for an AR approach to surgical telestration. Future work should be directed to evaluating the clinical value of the approach.

A Platform and Multisided Market for Translational, Software-Defined Medical Procedures in the Operating Room (OP 4.1): Proof-of-Concept Study.

BACKGROUND: Although digital and data-based technologies are widespread in various industries in the context of Industry 4.0, the use of smart connected devices in health care is still in its infancy. Innovative solutions for the medical environment are affected by difficult access to medical device data and high barriers to market entry because of proprietary systems. OBJECTIVE: In the proof-of-concept project OP 4.1, we show the business viability of connecting and augmenting medical devices and data through software add-ons by giving companies a technical and commercial platform for the development, implementation, distribution, and billing of innovative software solutions. METHODS: The creation of a central platform prototype requires the collaboration of several independent market contenders, including medical users, software developers, medical device manufacturers, and platform providers. A dedicated consortium of clinical and scientific partners as well as industry partners was set up. RESULTS: We demonstrate the successful development of the prototype of a user-centric, open, and extensible platform for the intelligent support of processes starting with the operating room. By connecting heterogeneous data sources and medical devices from different manufacturers and making them accessible for software developers and medical users, the cloud-based platform OP 4.1 enables the augmentation of medical devices and procedures through software-based solutions. The platform also allows for the demand-oriented billing of apps and medical devices, thus permitting software-based solutions to fast-track their economic development and become commercially successful. CONCLUSIONS: The technology and business platform OP 4.1 creates a multisided market for the successful development, implementation, distribution, and billing of new software solutions in the operating room and in the health care sector in general. Consequently, software-based medical innovation can be translated into clinical routine quickly, efficiently, and cost-effectively, optimizing the treatment of patients through smartly assisted procedures.

Surgical data science - from concepts toward clinical translation.

Recent developments in data science in general and machine learning in particular have transformed the way experts envision the future of surgery. Surgical Data Science (SDS) is a new research field that aims to improve the quality of interventional healthcare through the capture, organization, analysis and modeling of data. While an increasing number of data-driven approaches and clinical applications have been studied in the fields of radiological and clinical data science, translational success stories are still lacking in surgery. In this publication, we shed light on the underlying reasons and provide a roadmap for future advances in the field. Based on an international workshop involving leading researchers in the field of SDS, we review current practice, key achievements and initiatives as well as available standards and tools for a number of topics relevant to the field, namely (1) infrastructure for data acquisition, storage and access in the presence of regulatory constraints, (2) data annotation and sharing and (3) data analytics. We further complement this technical perspective with (4) a review of currently available SDS products and the translational progress from academia and (5) a roadmap for faster clinical translation and exploitation of the full potential of SDS, based on an international multi-round Delphi process.

Computer-assisted trajectory planning for percutaneous needle insertions.

PURPOSE: Computed tomography (CT) guided minimally invasive interventions such as biopsies or ablation therapies often involve insertion of a needle-shaped instrument into the target organ (e.g., the liver). Today, these interventions still require manual planning of a suitable trajectory to the target (e.g., the tumor) based on the slice data provided by the imaging modality. However, taking into account the critical structures and other parameters crucial to the success of the intervention--such as instrument shape and penetration angle--is challenging and requires a lot of experience. METHODS: To overcome these problems, we present a system for the automatic or semiautomatic planning of optimal trajectories to a target, based on 3D reconstructions of all relevant structures. The system determines possible insertion zones based on so-called hard constraints and rates the quality of these zones by so-called soft constraints. The concept of pareto optimality is utilized to allow for a weight-independent proposal of insertion trajectories. In order to demonstrate the benefits of our method, automatic trajectory planning was applied retrospectively to n = 10 data sets from interventions in which complications occurred. RESULTS: The efficient (graphics processing unit-based) implementation of the constraints results in a mean overall planning time of about 9 s. The examined trajectories, originally chosen by the physician, have been rated as follows: in six cases, the insertion point was labeled invalid by the planning system. For two cases, the system would have proposed points with a better rating according to the soft constraints. For the remaining two cases the system would have indicated poor rating with respect to one of the soft constraints. The paths proposed by our system were rated feasible and qualitatively good by experienced interventional radiologists. CONCLUSIONS: The proposed computer-assisted trajectory planning system is able to detect unsafe and propose safe insertion trajectories and may especially be helpful for interventional radiologist at the beginning or during their interventional training.

Can we predict the severe course of COVID-19 - a systematic review and meta-analysis of indicators of clinical outcome?

BACKGROUND: COVID-19 has been reported in over 40million people globally with variable clinical outcomes. In this systematic review and meta-analysis, we assessed demographic, laboratory and clinical indicators as predictors for severe courses of COVID-19. METHODS: This systematic review was registered at PROSPERO under CRD42020177154. We systematically searched multiple databases (PubMed, Web of Science Core Collection, MedRvix and bioRvix) for publications from December 2019 to May 31st 2020. Random-effects meta-analyses were used to calculate pooled odds ratios and differences of medians between (1) patients admitted to ICU versus non-ICU patients and (2) patients who died versus those who survived. We adapted an existing Cochrane risk-of-bias assessment tool for outcome studies. RESULTS: Of 6,702 unique citations, we included 88 articles with 69,762 patients. There was concern for bias across all articles included. Age was strongly associated with mortality with a difference of medians (DoM) of 13.15 years (95% confidence interval (CI) 11.37 to 14.94) between those who died and those who survived. We found a clinically relevant difference between non-survivors and survivors for C-reactive protein (CRP; DoM 69.10 mg/L, CI 50.43 to 87.77), lactate dehydrogenase (LDH; DoM 189.49 U/L, CI 155.00 to 223.98), cardiac troponin I (cTnI; DoM 21.88 pg/mL, CI 9.78 to 33.99) and D-Dimer (DoM 1.29mg/L, CI 0.9 to 1.69). Furthermore, cerebrovascular disease was the co-morbidity most strongly associated with mortality (Odds Ratio 3.45, CI 2.42 to 4.91) and ICU admission (Odds Ratio 5.88, CI 2.35 to 14.73). DISCUSSION: This comprehensive meta-analysis found age, cerebrovascular disease, CRP, LDH and cTnI to be the most important risk-factors that predict severe COVID-19 outcomes and will inform clinical scores to support early decision-making.

Tattoo tomography: Freehand 3D photoacoustic image reconstruction with an optical pattern.

PURPOSE: Photoacoustic tomography (PAT) is a novel imaging technique that can spatially resolve both morphological and functional tissue properties, such as vessel topology and tissue oxygenation. While this capacity makes PAT a promising modality for the diagnosis, treatment, and follow-up of various diseases, a current drawback is the limited field of view provided by the conventionally applied 2D probes. METHODS: In this paper, we present a novel approach to 3D reconstruction of PAT data (Tattoo tomography) that does not require an external tracking system and can smoothly be integrated into clinical workflows. It is based on an optical pattern placed on the region of interest prior to image acquisition. This pattern is designed in a way that a single tomographic image of it enables the recovery of the probe pose relative to the coordinate system of the pattern, which serves as a global coordinate system for image compounding. RESULTS: To investigate the feasibility of Tattoo tomography, we assessed the quality of 3D image reconstruction with experimental phantom data and in vivo forearm data. The results obtained with our prototype indicate that the Tattoo method enables the accurate and precise 3D reconstruction of PAT data and may be better suited for this task than the baseline method using optical tracking. CONCLUSIONS: In contrast to previous approaches to 3D ultrasound (US) or PAT reconstruction, the Tattoo approach neither requires complex external hardware nor training data acquired for a specific application. It could thus become a valuable tool for clinical freehand PAT.

Can we predict the severe course of COVID-19 - a systematic review and meta-analysis of indicators of clinical outcome?

Background: COVID-19 has been reported in over 40million people globally with variable clinical outcomes. In this systematic review and meta-analysis, we assessed demographic, laboratory and clinical indicators as predictors for severe courses of COVID-19. Methods: We systematically searched multiple databases (PubMed, Web of Science Core Collection, MedRvix and bioRvix) for publications from December 2019 to May 31st 2020. Random-effects meta-analyses were used to calculate pooled odds ratios and differences of medians between (1) patients admitted to ICU versus non-ICU patients and (2) patients who died versus those who survived. We adapted an existing Cochrane risk-of-bias assessment tool for outcome studies. Results: Of 6,702 unique citations, we included 88 articles with 69,762 patients. There was concern for bias across all articles included. Age was strongly associated with mortality with a difference of medians (DoM) of 13.15 years (95% confidence interval (CI) 11.37 to 14.94) between those who died and those who survived. We found a clinically relevant difference between non-survivors and survivors for C-reactive protein (CRP; DoM 69.10, CI 50.43 to 87.77), lactate dehydrogenase (LDH; DoM 189.49, CI 155.00 to 223.98), cardiac troponin I (cTnI; DoM 21.88, CI 9.78 to 33.99) and D-Dimer (DoM 1.29mg/L, CI 0.9 - 1.69). Furthermore, cerebrovascular disease was the co-morbidity most strongly associated with mortality (Odds Ratio 3.45, CI 2.42 to 4.91) and ICU admission (Odds Ratio 5.88, CI 2.35 to 14.73). Discussion: This comprehensive meta-analysis found age, cerebrovascular disease, CRP, LDH and cTnI to be the most important risk-factors in predicting severe COVID-19 outcomes and will inform decision analytical tools to support clinical decision-making.

Polhemus EM tracked Micro Sensor for CT-guided interventions.

PURPOSE: Electromagnetic (EM) tracking is a key technology in image-guided therapy. A new EM Micro Sensor was presented by Polhemus Inc.; it is the first to enable localization of medical instruments through their trackers. Different field generators (FGs) are available by Polhemus, one being almost as small as a sugar cube. As accuracy and robustness of tracking are known challenges to using EM trackers in clinical environments, the goal of this study was a standardized assessment of the Micro Sensor in both a laboratory (lab) and a computed tomography (CT) environment. METHODS: The Micro Sensor was assessed by means of Hummel et al.'s standardized protocol; it was assessed in conjunction with a Polhemus Liberty tracker and three FGs - with edge lengths of 1 (TX1), 2 (TX2), and 4 (TX4) inches. Precision as well as positional and rotational accuracy were determined in a lab and a CT suite. Distortions by four different metallic cylinders and tracking of two typical medical instruments - a hypodermic needle and a flexible endoscope - were also tested. RESULTS: A jitter of 0.02 mm or less was found for all FGs in the different environments, except for the TX2 FG for which no valid data could be obtained in the CT. Errors of 5 cm distance measurements were 0.6 mm or less for all FGs in the lab. While the distance errors of the TX1 FG were only slightly increased up to 1.6 mm in the CT, those of the TX4 FG were found to be up to around 10% of the measured distance (5.4 mm on average). The mean orientation error was found to be 0.9° /0.5° /0.1° for the TX4/TX2/TX1 FG in the lab. In the CT environment, rotation errors were in the same range: less than 1.2° /0.1° for the TX4/TX1 FG. Deviation under the presence of metallic cylinders stayed below 1 mm in most cases. Precision and orientational accuracy do not seem to be affected by instrument tracking and stayed in the same range as for the other measurements whereas distance errors were slightly increased up to 1.7 mm. CONCLUSION: This study shows that accurate tracking of medical instruments is possible with the new Micro Sensor; it demonstrated a jitter of 0.01 mm or less, position errors below 2 mm, and rotation errors of less than 0.3° . As with other EM trackers, errors increase when large tracking volumes with ranges of up to 50 cm are required in clinical environments. For smaller tracking volumes with ranges of up to 15 cm, a high accuracy and robustness was found. This is interesting especially for the TX1 FG which can easily be placed in close vicinity to the region of interest.

In vivo accuracy assessment of a needle-based navigation system for CT-guided radiofrequency ablation of the liver.

Computed tomography (CT)-guided percutaneous radiofrequency ablation (RFA) has become a commonly used procedure in the treatment of liver tumors. One of the main challenges related to the method is the exact placement of the instrument within the lesion. To address this issue, a system was developed for computer-assisted needle placement which uses a set of fiducial needles to compensate for organ motion in real time. The purpose of this study was to assess the accuracy of the system in vivo. Two medical experts with experience in CT-guided interventions and two nonexperts used the navigation system to perform 32 needle insertions into contrasted agar nodules injected into the livers of two ventilated swine. Skin-to-target path planning and real-time needle guidance were based on preinterventional 1 mm CT data slices. The lesions were hit in 97% of all trials with a mean user error of 2.4 +/- 2.1 mm, a mean target registration error (TRE) of 2.1 +/- 1.1 mm, and a mean overall targeting error of 3.7 +/- 2.3 mm. The nonexperts achieved significantly better results than the experts with an overall error of 2.8 +/- 1.4 mm (n=16) compared to 4.5 +/- 2.7 mm (n=16). The mean time for performing four needle insertions based on one preinterventional planning CT was 57 +/- 19 min with a mean setup time of 27 min, which includes the steps fiducial insertion (24 +/- 15 min), planning CT acquisition (1 +/- 0 min), and registration (2 +/- 1 min). The mean time for path planning and targeting was 5 +/- 4 and 2 +/- 1 min, respectively. Apart from the fiducial insertion step, experts and nonexperts performed comparably fast. It is concluded that the system allows for accurate needle placement into hepatic tumors based on one planning CT and could thus enable considerable improvement to the clinical treatment standard for RFA procedures and other CT-guided interventions in the liver. To support clinical application of the method, optimization of individual system modules to reduce intervention time is proposed.

Clickstream Analysis for Crowd-Based Object Segmentation with Confidence.

With the rapidly increasing interest in machine learning based solutions for automatic image annotation, the availability of reference annotations for algorithm training is one of the major bottlenecks in the field. Crowdsourcing has evolved as a valuable option for low-cost and large-scale data annotation; however, quality control remains a major issue which needs to be addressed. To our knowledge, we are the first to analyze the annotation process to improve crowd-sourced image segmentation. Our method involves training a regressor to estimate the quality of a segmentation from the annotator's clickstream data. The quality estimation can be used to identify spam and weight individual annotations by their (estimated) quality when merging multiple segmentations of one image. Using a total of 29,000 crowd annotations performed on publicly available data of different object classes, we show that (1) our method is highly accurate in estimating the segmentation quality based on clickstream data, (2) outperforms state-of-the-art methods for merging multiple annotations. As the regressor does not need to be trained on the object class that it is applied to it can be regarded as a low-cost option for quality control and confidence analysis in the context of crowd-based image annotation.