AR visualizations in laparoscopy: surgeon preferences and depth assessment of vascular anatomy.

Introduction: This study compares five augmented reality (AR) vasculature visualization techniques in a mixed-reality laparoscopy simulator with 50 medical professionals and analyzes their impact on the surgeon. Material and methods: ​​The different visualization techniques' abilities to convey depth were measured using the participant's accuracy in an objective depth sorting task. Demographic data and subjective measures, such as the preference of each AR visualization technique and potential application areas, were collected with questionnaires. Results: Despite measuring differences in objective measurements across the visualization techniques, they were not statistically significant. In the subjective measures, however, 55% of the participants rated visualization technique II, 'Opaque with single-color Fresnel highlights', as their favorite. Participants felt that AR could be useful for various surgeries, especially complex surgeries (100%). Almost all participants agreed that AR could potentially improve surgical parameters, such as patient safety (88%), complication rate (84%), and identifying risk structures (96%). Conclusions: More studies are needed on the effect of different visualizations on task performance, as well as more sophisticated and effective visualization techniques for the operating room. With the findings of this study, we encourage the development of new study setups to advance surgical AR.

Whole-body uptake classification and prostate cancer staging in 68Ga-PSMA-11 PET/CT using dual-tracer learning.

PURPOSE: In PSMA-ligand PET/CT imaging, standardized evaluation frameworks and image-derived parameters are increasingly used to support prostate cancer staging. Clinical applicability remains challenging wherever manual measurements of numerous suspected lesions are required. Deep learning methods are promising for automated image analysis, typically requiring extensive expert-annotated image datasets to reach sufficient accuracy. We developed a deep learning method to support image-based staging, investigating the use of training information from two radiotracers. METHODS: In 173 subjects imaged with 68Ga-PSMA-11 PET/CT, divided into development (121) and test (52) sets, we trained and evaluated a convolutional neural network to both classify sites of elevated tracer uptake as nonsuspicious or suspicious for cancer and assign them an anatomical location. We evaluated training strategies to leverage information from a larger dataset of 18F-FDG PET/CT images and expert annotations, including transfer learning and combined training encoding the tracer type as input to the network. We assessed the agreement between the N and M stage assigned based on the network annotations and expert annotations, according to the PROMISE miTNM framework. RESULTS: In the development set, including 18F-FDG training data improved classification performance in four-fold cross validation. In the test set, compared to expert assessment, training with 18F-FDG data and the development set yielded 80.4% average precision [confidence interval (CI): 71.1-87.8] for identification of suspicious uptake sites, 77% (CI: 70.0-83.4) accuracy for anatomical location classification of suspicious findings, 81% agreement for identification of regional lymph node involvement, and 77% agreement for identification of metastatic stage. CONCLUSION: The evaluated algorithm showed good agreement with expert assessment for identification and anatomical location classification of suspicious uptake sites in whole-body 68Ga-PSMA-11 PET/CT. With restricted PSMA-ligand data available, the use of training examples from a different radiotracer improved performance. The investigated methods are promising for enabling efficient assessment of cancer stage and tumor burden.

Reducing instability of inter-subject covariance of FDG uptake networks using structure-weighted sparse estimation approach.

PURPOSE: Sparse inverse covariance estimation (SICE) is increasingly utilized to estimate inter-subject covariance of FDG uptake (FDGcov) as proxy of metabolic brain connectivity. However, this statistical method suffers from the lack of robustness in the connectivity estimation. Patterns of FDGcov were observed to be spatially similar with patterns of structural connectivity as obtained from DTI imaging. Based on this similarity, we propose to regularize the sparse estimation of FDGcov using the structural connectivity. METHODS: We retrospectively analyzed the FDG-PET and DTI data of 26 healthy controls, 41 patients with Alzheimer's disease (AD), and 30 patients with frontotemporal lobar degeneration (FTLD). Structural connectivity matrix derived from DTI data was introduced as a regularization parameter to assign individual penalties to each potential metabolic connectivity. Leave-one-out cross validation experiments were performed to assess the differential diagnosis ability of structure weighted SICE approach. A few approaches of structure weighted were compared with the standard SICE. RESULTS: Compared to the standard SICE, structural weighting has shown more stable performance in the supervised classification, especially in the differentiation AD vs. FTLD (accuracy of 89-90%, while unweighted SICE only 85%). There was a significant positive relationship between the minimum number of metabolic connection and the robustness of the classification accuracy (r = 0.57, P < 0.001). Shuffling experiments showed significant differences between classification score derived with true structural weighting and those obtained by randomized structure (P < 0.05). CONCLUSION: The structure-weighted sparse estimation can enhance the robustness of metabolic connectivity, which may consequently improve the differentiation of pathological phenotypes.

The added value of PSMA PET/MR radiomics for prostate cancer staging.

PURPOSE: To evaluate the performance of combined PET and multiparametric MRI (mpMRI) radiomics for the group-wise prediction of postsurgical Gleason scores (psGSs) in primary prostate cancer (PCa) patients. METHODS: Patients with PCa, who underwent [68 Ga]Ga-PSMA-11 PET/MRI followed by radical prostatectomy, were included in this retrospective analysis (n = 101). Patients were grouped by psGS in three categories: ISUP grades 1-3, ISUP grade 4, and ISUP grade 5. mpMRI images included T1-weighted, T2-weighted, and apparent diffusion coefficient (ADC) map. Whole-prostate segmentations were performed on each modality, and image biomarker standardization initiative (IBSI)-compliant radiomic features were extracted. Nine support vector machine (SVM) models were trained: four single-modality radiomic models (PET, T1w, T2w, ADC); three PET + MRI double-modality models (PET + T1w, PET + T2w, PET + ADC), and two baseline models (one with patient data, one image-based) for comparison. A sixfold stratified cross-validation was performed, and balanced accuracies (bAcc) of the predictions of the best-performing models were reported and compared through Student's t-tests. The predictions of the best-performing model were compared against biopsy GS (bGS). RESULTS: All radiomic models outperformed the baseline models. The best-performing (mean ± stdv [%]) single-modality model was the ADC model (76 ± 6%), although not significantly better (p > 0.05) than other single-modality models (T1w: 72 ± 3%, T2w: 73 ± 2%; PET: 75 ± 5%). The overall best-performing model combined PET + ADC radiomics (82 ± 5%). It significantly outperformed most other double-modality (PET + T1w: 74 ± 5%, p = 0.026; PET + T2w: 71 ± 4%, p = 0.003) and single-modality models (PET: p = 0.042; T1w: p = 0.002; T2w: p = 0.003), except the ADC-only model (p = 0.138). In this initial cohort, the PET + ADC model outperformed bGS overall (82.5% vs 72.4%) in the prediction of psGS. CONCLUSION: All single- and double-modality models outperformed the baseline models, showing their potential in the prediction of GS, even with an unbalanced cohort. The best-performing model included PET + ADC radiomics, suggesting a complementary value of PSMA-PET and ADC radiomics.

PLAFOKON: a new concept for a patient-individual and intervention-specific flexible surgical platform.

BACKGROUND: Research in the field of surgery is mainly driven by aiming for trauma reduction as well as for personalized treatment concepts. Beyond laparoscopy, other proposed approaches for further reduction of the therapeutic trauma have failed to achieve clinical translation, with few notable exceptions. We believe that this is mainly due to a lack of flexibility and high associated costs. We aimed at addressing these issues by developing a novel minimally invasive operating platform and a preoperative design workflow for patient-individual adaptation and cost-effective rapid manufacturing of surgical manipulators. In this article, we report on the first in-vitro cholecystectomy performed with our operating platform. METHODS: The single-port overtube (SPOT) is a snake-like surgical manipulator for minimally invasive interventions. The system layout is highly flexible and can be adapted in design and dimensions for different kinds of surgery, based on patient- and disease-specific parameters. For collecting and analyzing this data, we developed a graphical user interface, which assists clinicians during the preoperative planning phase. Other major components of our operating platform include an instrument management system and a non-sterile user interface. For the trial surgery, we used a validated phantom which was further equipped with a porcine liver including the gallbladder. RESULTS: Following our envisioned preoperative design workflow, a suitable geometry of the surgical manipulator was determined for our trial surgery and rapidly manufactured by means of 3D printing. With this setup, we successfully performed a first in-vitro cholecystectomy, which was completed in 78 min. CONCLUSIONS: By conducting the trial surgery, we demonstrated the effectiveness of our PLAFOKON operating platform. While some aspects - especially regarding usability and ergonomics - can be further optimized, the overall performance of the system is highly promising, with sufficient flexibility and strength for conducting the necessary tissue manipulations.

How molecular imaging will enable robotic precision surgery : The role of artificial intelligence, augmented reality, and navigation.

Molecular imaging is one of the pillars of precision surgery. Its applications range from early diagnostics to therapy planning, execution, and the accurate assessment of outcomes. In particular, molecular imaging solutions are in high demand in minimally invasive surgical strategies, such as the substantially increasing field of robotic surgery. This review aims at connecting the molecular imaging and nuclear medicine community to the rapidly expanding armory of surgical medical devices. Such devices entail technologies ranging from artificial intelligence and computer-aided visualization technologies (software) to innovative molecular imaging modalities and surgical navigation (hardware). We discuss technologies based on their role at different steps of the surgical workflow, i.e., from surgical decision and planning, over to target localization and excision guidance, all the way to (back table) surgical verification. This provides a glimpse of how innovations from the technology fields can realize an exciting future for the molecular imaging and surgery communities.

PET- and SPECT-based navigation strategies to advance procedural accuracy in interventional radiology and image-guided surgery.

INTRODUCTION: Nuclear medicine has a crucial role in interventional strategies where a combination between the increasing use of targeted radiotracers and intraprocedural detection modalities enable novel, but often complex, targeted procedures in both the fields of interventional radiology and surgery. 3D navigation approaches could assist the interventional radiologist or surgeon in such complex procedures. EVIDENCE ACQUISITION: This review aimed to provide a comprehensive overview of the current application of computer-assisted navigation strategies based on nuclear imaging to assist in interventional radiology and image-guided surgery. This work starts with a brief overview of the typical navigation workflow from a technical perspective, which is followed by the different clinical applications organized based on their anatomical organ of interest. EVIDENCE SYNTHESIS: Although many studies have proven the feasibility of PET- and SPECT-based navigation strategies for various clinical applications in both interventional radiology and surgery, the strategies are spread widely in both navigation workflows and clinical indications, evaluated in small patient groups. Hence, no golden standard has yet been established. CONCLUSIONS: Despite that the clinical outcome is yet to be determined in large patient cohorts, navigation seems to be a promising technology to translate nuclear medicine findings, provided by PET- and SPECT-based molecular imaging, to the intervention and operating room. Interventional Nuclear Medicine (iNM) has an exciting future to come using both PET- and SPECT-based navigation.

Deep Convolution Neural Network for Laryngeal Cancer Classification on Contact Endoscopy-Narrow Band Imaging.

(1) Background: Contact Endoscopy (CE) and Narrow Band Imaging (NBI) are optical imaging modalities that can provide enhanced and magnified visualization of the superficial vascular networks in the laryngeal mucosa. The similarity of vascular structures between benign and malignant lesions causes a challenge in the visual assessment of CE-NBI images. The main objective of this study is to use Deep Convolutional Neural Networks (DCNN) for the automatic classification of CE-NBI images into benign and malignant groups with minimal human intervention. (2) Methods: A pretrained Res-Net50 model combined with the cut-off-layer technique was selected as the DCNN architecture. A dataset of 8181 CE-NBI images was used during the fine-tuning process in three experiments where several models were generated and validated. The accuracy, sensitivity, and specificity were calculated as the performance metrics in each validation and testing scenario. (3) Results: Out of a total of 72 trained and tested models in all experiments, Model 5 showed high performance. This model is considerably smaller than the full ResNet50 architecture and achieved the testing accuracy of 0.835 on the unseen data during the last experiment. (4) Conclusion: The proposed fine-tuned ResNet50 model showed a high performance to classify CE-NBI images into the benign and malignant groups and has the potential to be part of an assisted system for automatic laryngeal cancer detection.

CAI4CAI: The Rise of Contextual Artificial Intelligence in Computer Assisted Interventions.

Data-driven computational approaches have evolved to enable extraction of information from medical images with a reliability, accuracy and speed which is already transforming their interpretation and exploitation in clinical practice. While similar benefits are longed for in the field of interventional imaging, this ambition is challenged by a much higher heterogeneity. Clinical workflows within interventional suites and operating theatres are extremely complex and typically rely on poorly integrated intra-operative devices, sensors, and support infrastructures. Taking stock of some of the most exciting developments in machine learning and artificial intelligence for computer assisted interventions, we highlight the crucial need to take context and human factors into account in order to address these challenges. Contextual artificial intelligence for computer assisted intervention, or CAI4CAI, arises as an emerging opportunity feeding into the broader field of surgical data science. Central challenges being addressed in CAI4CAI include how to integrate the ensemble of prior knowledge and instantaneous sensory information from experts, sensors and actuators; how to create and communicate a faithful and actionable shared representation of the surgery among a mixed human-AI actor team; how to design interventional systems and associated cognitive shared control schemes for online uncertainty-aware collaborative decision making ultimately producing more precise and reliable interventions.

Applicability of augmented reality in orthopedic surgery - A systematic review.

BACKGROUND: Computer-assisted solutions are changing surgical practice continuously. One of the most disruptive technologies among the computer-integrated surgical techniques is Augmented Reality (AR). While Augmented Reality is increasingly used in several medical specialties, its potential benefit in orthopedic surgery is not yet clear. The purpose of this article is to provide a systematic review of the current state of knowledge and the applicability of AR in orthopedic surgery. METHODS: A systematic review of the current literature was performed to find the state of knowledge and applicability of AR in Orthopedic surgery. A systematic search of the following three databases was performed: "PubMed", "Cochrane Library" and "Web of Science". The systematic review followed the Preferred Reporting Items on Systematic Reviews and Meta-analysis (PRISMA) guidelines and it has been published and registered in the international prospective register of systematic reviews (PROSPERO). RESULTS: 31 studies and reports are included and classified into the following categories: Instrument / Implant Placement, Osteotomies, Tumor Surgery, Trauma, and Surgical Training and Education. Quality assessment could be performed in 18 studies. Among the clinical studies, there were six case series with an average score of 90% and one case report, which scored 81% according to the Joanna Briggs Institute Critical Appraisal Checklist (JBI CAC). The 11 cadaveric studies scored 81% according to the QUACS scale (Quality Appraisal for Cadaveric Studies). CONCLUSION: This manuscript provides 1) a summary of the current state of knowledge and research of Augmented Reality in orthopedic surgery presented in the literature, and 2) a discussion by the authors presenting the key remarks required for seamless integration of Augmented Reality in the future surgical practice. TRIAL REGISTRATION: PROSPERO registration number: CRD42019128569.

Augmented Reality for Acetabular Component Placement in Direct Anterior Total Hip Arthroplasty.

BACKGROUND: Malposition of the acetabular component of a hip prosthesis can lead to poor outcomes. Traditional placement with fluoroscopic guidance results in a 35% malpositioning rate. We compared the (1) accuracy and precision of component placement, (2) procedure time, (3) radiation dose, and (4) usability of a novel 3-dimensional augmented reality (AR) guidance system vs standard fluoroscopic guidance for acetabular component placement. METHODS: We simulated component placement using a radiopaque foam pelvis. Cone-beam computed tomographic data and optical data from a red-green-blue-depth camera were coregistered to create the AR environment. Eight orthopedic surgery trainees completed component placement using both methods. We measured component position (inclination, anteversion), procedure time, radiation dose, and usability (System Usability Scale score, Surgical Task Load Index value). Alpha = .05. RESULTS: Compared with fluoroscopic technique, AR technique was significantly more accurate for achieving target inclination (P = .01) and anteversion (P = .02) and more precise for achieving target anteversion (P < .01). AR technique was faster (mean ± standard deviation, 1.8 ± 0.25 vs 3.9 ± 1.6 minute; P < .01), and participants rated it as significantly easier to use according to both scales (P < .05). Radiation dose was not significantly different between techniques (P = .48). CONCLUSION: A novel 3-dimensional AR guidance system produced more accurate inclination and anteversion and more precise anteversion in the placement of the acetabular component of a hip prosthesis. AR guidance was faster and easier to use than standard fluoroscopic guidance and did not involve greater radiation dose.

Laryngeal Lesion Classification Based on Vascular Patterns in Contact Endoscopy and Narrow Band Imaging: Manual Versus Automatic Approach.

Longitudinal and perpendicular changes in the vocal fold's blood vessels are associated with the development of benign and malignant laryngeal lesions. The combination of Contact Endoscopy (CE) and Narrow Band Imaging (NBI) can provide intraoperative real-time visualization of the vascular changes in the laryngeal mucosa. However, the visual evaluation of vascular patterns in CE-NBI images is challenging and highly depends on the clinicians' experience. The current study aims to evaluate and compare the performance of a manual and an automatic approach for laryngeal lesion's classification based on vascular patterns in CE-NBI images. In the manual approach, six observers visually evaluated a series of CE+NBI images that belong to a patient and then classified the patient as benign or malignant. For the automatic classification, an algorithm based on characterizing the level of the vessel's disorder in combination with four supervised classifiers was used to classify CE-NBI images. The results showed that the manual approach's subjective evaluation could be reduced by using a computer-based approach. Moreover, the automatic approach showed the potential to work as an assistant system in case of disagreements among clinicians and to reduce the manual approach's misclassification issue.

Bayesian QuickNAT: Model uncertainty in deep whole-brain segmentation for structure-wise quality control.

We introduce Bayesian QuickNAT for the automated quality control of whole-brain segmentation on MRI T1 scans. Next to the Bayesian fully convolutional neural network, we also present inherent measures of segmentation uncertainty that allow for quality control per brain structure. For estimating model uncertainty, we follow a Bayesian approach, wherein, Monte Carlo (MC) samples from the posterior distribution are generated by keeping the dropout layers active at test time. Entropy over the MC samples provides a voxel-wise model uncertainty map, whereas expectation over the MC predictions provides the final segmentation. Next to voxel-wise uncertainty, we introduce four metrics to quantify structure-wise uncertainty in segmentation for quality control. We report experiments on four out-of-sample datasets comprising of diverse age range, pathology and imaging artifacts. The proposed structure-wise uncertainty metrics are highly correlated with the Dice score estimated with manual annotation and therefore present an inherent measure of segmentation quality. In particular, the intersection over union over all the MC samples is a suitable proxy for the Dice score. In addition to quality control at scan-level, we propose to incorporate the structure-wise uncertainty as a measure of confidence to do reliable group analysis on large data repositories. We envisage that the introduced uncertainty metrics would help assess the fidelity of automated deep learning based segmentation methods for large-scale population studies, as they enable automated quality control and group analyses in processing large data repositories.

QuickNAT: A fully convolutional network for quick and accurate segmentation of neuroanatomy.

Whole brain segmentation from structural magnetic resonance imaging (MRI) is a prerequisite for most morphological analyses, but is computationally intense and can therefore delay the availability of image markers after scan acquisition. We introduce QuickNAT, a fully convolutional, densely connected neural network that segments a MRI brain scan in 20 s. To enable training of the complex network with millions of learnable parameters using limited annotated data, we propose to first pre-train on auxiliary labels created from existing segmentation software. Subsequently, the pre-trained model is fine-tuned on manual labels to rectify errors in auxiliary labels. With this learning strategy, we are able to use large neuroimaging repositories without manual annotations for training. In an extensive set of evaluations on eight datasets that cover a wide age range, pathology, and different scanners, we demonstrate that QuickNAT achieves superior segmentation accuracy and reliability in comparison to state-of-the-art methods, while being orders of magnitude faster. The speed up facilitates processing of large data repositories and supports translation of imaging biomarkers by making them available within seconds for fast clinical decision making.

Toward an End-to-End Calibration for Mobile C-Arm in Combination with a Depth Sensor for Surgical Augmented Reality Applications.

C-arm X-ray imaging is commonly applied in operating rooms for guiding orthopedic surgeries. Augmented Reality (AR) with C-arm X-ray images during surgery is an efficient way to facilitate procedures for surgeons. However, the accurate calibration process for surgical AR based on C-arm is essential and still challenging due to the limitations of C-arm imaging systems, such as instability of C-arm calibration parameters and the narrow field of view. We extend existing methods using a depth camera and propose a new calibration procedure consisting of calibration of the C-arm imaging system, and 3D/2D calibration of an RGB-D camera and C-arm system with a new method to achieve reliable data and promising accuracy and, at the same time, consistent with standard surgical protocols. For the calibration procedure, we apply bundle adjustment equations with a 3D designed Lego multi-modal phantom, in contrast to the previous methods in which planar calibration phantoms were applied. By using our method, the visualization of the X-ray image upon the 3D data was done, and the achieved mean overlay error was 1.03 mm. The evaluations showed that the proposed calibration procedure provided promising accuracy for AR surgeries and it improved the flexibility and robustness of existing C-arm calibration methods for surgical augmented reality (using C-arm and RGB-D sensor). Moreover, the results showed the efficiency of our method to compensate for the effects of the C-arm movement on calibration parameters. It was shown that the obtained overlay error was improved for the non-zero rotation movement of C-arm by using a virtual detector.

Say, What Is on Your Mind? Surgeons' Evaluations of Realism and Usability of a Virtual Reality Vertebroplasty Simulator.

BACKGROUND: Virtual reality (VR)-based simulations offer rich opportunities for surgical skill training and assessment of surgical novices and experts. A structured evaluation and validation process of such training and assessment tools is necessary for effective surgical learning environments. OBJECTIVE: To develop and apply a classification system of surgeon-reported experience during operation of a VR vertebroplasty simulator. METHODS: A group of orthopedic, trauma surgeons and neurosurgeons (n = 13) with various levels of expertise performed on a VR vertebroplasty simulator. We established a mixed-methods design using think-aloud protocols, senior surgical expert evaluations, performance metrics, and a post-simulation questionnaire. Verbal content was systematically analyzed using structured qualitative content analysis. We established a category system for classification of surgeons' verbal evaluations during the simulation. Furthermore, we evaluated intraoperative performance metrics and explored potential associations with surgeons' characteristics and simulator evaluation. RESULTS: Overall, 244 comments on realism and usability of the vertebroplasty simulator were collected. This included positive and negative remarks, questions, and specific suggestions for improvement. Further findings included surgeons' approval of the realism and usability of the simulator and the observation that the haptic feedback of the VR patient's anatomy requires further improvement. Surgeon-reported evaluations were not associated with performance decrements. DISCUSSION: This study is the first to apply think-aloud protocols for evaluation of a surgical VR-based simulator. A novel classification approach is introduced that can be used to classify surgeons' verbalized experiences during simulator use. Our lessons learned may be valuable for future research with similar methodological approach.

Navigation of Fluorescence Cameras during Soft Tissue Surgery-Is it Possible to Use a Single Navigation Setup for Various Open and Laparoscopic Urological Surgery Applications?

PURPOSE: Real-time visualization fluorescence imaging can guide surgeons during tissue resection. Unfortunately tissue induced signal attenuation limits the value of this technique to superficial applications. By positioning the fluorescence camera via a dedicated navigation setup we reasoned that the technology could be made compatible with deeper lesions, increasing its impact on clinical care. Such an impact would benefit from the ability to implement the navigation technology in different surgical settings. For that reason we evaluated whether a single fluorescence camera could be navigated toward targeted lesions during open and laparoscopic surgery. MATERIALS AND METHODS: A fluorescence camera with scopes available for open and laparoscopic procedures was integrated with a navigation platform. Lymph nodes identified on SPECT/CT (single photon emission computerized tomography/computerized tomography) or free-hand single photon emission computerized tomography acted as navigation targets and were displayed as augmented overlays in the fluorescence camera video feed. The accuracy of this setup was evaluated in a phantom study of 4 scans per single photon emission computerized tomography imaging modality. This was followed by 4 first in human translations into sentinel lymph node biopsy procedures for penile (open surgery) and prostate (laparoscopic surgery) cancer. RESULTS: Overall the phantom studies revealed a tool-target distance accuracy of 2.1 mm for SPECT/CT and 3.2 mm for freehand single photon emission computerized tomography, and an augmented reality registration accuracy of 1.1 and 2.2 mm, respectively. Subsequently open and laparoscopic navigation efforts were accurate enough to localize the fluorescence signals of the targeted tissues in vivo. CONCLUSIONS: The phantom and human studies performed suggested that the single navigation setup is applicable in various open and laparoscopic urological surgery applications. Further evaluation in larger patient groups with a greater variety of malignancies is recommended to strengthen these results.

Computer-assisted surgery: virtual- and augmented-reality displays for navigation during urological interventions.

PURPOSE OF REVIEW: To provide an overview of the developments made for virtual- and augmented-reality navigation procedures in urological interventions/surgery. RECENT FINDINGS: Navigation efforts have demonstrated potential in the field of urology by supporting guidance for various disorders. The navigation approaches differ between the individual indications, but seem interchangeable to a certain extent. An increasing number of pre- and intra-operative imaging modalities has been used to create detailed surgical roadmaps, namely: (cone-beam) computed tomography, MRI, ultrasound, and single-photon emission computed tomography. Registration of these surgical roadmaps with the real-life surgical view has occurred in different forms (e.g. electromagnetic, mechanical, vision, or near-infrared optical-based), whereby the combination of approaches was suggested to provide superior outcome. Soft-tissue deformations demand the use of confirmatory interventional (imaging) modalities. This has resulted in the introduction of new intraoperative modalities such as drop-in US, transurethral US, (drop-in) gamma probes and fluorescence cameras. These noninvasive modalities provide an alternative to invasive technologies that expose the patients to X-ray doses. Whereas some reports have indicated navigation setups provide equal or better results than conventional approaches, most trials have been performed in relatively small patient groups and clear follow-up data are missing. SUMMARY: The reported computer-assisted surgery research concepts provide a glimpse in to the future application of navigation technologies in the field of urology.

Personalized augmented reality for anatomy education.

Anatomy education is a challenging but vital element in forming future medical professionals. In this work, a personalized and interactive augmented reality system is developed to facilitate education. This system behaves as a "magic mirror" which allows personalized in-situ visualization of anatomy on the user's body. Real-time volume visualization of a CT dataset creates the illusion that the user can look inside their body. The system comprises a RGB-D sensor as a real-time tracking device to detect the user moving in front of a display. In addition, the magic mirror system shows text information, medical images, and 3D models of organs that the user can interact with. Through the participation of 7 clinicians and 72 students, two user studies were designed to respectively assess the precision and acceptability of the magic mirror system for education. The results of the first study demonstrated that the average precision of the augmented reality overlay on the user body was 0.96 cm, while the results of the second study indicate 86.1% approval for the educational value of the magic mirror, and 91.7% approval for the augmented reality capability of displaying organs in three dimensions. The usefulness of this unique type of personalized augmented reality technology has been demonstrated in this paper.

Stratification of coronary artery disease patients for revascularization procedure based on estimating adverse effects.

BACKGROUND: Percutaneous coronary intervention (PCI) is the most commonly performed treatment for coronary atherosclerosis. It is associated with a higher incidence of repeat revascularization procedures compared to coronary artery bypass grafting surgery. Recent results indicate that PCI is only cost-effective for a subset of patients. Estimating risks of treatment options would be an effort toward personalized treatment strategy for coronary atherosclerosis. METHODS: In this paper, we propose to model clinical knowledge about the treatment of coronary atherosclerosis to identify patient-subgroup-specific classifiers to predict the risk of adverse events of different treatment options. We constructed one model for each patient subgroup to account for subgroup-specific interpretation and availability of features and hierarchically aggregated these models to cover the entire data. In addition, we deviated from the current clinical workflow only for patients with high probability of benefiting from an alternative treatment, as suggested by this model. Consequently, we devised a two-stage test with optimized negative and positive predictive values as the main indicators of performance. Our analysis was based on 2,377 patients that underwent PCI. Performance was compared with a conventional classification model and the existing clinical practice by estimating effectiveness, safety, and costs for different endpoints (6 month angiographic restenosis, 12 and 36 month hazardous events). RESULTS: Compared to the current clinical practice, the proposed method achieved an estimated reduction in adverse effects by 25.0% (95% CI, 17.8 to 30.2) for hazardous events at 36 months and 31.2% (95% CI, 25.4 to 39.0) for hazardous events at 12 months. Estimated total savings per patient amounted to $693 and $794 at 12 and 36 months, respectively. The proposed subgroup-specific method outperformed conventional population wide regression: The median area under the receiver operating characteristic curve increased from 0.57 to 0.61 for prediction of angiographic restenosis and from 0.76 to 0.85 for prediction of hazardous events. CONCLUSIONS: The results of this study demonstrated the efficacy of deployment of bare-metal stents and coronary artery bypass grafting surgery for subsets of patients. This is one effort towards development of personalized treatment strategies for patients with coronary atherosclerosis that could significantly impact associated treatment costs.