CardioGenesis4D: Interactive Morphological Transitions of Embryonic Heart Development in a Virtual Learning Environment.

In the embryonic human heart, complex dynamic shape changes take place in a short period of time on a microscopic scale, making this development difficult to visualize. However, spatial understanding of these processes is essential for students and future cardiologists to properly diagnose and treat congenital heart defects. Following a user centered approach, the most crucial embryological stages were identified and translated into a virtual reality learning environment (VRLE) to enable the understanding of the morphological transitions of these stages through advanced interactions. To address individual learning types, we implemented different features and evaluated the application regarding usability, perceived task load, and sense of presence in a user study. We also assessed spatial awareness and knowledge gain, and finally obtained feedback from domain experts. Overall, students and professionals rated the application positively. To minimize distraction from interactive learning content, such VRLEs should consider features for different learning types, allow for gradual habituation, and at the same time provide enough playful stimuli. Our work previews how VR can be integrated into a cardiac embryology education curriculum.

Real-Time Augmented Reality Annotation for Surgical Education during Laparoscopic Surgery: Results from a Single-Center Randomized Controlled Trial and Future Aspects.

BACKGROUND: We developed an interactive augmented reality tool (HoloPointer) that enables real-time annotation on a laparoscopy monitor for intraoperative guidance. This application operates exclusively via verbal commands and head movements to ensure a sterile workflow. STUDY DESIGN: Purpose of this randomized controlled clinical trial was to evaluate the integration of this new technology into the operating room. This prospective single-center study included 32 elective laparoscopic cholecystectomies (29 surgical teams, 15 trainees, and 13 trainers). The primary objective and assessment measure was the HoloPointer's influence on surgical performance (subjective assessment, global operative assessment of laparoscopic skills [GOALS] and critical view of safety [CVS]). The secondary objectives and outcome variables were its influence on operation time, quality of assistance (5-point Likert scale), and user-friendliness (system usability scale, 0 to 100 points). RESULTS: Gestural corrections were reduced by 59.4% (4.6 SD 8.1 vs 1.9 SD 4.7, p > 0.05) and verbal corrections by 36.1% (17.8 SD 12.9 vs 11.4 SD 8.1, p > 0.05). Subjective surgical performance could be improved by 84.6% of participants. No statistically significant differences were observed for objective parameters GOALS, CVS, and operation time. In the system usability scale, the application achieved an average score of 72.5 SD 16.3 (good user-friendliness). Of the participants, 69.2% wanted to use the HoloPointer more frequently. CONCLUSIONS: The majority of trainees improved their surgical performance using the HoloPointer in elective laparoscopic cholecystectomies, and the rate of classic but potentially misleading corrections was noticeably reduced. The HoloPointer has the potential to improve education in minimally invasive surgery.

AR-Supported Supervision of Conditional Autonomous Robots: Considerations for Pedicle Screw Placement in the Future.

Robotic assistance is applied in orthopedic interventions for pedicle screw placement (PSP). While current robots do not act autonomously, they are expected to have higher autonomy under surgeon supervision in the mid-term. Augmented reality (AR) is promising to support this supervision and to enable human-robot interaction (HRI). To outline a futuristic scenario for robotic PSP, the current workflow was analyzed through literature review and expert discussion. Based on this, a hypothetical workflow of the intervention was developed, which additionally contains the analysis of the necessary information exchange between human and robot. A video see-through AR prototype was designed and implemented. A robotic arm with an orthopedic drill mock-up simulated the robotic assistance. The AR prototype included a user interface to enable HRI. The interface provides data to facilitate understanding of the robot's "intentions", e.g., patient-specific CT images, the current workflow phase, or the next planned robot motion. Two-dimensional and three-dimensional visualization illustrated patient-specific medical data and the drilling process. The findings of this work contribute a valuable approach in terms of addressing future clinical needs and highlighting the importance of AR support for HRI.

Augmented reality visualization of automated path planning for percutaneous interventions: a phantom study.

PURPOSE: Insertion point identification is a major challenge for percutaneous interventions. Planning in 2D slice image data is time-consuming and inefficient. Automated path planning can help to overcome these challenges. However, the setup of the intervention room is difficult to consider. In addition, transferring the insertion point to the skin is often prone to error. Therefore, a visualization for an automated path planning was implemented. METHODS: A condition-based automated path planning was calculated with path length, distance to risk structures and insertion angle. The results were displayed on a phantom using projector-based augmented reality (AR) with an access point selection using the insertion needle. Two variants of the insertion visualization and three target displays were evaluated in a user study. RESULTS: A visualization of insertion points with a representation of the path quality resulted in a choice of safer paths, compared with no insertion point display or no coding of the path quality. A representation of the target was preferred in the final survey, but did not perform better. A target display separate from the insertion point visualization reduced interferences between visualizations. CONCLUSION: A projector-based AR visualization of automated path planning results supports insertion point identification for percutaneous interventions. A display of the path quality enables the choice of safe access paths especially for unexperienced users. Further research is needed to identify clinical benefits and applicability.

HoloPointer: a virtual augmented reality pointer for laparoscopic surgery training.

PURPOSE: In laparoscopic surgery training, experts guide novice physicians to desired instrument positions or indicate relevant areas of interest. These instructions are usually given via verbal communication or using physical pointing devices. To facilitate a sterile work flow and to improve training, new guiding methods are needed. This work proposes to use optical see-through augmented reality to visualize an interactive virtual pointer on the laparoscopic. METHODS: After an interdisciplinary development, the pointer's applicability and feasibility for training was evaluated and it was compared to a standard condition based on verbal and gestural communication only. In this study, ten surgical trainees were guided by an experienced trainer during cholecystectomies on a laparoscopic training simulator. All trainees completed a virtual cholecystectomy with and without the interactive virtual pointer in alternating order. Measures included procedure time, economy of movement and error rates. RESULTS: Results of standardized variables revealed significantly improved economy of movement (p = 0.047) and error rates (p = 0.047), as well as an overall improved user performance (Total z-score; p = 0.031) in conditions using the proposed method. CONCLUSION: The proposed HoloPointer is a feasible and applicable tool for laparoscopic surgery training. It improved objective performance metrics without prolongation of the task completion time in this pre-clinical setup.

Comparison of Augmented Reality Display Techniques to Support Medical Needle Insertion.

Augmented reality (AR) may be a useful technique to overcome issues of conventionally used navigation systems supporting medical needle insertions, like increased mental workload and complicated hand-eye coordination. Previous research primarily focused on the development of AR navigation systems designed for specific displaying devices, but differences between employed methods have not been investigated before. To this end, a user study involving a needle insertion task was conducted comparing different AR display techniques with a monitor-based approach as baseline condition for the visualization of navigation information. A video see-through stationary display, an optical see-through head-mounted display and a spatial AR projector-camera-system were investigated in this comparison. Results suggest advantages of using projected navigation information in terms of lower task completion time, lower angular deviation and affirmative subjective participant feedback. Techniques requiring the intermediate view on screens, i.e. the stationary display and the baseline condition, showed less favorable results. Thus, benefits of providing AR navigation information compared to a conventionally used method could be identified. Significant objective measures results, as well as an identification of advantages and disadvantages of individual display techniques contribute to the development and design of improved needle navigation systems.

Comparison of Projective Augmented Reality Concepts to Support Medical Needle Insertion.

Augmented reality (AR) is a promising tool to improve instrument navigation in needle-based interventions. Limited research has been conducted regarding suitable navigation visualizations. In this work, three navigation concepts based on existing approaches were compared in a user study using a projective AR setup. Each concept was implemented with three different scales for accuracy-to-color mapping and two methods of navigation indicator scaling. Participants were asked to perform simulated needle insertion tasks with each of the resulting 18 prototypes. Insertion angle and insertion depth accuracies were measured and analyzed, as well as task completion time and participants' subjectively perceived task difficulty. Results show a clear ranking of visualization concepts across variables. Less consistent results were obtained for the color and indicator scaling factors. Results suggest that logarithmic indicator scaling achieved better accuracy, but participants perceived it to be more difficult than linear scaling. With specific results for angle and depth accuracy, our study contributes to the future composition of improved navigation support and systems for precise needle insertion or similar applications.

Projector-based augmented reality system for interventional visualization inside MRI scanners.

BACKGROUND: Navigation support in interventional magnetic resonance imaging (MRI) is separated from the operating field, which makes it difficult to interpret positions and orientations and to coordinate the necessary hand movements. METHODS: We developed a projector-based augmented reality system to enable visual navigation of tracked instruments on pre-planned paths and visualization of risk structures directly on the patient inside the MRI bore. To assess the accuracy of the system, a user study was carried out with clinicians in a needle navigation test scenario. RESULTS: The targets were reached with an error of 1.7 ± 0.5 mm and the entry points with an error of 1.7 ± 0.8 mm. CONCLUSION: The accuracy results are similar to those reached by live image-guided interventions and related work and confirm that this projective augmented reality prototype for the interventional MRI can serve as a platform for current and future research in augmented reality visualization and dynamic registration.

HoloInjection: augmented reality support for CT-guided spinal needle injections.

The correct placement of needles is decisive for the success of many minimally-invasive interventions and therapies. These needle insertions are usually only guided by radiological imaging and can benefit from additional navigation support. Augmented reality (AR) is a promising tool to conveniently provide needed information and may thus overcome the limitations of existing approaches. To this end, a prototypical AR application was developed to guide the insertion of needles to spinal targets using the mixed reality glasses Microsoft HoloLens. The system's registration accuracy was attempted to measure and three guidance visualisation concepts were evaluated concerning achievable in-plane and out-of-plane needle orientation errors in a comparison study. Results suggested high registration accuracy and showed that the AR prototype is suitable for reducing out-of-plane orientation errors. Limitations, like comparatively high in-plane orientation errors, effects of the viewing position and missing image slices indicate potential for improvement that needs to be addressed before transferring the application to clinical trials.

Concepts for augmented reality visualisation to support needle guidance inside the MRI.

During MRI-guided interventions, navigation support is often separated from the operating field on displays, which impedes the interpretation of positions and orientations of instruments inside the patient's body as well as hand-eye coordination. To overcome these issues projector-based augmented reality can be used to support needle guidance inside the MRI bore directly in the operating field. The authors present two visualisation concepts for needle navigation aids which were compared in an accuracy and usability study with eight participants, four of whom were experienced radiologists. The results show that both concepts are equally accurate ( 2.0 ± 0.6 and 1.7 ± 0.5 mm ), useful and easy to use, with clear visual feedback about the state and success of the needle puncture. For easier clinical applicability, a dynamic projection on moving surfaces and organ movement tracking are needed. For now, tests with patients with respiratory arrest are feasible.

Molecular imaging for the characterization of breast tumors.

Recently, molecular imaging, using various techniques, has been assessed for breast imaging. Molecular imaging aims to quantify and visualize biological, physiological, and pathological processes at the cellular and molecular levels to further elucidate the development and progression of breast cancer and the response to treatment. Molecular imaging enables the depiction of tumor morphology, as well as the assessment of functional and metabolic processes involved in cancer development at different levels. To date, molecular imaging techniques comprise both nuclear medicine and radiological techniques. This review aims to summarize the current and emerging functional and metabolic techniques for the molecular imaging of breast tumors.