Ηand dexterities assessment in stroke patients based on augmented reality and machine learning through a box and block test.

A popular and widely suggested measure for assessing unilateral hand motor skills in stroke patients is the box and block test (BBT). Our study aimed to create an augmented reality enhanced version of the BBT (AR-BBT) and evaluate its correlation to the original BBT for stroke patients. Following G-power analysis, clinical examination, and inclusion-exclusion criteria, 31 stroke patients were included in this study. AR-BBT was developed using the Open Source Computer Vision Library (OpenCV). The MediaPipe's hand tracking library uses a palm and a hand landmark machine learning model to detect and track hands. A computer and a depth camera were employed in the clinical evaluation of AR-BBT following the principles of traditional BBT. A strong correlation was achieved between the number of blocks moved in the BBT and the AR-BBT on the hemiplegic side (Pearson correlation = 0.918) and a positive statistically significant correlation (p = 0.000008). The conventional BBT is currently the preferred assessment method. However, our approach offers an advantage, as it suggests that an AR-BBT solution could remotely monitor the assessment of a home-based rehabilitation program and provide additional hand kinematic information for hand dexterities in AR environment conditions. Furthermore, it employs minimal hardware equipment.

An Augmented Reality Visualization System for Simulated Multirotor Aerial Vehicles.

Multirotors Aerial Vehicles are special class of Unmanned Aerial Vehicles with many practical applications. The growing demand for this class of aircraft requires tools that speed up their development. Simulated environments have gained increasing importance, as they facilitate testing and prototyping solutions, where virtual environments allow real-time interaction with simulated models, with similar behavior to real systems. More recently, the use of Augmented Reality has allowed an increasing experience of immersion and integration between the virtual world and a real scenario. This work proposes the use of Augmented Reality technology and a simulated model of a multirotor to create an interactive flight environment, aiming to improve the user experience in the analysis of simulated models. For this purpose, a smartphone was adopted as a hardware platform, a game engine is used as a basis for the development of the Augmented Reality application, that represents a numerical simulation of the flight dynamics and the control system of a multirotor, and a game controller is adopted for user interaction. The resulting system demonstrates that Augmented Reality is a viable technology that can be used to increase the possibilities of evaluating simulated systems.

Robust tracking of deformable anatomical structures with severe occlusions using deformable geometrical primitives.

BACKGROUND AND OBJECTIVE: Surgical robotics tends to develop cognitive control architectures to provide certain degree of autonomy to improve patient safety and surgery outcomes, while decreasing the required surgeons' cognitive load dedicated to low level decisions. Cognition needs workspace perception, which is an essential step towards automatic decision-making and task planning capabilities. Robust and accurate detection and tracking in minimally invasive surgery suffers from limited visibility, occlusions, anatomy deformations and camera movements. METHOD: This paper develops a robust methodology to detect and track anatomical structures in real time to be used in automatic control of robotic systems and augmented reality. The work focuses on the experimental validation in highly challenging surgery: fetoscopic repair of Open Spina Bifida. The proposed method is based on two sequential steps: first, selection of relevant points (contour) using a Convolutional Neural Network and, second, reconstruction of the anatomical shape by means of deformable geometric primitives. RESULTS: The methodology performance was validated with different scenarios. Synthetic scenario tests, designed for extreme validation conditions, demonstrate the safety margin offered by the methodology with respect to the nominal conditions during surgery. Real scenario experiments have demonstrated the validity of the method in terms of accuracy, robustness and computational efficiency. CONCLUSIONS: This paper presents a robust anatomical structure detection in present of abrupt camera movements, severe occlusions and deformations. Even though the paper focuses on a case study, Open Spina Bifida, the methodology is applicable in all anatomies which contours can be approximated by geometric primitives. The methodology is designed to provide effective inputs to cognitive robotic control and augmented reality systems that require accurate tracking of sensitive anatomies.

Metaverse, virtual reality and augmented reality in total shoulder arthroplasty: a systematic review.

PURPOSE: This systematic review aims to provide an overview of the current knowledge on the role of the metaverse, augmented reality, and virtual reality in reverse shoulder arthroplasty. METHODS: A systematic review was performed using the PRISMA guidelines. A comprehensive review of the applications of the metaverse, augmented reality, and virtual reality in in-vivo intraoperative navigation, in the training of orthopedic residents, and in the latest innovations proposed in ex-vivo studies was conducted. RESULTS: A total of 22 articles were included in the review. Data on navigated shoulder arthroplasty was extracted from 14 articles: seven hundred ninety-three patients treated with intraoperative navigated rTSA or aTSA were included. Also, three randomized control trials (RCTs) reported outcomes on a total of fifty-three orthopedics surgical residents and doctors receiving VR-based training for rTSA, which were also included in the review. Three studies reporting the latest VR and AR-based rTSA applications and two proof of concept studies were also included in the review. CONCLUSIONS: The metaverse, augmented reality, and virtual reality present immense potential for the future of orthopedic surgery. As these technologies advance, it is crucial to conduct additional research, foster development, and seamlessly integrate them into surgical education to fully harness their capabilities and transform the field. This evolution promises enhanced accuracy, expanded training opportunities, and improved surgical planning capabilities.

Exploring the knowledge and awareness on applications of virtual reality and augmented reality technology among dental healthcare professionals - a crosssectional survey.

OBJECTIVE: To assess the knowledge and awareness of Virtual Reality (VR) and Augmented Reality (AR) technology in dentistry. METHODS: A questionnaire survey-based study was conducted using Google forms on a sample of 273 dental healthcare professionals (DHCP) from October- November 2023, after obtaining ethical approval. A validated questionnaire, divided into three sections, was used to assess the knowledge and awareness of dental healthcare professionals on virtual and augmented reality. Section A was about demographic statistics, section B assessed knowledge and awareness regarding VR and AR and section C consisted of future acceptability of VR and AR among DHCP. The frequency of each question was reported in percentages. To assess the difference of knowledge and awareness of AR and VR among different specialties of DHCP, one-way ANOVA test was applied and in case of significant results pairwise comparison was performed by post-hoc Tukey test. RESULTS: There was a statistically significant difference of knowledge (1.40 ± 0.49) among different dental healthcare professionals. On pairwise comparison, a statistically significant difference (p = ˂0.05) of knowledge and awareness of AR and VR was found among dental specialist and other dental health professionals. CONCLUSIONS: A concerning lack of knowledge and awareness among dental healthcare professionals regarding AR and VR technology in dentistry was found. Interestingly, within the spectrum of specialties, dental specialists demonstrated a comparatively higher awareness than their counterparts in other specialties. Addressing barriers, notably a lack of knowledge, is crucial for successful technology adoption in dental education and practice.

Application of augmented reality for crime scene investigation training and education.

The role of the crime scene investigator is complex, and investigators need to be able to conduct multiple frequently performed tasks. Appropriate training and education are critical to impart the crime scene investigator with the necessary capabilities. For a range of reasons, including the COVID-19 pandemic and the need for multi-disciplinary capabilities, training and education requirements have evolved in recent times to be more flexible and accessible. The skills of a crime scene investigator can be broadly categorised into two types of fundamental skills: decision-making and psychomotor skills. Both these skills need to be taught and assessed within training and education programs. The most common approach to impart these skills is by incorporating a crime scene simulation facility; however, not everyone has access to these facilities. Furthermore, crime scene staff often undertake refresher courses and are required to complete proficiency assessments. Conducting these activities in a dedicated crime scene simulation facility is time consuming and costly. Virtual tools have been developed in recent years to address this, but these tools only assess decision-making skills and not psychomotor skills. This paper argues that the implementation of augmented reality (AR) technology should be considered in crime scene investigator training and university education, because it can provide significant advantages when paired with conventional methods of training and education. When appropriately managed, AR can provide an avenue of training where both decision-making and psychomotor skills can be addressed simultaneously, while providing a more flexible and accessible approach. The implementation of AR has the potential to significantly improve the standards of teaching, resulting in better equipped crime scene investigators. The paper will explore how AR has the potential to improve accessibility of training, increase safety, enhance the student experience, enhance collaboration and feedback through connectivity and potentially reduce cost. This paper will also provide an insight into what would need to be considered before implementing AR technology into crime scene investigation training and education alongside current approaches. It is argued that the inclusion of AR into the current training and education arsenal provides significant benefits that are worthy of exploration.

Mismatch between Augmented Reality Navigation Images and Actual Location of a Cauda Equina Tumor:A Case Report.

BACKGROUND: Augmented reality navigation is the one of the navigation technologies that allows computer-generated virtual images to be projected onto a real-world environment. Augmented reality navigation can be used in spinal tumor surgery. However, it is unknown if there are any pitfalls when using this technique. CASE PRESENTATION: The patient in this report underwent complete resection of a cauda equina tumor at the L2-L3 level using microscope-based augmented reality navigation. Although the registration error of navigation was <1 mm, we found a discrepancy between the augmented reality navigation images and the actual location of the tumor, which we have called "navigation mismatch". This mismatch, which was caused by the mobility of the spinal tumor in the dura mater, seems to be one of the pitfalls of augmented reality navigation for spinal tumors. CONCLUSIONS: Combined use of intraoperative ultrasound and augmented reality navigation seems advisable in such cases. J. Med. Invest. 71 : 174-176, February, 2024.

The application of virtual reality and augmented reality in dentistry - a literature review.

In recent times, dentistry has seen significant technological advancements that have transformed various specialized areas within the field. Developed into applications for mobile devices, augmented reality (AR) seamlessly merges digital components with the physical world, enhancing both realms while maintaining their individual separateness. On the other hand, virtual reality (VR) relies on advanced, tailored software to visualize a digital 3D environment stimulating the operator's senses through computer generated sensations and feedback. The current advances use the application of VR, haptic simulators, the use of an AI algorithm and many more that provides new opportunities for smart learning and enhance the teaching environment. As this technology continues to evolve, it is poised to become even more remarkable, enabling specialists to potentially visualize both soft and hard tissues within the patient's body for effective treatment planning. This literature aims to present the newest advancements and ongoing development of AR and VR in dentistry and medicine. It highlights their diverse applications while identifying areas needing further research for effective integration into clinical practice.

Assessment of pulmonary vascular anatomy: comparing augmented reality by holograms versus standard CT images/reconstructions using surgical findings as reference standard.

BACKGROUND: We compared computed tomography (CT) images and holograms (HG) to assess the number of arteries of the lung lobes undergoing lobectomy and assessed easiness in interpretation by radiologists and thoracic surgeons with both techniques. METHODS: Patients scheduled for lobectomy for lung cancer were prospectively included and underwent CT for staging. A patient-specific three-dimensional model was generated and visualized in an augmented reality setting. One radiologist and one thoracic surgeon evaluated CT images and holograms to count lobar arteries, having as reference standard the number of arteries recorded at surgery. The easiness of vessel identification was graded according to a Likert scale. Wilcoxon signed-rank test and κ statistics were used. RESULTS: Fifty-two patients were prospectively included. The two doctors detected the same number of arteries in 44/52 images (85%) and in 51/52 holograms (98%). The mean difference between the number of artery branches detected by surgery and CT images was 0.31 ± 0.98, whereas it was 0.09 ± 0.37 between surgery and HGs (p = 0.433). In particular, the mean difference in the number of arteries detected in the upper lobes was 0.67 ± 1.08 between surgery and CT images and 0.17 ± 0.46 between surgery and holograms (p = 0.029). Both radiologist and surgeon showed a higher agreement for holograms (κ = 0.99) than for CT (κ = 0.81) and found holograms easier to evaluate than CTs (p < 0.001). CONCLUSIONS: Augmented reality by holograms is an effective tool for preoperative vascular anatomy assessment of lungs, especially when evaluating the upper lobes, more prone to anatomical variations. TRIAL REGISTRATION: ClinicalTrials.gov, NCT04227444 RELEVANCE STATEMENT: Preoperative evaluation of the lung lobe arteries through augmented reality may help the thoracic surgeons to carefully plan a lobectomy, thus contributing to optimize patients' outcomes. KEY POINTS: • Preoperative assessment of the lung arteries may help surgical planning. • Lung artery detection by augmented reality was more accurate than that by CT images, particularly for the upper lobes. • The assessment of the lung arterial vessels was easier by using holograms than CT images.

A decade of progress: bringing mixed reality image-guided surgery systems in the operating room.

Advancements in mixed reality (MR) have led to innovative approaches in image-guided surgery (IGS). In this paper, we provide a comprehensive analysis of the current state of MR in image-guided procedures across various surgical domains. Using the Data Visualization View (DVV) Taxonomy, we analyze the progress made since a 2013 literature review paper on MR IGS systems. In addition to examining the current surgical domains using MR systems, we explore trends in types of MR hardware used, type of data visualized, visualizations of virtual elements, and interaction methods in use. Our analysis also covers the metrics used to evaluate these systems in the operating room (OR), both qualitative and quantitative assessments, and clinical studies that have demonstrated the potential of MR technologies to enhance surgical workflows and outcomes. We also address current challenges and future directions that would further establish the use of MR in IGS.

How different text display patterns affect cybersickness in augmented reality.

Cybersickness remains a pivotal factor that impacts user experience in Augmented Reality (AR). Research probing into the relationship between AR reading tasks and cybersickness, particularly focusing on text display patterns and user characteristics, has been scant. Moreover, the influence of cybersickness on searching ability and the broader spectrum of user experience has not been rigorously tested. Recent investigations have aimed to pinpoint the variables that contribute to cybersickness during AR reading sessions. In one such study, 40 participants underwent a series of controlled experiments with randomized text display patterns, including variations in text speed and text movement modes. Post-experiment, participants completed a questionnaire that helped quantify their experiences and the degree of cybersickness encountered. The data highlighted that satiety, text speed, and text movement mode are significant contributors to cybersickness. When participants experienced higher levels of cybersickness, font color stood out as a particularly influential factor, whereas gender differences seemed to affect the onset of cybersickness more noticeably at lower levels. This study also drew attention to the impact of cybersickness on search ability within AR environments. It was noted that as cybersickness intensity increased, search ability was markedly compromised. In sum, the research underscores the importance of text display patterns and user characteristics, such as past AR experience, in understanding cybersickness and its detrimental effects on user experience and search ability, particularly under conditions of intense cybersickness.

Current Applications of VR/AR (Virtual Reality/Augmented Reality) in Pediatric Neurosurgery.

Neurosurgical procedures are some of the most complex procedures in medicine and since the advent of the field, planning, performing, and learning them has challenged the neurosurgeon. Virtual reality (VR) and augmented reality (AR) are making these challenges more manageable. VR refers to a virtual digital environment that can be experienced usually through use of stereoscopic glasses and controllers. AR, on the other hand, fuses the natural environment with virtual images, such as superimposing a preoperative MRI image on to the surgical field [1]. They initially were used primarily as neuronavigational tools but soon their potential in other areas of surgery, such as planning, education, and assessment, was noted and explored. Through this chapter, we outline the history and evolution of these two technologies over the past few decades, describe the current state of the technology and its uses, and postulate future directions for research and implementation.

A Systematic Review of the Application of Computational Technology in Microtia.

Microtia is a congenital and morphological anomaly of one or both ears, which results from a confluence of genetic and external environmental factors. Up to now, extensive research has explored the potential utilization of computational methodologies in microtia and has obtained promising results. Thus, the authors reviewed the achievements and shortcomings of the research mentioned previously, from the aspects of artificial intelligence, computer-aided design and surgery, computed tomography, medical and biological data mining, and reality-related technology, including virtual reality and augmented reality. Hoping to offer novel concepts and inspire further studies within this field.

Accuracy of augmented reality-guided needle placement for pulsed radiofrequency treatment of pudendal neuralgia: a pilot study on a phantom model.

Background: Pudendal neuralgia (PN) is a chronic neuropathy that causes pain, numbness, and dysfunction in the pelvic region. The current state-of-the-art treatment is pulsed radiofrequency (PRF) in which a needle is supposed to be placed close to the pudendal nerve for neuromodulation. Given the effective range of PRF of 5 mm, the accuracy of needle placement is important. This study aimed to investigate the potential of augmented reality guidance for improving the accuracy of needle placement in pulsed radiofrequency treatment for pudendal neuralgia. Methods: In this pilot study, eight subjects performed needle placements onto an in-house developed phantom model of the pelvis using AR guidance. AR guidance is provided using an in-house developed application on the HoloLens 2. The accuracy of needle placement was calculated based on the virtual 3D models of the needle and targeted phantom nerve, derived from CBCT scans. Results: The median Euclidean distance between the tip of the needle and the target is found to be 4.37 (IQR 5.16) mm, the median lateral distance is 3.25 (IQR 4.62) mm and the median depth distance is 1.94 (IQR 7.07) mm. Conclusion: In this study, the first method is described in which the accuracy of patient-specific needle placement using AR guidance is determined. This method could potentially improve the accuracy of PRF needle placement for pudendal neuralgia, resulting in improved treatment outcomes.

[Experts consensus on transbronchial diagnosis, localization and treatment of peripheral pulmonary nodules guided by the augmented reality optical lung navigation].

Lung cancer is the second most common malignancy with the highest mortality rate worldwide. In recent years, the rapid development of various bronchoscopic navigation techniques has provided conditions for the minimally invasive diagnosis and treatment of peripheral pulmonary nodules through the airway.Augmented reality optical lung navigation is a new technology that combined virtual bronchoscopy navigation (VBN) with augmented reality (AR) and optical navigation technology, which could assist bronchoscopist and has been widely applied in clinics. The clinical evidence certified that the navigation, has the advantages of safety and efficacy in guiding transbronchial diagnosis, localization, and treatment of pulmonary nodules. In order to standardize the clinical operation of augmented reality optical lung navigation technology and guide its application in clinical practice, Interventional Group, Society of Respiratory Diseases, Chinese Medical Association/Interventional Pulmonology Group of the Zhejiang Medical Association organized multidisciplinary experts to take the lead in formulating the Consensus of experts on transbronchial diagnosis, localization and treatment of peripheral pulmonary nodules guided by the augmented reality optical lung navigation after multiple rounds of discussion, and provided recommendation opinions and clinical guidance for the indications and contraindications, equipment and devices, perioperative treatment, operating process and complication management of peripheral pulmonary nodules applicable to augmented reality optical lung diagnosis navigation technology.

Using Mixed Reality Simulation to Improve Junior Medical Trainees' Preparedness to Manage High-Acuity Trauma.

High-acuity trauma necessitates experienced and rapid intervention to prevent patient harm. However, upskilling junior trainees through hands-on management of real trauma cases is rarely feasible without compromising patient safety. This quality education report sought to investigate whether a simulation course operated via mixed reality (MR) headset devices (Microsoft HoloLens) could enhance the clinical knowledge recall and preparedness to practice of junior trainees with no prior experience managing trauma.The Plan-Do-Study-Act quality improvement method was used to refine six emergency trauma vignettes compatible with an MR teaching platform. Each vignette was curated by a multidisciplinary team of orthopaedic surgeons, clinical fellows and experts in simulation-based medical education. As a baseline assessment, a 2-hour emergency trauma course was delivered using traditional didactic methods to a cohort of pre-registration medical students with no clinical exposure to high-acuity trauma (n=16). Next, we delivered the MR simulation to an equivalent cohort (n=32). Clinical knowledge scores derived from written test papers were recorded for each group during and 2 weeks after each course. Each attendee's end-of-rotation clinical supervisor appraisal grade was recorded, as determined by a consultant surgeon who supervised participants during a 2-week placement on a major trauma ward. Balancing measures included participant feedback and validated cognitive load questionnaires (National Aeronautics and Space Administration-Task Load Index).Overall, attendees of the MR simulation course achieved and sustained higher clinical knowledge scores and were more likely to receive a positive consultant supervisor appraisal. This project serves as a proof of concept that MR wearable technologies can be used to improve clinical knowledge recall and enhance the preparedness to practice of novice learners with otherwise limited clinical exposure to high-acuity trauma.

Comparison between accuracy of augmented reality computed tomography-based and portable augmented reality-based navigation systems for cup insertion in total hip arthroplasty.

Augmented reality (AR) has been used for navigation during total hip arthroplasty (THA). AR computed tomography (CT)-based navigation systems and AR-based portable navigation systems that use smartphones can also be used. This study compared the accuracy of cup insertion during THA using AR-CT-based and portable AR-based navigation systems. Patients with symptomatic hip disease who underwent primary THA in the supine position using both AR CT-based and portable AR-based navigation systems simultaneously between October 2021 and July 2023 were included. The primary outcome of this study was the absolute difference between cup angles in the intraoperative navigation record and those measured on postoperative CT. The secondary outcome was to determine the factors affecting the absolute value of the navigation error in radiographic inclination (RI) and radiographic anteversion (RA) of the cup, including sex, age, body mass index, left or right side, approach, and preoperative pelvic tilt. This study included 94 consecutive patients. There were 11 men and 83 women, with a mean age of 68 years. The mean absolute errors of RI were 2.7° ± 2.0° in the AR CT-based and 3.3° ± 2.4° in the portable AR-based navigation system. The mean absolute errors of RA were 2.5° ± 2.1° in the AR CT-based navigation system and 2.3° ± 2.2° in the portable AR-based navigation system. No significant differences were observed in RI or RA of the cup between the two navigation systems (RI: p = 0.706; RA: p = 0.329). No significant factors affected the absolute value of the navigation errors in RI and RA. In conclusion, there were no differences in the accuracy of cup insertion between the AR CT-based and portable AR-based navigation systems.

Reflections on augmented reality codes for teaching fundamental defensive techniques to boxing beginners.

AR technology allows users to interact with virtual objects in real-world settings. Immersive AR experiences can enhance creativity and possibilities. Learners can explore real-life situations in a safe, controlled environment, understand abstract concepts and solve problems. This study investigates whether AR-codes affect boxing beginners' performance in some fundamental defensive techniques. An experimental and control group were created to implement a quasi-experimental design. By using the ASSURE instructional design model, AR technology was incorporated into the educational program and delivered in flipped classroom method to the experimental group. Comparatively, the control group is taught a program using a teacher's command style. A post-measurement of defensive boxing skills was conducted for both groups. Participants were 60 boxing beginners aged 12 to 14 who had enrolled in Port Fouad Sports Club's 2023/2024 training season in Port Said, Egypt. Randomly, participants were divided into control and experimental groups. They were homogenized and equivalent in terms of age, height, weight, IQ, physical fitness, and skill level. According to the study results, the experimental group performed better in post-measurements than the control group. The AR Codes technology had a large effect size on the learning of boxing defensive skills under study. Consequently, it is necessary to use AR Codes technology as an educational resource to enhance the learning process, integrate it with active learning strategies, and use it to teach defensive boxing skills and apply them to offensive and counterattack skills, thereby improving the learning process.