Ηand dexterities assessment in stroke patients based on augmented reality and machine learning through a box and block test.

A popular and widely suggested measure for assessing unilateral hand motor skills in stroke patients is the box and block test (BBT). Our study aimed to create an augmented reality enhanced version of the BBT (AR-BBT) and evaluate its correlation to the original BBT for stroke patients. Following G-power analysis, clinical examination, and inclusion-exclusion criteria, 31 stroke patients were included in this study. AR-BBT was developed using the Open Source Computer Vision Library (OpenCV). The MediaPipe's hand tracking library uses a palm and a hand landmark machine learning model to detect and track hands. A computer and a depth camera were employed in the clinical evaluation of AR-BBT following the principles of traditional BBT. A strong correlation was achieved between the number of blocks moved in the BBT and the AR-BBT on the hemiplegic side (Pearson correlation = 0.918) and a positive statistically significant correlation (p = 0.000008). The conventional BBT is currently the preferred assessment method. However, our approach offers an advantage, as it suggests that an AR-BBT solution could remotely monitor the assessment of a home-based rehabilitation program and provide additional hand kinematic information for hand dexterities in AR environment conditions. Furthermore, it employs minimal hardware equipment.

The application of virtual reality and augmented reality in dentistry - a literature review.

In recent times, dentistry has seen significant technological advancements that have transformed various specialized areas within the field. Developed into applications for mobile devices, augmented reality (AR) seamlessly merges digital components with the physical world, enhancing both realms while maintaining their individual separateness. On the other hand, virtual reality (VR) relies on advanced, tailored software to visualize a digital 3D environment stimulating the operator's senses through computer generated sensations and feedback. The current advances use the application of VR, haptic simulators, the use of an AI algorithm and many more that provides new opportunities for smart learning and enhance the teaching environment. As this technology continues to evolve, it is poised to become even more remarkable, enabling specialists to potentially visualize both soft and hard tissues within the patient's body for effective treatment planning. This literature aims to present the newest advancements and ongoing development of AR and VR in dentistry and medicine. It highlights their diverse applications while identifying areas needing further research for effective integration into clinical practice.

Assessment of pulmonary vascular anatomy: comparing augmented reality by holograms versus standard CT images/reconstructions using surgical findings as reference standard.

BACKGROUND: We compared computed tomography (CT) images and holograms (HG) to assess the number of arteries of the lung lobes undergoing lobectomy and assessed easiness in interpretation by radiologists and thoracic surgeons with both techniques. METHODS: Patients scheduled for lobectomy for lung cancer were prospectively included and underwent CT for staging. A patient-specific three-dimensional model was generated and visualized in an augmented reality setting. One radiologist and one thoracic surgeon evaluated CT images and holograms to count lobar arteries, having as reference standard the number of arteries recorded at surgery. The easiness of vessel identification was graded according to a Likert scale. Wilcoxon signed-rank test and κ statistics were used. RESULTS: Fifty-two patients were prospectively included. The two doctors detected the same number of arteries in 44/52 images (85%) and in 51/52 holograms (98%). The mean difference between the number of artery branches detected by surgery and CT images was 0.31 ± 0.98, whereas it was 0.09 ± 0.37 between surgery and HGs (p = 0.433). In particular, the mean difference in the number of arteries detected in the upper lobes was 0.67 ± 1.08 between surgery and CT images and 0.17 ± 0.46 between surgery and holograms (p = 0.029). Both radiologist and surgeon showed a higher agreement for holograms (κ = 0.99) than for CT (κ = 0.81) and found holograms easier to evaluate than CTs (p < 0.001). CONCLUSIONS: Augmented reality by holograms is an effective tool for preoperative vascular anatomy assessment of lungs, especially when evaluating the upper lobes, more prone to anatomical variations. TRIAL REGISTRATION: ClinicalTrials.gov, NCT04227444 RELEVANCE STATEMENT: Preoperative evaluation of the lung lobe arteries through augmented reality may help the thoracic surgeons to carefully plan a lobectomy, thus contributing to optimize patients' outcomes. KEY POINTS: • Preoperative assessment of the lung arteries may help surgical planning. • Lung artery detection by augmented reality was more accurate than that by CT images, particularly for the upper lobes. • The assessment of the lung arterial vessels was easier by using holograms than CT images.

Exploring the knowledge and awareness on applications of virtual reality and augmented reality technology among dental healthcare professionals - a crosssectional survey.

OBJECTIVE: To assess the knowledge and awareness of Virtual Reality (VR) and Augmented Reality (AR) technology in dentistry. METHODS: A questionnaire survey-based study was conducted using Google forms on a sample of 273 dental healthcare professionals (DHCP) from October- November 2023, after obtaining ethical approval. A validated questionnaire, divided into three sections, was used to assess the knowledge and awareness of dental healthcare professionals on virtual and augmented reality. Section A was about demographic statistics, section B assessed knowledge and awareness regarding VR and AR and section C consisted of future acceptability of VR and AR among DHCP. The frequency of each question was reported in percentages. To assess the difference of knowledge and awareness of AR and VR among different specialties of DHCP, one-way ANOVA test was applied and in case of significant results pairwise comparison was performed by post-hoc Tukey test. RESULTS: There was a statistically significant difference of knowledge (1.40 ± 0.49) among different dental healthcare professionals. On pairwise comparison, a statistically significant difference (p = ˂0.05) of knowledge and awareness of AR and VR was found among dental specialist and other dental health professionals. CONCLUSIONS: A concerning lack of knowledge and awareness among dental healthcare professionals regarding AR and VR technology in dentistry was found. Interestingly, within the spectrum of specialties, dental specialists demonstrated a comparatively higher awareness than their counterparts in other specialties. Addressing barriers, notably a lack of knowledge, is crucial for successful technology adoption in dental education and practice.

Current Applications of VR/AR (Virtual Reality/Augmented Reality) in Pediatric Neurosurgery.

Neurosurgical procedures are some of the most complex procedures in medicine and since the advent of the field, planning, performing, and learning them has challenged the neurosurgeon. Virtual reality (VR) and augmented reality (AR) are making these challenges more manageable. VR refers to a virtual digital environment that can be experienced usually through use of stereoscopic glasses and controllers. AR, on the other hand, fuses the natural environment with virtual images, such as superimposing a preoperative MRI image on to the surgical field [1]. They initially were used primarily as neuronavigational tools but soon their potential in other areas of surgery, such as planning, education, and assessment, was noted and explored. Through this chapter, we outline the history and evolution of these two technologies over the past few decades, describe the current state of the technology and its uses, and postulate future directions for research and implementation.

Application of augmented reality for crime scene investigation training and education.

The role of the crime scene investigator is complex, and investigators need to be able to conduct multiple frequently performed tasks. Appropriate training and education are critical to impart the crime scene investigator with the necessary capabilities. For a range of reasons, including the COVID-19 pandemic and the need for multi-disciplinary capabilities, training and education requirements have evolved in recent times to be more flexible and accessible. The skills of a crime scene investigator can be broadly categorised into two types of fundamental skills: decision-making and psychomotor skills. Both these skills need to be taught and assessed within training and education programs. The most common approach to impart these skills is by incorporating a crime scene simulation facility; however, not everyone has access to these facilities. Furthermore, crime scene staff often undertake refresher courses and are required to complete proficiency assessments. Conducting these activities in a dedicated crime scene simulation facility is time consuming and costly. Virtual tools have been developed in recent years to address this, but these tools only assess decision-making skills and not psychomotor skills. This paper argues that the implementation of augmented reality (AR) technology should be considered in crime scene investigator training and university education, because it can provide significant advantages when paired with conventional methods of training and education. When appropriately managed, AR can provide an avenue of training where both decision-making and psychomotor skills can be addressed simultaneously, while providing a more flexible and accessible approach. The implementation of AR has the potential to significantly improve the standards of teaching, resulting in better equipped crime scene investigators. The paper will explore how AR has the potential to improve accessibility of training, increase safety, enhance the student experience, enhance collaboration and feedback through connectivity and potentially reduce cost. This paper will also provide an insight into what would need to be considered before implementing AR technology into crime scene investigation training and education alongside current approaches. It is argued that the inclusion of AR into the current training and education arsenal provides significant benefits that are worthy of exploration.

Mismatch between Augmented Reality Navigation Images and Actual Location of a Cauda Equina Tumor:A Case Report.

BACKGROUND: Augmented reality navigation is the one of the navigation technologies that allows computer-generated virtual images to be projected onto a real-world environment. Augmented reality navigation can be used in spinal tumor surgery. However, it is unknown if there are any pitfalls when using this technique. CASE PRESENTATION: The patient in this report underwent complete resection of a cauda equina tumor at the L2-L3 level using microscope-based augmented reality navigation. Although the registration error of navigation was <1 mm, we found a discrepancy between the augmented reality navigation images and the actual location of the tumor, which we have called "navigation mismatch". This mismatch, which was caused by the mobility of the spinal tumor in the dura mater, seems to be one of the pitfalls of augmented reality navigation for spinal tumors. CONCLUSIONS: Combined use of intraoperative ultrasound and augmented reality navigation seems advisable in such cases. J. Med. Invest. 71 : 174-176, February, 2024.

A Framework for Real-Time Gestural Recognition and Augmented Reality for Industrial Applications.

In recent decades, technological advancements have transformed the industry, highlighting the efficiency of automation and safety. The integration of augmented reality (AR) and gesture recognition has emerged as an innovative approach to create interactive environments for industrial equipment. Gesture recognition enhances AR applications by allowing intuitive interactions. This study presents a web-based architecture for the integration of AR and gesture recognition, designed to interact with industrial equipment. Emphasizing hardware-agnostic compatibility, the proposed structure offers an intuitive interaction with equipment control systems through natural gestures. Experimental validation, conducted using Google Glass, demonstrated the practical viability and potential of this approach in industrial operations. The development focused on optimizing the system's software and implementing techniques such as normalization, clamping, conversion, and filtering to achieve accurate and reliable gesture recognition under different usage conditions. The proposed approach promotes safer and more efficient industrial operations, contributing to research in AR and gesture recognition. Future work will include improving the gesture recognition accuracy, exploring alternative gestures, and expanding the platform integration to improve the user experience.

Navigate biopsy with ultrasound under augmented reality device: Towards higher system performance.

PURPOSE: Biopsies play a crucial role in determining the classification and staging of tumors. Ultrasound is frequently used in this procedure to provide real-time anatomical information. Using augmented reality (AR), surgeons can visualize ultrasound data and spatial navigation information seamlessly integrated with real tissues. This innovation facilitates faster and more precise biopsy operations. METHODS: We have developed an augmented reality biopsy navigation system characterized by low display latency and high accuracy. Ultrasound data is initially read by an image capture card and streamed to Unity via net communication. In Unity, navigation information is rendered and transmitted to the HoloLens 2 device using holographic remoting. Concurrently, a retro-reflective tool tracking method is implemented on the HoloLens 2, enabling the simultaneous tracking of the ultrasound probe and biopsy needle. Distinct navigation information is provided during in-plane and out-of-plane punctuation. To evaluate the effectiveness of our system, we conducted a study involving ten participants, assessing puncture accuracy and biopsy time in comparison to traditional methods. RESULTS: Ultrasound image was streamed from the ultrasound device to augmented reality headset with 122.49±11.61ms latency, while only 16.22±11.25ms was taken after data acquisition from image capture card. Navigation accuracy reached 1.23±0.68mm in the image plane and 0.95±0.70mm outside the image plane, within a depth range of 200 millimeters. Remarkably, the utilization of our system led to 98% and 95% success rate in out-of-plane and in-plane biopsy, among ten participants with little ultrasound experience. CONCLUSION: To sum up, this paper introduces an AR-based ultrasound biopsy navigation system characterized by high navigation accuracy and minimal latency. The system provides distinct visualization contents during in-plane and out-of-plane operations according to their different characteristics. Use case study in this paper proved that our system can help young surgeons perform biopsy faster and more accurately.

Extended reality in cranial and spinal neurosurgery - a bibliometric analysis.

PURPOSE: This bibliometric analysis of the top 100 cited articles on extended reality (XR) in neurosurgery aimed to reveal trends in this research field. Gender differences in authorship and global distribution of the most-cited articles were also addressed. METHODS: A Web of Science electronic database search was conducted. The top 100 most-cited articles related to the scope of this review were retrieved and analyzed for trends in publications, journal characteristics, authorship, global distribution, study design, and focus areas. After a brief description of the top 100 publications, a comparative analysis between spinal and cranial publications was performed. RESULTS: From 2005, there was a significant increase in spinal neurosurgery publications with a focus on pedicle screw placement. Most articles were original research studies, with an emphasis on augmented reality (AR). In cranial neurosurgery, there was no notable increase in publications. There was an increase in studies assessing both AR and virtual reality (VR) research, with a notable emphasis on VR compared to AR. Education, surgical skills assessment, and surgical planning were more common themes in cranial studies compared to spinal studies. Female authorship was notably low in both groups, with no significant increase over time. The USA and Canada contributed most of the publications in the research field. CONCLUSIONS: Research regarding the use of XR in neurosurgery increased significantly from 2005. Cranial research focused on VR and resident education while spinal research focused on AR and neuronavigation. Female authorship was underrepresented. North America provides most of the high-impact research in this area.

[Experts consensus on transbronchial diagnosis, localization and treatment of peripheral pulmonary nodules guided by the augmented reality optical lung navigation].

Lung cancer is the second most common malignancy with the highest mortality rate worldwide. In recent years, the rapid development of various bronchoscopic navigation techniques has provided conditions for the minimally invasive diagnosis and treatment of peripheral pulmonary nodules through the airway.Augmented reality optical lung navigation is a new technology that combined virtual bronchoscopy navigation (VBN) with augmented reality (AR) and optical navigation technology, which could assist bronchoscopist and has been widely applied in clinics. The clinical evidence certified that the navigation, has the advantages of safety and efficacy in guiding transbronchial diagnosis, localization, and treatment of pulmonary nodules. In order to standardize the clinical operation of augmented reality optical lung navigation technology and guide its application in clinical practice, Interventional Group, Society of Respiratory Diseases, Chinese Medical Association/Interventional Pulmonology Group of the Zhejiang Medical Association organized multidisciplinary experts to take the lead in formulating the Consensus of experts on transbronchial diagnosis, localization and treatment of peripheral pulmonary nodules guided by the augmented reality optical lung navigation after multiple rounds of discussion, and provided recommendation opinions and clinical guidance for the indications and contraindications, equipment and devices, perioperative treatment, operating process and complication management of peripheral pulmonary nodules applicable to augmented reality optical lung diagnosis navigation technology.

Using Mixed Reality Simulation to Improve Junior Medical Trainees' Preparedness to Manage High-Acuity Trauma.

High-acuity trauma necessitates experienced and rapid intervention to prevent patient harm. However, upskilling junior trainees through hands-on management of real trauma cases is rarely feasible without compromising patient safety. This quality education report sought to investigate whether a simulation course operated via mixed reality (MR) headset devices (Microsoft HoloLens) could enhance the clinical knowledge recall and preparedness to practice of junior trainees with no prior experience managing trauma.The Plan-Do-Study-Act quality improvement method was used to refine six emergency trauma vignettes compatible with an MR teaching platform. Each vignette was curated by a multidisciplinary team of orthopaedic surgeons, clinical fellows and experts in simulation-based medical education. As a baseline assessment, a 2-hour emergency trauma course was delivered using traditional didactic methods to a cohort of pre-registration medical students with no clinical exposure to high-acuity trauma (n=16). Next, we delivered the MR simulation to an equivalent cohort (n=32). Clinical knowledge scores derived from written test papers were recorded for each group during and 2 weeks after each course. Each attendee's end-of-rotation clinical supervisor appraisal grade was recorded, as determined by a consultant surgeon who supervised participants during a 2-week placement on a major trauma ward. Balancing measures included participant feedback and validated cognitive load questionnaires (National Aeronautics and Space Administration-Task Load Index).Overall, attendees of the MR simulation course achieved and sustained higher clinical knowledge scores and were more likely to receive a positive consultant supervisor appraisal. This project serves as a proof of concept that MR wearable technologies can be used to improve clinical knowledge recall and enhance the preparedness to practice of novice learners with otherwise limited clinical exposure to high-acuity trauma.

Comparison between accuracy of augmented reality computed tomography-based and portable augmented reality-based navigation systems for cup insertion in total hip arthroplasty.

Augmented reality (AR) has been used for navigation during total hip arthroplasty (THA). AR computed tomography (CT)-based navigation systems and AR-based portable navigation systems that use smartphones can also be used. This study compared the accuracy of cup insertion during THA using AR-CT-based and portable AR-based navigation systems. Patients with symptomatic hip disease who underwent primary THA in the supine position using both AR CT-based and portable AR-based navigation systems simultaneously between October 2021 and July 2023 were included. The primary outcome of this study was the absolute difference between cup angles in the intraoperative navigation record and those measured on postoperative CT. The secondary outcome was to determine the factors affecting the absolute value of the navigation error in radiographic inclination (RI) and radiographic anteversion (RA) of the cup, including sex, age, body mass index, left or right side, approach, and preoperative pelvic tilt. This study included 94 consecutive patients. There were 11 men and 83 women, with a mean age of 68 years. The mean absolute errors of RI were 2.7° ± 2.0° in the AR CT-based and 3.3° ± 2.4° in the portable AR-based navigation system. The mean absolute errors of RA were 2.5° ± 2.1° in the AR CT-based navigation system and 2.3° ± 2.2° in the portable AR-based navigation system. No significant differences were observed in RI or RA of the cup between the two navigation systems (RI: p = 0.706; RA: p = 0.329). No significant factors affected the absolute value of the navigation errors in RI and RA. In conclusion, there were no differences in the accuracy of cup insertion between the AR CT-based and portable AR-based navigation systems.

Keyhole-aware laparoscopic augmented reality.

Augmented Reality (AR) from preoperative data is a promising approach to improve intraoperative tumour localisation in Laparoscopic Liver Resection (LLR). Existing systems register the preoperative tumour model with the laparoscopic images and render it by direct camera projection, as if the organ were transparent. However, a simple geometric reasoning shows that this may induce serious surgeon misguidance. This is because the tools enter in a different keyhole than the laparoscope. As AR is particularly important for deep tumours, this problem potentially hinders the whole interest of AR guidance. A remedy to this issue is to project the tumour from its internal position to the liver surface towards the tool keyhole, and only then to the camera. This raises the problem of estimating the tool keyhole position in laparoscope coordinates. We propose a keyhole-aware pipeline which resolves the problem by using the observed tool to probe the keyhole position and by showing a keyhole-aware visualisation of the tumour. We assess the benefits of our pipeline quantitatively on a geometric in silico model and on a liver phantom model, as well as qualitatively on three patient data.

Reflections on augmented reality codes for teaching fundamental defensive techniques to boxing beginners.

AR technology allows users to interact with virtual objects in real-world settings. Immersive AR experiences can enhance creativity and possibilities. Learners can explore real-life situations in a safe, controlled environment, understand abstract concepts and solve problems. This study investigates whether AR-codes affect boxing beginners' performance in some fundamental defensive techniques. An experimental and control group were created to implement a quasi-experimental design. By using the ASSURE instructional design model, AR technology was incorporated into the educational program and delivered in flipped classroom method to the experimental group. Comparatively, the control group is taught a program using a teacher's command style. A post-measurement of defensive boxing skills was conducted for both groups. Participants were 60 boxing beginners aged 12 to 14 who had enrolled in Port Fouad Sports Club's 2023/2024 training season in Port Said, Egypt. Randomly, participants were divided into control and experimental groups. They were homogenized and equivalent in terms of age, height, weight, IQ, physical fitness, and skill level. According to the study results, the experimental group performed better in post-measurements than the control group. The AR Codes technology had a large effect size on the learning of boxing defensive skills under study. Consequently, it is necessary to use AR Codes technology as an educational resource to enhance the learning process, integrate it with active learning strategies, and use it to teach defensive boxing skills and apply them to offensive and counterattack skills, thereby improving the learning process.

Accuracy of augmented reality-guided needle placement for pulsed radiofrequency treatment of pudendal neuralgia: a pilot study on a phantom model.

Background: Pudendal neuralgia (PN) is a chronic neuropathy that causes pain, numbness, and dysfunction in the pelvic region. The current state-of-the-art treatment is pulsed radiofrequency (PRF) in which a needle is supposed to be placed close to the pudendal nerve for neuromodulation. Given the effective range of PRF of 5 mm, the accuracy of needle placement is important. This study aimed to investigate the potential of augmented reality guidance for improving the accuracy of needle placement in pulsed radiofrequency treatment for pudendal neuralgia. Methods: In this pilot study, eight subjects performed needle placements onto an in-house developed phantom model of the pelvis using AR guidance. AR guidance is provided using an in-house developed application on the HoloLens 2. The accuracy of needle placement was calculated based on the virtual 3D models of the needle and targeted phantom nerve, derived from CBCT scans. Results: The median Euclidean distance between the tip of the needle and the target is found to be 4.37 (IQR 5.16) mm, the median lateral distance is 3.25 (IQR 4.62) mm and the median depth distance is 1.94 (IQR 7.07) mm. Conclusion: In this study, the first method is described in which the accuracy of patient-specific needle placement using AR guidance is determined. This method could potentially improve the accuracy of PRF needle placement for pudendal neuralgia, resulting in improved treatment outcomes.

KIJANI: Designing a Physical Activity Promoting Collaborative Augmented Reality Game.

BACKGROUND: There is an increased need for physical activity among children and adolescents. KIJANI, a mobile augmented reality game, is designed to increase physical activity through gamified exercises. OBJECTIVES: The primary aim of this study is to get feedback on the design and implementation of potentially physical activity-increasing features in KIJANI. METHODS: A mixed-method study (n=13) evaluates newly implemented game design features quantitatively through measuring physical activity and qualitatively through participant feedback. RESULTS: Preliminary results are limited and need further studies. Participants' feedback shows a positive trend and highlights the game's potential effectiveness. CONCLUSION: KIJANI shows potential for increasing physical activity among children and adolescents through gamified exercise. Future work will refine the game based on user feedback and findings presented in related work. The game's long-term impact is to be explored.

[The impact of augmented reality glasses on human visual efficiency and digital eye fatigue].

Objective: To investigate the differences in reading efficiency and visual fatigue between the use of augmented reality (AR) glasses and laptops. Methods: A prospective self-controlled study was conducted. Healthy students from Capital Medical University who frequently engaged in long-term near work and used laptops and other digital display devices were recruited as subjects at Beijing Tongren Hospital, Capital Medical University between November 1 and November 15, 2023. LogMAR visual acuity, visual functions (accommodation, convergence, and fusion), and visual fatigue scores (Likert visual fatigue scale) of the participants were assessed. The order of using the laptop and AR glasses for each participant was determined by a coin toss. Reading efficiency (reading speed and error rate multiplied by the detection rate of incorrect numbers) with different devices for 10 minutes at the same time on different dates and visual fatigue scores after watching a 20-minute video were measured. Statistical analyses were performed using paired t-tests and Wilcoxon signed-rank tests. Results: A total of 20 eligible subjects were included, comprising 7 males and 13 females, with a mean age of (25.45±2.27) years. There was no significant change in binocular visual acuity before and after using AR glasses and laptops (both P>0.05). The reading speed and reading efficiency of using AR glasses [(34.03±9.25) and (29.19±7.62) digits/min, respectively] were significantly lower than those of using laptops [(39.43±10.36) and (35.67±9.87) digits/min, respectively] (t=4.36, P<0.001), while the difference in error detection rate was not statistically significant (t=1.29, P=0.213). There was no statistically significant difference in visual fatigue scores before watching videos with the two devices (Z=-0.71, P=0.480). However, the visual fatigue score after watching videos with AR glasses [(20.55±5.04) points] was significantly higher than that with laptops [16.50 (13.00, 19.75) points] (Z=-2.85, P=0.004). The visual fatigue scores after watching videos with both devices were significantly higher than before (P<0.05), with a more significant increase observed with AR glasses [(6.05±3.50) points] (Z=-3.41, P<0.001). Conclusion: Compared with using laptops, the reading speed and efficiency were lower, and the visual fatigue was more pronounced with the use of AR glasses at the current technical level. Further optimization and improvement of AR glasses are warranted.