Reliability of ChatGPT for performing triage task in the emergency department using the Korean Triage and Acuity Scale.

Background: Artificial intelligence (AI) technology can enable more efficient decision-making in healthcare settings. There is a growing interest in improving the speed and accuracy of AI systems in providing responses for given tasks in healthcare settings. Objective: This study aimed to assess the reliability of ChatGPT in determining emergency department (ED) triage accuracy using the Korean Triage and Acuity Scale (KTAS). Methods: Two hundred and two virtual patient cases were built. The gold standard triage classification for each case was established by an experienced ED physician. Three other human raters (ED paramedics) were involved and rated the virtual cases individually. The virtual cases were also rated by two different versions of the chat generative pre-trained transformer (ChatGPT, 3.5 and 4.0). Inter-rater reliability was examined using Fleiss' kappa and intra-class correlation coefficient (ICC). Results: The kappa values for the agreement between the four human raters and ChatGPTs were .523 (version 4.0) and .320 (version 3.5). Of the five levels, the performance was poor when rating patients at levels 1 and 5, as well as case scenarios with additional text descriptions. There were differences in the accuracy of the different versions of GPTs. The ICC between version 3.5 and the gold standard was .520, and that between version 4.0 and the gold standard was .802. Conclusions: A substantial level of inter-rater reliability was revealed when GPTs were used as KTAS raters. The current study showed the potential of using GPT in emergency healthcare settings. Considering the shortage of experienced manpower, this AI method may help improve triaging accuracy.

Smart glasses and telehealth services by professionals in isolated areas in Korea: Acceptability and concerns.

BACKGROUND: Smart glass technology offers remote interaction between health professionals for telehealth, alleviating healthcare disparities in isolated areas. OBJECTIVE: To evaluate the professionals' perceptions of smart glass technology as a tool for telehealth and distance learning. METHOD: This mixed-method study on health professionals in 10 different island areas in Korea involved participants experiencing a smart glass-based telehealth system using the scenario of clinical consultation with remote specialists. A group pre- and post-test design was used to examine the change in attitude and perceived importance among health professionals about using smart glasses in telehealth. RESULTS: Forty-seven participants completed both pre-and post-evaluation of smart glasses. A positive, statistically significant change in participants' perceptions of smart glasses regarding their implications for telehealth and distance learning (p< 0.05) was found. Fifty-one health workers provided feedback on smart glasses, and a majority expressed their expectations of quality care with telehealth using advanced technology. The main concerns were patients' privacy issues and inadequate technology for seamless application. CONCLUSION: The incorporation of smart glass technology offers great potential to enrich telehealth as well as distance learning for unskilled health professionals in isolated areas. Future studies are needed to increase efforts to secure a high level of acceptance for clinical consultation with remote specialists on this newly developed device.

Google Glass-Supported Cooperative Training for Health Professionals: A Case Study Based on Using Remote Desktop Virtual Support.

PURPOSE: Observation of medical trainees' care performance by experts can be extremely helpful for ensuring safety and providing quality care. The advanced technology of smart glasses enables health professionals to video stream their operations to remote supporters for collaboration and cooperation. This study monitored the clinical situation by using smart glasses for remote cooperative training via video streaming and clinical decision-making through simulation based on a scenario of emergency nursing care for patients with arrhythmia. PARTICIPANTS AND METHODS: The clinical operations of bedside trainees, who is Google Glass Enterprise Edition 2(Glass EE2) wearers, were live streamed via their Google Glasses, which were viewed at a remote site by remote supporters via a desktop computer. Data were obtained from 31 nursing students using eight essay questions regarding their experience as desktop-side remote supporters. RESULTS: Most of the participants reported feeling uneasy about identifying clinical situations (84%), patients' condition (72%), and trainees' performance (69%). The current system demonstrated sufficient performance with a satisfactory level of image quality and auditory communication, while network and connectivity are areas that require further improvement. The reported barriers to identifying situations on the remote desktop were predominantly a narrow field of view and motion blur in videos captured by Glass EE2s, and using the customized mirror mode. CONCLUSION: The current commercial Glass EE2 can facilitate enriched communication between remotely located supporters and trainees by sharing live videos and audio during clinical operations. Further improvement of hardware and software user interfaces will ensure better applicability of smart glasses and video streaming functions to clinical practice settings.

Adaptation of Extended Reality Smart Glasses for Core Nursing Skill Training Among Undergraduate Nursing Students: Usability and Feasibility Study.

BACKGROUND: Skill training in nursing education has been highly dependent on self-training because of Korea's high student-faculty ratio. Students tend to have a passive attitude in self-practice, and it is hard to expect effective learning outcomes with traditional checklist-dependent self-practice. Smart glasses have a high potential to assist nursing students with timely information, and a hands-free device does not interrupt performance. OBJECTIVE: This study aimed to develop a smart glass-based nursing skill training program and evaluate its usability and feasibility for the implementation of self-practice. METHODS: We conducted a usability and feasibility study with 30 undergraduate nursing students during a 2-hour open lab for self-practice of core nursing skills, wearing smart glasses for visualized guidance. The usability test was conducted using a 16-item self-reporting questionnaire and 7 open-ended questions. Learning satisfaction was assessed using a 7-item questionnaire. The number of practice sessions was recorded, and perceived competency in core nursing skills was measured before and after the intervention. At the final evaluation, performance accuracy and time consumed for completion were recorded. RESULTS: Smart glass-assisted self-practice of nursing skills was perceived as helpful, convenient, and interesting. Participants reported improved recollection of sequences of skills, and perceived competency was significantly improved. Several issues were raised by participants regarding smart glasses, including small screen size, touch sensors, fogged lenses with masks, heaviness, and heat after a period of time. CONCLUSIONS: Smart glasses have the potential to assist self-practice, providing timely information at students' own paces. Having both hands free from holding a device, participants reported the convenience of learning as they could practice and view the information simultaneously. Further revision correcting reported issues would improve the applicability of smart glasses in other areas of nursing education.