Performance of ChatGPT Across Different Versions in Medical Licensing Examinations Worldwide: Systematic Review and Meta-Analysis.

BACKGROUND: Over the past 2 years, researchers have used various medical licensing examinations to test whether ChatGPT (OpenAI) possesses accurate medical knowledge. The performance of each version of ChatGPT on the medical licensing examination in multiple environments showed remarkable differences. At this stage, there is still a lack of a comprehensive understanding of the variability in ChatGPT's performance on different medical licensing examinations. OBJECTIVE: In this study, we reviewed all studies on ChatGPT performance in medical licensing examinations up to March 2024. This review aims to contribute to the evolving discourse on artificial intelligence (AI) in medical education by providing a comprehensive analysis of the performance of ChatGPT in various environments. The insights gained from this systematic review will guide educators, policymakers, and technical experts to effectively and judiciously use AI in medical education. METHODS: We searched the literature published between January 1, 2022, and March 29, 2024, by searching query strings in Web of Science, PubMed, and Scopus. Two authors screened the literature according to the inclusion and exclusion criteria, extracted data, and independently assessed the quality of the literature concerning Quality Assessment of Diagnostic Accuracy Studies-2. We conducted both qualitative and quantitative analyses. RESULTS: A total of 45 studies on the performance of different versions of ChatGPT in medical licensing examinations were included in this study. GPT-4 achieved an overall accuracy rate of 81% (95% CI 78-84; P<.01), significantly surpassing the 58% (95% CI 53-63; P<.01) accuracy rate of GPT-3.5. GPT-4 passed the medical examinations in 26 of 29 cases, outperforming the average scores of medical students in 13 of 17 cases. Translating the examination questions into English improved GPT-3.5's performance but did not affect GPT-4. GPT-3.5 showed no difference in performance between examinations from English-speaking and non-English-speaking countries (P=.72), but GPT-4 performed better on examinations from English-speaking countries significantly (P=.02). Any type of prompt could significantly improve GPT-3.5's (P=.03) and GPT-4's (P<.01) performance. GPT-3.5 performed better on short-text questions than on long-text questions. The difficulty of the questions affected the performance of GPT-3.5 and GPT-4. In image-based multiple-choice questions (MCQs), ChatGPT's accuracy rate ranges from 13.1% to 100%. ChatGPT performed significantly worse on open-ended questions than on MCQs. CONCLUSIONS: GPT-4 demonstrates considerable potential for future use in medical education. However, due to its insufficient accuracy, inconsistent performance, and the challenges posed by differing medical policies and knowledge across countries, GPT-4 is not yet suitable for use in medical education. TRIAL REGISTRATION: PROSPERO CRD42024506687; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=506687.

Immersive and Nonimmersive Virtual Reality-Assisted Active Training in Chronic Musculoskeletal Pain: Systematic Review and Meta-Analysis.

BACKGROUND: Virtual reality (VR) in different immersive conditions has been increasingly used as a nonpharmacological method for managing chronic musculoskeletal pain. OBJECTIVE: We aimed to assess the effectiveness of VR-assisted active training versus conventional exercise or physiotherapy in chronic musculoskeletal pain and to analyze the effects of immersive versus nonimmersive VR on pain outcomes. METHODS: This systematic review of randomized control trials (RCTs) searched PubMed, Scopus, and Web of Science databases from inception to June 9, 2024. RCTs comparing adults with chronic musculoskeletal pain receiving VR-assisted training were included. The primary outcome was pain intensity; secondary outcomes included functional disability and kinesiophobia. Available data were pooled in a meta-analysis. Studies were graded using the Cochrane Risk-of-Bias Tool version 2. RESULTS: In total, 28 RCTs including 1114 participants with some concerns for a high risk of bias were identified, and 25 RCTs were included in the meta-analysis. In low back pain, short-term outcomes measured post intervention showed that nonimmersive VR is effective in reducing pain (standardized mean difference [SMD] -1.79, 95% CI -2.72 to -0.87; P<.001), improving disability (SMD -0.44, 95% CI -0.72 to -0.16; P=.002), and kinesiophobia (SMD -2.94, 95% CI -5.20 to -0.68; P=.01). Intermediate-term outcomes measured at 6 months also showed that nonimmersive VR is effective in reducing pain (SMD -8.15, 95% CI -15.29 to -1.01; P=.03), and kinesiophobia (SMD -4.28, 95% CI -8.12 to -0.44; P=.03) compared to conventional active training. For neck pain, immersive VR reduced pain intensity (SMD -0.55, 95% CI -1.02 to -0.08; P=.02) but not disability and kinesiophobia in the short term. No statistical significances were detected for knee pain or other pain regions at all time points. In addition, 2 (8%) studies had a high risk of bias. CONCLUSIONS: Both nonimmersive and immersive VR-assisted active training is effective in reducing back and neck pain symptoms. Our study findings suggest that VR is effective in alleviating chronic musculoskeletal pain. TRIAL REGISTRATION: PROSPERO CRD42022302912; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=302912.

Mobile Apps to Support Mental Health Response in Natural Disasters: Scoping Review.

BACKGROUND: Disasters are becoming more frequent due to the impact of extreme weather events attributed to climate change, causing loss of lives, property, and psychological trauma. Mental health response to disasters emphasizes prevention and mitigation, and mobile health (mHealth) apps have been used for mental health promotion and treatment. However, little is known about their use in the mental health components of disaster management. OBJECTIVE: This scoping review was conducted to explore the use of mobile phone apps for mental health responses to natural disasters and to identify gaps in the literature. METHODS: We identified relevant keywords and subject headings and conducted comprehensive searches in 6 electronic databases. Studies in which participants were exposed to a man-made disaster were included if the sample also included some participants exposed to a natural hazard. Only full-text studies published in English were included. The initial titles and abstracts of the unique papers were screened by 2 independent review authors. Full texts of the selected papers that met the inclusion criteria were reviewed by the 2 independent reviewers. Data were extracted from each selected full-text paper and synthesized using a narrative approach based on the outcome measures, duration, frequency of use of the mobile phone apps, and the outcomes. This scoping review was reported according to the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews). RESULTS: Of the 1398 papers retrieved, 5 were included in this review. A total of 3 studies were conducted on participants exposed to psychological stress following a disaster while 2 were for disaster relief workers. The mobile phone apps for the interventions included Training for Life Skills, Sonoma Rises, Headspace, Psychological First Aid, and Substance Abuse and Mental Health Services Administration (SAMHSA) Behavioural Health Disaster Response Apps. The different studies assessed the effectiveness or efficacy of the mobile app, feasibility, acceptability, and characteristics of app use or predictors of use. Different measures were used to assess the effectiveness of the apps' use as either the primary or secondary outcome. CONCLUSIONS: A limited number of studies are exploring the use of mobile phone apps for mental health responses to disasters. The 5 studies included in this review showed promising results. Mobile apps have the potential to provide effective mental health support before, during, and after disasters. However, further research is needed to explore the potential of mobile phone apps in mental health responses to all hazards.

Automated Paper Screening for Clinical Reviews Using Large Language Models: Data Analysis Study.

BACKGROUND: The systematic review of clinical research papers is a labor-intensive and time-consuming process that often involves the screening of thousands of titles and abstracts. The accuracy and efficiency of this process are critical for the quality of the review and subsequent health care decisions. Traditional methods rely heavily on human reviewers, often requiring a significant investment of time and resources. OBJECTIVE: This study aims to assess the performance of the OpenAI generative pretrained transformer (GPT) and GPT-4 application programming interfaces (APIs) in accurately and efficiently identifying relevant titles and abstracts from real-world clinical review data sets and comparing their performance against ground truth labeling by 2 independent human reviewers. METHODS: We introduce a novel workflow using the Chat GPT and GPT-4 APIs for screening titles and abstracts in clinical reviews. A Python script was created to make calls to the API with the screening criteria in natural language and a corpus of title and abstract data sets filtered by a minimum of 2 human reviewers. We compared the performance of our model against human-reviewed papers across 6 review papers, screening over 24,000 titles and abstracts. RESULTS: Our results show an accuracy of 0.91, a macro F1-score of 0.60, a sensitivity of excluded papers of 0.91, and a sensitivity of included papers of 0.76. The interrater variability between 2 independent human screeners was κ=0.46, and the prevalence and bias-adjusted κ between our proposed methods and the consensus-based human decisions was κ=0.96. On a randomly selected subset of papers, the GPT models demonstrated the ability to provide reasoning for their decisions and corrected their initial decisions upon being asked to explain their reasoning for incorrect classifications. CONCLUSIONS: Large language models have the potential to streamline the clinical review process, save valuable time and effort for researchers, and contribute to the overall quality of clinical reviews. By prioritizing the workflow and acting as an aid rather than a replacement for researchers and reviewers, models such as GPT-4 can enhance efficiency and lead to more accurate and reliable conclusions in medical research.

Design of Digital Mental Health Platforms for Family Member Cocompletion: Scoping Review.

BACKGROUND: The COVID-19 pandemic placed an additional mental health burden on individuals and families, resulting in widespread service access problems. Digital mental health interventions suggest promise for improved accessibility. Recent reviews have shown emerging evidence for individual use and early evidence for multiusers. However, attrition rates remain high for digital mental health interventions, and additional complexities exist when engaging multiple family members together. OBJECTIVE: As such, this scoping review aims to detail the reported evidence for digital mental health interventions designed for family use with a focus on the build and design characteristics that promote accessibility and engagement and enable cocompletion by families. METHODS: A systematic literature search of MEDLINE, Embase, PsycINFO, Web of Science, and CINAHL databases was conducted for articles published in the English language from January 2002 to March 2024. Eligible records included empirical studies of digital platforms containing some elements designed for cocompletion by related people as well as some components intended to be completed without therapist engagement. Platforms were included in cases in which clinical evidence had been documented. RESULTS: Of the 9527 papers reviewed, 85 (0.89%) met the eligibility criteria. A total of 24 unique platforms designed for co-use by related parties were identified. Relationships between participants included couples, parent-child dyads, family caregiver-care recipient dyads, and families. Common platform features included the delivery of content via structured interventions with no to minimal tailoring or personalization offered. Some interventions provided live contact with therapists. User engagement indicators and findings varied and included user experience, satisfaction, completion rates, and feasibility. Our findings are more remarkable for what was absent in the literature than what was present. Contrary to expectations, few studies reported any design and build characteristics that enabled coparticipation. No studies reported on platform features for enabling cocompletion or considerations for ensuring individual privacy and safety. None examined platform build or design characteristics as moderators of intervention effect, and none offered a formative evaluation of the platform itself. CONCLUSIONS: In this early era of digital mental health platform design, this novel review demonstrates a striking absence of information about design elements associated with the successful engagement of multiple related users in any aspect of a therapeutic process. There remains a large gap in the literature detailing and evaluating platform design, highlighting a significant opportunity for future cross-disciplinary research. This review details the incentive for undertaking such research; suggests design considerations when building digital mental health platforms for use by families; and offers recommendations for future development, including platform co-design and formative evaluation.

Effectiveness of eHealth Interventions in Improving Medication Adherence Among Patients With Cardiovascular Disease: Systematic Review and Meta-Analysis.

BACKGROUND: Nonadherence to medication among patients with cardiovascular diseases undermines the desired therapeutic outcomes. eHealth interventions emerge as promising strategies to effectively tackle this issue. OBJECTIVE: The aim of this study was to conduct a network meta-analysis (NMA) to compare and rank the efficacy of various eHealth interventions in improving medication adherence among patients with cardiovascular diseases (CVDs). METHODS: A systematic search strategy was conducted in PubMed, Embase, Web of Science, Cochrane, China National Knowledge Infrastructure Library (CNKI), China Science and Technology Journal Database (Weipu), and WanFang databases to search for randomized controlled trials (RCTs) published from their inception on January 15, 2024. We carried out a frequentist NMA to compare the efficacy of various eHealth interventions. The quality of the literature was assessed using the risk of bias tool from the Cochrane Handbook (version 2.0), and extracted data were analyzed using Stata16.0 (StataCorp LLC) and RevMan5.4 software (Cochrane Collaboration). The certainty of evidence was evaluated using the Grading of Recommendations, Assessment, Development, and Evaluations (GRADE) approach. RESULTS: A total of 21 RCTs involving 3904 patients were enrolled. The NMA revealed that combined interventions (standardized mean difference [SMD] 0.89, 95% CI 0.22-1.57), telephone support (SMD 0.68, 95% CI 0.02-1.33), telemonitoring interventions (SMD 0.70, 95% CI 0.02-1.39), and mobile phone app interventions (SMD 0.65, 95% CI 0.01-1.30) were statistically superior to usual care. However, SMS compared to usual care showed no statistical difference. Notably, the combined intervention, with a surface under the cumulative ranking curve of 79.3%, appeared to be the most effective option for patients with CVDs. Regarding systolic blood pressure and diastolic blood pressure outcomes, the combined intervention also had the highest probability of being the best intervention. CONCLUSIONS: The research indicates that the combined intervention (SMS text messaging and telephone support) has the greatest likelihood of being the most effective eHealth intervention to improve medication adherence in patients with CVDs, followed by telemonitoring, telephone support, and app interventions. The results of these network meta-analyses can provide crucial evidence-based support for health care providers to enhance patients' medication adherence. Given the differences in the design and implementation of eHealth interventions, further large-scale, well-designed multicenter trials are needed. TRIAL REGISTRATION: INPLASY 2023120063; https://inplasy.com/inplasy-2023-12-0063/.

Race, Ethnicity, and Other Cultural Background Factors in Trials of Internet-Based Cognitive Behavioral Therapy for Depression: Systematic Review.

BACKGROUND: There is a growing interest in developing scalable interventions, including internet-based cognitive behavioral therapy (iCBT), to meet the increasing demand for mental health services. Given the growth in diversity worldwide, it is essential that the clinical trials of iCBT for depression include diverse samples or, at least, report information on the race, ethnicity, or other background indicators of their samples. Unfortunately, the field lacks data on how well diversity is currently reported and represented in the iCBT literature. OBJECTIVE: Thus, the main objective of this systematic review was to examine the overall reporting of racial and ethnic identities in published clinical trials of iCBT for depression. We also aimed to review the representation of specific racial and ethnic minoritized groups and the inclusion of alternative background indicators such as migration status or country of residence. METHODS: Studies were included if they were randomized controlled trials in which iCBT was compared to a waiting list, care-as-usual, active control, or another iCBT. The included papers also had to have a focus on acute treatment (eg, 4 weeks to 6 months) of depression, be delivered via the internet on a website or a smartphone app and use guided or unguided self-help. Studies were initially identified from the METAPSY database (n=59) and then extended to include papers up to 2022, with papers retrieved from Embase, PubMed, PsycINFO, and Cochrane (n=3). Risk of bias assessment suggested that reported studies had at least some risk of bias due to use of self-report outcome measures. RESULTS: A total of 62 iCBT randomized controlled trials representing 17,210 participants are summarized in this study. Out of those 62 papers, only 17 (27%) of the trials reported race, and only 12 (19%) reported ethnicity. Reporting outside of the United States was very poor, with the United States accounting for 15 (88%) out of 17 of studies that reported race and 9 (75%) out of 12 for ethnicity. Out of 3,623 participants whose race was reported in the systematic review, the racial category reported the most was White (n=2716, 74.9%), followed by Asian (n=209, 5.8%) and Black (n=274, 7.6%). Furthermore, only 25 (54%) out of the 46 papers conducted outside of the United States reported other background demographics. CONCLUSIONS: It is important to note that the underreporting observed in this study does not necessarily indicate an underrepresentation in the actual study population. However, these findings highlight the poor reporting of race and ethnicity in iCBT trials for depression found in the literature. This lack of diversity reporting may have significant implications for the scalability of these interventions.

Longitudinal Coadaptation of Older Adults With Wearables and Voice-Activated Virtual Assistants: Scoping Review.

BACKGROUND: The integration of smart technologies, including wearables and voice-activated devices, is increasingly recognized for enhancing the independence and well-being of older adults. However, the long-term dynamics of their use and the coadaptation process with older adults remain poorly understood. This scoping review explores how interactions between older adults and smart technologies evolve over time to improve both user experience and technology utility. OBJECTIVE: This review synthesizes existing research on the coadaptation between older adults and smart technologies, focusing on longitudinal changes in use patterns, the effectiveness of technological adaptations, and the implications for future technology development and deployment to improve user experiences. METHODS: Following the Joanna Briggs Institute Reviewer's Manual and PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews) guidelines, this scoping review examined peer-reviewed papers from databases including Ovid MEDLINE, Ovid Embase, PEDro, Ovid PsycINFO, and EBSCO CINAHL from the year 2000 to August 28, 2023, and included forward and backward searches. The search was updated on March 1, 2024. Empirical studies were included if they involved (1) individuals aged 55 years or older living independently and (2) focused on interactions and adaptations between older adults and wearables and voice-activated virtual assistants in interventions for a minimum period of 8 weeks. Data extraction was informed by the selection and optimization with compensation framework and the sex- and gender-based analysis plus theoretical framework and used a directed content analysis approach. RESULTS: The search yielded 16,143 papers. Following title and abstract screening and a full-text review, 5 papers met the inclusion criteria. Study populations were mostly female participants and aged 73-83 years from the United States and engaged with voice-activated virtual assistants accessed through smart speakers and wearables. Users frequently used simple commands related to music and weather, integrating devices into daily routines. However, communication barriers often led to frustration due to devices' inability to recognize cues or provide personalized responses. The findings suggest that while older adults can integrate smart technologies into their lives, a lack of customization and user-friendly interfaces hinder long-term adoption and satisfaction. The studies highlight the need for technology to be further developed so they can better meet this demographic's evolving needs and call for research addressing small sample sizes and limited diversity. CONCLUSIONS: Our findings highlight a critical need for continued research into the dynamic and reciprocal relationship between smart technologies and older adults over time. Future studies should focus on more diverse populations and extend monitoring periods to provide deeper insights into the coadaptation process. Insights gained from this review are vital for informing the development of more intuitive, user-centric smart technology solutions to better support the aging population in maintaining independence and enhancing their quality of life. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.2196/51129.

Existing Barriers Faced by and Future Design Recommendations for Direct-to-Consumer Health Care Artificial Intelligence Apps: Scoping Review.

BACKGROUND: Direct-to-consumer (DTC) health care artificial intelligence (AI) apps hold the potential to bridge the spatial and temporal disparities in health care resources, but they also come with individual and societal risks due to AI errors. Furthermore, the manner in which consumers interact directly with health care AI is reshaping traditional physician-patient relationships. However, the academic community lacks a systematic comprehension of the research overview for such apps. OBJECTIVE: This paper systematically delineated and analyzed the characteristics of included studies, identified existing barriers and design recommendations for DTC health care AI apps mentioned in the literature and also provided a reference for future design and development. METHODS: This scoping review followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses Extension for Scoping Reviews guidelines and was conducted according to Arksey and O'Malley's 5-stage framework. Peer-reviewed papers on DTC health care AI apps published until March 27, 2023, in Web of Science, Scopus, the ACM Digital Library, IEEE Xplore, PubMed, and Google Scholar were included. The papers were analyzed using Braun and Clarke's reflective thematic analysis approach. RESULTS: Of the 2898 papers retrieved, 32 (1.1%) covering this emerging field were included. The included papers were recently published (2018-2023), and most (23/32, 72%) were from developed countries. The medical field was mostly general practice (8/32, 25%). In terms of users and functionalities, some apps were designed solely for single-consumer groups (24/32, 75%), offering disease diagnosis (14/32, 44%), health self-management (8/32, 25%), and health care information inquiry (4/32, 13%). Other apps connected to physicians (5/32, 16%), family members (1/32, 3%), nursing staff (1/32, 3%), and health care departments (2/32, 6%), generally to alert these groups to abnormal conditions of consumer users. In addition, 8 barriers and 6 design recommendations related to DTC health care AI apps were identified. Some more subtle obstacles that are particularly worth noting and corresponding design recommendations in consumer-facing health care AI systems, including enhancing human-centered explainability, establishing calibrated trust and addressing overtrust, demonstrating empathy in AI, improving the specialization of consumer-grade products, and expanding the diversity of the test population, were further discussed. CONCLUSIONS: The booming DTC health care AI apps present both risks and opportunities, which highlights the need to explore their current status. This paper systematically summarized and sorted the characteristics of the included studies, identified existing barriers faced by, and made future design recommendations for such apps. To the best of our knowledge, this is the first study to systematically summarize and categorize academic research on these apps. Future studies conducting the design and development of such systems could refer to the results of this study, which is crucial to improve the health care services provided by DTC health care AI apps.

Evaluation of the Current State of Chatbots for Digital Health: Scoping Review.

BACKGROUND: Chatbots have become ubiquitous in our daily lives, enabling natural language conversations with users through various modes of communication. Chatbots have the potential to play a significant role in promoting health and well-being. As the number of studies and available products related to chatbots continues to rise, there is a critical need to assess product features to enhance the design of chatbots that effectively promote health and behavioral change. OBJECTIVE: This scoping review aims to provide a comprehensive assessment of the current state of health-related chatbots, including the chatbots' characteristics and features, user backgrounds, communication models, relational building capacity, personalization, interaction, responses to suicidal thoughts, and users' in-app experiences during chatbot use. Through this analysis, we seek to identify gaps in the current research, guide future directions, and enhance the design of health-focused chatbots. METHODS: Following the scoping review methodology by Arksey and O'Malley and guided by the PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist, this study used a two-pronged approach to identify relevant chatbots: (1) searching the iOS and Android App Stores and (2) reviewing scientific literature through a search strategy designed by a librarian. Overall, 36 chatbots were selected based on predefined criteria from both sources. These chatbots were systematically evaluated using a comprehensive framework developed for this study, including chatbot characteristics, user backgrounds, building relational capacity, personalization, interaction models, responses to critical situations, and user experiences. Ten coauthors were responsible for downloading and testing the chatbots, coding their features, and evaluating their performance in simulated conversations. The testing of all chatbot apps was limited to their free-to-use features. RESULTS: This review provides an overview of the diversity of health-related chatbots, encompassing categories such as mental health support, physical activity promotion, and behavior change interventions. Chatbots use text, animations, speech, images, and emojis for communication. The findings highlight variations in conversational capabilities, including empathy, humor, and personalization. Notably, concerns regarding safety, particularly in addressing suicidal thoughts, were evident. Approximately 44% (16/36) of the chatbots effectively addressed suicidal thoughts. User experiences and behavioral outcomes demonstrated the potential of chatbots in health interventions, but evidence remains limited. CONCLUSIONS: This scoping review underscores the significance of chatbots in health-related applications and offers insights into their features, functionalities, and user experiences. This study contributes to advancing the understanding of chatbots' role in digital health interventions, thus paving the way for more effective and user-centric health promotion strategies. This study informs future research directions, emphasizing the need for rigorous randomized control trials, standardized evaluation metrics, and user-centered design to unlock the full potential of chatbots in enhancing health and well-being. Future research should focus on addressing limitations, exploring real-world user experiences, and implementing robust data security and privacy measures.

Rapid review: A review of methods and recommendations based on current evidence.

Rapid review (RR) could accelerate the traditional systematic review (SR) process by simplifying or omitting steps using various shortcuts. With the increasing popularity of RR, numerous shortcuts had emerged, but there was no consensus on how to choose the most appropriate ones. This study conducted a literature search in PubMed from inception to December 21, 2023, using terms such as "rapid review" "rapid assessment" "rapid systematic review" and "rapid evaluation". We also scanned the reference lists and performed citation tracking of included impact studies to obtain more included studies. We conducted a narrative synthesis of all RR approaches, shortcuts and studies assessing their effectiveness at each stage of RRs. Based on the current evidence, we provided recommendations on utilizing certain shortcuts in RRs. Ultimately, we identified 185 studies focusing on summarizing RR approaches and shortcuts, or evaluating their impact. There was relatively sufficient evidence to support the use of the following shortcuts in RRs: limiting studies to those published in English-language; conducting abbreviated database searches (e.g., only searching PubMed/MEDLINE, Embase, and CENTRAL); omitting retrieval of grey literature; restricting the search timeframe to the recent 20 years for medical intervention and the recent 15 years for reviewing diagnostic test accuracy; conducting a single screening by an experienced screener. To some extent, the above shortcuts were also applicable to SRs. This study provided a reference for future RR researchers in selecting shortcuts, and it also presented a potential research topic for methodologists.

Only half of the authors of overviews of exercise-related interventions use some strategy to manage overlapping primary studies-a metaresearch study.

OBJECTIVES: The conduct of systematic reviews (SRs) and overviews share several similarities. However, because the unit of analysis for overviews is the SRs, there are some unique challenges. One of the most critical issues to manage when conducting an overview is the overlap of data across the primary studies included in the SRs. This metaresearch study aimed to describe the frequency of strategies to manage the overlap in overviews of exercise-related interventions. STUDY DESIGN AND SETTING: A systematic search in MEDLINE (Ovid), Embase (Ovid), Cochrane Library, Epistemonikos, and other sources was conducted from inception to June 2022. We included overviews of SRs that considered primary studies and evaluated the effectiveness of exercise-related interventions for any health condition. The overviews were screened by two authors independently, and the extraction was performed by one author and checked by a second. We found 353 overviews published between 2005 and 2022 that met the inclusion criteria. RESULTS: One hundred and sixty-four overviews (46%) used at least one strategy to visualize, quantify, or resolve overlap, with a matrix (32/164; 20%), absolute frequency (34/164; 21%), and authors' algorithms (24/164; 15%) being the most used methods, respectively. From 2016 onwards, there has been a trend toward increasing the use of some strategies to manage overlap. Of the 108 overviews that used some strategy to resolve the overlap, ie, avoiding double or multiple counting of primary study data, 79 (73%) succeeded. In overviews where no strategies to manage overlap were reported (n = 189/353; 54%), 16 overview authors (8%) recognized this as a study limitation. CONCLUSION: Although there is a trend toward increasing its use, only half of the authors of the overviews of exercise-related interventions used a strategy to visualize, quantify, or resolve overlap in the primary studies' data. In the future, authors should report such strategies to communicate more valid results.

Predictive Data Analytics in Telecare and Telehealth: Systematic Scoping Review.

BACKGROUND: Telecare and telehealth are important care-at-home services used to support individuals to live more independently at home. Historically, these technologies have reactively responded to issues. However, there has been a recent drive to make better use of the data from these services to facilitate more proactive and predictive care. OBJECTIVE: This review seeks to explore the ways in which predictive data analytics techniques have been applied in telecare and telehealth in at-home settings. METHODS: The PRISMA-ScR (Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews) checklist was adhered to alongside Arksey and O'Malley's methodological framework. English language papers published in MEDLINE, Embase, and Social Science Premium Collection between 2012 and 2022 were considered and results were screened against inclusion or exclusion criteria. RESULTS: In total, 86 papers were included in this review. The types of analytics featuring in this review can be categorized as anomaly detection (n=21), diagnosis (n=32), prediction (n=22), and activity recognition (n=11). The most common health conditions represented were Parkinson disease (n=12) and cardiovascular conditions (n=11). The main findings include: a lack of use of routinely collected data; a dominance of diagnostic tools; and barriers and opportunities that exist, such as including patient-reported outcomes, for future predictive analytics in telecare and telehealth. CONCLUSIONS: All papers in this review were small-scale pilots and, as such, future research should seek to apply these predictive techniques into larger trials. Additionally, further integration of routinely collected care data and patient-reported outcomes into predictive models in telecare and telehealth offer significant opportunities to improve the analytics being performed and should be explored further. Data sets used must be of suitable size and diversity, ensuring that models are generalizable to a wider population and can be appropriately trained, validated, and tested.

Predicting Hypoxia Using Machine Learning: Systematic Review.

Background: Hypoxia is an important risk factor and indicator for the declining health of inpatients. Predicting future hypoxic events using machine learning is a prospective area of study to facilitate time-critical interventions to counter patient health deterioration. Objective: This systematic review aims to summarize and compare previous efforts to predict hypoxic events in the hospital setting using machine learning with respect to their methodology, predictive performance, and assessed population. Methods: A systematic literature search was performed using Web of Science, Ovid with Embase and MEDLINE, and Google Scholar. Studies that investigated hypoxia or hypoxemia of hospitalized patients using machine learning models were considered. Risk of bias was assessed using the Prediction Model Risk of Bias Assessment Tool. Results: After screening, a total of 12 papers were eligible for analysis, from which 32 models were extracted. The included studies showed a variety of population, methodology, and outcome definition. Comparability was further limited due to unclear or high risk of bias for most studies (10/12, 83%). The overall predictive performance ranged from moderate to high. Based on classification metrics, deep learning models performed similar to or outperformed conventional machine learning models within the same studies. Models using only prior peripheral oxygen saturation as a clinical variable showed better performance than models based on multiple variables, with most of these studies (2/3, 67%) using a long short-term memory algorithm. Conclusions: Machine learning models provide the potential to accurately predict the occurrence of hypoxic events based on retrospective data. The heterogeneity of the studies and limited generalizability of their results highlight the need for further validation studies to assess their predictive performance.

Machine Learning-Based Approach for Identifying Research Gaps: COVID-19 as a Case Study.

BACKGROUND: Research gaps refer to unanswered questions in the existing body of knowledge, either due to a lack of studies or inconclusive results. Research gaps are essential starting points and motivation in scientific research. Traditional methods for identifying research gaps, such as literature reviews and expert opinions, can be time consuming, labor intensive, and prone to bias. They may also fall short when dealing with rapidly evolving or time-sensitive subjects. Thus, innovative scalable approaches are needed to identify research gaps, systematically assess the literature, and prioritize areas for further study in the topic of interest. OBJECTIVE: In this paper, we propose a machine learning-based approach for identifying research gaps through the analysis of scientific literature. We used the COVID-19 pandemic as a case study. METHODS: We conducted an analysis to identify research gaps in COVID-19 literature using the COVID-19 Open Research (CORD-19) data set, which comprises 1,121,433 papers related to the COVID-19 pandemic. Our approach is based on the BERTopic topic modeling technique, which leverages transformers and class-based term frequency-inverse document frequency to create dense clusters allowing for easily interpretable topics. Our BERTopic-based approach involves 3 stages: embedding documents, clustering documents (dimension reduction and clustering), and representing topics (generating candidates and maximizing candidate relevance). RESULTS: After applying the study selection criteria, we included 33,206 abstracts in the analysis of this study. The final list of research gaps identified 21 different areas, which were grouped into 6 principal topics. These topics were: "virus of COVID-19," "risk factors of COVID-19," "prevention of COVID-19," "treatment of COVID-19," "health care delivery during COVID-19," "and impact of COVID-19." The most prominent topic, observed in over half of the analyzed studies, was "the impact of COVID-19." CONCLUSIONS: The proposed machine learning-based approach has the potential to identify research gaps in scientific literature. This study is not intended to replace individual literature research within a selected topic. Instead, it can serve as a guide to formulate precise literature search queries in specific areas associated with research questions that previous publications have earmarked for future exploration. Future research should leverage an up-to-date list of studies that are retrieved from the most common databases in the target area. When feasible, full texts or, at minimum, discussion sections should be analyzed rather than limiting their analysis to abstracts. Furthermore, future studies could evaluate more efficient modeling algorithms, especially those combining topic modeling with statistical uncertainty quantification, such as conformal prediction.

Effectiveness and User Experience of Virtual Reality for Social Anxiety Disorder: Systematic Review.

BACKGROUND: Social anxiety disorder (SAD) is a debilitating psychiatric disorder that affects occupational and social functioning. Virtual reality (VR) therapies can provide effective treatment for people with SAD. However, with rapid innovations in immersive VR technology, more contemporary research is required to examine the effectiveness and concomitant user experience outcomes (ie, safety, usability, acceptability, and attrition) of emerging VR interventions for SAD. OBJECTIVE: The aim of this systematic review was to examine the effectiveness and user experience of contemporary VR interventions among people with SAD. METHODS: The Cochrane Library, Emcare, PsycINFO, PubMed, ScienceDirect, Scopus, and Web of Science databases were searched between January 1, 2012, and April 26, 2022. Deduplicated search results were screened based on title and abstract information. Full-text examination was conducted on 71 articles. Studies of all designs and comparator groups were included if they appraised the effectiveness and user experience outcomes of any immersive VR intervention among people with SAD. A standardized coding sheet was used to extract data on key participant, intervention, comparator, outcome, and study design items. RESULTS: The findings were tabulated and discussed using a narrative synthesis. A total of 18 studies met the inclusion criteria. CONCLUSIONS: The findings showed that VR exposure therapy-based interventions can generally provide effective, safe, usable, and acceptable treatments for adults with SAD. The average attrition rate from VR treatment was low (11.36%) despite some reported user experience difficulties, including potential simulator sickness, exposure-based emotional distress, and problems with managing treatment delivered in a synchronous group setting. This review also revealed several research gaps, including a lack of VR treatment studies on children and adolescents with SAD as well as a paucity of standardized assessments of VR user experience interactions. More studies are required to address these issues. TRIAL REGISTRATION: PROSPERO CRD42022353891; https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=353891.

The Potential of Exercise on Lifestyle and Skin Function: Narrative Review.

BACKGROUND: The skin is an important organ of the human body and has moisturizing and barrier functions. Factors such as sunlight and lifestyle significantly affect these skin functions, with sunlight being extremely damaging. The effects of lifestyle habits such as smoking, diet, and sleep have been studied extensively. It has been found that smoking increases the risk of wrinkles, while excessive fat and sugar intake leads to skin aging. Lack of sleep and stress are also dangerous for the skin's barrier function. In recent years, the impact of exercise habits on skin function has been a focus of study. Regular exercise is associated with increased blood flow to the skin, elevated skin temperature, and improved skin moisture. Furthermore, it has been shown to improve skin structure and rejuvenate its appearance, possibly through promoting mitochondrial biosynthesis and affecting hormone secretion. Further research is needed to understand the effects of different amounts and content of exercise on the skin. OBJECTIVE: This study aims to briefly summarize the relationship between lifestyle and skin function and the mechanisms that have been elucidated so far and introduce the expected effects of exercise on skin function. METHODS: We conducted a review of the literature using PubMed and Google Scholar repositories for relevant literature published between 2000 and 2022 with the following keywords: exercise, skin, and life habits. RESULTS: Exercise augments the total spectrum power density of cutaneous blood perfusion by a factor of approximately 8, and vasodilation demonstrates an enhancement of approximately 1.5-fold. Regular exercise can also mitigate age-related skin changes by promoting mitochondrial biosynthesis. However, not all exercise impacts are positive; for instance, swimming in chlorinated pools may harm the skin barrier function. Hence, the exercise environment should be considered for its potential effects on the skin. CONCLUSIONS: This review demonstrates that exercise can potentially enhance skin function retention.

Evaluation of Patient-Facing Mobile Apps to Support Physiotherapy Care: Systematic Review.

BACKGROUND: Mobile health interventions delivered through mobile apps are increasingly used in physiotherapy care. This may be because of the potential of apps to facilitate changes in behavior, which is central to the aims of care delivered by physiotherapists. A benefit of using apps is their ability to incorporate behavior change techniques (BCTs) that can optimize the effectiveness of physiotherapeutic interventions. Research continues to suggest that despite their importance, behavior change strategies are often missing in patient management. Evaluating mobile apps that physiotherapists can use to drive behavior change may inform clinical practice and potentially improve patient outcomes. Examining the quality of apps and exploring their key features that can support behavior change and physiotherapy care are important aspects of such an evaluation. OBJECTIVE: The primary aim of this study was to describe the range of mobile apps in app stores that are intended for use by patients to support physiotherapy care. The secondary aims were to assess app quality, BCTs, and their behavior change potential. METHODS: A systematic review of mobile apps in app stores was undertaken. The Apple App Store and Google Play were searched using a 2-step search strategy, using terms relevant to the physiotherapy discipline. Strict inclusion and exclusion criteria were applied: apps had to be intended for use by patients and be self-contained (or stand-alone) without the requirement to be used in conjunction with a partner wearable device or another plugin. Included apps were coded for BCTs using the Behavior Change Technique Taxonomy version 1. App quality was assessed using the Mobile App Rating Scale, and the App Behavior Change Scale was used to assess the app's potential to change behavior. RESULTS: In total, 1240 apps were screened, and 35 were included. Of these 35 apps, 22 (63%) were available on both the Apple App Store and Google Play platforms. In total, 24 (69%) were general in their focus (eg, not condition-specific), with the remaining 11 (31%) being more specific (eg, knee rehabilitation and pelvic floor training). The mean app quality score (Mobile App Rating Scale) was 3.7 (SD 0.4) of 5 (range 2.8-4.5). The mean number of BCTs identified per app was 8.5 (SD 3.6). BCTs most frequently included in the apps were instruction on how to perform a behavior (n=32), action planning (n=30), and self-monitoring of behavior (n=28). The mean behavior change potential score (App Behavior Change Scale) was 8.5 (SD 3.1) of 21 (range 3-15). CONCLUSIONS: Mobile apps available to support patient care received from a physiotherapist are of variable quality. Although they contain some BCTs, the potential for behavior change varied widely across apps. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.2196/29047.

Strategies to Mitigate Age-Related Bias in Machine Learning: Scoping Review.

BACKGROUND: Research suggests that digital ageism, that is, age-related bias, is present in the development and deployment of machine learning (ML) models. Despite the recognition of the importance of this problem, there is a lack of research that specifically examines the strategies used to mitigate age-related bias in ML models and the effectiveness of these strategies. OBJECTIVE: To address this gap, we conducted a scoping review of mitigation strategies to reduce age-related bias in ML. METHODS: We followed a scoping review methodology framework developed by Arksey and O'Malley. The search was developed in conjunction with an information specialist and conducted in 6 electronic databases (IEEE Xplore, Scopus, Web of Science, CINAHL, EMBASE, and the ACM digital library), as well as 2 additional gray literature databases (OpenGrey and Grey Literature Report). RESULTS: We identified 8 publications that attempted to mitigate age-related bias in ML approaches. Age-related bias was introduced primarily due to a lack of representation of older adults in the data. Efforts to mitigate bias were categorized into one of three approaches: (1) creating a more balanced data set, (2) augmenting and supplementing their data, and (3) modifying the algorithm directly to achieve a more balanced result. CONCLUSIONS: Identifying and mitigating related biases in ML models is critical to fostering fairness, equity, inclusion, and social benefits. Our analysis underscores the ongoing need for rigorous research and the development of effective mitigation approaches to address digital ageism, ensuring that ML systems are used in a way that upholds the interests of all individuals. TRIAL REGISTRATION: Open Science Framework AMG5P; https://osf.io/amg5p.

Factors Underlying Vaccine Hesitancy and Their Mitigations in Saudi Arabia: Protocol for a Systematic Review.

BACKGROUND: Vaccine hesitancy is a growing concern in Saudi Arabia, impacting even well-educated parents. The decision-making process involves various factors such as accessibility, trustworthy information, and the influence of social networks, reflecting a complex interplay of emotional, cultural, social, spiritual, and political dimensions. OBJECTIVE: This review seeks to evaluate the prevalence and trends of vaccine hesitancy, identify contributing factors, and explore potential solutions to enhance immunization rates. This review aligns with global concerns, as the World Health Organization has identified vaccine hesitancy as a top global health threat. METHODS: Our systematic review will follow the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines and PICOS (Population, Intervention, Comparison, Outcomes, and Study) criteria for comprehensive assessment. We will conduct a thorough search across various databases, encompassing a wide range of vaccines, and pay special attention to vaccination campaigns and refusals. Inclusion criteria involve descriptive, observational, and analytical studies focusing on factors influencing vaccine acceptance or hesitancy. The study will use the Crowe Critical Appraisal Tool for quality assessment and perform a narrative synthesis to summarize findings thematically. RESULTS: This systematic review is expected to unveil the prevalence and trends of vaccine hesitancy in diverse populations in Saudi Arabia, shedding light on cultural, religious, and social factors contributing to hesitancy. It aims to assess the effectiveness of implemented strategies, enable regional and global comparisons, and provide implications for tailored vaccination policies. Additionally, the review may pinpoint research gaps, guiding future investigations to address and mitigate vaccine hesitancy effectively. CONCLUSIONS: The findings are expected to have direct policy implications and guide interventions to strengthen vaccination programs and improve public health outcomes. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): PRR1-10.2196/54680.