The relationship between patient experience and real-world digital health access in primary care: A population-based cross-sectional study.

Implementing digital health technologies in primary care is anticipated to improve patient experience. We examined the relationships between patient experience and digital health access in primary care settings in Ontario, Canada. We conducted a retrospective cross-sectional study using patient responses to the Health Care Experience Survey linked to health and administrative data between April 2019-February 2020. We measured patient experience by summarizing HCES questions. We used multivariable logistic regression stratified by the number of primary care visits to investigate associations between patient experience with digital health access and moderating variables. Our cohort included 2,692 Ontario adults, of which 63.0% accessed telehealth, 2.6% viewed medical records online, and 3.6% booked appointments online. Although patients reported overwhelmingly positive experiences, we found no consistent relationship with digital health access. Online appointment booking access was associated with lower odds of poor experience for patients with three or more primary care visits in the past 12 months (adjusted odds ratio 0.16, 95% CI 0.02-0.56). Younger age, tight financial circumstances, English as a second language, and knowing their primary care provider for fewer years had greater odds of poor patient experience. In 2019/2020, we found limited uptake of digital health in primary care and no clear association between real-world digital health adoption and patient experience in Ontario. Our findings provide an essential context for ensuing rapid shifts in digital health adoption during the COVID-19 pandemic, serving as a baseline to reexamine subsequent improvements in patient experience.

Seeing the random forest through the decision trees. Supporting learning health systems from histopathology with machine learning models: Challenges and opportunities.

This paper discusses some overlooked challenges faced when working with machine learning models for histopathology and presents a novel opportunity to support "Learning Health Systems" with them. Initially, the authors elaborate on these challenges after separating them according to their mitigation strategies: those that need innovative approaches, time, or future technological capabilities and those that require a conceptual reappraisal from a critical perspective. Then, a novel opportunity to support "Learning Health Systems" by integrating hidden information extracted by ML models from digitalized histopathology slides with other healthcare big data is presented.

A scoping review reveals candidate quality indicators of knowledge translation and implementation science practice tools.

OBJECTIVES: To identify candidate quality indicators from existing tools that provide guidance on how to practice knowledge translation and implemenation science (KT practice tools) across KT domains (dissemination, implementation, sustainability, and scalability). STUDY DESIGN AND SETTING: We conducted a scoping review using the Joanna Briggs Institute Manual for Evidence Synthesis. We systematically searched multiple electronic databases and the gray literature. Documents were independently screened, selected, and extracted by pairs of reviewers. Data about the included articles, KT practice tools, and candidate quality indicators were analyzed, categorized, and summarized descriptively. RESULTS: Of 43,060 titles and abstracts that were screened from electronic databases and gray literature, 850 potentially relevant full-text articles were identified, and 253 articles were included in the scoping review. Of these, we identified 232 unique KT practice tools from which 27 unique candidate quality indicators were generated. The identified candidate quality indicators were categorized according to the development (n = 17), evaluation (n = 5) and adaptation (n = 3) of the tools, and engagement of knowledge users (n = 2). No tools were identified that appraised the quality of KT practice tools. CONCLUSIONS: The development of a quality appraisal instrument of KT practice tools is needed. The results will be further refined and finalized in order to develop a quality appraisal instrument for KT practice tools.

Performance of externally validated machine learning models based on histopathology images for the diagnosis, classification, prognosis, or treatment outcome prediction in female breast cancer: A systematic review.

Numerous machine learning (ML) models have been developed for breast cancer using various types of data. Successful external validation (EV) of ML models is important evidence of their generalizability. The aim of this systematic review was to assess the performance of externally validated ML models based on histopathology images for diagnosis, classification, prognosis, or treatment outcome prediction in female breast cancer. A systematic search of MEDLINE, EMBASE, CINAHL, IEEE, MICCAI, and SPIE conferences was performed for studies published between January 2010 and February 2022. The Prediction Model Risk of Bias Assessment Tool (PROBAST) was employed, and the results were narratively described. Of the 2011 non-duplicated citations, 8 journal articles and 2 conference proceedings met inclusion criteria. Three studies externally validated ML models for diagnosis, 4 for classification, 2 for prognosis, and 1 for both classification and prognosis. Most studies used Convolutional Neural Networks and one used logistic regression algorithms. For diagnostic/classification models, the most common performance metrics reported in the EV were accuracy and area under the curve, which were greater than 87% and 90%, respectively, using pathologists' annotations/diagnoses as ground truth. The hazard ratios in the EV of prognostic ML models were between 1.7 (95% CI, 1.2-2.6) and 1.8 (95% CI, 1.3-2.7) to predict distant disease-free survival; 1.91 (95% CI, 1.11-3.29) for recurrence, and between 0.09 (95% CI, 0.01-0.70) and 0.65 (95% CI, 0.43-0.98) for overall survival, using clinical data as ground truth. Despite EV being an important step before the clinical application of a ML model, it hasn't been performed routinely. The large variability in the training/validation datasets, methods, performance metrics, and reported information limited the comparison of the models and the analysis of their results. Increasing the availability of validation datasets and implementing standardized methods and reporting protocols may facilitate future analyses.

Public Health Surveillance of Behavioral Cancer Risk Factors During the COVID-19 Pandemic: Sentiment and Emotion Analysis of Twitter Data.

BACKGROUND: The COVID-19 pandemic and its associated public health mitigation strategies have dramatically changed patterns of daily life activities worldwide, resulting in unintentional consequences on behavioral risk factors, including smoking, alcohol consumption, poor nutrition, and physical inactivity. The infodemic of social media data may provide novel opportunities for evaluating changes related to behavioral risk factors during the pandemic. OBJECTIVE: We explored the feasibility of conducting a sentiment and emotion analysis using Twitter data to evaluate behavioral cancer risk factors (physical inactivity, poor nutrition, alcohol consumption, and smoking) over time during the first year of the COVID-19 pandemic. METHODS: Tweets during 2020 relating to the COVID-19 pandemic and the 4 cancer risk factors were extracted from the George Washington University Libraries Dataverse. Tweets were defined and filtered using keywords to create 4 data sets. We trained and tested a machine learning classifier using a prelabeled Twitter data set. This was applied to determine the sentiment (positive, negative, or neutral) of each tweet. A natural language processing package was used to identify the emotions (anger, anticipation, disgust, fear, joy, sadness, surprise, and trust) based on the words contained in the tweets. Sentiments and emotions for each of the risk factors were evaluated over time and analyzed to identify keywords that emerged. RESULTS: The sentiment analysis revealed that 56.69% (51,479/90,813) of the tweets about physical activity were positive, 16.4% (14,893/90,813) were negative, and 26.91% (24,441/90,813) were neutral. Similar patterns were observed for nutrition, where 55.44% (27,939/50,396), 15.78% (7950/50,396), and 28.79% (14,507/50,396) of the tweets were positive, negative, and neutral, respectively. For alcohol, the proportions of positive, negative, and neutral tweets were 46.85% (34,897/74,484), 22.9% (17,056/74,484), and 30.25% (22,531/74,484), respectively, and for smoking, they were 41.2% (11,628/28,220), 24.23% (6839/28,220), and 34.56% (9753/28,220), respectively. The sentiments were relatively stable over time. The emotion analysis suggests that the most common emotion expressed across physical activity and nutrition tweets was trust (69,495/320,741, 21.67% and 42,324/176,564, 23.97%, respectively); for alcohol, it was joy (49,147/273,128, 17.99%); and for smoking, it was fear (23,066/110,256, 20.92%). The emotions expressed remained relatively constant over the observed period. An analysis of the most frequent words tweeted revealed further insights into common themes expressed in relation to some of the risk factors and possible sources of bias. CONCLUSIONS: This analysis provided insight into behavioral cancer risk factors as expressed on Twitter during the first year of the COVID-19 pandemic. It was feasible to extract tweets relating to all 4 risk factors, and most tweets had a positive sentiment with varied emotions across the different data sets. Although these results can play a role in promoting public health, a deeper dive via qualitative analysis can be conducted to provide a contextual examination of each tweet.

Enhanced recovery support for people with eating disorders during the COVID-19 pandemic: quality improvement using a web-based, stepped-care programme in Canada.

BACKGROUND: Throughout the COVID-19 pandemic, the number of individuals struggling with eating disorders (EDs) increased substantially. Body Brave (a not-for-profit) created and implemented a web-based stepped-care Recovery Support Programme (RSP) to improve access to community-based ED services. This quality improvement study describes the RSP and assesses its ability to deliver timely access to treatment and platform engagement. METHODS: We conducted a retrospective cohort study comparing access to, and use of Body Brave services 6 months before and 12 months after implementation of the RSP platform (using 6-month increments for two postimplementation periods). Primary programme quality measures included registration requests, number of participants onboarded and time to access services; secondary measures included use of RSP action plans, attendance for recovery sessions and workshops, number of participants accessing treatment and text-based patient experience data. RESULTS: A substantial increase in registration requests was observed during the first postimplementation period compared with the preimplementation period (176.5 vs 85.5; p=0.028). When compared with the preimplementation period, the second postimplementation observed a significantly larger percentage of successfully onboarded participants (76.6 vs 37.9; p<0.01) and a reduction in the number of days to access services (2 days vs 31 days; p<0.01). Although participant feedback rates were low, many users found the RSP helpful, easy to access, user-friendly and were satisfied overall. Users provided suggestions for improvement (eg, a platform instructional video, offer multiple times of day for live sessions and drop-in hours). CONCLUSIONS: Although clinical benefit needs to be assessed, our findings demonstrate that the RSP enabled participants to quickly onboard and access initial services and have informed subsequent improvements. Understanding initial programme effects and usage will help assess the feasibility of adapting and expanding the RSP across Canada to address the urgent need for low-barrier, patient-centred ED care.

Deep learning to refine the identification of high-quality clinical research articles from the biomedical literature: Performance evaluation.

BACKGROUND: Identifying practice-ready evidence-based journal articles in medicine is a challenge due to the sheer volume of biomedical research publications. Newer approaches to support evidence discovery apply deep learning techniques to improve the efficiency and accuracy of classifying sound evidence. OBJECTIVE: To determine how well deep learning models using variants of Bidirectional Encoder Representations from Transformers (BERT) identify high-quality evidence with high clinical relevance from the biomedical literature for consideration in clinical practice. METHODS: We fine-tuned variations of BERT models (BERTBASE, BioBERT, BlueBERT, and PubMedBERT) and compared their performance in classifying articles based on methodological quality criteria. The dataset used for fine-tuning models included titles and abstracts of >160,000 PubMed records from 2012 to 2020 that were of interest to human health which had been manually labeled based on meeting established critical appraisal criteria for methodological rigor. The data was randomly divided into 80:10:10 sets for training, validating, and testing. In addition to using the full unbalanced set, the training data was randomly undersampled into four balanced datasets to assess performance and select the best performing model. For each of the four sets, one model that maintained sensitivity (recall) at ≥99% was selected and were ensembled. The best performing model was evaluated in a prospective, blinded test and applied to an established reference standard, the Clinical Hedges dataset. RESULTS: In training, three of the four selected best performing models were trained using BioBERTBASE. The ensembled model did not boost performance compared with the best individual model. Hence a solo BioBERT-based model (named DL-PLUS) was selected for further testing as it was computationally more efficient. The model had high recall (>99%) and 60% to 77% specificity in a prospective evaluation conducted with blinded research associates and saved >60% of the work required to identify high quality articles. CONCLUSIONS: Deep learning using pretrained language models and a large dataset of classified articles produced models with improved specificity while maintaining >99% recall. The resulting DL-PLUS model identifies high-quality, clinically relevant articles from PubMed at the time of publication. The model improves the efficiency of a literature surveillance program, which allows for faster dissemination of appraised research.

Identifying candidate quality indicators of tools that support the practice of knowledge translation: a scoping review protocol.

OBJECTIVE: The objective of this scoping review is to identify and characterize relevant knowledge translation methods tools (those that provide guidance for optimized knowledge translation practice) to uncover candidate quality indicators to inform a future quality assessment tool for knowledge translation strategies. INTRODUCTION: Knowledge translation strategies (defined as including knowledge translation interventions, tools, and products) target various knowledge users, including patients, clinicians, researchers, and policy-makers. The development and use of strategies that support knowledge translation practice have been rapidly increasing, making it difficult for knowledge users to decide which to use. There is limited evidence-based guidance or measures to help assess the overall quality of knowledge translation strategies. INCLUSION CRITERIA: Empirical and non-empirical documents will be considered if they explicitly describe a knowledge translation methods tool and its development, evaluation or validation, methodological strengths or limitations, and/or use over time. The review will consider a knowledge translation methods tool if it falls within at least one knowledge translation domain (ie, implementation, dissemination, sustainability, scalability, integrated knowledge translation) in the health field. METHODS: We will conduct a systematic search of relevant electronic databases and gray literature. The search strategy will be developed iteratively by an experienced medical information specialist and peer-reviewed with the PRESS checklist. The search will be limited to English-only documents published from 2005 onward. Documents will be independently screened, selected, and extracted by 2 researchers. Data will be analyzed and summarized descriptively, including the characteristics of the included documents, knowledge translation methods tools, and candidate quality indicators. SCOPING REVIEW REGISTRATION: Open Science Framework ( https://osf.io/chxvq ).

Considerations for Conducting Bring Your Own "Device" (BYOD) Clinical Studies.

Background: Digital health technologies are attracting attention as novel tools for data collection in clinical research. They present alternative methods compared to in-clinic data collection, which often yields snapshots of the participants' physiology, behavior, and function that may be prone to biases and artifacts, e.g., white coat hypertension, and not representative of the data in free-living conditions. Modern digital health technologies equipped with multi-modal sensors combine different data streams to derive comprehensive endpoints that are important to study participants and are clinically meaningful. Used for data collection in clinical trials, they can be deployed as provisioned products where technology is given at study start or in a bring your own "device" (BYOD) manner where participants use their technologies to generate study data. Summary: The BYOD option has the potential to be more user-friendly, allowing participants to use technologies that they are familiar with, ensuring better participant compliance, and potentially reducing the bias that comes with introducing new technologies. However, this approach presents different technical, operational, regulatory, and ethical challenges to study teams. For example, BYOD data can be more heterogeneous, and recruiting historically underrepresented populations with limited access to technology and the internet can be challenging. Despite the rapid increase in digital health technologies for clinical and healthcare research, BYOD use in clinical trials is limited, and regulatory guidance is still evolving. Key Messages: We offer considerations for academic researchers, drug developers, and patient advocacy organizations on the design and deployment of BYOD models in clinical research. These considerations address: (1) early identification and engagement with internal and external stakeholders; (2) study design including informed consent and recruitment strategies; (3) outcome, endpoint, and technology selection; (4) data management including compliance and data monitoring; (5) statistical considerations to meet regulatory requirements. We believe that this article acts as a primer, providing insights into study design and operational requirements to ensure the successful implementation of BYOD clinical studies.

Patient and physician perspectives on the use and outcome measures of mHealth apps: Exploratory survey and focus group study.

Objective: Factors that physicians and patients consider when making decisions about using or recommending health apps are not well understood. We explored these factors to better assess how to support such decision making. Methods: We conducted an exploratory cross-sectional study in Ontario using qualitative focus groups and quantitative surveys. 133 physicians and 94 community dwelling adults completed online surveys and we held two focus groups of nine community dwelling participants who had cardiovascular risk factors and an interest in using mHealth apps. Quantitative survey data was analyzed descriptively. Focus groups were audio-recorded and transcribed verbatim prior to inductive thematic content analysis. We integrated the results from the surveys and focus groups to understand factors that influence physicians' and patients' selection and use of such apps. Results: Physicians recommend apps to patients but the level of evidence they prefer to use to guide selection did not align with what they were currently using. Patients trusted recommendations and reviews from medical organizations and healthcare professionals when selecting apps and were motivated to continue using apps when they supported goal setting and tracking, data sharing, decision making, and empowerment. Conclusions: The findings highlight the significance of evaluating mHealth apps based on metrics that patients and physicians value beyond usage and clinical outcome data. Patients engage with apps that support them in confidently managing their health. Increased training and awareness of apps and creating a more rigorous evidence base showing the value of apps to supporting health goals will support greater adoption and acceptance of mHealth apps.

Challenges in eating disorder diagnosis and management among family physicians and trainees: a qualitative study.

BACKGROUND: Family physicians are one of the first points of contact for individuals with eating disorders (EDs) seeking care and treatment, but training in this area is suboptimal and insufficient. Specialized ED treatment programs often have long wait lists, and family physicians are responsible for patients care in the interim. The aim of this study was to identify the learning needs and challenges faced by Canadian family physicians and trainees when caring for patients with EDs. METHODS: We recruited six family medicine residents and five family physicians practicing in an academic unit in the Department of Family Medicine of a medical school in urban southwestern Ontario, Canada. We used purposive sampling, focusing on residents and faculty physicians from the department and conducted one focus group for the residents and another for the faculty physicians, exploring their clinical knowledge and challenges when managing ED patients. The focus groups were audio-recorded and transcribed verbatim prior to thematic coding. RESULTS: Physicians and residents faced challenges in discussing, screening, and managing patients with EDs. Three themes that emerged from the qualitative data highlighted training needs related to: (a) improving communication skills when treating a patient with an ED, (b) more effective screening and diagnosis in primary care practice, and (c) optimizing management strategies for patients with an ED, especially patients who are waiting for more intensive treatment. A fourth theme that emerged was the distress experienced by family physicians as they try best to manage and access care for their patients with EDs. CONCLUSION: Addressing the learning needs identified in this study through continuing education offerings could aid family physicians in confidently providing effective, evidence-based care to patients with EDs. Improvement in training and education could also alleviate some of the distress faced by family physicians in managing patients with EDs. Ultimately, system changes to allow more efficient and appropriate levels of care for patients with EDs, removing the burden from family medicine, are critical as EDs are on the rise. A person with an eating disorder will normally seek care from their family physician first. These conditions can dramatically reduce the quality of a person's life and health. Family physicians therefore need to know how best to help these patients or refer them to a more intensive level of care, which often has long wait lists. We asked a group of family physicians and a group of family medicine trainees about their experiences with patients with eating disorders and about the information they wished they had to help these patients. The results show that they need more information on how to talk to a patient about eating disorders without judgement, how to diagnose a patient with an eating disorder, and then what treatment and management is needed while they wait for more intensive treatment for sicker patients. The physicians and trainees both talked about the stress and worry that they faced when treating patients with eating disorders. Besides their lack of training about these conditions, family physicians also described difficulties when trying to access timely specialized services for their patients. Physicians can experience moral distress when they know that their patients need higher level care, but there are systemic barriers to specialized programs that block their patients from getting the care they need when they need it.

Infectious diseases, comorbidities and outcomes in hospitalized people who inject drugs (PWID) infections in persons who inject drugs.

Injection drug use poses a public health challenge. Clinical experience indicates that people who inject drugs (PWID) are hospitalized frequently for infectious diseases, but little is known about outcomes when admitted. Charts were identified from local hospitals between 2013-2018 using consultation lists and hospital record searches. Included individuals injected drugs in the past six months and presented with infection. Charts were accessed using the hospital information system, undergoing primary and secondary reviews using Research Electronic Data Capture (REDCap). The Wilcoxon rank-sum test was used for comparisons between outcome categories. Categorical data were summarized as count and frequency, and compared using Fisher's exact test. Of 240 individuals, 33% were admitted to the intensive care unit, 36% underwent surgery, 12% left against medical advice (AMA), and 9% died. Infectious diagnoses included bacteremia (31%), abscess (29%), endocarditis (29%), cellulitis (20%), sepsis (10%), osteomyelitis (9%), septic arthritis (8%), pneumonia (7%), discitis (2%), meningitis/encephalitis (2%), or other (7%). Sixty-six percent had stable housing and 60% had a family physician. Fifty-four percent of patient-initiated discharges were seen in the emergency department within 30 days and 29% were readmitted. PWID are at risk for infections. Understanding their healthcare trajectory is essential to improve their care.

Tracking discussions of complementary, alternative, and integrative medicine in the context of the COVID-19 pandemic: a month-by-month sentiment analysis of Twitter data.

BACKGROUND: Coronavirus disease 2019 (COVID-19) is a novel infectious disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Despite the paucity of evidence, various complementary, alternative and integrative medicines (CAIMs) have been being touted as both preventative and curative. We conducted sentiment and emotion analysis with the intent of understanding CAIM content related to COVID-19 being generated on Twitter across 9 months. METHODS: Tweets relating to CAIM and COVID-19 were extracted from the George Washington University Libraries Dataverse Coronavirus tweets dataset from March 03 to November 30, 2020. We trained and tested a machine learning classifier using a large, pre-labelled Twitter dataset, which was applied to predict the sentiment of each CAIM-related tweet, and we used a natural language processing package to identify the emotions based on the words contained in the tweets. RESULTS: Our dataset included 28 713 English-language Tweets. The number of CAIM-related tweets during the study period peaked in May 2020, then dropped off sharply over the subsequent three months; the fewest CAIM-related tweets were collected during August 2020 and remained low for the remainder of the collection period. Most tweets (n = 15 612, 54%) were classified as positive, 31% were neutral (n = 8803) and 15% were classified as negative (n = 4298). The most frequent emotions expressed across tweets were trust, followed by fear, while surprise and disgust were the least frequent. Though volume of tweets decreased over the 9 months of the study, the expressed sentiments and emotions remained constant. CONCLUSION: The results of this sentiment analysis enabled us to establish key CAIMs being discussed at the intersection of COVID-19 across a 9-month period on Twitter. Overall, the majority of our subset of tweets were positive, as were the emotions associated with the words found within them. This may be interpreted as public support for CAIM, however, further qualitative investigation is warranted. Such future directions may be used to combat misinformation and improve public health strategies surrounding the use of social media information.

Frontline connect: Evaluating a virtual technology program to enhance patient and provider communication during COVID-19.

RATIONALE: Since the beginning of the COVID-19 pandemic, many hospitals have reduced in-hospital visitation. In these situations, virtual communication tools have helped maintain interaction between parties. The Frontline Connect program was designed to address communication and patient care challenges by providing data-enabled devices to clinical staff in hospitals. OBJECTIVE: This study aimed to identify areas of improvement for the Frontline Connect program by: (a) evaluating communication needs, user experience, and program satisfaction; and (b) identifying potential barriers to device access or use. METHODS: We administered pre-implementation needs assessment, post-use, and exit surveys to healthcare staff at a pilot hospital site in Ontario. Recruitment was through email lists and site champions using convenience sampling. We descriptively analysed survey responses and compared the initial need statements to post-implementation use-cases identified by users. RESULTS: We received 139 needs assessments, 31 user experience assessments, and 47 exit survey responses. Most device use occurred in the emergency department and intensive care units and was facilitated by social workers, nurses, and physicians to connect patients, families, and care providers. Pre-implementation concerns were related to infection control, data security, and device privacy. In the exit survey, these were replaced by other concerns including Internet connectivity and time-intensiveness. Device utility and ease-of-use were rated 9.7/10 and 9.6/10 respectively in the user experience survey, though overall experience was rated 7.2/10 in the exit survey. Overall, respondents viewed the devices as useful and we agree with participants who suggested increased program promotion and training would likely improve adoption. CONCLUSIONS: We found that our virtual technology program for facilitating communication was positively perceived. Survey feedback indicates that a rapid rollout in response to urgent pandemic-related needs was feasible, though program logistics could be improved. The current work supports the need to improve, standardize, and sustain virtual communication programs in hospitals.

Online Educational Tools for Caregivers of People with Dementia: A Scoping Literature Review.

BACKGROUND: Informal caregivers of people with dementia provide the majority of health-based care to people with dementia. Providing this care requires knowledge and access to resources, which caregivers often do not receive. We set out to evaluate the effect of online educational tools on informal caregiver self-efficacy, quality of life, burden/stress, depression, and anxiety, and to identify effective processes for online educational tool development. METHODS: We conducted a scoping review of articles on online educational interventions for informal caregivers of people with dementia searching CINAHL, MEDLINE, EMBASE, and PubMed from 1990 to March 2018, with an updated search conducted in 2020. The identified articles were screened and the data were charted. RESULTS: 33 articles that reported on 24 interventions were included. There is some evidence that online interventions improve caregiver-related outcomes such as self-efficacy, depression, dementia knowledge, and quality of life; and decrease caregiver burden. Common findings across the studies included the need for tailored, stage-specific information applicable to the caregiver's situation and the use of psychosocial techniques to develop the knowledge components of the interventions. CONCLUSION: We demonstrate the importance of having caregivers and health-care professionals involved at all stages of tool conceptualization and development. Online tools should be evaluated with robust trials that focus on how increased knowledge and development approaches affect caregiver-related outcomes.

A Deep Learning Approach to Refine the Identification of High-Quality Clinical Research Articles From the Biomedical Literature: Protocol for Algorithm Development and Validation.

BACKGROUND: A barrier to practicing evidence-based medicine is the rapidly increasing body of biomedical literature. Use of method terms to limit the search can help reduce the burden of screening articles for clinical relevance; however, such terms are limited by their partial dependence on indexing terms and usually produce low precision, especially when high sensitivity is required. Machine learning has been applied to the identification of high-quality literature with the potential to achieve high precision without sacrificing sensitivity. The use of artificial intelligence has shown promise to improve the efficiency of identifying sound evidence. OBJECTIVE: The primary objective of this research is to derive and validate deep learning machine models using iterations of Bidirectional Encoder Representations from Transformers (BERT) to retrieve high-quality, high-relevance evidence for clinical consideration from the biomedical literature. METHODS: Using the HuggingFace Transformers library, we will experiment with variations of BERT models, including BERT, BioBERT, BlueBERT, and PubMedBERT, to determine which have the best performance in article identification based on quality criteria. Our experiments will utilize a large data set of over 150,000 PubMed citations from 2012 to 2020 that have been manually labeled based on their methodological rigor for clinical use. We will evaluate and report on the performance of the classifiers in categorizing articles based on their likelihood of meeting quality criteria. We will report fine-tuning hyperparameters for each model, as well as their performance metrics, including recall (sensitivity), specificity, precision, accuracy, F-score, the number of articles that need to be read before finding one that is positive (meets criteria), and classification probability scores. RESULTS: Initial model development is underway, with further development planned for early 2022. Performance testing is expected to star in February 2022. Results will be published in 2022. CONCLUSIONS: The experiments will aim to improve the precision of retrieving high-quality articles by applying a machine learning classifier to PubMed searching. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): DERR1-10.2196/29398.

Knowledge translation strategies for sharing evidence-based health information with older adults and their caregivers: findings from a persona-scenario method.

BACKGROUND: By understanding the information seeking behaviors of older adults, we can better develop or iterate effective information technologies, such as the McMaster Optimal Aging Portal, that provide evidence-based health information to the public. This paper reports health-related information seeking and searching behaviours and provides strategies for effective knowledge translation (KT) to increase awareness and use of reliable health information. METHODS: We conducted a qualitative study with eighteen older adults using the persona-scenario method, whereby participants created personas and scenarios describing older adults seeking health information. Scenarios were analyzed using a two-phase inductive qualitative approach, with the personas as context. From the findings related to pathways of engaging with health information, we identified targeted KT strategies to raise awareness and uptake of evidence-based information resources. RESULTS: Twelve women and six men, 60 to 81 years of age, participated. In pairs, they created twelve personas that captured rural and urban, male and female, and immigrant perspectives. Some scenarios described older adults who did not engage directly with technology, but rather accessed information indirectly through other sources or preferred nondigital modes of delivery. Two major themes regarding KT considerations were identified: connecting to information via other people and personal venues (people included healthcare professionals, librarians, and personal networks; personal venues included clinics, libraries, pharmacies, and community gatherings); and health information delivery formats, (e.g., printed and multimedia formats for web-based resources). For each theme, and any identified subthemes, corresponding sets of suggested KT strategies are presented. CONCLUSIONS: Our findings underline the importance of people, venues, and formats in the actions of older adults seeking trusted health information and highlight the need for enhanced KT strategies to share information across personal and professional networks of older adults. KT strategies that could be employed by organizations or communities sharing evidence-based, reliable health information include combinations of educational outreach and materials, decision support tools, small group sessions, publicity campaigns, champions/opinion leaders, and conferences.

Machine Learning Approaches to Retrieve High-Quality, Clinically Relevant Evidence From the Biomedical Literature: Systematic Review.

BACKGROUND: The rapid growth of the biomedical literature makes identifying strong evidence a time-consuming task. Applying machine learning to the process could be a viable solution that limits effort while maintaining accuracy. OBJECTIVE: The goal of the research was to summarize the nature and comparative performance of machine learning approaches that have been applied to retrieve high-quality evidence for clinical consideration from the biomedical literature. METHODS: We conducted a systematic review of studies that applied machine learning techniques to identify high-quality clinical articles in the biomedical literature. Multiple databases were searched to July 2020. Extracted data focused on the applied machine learning model, steps in the development of the models, and model performance. RESULTS: From 3918 retrieved studies, 10 met our inclusion criteria. All followed a supervised machine learning approach and applied, from a limited range of options, a high-quality standard for the training of their model. The results show that machine learning can achieve a sensitivity of 95% while maintaining a high precision of 86%. CONCLUSIONS: Machine learning approaches perform well in retrieving high-quality clinical studies. Performance may improve by applying more sophisticated approaches such as active learning and unsupervised machine learning approaches.

Health programmes and services addressing the prevention and management of infectious diseases in people who inject drugs in Canada: a systematic integrative review.

OBJECTIVES: People who inject drugs (PWID) experience a high burden of injection drug use-related infectious disease and challenges in accessing adequate care. This study sought to identify programmes and services in Canada addressing the prevention and management of infectious disease in PWID. DESIGN: This study employed a systematic integrative review methodology. Electronic databases (PubMed, CINAHL and Web of Science Core Collection) and relevant websites were searched for literature published between 2008 and 2019 (last search date was 6 June 2019). Eligible articles and documents were required to address injection or intravenous drug use and health programmes or services relating to the prevention or management of infectious diseases in Canada. RESULTS: This study identified 1607 unique articles and 97 were included in this study. The health programmes and services identified included testing and management of HIV and hepatitis C virus (n=27), supervised injection facilities (n=19), medication treatment for opioid use disorder (n=12), integrated infectious disease and addiction programmes (n=10), needle exchange programmes (n=9), harm reduction strategies broadly (n=6), mobile care initiatives (n=5), peer-delivered services (n=3), management of IDU-related bacterial infections (n=2) and others (n=4). Key implications for policy, practice and future research were identified based on the results of the included studies, which include addressing individual and systemic factors that impede care, furthering evaluation of programmes and the need to provide comprehensive care to PWID, involving medical care, social support and harm reduction. CONCLUSIONS: These results demonstrate the need for expanded services across a variety of settings and populations. Our study emphasises the importance of addressing social and structural factors that impede infectious disease care for PWID. Further research is needed to improve evaluation of health programmes and services and contextual factors surrounding accessing services or returning to care. PROSPERO REGISTRATION NUMBER: CRD42020142947.

eHealth Implementation Issues in Low-Resource Countries: Model, Survey, and Analysis of User Experience.

BACKGROUND: The implementation of eHealth in low-resource countries (LRCs) is challenged by limited resources and infrastructure, lack of focus on eHealth agendas, ethical and legal considerations, lack of common system interoperability standards, unreliable power, and shortage of trained workers. OBJECTIVE: The aim of this study is to describe and study the current situation of eHealth implementation in a small number of LRCs from the perspectives of their professional eHealth users. METHODS: We developed a structural equation model that reflects the opinions of professional eHealth users who work on LRC health care front lines. We recruited country coordinators from 4 LRCs to help recruit survey participants: India, Egypt, Nigeria, and Kenya. Through a web-based survey that focused on barriers to eHealth implementation, we surveyed 114 participants. We analyzed the information using a structural equation model to determine the relationships among the constructs in the model, including the dependent variable, eHealth utilization. RESULTS: Although all the model constructs were important to participants, some constructs, such as user characteristics, perceived privacy, and perceived security, did not play a significant role in eHealth utilization. However, the constructs related to technology infrastructure tended to reduce the impact of concerns and uncertainties (path coefficient=-0.32; P=.001), which had a negative impact on eHealth utilization (path coefficient=-0.24; P=.01). Constructs that were positively related to eHealth utilization were implementation effectiveness (path coefficient=0.45; P<.001), the countries where participants worked (path coefficient=0.29; P=.004), and whether they worked for privately or publicly funded institutions (path coefficient=0.18; P<.001). As exploratory research, the model had a moderately good fit for eHealth utilization (adjusted R2=0.42). CONCLUSIONS: eHealth success factors can be categorized into 5 groups; our study focused on frontline eHealth workers' opinions concerning 2 of these groups: technology and its support infrastructure and user acceptance. We found significant disparities among the responses from different participant groups. Privately funded organizations tended to be further ahead with eHealth utilization than those that were publicly funded. Moreover, participant comments identified the need for more use of telemedicine in remote and rural regions in these countries. An understanding of these differences can help regions or countries that are lagging in the implementation and use of eHealth technologies. Our approach could also be applied to detailed studies of the other 3 categories of success factors: short- and long-term funding, organizational factors, and political or legislative aspects.