Deep learning to refine the identification of high-quality clinical research articles from the biomedical literature: Performance evaluation.

BACKGROUND: Identifying practice-ready evidence-based journal articles in medicine is a challenge due to the sheer volume of biomedical research publications. Newer approaches to support evidence discovery apply deep learning techniques to improve the efficiency and accuracy of classifying sound evidence. OBJECTIVE: To determine how well deep learning models using variants of Bidirectional Encoder Representations from Transformers (BERT) identify high-quality evidence with high clinical relevance from the biomedical literature for consideration in clinical practice. METHODS: We fine-tuned variations of BERT models (BERTBASE, BioBERT, BlueBERT, and PubMedBERT) and compared their performance in classifying articles based on methodological quality criteria. The dataset used for fine-tuning models included titles and abstracts of >160,000 PubMed records from 2012 to 2020 that were of interest to human health which had been manually labeled based on meeting established critical appraisal criteria for methodological rigor. The data was randomly divided into 80:10:10 sets for training, validating, and testing. In addition to using the full unbalanced set, the training data was randomly undersampled into four balanced datasets to assess performance and select the best performing model. For each of the four sets, one model that maintained sensitivity (recall) at ≥99% was selected and were ensembled. The best performing model was evaluated in a prospective, blinded test and applied to an established reference standard, the Clinical Hedges dataset. RESULTS: In training, three of the four selected best performing models were trained using BioBERTBASE. The ensembled model did not boost performance compared with the best individual model. Hence a solo BioBERT-based model (named DL-PLUS) was selected for further testing as it was computationally more efficient. The model had high recall (>99%) and 60% to 77% specificity in a prospective evaluation conducted with blinded research associates and saved >60% of the work required to identify high quality articles. CONCLUSIONS: Deep learning using pretrained language models and a large dataset of classified articles produced models with improved specificity while maintaining >99% recall. The resulting DL-PLUS model identifies high-quality, clinically relevant articles from PubMed at the time of publication. The model improves the efficiency of a literature surveillance program, which allows for faster dissemination of appraised research.

Knowledge translation strategies for sharing evidence-based health information with older adults and their caregivers: findings from a persona-scenario method.

BACKGROUND: By understanding the information seeking behaviors of older adults, we can better develop or iterate effective information technologies, such as the McMaster Optimal Aging Portal, that provide evidence-based health information to the public. This paper reports health-related information seeking and searching behaviours and provides strategies for effective knowledge translation (KT) to increase awareness and use of reliable health information. METHODS: We conducted a qualitative study with eighteen older adults using the persona-scenario method, whereby participants created personas and scenarios describing older adults seeking health information. Scenarios were analyzed using a two-phase inductive qualitative approach, with the personas as context. From the findings related to pathways of engaging with health information, we identified targeted KT strategies to raise awareness and uptake of evidence-based information resources. RESULTS: Twelve women and six men, 60 to 81 years of age, participated. In pairs, they created twelve personas that captured rural and urban, male and female, and immigrant perspectives. Some scenarios described older adults who did not engage directly with technology, but rather accessed information indirectly through other sources or preferred nondigital modes of delivery. Two major themes regarding KT considerations were identified: connecting to information via other people and personal venues (people included healthcare professionals, librarians, and personal networks; personal venues included clinics, libraries, pharmacies, and community gatherings); and health information delivery formats, (e.g., printed and multimedia formats for web-based resources). For each theme, and any identified subthemes, corresponding sets of suggested KT strategies are presented. CONCLUSIONS: Our findings underline the importance of people, venues, and formats in the actions of older adults seeking trusted health information and highlight the need for enhanced KT strategies to share information across personal and professional networks of older adults. KT strategies that could be employed by organizations or communities sharing evidence-based, reliable health information include combinations of educational outreach and materials, decision support tools, small group sessions, publicity campaigns, champions/opinion leaders, and conferences.

eHealth Implementation Issues in Low-Resource Countries: Model, Survey, and Analysis of User Experience.

BACKGROUND: The implementation of eHealth in low-resource countries (LRCs) is challenged by limited resources and infrastructure, lack of focus on eHealth agendas, ethical and legal considerations, lack of common system interoperability standards, unreliable power, and shortage of trained workers. OBJECTIVE: The aim of this study is to describe and study the current situation of eHealth implementation in a small number of LRCs from the perspectives of their professional eHealth users. METHODS: We developed a structural equation model that reflects the opinions of professional eHealth users who work on LRC health care front lines. We recruited country coordinators from 4 LRCs to help recruit survey participants: India, Egypt, Nigeria, and Kenya. Through a web-based survey that focused on barriers to eHealth implementation, we surveyed 114 participants. We analyzed the information using a structural equation model to determine the relationships among the constructs in the model, including the dependent variable, eHealth utilization. RESULTS: Although all the model constructs were important to participants, some constructs, such as user characteristics, perceived privacy, and perceived security, did not play a significant role in eHealth utilization. However, the constructs related to technology infrastructure tended to reduce the impact of concerns and uncertainties (path coefficient=-0.32; P=.001), which had a negative impact on eHealth utilization (path coefficient=-0.24; P=.01). Constructs that were positively related to eHealth utilization were implementation effectiveness (path coefficient=0.45; P<.001), the countries where participants worked (path coefficient=0.29; P=.004), and whether they worked for privately or publicly funded institutions (path coefficient=0.18; P<.001). As exploratory research, the model had a moderately good fit for eHealth utilization (adjusted R2=0.42). CONCLUSIONS: eHealth success factors can be categorized into 5 groups; our study focused on frontline eHealth workers' opinions concerning 2 of these groups: technology and its support infrastructure and user acceptance. We found significant disparities among the responses from different participant groups. Privately funded organizations tended to be further ahead with eHealth utilization than those that were publicly funded. Moreover, participant comments identified the need for more use of telemedicine in remote and rural regions in these countries. An understanding of these differences can help regions or countries that are lagging in the implementation and use of eHealth technologies. Our approach could also be applied to detailed studies of the other 3 categories of success factors: short- and long-term funding, organizational factors, and political or legislative aspects.

AIMD - a validated, simplified framework of interventions to promote and integrate evidence into health practices, systems, and policies.

BACKGROUND: Proliferation of terms describing the science of effectively promoting and supporting the use of research evidence in healthcare policy and practice has hampered understanding and development of the field. To address this, an international Terminology Working Group developed and published a simplified framework of interventions to promote and integrate evidence into health practices, systems, and policies. This paper presents results of validation work and a second international workgroup meeting, culminating in the updated AIMD framework [Aims, Ingredients, Mechanism, Delivery]. METHODS: Framework validity was evaluated against terminology schemas (n = 51); primary studies (n = 37); and reporting guidelines (n = 10). Framework components were independently categorized as fully represented, partly represented, or absent by two researchers. Opportunities to refine the framework were systematically recorded. A meeting of the expanded international Terminology Working Group updated the framework by reviewing and deliberating upon validation findings and refinement proposals. RESULTS: There was variation in representativeness of the components across the three types of literature, in particular for the component 'causal mechanisms'. Analysis of primary studies revealed that representativeness of this concept lowered from 92 to 68% if only explicit, rather than explicit and non-explicit references to causal mechanisms were included. All components were very well represented in reporting guidelines, however the level of description of these was lower than in other types of literature. Twelve opportunities were identified to improve the framework, 9 of which were operationalized at the meeting. The updated AIMD framework comprises four components: (1) Aims: what do you want your intervention to achieve and for whom? (2) Ingredients: what comprises the intervention? (3) Mechanisms: how do you propose the intervention will work? and (4) Delivery: how will you deliver the intervention? CONCLUSIONS: The draft simplified framework was validated with reference to a wide range of relevant literature and improvements have enhanced useability. The AIMD framework could aid in the promotion of evidence into practice, remove barriers to understanding how interventions work, enhance communication of interventions and support knowledge synthesis. Future work needs to focus on developing and testing resources and educational initiatives to optimize use of the AIMD framework in collaboration with relevant end-user groups.

Limits of search filter development.

OBJECTIVE: The research attempted to develop search filters for biomedical literature databases that improve retrieval of studies of clinical relevance for the nursing and rehabilitation professions. METHODS: Diagnostic testing framework compared machine-culled and practitioner-nominated search terms with a hand-tagged clinical literature database. RESULTS: We were unable to: (1) develop filters for nursing, likely because of the overlapping and expanding scope of practice for nurses in comparison with medical professionals, or (2) develop filters for rehabilitation, because of its broad scope and the profession's multifaceted understanding of "health and ability." CONCLUSIONS: We found limitations on search filter development for these health professions: nursing and rehabilitation.

Engaging with a Wiki related to knowledge translation: a survey of WhatisKT Wiki users.

BACKGROUND: In 2008, WhatisKT wiki was launched as a collaborative platform for knowledge translation (KT) researchers and stakeholders to debate the use and definitions of KT-related terms. The wiki has definitions for over 110 terms from disciplines including health care, information technology, education, accounting, and business. WhatisKT wiki has over 115 registered users. Approximately 73,000 unique visitors have visited the wiki since 2008. Despite annual increases in visitors and regular maintenance of the wiki, no visitors have contributed content or started a discussion. OBJECTIVE: We surveyed wiki users to gain an understanding of the perceived value of the website, reasons for not engaging in the wiki, and suggestions to facilitate collaboration and improve the usability of the wiki. METHODS: We surveyed three cohorts: KT Canada members who were previously invited to join the wiki, registered wiki members, and unregistered visitors. The first two cohorts completed a Web-based survey that included the System Usability Scale (SUS) questionnaire to assess usability; additionally 3 participants were interviewed. Unregistered wiki visitors were surveyed with polls posted on the wiki. The study received ethics approval from the McMaster University Faculty of Health Sciences Research Ethics Board. RESULTS: Twenty-three participants completed the Web-based and SUS surveys; 15 participants indicated that they would collaborate on the wiki. The mean SUS score of 67 (95% CI 56-77) indicated that the wiki could be considered for design improvements. Study participants indicated that the wiki could be improved by email notification regarding new terms, better grouping of terms, user friendly interface, and training for users interested in editing content. CONCLUSIONS: The findings from this survey will be used to enhance the design and content of WhatisKT wiki. Further feedback from participants will be used to make the wiki an ideal collaboration platform for KT researchers interested in terminology.

Seeing the random forest through the decision trees. Supporting learning health systems from histopathology with machine learning models: Challenges and opportunities.

This paper discusses some overlooked challenges faced when working with machine learning models for histopathology and presents a novel opportunity to support "Learning Health Systems" with them. Initially, the authors elaborate on these challenges after separating them according to their mitigation strategies: those that need innovative approaches, time, or future technological capabilities and those that require a conceptual reappraisal from a critical perspective. Then, a novel opportunity to support "Learning Health Systems" by integrating hidden information extracted by ML models from digitalized histopathology slides with other healthcare big data is presented.

Performance of externally validated machine learning models based on histopathology images for the diagnosis, classification, prognosis, or treatment outcome prediction in female breast cancer: A systematic review.

Numerous machine learning (ML) models have been developed for breast cancer using various types of data. Successful external validation (EV) of ML models is important evidence of their generalizability. The aim of this systematic review was to assess the performance of externally validated ML models based on histopathology images for diagnosis, classification, prognosis, or treatment outcome prediction in female breast cancer. A systematic search of MEDLINE, EMBASE, CINAHL, IEEE, MICCAI, and SPIE conferences was performed for studies published between January 2010 and February 2022. The Prediction Model Risk of Bias Assessment Tool (PROBAST) was employed, and the results were narratively described. Of the 2011 non-duplicated citations, 8 journal articles and 2 conference proceedings met inclusion criteria. Three studies externally validated ML models for diagnosis, 4 for classification, 2 for prognosis, and 1 for both classification and prognosis. Most studies used Convolutional Neural Networks and one used logistic regression algorithms. For diagnostic/classification models, the most common performance metrics reported in the EV were accuracy and area under the curve, which were greater than 87% and 90%, respectively, using pathologists' annotations/diagnoses as ground truth. The hazard ratios in the EV of prognostic ML models were between 1.7 (95% CI, 1.2-2.6) and 1.8 (95% CI, 1.3-2.7) to predict distant disease-free survival; 1.91 (95% CI, 1.11-3.29) for recurrence, and between 0.09 (95% CI, 0.01-0.70) and 0.65 (95% CI, 0.43-0.98) for overall survival, using clinical data as ground truth. Despite EV being an important step before the clinical application of a ML model, it hasn't been performed routinely. The large variability in the training/validation datasets, methods, performance metrics, and reported information limited the comparison of the models and the analysis of their results. Increasing the availability of validation datasets and implementing standardized methods and reporting protocols may facilitate future analyses.

The relationship between patient experience and real-world digital health access in primary care: A population-based cross-sectional study.

Implementing digital health technologies in primary care is anticipated to improve patient experience. We examined the relationships between patient experience and digital health access in primary care settings in Ontario, Canada. We conducted a retrospective cross-sectional study using patient responses to the Health Care Experience Survey linked to health and administrative data between April 2019-February 2020. We measured patient experience by summarizing HCES questions. We used multivariable logistic regression stratified by the number of primary care visits to investigate associations between patient experience with digital health access and moderating variables. Our cohort included 2,692 Ontario adults, of which 63.0% accessed telehealth, 2.6% viewed medical records online, and 3.6% booked appointments online. Although patients reported overwhelmingly positive experiences, we found no consistent relationship with digital health access. Online appointment booking access was associated with lower odds of poor experience for patients with three or more primary care visits in the past 12 months (adjusted odds ratio 0.16, 95% CI 0.02-0.56). Younger age, tight financial circumstances, English as a second language, and knowing their primary care provider for fewer years had greater odds of poor patient experience. In 2019/2020, we found limited uptake of digital health in primary care and no clear association between real-world digital health adoption and patient experience in Ontario. Our findings provide an essential context for ensuing rapid shifts in digital health adoption during the COVID-19 pandemic, serving as a baseline to reexamine subsequent improvements in patient experience.

A scoping review reveals candidate quality indicators of knowledge translation and implementation science practice tools.

OBJECTIVES: To identify candidate quality indicators from existing tools that provide guidance on how to practice knowledge translation and implemenation science (KT practice tools) across KT domains (dissemination, implementation, sustainability, and scalability). STUDY DESIGN AND SETTING: We conducted a scoping review using the Joanna Briggs Institute Manual for Evidence Synthesis. We systematically searched multiple electronic databases and the gray literature. Documents were independently screened, selected, and extracted by pairs of reviewers. Data about the included articles, KT practice tools, and candidate quality indicators were analyzed, categorized, and summarized descriptively. RESULTS: Of 43,060 titles and abstracts that were screened from electronic databases and gray literature, 850 potentially relevant full-text articles were identified, and 253 articles were included in the scoping review. Of these, we identified 232 unique KT practice tools from which 27 unique candidate quality indicators were generated. The identified candidate quality indicators were categorized according to the development (n = 17), evaluation (n = 5) and adaptation (n = 3) of the tools, and engagement of knowledge users (n = 2). No tools were identified that appraised the quality of KT practice tools. CONCLUSIONS: The development of a quality appraisal instrument of KT practice tools is needed. The results will be further refined and finalized in order to develop a quality appraisal instrument for KT practice tools.

Net improvement of correct answers to therapy questions after pubmed searches: pre/post comparison.

BACKGROUND: Clinicians search PubMed for answers to clinical questions although it is time consuming and not always successful. OBJECTIVE: To determine if PubMed used with its Clinical Queries feature to filter results based on study quality would improve search success (more correct answers to clinical questions related to therapy). METHODS: We invited 528 primary care physicians to participate, 143 (27.1%) consented, and 111 (21.0% of the total and 77.6% of those who consented) completed the study. Participants answered 14 yes/no therapy questions and were given 4 of these (2 originally answered correctly and 2 originally answered incorrectly) to search using either the PubMed main screen or PubMed Clinical Queries narrow therapy filter via a purpose-built system with identical search screens. Participants also picked 3 of the first 20 retrieved citations that best addressed each question. They were then asked to re-answer the original 14 questions. RESULTS: We found no statistically significant differences in the rates of correct or incorrect answers using the PubMed main screen or PubMed Clinical Queries. The rate of correct answers increased from 50.0% to 61.4% (95% CI 55.0%-67.8%) for the PubMed main screen searches and from 50.0% to 59.1% (95% CI 52.6%-65.6%) for Clinical Queries searches. These net absolute increases of 11.4% and 9.1%, respectively, included previously correct answers changing to incorrect at a rate of 9.5% (95% CI 5.6%-13.4%) for PubMed main screen searches and 9.1% (95% CI 5.3%-12.9%) for Clinical Queries searches, combined with increases in the rate of being correct of 20.5% (95% CI 15.2%-25.8%) for PubMed main screen searches and 17.7% (95% CI 12.7%-22.7%) for Clinical Queries searches. CONCLUSIONS: PubMed can assist clinicians answering clinical questions with an approximately 10% absolute rate of improvement in correct answers. This small increase includes more correct answers partially offset by a decrease in previously correct answers.

Enhanced recovery support for people with eating disorders during the COVID-19 pandemic: quality improvement using a web-based, stepped-care programme in Canada.

BACKGROUND: Throughout the COVID-19 pandemic, the number of individuals struggling with eating disorders (EDs) increased substantially. Body Brave (a not-for-profit) created and implemented a web-based stepped-care Recovery Support Programme (RSP) to improve access to community-based ED services. This quality improvement study describes the RSP and assesses its ability to deliver timely access to treatment and platform engagement. METHODS: We conducted a retrospective cohort study comparing access to, and use of Body Brave services 6 months before and 12 months after implementation of the RSP platform (using 6-month increments for two postimplementation periods). Primary programme quality measures included registration requests, number of participants onboarded and time to access services; secondary measures included use of RSP action plans, attendance for recovery sessions and workshops, number of participants accessing treatment and text-based patient experience data. RESULTS: A substantial increase in registration requests was observed during the first postimplementation period compared with the preimplementation period (176.5 vs 85.5; p=0.028). When compared with the preimplementation period, the second postimplementation observed a significantly larger percentage of successfully onboarded participants (76.6 vs 37.9; p<0.01) and a reduction in the number of days to access services (2 days vs 31 days; p<0.01). Although participant feedback rates were low, many users found the RSP helpful, easy to access, user-friendly and were satisfied overall. Users provided suggestions for improvement (eg, a platform instructional video, offer multiple times of day for live sessions and drop-in hours). CONCLUSIONS: Although clinical benefit needs to be assessed, our findings demonstrate that the RSP enabled participants to quickly onboard and access initial services and have informed subsequent improvements. Understanding initial programme effects and usage will help assess the feasibility of adapting and expanding the RSP across Canada to address the urgent need for low-barrier, patient-centred ED care.

Public Health Surveillance of Behavioral Cancer Risk Factors During the COVID-19 Pandemic: Sentiment and Emotion Analysis of Twitter Data.

BACKGROUND: The COVID-19 pandemic and its associated public health mitigation strategies have dramatically changed patterns of daily life activities worldwide, resulting in unintentional consequences on behavioral risk factors, including smoking, alcohol consumption, poor nutrition, and physical inactivity. The infodemic of social media data may provide novel opportunities for evaluating changes related to behavioral risk factors during the pandemic. OBJECTIVE: We explored the feasibility of conducting a sentiment and emotion analysis using Twitter data to evaluate behavioral cancer risk factors (physical inactivity, poor nutrition, alcohol consumption, and smoking) over time during the first year of the COVID-19 pandemic. METHODS: Tweets during 2020 relating to the COVID-19 pandemic and the 4 cancer risk factors were extracted from the George Washington University Libraries Dataverse. Tweets were defined and filtered using keywords to create 4 data sets. We trained and tested a machine learning classifier using a prelabeled Twitter data set. This was applied to determine the sentiment (positive, negative, or neutral) of each tweet. A natural language processing package was used to identify the emotions (anger, anticipation, disgust, fear, joy, sadness, surprise, and trust) based on the words contained in the tweets. Sentiments and emotions for each of the risk factors were evaluated over time and analyzed to identify keywords that emerged. RESULTS: The sentiment analysis revealed that 56.69% (51,479/90,813) of the tweets about physical activity were positive, 16.4% (14,893/90,813) were negative, and 26.91% (24,441/90,813) were neutral. Similar patterns were observed for nutrition, where 55.44% (27,939/50,396), 15.78% (7950/50,396), and 28.79% (14,507/50,396) of the tweets were positive, negative, and neutral, respectively. For alcohol, the proportions of positive, negative, and neutral tweets were 46.85% (34,897/74,484), 22.9% (17,056/74,484), and 30.25% (22,531/74,484), respectively, and for smoking, they were 41.2% (11,628/28,220), 24.23% (6839/28,220), and 34.56% (9753/28,220), respectively. The sentiments were relatively stable over time. The emotion analysis suggests that the most common emotion expressed across physical activity and nutrition tweets was trust (69,495/320,741, 21.67% and 42,324/176,564, 23.97%, respectively); for alcohol, it was joy (49,147/273,128, 17.99%); and for smoking, it was fear (23,066/110,256, 20.92%). The emotions expressed remained relatively constant over the observed period. An analysis of the most frequent words tweeted revealed further insights into common themes expressed in relation to some of the risk factors and possible sources of bias. CONCLUSIONS: This analysis provided insight into behavioral cancer risk factors as expressed on Twitter during the first year of the COVID-19 pandemic. It was feasible to extract tweets relating to all 4 risk factors, and most tweets had a positive sentiment with varied emotions across the different data sets. Although these results can play a role in promoting public health, a deeper dive via qualitative analysis can be conducted to provide a contextual examination of each tweet.

Identifying candidate quality indicators of tools that support the practice of knowledge translation: a scoping review protocol.

OBJECTIVE: The objective of this scoping review is to identify and characterize relevant knowledge translation methods tools (those that provide guidance for optimized knowledge translation practice) to uncover candidate quality indicators to inform a future quality assessment tool for knowledge translation strategies. INTRODUCTION: Knowledge translation strategies (defined as including knowledge translation interventions, tools, and products) target various knowledge users, including patients, clinicians, researchers, and policy-makers. The development and use of strategies that support knowledge translation practice have been rapidly increasing, making it difficult for knowledge users to decide which to use. There is limited evidence-based guidance or measures to help assess the overall quality of knowledge translation strategies. INCLUSION CRITERIA: Empirical and non-empirical documents will be considered if they explicitly describe a knowledge translation methods tool and its development, evaluation or validation, methodological strengths or limitations, and/or use over time. The review will consider a knowledge translation methods tool if it falls within at least one knowledge translation domain (ie, implementation, dissemination, sustainability, scalability, integrated knowledge translation) in the health field. METHODS: We will conduct a systematic search of relevant electronic databases and gray literature. The search strategy will be developed iteratively by an experienced medical information specialist and peer-reviewed with the PRESS checklist. The search will be limited to English-only documents published from 2005 onward. Documents will be independently screened, selected, and extracted by 2 researchers. Data will be analyzed and summarized descriptively, including the characteristics of the included documents, knowledge translation methods tools, and candidate quality indicators. SCOPING REVIEW REGISTRATION: Open Science Framework ( https://osf.io/chxvq ).

How current are leading evidence-based medical textbooks? An analytic survey of four online textbooks.

BACKGROUND: The consistency of treatment recommendations of evidence-based medical textbooks with more recently published evidence has not been investigated to date. Inconsistencies could affect the quality of medical care. OBJECTIVE: To determine the frequency with which topics in leading online evidence-based medical textbooks report treatment recommendations consistent with more recently published research evidence. METHODS: Summarized treatment recommendations in 200 clinical topics (ie, disease states) covered in four evidence-based textbooks--UpToDate, Physicians' Information Education Resource (PIER), DynaMed, and Best Practice--were compared with articles identified in an evidence rating service (McMaster Premium Literature Service, PLUS) since the date of the most recent topic updates in each textbook. Textbook treatment recommendations were compared with article results to determine if the articles provided different, new conclusions. From these findings, the proportion of topics which potentially require updating in each textbook was calculated. RESULTS: 478 clinical topics were assessed for inclusion to find 200 topics that were addressed by all four textbooks. The proportion of topics for which there was 1 or more recently published articles found in PLUS with evidence that differed from the textbooks' treatment recommendations was 23% (95% CI 17-29%) for DynaMed, 52% (95% CI 45-59%) for UpToDate, 55% (95% CI 48-61%) for PIER, and 60% (95% CI 53-66%) for Best Practice (χ(2) (3)=65.3, P<.001). The time since the last update for each textbook averaged from 170 days (range 131-209) for DynaMed, to 488 days (range 423-554) for PIER (P<.001 across all textbooks). CONCLUSIONS: In online evidence-based textbooks, the proportion of topics with potentially outdated treatment recommendations varies substantially.

How well are journal and clinical article characteristics associated with the journal impact factor? a retrospective cohort study.

OBJECTIVE: Journal impact factor (JIF) is often used as a measure of journal quality. A retrospective cohort study determined the ability of clinical article and journal characteristics, including appraisal measures collected at the time of publication, to predict subsequent JIFs. METHODS: Clinical research articles that passed methods quality criteria were included. Each article was rated for relevance and newsworthiness by 3 to 24 physicians from a panel of more than 4,000 practicing clinicians. The 1,267 articles (from 103 journals) were divided 60∶40 into derivation (760 articles) and validation sets (507 articles), representing 99 and 88 journals, respectively. A multiple regression model was produced determining the association of 10 journal and article measures with the 2007 JIF. RESULTS: Four of the 10 measures were significant in the regression model: number of authors, number of databases indexing the journal, proportion of articles passing methods criteria, and mean clinical newsworthiness scores. With the number of disciplines rating the article, the 5 variables accounted for 61% of the variation in JIF (R(2) = 0.607, 95% CI 0.444 to 0.706, P<0.001). CONCLUSION: For the clinical literature, measures of scientific quality and clinical newsworthiness available at the time of publication can predict JIFs with 60% accuracy.

Tracking discussions of complementary, alternative, and integrative medicine in the context of the COVID-19 pandemic: a month-by-month sentiment analysis of Twitter data.

BACKGROUND: Coronavirus disease 2019 (COVID-19) is a novel infectious disease caused by the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2). Despite the paucity of evidence, various complementary, alternative and integrative medicines (CAIMs) have been being touted as both preventative and curative. We conducted sentiment and emotion analysis with the intent of understanding CAIM content related to COVID-19 being generated on Twitter across 9 months. METHODS: Tweets relating to CAIM and COVID-19 were extracted from the George Washington University Libraries Dataverse Coronavirus tweets dataset from March 03 to November 30, 2020. We trained and tested a machine learning classifier using a large, pre-labelled Twitter dataset, which was applied to predict the sentiment of each CAIM-related tweet, and we used a natural language processing package to identify the emotions based on the words contained in the tweets. RESULTS: Our dataset included 28 713 English-language Tweets. The number of CAIM-related tweets during the study period peaked in May 2020, then dropped off sharply over the subsequent three months; the fewest CAIM-related tweets were collected during August 2020 and remained low for the remainder of the collection period. Most tweets (n = 15 612, 54%) were classified as positive, 31% were neutral (n = 8803) and 15% were classified as negative (n = 4298). The most frequent emotions expressed across tweets were trust, followed by fear, while surprise and disgust were the least frequent. Though volume of tweets decreased over the 9 months of the study, the expressed sentiments and emotions remained constant. CONCLUSION: The results of this sentiment analysis enabled us to establish key CAIMs being discussed at the intersection of COVID-19 across a 9-month period on Twitter. Overall, the majority of our subset of tweets were positive, as were the emotions associated with the words found within them. This may be interpreted as public support for CAIM, however, further qualitative investigation is warranted. Such future directions may be used to combat misinformation and improve public health strategies surrounding the use of social media information.

Challenges in eating disorder diagnosis and management among family physicians and trainees: a qualitative study.

BACKGROUND: Family physicians are one of the first points of contact for individuals with eating disorders (EDs) seeking care and treatment, but training in this area is suboptimal and insufficient. Specialized ED treatment programs often have long wait lists, and family physicians are responsible for patients care in the interim. The aim of this study was to identify the learning needs and challenges faced by Canadian family physicians and trainees when caring for patients with EDs. METHODS: We recruited six family medicine residents and five family physicians practicing in an academic unit in the Department of Family Medicine of a medical school in urban southwestern Ontario, Canada. We used purposive sampling, focusing on residents and faculty physicians from the department and conducted one focus group for the residents and another for the faculty physicians, exploring their clinical knowledge and challenges when managing ED patients. The focus groups were audio-recorded and transcribed verbatim prior to thematic coding. RESULTS: Physicians and residents faced challenges in discussing, screening, and managing patients with EDs. Three themes that emerged from the qualitative data highlighted training needs related to: (a) improving communication skills when treating a patient with an ED, (b) more effective screening and diagnosis in primary care practice, and (c) optimizing management strategies for patients with an ED, especially patients who are waiting for more intensive treatment. A fourth theme that emerged was the distress experienced by family physicians as they try best to manage and access care for their patients with EDs. CONCLUSION: Addressing the learning needs identified in this study through continuing education offerings could aid family physicians in confidently providing effective, evidence-based care to patients with EDs. Improvement in training and education could also alleviate some of the distress faced by family physicians in managing patients with EDs. Ultimately, system changes to allow more efficient and appropriate levels of care for patients with EDs, removing the burden from family medicine, are critical as EDs are on the rise. A person with an eating disorder will normally seek care from their family physician first. These conditions can dramatically reduce the quality of a person's life and health. Family physicians therefore need to know how best to help these patients or refer them to a more intensive level of care, which often has long wait lists. We asked a group of family physicians and a group of family medicine trainees about their experiences with patients with eating disorders and about the information they wished they had to help these patients. The results show that they need more information on how to talk to a patient about eating disorders without judgement, how to diagnose a patient with an eating disorder, and then what treatment and management is needed while they wait for more intensive treatment for sicker patients. The physicians and trainees both talked about the stress and worry that they faced when treating patients with eating disorders. Besides their lack of training about these conditions, family physicians also described difficulties when trying to access timely specialized services for their patients. Physicians can experience moral distress when they know that their patients need higher level care, but there are systemic barriers to specialized programs that block their patients from getting the care they need when they need it.

Frontline connect: Evaluating a virtual technology program to enhance patient and provider communication during COVID-19.

RATIONALE: Since the beginning of the COVID-19 pandemic, many hospitals have reduced in-hospital visitation. In these situations, virtual communication tools have helped maintain interaction between parties. The Frontline Connect program was designed to address communication and patient care challenges by providing data-enabled devices to clinical staff in hospitals. OBJECTIVE: This study aimed to identify areas of improvement for the Frontline Connect program by: (a) evaluating communication needs, user experience, and program satisfaction; and (b) identifying potential barriers to device access or use. METHODS: We administered pre-implementation needs assessment, post-use, and exit surveys to healthcare staff at a pilot hospital site in Ontario. Recruitment was through email lists and site champions using convenience sampling. We descriptively analysed survey responses and compared the initial need statements to post-implementation use-cases identified by users. RESULTS: We received 139 needs assessments, 31 user experience assessments, and 47 exit survey responses. Most device use occurred in the emergency department and intensive care units and was facilitated by social workers, nurses, and physicians to connect patients, families, and care providers. Pre-implementation concerns were related to infection control, data security, and device privacy. In the exit survey, these were replaced by other concerns including Internet connectivity and time-intensiveness. Device utility and ease-of-use were rated 9.7/10 and 9.6/10 respectively in the user experience survey, though overall experience was rated 7.2/10 in the exit survey. Overall, respondents viewed the devices as useful and we agree with participants who suggested increased program promotion and training would likely improve adoption. CONCLUSIONS: We found that our virtual technology program for facilitating communication was positively perceived. Survey feedback indicates that a rapid rollout in response to urgent pandemic-related needs was feasible, though program logistics could be improved. The current work supports the need to improve, standardize, and sustain virtual communication programs in hospitals.

Considerations for Conducting Bring Your Own "Device" (BYOD) Clinical Studies.

Background: Digital health technologies are attracting attention as novel tools for data collection in clinical research. They present alternative methods compared to in-clinic data collection, which often yields snapshots of the participants' physiology, behavior, and function that may be prone to biases and artifacts, e.g., white coat hypertension, and not representative of the data in free-living conditions. Modern digital health technologies equipped with multi-modal sensors combine different data streams to derive comprehensive endpoints that are important to study participants and are clinically meaningful. Used for data collection in clinical trials, they can be deployed as provisioned products where technology is given at study start or in a bring your own "device" (BYOD) manner where participants use their technologies to generate study data. Summary: The BYOD option has the potential to be more user-friendly, allowing participants to use technologies that they are familiar with, ensuring better participant compliance, and potentially reducing the bias that comes with introducing new technologies. However, this approach presents different technical, operational, regulatory, and ethical challenges to study teams. For example, BYOD data can be more heterogeneous, and recruiting historically underrepresented populations with limited access to technology and the internet can be challenging. Despite the rapid increase in digital health technologies for clinical and healthcare research, BYOD use in clinical trials is limited, and regulatory guidance is still evolving. Key Messages: We offer considerations for academic researchers, drug developers, and patient advocacy organizations on the design and deployment of BYOD models in clinical research. These considerations address: (1) early identification and engagement with internal and external stakeholders; (2) study design including informed consent and recruitment strategies; (3) outcome, endpoint, and technology selection; (4) data management including compliance and data monitoring; (5) statistical considerations to meet regulatory requirements. We believe that this article acts as a primer, providing insights into study design and operational requirements to ensure the successful implementation of BYOD clinical studies.