National learning curves among robotic thoracic surgeons in the United States: Quantifying the impact of procedural experience on efficiency and productivity gains.

OBJECTIVE: This study aims to characterize the aggregate learning curves of US surgeons for robotic thoracic procedures and to quantify the impact on productivity. METHODS: National average console times relative to cumulative case number were extracted from the My Intuitive application (Version 1.7.0). Intuitive da Vinci robotic system data for 56,668 lung resections performed by 870 individual surgeons between 2021 and 2022 were reviewed. Console time and hourly productivity (work relative value units/hour) were analyzed using linear regression models. RESULTS: Average console times improved for all robotic procedures with cumulative case experience (P = .003). Segmentectomy and thymectomy had the steepest initial learning curves with a 33% and 34% reduction of the average console time for proficient (51-100 cases) relative to novice surgeons (1-10 cases), respectively. The hourly productivity increase for proficient surgeons ranged from 11.4 work relative value units/hour (+26%) for lobectomy to 17.0 work relative value units/hour (+50%) for segmentectomy. At the expert level (101+ cases), average console times continued to decrease significantly for esophagectomy (-18%) and lobectomy (-23%), but only minimally for wedge resections (-1%) (P = .003). The work relative value units/hour increase at the expert level reached 50% for lobectomy and 40% for esophagectomy. Surgeon experience level, dual console use, system model, and robotic stapler use were factors independently associated with console time for robotic lobectomy. CONCLUSIONS: The aggregate learning curve for robotic thoracic surgeons in the United States varies significantly by procedure type and demonstrate continued improvements in efficiency beyond 100 cases for lobectomy and esophagectomy. Improvements in efficiency with growing experiences translate to substantial productivity gains.

Artificial Intelligence-enabled Decision Support in Surgery: State-of-the-art and Future Directions.

OBJECTIVE: To summarize state-of-the-art artificial intelligence-enabled decision support in surgery and to quantify deficiencies in scientific rigor and reporting. BACKGROUND: To positively affect surgical care, decision-support models must exceed current reporting guideline requirements by performing external and real-time validation, enrolling adequate sample sizes, reporting model precision, assessing performance across vulnerable populations, and achieving clinical implementation; the degree to which published models meet these criteria is unknown. METHODS: Embase, PubMed, and MEDLINE databases were searched from their inception to September 21, 2022 for articles describing artificial intelligence-enabled decision support in surgery that uses preoperative or intraoperative data elements to predict complications within 90 days of surgery. Scientific rigor and reporting criteria were assessed and reported according to Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines. RESULTS: Sample size ranged from 163-2,882,526, with 8/36 articles (22.2%) featuring sample sizes of less than 2000; 7 of these 8 articles (87.5%) had below-average (<0.83) area under the receiver operating characteristic or accuracy. Overall, 29 articles (80.6%) performed internal validation only, 5 (13.8%) performed external validation, and 2 (5.6%) performed real-time validation. Twenty-three articles (63.9%) reported precision. No articles reported performance across sociodemographic categories. Thirteen articles (36.1%) presented a framework that could be used for clinical implementation; none assessed clinical implementation efficacy. CONCLUSIONS: Artificial intelligence-enabled decision support in surgery is limited by reliance on internal validation, small sample sizes that risk overfitting and sacrifice predictive performance, and failure to report confidence intervals, precision, equity analyses, and clinical implementation. Researchers should strive to improve scientific quality.

Competencies for the Use of Artificial Intelligence-Based Tools by Health Care Professionals.

PURPOSE: The expanded use of clinical tools that incorporate artificial intelligence (AI) methods has generated calls for specific competencies for effective and ethical use. This qualitative study used expert interviews to define AI-related clinical competencies for health care professionals. METHOD: In 2021, a multidisciplinary team interviewed 15 experts in the use of AI-based tools in health care settings about the clinical competencies health care professionals need to work effectively with such tools. Transcripts of the semistructured interviews were coded and thematically analyzed. Draft competency statements were developed and provided to the experts for feedback. The competencies were finalized using a consensus process across the research team. RESULTS: Six competency domain statements and 25 subcompetencies were formulated from the thematic analysis. The competency domain statements are: (1) basic knowledge of AI: explain what AI is and describe its health care applications; (2) social and ethical implications of AI: explain how social, economic, and political systems influence AI-based tools and how these relationships impact justice, equity, and ethics; (3) AI-enhanced clinical encounters: carry out AI-enhanced clinical encounters that integrate diverse sources of information in creating patient-centered care plans; (4) evidence-based evaluation of AI-based tools: evaluate the quality, accuracy, safety, contextual appropriateness, and biases of AI-based tools and their underlying data sets in providing care to patients and populations; (5) workflow analysis for AI-based tools: analyze and adapt to changes in teams, roles, responsibilities, and workflows resulting from implementation of AI-based tools; and (6) practice-based learning and improvement regarding AI-based tools: participate in continuing professional development and practice-based improvement activities related to use of AI tools in health care. CONCLUSIONS: The 6 clinical competencies identified can be used to guide future teaching and learning programs to maximize the potential benefits of AI-based tools and diminish potential harms.

Parent Activation in the Neonatal Intensive Care Unit.

OBJECTIVE: Patient activation is the knowledge, skills, and confidence to manage one's health; parent activation is a comparable concept related to a parent's ability to manage a child's health. Activation in adults is a modifiable risk factor and associated with clinical outcomes and health care utilization. We examined activation in parents of hospitalized newborns observing temporal trends and associations with sociodemographic characteristics, neonate characteristics, and outcomes. STUDY DESIGN: Participants included adult parents of neonates admitted to a level-IV neonatal intensive care unit in an academic medical center. Activation was measured with the 10-item Parent version of the Patient Activation Measure (P-PAM) at admission, discharge, and 30 days after discharge. Associations with sociodemographic variables, health literacy, clinical variables, and health care utilization were evaluated. RESULTS: A total of 96 adults of 64 neonates were enrolled. The overall mean P-PAM score on admission was 81.8 (standard deviation [SD] = 18), 88.8 (SD = 13) at discharge, and 86.8 (SD = 16) at 30-day follow-up. Using linear mixed regression model, P-PAM score was significantly associated with timing of measurement. Higher P-PAM scores were associated with higher health literacy (p = 0.002) and higher in mothers compared to fathers (p = 0.040). There were no significant associations of admission P-PAM scores with sociodemographic characteristics. Parents of neonates who had a surgical diagnosis had a statistically significant (p = 0.003) lower score than those who did not. There were no associations between discharge P-PAM scores and neonates' lengths of stay or other indicators of illness severity. CONCLUSION: Parental activation in the NICU setting was higher than reported in the adult and limited pediatric literature; scores increased from admission to discharge and 30-day postdischarge. Activation was higher in mothers and parents with higher health literacy. Additional larger scale studies are needed to determine whether parental activation is associated with long-term health care outcomes as seen in adults. KEY POINTS: · Little is known about activation in parents of neonates.. · Activation plays a role in health outcomes in adults.. · Larger studies are needed to explore parent activation..

Mutational profiles of head and neck squamous cell carcinomas based upon human papillomavirus status in the Veterans Affairs National Precision Oncology Program.

BACKGROUND: Patients with advanced head and neck squamous cell carcinoma (HNSCC) associated with human papillomavirus (HPV) demonstrate favorable clinical outcomes compared to patients bearing HPV-negative HNSCC. We sought to characterize the association between HPV status and mutational profiles among patients served by the Veterans Health Administration (VHA). METHODS: We performed a retrospective analysis of all Veterans with primary HNSCC tumors who underwent next-generation sequencing (NGS) through the VHA's National Precision Oncology Program between July 2016 and February 2019. HPV status was determined by clinical pathology reports of p16 immunohistochemical staining; gene variant pathogenicity was classified using OncoKB, an online precision oncology knowledge database, and mutation frequencies were compared using Fisher's exact test. RESULTS: A total of 79 patients met inclusion criteria, of which 48 (60.8%) had p16-positive tumors. Patients with p16-negative HNSCC were more likely to have mutations in TP53 (p < 0.0001), and a trend towards increased mutation frequency was observed within NOTCH1 (p = 0.032) and within the composite CDK/Rb pathway (p = 0.065). Mutations in KRAS, NRAS, HRAS, and FBXW7 were exclusively identified within p16-positive tumors, and a trend towards increased frequency was observed within the PI3K pathway (p = 0.051). No difference in overall mutational burden was observed between the two groups. CONCLUSIONS: In accordance with the previous studies, no clear molecular basis for improved prognosis among patients harboring HPV-positive disease has been elucidated. Though no targeted therapies are approved based upon HPV-status, current efforts to trial PI3K inhibitors and mTOR inhibitors across patients with HPV-positive disease bear genomic rationale based upon the current findings.

Understanding a Care Management System's Role in Influencing a Transitional-Aged Youth Program's Practice: Mixed Methods Study.

BACKGROUND: Extended foster care programs help prepare transitional-aged youth (TAY) to step into adulthood and live independent lives. Aspiranet, one of California's largest social service organizations, used a social care management solution (SCMS) to meet TAY's needs. OBJECTIVE: We aimed to investigate the impact of an SCMS, IBM Watson Care Manager (WCM), in transforming foster program service delivery and improving TAY outcomes. METHODS: We used a mixed methods study design by collecting primary data from stakeholders through semistructured interviews in 2021 and by pulling secondary data from annual reports, system use logs, and data repositories from 2014 to 2021. Thematic analysis based on grounded theory was used to analyze qualitative data using NVivo software. Descriptive analysis of aggregated outcome metrics in the quantitative data was performed and compared across 2 periods: pre-SCMS implementation (before October 31, 2016) and post-SCMS implementation (November 1, 2016, and March 31, 2021). RESULTS: In total, 6 Aspiranet employees (4 leaders and 2 life coaches) were interviewed, with a median time of 56 (IQR 53-67) minutes. The majority (5/6, 83%) were female, over 30 years of age (median 37, IQR 32-39) with a median of 6 (IQR 5-10) years of experience at Aspiranet and overall field experience of 10 (IQR 7-14) years. Most (4/6, 67%) participants rated their technological skills as expert. Thematic analysis of participants' interview transcripts yielded 24 subthemes that were grouped into 6 superordinate themes: study context, the impact of the new tool, key strengths, commonly used features, expectations with WCM, and limitations and recommendations. The tool met users' initial expectations of streamlining tasks and adopting essential functionalities. Median satisfaction scores around pre- and post-WCM workflow processes remained constant between 2 life coaches (3.25, IQR 2.5-4); however, among leaders, post-WCM scores (median 4, IQR 4-5) were higher than pre-WCM scores (median 3, IQR 3-3). Across the 2 study phases, Aspiranet served 1641 TAY having consistent population demographics (median age of 18, IQR 18-19 years; female: 903/1641, 55.03%; race and ethnicity: Hispanic or Latino: 621/1641, 37.84%; Black: 470/1641, 28.64%; White: 397/1641, 24.19%; Other: 153/1641, 9.32%). Between the pre- and post-WCM period, there was an increase in full-time school enrollment (359/531, 67.6% to 833/1110, 75.04%) and a reduction in part-time school enrollment (61/531, 11.5% to 91/1110, 8.2%). The median number of days spent in the foster care program remained the same (247, IQR 125-468 years); however, the number of incidents reported monthly per hundred youth showed a steady decline, even with an exponentially increasing number of enrolled youth and incidents. CONCLUSIONS: The SCMS for coordinating care and delivering tailored services to TAY streamlined Aspiranet's workflows and processes and positively impacted youth outcomes. Further enhancements are needed to better align with user and youth needs.

Quantifying the phenome-wide disease burden of obesity using electronic health records and genomics.

OBJECTIVE: High BMI is associated with many comorbidities and mortality. This study aimed to elucidate the overall clinical risk of obesity using a genome- and phenome-wide approach. METHODS: This study performed a phenome-wide association study of BMI using a clinical cohort of 736,726 adults. This was followed by genetic association studies using two separate cohorts: one consisting of 65,174 adults in the Electronic Medical Records and Genomics (eMERGE) Network and another with 405,432 participants in the UK Biobank. RESULTS: Class 3 obesity was associated with 433 phenotypes, representing 59.3% of all billing codes in individuals with severe obesity. A genome-wide polygenic risk score for BMI, accounting for 7.5% of variance in BMI, was associated with 296 clinical diseases, including strong associations with type 2 diabetes, sleep apnea, hypertension, and chronic liver disease. In all three cohorts, 199 phenotypes were associated with class 3 obesity and polygenic risk for obesity, including novel associations such as increased risk of renal failure, venous insufficiency, and gastroesophageal reflux. CONCLUSIONS: This combined genomic and phenomic systematic approach demonstrated that obesity has a strong genetic predisposition and is associated with a considerable burden of disease across all disease classes.

Electronic Patient Portal Access, Retention in Care, and Viral Suppression Among People Living With HIV in Southeastern United States: Observational Study.

BACKGROUND: Approximately 1.1 million people living with HIV live in the United States, and the incidence is highest in Southeastern United States. Electronic patient portal prevalence is increasing and can improve engagement in primary medical care. Retention in care and viral suppression-measures of engagement in HIV care-are associated with decreased HIV transmission, morbidity, and mortality. OBJECTIVE: We aimed to determine if patient portal access among people living with HIV was associated with retention and viral suppression. METHODS: We conducted an observational cohort study among people living with HIV in care at the Vanderbilt Comprehensive Care Clinic (Nashville, Tennessee) from 2011-2016. Individual access was defined as patient portal account registration at any point in the year prior. Retention was defined as ≥2 kept appointments or HIV lab measurements ≥3 months apart within a 12-month period. Viral suppression was defined as the last viral load in the calendar year <200 copies/mL. We calculated adjusted prevalence ratios (aPRs) and 95% CIs using modified Poisson regression with generalized estimating equations to estimate the association of portal access with retention and viral suppression. RESULTS: We included 4237 people living with HIV contributing 16,951 person-years of follow-up (median 5, IQR 3-5 person-years). The median age was 43 (IQR 33-50) years. Of the 4237 people living with HIV, 78.1% (n=4237) were male, 40.8% (n=1727) were Black non-Hispanic, and 56.5% (n=2395) had access. Access was independently associated with retention (aPR 1.13, 95% CI 1.10-1.17) and viral suppression (aPR 1.18, 95% CI 1.14-1.22). CONCLUSIONS: In this population, patient portal access was associated with retention and viral suppression. Future prospective studies should assess the impact of increasing portal access among people living with HIV on these HIV outcomes.

A tribute to Karen Greenwood and her contributions to the American Medical Informatics Association.

After 25 years of service to the American Medical Informatics Association (AMIA), Ms Karen Greenwood, the Executive Vice President and Chief Operating Officer, is leaving the organization. In this perspective, we reflect on her accomplishments and her effect on the organization and the field of informatics nationally and globally. We also express our appreciation and gratitude for Ms Greenwood's role at AMIA.

SARS-CoV-2 Testing of Aerosols Emitted During Pediatric Minimally Invasive Surgery: A Prospective, Case-Controlled Study.

BACKGROUND: The COVID-19 pandemic has presented significant safety concerns for healthcare providers, especially those performing aerosol-generating procedures. Several surgical societies issued early warnings that aerosols generated during minimally invasive surgery (MIS) could harbor infectious quantities of severe acute respiratory syndrome coronavirus-2 (SARS-CoV-2). This study tested the hypothesis that MIS-aerosols contain SARS-CoV-2. METHODS: To evaluate SARS-CoV-2 presence in aerosols emitted during intracavitary MIS, children <18 years who required emergent MIS and were discovered to be SARS-CoV-2-positive were enrolled. Swabs were obtained from the port in-line with a filtered smoke evacuation system, the tubing adjacent to this port, the fluid collection chamber and filter, and the distal endotracheal tube (ETT). All swabs were analyzed for SARS-CoV-2 using quantitative reverse-transcription polymerase chain reaction. To evaluate viral distribution in tissues, fluorescence in situ hybridization for SARS-CoV-2 was performed on resected specimens. Outcomes were recorded, and participating healthcare workers were tracked for SARS-CoV-2 conversion. RESULTS: From July 1, 2020, to June 30, 2021, 11 children requiring emergent MIS were discovered preoperatively to be SARS-CoV-2 positive (median age: 14 years [5-17]). SARS-CoV-2 was detected only in ETT swabs and not in surgical aerosols or specimens. Median operative time was 56.5 minutes (IQR: 46-66), and postoperative stay was 21.2 hours (IQR: 1.97-57.57). No complications or viral eruption were recorded, and none of 63 healthcare workers tested positive for SARS-CoV-2 within 6 weeks. DISCUSSION: SARS-CoV-2 was detected only in ETT secretions and not in surgical aerosols or specimens among a pediatric cohort of asymptomatic patients having emergent MIS.

Differences in information accessed in a pharmacologic knowledge base using a conversational agent vs traditional search methods.

INTRODUCTION: Clinicians rely on pharmacologic knowledge bases to answer medication questions and avoid potential adverse drug events. In late 2018, an artificial intelligence-based conversational agent, Watson Assistant (WA), was made available to online subscribers to the pharmacologic knowledge base, Micromedex®. WA allows users to ask medication-related questions in natural language. This study evaluated search method-dependent differences in the frequency of information accessed by traditional methods (keyword search and heading navigation) vs conversational agent search. MATERIALS AND METHODS: We compared the proportion of information types accessed through the conversational agent to the proportion of analogous information types accessed by traditional methods during the first 6 months of 2020. RESULTS: Addition of the conversational agent allowed early adopters to access 22 different information types contained in the 'quick answers' portion of the knowledge base. These information types were accessed 117,550 times with WA during the study period, compared to 33,649,651 times using traditional search methods. The distribution across information types differed by method employed (c2 test, P < .0001). Single drug/dosing, FDA/non-FDA uses, adverse effects, and drug administration emerged as 4 of the top 5 information types accessed by either method. Intravenous compatibility was accessed more frequently using the conversational agent (7.7% vs. 0.6% for traditional methods), whereas dose adjustments were accessed more frequently via traditional methods (4.8% vs. 1.4% for WA). CONCLUSION: In a widely used pharmacologic knowledge base, information accessed through conversational agents versus traditional methods differed. User-centered studies are needed to understand these differences.

Effectiveness of non-pharmaceutical interventions related to social distancing on respiratory viral infectious disease outcomes: A rapid evidence-based review and meta-analysis.

OBJECTIVES: Non-pharmaceutical interventions (e.g. quarantine and isolation) are used to mitigate and control viral infectious disease, but their effectiveness has not been well studied. For COVID-19, disease control efforts will rely on non-pharmaceutical interventions until pharmaceutical interventions become widely available, while non-pharmaceutical interventions will be of continued importance thereafter. METHODS: This rapid evidence-based review provides both qualitative and quantitative analyses of the effectiveness of social distancing non-pharmaceutical interventions on disease outcomes. Literature was retrieved from MEDLINE, Google Scholar, and pre-print databases (BioRxiv.org, MedRxiv.org, and Wellcome Open Research). RESULTS: Twenty-eight studies met inclusion criteria (n = 28). Early, sustained, and combined application of various non-pharmaceutical interventions could mitigate and control primary outbreaks and prevent more severe secondary or tertiary outbreaks. The strategic use of non-pharmaceutical interventions decreased incidence, transmission, and/or mortality across all interventions examined. The pooled attack rates for no non-pharmaceutical intervention, single non-pharmaceutical interventions, and multiple non-pharmaceutical interventions were 42% (95% confidence interval = 30% - 55%), 29% (95% confidence interval = 23% - 36%), and 22% (95% confidence interval = 16% - 29%), respectively. CONCLUSION: Implementation of multiple non-pharmaceutical interventions at key decision points for public health could effectively facilitate disease mitigation and suppression until pharmaceutical interventions become available. Dynamics around R 0 values, the susceptibility of certain high-risk patient groups to infection, and the probability of asymptomatic cases spreading disease should be considered.

Artificial intelligence in oncology: Path to implementation.

In recent years, the field of artificial intelligence (AI) in oncology has grown exponentially. AI solutions have been developed to tackle a variety of cancer-related challenges. Medical institutions, hospital systems, and technology companies are developing AI tools aimed at supporting clinical decision making, increasing access to cancer care, and improving clinical efficiency while delivering safe, high-value oncology care. AI in oncology has demonstrated accurate technical performance in image analysis, predictive analytics, and precision oncology delivery. Yet, adoption of AI tools is not widespread, and the impact of AI on patient outcomes remains uncertain. Major barriers for AI implementation in oncology include biased and heterogeneous data, data management and collection burdens, a lack of standardized research reporting, insufficient clinical validation, workflow and user-design challenges, outdated regulatory and legal frameworks, and dynamic knowledge and data. Concrete actions that major stakeholders can take to overcome barriers to AI implementation in oncology include training and educating the oncology workforce in AI; standardizing data, model validation methods, and legal and safety regulations; funding and conducting future research; and developing, studying, and deploying AI tools through multidisciplinary collaboration.

The Perceived Impact and Usability of a Care Management and Coordination System in Delivering Services to Vulnerable Populations: Mixed Methods Study.

BACKGROUND: People with complex needs, such as those experiencing homelessness, require concurrent, seamless support from multiple social service agencies. Sonoma County, California has one of the nation's largest homeless populations among largely suburban communities. To support client-centered care, the county deployed a Care Management and Coordination System (CMCS). This system comprised the Watson Care Manager (WCM), a front-end system, and Connect 360, which is an integrated data hub that aggregates information from various systems into a single client record. OBJECTIVE: The aim of this study is to evaluate the perceived impact and usability of WCM in delivering services to the homeless population in Sonoma County. METHODS: A mixed methods study was conducted to identify ways in which WCM helps to coordinate care. Interviews, observations, and surveys were conducted, and transcripts and field notes were thematically analyzed and directed by a grounded theory approach. Responses to the Technology Acceptance Model survey were analyzed. RESULTS: A total of 16 participants were interviewed, including WCM users (n=8) and department leadership members (n=8). In total, 3 interdisciplinary team meetings were observed, and 8 WCM users were surveyed. WCM provided a central shared platform where client-related, up-to-date, comprehensive, and reliable information from participating agencies was consolidated. Factors that facilitated WCM use were users' enthusiasm regarding the tool functionalities, scalability, and agency collaboration. Constraining factors included the suboptimal awareness of care delivery goals and functionality of the system among the community, sensitivities about data sharing and legal requirements, and constrained funding from government and nongovernment organizations. Overall, users found WCM to be a useful tool that was easy to use and helped to enhance performance. CONCLUSIONS: WCM supports the delivery of care to individuals with complex needs. Integration of data and information in a CMCS can facilitate coordinated care. Future research should examine WCM and similar CMCSs in diverse populations and settings.

Barriers to Prescribing Targeted Therapies for Patients With NSCLC With Highly Actionable Gene Variants in the Veterans Affairs National Precision Oncology Program.

PURPOSE: Next-generation sequencing (NGS) gene panels are frequently completed for patients with advanced non-small-cell lung cancer (NSCLC). Patients with highly actionable gene variants have improved outcomes and reduced toxicities with the use of corresponding targeted agents. We sought to identify barriers to targeted agent use within the Veterans Health Affairs' National Precision Oncology Program (NPOP). METHODS: A retrospective evaluation of patients with NSCLC who underwent NGS multigene panels through NPOP between July 2015 and February 2019 was conducted. Patients who were assigned level 1 or 2A evidence for oncogenic gene variants by an artificial intelligence offering (IBM Watson for Genomics [WfG]) and NPOP staff were selected. Antineoplastic drug prescriptions and provider notes were reviewed. Reasons for withholding targeted treatments were categorized. RESULTS: Of 1,749 patients with NSCLC who successfully underwent NGS gene panel testing, 112 (6.4%) patients were assigned level 1 and/or 2A evidence for available targeted treatments by WfG and NPOP staff. All highly actionable gene variants were within ALK, BRAF, EGFR, ERBB2, MET, RET, and ROS1. Of these, 36 (32.1%) patients were not prescribed targeted agents. The three most common reasons were (1) patient did not carry a diagnosis of metastatic disease (33.3%), (2) treating provider did not comment on the NGS results (25.0%), and (3) provider felt that patient could not tolerate therapy (19.4%). No patients were denied access to level 1 or 2A targeted drugs because of rejection of a nonformulary drug request. CONCLUSION: A substantial minority of patients with NSCLC bearing highly actionable gene variants are not prescribed targeted agents. Further provider- and pathologist-directed educational efforts and implementation of health informatics systems to provide real-time decision support for test ordering and interpretation are needed.

The burden of the digital environment: a systematic review on organization-directed workplace interventions to mitigate physician burnout.

OBJECTIVE: To conduct a systematic review identifying workplace interventions that mitigate physician burnout related to the digital environment including health information technologies (eg, electronic health records) and decision support systems) with or without the application of advanced analytics for clinical care. MATERIALS AND METHODS: Literature published from January 1, 2007 to June 3, 2020 was systematically reviewed from multiple databases and hand searches. Subgroup analysis identified relevant physician burnout studies with interventions examining digital tool burden, related workflow inefficiencies, and measures of burnout, stress, or job satisfaction in all practice settings. RESULTS: The search strategy identified 4806 citations of which 81 met inclusion criteria. Thirty-eight studies reported interventions to decrease digital tool burden. Sixty-eight percent of these studies reported improvement in burnout and/or its proxy measures. Burnout was decreased by interventions that optimized technologies (primarily electronic health records), provided training, reduced documentation and task time, expanded the care team, and leveraged quality improvement processes in workflows. DISCUSSION: The contribution of digital tools to physician burnout can be mitigated by careful examination of usability, introducing technologies to save or optimize time, and applying quality improvement to workflows. CONCLUSION: Physician burnout is not reduced by technology implementation but can be mitigated by technology and workflow optimization, training, team expansion, and careful consideration of factors affecting burnout, including specialty, practice setting, regulatory pressures, and how physicians spend their time.

Artificial Intelligence Clinical Evidence Engine for Automatic Identification, Prioritization, and Extraction of Relevant Clinical Oncology Research.

PURPOSE: We developed a system to automate analysis of the clinical oncology scientific literature from bibliographic databases and match articles to specific patient cohorts to answer specific questions regarding the efficacy of a treatment. The approach attempts to replicate a clinician's mental processes when reviewing published literature in the context of a patient case. We describe the system and evaluate its performance. METHODS: We developed separate ground truth data sets for each of the tasks described in the paper. The first ground truth was used to measure the natural language processing (NLP) accuracy from approximately 1,300 papers covering approximately 3,100 statements and approximately 25 concepts; performance was evaluated using a standard F1 score. The ground truth for the expert classifier model was generated by dividing papers cited in clinical guidelines into a training set and a test set in an 80:20 ratio, and performance was evaluated for accuracy, sensitivity, and specificity. RESULTS: The NLP models were able to identify individual attributes with a 0.7-0.9 F1 score, depending on the attribute of interest. The expert classifier machine learning model was able to classify the individual records with a 0.93 accuracy (95% CI, 0.9 to 0.96, P < .0001), and sensitivity and specificity of 0.95 and 0.91, respectively. Using a decision boundary of 0.5 for the positive (expert) label, the classifier demonstrated an F1 score of 0.92. CONCLUSION: The system identified and extracted evidence from the oncology literature with a high degree of accuracy, sensitivity, and specificity. This tool enables timely access to the most relevant biomedical literature, providing critical support to evidence-based practice in areas of rapidly evolving science.

Predictive article recommendation using natural language processing and machine learning to support evidence updates in domain-specific knowledge graphs.

OBJECTIVES: Describe an augmented intelligence approach to facilitate the update of evidence for associations in knowledge graphs. METHODS: New publications are filtered through multiple machine learning study classifiers, and filtered publications are combined with articles already included as evidence in the knowledge graph. The corpus is then subjected to named entity recognition, semantic dictionary mapping, term vector space modeling, pairwise similarity, and focal entity match to identify highly related publications. Subject matter experts review recommended articles to assess inclusion in the knowledge graph; discrepancies are resolved by consensus. RESULTS: Study classifiers achieved F-scores from 0.88 to 0.94, and similarity thresholds for each study type were determined by experimentation. Our approach reduces human literature review load by 99%, and over the past 12 months, 41% of recommendations were accepted to update the knowledge graph. CONCLUSION: Integrated search and recommendation exploiting current evidence in a knowledge graph is useful for reducing human cognition load.

A Narrative Review of Emerging Therapeutics for COVID-19.

The novel severe acute respiratory syndrome coronavirus 2, the causal agent of coronavirus disease 2019 (COVID-19), quickly spread around the world, resulting in the most aggressive pandemic experienced in more than 100 years. Research on targeted therapies and vaccines has been initiated on an unprecedented scale and speed but will take months and even years to come to fruition. Meanwhile, the efficacy of emerging therapeutics for use in treating COVID-19 is feverishly being investigated to identify the best available treatment options for dealing with the current wave of disease. This review of publications with a "treatment" tag through June 29, 2020 in the National Library of Medicine's LitCovid literature hub, provides frontline clinicians with a pragmatic summary of the current state of the rapidly evolving evidence supporting emerging candidate therapeutics for COVID-19. Two main categories of pharmaceutical therapeutics are showing promise: those with antiviral activity directly addressing infection and those that counteract the inflammatory cytokine storm induced by severe disease. Preliminary results suggest that other approaches such as convalescent plasma therapy and lung radiation therapy may have some efficacy. The current clinical evidence for potential treatments is preliminary-often small retrospective series or early results of randomized trials-and the science is evolving rapidly. The long-term results from large, well-designed randomized controlled trials will provide definitive evidence for therapeutic effectiveness and are likely months away. The trial landscape for promising therapies is described.