Evaluating Factors Associated With Telehealth Appropriateness in Outpatient Rheumatoid Arthritis Encounters Using the Encounter Appropriateness Score for You (EASY).

OBJECTIVE: Telehealth has been proposed as a safe and effective alternative to in-person care for rheumatoid arthritis (RA). The purpose of this study was to evaluate factors associated with telehealth appropriateness in outpatient RA encounters. METHODS: A prospective cohort study (January 1, 2021, to August 31, 2021) was conducted using electronic health record data from outpatient RA encounters in a single academic rheumatology practice. Rheumatology providers rated the telehealth appropriateness of their own encounters using the Encounter Appropriateness Score for You (EASY) immediately following each encounter. Robust Poisson regression with generalized estimating equations modeling was used to evaluate the association of telehealth appropriateness with patient demographics, RA clinical characteristics, comorbid noninflammatory causes of joint pain, previous and current encounter characteristics, and provider characteristics. RESULTS: During the study period, 1823 outpatient encounters with 1177 unique patients with RA received an EASY score from 25 rheumatology providers. In the final multivariate model, factors associated with increased telehealth appropriateness included higher average provider preference for telehealth in prior encounters (relative risk [RR] 1.26, 95% CI 1.21-1.31), telehealth as the current encounter modality (RR 2.27, 95% CI 1.95-2.64), and increased patient age (RR 1.05, 95% CI 1.01-1.09). Factors associated with decreased telehealth appropriateness included moderate (RR 0.81, 95% CI 0.68-0.96) and high (RR 0.57, 95% CI 0.46-0.70) RA disease activity and if the previous encounters were conducted by telehealth (RR 0.83, 95% CI 0.73-0.95). CONCLUSION: In this study, telehealth appropriateness was most associated with provider preference, the current and previous encounter modality, and RA disease activity. Other factors like patient demographics, RA medications, and comorbid noninflammatory causes of joint pain were not associated with telehealth appropriateness.

Use of artificial intelligence in critical care: opportunities and obstacles.

BACKGROUND: Perhaps nowhere else in the healthcare system than in the intensive care unit environment are the challenges to create useful models with direct time-critical clinical applications more relevant and the obstacles to achieving those goals more massive. Machine learning-based artificial intelligence (AI) techniques to define states and predict future events are commonplace activities of modern life. However, their penetration into acute care medicine has been slow, stuttering and uneven. Major obstacles to widespread effective application of AI approaches to the real-time care of the critically ill patient exist and need to be addressed. MAIN BODY: Clinical decision support systems (CDSSs) in acute and critical care environments support clinicians, not replace them at the bedside. As will be discussed in this review, the reasons are many and include the immaturity of AI-based systems to have situational awareness, the fundamental bias in many large databases that do not reflect the target population of patient being treated making fairness an important issue to address and technical barriers to the timely access to valid data and its display in a fashion useful for clinical workflow. The inherent "black-box" nature of many predictive algorithms and CDSS makes trustworthiness and acceptance by the medical community difficult. Logistically, collating and curating in real-time multidimensional data streams of various sources needed to inform the algorithms and ultimately display relevant clinical decisions support format that adapt to individual patient responses and signatures represent the efferent limb of these systems and is often ignored during initial validation efforts. Similarly, legal and commercial barriers to the access to many existing clinical databases limit studies to address fairness and generalizability of predictive models and management tools. CONCLUSIONS: AI-based CDSS are evolving and are here to stay. It is our obligation to be good shepherds of their use and further development.

Evaluating Provider Perceptions of Telehealth Utility in Outpatient Rheumatology Telehealth Encounters.

OBJECTIVE: This study aims to explore the factors associated with rheumatology providers' perceptions of telehealth utility in real-world telehealth encounters. METHODS: From September 14, 2020 to January 31, 2021, 6 providers at an academic medical center rated their telehealth visits according to perceived utility in making treatment decisions using the following Telehealth Utility Score (TUS) (1 = very low utility to 5 = very high utility). Modified Poisson regression models were used to assess the association between TUS scores and encounter diagnoses, disease activity measures, and immunomodulatory therapy changes during the encounter. RESULTS: A total of 481 telehealth encounters were examined, of which 191 (39.7%) were rated as "low telehealth utility" (TUS 1-3) and 290 (60.3%) were rated as "high telehealth utility" (TUS 4-5). Encounters with a diagnosis of inflammatory arthritis were significantly less likely to be rated as high telehealth utility (adjusted relative risk [aRR], 0.8061; p = 0.004), especially in those with a concurrent noninflammatory musculoskeletal diagnosis (aRR, 0.54; p = 0.006). Other factors significantly associated with low telehealth utility included higher disease activity according to current and prior RAPID3 scores (aRR, 0.87 and aRR, 0.89, respectively; p < 0.001) and provider global scores (aRR, 0.83; p < 0.001), as well as an increase in immunomodulatory therapy (aRR, 0.70; p = 0.015). CONCLUSIONS: Provider perceptions of telehealth utility in real-world encounters are significantly associated with patient diagnoses, current and prior disease activity, and the need for changes in immunomodulatory therapy. These findings inform efforts to optimize the appropriate utilization of telehealth in rheumatology.

Impact of the COVID-19 pandemic on patterns of outpatient cardiovascular care.

BACKGROUND: The coronavirus disease 2019 (COVID-19) pandemic brought about abrupt changes in the way health care is delivered, and the impact of transitioning outpatient clinic visits to telehealth visits on processes of care and outcomes is unclear. METHODS: We evaluated ordering patterns during cardiovascular telehealth clinic visits in the Duke University Health System between March 15 and June 30, 2020 and 30-day outcomes compared with in-person visits in the same time frame in 2020 and in 2019. RESULTS: Within the Duke University Health System, there was a 33.1% decrease in the number of outpatient cardiovascular visits conducted in the first 15 weeks of the COVID-19 pandemic, compared with the same time period in 2019. As a proportion of total visits initially booked, 53% of visits were cancelled in 2020 compared to 35% in 2019. However, patients with cancelled visits had similar demographics and comorbidities in 2019 and 2020. Telehealth visits comprised 9.3% of total visits initially booked in 2020, with younger and healthier patients utilizing telehealth compared with those utilizing in-person visits. Compared with in-person visits in 2020, telehealth visits were associated with fewer new (31.6% for telehealth vs 44.6% for in person) or refill (12.9% vs 15.6%, respectively) medication prescriptions, electrocardiograms (4.3% vs 31.4%), laboratory orders (5.9% vs 21.8%), echocardiograms (7.3% vs 98%), and stress tests (4.4% vs 6.6%). When adjusted for age, race, and insurance status, those who had a telehealth visit or cancelled their visit were less likely to have an emergency department or hospital encounter within 30 days compared with those who had in-person visits (adjusted rate ratios (aRR) 0.76 [95% 0.65, 0.89] and aRR 0.71 [95% 0.65, 0.78], respectively). CONCLUSIONS: In response to the perceived risks of routine medical care affected by the COVID-19 pandemic, different phenotypes of patients chose different types of outpatient cardiology care. A better understanding of these differences could help define necessary and appropriate mode of care for cardiology patients.

BOLD: Blood-gas and Oximetry Linked Dataset.

Pulse oximeters measure peripheral arterial oxygen saturation (SpO2) noninvasively, while the gold standard (SaO2) involves arterial blood gas measurement. There are known racial and ethnic disparities in their performance. BOLD is a dataset that aims to underscore the importance of addressing biases in pulse oximetry accuracy, which disproportionately affect darker-skinned patients. The dataset was created by harmonizing three Electronic Health Record databases (MIMIC-III, MIMIC-IV, eICU-CRD) comprising Intensive Care Unit stays of US patients. Paired SpO2 and SaO2 measurements were time-aligned and combined with various other sociodemographic and parameters to provide a detailed representation of each patient. BOLD includes 49,099 paired measurements, within a 5-minute window and with oxygen saturation levels between 70-100%. Minority racial and ethnic groups account for ~25% of the data - a proportion seldom achieved in previous studies. The codebase is publicly available. Given the prevalent use of pulse oximeters in the hospital and at home, we hope that BOLD will be leveraged to develop debiasing algorithms that can result in more equitable healthcare solutions.

Translating ethical and quality principles for the effective, safe and fair development, deployment and use of artificial intelligence technologies in healthcare.

OBJECTIVE: The complexity and rapid pace of development of algorithmic technologies pose challenges for their regulation and oversight in healthcare settings. We sought to improve our institution's approach to evaluation and governance of algorithmic technologies used in clinical care and operations by creating an Implementation Guide that standardizes evaluation criteria so that local oversight is performed in an objective fashion. MATERIALS AND METHODS: Building on a framework that applies key ethical and quality principles (clinical value and safety, fairness and equity, usability and adoption, transparency and accountability, and regulatory compliance), we created concrete guidelines for evaluating algorithmic technologies at our institution. RESULTS: An Implementation Guide articulates evaluation criteria used during review of algorithmic technologies and details what evidence supports the implementation of ethical and quality principles for trustworthy health AI. Application of the processes described in the Implementation Guide can lead to algorithms that are safer as well as more effective, fair, and equitable upon implementation, as illustrated through 4 examples of technologies at different phases of the algorithmic lifecycle that underwent evaluation at our academic medical center. DISCUSSION: By providing clear descriptions/definitions of evaluation criteria and embedding them within standardized processes, we streamlined oversight processes and educated communities using and developing algorithmic technologies within our institution. CONCLUSIONS: We developed a scalable, adaptable framework for translating principles into evaluation criteria and specific requirements that support trustworthy implementation of algorithmic technologies in patient care and healthcare operations.

Predictive Modeling to Identify Children With Complex Health Needs At Risk for Hospitalization.

BACKGROUND: Identifying children at high risk with complex health needs (CCHN) who have intersecting medical and social needs is challenging. This study's objectives were to (1) develop and evaluate an electronic health record (EHR)-based clinical predictive model ("model") for identifying high-risk CCHN and (2) compare the model's performance as a clinical decision support (CDS) to other CDS tools available for identifying high-risk CCHN. METHODS: This retrospective cohort study included children aged 0 to 20 years with established care within a single health system. The model development/validation cohort included 33 months (January 1, 2016-September 30, 2018) and the testing cohort included 18 months (October 1, 2018-March 31, 2020) of EHR data. Machine learning methods generated a model that predicted probability (0%-100%) for hospitalization within 6 months. Model performance measures included sensitivity, positive predictive value, area under receiver-operator curve, and area under precision-recall curve. Three CDS rules for identifying high-risk CCHN were compared: (1) hospitalization probability ≥10% (model-predicted); (2) complex chronic disease classification (using Pediatric Medical Complexity Algorithm [PMCA]); and (3) previous high hospital utilization. RESULTS: Model development and testing cohorts included 116 799 and 27 087 patients, respectively. The model demonstrated area under receiver-operator curve = 0.79 and area under precision-recall curve = 0.13. PMCA had the highest sensitivity (52.4%) and classified the most children as high risk (17.3%). Positive predictive value of the model-based CDS rule (19%) was higher than CDS based on the PMCA (1.9%) and previous hospital utilization (15%). CONCLUSIONS: A novel EHR-based predictive model was developed and validated as a population-level CDS tool for identifying CCHN at high risk for future hospitalization.

BOLD: Blood-gas and Oximetry Linked Dataset - Open Source Research.

Pulse oximeters measure peripheral arterial oxygen saturation (SpO 2 ) noninvasively, while the gold standard (SaO 2 ) involves arterial blood gas measurement. There are known racial and ethnic disparities in their performance. BOLD is a new comprehensive dataset that aims to underscore the importance of addressing biases in pulse oximetry accuracy, which disproportionately affect darker-skinned patients. The dataset was created by harmonizing three Electronic Health Record databases (MIMIC-III, MIMIC-IV, eICU-CRD) comprising Intensive Care Unit stays of US patients. Paired SpO 2 and SaO 2 measurements were time-aligned and combined with various other sociodemographic and parameters to provide a detailed representation of each patient. BOLD includes 49,099 paired measurements, within a 5-minute window and with oxygen saturation levels between 70-100%. Minority racial and ethnic groups account for ∼25% of the data - a proportion seldom achieved in previous studies. The codebase is publicly available. Given the prevalent use of pulse oximeters in the hospital and at home, we hope that BOLD will be leveraged to develop debiasing algorithms that can result in more equitable healthcare solutions.

Mitigating Racial And Ethnic Bias And Advancing Health Equity In Clinical Algorithms: A Scoping Review.

In August 2022 the Department of Health and Human Services (HHS) issued a notice of proposed rulemaking prohibiting covered entities, which include health care providers and health plans, from discriminating against individuals when using clinical algorithms in decision making. However, HHS did not provide specific guidelines on how covered entities should prevent discrimination. We conducted a scoping review of literature published during the period 2011-22 to identify health care applications, frameworks, reviews and perspectives, and assessment tools that identify and mitigate bias in clinical algorithms, with a specific focus on racial and ethnic bias. Our scoping review encompassed 109 articles comprising 45 empirical health care applications that included tools tested in health care settings, 16 frameworks, and 48 reviews and perspectives. We identified a wide range of technical, operational, and systemwide bias mitigation strategies for clinical algorithms, but there was no consensus in the literature on a single best practice that covered entities could employ to meet the HHS requirements. Future research should identify optimal bias mitigation methods for various scenarios, depending on factors such as patient population, clinical setting, algorithm design, and types of bias to be addressed.

Facilitating Harmonization of Variables in Framingham, MESA, ARIC, and REGARDS Studies Through a Metadata Repository.

BACKGROUND: High-quality research in cardiovascular prevention, as in other fields, requires inclusion of a broad range of data sets from different sources. Integrating and harmonizing different data sources are essential to increase generalizability, sample size, and representation of understudied populations-strengthening the evidence for the scientific questions being addressed. METHODS: Here, we describe an effort to build an open-access repository and interactive online portal for researchers to access the metadata and code harmonizing data from 4 well-known cohort studies-the REGARDS (Reasons for Geographic and Racial Differences in Stroke) study, FHS (Framingham Heart Study), MESA (Multi-Ethnic Study of Atherosclerosis), and ARIC (Atherosclerosis Risk in Communities) study. We introduce a methodology and a framework used for preprocessing and harmonizing variables from multiple studies. RESULTS: We provide a real-case study and step-by-step guidance to demonstrate the practical utility of our repository and interactive web page. In addition to our successful development of such an open-access repository and interactive web page, this exercise in harmonizing data from multiple cohort studies has revealed several key themes. These themes include the importance of careful preprocessing and harmonization of variables, the value of creating an open-access repository to facilitate collaboration and reproducibility, and the potential for using harmonized data to address important scientific questions and disparities in cardiovascular disease research. CONCLUSIONS: By integrating and harmonizing these large-scale cohort studies, such a repository may improve the statistical power and representation of understudied cohorts, enabling development and validation of risk prediction models, identification and investigation of risk factors, and creating a platform for racial disparities research. REGISTRATION: URL: https://precision.heart.org/duke-ninds.

A framework for the oversight and local deployment of safe and high-quality prediction models.

Artificial intelligence/machine learning models are being rapidly developed and used in clinical practice. However, many models are deployed without a clear understanding of clinical or operational impact and frequently lack monitoring plans that can detect potential safety signals. There is a lack of consensus in establishing governance to deploy, pilot, and monitor algorithms within operational healthcare delivery workflows. Here, we describe a governance framework that combines current regulatory best practices and lifecycle management of predictive models being used for clinical care. Since January 2021, we have successfully added models to our governance portfolio and are currently managing 52 models.

Telehealth Made EASY: Understanding Provider Perceptions of Telehealth Appropriateness in Outpatient Rheumatology Encounters.

OBJECTIVE: The purpose of this study was to evaluate a novel scoring system, the Encounter Appropriateness Score for You (EASY), to assess provider perceptions of telehealth appropriateness in rheumatology encounters. METHODS: The EASY scoring system prompts providers to rate their own encounters as follows: in-person or telehealth acceptable, EASY = 1; in-person preferred, EASY = 2; or telehealth preferred, EASY = 3. Assessment of the EASY scoring system occurred at a single academic institution from January 1, 2021, to August 31, 2021. Data were collected in three rounds: 1) initial survey (31 providers) assessing EASY responsiveness to five hypothetical scenarios, 2) follow-up survey (34 providers) exploring EASY responsiveness to 11 scenario modifications, and 3) assessment of EASYs documented in clinic care. RESULTS: The initial and follow-up surveys demonstrated responsiveness of EASYs to different clinical and nonclinical factors. For instance, less than 20% of providers accepted telehealth when starting a biologic for active rheumatoid arthritis, although more than 35% accepted telehealth in the same scenario if the patient lived far away or was well known to the provider. Regarding EASY documentation, 27 providers provided EASYs for 12,381 encounters. According to these scores, telehealth was acceptable or preferred for 29.7% of all encounters, including 21.4% of in-person encounters. Conversely, 24.4% of telehealth encounters were scored as in-person preferred. CONCLUSION: EASY is simple, understandable, and responsive to changes in the clinical scenario. We have successfully accumulated 12,381 EASYs that can be studied in future work to better understand telehealth utility and optimize telehealth triage.

Artificial Intelligence Can Be Regulated Using Current Patient Safety Procedures and Infrastructure in Hospitals.

This Viewpoint describes the potential benefits and harms of using artificial intelligence (AI) in health care decision-making processes.