Development and application of Breadth-Depth-Context (BDC), a conceptual framework for measuring technology engagement with a qualified clinical data registry.

Objectives: Despite the proliferation of dashboards that display performance data derived from Qualified Clinical Data Registries (QCDR), the degree to which clinicians and practices engage with such dashboards has not been well described. We aimed to develop a conceptual framework for assessing user engagement with dashboard technology and to demonstrate its application to a rheumatology QCDR. Materials and Methods: We developed the BDC (Breadth-Depth-Context) framework, which included concepts of breadth (derived from dashboard sessions), depth (derived from dashboard actions), and context (derived from practice characteristics). We demonstrated its application via user log data from the American College of Rheumatology's Rheumatology Informatics System for Effectiveness (RISE) registry to define engagement profiles and characterize practice-level factors associated with different profiles. Results: We applied the BDC framework to 213 ambulatory practices from the RISE registry in 2020-2021, and classified practices into 4 engagement profiles: not engaged (8%), minimally engaged (39%), moderately engaged (34%), and most engaged (19%). Practices with more patients and with specific electronic health record vendors (eClinicalWorks and eMDs) had a higher likelihood of being in the most engaged group, even after adjusting for other factors. Discussion: We developed the BDC framework to characterize user engagement with a registry dashboard and demonstrated its use in a specialty QCDR. The application of the BDC framework revealed a wide range of breadth and depth of use and that specific contextual factors were associated with nature of engagement. Conclusion: Going forward, the BDC framework can be used to study engagement with similar dashboards.

Structuring medication signeturs as a language regression task: comparison of zero- and few-shot GPT with fine-tuned models.

Importance: Electronic health record textual sources such as medication signeturs (sigs) contain valuable information that is not always available in structured form. Commonly processed through manual annotation, this repetitive and time-consuming task could be fully automated using large language models (LLMs). While most sigs include simple instructions, some include complex patterns. Objectives: We aimed to compare the performance of GPT-3.5 and GPT-4 with smaller fine-tuned models (ClinicalBERT, BlueBERT) in extracting the average daily dose of 2 immunomodulating medications with frequent complex sigs: hydroxychloroquine, and prednisone. Methods: Using manually annotated sigs as the gold standard, we compared the performance of these models in 702 hydroxychloroquine and 22 104 prednisone prescriptions. Results: GPT-4 vastly outperformed all other models for this task at any level of in-context learning. With 100 in-context examples, the model correctly annotates 94% of hydroxychloroquine and 95% of prednisone sigs to within 1 significant digit. Error analysis conducted by 2 additional manual annotators on annotator-model disagreements suggests that the vast majority of disagreements are model errors. Many model errors relate to ambiguous sigs on which there was also frequent annotator disagreement. Discussion: Paired with minimal manual annotation, GPT-4 achieved excellent performance for language regression of complex medication sigs and vastly outperforms GPT-3.5, ClinicalBERT, and BlueBERT. However, the number of in-context examples needed to reach maximum performance was similar to GPT-3.5. Conclusion: LLMs show great potential to rapidly extract structured data from sigs in no-code fashion for clinical and research applications.

Monitoring and Achievement of Target Serum Urate Among Gout Patients Receiving Long-Term Urate-Lowering Therapy in the American College of Rheumatology RISE Registry.

OBJECTIVE: The American College of Rheumatology's (ACR) 2020 guidelines for the management of gout recommend using a treat-to-target approach to lower serum urate (SU). Using the ACR's Rheumatology Informatics System for Effectiveness registry, we examined the use of a treat-to-target approach among gout patients receiving long-term urate-lowering therapy (ULT) and followed longitudinally by rheumatologists. METHODS: Included patients had one or more diagnoses for gout in 2018-2019 and continuous use of ULT for ≥12 months. We assessed the proportions of patients with SU monitoring and, among those tested, who achieved SU <6.0 mg/dl during the measurement year. Multilevel logistic regression adjusting for sociodemographics, comorbidities, region, and health care utilization was used to determine factors associated with SU monitoring and achievement of target SU. RESULTS: A total of 9,560 were included. The mean ± SD age was 67.2 ± 12.7 years, 73.5% of patients were male, and 32.3% were non-White. Fifty-six percent of patients had at least 1 SU recorded during the measurement year; among patients with at least 1 SU recorded, 74% achieved the SU target. In multivariate analyses, non-White patients were slightly less likely to be tested or achieve a target SU. CONCLUSION: Among gout patients receiving long-term ULT followed longitudinally by rheumatologists, more than half had a documented SU, and among those tested, three-quarters achieved the recommended SU target. Routine monitoring of SU is a first step toward improving quality of care for patients with gout.

Treatment of Sarcoidosis in US Rheumatology Practices: Data From the American College of Rheumatology's Rheumatology Informatics System for Effectiveness (RISE) Registry.

OBJECTIVE: Sarcoidosis is often treated with glucocorticoids, although the use of biologics is growing. Prescribing patterns for biologics for patients with sarcoidosis in US rheumatology practices have never been examined. Given that there are no steroid-sparing US Food and Drug Administration-approved therapies for sarcoidosis, we sought to characterize the real-world treatment of sarcoidosis and to assess practice-level variation in prescribing patterns. METHODS: We conducted an observational study of patients with sarcoidosis using data from the Rheumatology Informatics System for Effectiveness (RISE) registry (2014-2018). The RISE registry represents an estimated 32% of the US clinical rheumatology workforce. Adult patients with ≥2 codes for sarcoidosis ≥30 days apart were included. We examined sarcoidosis-specific medication use at any time during the study period. Data were analyzed at the practice level. RESULTS: A total of 3,276 patients with sarcoidosis from 184 practices were included. Of those patients, 75.1% were women, with a mean age of 59.0 ± 12.5 years; 48.3% were White and 27.6% were Black. Overall, 59.3% of patients were prescribed glucocorticoids, and 24.7% received prolonged glucocorticoid therapy (≥10 mg/day for ≥90 days). In all, 12.1% received a biologic or targeted synthetic disease-modifying antirheumatic drug (tsDMARD), most commonly tumor necrosis factor inhibitors. There was wide practice-level variation among 31 practices with ≥30 patients with sarcoidosis; biologic use ranged from 15.6% to 69.2%. Infliximab represented the most common biologic prescribed. CONCLUSION: In a large sample of US rheumatology practices, 12.1% of patients with sarcoidosis received biologics or tsDMARDs. We found high variability in biologic use across practices. The significant use of long-term glucocorticoids suggests unmet therapeutic needs in this patient population.

Quality of Care for Patients With Systemic Lupus Erythematosus: Data From the American College of Rheumatology RISE Registry.

OBJECTIVE: Although multiple national quality measures focus on the management and safety of rheumatoid arthritis, few measures address the care of patients with systemic lupus erythematosus (SLE). Our objective was to apply a group of quality measures relevant to the care of patients with SLE, and we used the American College of Rheumatology's Rheumatology Informatics System for Effectiveness (RISE) registry to assess nationwide variations in care. METHODS: The data derived from RISE and included patients who had ≥2 visits with SLE codes ≥30 days apart in 2017-2018. We calculated performance on 5 quality measures: renal disease screening, blood pressure assessment and management, hydroxychloroquine (HCQ) prescribing, safe dosing for HCQ, and prolonged glucocorticoid use at doses of >7.5 mg/day. We reported performance on these measures at the practice level. We used logistic regression to assess independent predictors of performance after adjusting for sociodemographic and utilization factors. RESULTS: We included 27,567 unique patients from 186 practices; 91.7% were female and 48% White, with a mean age of 53.5 ± 15.2 years. Few patients had adequate screening for the development of renal manifestations (39.5%). Although blood pressure assessment was common (94.4%), a meaningful fraction of patients had untreated hypertension (17.7%). Many received HCQ (71.5%), but only 62% at doses of ≤5.0 mg/kg/day. Some received at least moderate-dose steroids for ≥90 days (18.5%). We observed significant practice variation on every measure. CONCLUSION: We found potential gaps in care for patients with SLE across the US. Although some performance variation may be explained by differences in disease severity, dramatic differences suggest that developing quality measures to address important health care processes in SLE may improve care.

The Relationship Between Electronic Health Record System and Performance on Quality Measures in the American College of Rheumatology's Rheumatology Informatics System for Effectiveness (RISE) Registry: Observational Study.

BACKGROUND: Routine collection of disease activity (DA) and patient-reported outcomes (PROs) in rheumatoid arthritis (RA) are nationally endorsed quality measures and critical components of a treat-to-target approach. However, little is known about the role electronic health record (EHR) systems play in facilitating performance on these measures. OBJECTIVE: Using the American College Rheumatology's (ACR's) RISE registry, we analyzed the relationship between EHR system and performance on DA and functional status (FS) quality measures. METHODS: We analyzed data collected in 2018 from practices enrolled in RISE. We assessed practice-level performance on quality measures that require DA and FS documentation. Multivariable linear regression and zero-inflated negative binomial models were used to examine the independent effect of EHR system on practice-level quality measure performance, adjusting for practice characteristics and patient case-mix. RESULTS: In total, 220 included practices cared for 314,793 patients with RA. NextGen was the most commonly used EHR system (34.1%). We found wide variation in performance on DA and FS quality measures by EHR system (median 30.1, IQR 0-74.8, and median 9.0, IQR 0-74.2), respectively). Even after adjustment, NextGen practices performed significantly better than Allscripts on the DA measure (51.4% vs 5.0%; P<.05) and significantly better than eClinicalWorks and eMDs on the FS measure (49.3% vs 29.0% and 10.9%; P<.05). CONCLUSIONS: Performance on national RA quality measures was associated with the EHR system, even after adjusting for practice and patient characteristics. These findings suggest that future efforts to improve quality of care in RA should focus not only on provider performance reporting but also on developing and implementing rheumatology-specific standards across EHRs.

Epidemiology and treatment of Behçet's disease in the USA: insights from the Rheumatology Informatics System for Effectiveness (RISE) Registry with a comparison with other published cohorts from endemic regions.

BACKGROUND: Behçet's disease (BD), a chronic systemic vasculitis, has distinct geographical and ethnic variation. Data regarding the epidemiology of patients with BD in the U.S. are limited; therefore, we sought to describe BD patient characteristics and medication use in the U.S., and compared them with data from patients from endemic regions. METHODS: We conducted a cross-sectional study using data from the RISE registry (2014-2018). Patients aged ≥ 18 years with BD were included. Sociodemographic and treatment information was extracted. We compared patients from the RISE registry to data from other published studies of patients with BD from endemic areas. RESULTS: One thousand three hundred twenty-three subjects with BD from the RISE registry were included. Mean age was 48.7 ± 16.3 years, female to male ratio was 3.8:1, and 66.7% were White. The most frequently used medications included glucocorticoids (67.6%) and colchicine (55.0%). Infliximab and adalimumab were the most used biologics (14.5% and 14.1%, respectively); 3.2% of patients used apremilast. The RISE registry had more women (79.3%), and patients were older compared to previously published BD studies from endemic areas. Methotrexate and TNFi were more commonly reported in RISE (21.8% and 29.4%) compared to studies from Egypt and Turkey. Colchicine, cyclosporine, and cyclophosphamide were more commonly used in cohorts from Egypt, Turkey, and Iran. CONCLUSIONS: Findings from the largest BD dataset in the U.S. suggest that BD patients are predominantly female. Further research is needed to explore the reasons for the higher prevalence of BD among women in the U.S. and its possible impact on disease severity and management.

RISE registry reveals potential gaps in medication safety for new users of biologics and targeted synthetic DMARDs.

OBJECTIVE: Immunosuppressant drugs can increase the risk of hepatitis B virus (HBV) and hepatitis C virus (HCV) and tuberculosis (TB) reactivation. Using the American College of Rheumatology's Rheumatology Informatics System for Effectiveness (RISE) registry, we examined pre-treatment screening among new users of biologic or targeted synthetic disease modifying drugs (DMARDs). METHODS: Data, derived from RISE, included patients ≥ 18 years old who were new users of biologic or targeted synthetic DMARDs. We developed quality measures related to pre-treatment screening for HBV, HCV, and TB in addition to a "composite" measure for all applicable tests. We assessed patient-level screening rates, practice-level variation among practices reporting on ≥ 20 patients, and the frequency of positive results. RESULTS: We included 26,802 patients across 213 rheumatology practices nationwide. Patients were 58 (14) years old, 75.9% female; 59.6% had rheumatoid arthritis, and TNFi were the most common index DMARDs (64.9%). Overall, 44.8% and 40.5% patients had any documented HBV or HCV screening, respectively, prior to the index date; 29.7% had TB screening in the year prior to drug start. Only 15.5% had documentation of screening for all appropriate infections prior to drug start. Practice-level performance on the composite measure was low (range 0 to 48.3%). 2.4% of screening tests were positive. CONCLUSION: We found gaps in documentation of key safety measures among practices participating in RISE. Given the small but significant number of patients with active or latent infections that pose safety risks, developing standardized and reliable strategies to capture safety screening measures is paramount.