Structuring medication signeturs as a language regression task: comparison of zero- and few-shot GPT with fine-tuned models.

Importance: Electronic health record textual sources such as medication signeturs (sigs) contain valuable information that is not always available in structured form. Commonly processed through manual annotation, this repetitive and time-consuming task could be fully automated using large language models (LLMs). While most sigs include simple instructions, some include complex patterns. Objectives: We aimed to compare the performance of GPT-3.5 and GPT-4 with smaller fine-tuned models (ClinicalBERT, BlueBERT) in extracting the average daily dose of 2 immunomodulating medications with frequent complex sigs: hydroxychloroquine, and prednisone. Methods: Using manually annotated sigs as the gold standard, we compared the performance of these models in 702 hydroxychloroquine and 22 104 prednisone prescriptions. Results: GPT-4 vastly outperformed all other models for this task at any level of in-context learning. With 100 in-context examples, the model correctly annotates 94% of hydroxychloroquine and 95% of prednisone sigs to within 1 significant digit. Error analysis conducted by 2 additional manual annotators on annotator-model disagreements suggests that the vast majority of disagreements are model errors. Many model errors relate to ambiguous sigs on which there was also frequent annotator disagreement. Discussion: Paired with minimal manual annotation, GPT-4 achieved excellent performance for language regression of complex medication sigs and vastly outperforms GPT-3.5, ClinicalBERT, and BlueBERT. However, the number of in-context examples needed to reach maximum performance was similar to GPT-3.5. Conclusion: LLMs show great potential to rapidly extract structured data from sigs in no-code fashion for clinical and research applications.

Treatment of Sarcoidosis in US Rheumatology Practices: Data From the American College of Rheumatology's Rheumatology Informatics System for Effectiveness (RISE) Registry.

OBJECTIVE: Sarcoidosis is often treated with glucocorticoids, although the use of biologics is growing. Prescribing patterns for biologics for patients with sarcoidosis in US rheumatology practices have never been examined. Given that there are no steroid-sparing US Food and Drug Administration-approved therapies for sarcoidosis, we sought to characterize the real-world treatment of sarcoidosis and to assess practice-level variation in prescribing patterns. METHODS: We conducted an observational study of patients with sarcoidosis using data from the Rheumatology Informatics System for Effectiveness (RISE) registry (2014-2018). The RISE registry represents an estimated 32% of the US clinical rheumatology workforce. Adult patients with ≥2 codes for sarcoidosis ≥30 days apart were included. We examined sarcoidosis-specific medication use at any time during the study period. Data were analyzed at the practice level. RESULTS: A total of 3,276 patients with sarcoidosis from 184 practices were included. Of those patients, 75.1% were women, with a mean age of 59.0 ± 12.5 years; 48.3% were White and 27.6% were Black. Overall, 59.3% of patients were prescribed glucocorticoids, and 24.7% received prolonged glucocorticoid therapy (≥10 mg/day for ≥90 days). In all, 12.1% received a biologic or targeted synthetic disease-modifying antirheumatic drug (tsDMARD), most commonly tumor necrosis factor inhibitors. There was wide practice-level variation among 31 practices with ≥30 patients with sarcoidosis; biologic use ranged from 15.6% to 69.2%. Infliximab represented the most common biologic prescribed. CONCLUSION: In a large sample of US rheumatology practices, 12.1% of patients with sarcoidosis received biologics or tsDMARDs. We found high variability in biologic use across practices. The significant use of long-term glucocorticoids suggests unmet therapeutic needs in this patient population.

Quality of Care for Patients With Systemic Lupus Erythematosus: Data From the American College of Rheumatology RISE Registry.

OBJECTIVE: Although multiple national quality measures focus on the management and safety of rheumatoid arthritis, few measures address the care of patients with systemic lupus erythematosus (SLE). Our objective was to apply a group of quality measures relevant to the care of patients with SLE, and we used the American College of Rheumatology's Rheumatology Informatics System for Effectiveness (RISE) registry to assess nationwide variations in care. METHODS: The data derived from RISE and included patients who had ≥2 visits with SLE codes ≥30 days apart in 2017-2018. We calculated performance on 5 quality measures: renal disease screening, blood pressure assessment and management, hydroxychloroquine (HCQ) prescribing, safe dosing for HCQ, and prolonged glucocorticoid use at doses of >7.5 mg/day. We reported performance on these measures at the practice level. We used logistic regression to assess independent predictors of performance after adjusting for sociodemographic and utilization factors. RESULTS: We included 27,567 unique patients from 186 practices; 91.7% were female and 48% White, with a mean age of 53.5 ± 15.2 years. Few patients had adequate screening for the development of renal manifestations (39.5%). Although blood pressure assessment was common (94.4%), a meaningful fraction of patients had untreated hypertension (17.7%). Many received HCQ (71.5%), but only 62% at doses of ≤5.0 mg/kg/day. Some received at least moderate-dose steroids for ≥90 days (18.5%). We observed significant practice variation on every measure. CONCLUSION: We found potential gaps in care for patients with SLE across the US. Although some performance variation may be explained by differences in disease severity, dramatic differences suggest that developing quality measures to address important health care processes in SLE may improve care.

Epidemiology and treatment of Behçet's disease in the USA: insights from the Rheumatology Informatics System for Effectiveness (RISE) Registry with a comparison with other published cohorts from endemic regions.

BACKGROUND: Behçet's disease (BD), a chronic systemic vasculitis, has distinct geographical and ethnic variation. Data regarding the epidemiology of patients with BD in the U.S. are limited; therefore, we sought to describe BD patient characteristics and medication use in the U.S., and compared them with data from patients from endemic regions. METHODS: We conducted a cross-sectional study using data from the RISE registry (2014-2018). Patients aged ≥ 18 years with BD were included. Sociodemographic and treatment information was extracted. We compared patients from the RISE registry to data from other published studies of patients with BD from endemic areas. RESULTS: One thousand three hundred twenty-three subjects with BD from the RISE registry were included. Mean age was 48.7 ± 16.3 years, female to male ratio was 3.8:1, and 66.7% were White. The most frequently used medications included glucocorticoids (67.6%) and colchicine (55.0%). Infliximab and adalimumab were the most used biologics (14.5% and 14.1%, respectively); 3.2% of patients used apremilast. The RISE registry had more women (79.3%), and patients were older compared to previously published BD studies from endemic areas. Methotrexate and TNFi were more commonly reported in RISE (21.8% and 29.4%) compared to studies from Egypt and Turkey. Colchicine, cyclosporine, and cyclophosphamide were more commonly used in cohorts from Egypt, Turkey, and Iran. CONCLUSIONS: Findings from the largest BD dataset in the U.S. suggest that BD patients are predominantly female. Further research is needed to explore the reasons for the higher prevalence of BD among women in the U.S. and its possible impact on disease severity and management.

RISE registry reveals potential gaps in medication safety for new users of biologics and targeted synthetic DMARDs.

OBJECTIVE: Immunosuppressant drugs can increase the risk of hepatitis B virus (HBV) and hepatitis C virus (HCV) and tuberculosis (TB) reactivation. Using the American College of Rheumatology's Rheumatology Informatics System for Effectiveness (RISE) registry, we examined pre-treatment screening among new users of biologic or targeted synthetic disease modifying drugs (DMARDs). METHODS: Data, derived from RISE, included patients ≥ 18 years old who were new users of biologic or targeted synthetic DMARDs. We developed quality measures related to pre-treatment screening for HBV, HCV, and TB in addition to a "composite" measure for all applicable tests. We assessed patient-level screening rates, practice-level variation among practices reporting on ≥ 20 patients, and the frequency of positive results. RESULTS: We included 26,802 patients across 213 rheumatology practices nationwide. Patients were 58 (14) years old, 75.9% female; 59.6% had rheumatoid arthritis, and TNFi were the most common index DMARDs (64.9%). Overall, 44.8% and 40.5% patients had any documented HBV or HCV screening, respectively, prior to the index date; 29.7% had TB screening in the year prior to drug start. Only 15.5% had documentation of screening for all appropriate infections prior to drug start. Practice-level performance on the composite measure was low (range 0 to 48.3%). 2.4% of screening tests were positive. CONCLUSION: We found gaps in documentation of key safety measures among practices participating in RISE. Given the small but significant number of patients with active or latent infections that pose safety risks, developing standardized and reliable strategies to capture safety screening measures is paramount.