Racial and Ethnic Differences in Telemedicine Use.

Importance: Individuals of racial and ethnic minority groups may be less likely to use telemedicine in part due to lack of access to technology (ie, digital divide). To date, some studies have found less telemedicine use by individuals of racial and ethnic minority groups compared with White individuals, and others have found the opposite. What explains these different findings is unclear. Objective: To quantify racial and ethnic differences in the receipt of telemedicine and total visits with and without accounting for demographic and clinical characteristics and geography. Design, Setting, and Participants: This cross-sectional study included individuals who were continuously enrolled in traditional Medicare from March 2020 to February 2022 or until death. Exposure: Race and ethnicity, which was categorized as Black non-Hispanic, Hispanic, White non-Hispanic, other (defined as American Indian/Pacific Islander, Alaska Native, and Asian), and unknown/missing. Main Outcomes and Measures: Total telemedicine visits (audio-video or audio); total visits (telemedicine or in-person) per individual during the study period. Multivariable models were used that sequentially adjusted for demographic and clinical characteristics and geographic area to examine their association with differences in telemedicine and total visit utilization by documented race and ethnicity. Results: In this national sample of 14 305 819 individuals, 7.4% reported that they were Black, 5.6% Hispanic, and 4.2% other race. In unadjusted results, compared with White individuals, Black individuals, Hispanic individuals, and individuals of other racial groups had 16.7 (95% CI, 16.1-17.3), 32.9 (95% CI, 32.3-33.6), and 20.9 (95% CI, 20.2-21.7) more telemedicine visits per 100 beneficiaries, respectively. After adjustment for clinical and demographic characteristics and geography, compared with White individuals, Black individuals, Hispanic individuals, and individuals of other racial groups had 7.9 (95% CI, -8.5 to -7.3), 13.2 (95% CI, -13.9 to -12.6), and 9.2 (95% CI, -10.0 to -8.5) fewer telemedicine visits per 100 beneficiaries, respectively. In unadjusted and fully adjusted models, and in 2019 and the second year of the COVID-19 pandemic, Black individuals, Hispanic individuals, and individuals of other racial groups continued to have fewer total visits than White individuals. Conclusions and Relevance: The results of this cross-sectional study of US Medicare enrollees suggest that although nationally, Black individuals, Hispanic individuals, and individuals of other racial groups received more telemedicine visits during the pandemic and disproportionately lived in geographic regions with higher telemedicine use, after controlling for geographic region, Black individuals, Hispanic individuals, and individuals of other racial groups received fewer telemedicine visits than White individuals.

Questioning a Sensible Result.

Contemporary data collection strategies, storage capabilities, and modern statistical methodology have made retrospective analyses of observational databases commonplace. Such databases afford opportunities to learn about the effectiveness and risks of interventions or health behaviors that generally cannot be randomized. In this issue of NEJM Evidence, Cho et al.1 assemble survey data and cohort data from four countries to quantify the association between age-sex-specific smoking cessation and mortality. The authors conclude that smoking cessation at any age is associated with lower excess overall mortality risk and lower death from diseases made more common by smoking. It is difficult to argue with this conclusion - to question the magnitude of the associations is not.

Trends in Use of Telemedicine for Stimulant Initiation Among Children and Adults.

OBJECTIVE: The authors sought to examine trends in stimulant initiation and follow-up care for attention-deficit hyperactivity disorder (ADHD) via telemedicine. METHODS: This retrospective longitudinal study used national, deidentified commercial health insurance outpatient claims among children (ages 2-17 years; N=535,629) and adults (ages 18-64 years; N=2,116,160) from January 2019 through April 2022. Regression analyses were used to examine risk for stimulant initiation, whether initiation occurred via telemedicine or in-person care, and receipt of a follow-up visit. RESULTS: The mean monthly adjusted number of stimulant initiations per 100,000 enrollees was similar for children before and during the COVID-19 pandemic (prepandemic, 57 initiations; during pandemic, 56 initiations) but increased for adults (prepandemic, 27 initiations; during pandemic, 33 initiations). Initiations via telemedicine peaked at 53%-57% in April 2020 and dropped to about 14% among children and 28% among adults in April 2022. Telemedicine initiations were significantly more common among psychiatrists than among other prescribers (OR=3.70, 95% CI=3.38-4.06 [children]; OR=3.02, 95% CI=2.87-3.17 [adults]) and less common for rural residents (OR=0.57, 95% CI=0.40-0.82 [children]; OR=0.75, 95% CI=0.61-0.92 [adults]). Follow-up care was significantly more common among individuals whose care was initiated via telemedicine than among those receiving in-person care (OR=1.09, 95% CI=1.00-1.19 [children]; OR=1.61, 95% CI=1.53-1.69 [adults]). CONCLUSIONS: Many stimulant treatments were initiated via telemedicine. Proposed rules to prohibit controlled substance prescribing without an in-person evaluation would require significant changes in current practice, potentially limiting access to stimulant medications for ADHD.

Classification Algorithm to Distinguish Between Type 1 and Type 2 Myocardial Infarction in Administrative Claims Data.

BACKGROUND: Type 2 myocardial infarction (T2MI) and type 1 myocardial infarction (T1MI) differ with respect to demographics, comorbidities, treatments, and clinical outcomes. Reliable quality and outcomes assessment depends on the ability to distinguish between T1MI and T2MI in administrative claims data. As such, we aimed to develop a classification algorithm to distinguish between T1MI and T2MI that could be applied to claims data. METHODS: Using data for beneficiaries in a Medicare accountable care organization contract in a large health care system in New England, we examined the distribution of MI diagnosis codes between 2018 to 2021 and the patterns of care and coding for beneficiaries with a hospital discharge diagnosis International Classification of Diseases, Tenth Revision code for T2MI, compared with those for T1MI. We then assessed the probability that each hospitalization was for a T2MI versus T1MI and examined care occurring in 2017 before the introduction of the T2MI code. RESULTS: After application of inclusion and exclusion criteria, 7759 hospitalizations for myocardial infarction remained (46.5% T1MI and 53.5% T2MI; mean age, 79±10.3 years; 47% female). In the classification algorithm, female gender (odds ratio, 1.26 [95% CI, 1.11-1.44]), Black race relative to White race (odds ratio, 2.48 [95% CI, 1.76-3.48]), and diagnoses of COVID-19 (odds ratio, 1.74 [95% CI, 1.11-2.71]) or hypertensive emergency (odds ratio, 1.46 [95% CI, 1.00-2.14]) were associated with higher odds of the hospitalization being for T2MI versus T1MI. When applied to the testing sample, the C-statistic of the full model was 0.83. Comparison of classified T2MI and observed T2MI suggest the possibility of substantial misclassification both before and after the T2MI code. CONCLUSIONS: A simple classification algorithm appears to be able to differentiate between hospitalizations for T1MI and T2MI before and after the T2MI code was introduced. This could facilitate more accurate longitudinal assessments of acute myocardial infarction quality and outcomes.

Healthcare Access for a Diverse Population with Schizophrenia Following the Onset of the COVID-19 Pandemic.

COVID-19 has had a disproportionate impact on the most disadvantaged members of society, including minorities and those with disabling chronic illnesses such as schizophrenia. We examined the pandemic's impacts among New York State's Medicaid beneficiaries with schizophrenia in the immediate post-pandemic surge period, with a focus on equity of access to critical healthcare. We compared changes in utilization of key behavioral health outpatient services and inpatient services for life-threatening conditions between the pre-pandemic and surge periods for White and non-White beneficiaries. We found racial and ethnic differences across all outcomes, with most differences stable over time. The exception was pneumonia admissions-while no differences existed in the pre-pandemic period, Black and Latinx beneficiaries were less likely than Whites to be hospitalized in the surge period despite minorities' heavier COVID-19 disease burden. The emergence of racial and ethnic differences in access to scarce life-preserving healthcare may hold lessons for future crises.

Use of Telemedicine and Quality of Care Among Medicare Enrollees With Serious Mental Illness.

Importance: During the COVID-19 pandemic, a large fraction of mental health care was provided via telemedicine. The implications of this shift in care for use of mental health service and quality of care have not been characterized. Objective: To compare changes in care patterns and quality during the first year of the pandemic among Medicare beneficiaries with serious mental illness (schizophrenia or bipolar I disorder) cared for at practices with higher vs lower telemedicine use. Design, Setting, and Participants: In this cohort study, Medicare fee-for-service beneficiaries with schizophrenia or bipolar I disorder were attributed to specialty mental health practices that delivered the majority of their mental health care in 2019. Practices were categorized into 3 groups based on the proportion of telemental health visits provided during the first year of the pandemic (March 2020-February 2021): lowest use (0%-49%), middle use (50%-89%), or highest use (90%-100%). Across the 3 groups of practices, differential changes in patient outcomes were calculated from the year before the pandemic started to the year after. These changes were also compared with differential changes from a 2-year prepandemic period. Analyses were conducted in November 2022. Exposure: Practice-level use of telemedicine during the first year of the COVID-19 pandemic. Main Outcomes and Measures: The primary outcome was the total number of mental health visits (telemedicine plus in-person) per person. Secondary outcomes included the number of acute hospital and emergency department encounters, all-cause mortality, and quality outcomes, including adherence to antipsychotic and mood-stabilizing medications (as measured by the number of months of medication fills) and 7- and 30-day outpatient follow-up rates after discharge for a mental health hospitalization. Results: The pandemic cohort included 120 050 Medicare beneficiaries (mean [SD] age, 56.5 [14.5] years; 66 638 females [55.5%]) with serious mental illness. Compared with prepandemic changes and relative to patients receiving care at practices with the lowest telemedicine use: patients receiving care at practices in the middle and highest telemedicine use groups had 1.11 (95% CI, 0.45-1.76) and 1.94 (95% CI, 1.28-2.59) more mental health visits per patient per year (or 7.5% [95% CI, 3.0%-11.9%] and 13.0% [95% CI, 8.6%-17.4%] more mental health visits per year, respectively). Among patients of practices with middle and highest telemedicine use, changes in adherence to antipsychotic and mood-stabilizing medications were -0.4% (95% CI, -1.3% to 0.5%) and -0.1% (95% CI, -1.0% to 0.8%), and hospital and emergency department use for any reason changed by 2.4% (95% CI, -1.5% to 6.2%) and 2.8% (95% CI, -1.2% to 6.8%), respectively. There were no significant differential changes in postdischarge follow-up or mortality rates according to the level of telemedicine use. Conclusions and Relevance: In this cohort study of Medicare beneficiaries with serious mental illness, patients receiving care from practices that had a higher level of telemedicine use during the COVID-19 pandemic had more mental health visits per year compared with prepandemic levels, with no differential changes in other observed quality metrics over the same period.

Antipsychotics and the risk of diabetes and death among adults with serious mental illnesses.

BACKGROUND: Individuals with schizophrenia exposed to second-generation antipsychotics (SGA) have an increased risk for diabetes, with aripiprazole purportedly a safer drug. Less is known about the drugs' mortality risk or whether serious mental illness (SMI) diagnosis or race/ethnicity modify these effects. METHODS: Authors created a retrospective cohort of non-elderly adults with SMI initiating monotherapy with an SGA (olanzapine, quetiapine, risperidone, and ziprasidone, aripiprazole) or haloperidol during 2008-2013. Three-year diabetes incidence or all-cause death risk differences were estimated between each drug and aripiprazole, the comparator, as well as effects within SMI diagnosis and race/ethnicity. Sensitivity analyses evaluated potential confounding by indication. RESULTS: 38 762 adults, 65% White and 55% with schizophrenia, initiated monotherapy, with haloperidol least (6%) and quetiapine most (26·5%) frequent. Three-year mortality was 5% and diabetes incidence 9.3%. Compared with aripiprazole, haloperidol and olanzapine reduced diabetes risk by 1.9 (95% CI 1.2-2.6) percentage points, or a 18.6 percentage point reduction relative to aripiprazole users' unadjusted risk (10.2%), with risperidone having a smaller advantage. Relative to aripiprazole users' unadjusted risk (3.4%), all antipsychotics increased mortality risk by 1.1-2.2 percentage points, representing 32.4-64.7 percentage point increases. Findings within diagnosis and race/ethnicity were generally consistent with overall findings. Only quetiapine's higher mortality risk held in sensitivity analyses. CONCLUSIONS: Haloperidol's, olanzapine's, and risperidone's lower diabetes risks relative to aripiprazole were not robust in sensitivity analyses but quetiapine's higher mortality risk proved robust. Findings expand the evidence on antipsychotics' risks, suggesting a need for caution in the use of quetiapine among individuals with SMI.

Relationship Between In-Hospital Adverse Events and Hospital Performance on 30-Day All-cause Mortality and Readmission for Patients With Heart Failure.

BACKGROUND: Hospitals with high mortality and readmission rates for patients with heart failure (HF) might also perform poorly in other quality concepts. We sought to evaluate the association between hospital performance on mortality and readmission with hospital performance rates of safety adverse events. METHODS: This cross-sectional study linked the 2009 to 2019 patient-level adverse events data from the Medicare Patient Safety Monitoring System, a randomly selected medical records-abstracted patient safety database, to the 2005 to 2016 hospital-level HF-specific 30-day all-cause mortality and readmissions data from the United States Centers for Medicare & Medicaid Services. Hospitals were classified to one of 3 performance categories based on their risk-standardized 30-day all-cause mortality and readmission rates: better (both in <25th percentile), worse (both >75th percentile), and average (otherwise). Our main outcome was the occurrence (yes/no) of one or more adverse events during hospitalization. A mixed-effect model was fit to assess the relationship between a patient's risk of having adverse events and hospital performance categories, adjusted for patient and hospital characteristics. RESULTS: The study included 39 597 patients with HF from 3108 hospitals, of which 252 hospitals (8.1%) and 215 (6.9%) were in the better and worse categories, respectively. The rate of patients with one or more adverse events during a hospitalization was 12.5% (95% CI, 12.1-12.8). Compared with patients admitted to better hospitals, patients admitted to worse hospitals had a higher risk of one or more hospital-acquired adverse events (adjusted risk ratio, 1.24 [95% CI, 1.06-1.44]). CONCLUSIONS: Patients admitted with HF to hospitals with high 30-day all-cause mortality and readmission rates had a higher risk of in-hospital adverse events. There may be common quality issues among these 3 measure concepts in these hospitals that produce poor performance for patients with HF.

Evaluating The Accuracy Of Medicare Risk Adjustment For Alzheimer's Disease And Related Dementias.

In 2020 Medicare reintroduced Alzheimer's disease and related dementias (ADRD) Hierarchical Condition Categories (HCCs) to risk-adjust Medicare Advantage and accountable care organization (ACO) payments. The potential for Medicare spending increases from this policy change are not well understood because the baseline accuracy of ADRD HCCs is uncertain. Using linked 2016-18 claims and electronic health record data from a large ACO, we evaluated the accuracy of claims-based ADRD HCCs against a reference standard of clinician-adjudicated disease. An estimated 7.5 percent of beneficiaries had clinician-adjudicated ADRD. Among those with ADRD HCCs, 34 percent did not have clinician-adjudicated disease. The false-negative and false-positive rates were 22.7 percent and 3.2 percent, respectively. Medicare spending for those with false-negative ADRD HCCs exceeded that of true positives by $14,619 per beneficiary. If, after the reintroduction of risk adjustment for ADRD, all false negatives were coded as having ADRD, expenditure benchmarks for beneficiaries with ADRD would increase by 9 percent. Monitoring ADRD coding could become challenging in the setting of concurrent incentives to decrease false-negative rates and increase false-positive rates.

Analysis of Hospital-Level Readmission Rates and Variation in Adverse Events Among Patients With Pneumonia in the United States.

Importance: It is known that hospitalized patients who experience adverse events are at greater risk of readmission; however, it is unknown whether patients admitted to hospitals with higher risk-standardized readmission rates had a higher risk of in-hospital adverse events. Objective: To evaluate whether patients with pneumonia admitted to hospitals with higher risk-standardized readmission rates had a higher risk of adverse events. Design, Setting, and Participants: This cross-sectional study linked patient-level adverse events data from the Medicare Patient Safety Monitoring System (MPSMS), a randomly selected medical record abstracted database, to the hospital-level pneumonia-specific all-cause readmissions data from the Centers for Medicare & Medicaid Services. Patients with pneumonia discharged from July 1, 2010, through December 31, 2019, in the MPSMS data were included. Hospital performance on readmissions was determined by the risk-standardized 30-day all-cause readmission rate. Mixed-effects models were used to examine the association between adverse events and hospital performance on readmissions, adjusted for patient and hospital characteristics. Analysis was completed from October 2019 through July 2020 for data from 2010 to 2017 and from March through April 2022 for data from 2018 to 2019. Exposures: Patients hospitalized for pneumonia. Main Outcomes and Measures: Adverse events were measured by the rate of occurrence of hospital-acquired events and the number of events per 1000 discharges. Results: The sample included 46 047 patients with pneumonia, with a median (IQR) age of 71 (58-82) years, with 23 943 (52.0%) women, 5305 (11.5%) Black individuals, 37 763 (82.0%) White individuals, and 2979 (6.5%) individuals identifying as another race, across 2590 hospitals. The median hospital-specific risk-standardized readmission rate was 17.0% (95% CI, 16.3%-17.7%), the occurrence rate of adverse events was 2.6% (95% CI, 2.54%-2.65%), and the number of adverse events per 1000 discharges was 157.3 (95% CI, 152.3-162.5). An increase by 1 IQR in the readmission rate was associated with a relative 13% higher patient risk of adverse events (adjusted odds ratio, 1.13; 95% CI, 1.08-1.17) and 5.0 (95% CI, 2.8-7.2) more adverse events per 1000 discharges at the patient and hospital levels, respectively. Conclusions and Relevance: Patients with pneumonia admitted to hospitals with high all-cause readmission rates were more likely to develop adverse events during the index hospitalization. This finding strengthens the evidence that readmission rates reflect the quality of hospital care for pneumonia.

Statistical Approaches for Quantifying the Quality of Neurosurgical Care.

BACKGROUND: Quantifying quality of health care can provide valuable information to patients, providers, and policy makers. However, the observational nature of measuring quality complicates assessments. METHODS: We describe a conceptual model for defining quality and its implications about the data collected, how to make inferences about quality, and the assumptions required to provide statistically valid estimates. Twenty-one binary or polytomous quality measures collected from 101,051 adult Medicaid beneficiaries aged 18-64 years with schizophrenia from 5 U.S. states show methodology. A categorical principal components analysis establishes dimensionality of quality, and item response theory models characterize the relationship between each quality measure and a unidimensional quality construct. Latent regression models estimate racial/ethnic and geographic quality disparities. RESULTS: More than 90% of beneficiaries filled at least 1 antipsychotic prescription and 19% were hospitalized for schizophrenia during a 12-month observational period in our multistate cohort with approximately 2/3 nonwhite beneficiaries. Four quality constructs emerged: inpatient, emergency room, pharmacologic/ambulatory, and ambulatory only. Using a 2-parameter logistic model, pharmacologic/ambulatory care quality varied from -2.35 to 1.26 (higher = better quality). Black and Latinx beneficiaries had lower pharmacologic/ambulatory quality compared with whites. Race/ethnicity modified the association of state and pharmacologic/ambulatory care quality in latent regression modeling. Average quality ranged from -0.28 (95% confidence interval, -2.15 to 1.04) for blacks in New Jersey to 0.46 [95% confidence interval, -0.89 to 1.40] for whites in Michigan. CONCLUSIONS: By combining multiple quality measures using item response theory models, a composite measure can be estimated that has more statistical power to detect differences among subjects than the observed mean per subject.

Reevaluating Congenital Heart Surgery Center Performance Using Operative Mortality.

BACKGROUND: The Society of Thoracic Surgeons Congenital Heart Surgery Database (STS-CHSD) provides observed-to-expected (O/E) operative mortality ratios to more than 100 congenital heart centers in North America. We compared the current approach for estimating O/E ratios to approaches incorporating information on diagnosis as moderators of procedures, other unused risk factors, and additional variation in confidence interval construction to characterize center performance. METHODS: Bayesian additive regression trees (BART) and lasso models linked operative mortality to diagnosis-procedure categories, procedure-specific risk factors, and syndromes/abnormalities. Bootstrapping accounted for variation in the STS-CHSD (STS bootstrap) and lasso CIs. We compared O/E estimates, interquartile range of CI widths, and concordance of center performance categorizations (worse-than-, as-, or better-than-expected mortality) of the new approaches to the STS-CHSD. RESULTS: In 110 surgical centers including 98,822 surgical operative encounters, there were 2818 (2.85%) operative mortalities (center range, 0.37%-10%). Compared with the STS-CHSD, BART- and lasso-estimated O/E ratios varied more and had narrower confidence intervals (interquartile range of confidence interval: STS-CHSD = 1.11, STS bootstrap = 0.98; lasso = 0.80; BART = 0.96). Concordance of performance categorization with the STS-CHSD ranged from 84% (lasso) to 91% (STS Bootstrap); more than 70% of discordant centers improved categories. Discordant centers had smaller volumes, fewer operative mortalities, and treated more patients with congenital lung abnormalities. CONCLUSIONS: Relative to the STS-CHSD, up to 16% of hospitals changed performance categories, most improving performance. Given the significance of quality reports for congenital heart centers, inclusion of additional risk factors and unaddressed variation should be considered.

Mortality Prediction After Cardiac Surgery in Children: An STS Congenital Heart Surgery Database Analysis.

BACKGROUND: The Society of Thoracic Surgeons (STS) Congenital Heart Surgery Database (CHSD) provides risk-adjusted operative mortality rates to approximately 120 North American congenital heart centers. Optimal case-mix adjustment methods for operative mortality risk prediction in this population remain unclear. METHODS: A panel created diagnosis-procedure combinations of encounters in the CHSD. Models for operative mortality using the new diagnosis-procedure categories, procedure-specific risk factors, and syndromes or abnormalities included in the CHSD were estimated using Bayesian additive regression trees and least absolute shrinkage and selector operator (lasso) models. Performance of the new models was compared with the current STS CHSD risk model. RESULTS: Of 98 825 operative encounters (69 063 training; 29 762 testing), 2818 (2.85%) STS-defined operative mortalities were observed. Differences in sensitivity, specificity, and true and false positive predicted values were negligible across models. Calibration for mortality predictions at the higher end of risk from the lasso and Bayesian additive regression trees models was better than predictions from the STS CHSD model, likely because of the new models' inclusion of diagnosis-palliative procedure variables affecting <1% of patients overall but accounting for 27% of mortalities. Model discrimination varied across models for high-risk procedures, hospital volume, and hospitals. CONCLUSIONS: Overall performance of the new models did not differ meaningfully from the STS CHSD risk model. Adding procedure-specific risk factors and allowing diagnosis to modify predicted risk for palliative operations may augment model performance for very high-risk surgical procedures. Given the importance of risk adjustment in estimating hospital quality, a comparative assessment of surgical program quality evaluations using the different models is warranted.

Significance and Factors Associated with Antipsychotic Polypharmacy Utilization Among Publicly Insured US Adults.

Antipsychotic polypharmacy (APP) lacks evidence of effectiveness in the care of schizophrenia or other disorders for which antipsychotic drugs are indicated, also exposing patients to more risks. Authors assessed APP prevalence and APP association with beneficiary race/ethnicity and payer among publicly-insured adults regardless of diagnosis. Retrospective repeated panel study of fee-for-service (FFS) Medicare, Medicaid, and dually-eligible white, black, and Latino adults residing in California, Georgia, Iowa, Mississippi, Oklahoma, South Dakota, or West Virginia, filling antipsychotic prescriptions between July 2008 and June 2013. Primary outcome was any monthly APP utilization. Across states and payers, 11% to 21% of 397,533 antipsychotic users and 12% to 19% of 9,396,741 person-months had some APP utilization. Less than 50% of person-months had a schizophrenia diagnosis and up to 19% had no diagnosed mental illness. Payer modified race/ethnicity effects on APP utilization only in CA; however, the odds of APP utilization remained lower for minorities than for whites. Elsewhere, the odds varied by race/ethnicity only in OK, with Latinos having lower odds than whites (odds ratio 0.76; 95% confidence interval 0.60-0.96). The odds of APP utilization varied by payer in several study states, with odds generally higher for Dual eligibles, although the differences were generally small; the odds also varied by year (lower at study end). APP was frequently utilized but mostly declined over time. APP utilization patterns varied across states, with no consistent association with race/ethnicity and small payer effects. Greater use of APP-reducing strategies are needed, particularly among non-schizophrenia populations.

Performance Metrics for the Comparative Analysis of Clinical Risk Prediction Models Employing Machine Learning.

BACKGROUND: New methods such as machine learning techniques have been increasingly used to enhance the performance of risk predictions for clinical decision-making. However, commonly reported performance metrics may not be sufficient to capture the advantages of these newly proposed models for their adoption by health care professionals to improve care. Machine learning models often improve risk estimation for certain subpopulations that may be missed by these metrics. METHODS AND RESULTS: This article addresses the limitations of commonly reported metrics for performance comparison and proposes additional metrics. Our discussions cover metrics related to overall performance, discrimination, calibration, resolution, reclassification, and model implementation. Models for predicting acute kidney injury after percutaneous coronary intervention are used to illustrate the use of these metrics. CONCLUSIONS: We demonstrate that commonly reported metrics may not have sufficient sensitivity to identify improvement of machine learning models and propose the use of a comprehensive list of performance metrics for reporting and comparing clinical risk prediction models.

All-Cause Mortality and Progression to End-Stage Kidney Disease Following Percutaneous Revascularization or Surgical Coronary Revascularization in Patients with CKD.

INTRODUCTION: Relative impacts of coronary artery bypass grafting (CABG) and percutaneous coronary intervention (PCI) on mortality and end-stage kidney disease (ESKD) in chronic kidney disease (CKD) are uncertain. METHODS: Data from Massachusetts residents with CKD undergoing CABG or PCI from 2003 to 2012 were linked to the United States Renal Data System. Associations with death, ESKD, and combined death and ESKD were analyzed in propensity score-matched multivariable survival models. RESULTS: We identified 6805 CABG and 17,494 PCI patients. Among 3775 matched-pairs, multi-vessel disease was present in 97%, and stage 4 CKD was present in 11.9% of CABG and 12.2% of PCI patients. One-year mortality (CABG 7.7%, PCI 11.0%) was more frequent than ESKD (CABG 1.4%, PCI 1.7%). Overall survival was improved and ESKD risk decreased with CABG compared to PCI, but effects differed in the presence of left main disease and prior myocardial infarction (MI). Survival was worse following PCI than following CABG among patients with left main disease and without MI (hazard ratio = 3.7, 95% confidence interval = 1.3-10.5). ESKD risk was higher with PCI for individuals with left main disease and prior infarction (hazard ratio = 8.1, 95% confidence interval = 1.7-39.2). CONCLUSION: Risks following CABG and PCI were modified by left main disease and prior MI. In individuals with CKD, survival was greater after CABG than after PCI in patients with left main disease but without MI, whereas ESKD risk was lower with CABG in those with left main and MI. Absolute risks of ESKD were markedly lower than for mortality, suggesting prioritizing mortality over ESKD in clinical decision making.

Factors Associated With Off-Label Utilization of Second-Generation Antipsychotics Among Publicly Insured Adults.

OBJECTIVE: Off-label utilization of second-generation antipsychotic medications may expose patients to significant risks. The authors examined the prevalence, temporal trends, and factors associated with off-label utilization of second-generation antipsychotics among publicly insured adults. METHODS: A retrospective repeated panel was used to examine monthly off-label utilization of second-generation antipsychotics among fee-for-service Medicare, Medicaid, and dually eligible White, Black, and Latino adult beneficiaries filling prescriptions for second-generation antipsychotics in California, Georgia, Mississippi, and Oklahoma from July 2008 through June 2013. RESULTS: Among 301,367 users of second-generation antipsychotics, between 36.5% and 41.9% had utilization that was always off-label. Payer did not modify effects of race-ethnicity on off-label utilization. Compared with Whites, Blacks had lower monthly odds of off-label utilization in all four states, and Latinos had lower odds of utilization in California and Georgia. Payer was associated with off-label utilization in California, Mississippi, and Oklahoma. California Medicaid beneficiaries were 1.12 (95% confidence interval=1.10-1.13) times as likely as dually eligible beneficiaries to have off-label utilization. Off-label utilization increased relative to the baseline year in all states, but a downward trend followed in three states. CONCLUSIONS: Off-label utilization of second-generation antipsychotics was prevalent despite the drugs' cardiometabolic risks and little evidence of their effectiveness. The lower likelihood of off-label utilization among patients from racial-ethnic minority groups might stem from prescribers' efforts to minimize risks, given a higher baseline risk for these groups, or from disparities-associated factors. Variation among payers suggests that payer policies can affect off-label utilization.

Medication Utilization for Alcohol Use Disorder in a Commercially Insured Population.

OBJECTIVE: Examine patterns of alcohol use disorder (AUD) medication use and identify factors associated with prescription fill among commercially insured individuals with an index AUD visit. DESIGN: Using 2008-2018 claims data from a large national insurer, estimate days to first AUD medication using cause-specific hazards approach to account for competing risk of benefits loss. PARTICIPANTS: Aged 17-64 with ≥ 1 AUD visit. MAIN MEASURE: Days to AUD medication fill. KEY RESULTS: A total of 13.3% of the 151,128 with an index visit filled an AUD prescription after that visit, while 69.8% lost benefits before filling and 17.0% remained enrolled but did not fill (median days observed = 305). Almost half (46.3%) of those who filled a prescription received substance use disorder (SUD) inpatient care within 7 days before the fill, and 63.4% received SUD outpatient care. Likelihood of medication use was higher for those aged 26-35, 36-45, and 46-55 years relative to 56-64 years (e.g., 26-35: hazard ratio = 1.29 [95% confidence interval 1.23-1.36]); those diagnosed with moderate/severe AUD (2.05 [1.98-2.12]), co-occurring opioid use disorder (OUD) (1.33 [1.26-1.39]), or severe mental illness (1.31 [1.27-1.35]); those with a chronic alcohol-related diagnosis (1.08 [1.04-1.12]); and those whose index visit was in an inpatient/emergency department (1.27 [1.23-1.31]) or intermediate care setting (1.13 [1.07-1.20]) relative to outpatient. Likelihood of use was higher in later years relative to 2008 (e.g., 2018:2.02 [1.89-2.15]) and higher for those who received the majority of AUD care in a practice with a psychiatrist/addiction medicine specialist (1.13 [1.10-1.16]). Likelihood of use was lower for those diagnosed with a SUD other than AUD or OUD (0.88 [0.85-0.92]), those with an acute alcohol-related condition (0.79 [0.75-0.84]), and males (0.71 [0.69-0.73]). CONCLUSIONS: While AUD medication use increased and was more common among individuals with greater severity, few patients who could benefit from medications are using them. More efforts are needed to identify and treat individuals in non-acute care settings earlier in their course of AUD.