Generalizability of an Automatic Explanation Method for Machine Learning Prediction Results on Asthma-Related Hospital Visits in Patients With Asthma: Quantitative Analysis.

BACKGROUND: Asthma exerts a substantial burden on patients and health care systems. To facilitate preventive care for asthma management and improve patient outcomes, we recently developed two machine learning models, one on Intermountain Healthcare data and the other on Kaiser Permanente Southern California (KPSC) data, to forecast asthma-related hospital visits, including emergency department visits and hospitalizations, in the succeeding 12 months among patients with asthma. As is typical for machine learning approaches, these two models do not explain their forecasting results. To address the interpretability issue of black-box models, we designed an automatic method to offer rule format explanations for the forecasting results of any machine learning model on imbalanced tabular data and to suggest customized interventions with no accuracy loss. Our method worked well for explaining the forecasting results of our Intermountain Healthcare model, but its generalizability to other health care systems remains unknown. OBJECTIVE: The objective of this study is to evaluate the generalizability of our automatic explanation method to KPSC for forecasting asthma-related hospital visits. METHODS: Through a secondary analysis of 987,506 data instances from 2012 to 2017 at KPSC, we used our method to explain the forecasting results of our KPSC model and to suggest customized interventions. The patient cohort covered a random sample of 70% of patients with asthma who had a KPSC health plan for any period between 2015 and 2018. RESULTS: Our method explained the forecasting results for 97.57% (2204/2259) of the patients with asthma who were correctly forecasted to undergo asthma-related hospital visits in the succeeding 12 months. CONCLUSIONS: For forecasting asthma-related hospital visits, our automatic explanation method exhibited an acceptable generalizability to KPSC. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.2196/resprot.5039.

Development of a pediatric asthma predictive index for hospitalization.

BACKGROUND: Variation in emergency department (ED) management for asthma exacerbation leads to disparities in care. Current asthma severity scores are insufficient to be used for hospitalization decisions. OBJECTIVE: To develop and internally validate an asthma predictive index for hospitalization (APIH) to guide practitioners in their admission decision for children with asthma exacerbations. METHODS: Data were collected from 12,066 children between 5 and 18 years old diagnosed with asthma exacerbation in the ED. Epidemiologic findings, number of inhaled corticosteroid canisters, short-acting ß-blocker canisters, oral steroids, coexisting atopy, family history of atopy, insurance, and prior asthma ED visits or hospitalizations were compared between patients hospitalized and discharged. We used univariate analysis and multivariate analysis to determine the best predictor variables for hospitalization. Our study internally validated the prediction index to estimate future performance of the prediction rule. RESULTS: The highest risk factors associated with asthma hospitalization from the ED are oxygen saturation less than 94%, respiratory rate greater than 31/min, history of pneumonia, and asthma ED visits in past 12 months. With a reduced predictive model that combined these risk factors, the odds ratio was 44.9 (95% CI, 32.8-61.4), which is extremely significant. Our C index of discrimination of 0.77 was similar to the validation C index of 0.78, which confirms a solid prediction model. CONCLUSION: We have developed and internally validated a pediatric hospitalization prediction index for acute asthma exacerbation in the ED. Further studies are needed to externally validate the APIH before its implementation into clinical practice.

Self-Reported Physical Activity and Asthma Risk in Children.

BACKGROUND: Increased physical activity (PA) may protect against asthma but PA can trigger asthma symptoms. OBJECTIVE: To investigate relationships between moderate-to-vigorous PA (MVPA) assessed during routine care visits and incident asthma. METHODS: For this retrospective cohort, 542,486 children between 2 and 17 years from 2010 to 2017 were included who had an MVPA assessment (exercise vital sign) during routine care visits. The association of MVPA and asthma was analyzed using Cox proportional hazards regression models as a function of age, with MVPA and body mass index (BMI) being time-varying factors, adjusted for race and ethnicity, socioeconomic status, and air pollution. RESULTS: The mean MVPA was 5.4 (standard deviation: 4.4) hours/week. Crude asthma incidence density rate (IDR) was highest in children with <1 hour/week of MVPA (IDR: 9.07, 95% confidence interval [CI]: 8.79, 9.36) and lowest in children engaging in 4 to 7 hours/week of MVPA (IDR: 6.55, 95% CI: 6.33, 6.77). In adjusted models, an increase in MVPA was associated with lower asthma risk in children reporting 0 hour/week of MVPA (hazard ratio: 0.981, 95% CI: 0.973, 0.990). In children with ≥8 hours/week of MVPA, an increase in MVPA was associated with higher asthma risk (1.005, 95% CI: 1.002, 1.009). There was no significant BMI by MVPA interaction. CONCLUSION: Increasing MVPA in children with low activity levels is associated with lower asthma risk; children reporting high levels of activity may experience greater asthma risk as their activity levels increase further. Understanding the role of PA in the development of asthma and assessing MVPA during routine care visits in children may help to develop targeted interventions and guide asthma management.

Population-Based Incidence of New Ampicillin, Cephalexin, Cefaclor, and Sulfonamide Antibiotic "Allergies" in Exposed Individuals with and without Preexisting Ampicillin, Cephalexin, or Cefaclor "Allergies".

BACKGROUND: There is a theoretical concern, unconfirmed by population-based challenge data, that clinically significant, immunologically mediated hypersensitivity occurs among ß-lactams sharing side chains. OBJECTIVE: To determine the population-based allergy incidence associated with the use of ß-lactams sharing exact R1 side chains (ampicillin, cephalexin, and cefaclor [ACC]), with or without a current ACC allergy or a sulfonamide antibiotic allergy for comparison. METHODS: All courses of ACC and trimethoprim-sulfamethoxazole used by any Kaiser Permanente California members in 2017 and 2018, with follow-up through January 2019, were identified along with their preexisting antibiotic allergy status and all new antibiotic-specific allergies reported within 30 days of course initiation. RESULTS: A total of 1,167,713 courses of ACC were administered to individuals with no sulfonamide antibiotic or ACC allergy and 4,771 new ACC allergies (0.41%) were reported. Moreover, 130,032 courses of ACC were administered to individuals with a sulfonamide antibiotic allergy and no ACC allergy, and 904 new ACC allergies (0.70%) were reported. There were 5,958 courses of ACC administered to individuals with an ACC allergy, 2,341 who also had sulfonamide antibiotic allergy, and 52 new ACC allergies (0.87%) were reported. CONCLUSIONS: The incidence of new ACC allergy reports is minimally and non-specifically increased among individuals with a preexisting ACC or sulfonamide antibiotic allergy compared to the baseline incidence in the population. This argues against clinically significant, immunologically mediated cross-reactivity among ß-lactams sharing exact side chains in individuals with preexisting but unconfirmed ß-lactam allergy. Any previously reported, even unrelated antibiotic allergy appears to be a risk factor for reporting a new antibiotic allergy.

Association of exhaled nitric oxide to asthma burden in asthmatics on inhaled corticosteroids.

BACKGROUND: Fractional exhaled nitric oxide (FENO) is a marker of airway inflammation. Its role in assessing asthma burden in clinical practice needs more study. OBJECTIVE: To determine whether higher FENO levels are associated with greater asthma burden. METHODS: This was a multicenter cross-sectional retrospective study of atopic 12- to 56-year-old persistent asthmatics on inhaled corticosteroids (ICS). Questionnaire and 1-year retrospective administrative data were used to analyze by unadjusted and adjusted robust Poisson regression (relative risks) and negative binomial regression [incidence rate ratios (IRRs)] the associations of masked FENO levels (NIOX MINO®) to short-acting beta-agonist (SABA) dispensings and oral corticosteroid (OCS) use in the past year independent of spirometry and an asthma control tool [Asthma Control Test (ACT)]. RESULTS: FENO levels ranged from 7-215 ppb (median 28 ppb) in 325 patients. Higher FENO levels significantly correlated with more SABA dispensings and OCS courses in the past year, lower FEV(1)% predicted levels, but not ACT score. FENO highest (≥48 ppb) versus lowest (≤19 ppb) quartile values were associated independently in the past year with ≥7 SABA canisters dispensed (relative risk=2.40, 95% CI=1.25-4.62) and total number of SABA canisters dispensed (IRR=1.46, 95% CI=1.12-1.99) and with ≥1 OCS course (relative risk=1.48, 95% CI=1.06-2.07) and total number of OCS courses (IRR=1.71, 95% CI=1.09-2.66). The significant independent relationship of higher FENO levels to increasing SABA dispensings and OCS courses was confirmed by linear trend analyses. CONCLUSIONS: Independent and clinically meaningful associations between higher FENO levels and greater asthma burden during a prior year in persistent asthmatics on ICS suggest that FENO measurement may be a complementary tool to help clinicians assess asthma burden.

Association Between Removal of a Warning Against Cephalosporin Use in Patients With Penicillin Allergy and Antibiotic Prescribing.

Importance: Electronic health records (EHRs) often include default alerts that can influence physician selection of antibiotics, which in turn may be associated with a suboptimal choice of agents and increased antibiotic resistance. Objective: To examine whether removal of a default alert in the EHR to avoid cephalosporin use in patients with penicillin allergies is associated with changes in cephalosporin dispensing or administration in these patients. Design, Setting, and Participants: This retrospective cohort study of a natural experiment included data on patients who had received antibiotic treatment in the hospital or outpatient setting in 2 regions of a large, integrated health system in California from January 1, 2017, to December 31, 2018. Of 4 398 792 patients, 4 206 480 met the eligibility criteria: enrollment in the health system during antibiotic use, availability of complete demographic data, and use of antibiotics outside of the washout period. Interventions or Exposures: Oral or parenteral antibiotics dispensed or administered after removal of an EHR alert to avoid cephalosporin use in patients with a recorded penicillin allergy. Main Outcomes and Measures: Probability that an antibiotic course was a cephalosporin. A multinomial logistic regression model was used to examine the change in rates of cephalosporin use before and after an EHR penicillin allergy alert was removed in 1 of the study regions. Temporal changes in use rates were controlled for by comparing changes in cephalosporin use among patients with or without a penicillin allergy at the site that removed the warning and among patients at a comparison site that retained the warning. Regression models were used to examine adverse events. Results: Of the 4 206 480 patients who met all inclusion criteria, 2 465 849 (58.6%) were women; the mean (SD) age was 40.5 (23.2) years. A total of 10 652 014 antibiotic courses were administered or dispensed, divided approximately evenly between the period before and after removal of the warning. Before removal of an alert in the electronic health record system to avoid prescribing of cephalosporins to patients with a penicillin allergy at 1 of the 2 sites, 58 228 courses of cephalosporins (accounting for 17.9% of all antibiotic use at the site) were used among patients with a penicillin allergy; after removal of the alert, administration or dispensing of cephalosporins increased by 47% compared with cephalosporin administration or dispensing among patients without a penicillin allergy at the same site and patients at the comparison site that retained the warning (ratio of ratios of odds ratios [RROR], 1.47; 95% CI, 1.38-1.56) . No significant differences in anaphylaxis (9 total cases), new allergies (RROR, 1.02; 95% CI, 0.93-1.12), or treatment failures (RROR, 1.02; 95% CI, 0.99-1.05) were found at the course level. No significant differences were found in all-cause mortality (ratio of ratios of rate ratios [RRRR], 1.03; 95% CI, 0.94-1.13), hospital days (RRRR, 1.04; 95% CI, 0.99-1.10), and new infections (Clostridioides difficile: RRRR, 1.02; 95% CI, 0.84-1.22; methicillin-resistant Staphylococcus aureus: RRRR, 0.87; 95% CI, 0.75-1.00; and vancomycin-resistant Enterococcus: RRRR, 0.82; 95% CI, 0.55-1.22) at the patient level. Conclusions and Relevance: In this cohort study, removal of a warning in the electronic health record to avoid cephalosporin use in patients with penicillin allergies was associated with increased administration and dispensing of cephalosporin. This simple and rapidly implementable system-level intervention may be useful for improvement in antibiotic stewardship.

Developing a Predictive Model for Asthma-Related Hospital Encounters in Patients With Asthma in a Large, Integrated Health Care System: Secondary Analysis.

BACKGROUND: Asthma causes numerous hospital encounters annually, including emergency department visits and hospitalizations. To improve patient outcomes and reduce the number of these encounters, predictive models are widely used to prospectively pinpoint high-risk patients with asthma for preventive care via care management. However, previous models do not have adequate accuracy to achieve this goal well. Adopting the modeling guideline for checking extensive candidate features, we recently constructed a machine learning model on Intermountain Healthcare data to predict asthma-related hospital encounters in patients with asthma. Although this model is more accurate than the previous models, whether our modeling guideline is generalizable to other health care systems remains unknown. OBJECTIVE: This study aims to assess the generalizability of our modeling guideline to Kaiser Permanente Southern California (KPSC). METHODS: The patient cohort included a random sample of 70.00% (397,858/568,369) of patients with asthma who were enrolled in a KPSC health plan for any duration between 2015 and 2018. We produced a machine learning model via a secondary analysis of 987,506 KPSC data instances from 2012 to 2017 and by checking 337 candidate features to project asthma-related hospital encounters in the following 12-month period in patients with asthma. RESULTS: Our model reached an area under the receiver operating characteristic curve of 0.820. When the cutoff point for binary classification was placed at the top 10.00% (20,474/204,744) of patients with asthma having the largest predicted risk, our model achieved an accuracy of 90.08% (184,435/204,744), a sensitivity of 51.90% (2259/4353), and a specificity of 90.91% (182,176/200,391). CONCLUSIONS: Our modeling guideline exhibited acceptable generalizability to KPSC and resulted in a model that is more accurate than those formerly built by others. After further enhancement, our model could be used to guide asthma care management. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID): RR2-10.2196/resprot.5039.

Relationship of asthma control to asthma exacerbations using surrogate markers within a managed care database.

OBJECTIVE: To compare the relationship of surrogate markers of asthma control to subsequent asthma exacerbations. STUDY DESIGN: Retrospective cohort. METHODS: Administrative data were used to identify patients who met the Healthcare Effectiveness Data and Information Set (HEDIS) criteria for persistent asthma in 2006 and 2007. The following potential surrogate markers of asthma control assessed in 2007 were compared for their ability to predict asthma exacerbations in 2008 (defined as oral corticosteroid dispensing or an asthma hospitalization or emergency department visit): dispensing of any controller, unweighted medication ratio (the ratio of controller to total medication), weighted medication ratio, and the number of short-acting beta-agonist (SABA) canisters dispensed. Weighted medication ratios were weighted for controller potency and for doses per container. RESULTS: Meeting the HEDIS criteria for persistent asthma were 8634 patients (60.5% female) aged 18 to 56 years (mean age, 42.7 years), of whom 6.5% experienced emergency hospital care and 27.3% received oral corticosteroids in 2008. The largest effect size for predicting reduced emergency hospital care was for the number of SABA canisters dispensed (odds ratio [OR], 0.49; 95% confidence interval [CI], 0.40-0.60), followed by the unweighted medication ratio (OR, 0.54; 95% CI, 0.40-0.72), and then the weighted medication ratio (OR, 0.57; 95% CI, 0.45-0.73). Dispensing of any controller was associated with a nonsignificant increased risk of emergency hospital care (OR, 1.41; 95% CI, 0.95-2.09). CONCLUSIONS: The number of SABA canisters dispensed is most strongly related to improved asthma outcomes, followed by the unweighted medication ratio; dispensing of any controller is least related. Health plans can use the number of SABA canisters dispensed and the unweighted medication ratio for asthma population management or for provider quality-of-care assessment to reduce asthma exacerbations, which exact a high economic and humanistic cost.

Persistent asthma defined using HEDIS versus survey criteria.

OBJECTIVES: To describe the relationships between persistent asthma defined by administrative versus survey data and their stability over time. STUDY DESIGN: Longitudinal survey and retrospective administrative database. METHODS: Administrative data were used to identify patients meeting the Healthcare Effectiveness Data and Information Set (HEDIS) criteria for persistent asthma in year 1 (2006). At the end of year 2 and on 3 occasions during year 3, patients were mailed a survey to define persistent asthma based on symptoms and medication use in the prior month and exacerbations in the prior 12 months. Administrative data were also used to define medical utilization for asthma in year 3. RESULTS: Of 13,833 eligible patients, 2895 (20.9%) returned the survey; 2751 of these respondents reported physician-diagnosed asthma, of whom 2517 (91.5%) had survey-defined persistent asthma. Patients having survey-defined persistent asthma (68.0%) were more likely to requalify as having HEDIS-defined persistent asthma in year 2 than patients not having survey-defined persistent asthma (22.2%). However, 81.6% of survey respondents who did not requalify as having HEDIS-defined persistent asthma in year 2 had survey-defined persistent asthma. Patients with survey-defined persistent asthma in year 2 had significantly more medical utilization for asthma in year 3 than patients without survey-defined persistent asthma. Approximately 82% of the 799 patients completing all 4 surveys had persistent asthma on all surveys. CONCLUSIONS: HEDIS-defined persistent asthma is generally consistent with survey-defined persistent asthma. Persistent asthma usually remains persistent over a 3-year period, indicating that it is a stable characteristic of asthma for most patients. The low survey response rate suggests that further population-based studies will be necessary to confirm the validity and generalizability of our study findings regarding persistent asthma.

Emergency department utilization by patients not meeting Health Plan and Employer Data and Information Set (HEDIS) criteria for persistent asthma.

Emergency hospital utilization rates for asthma remain high despite advances in asthma controller medications and the presence of widely accepted asthma treatment guidelines. To explore this phenomenon, we analyzed administrative data to determine characteristics of patients seen in the emergency department (ED) for asthma. Complete pharmacy and diagnostic coding records were obtained from consecutive adults (aged 19-56 years) treated for asthma in the ED of a closed-network health maintenance organization between April and July of 2002. Subjects were stratified into asthma severity categories (persistent or non-persistent) based on the National Committee for Quality Assurance 2006 Health Plan and Employer Data and Information Set (HEDIS) criteria for persistent asthma. Eighty-one unique patients made a total of 89 ED visits for asthma during the study period. Of the 89 total ED visits for asthma, 44 (49%) occurred in patients that did not meet HEDIS criteria for persistent asthma. Of the 81 unique patients making asthma-related ED visits, 41 (51%) did not meet HEDIS criteria for persistent asthma. Over one-half (51%) of this nonpersistent population were not given either asthma reliever or asthma controller medications during the 12-month period before their index ED visit. Over the 24-month period before their index ED visit, 37% of nonpersistent patients were dispensed neither asthma reliever nor controller medications. Patients that do not meet HEDIS criteria for persistent asthma account for a substantial percentage of asthma-related ED visits. These patients have a history of low use of asthma medications before their ED visit.

The effects of quinapril and atorvastatin on the responsiveness to sildenafil in men with erectile dysfunction.

Phosphodiesterase-5 (PDE-5) inhibitors are an effective therapy for the majority of men with erectile dysfunction (ED). However, many men with ED still report a suboptimal or partial response even after an adequate trial of oral PDE-5 therapy. Since ED is associated with impaired vascular function and both atorvastatin and quinapril have been previously shown to improve vascular function, we examined the effects of adjunctive treatment with these medications in men with vasculogenic ED who were suboptimal responders to 100 mg of sildenafil. Men with ED and suboptimal response to sildenafil were randomly assigned to 3 months of treatment with atorvastatin 40 mg (n = 12), quinapril 10 mg (n = 10), or placebo (n = 13), along with continued adjunctive sildenafil use. Measured variables included: International Index of Erectile Function (IIEF) questionnaire, brachial artery flow-mediated dilation (FMD), endothelium-independent dilation (EID) via nitroglycerin, penile Doppler blood flow, blood pressure (BP), lipids, and C-reactive protein (CRP). Compared to placebo, quinapril (p < 0.01) significantly improved symptoms of ED as measured by the IIEF-5 questionnaire. There was a trend toward a significant improvement in IIEF-5 with atorvastatin. Similarly, quinapril significantly improved the IIEF ED Domain (p < 0.05). Other peripheral and penile vascular parameters were unchanged with drug therapy as compared to placebo. In conclusion, treatment with quinapril, in combination with sildenafil, improved ED in men with suboptimal response to sildenafil alone. Atorvastatin demonstrated a trend toward improved ED in this group.

Age effects on objective measures of atopy in adult asthma and rhinitis.

A cross-sectional survey of 132 adult men referred to the outpatient allergy clinic at the West Los Angeles Veterans Affairs Medical Center was performed to assess age effects on allergic disease in the elderly. Total serum immunoglobulin E (IgE), immediate hypersensitivity skin testing, and serum eosinophil count were measured in all subjects. Subjects were stratified by age into one of five groups for comparison. In asthma, prevalence of allergy skin test reactivity and mean total serum IgE levels did not decline with advancing age, suggesting that IgE-dependent mechanisms continue to be significant in elderly patients with asthma. In subjects with rhinitis, prevalence of allergy skin test reactivity and mean total serum IgE did decline among elderly subjects relative to younger subjects. However, the prevalence of allergic rhinitis did not decline in the elderly. This suggests that although allergic rhinitis is common in elderly patients, nonallergic causes of rhinitis may become more prevalent with advancing age.