Using electronic health records to identify candidates for human immunodeficiency virus pre-exposure prophylaxis: An application of super learning to risk prediction when the outcome is rare.

Human immunodeficiency virus (HIV) pre-exposure prophylaxis (PrEP) protects high risk patients from becoming infected with HIV. Clinicians need help to identify candidates for PrEP based on information routinely collected in electronic health records (EHRs). The greatest statistical challenge in developing a risk prediction model is that acquisition is extremely rare. METHODS: Data consisted of 180 covariates (demographic, diagnoses, treatments, prescriptions) extracted from records on 399 385 patient (150 cases) seen at Atrius Health (2007-2015), a clinical network in Massachusetts. Super learner is an ensemble machine learning algorithm that uses k-fold cross validation to evaluate and combine predictions from a collection of algorithms. We trained 42 variants of sophisticated algorithms, using different sampling schemes that more evenly balanced the ratio of cases to controls. We compared super learner's cross validated area under the receiver operating curve (cv-AUC) with that of each individual algorithm. RESULTS: The least absolute shrinkage and selection operator (lasso) using a 1:20 class ratio outperformed the super learner (cv-AUC = 0.86 vs 0.84). A traditional logistic regression model restricted to 23 clinician-selected main terms was slightly inferior (cv-AUC = 0.81). CONCLUSION: Machine learning was successful at developing a model to predict 1-year risk of acquiring HIV based on a physician-curated set of predictors extracted from EHRs.

Development and validation of an automated HIV prediction algorithm to identify candidates for pre-exposure prophylaxis: a modelling study.

BACKGROUND: HIV pre-exposure prophylaxis (PrEP) is effective but underused, in part because clinicians do not have the tools to identify PrEP candidates. We developed and validated an automated prediction algorithm that uses electronic health record (EHR) data to identify individuals at increased risk for HIV acquisition. METHODS: We used machine learning algorithms to predict incident HIV infections with 180 potential predictors of HIV risk drawn from EHR data from 2007-15 at Atrius Health, an ambulatory group practice in Massachusetts, USA. We included EHRs of all patients aged 15 years or older with at least one clinical encounter during 2007-15. We used ten-fold cross-validated area under the receiver operating characteristic curve (cv-AUC) with 95% CIs to assess the model's performance at identifying individuals with incident HIV and patients independently prescribed PrEP by clinicians. The best-performing model was validated prospectively with 2016 data from Atrius Health and externally with 2011-16 data from Fenway Health, a community health centre specialising in sexual health care in Boston (MA, USA). We calculated HIV risk scores (ie, probability of an incident HIV diagnosis) for every HIV-uninfected patient not on PrEP during 2007-15 at Atrius Health and assessed the distribution of scores for thresholds to determine possible candidates for PrEP in the three study cohorts. FINDINGS: We included 1 155 966 Atrius Health patients from 2007-15 (150 [<0·1%] patients with incident HIV) in our development cohort, 537 257 Atrius Health patients in 2016 (16 [<0·1%] with incident HIV) in our prospective validation cohort, and 33 404 Fenway Health patients from 2011-16 (423 [1·3%] with incident HIV) in our external validation cohort. The best-performing algorithm was obtained with least absolute shrinkage and selection operator (LASSO) and had a cv-AUC of 0·86 (95% CI 0·82-0·90) for identification of incident HIV infections in the development cohort, 0·91 (0·81-1·00) on prospective validation, and 0·77 (0·74-0·79) on external validation. The LASSO model successfully identified patients independently prescribed PrEP by clinicians at Atrius Health in 2016 (cv-AUC 0·93, 95% CI 0·90-0·96) or Fenway Health (0·79, 0·78-0·80). HIV risk scores increased steeply at the 98th percentile. Using this score as a threshold, we prospectively identified 9515 (1·8%) of 536 384 patients at Atrius Health in 2016 and 4385 (15·3%) of 28 702 Fenway Health patients as potential PrEP candidates. INTERPRETATION: Automated algorithms can efficiently identify patients at increased risk for HIV acquisition. Integrating these models into EHRs to alert providers about patients who might benefit from PrEP could improve prescribing and prevent new HIV infections. FUNDING: Harvard University Center for AIDS Research, Providence/Boston Center for AIDS Research, Rhode Island IDeA-CTR, the National Institute of Mental Health, and the US Centers for Disease Control and Prevention.

Temporal Patterns in Chlamydia Repeat Testing in Massachusetts.

INTRODUCTION: National guidelines recommend test-of-cure for pregnant women and test-of-reinfection for all patients with chlamydia infections in order to interrupt transmission and prevent adverse sequelae for patients, partners, and newborns. Little is known about retesting and positivity rates, and whether they are changing over time, particularly in private sector practices. METHODS: Electronic health record data on patients with chlamydia tests were extracted from three independent clinical practice groups serving ≅20% of the Massachusetts population. Records were extracted using the Electronic medical record Support for Public Health platform (esphealth.org). These data were analyzed for temporal trends in annual repeat testing rates by using generalized estimating equations after index positive chlamydia tests between 2010 and 2015 and for differences in intervals to first repeat tests among pregnant females, non-pregnant females, and males. Data extraction and analysis were performed during calendar years 2017 and 2018. RESULTS: An index positive C. trachomatis result was identified for 972 pregnant female cases, 10,309 non-pregnant female cases, and 4,973 male cases. Test-of-cure 3-5 weeks after an index positive test occurred in 37% of pregnant females. Test-of-reinfection 8-16 weeks after an index positive test occurred in 39% of pregnant females, 18% of non-pregnant females, and 9% of males. There were no significant increases in test-of-cure or test-of-reinfection rates from 2010 to 2015. Among cases with repeat tests, 16% of pregnant females, 15% of non-pregnant females, and 16% of males had positive results. CONCLUSIONS: Chlamydia test-of-cure and test-of-reinfection rates are low, with no evidence of improvement over time. There are substantial opportunities to improve adherence to chlamydia repeat testing recommendations.

Automated influenza-like illness reporting--an efficient adjunct to traditional sentinel surveillance.

OBJECTIVES: We compared an electronic health record-based influenza-like illness (ILI) surveillance system with manual sentinel surveillance and virologic data to evaluate the utility of the automated system for routine ILI surveillance. METHODS: We obtained weekly aggregate ILI reports from the Electronic medical record Support for Public Health (ESP) disease-detection and reporting system, which used an automated algorithm to identify ILI visits among a patient population of about 700,000 in Eastern Massachusetts. The percentage of total visits for ILI ("percent ILI") in ESP, percent ILI in the Massachusetts Department of Public Health's sentinel surveillance system, and percentage of laboratory specimens submitted to participating Massachusetts laboratories that tested positive for influenza were compared for the period October 2007-September 2011. We calculated Spearman's correlation coefficients and compared ESP and sentinel surveillance systems qualitatively, in terms of simplicity, flexibility, data quality, acceptability, timeliness, and usefulness. RESULTS: ESP and sentinel surveillance percent ILI always peaked within one week of each other. There was 80% correlation between the two and 71%-73% correlation with laboratory data. Sentinel surveillance percent ILI was higher than ESP percent ILI during influenza seasons. The amplitude of variation in ESP percent ILI was greatest for 5- to 49-year-olds and typically peaked for the 5- to 24-year-old age group before the others. CONCLUSIONS: The ESP system produces percent ILI data of similar quality to sentinel surveillance and offers the advantages of shifting disease reporting burden from clinicians to information systems, allowing tracking of disease by age group, facilitating efficient surveillance for very large populations, and producing consistent and timely reports.

Initial Antibiotic Choice in the Treatment of Group A Streptococcal Pharyngitis and Return Visit Rates.

BACKGROUND: Our objectives were to describe the incidence of return visits for children with Group A Streptococcal (GAS) pharyngitis (ie, clinical treatment failure) and to assess whether initial treatment with amoxicillin or penicillin was associated with the rate of retreatment for GAS pharyngitis. METHODS: This analysis was a retrospective cohort study of 5533 children 0-17 years from a multisite practice. Eligible visits (n = 6585) were associated with a positive test for GAS, receipt of antibiotics within 7 days, no allergies to penicillins or cephalosporins, and no codiagnoses requiring antibiotic treatment. Retreatment for GAS pharyngitis was defined as an index visit followed by another visit within 1-4 weeks. Five hundred episodes (250 treatment failures and 250 controls) were randomly selected for chart review to validate cases. RESULTS: Amoxicillin or penicillin was the initial antibiotic treatment at 76.1% of visits, and retreatment for GAS pharyngitis occurred after 5.8% of initial visits. Children initially prescribed amoxicillin or penicillin had higher odds of retreatment of GAS pharyngitis even after adjusting for age, sex, symptoms, and community-level covariates such as race, income, and education (odds ratio, 1.51; 95% confidence interval, 1.07-2.13). CONCLUSIONS: Retreatment for GAS pharyngitis was uncommon and associated with receipt of amoxicillin or penicillin, although the impact of GAS carriage is unknown. Recommendations for initial treatment of GAS pharyngitis should reflect both individual and societal considerations, including the potential impact on antibiotic resistance in the community.

Integrating clinical practice and public health surveillance using electronic medical record systems.

Electronic medical record (EMR) systems have rich potential to improve integration between primary care and the public health system at the point of care. EMRs make it possible for clinicians to contribute timely, clinically detailed surveillance data to public health practitioners without changing their existing workflows or incurring extra work. New surveillance systems can extract raw data from providers' EMRs, analyze them for conditions of public health interest, and automatically communicate results to health departments. We describe a model EMR-based public health surveillance platform called Electronic Medical Record Support for Public Health (ESP). The ESP platform provides live, automated surveillance for notifiable diseases, influenza-like illness, and diabetes prevalence, care, and complications. Results are automatically transmitted to state health departments.

Integrating clinical practice and public health surveillance using electronic medical record systems.

Electronic medical record (EMR) systems have rich potential to improve integration between primary care and the public health system at the point of care. EMRs make it possible for clinicians to contribute timely, clinically detailed surveillance data to public health practitioners without changing their existing workflows or incurring extra work. New surveillance systems can extract raw data from providers' EMRs, analyze them for conditions of public health interest, and automatically communicate results to health departments. The current paper describes a model EMR-based public health surveillance platform called Electronic Medical Record Support for Public Health (ESP). The ESP platform provides live, automated surveillance for notifiable diseases, influenza-like illness, and diabetes prevalence, care, and complications. Results are automatically transmitted to state health departments.

Evaluating real-time syndromic surveillance signals from ambulatory care data in four states.

OBJECTIVES: We evaluated a real-time ambulatory care-based syndromic surveillance system in four metropolitan areas of the United States. METHODS: Health-care organizations and health departments in California, Massachusetts, Minnesota, and Texas participated during 2007-2008. Syndromes were defined using International Classification of Diseases, Ninth Revision diagnostic codes in electronic medical records. Health-care organizations transmitted daily counts of new episodes of illness by syndrome, date, and patient zip code. A space-time permutation scan statistic was used to detect unusual clustering. Health departments followed up on e-mailed alerts. Distinct sets of related alerts ("signals") were compared with known outbreaks or clusters found using traditional surveillance. RESULTS: The 62 alerts generated corresponded to 17 distinct signals of a potential outbreak. The signals had a median of eight cases (range: 3-106), seven zip code areas (range: 1-88), and seven days (range: 3-14). Two signals resulted from true clusters of varicella; six were plausible but unconfirmed indications of disease clusters, six were considered spurious, and three were not investigated. The median investigation time per signal by health departments was 50 minutes (range: 0-8 hours). Traditional surveillance picked up 124 clusters of illness in the same period, with a median of six ill per cluster (range: 2-75). None was related to syndromic signals. CONCLUSIONS: The system was able to detect two true clusters of illness, but none was of public health interest. Possibly due to limited population coverage, the system did not detect any of 124 known clusters, many of which were small. The number of false alarms was reasonable.

Th 1 and Th 2 cytokines cooperate to stimulate mannose-receptor-mediated phagocytosis.

The mannose receptor is a macrophage surface receptor that mediates both endocytosis and phagocytosis. Previous work has demonstrated that the prototypical Th 2 cytokine, interleukin-4 (IL-4), increases both cell-surface receptor expression and mannose receptor-mediated endocytosis, whereas the prototypical Th 1 cytokine, interferon-γ (IFN-γ), decreases both surface expression and endocytosis. In many aspects of the immune response, Th 1 and Th 2 cytokines oppose each others' actions. We demonstrate that IL-4 and IFN-γ alone and together enhance mannose receptor-mediated phagocytosis, despite opposing effects on cell-surface mannose receptor expression and endocytosis. Thus these usually antagonistic cytokines cooperate in increasing mannose receptor phagocytic function. The cooperative effect of these cytokines is not observed for Fc receptor-mediated phagocytosis. The Th 2 cytokine IL-13 exerts similar effects to IL-4. Our results suggest that Th 1 and Th 2 cytokines may act in concert at sites of inflammation to enhance mannose receptor-mediated phagocytosis of microorganisms. J. Leukoc. Biol. 64: 108-113; 1998.