Proof-of-Concept Study of Using Supervised Machine Learning Algorithms to Predict Self-Care and Glycemic Control in Type 1 Diabetes Patients on Insulin Pump Therapy.

OBJECTIVE: Using supervised machine learning algorithms (SMLAs), we built models to predict the probability of type 1 diabetes mellitus patients on insulin pump therapy for meeting insulin pump self-management behavioral (IPSMB) criteria and achieving good glycemic response within 6 months. METHODS: This was a single-center retrospective chart review of 100 adult type 1 diabetes mellitus patients on insulin pump therapy (≥6 months). Three SMLAs were deployed: multivariable logistic regression (LR), random forest (RF), and K-nearest neighbor (k-NN); validated using repeated three-fold cross-validation. Performance metrics included area under the curve-Receiver of characteristics for discrimination and Brier scores for calibration. RESULTS: Variables predictive of adherence with IPSMB criteria were baseline hemoglobin A1c, continuous glucose monitoring, and sex. The models had comparable discriminatory power (LR = 0.74; RF = 0.74; k-NN = 0.72), with the RF model showing better calibration (Brier = 0.151). Predictors of the good glycemic response included baseline hemoglobin A1c, entering carbohydrates, and following the recommended bolus dose, with models comparable in discriminatory power (LR = 0.81, RF = 0.80, k-NN = 0.78) but the RF model being better calibrated (Brier = 0.099). CONCLUSION: These proof-of-concept analyses demonstrate the feasibility of using SMLAs to develop clinically relevant predictive models of adherence with IPSMB criteria and glycemic control within 6 months. Subject to further study, nonlinear prediction models may perform better.

Impact of Non-Tailored One-Way Automated Short Messaging Service (OASMS) on Glycemic Control in Type 2 Diabetes: A Retrospective Feasibility Study.

Short message service (SMS) is easily accessible and potentially an ideal platform for delivering patient-targeted messages. However, an effective SMS dosing strategy is not well established. Our purpose was to evaluate the impact of diabetes self-care promoting messages via non-tailored one-way automated SMS (OASMS) on glycemic control in type 2 diabetes (T2DM). The change in hemoglobin A1c (HbA1c) was compared between patients who received the service and those who did not. This retrospective quasi-experimental pre-post feasibility study was conducted at an academic medical center endocrinology clinic. English-speaking adults (≥18 years) with uncontrolled T2DM (HbA1c ≥ 8%) were included. A total of 69 patients (intervention n = 34; control n = 35) met the inclusion criteria. The mean (±SD) baseline HbA1c values were 10.2% (±1.9%) and 9.9% (±1.7%) in the intervention and control arms, respectively. Median follow-up was 3.3 months (IQR = 3-4.2). An ANCOVA model adjusted for baseline HbA1c and age showed an estimated HbA1c reduction difference of -0.97% (95% CI, -1.73 to -0.20%, p = 0.014), favoring the intervention arm. Inverse propensity score weighting confirmed the ANCOVA results. Our study suggests that adding diabetes self-care promoting messages via non-tailored OASMS to usual care improves glycemic control in poorly controlled T2DM. Larger and longer studies are needed to evaluate different features of the non-tailored OASMS strategy.

Effect of board certification on antihypertensive treatment intensification in patients with diabetes mellitus.

BACKGROUND: Regular recertification is mandatory to maintain board certification status in all specialties. However, the evidence that physicians' performance decreases with time since initial certification is limited. We therefore carried out a study to determine whether the frequency of antihypertensive treatment intensification for diabetic patients changes with time since their physicians' last board certification. METHODS AND RESULTS: In this retrospective cohort study, we analyzed treatment of 8127 hypertensive patients with diabetes mellitus treated by 301 internists at primary care practices affiliated with 2 large academic hospitals. Patient visits with documented blood pressure > or = 130/85 mm Hg between January 1, 2000, and August 31, 2005, were studied. The association between the number of years since the physician's last board certification and the probability of pharmacological antihypertensive treatment intensification at a given visit was analyzed. Frequency of treatment intensification decreased from 26.7% for physicians who were board certified the previous year to 6.9% for physicians who were board certified 31 years before the visit. Treatment intensification rate was 22.5% for physicians certified < or = 10 years ago versus 16.9% for physicians last certified > 10 years ago (P<0.0001). Multivariable analysis adjusted for patient and visit characteristics and physician age showed that for every decade since the physician's last board certification, the probability of treatment intensification decreased by 21.3% (P=0.0097). CONCLUSIONS: Physician intensification of pharmacological therapy for blood pressure levels above the recommended treatment goals decreases with time since the last board certification. This finding supports the current policy of mandatory recertification.

Comparison of information content of structured and narrative text data sources on the example of medication intensification.

OBJECTIVE To compare information obtained from narrative and structured electronic sources using anti-hypertensive medication intensification as an example clinical issue of interest. DESIGN A retrospective cohort study of 5,634 hypertensive patients with diabetes from 2000 to 2005. MEASUREMENTS The authors determined the fraction of medication intensification events documented in both narrative and structured data in the electronic medical record. The authors analyzed the relationship between provider characteristics and concordance between intensifications in narrative and structured data. As there is no gold standard data source for medication information, the authors clinically validated medication intensification information by assessing the relationship between documented medication intensification and the patients' blood pressure in univariate and multivariate models. RESULTS Overall, 5,627 (30.9%) of 18,185 medication intensification events were documented in both sources. For a medication intensification event documented in narrative notes the probability of a concordant entry in structured records increased by 11% for each study year (p < 0.0001) and decreased by 19% for each decade of provider age (p = 0.035). In a multivariate model that adjusted for patient demographics and intraphysician correlations, an increase of one medication intensification per month documented in either narrative or structured data were associated with a 5-8 mm Hg monthly decrease in systolic and 1.5-4 mm Hg decrease in diastolic blood pressure (p < 0.0001 for all). CONCLUSION Narrative and structured electronic data sources provide complementary information on anti-hypertensive medication intensification. Clinical validity of information in both sources was demonstrated by correlation with changes in blood pressure.

Evaluation of compliance with a paper-based, multiplication-factor, intravenous insulin protocol.

BACKGROUND: Hyperglycemia is common in critically ill patients and is an independent risk factor for in-hospital morbidity and mortality. OBJECTIVE: To assess compliance with a paper-based, multiplication-factor, intravenous insulin protocol. METHODS: A retrospective chart review was conducted in a 720-bed urban, academic medical center in Boston, Massachusetts. During a 1-month period, compliance with and the consequent safety and efficacy of the Brigham and Women's Hospital paper-based, multiplication-factor, intravenous insulin protocol was evaluated. RESULTS: The primary endpoint of protocol compliance, defined as correct adjustment to insulin infusion rate and correct timing of bedside blood glucose concentration (BBGC) checks +/-10 minutes of prespecified BBGC check according to the Brigham and Women's Hospital Intravenous Insulin Protocol (BHIP), was 47.2%. Seventy-two patients met inclusion criteria. Appropriate adjustment of infusion rates occurred 68.2% (1206/1768) of the time. Compliance with the timing of BBGC checks was found to be the majority of protocol violations. BBGCs were monitored +/-5 minutes of indicated time per the protocol 26.2% (463/1768) of the time. Blood glucose concentration checks within extended timing of +/-10 minutes of indicated time per the protocol occurred 793 (44.8%) times. Blood glucose concentration monitoring took place greater than 20 minutes past indicated time 450 (25.5%) times. In 1768 measurements, blood glucose concentrations between 40 and 60 mg/dL occurred 23 (1.3%) times in 12 (16.7%) patients. Blood glucose concentrations 40 mg/dL or less were detected 3 (0.17%) times in 2 (2.7%) patients. None of these hypoglycemic events led to documented complications. CONCLUSIONS: Overall, a rather low level of compliance with a paper-based, multiplication-factor, intravenous insulin protocol was observed, which warrants further investigation. Compliance rates in this evaluation were found to be similar to the rates observed in previously evaluated fixed-dose intravenous insulin protocols. Protocol noncompliance may be associated with hypo- and hyperglycemia.

Treatment intensification and blood glucose control among hospitalized diabetic patients.

BACKGROUND: Hyperglycemia is common among diabetic inpatients, and has been linked to adverse outcomes. However, antihyperglycemic treatment is seldom intensified in noncritical care patients, and the relationship between intensification frequency and glucose control is poorly understood. We evaluated the relationship between treatment intensification and changes in blood glucose in hospitalized diabetic patients. DESIGN: We retrospectively analyzed 3,613 hospitalized diabetic patients who were admitted to the hospital between January 2003 and August 2004, were not hospitalized in an ICU, were not prescribed IV insulin or total parenteral nutrition (TPN), had a length of stay of at least 3 days, and had at least one point-of-care blood glucose measurement. A linear model was used to assess the relationship between intensification of antihyperglycemic medications and the average daily change in point-of-care glucose measurements. RESULTS: Hyperglycemia (>180 mg/dL) was documented at least once for 82.5% of patient admissions. Antihyperglycemic treatment was intensified for 22.0% of days with hyperglycemia. Intensifications of scheduled and sliding scale insulin, but not oral medications, were associated with a 11.1 mg/dL (p < 0.0001) and 12.2 mg/dL (p < 0.0001) reduction in the average daily glucose, respectively. Hypoglycemia (<50 mg/dL) was documented on 2.2% of days after antihyperglycemic treatment intensification. CONCLUSION: In this cohort, lack of treatment intensification in response to inpatient hyperglycemia was common. Antihyperglycemic treatment intensification was strongly associated with decrease in average daily glucose, while hypoglycemia was uncommon. This suggests that increasing the frequency of treatment intensifications could lead to improved glycemic control in inpatients with diabetes.

Impact of supplemental training programs on improving medical students' confidence in providing diabetes self-management education and support.

BACKGROUND/PURPOSE: The purpose of this study was to evaluate the effectiveness of supplemental diabetes-related training modalities and volunteer activities in increasing first-year medical students' knowledge/comfort in providing diabetes self-management education and support (DSMES) to patients. METHODS: A group of medical students developed supplemental diabetes-related training/volunteer programs. The training modalities included an optional 7-session interprofessionally taught Diabetes Enrichment Elective and a 3-hour endocrinologist-led training session intended to prepare students for involvement in an inpatient DSMES volunteer program. The volunteer program provided the students with the opportunity to provide DSMES to patients with diabetes admitted to an academic medical center. Those participating in any of the stated programs were compared to those with no such training regarding confidence in providing DSMES using an optional online survey. The results were analyzed by using Mann-Whitney U test and descriptive analyses. RESULTS: A total of 18 first-year medical students responded to the optional survey with a response rate of ~30% (10 of 33) among participants in any training/volunteer program. First-year medical students who attended any of the offered optional programs had statistically significant higher comfort level in 4 of the 6 areas assessed regarding providing DSMES compared with those with no such training (p<0.05), with medium to large effect size (r=0.48-0.59). CONCLUSION: This study suggests that the supplemental preclerkship diabetes-specific training modalities/volunteer programs can provide benefit in providing medical students with practical knowledge while improving their confidence in providing DSMES to patients with diabetes.

Using regular expressions to abstract blood pressure and treatment intensification information from the text of physician notes.

This case study examined the utility of regular expressions to identify clinical data relevant to the epidemiology of treatment of hypertension. We designed a software tool that employed regular expressions to identify and extract instances of documented blood pressure values and anti-hypertensive treatment intensification from the text of physician notes. We determined sensitivity, specificity and precision of identification of blood pressure values and anti-hypertensive treatment intensification using a gold standard of manual abstraction of 600 notes by two independent reviewers. The software processed 370 Mb of text per hour, and identified elevated blood pressure documented in free text physician notes with sensitivity and specificity of 98%, and precision of 93.2%. Anti-hypertensive treatment intensification was identified with sensitivity 83.8%, specificity of 95.0%, and precision of 85.9%. Regular expressions can be an effective method for focused information extraction tasks related to high-priority disease areas such as hypertension.

A student-implemented elective to improve medical student confidence in providing diabetes self-management support.

BACKGROUND: The purpose of this study was to develop a preclerkship elective and assess its effectiveness in supplementing medical students' education. METHODS: A group of medical students under the guidance of two faculty advisors developed an elective consisting of six sessions covering a variety of practical aspects of diabetes care/education taught by an interprofessional team. Following the course completion, a survey was emailed to the enrollees who attended at least one session. The results were analyzed using Wilcoxon signed-rank and descriptive analyses. RESULTS: A total of 14 medical students were enrolled (nine first year and five second year). An average of 4.4 sessions/student was attended. Thirteen students attended at least one session and were surveyed. The survey response rate was ~62% (8/13). All eight students indicated that the course was valuable and would recommend it to their colleagues. A Wilcoxon signed-rank test revealed a statistically significant increase in students' confidence in all five areas assessed following participation in the course, P<0.05 with a large effect (r>0.5). CONCLUSION: This study suggests the feasibility of developing disease state-specific preclerkship elective courses and that such courses can be beneficial in supplementing medical student education with practical knowledge.

Quality of care for diabetic patients in a large urban public hospital.

OBJECTIVE: We compared diabetes quality-of-care indicators for patients receiving medical treatment in three practice settings of the same hospital. RESEARCH DESIGN AND METHODS: A cross-sectional medical record review for patients receiving care between 1 July 2000 and 30 June 2001 was conducted. Records were abstracted from three practice settings: the Diabetes Clinic (DIABETES), a general medicine clinic staffed by internal medicine residents (RESIDENT), and a general medicine clinic whose providers were medical school faculty physicians (FACULTY). Record review (n = 791) yielded data on diabetes indicators that were derived primarily from the Diabetes Quality Improvement Project. RESULTS: There were significant differences between the DIABETES, RESIDENT, and FACULTY clinics for the percentages of patients with HbA(1c) testing (94 vs. 92 vs. 76%, P < 0.001), HbA(1c) >9.5% (31 vs. 36 vs. 43%, P < 0.05), nephropathy assessment (79 vs. 67 vs. 58%, P < 0.001), lipid assessment (86 vs. 79 vs. 76%, P < 0.050), LDL <130 mg/dl (54 vs. 44 vs. 43%, P < 0.05), blood pressure <140/90 mmHg (63 vs. 55 vs. 49%, P < 0.025), eye examinations (64 vs. 50 vs. 31%, P < 0.001), foot examinations (97 vs. 55 vs. 24%, P < 0.001), ACE inhibitor treatment (66 vs. 69 vs. 35%, P < 0.001), and aspirin treatment (71 vs. 59 vs. 15%, P < 0.001). CONCLUSIONS: There is considerable variation in diabetes management in different primary care settings of the same hospital. Although management in all settings was suboptimal, the results attained by the patients in the Diabetes Clinic represent minimal achievable goals for all diabetic patients in this hospital.

Long-acting subcutaneously administered insulin for glycemic control immediately after cardiac surgery.

OBJECTIVE: To test the hypothesis that subcutaneous administration of basal insulin begun immediately after cardiac surgery can decrease the need for insulin infusion in patients without diabetes and save nursing time. METHODS: After cardiac surgery, 36 adult patients without diabetes were randomly assigned to receive either standard treatment (control group) or insulin glargine once daily in addition to standard treatment (basal insulin group). Standard treatment included blood glucose measurements every 1 to 4 hours and intermittent insulin infusion to maintain blood glucose levels between 100 and 150 mg/dL. The study period lasted up to 72 hours. RESULTS: There were no differences in demographics or baseline laboratory characteristics of the 2 study groups. Mean daily blood glucose levels were lower in the basal insulin group in comparison with the control group, but the difference was not statistically significant (129.3 ± 9.4 mg/dL versus 132.6 ± 7.3 mg/dL; P = .25). The mean duration of insulin infusion was significantly shorter in the basal insulin group than in the control group (16.3 ± 10.7 hours versus 26.6 ± 17.3 hours; P = .04). Nurses tested blood glucose a mean of 8.3 ± 3.5 times per patient per day in the basal insulin group and 12.0 ± 4.7 times per patient per day in the control group (P = .01). There was no occurrence of hypoglycemia (blood glucose level <60 mg/dL) in either group. CONCLUSION: Once-daily insulin glargine is safe and may decrease the duration of insulin infusion and reduce nursing time in patients without diabetes who have hyperglycemia after cardiac surgery.

Effects of a computerized order set on the inpatient management of hyperglycemia: a cluster-randomized controlled trial.

OBJECTIVE: To determine the effects of a computerized order set on the inpatient management of diabetes and hyperglycemia. METHODS: We conducted a cluster-randomized controlled trial on the general medical service of an academic medical center staffed by residents and hospitalists. Consecutively enrolled patients with diabetes mellitus or inpatient hyperglycemia were randomized on the basis of their medical team to usual care (control group) or an admission order set built into the hospital's computer provider order entry (CPOE) system (intervention group). All teams received a detailed subcutaneous insulin protocol and case-based education. The primary outcome was the mean percent of glucose readings per patient between 60 and 180 mg/dL. RESULTS: Between April 5 and June 22, 2006, we identified 179 eligible study subjects. The mean percent of glucose readings per patient between 60 and 180 mg/dL was 75% in the intervention group and 71% in the usual care group (adjusted relative risk, 1.36; 95% confidence interval, 1.03 to 1.80). In comparison with usual care, the intervention group also had a lower patient-day weighted mean glucose (148 mg/dL versus 158 mg/dL, P = .04), less use of sliding-scale insulin by itself (25% versus 58%, P = .01), and no significant difference in the rate of severe hypoglycemia (glucose <40 mg/dL; 0.5% versus 0.3% of patient-days, P = .58). CONCLUSION: The use of an order set built into a hospital's CPOE system led to improvements in glycemic control and insulin ordering without causing a significant increase in hypoglycemia. Other institutions with CPOE should consider adopting similar order sets as part of a comprehensive inpatient glycemic management program.

Effects of a subcutaneous insulin protocol, clinical education, and computerized order set on the quality of inpatient management of hyperglycemia: results of a clinical trial.

BACKGROUND: Inpatient hyperglycemia is associated with poor patient outcomes. It is unknown how best to implement glycemic management strategies in the non-intensive care unit (ICU) setting. OBJECTIVE: To determine the effects of a multifaceted quality improvement intervention on the management of medical inpatients with diabetes mellitus or hyperglycemia. DESIGN: Before-after trial. SETTING: Geographically localized general medical service staffed by physician's assistants (PAs) and hospitalists. PATIENTS: Consecutively enrolled patients with type 2 diabetes or inpatient hyperglycemia. INTERVENTION: A detailed subcutaneous insulin protocol, an admission order set built into the hospital's computerized order entry system, and case-based educational workshops and lectures to nurses, physicians, and PAs. MEASUREMENTS: Mean percent of glucose readings per patient between 60 and 180 mg/dL; percent patient-days with hypoglycemia; insulin use patterns; and hospital length of stay. RESULTS: The mean percent of readings per patient between 60 and 180 mg/dL was 59% prior to the intervention and 65% afterward (adjusted effect size 9.7%; 95% confidence interval [CI], 0.6%-18.8%). The percent of patient days with any hypoglycemia was 5.5% preintervention and 6.1% afterward (adjusted odds ratio 1.1; 95% CI, 0.6-2.1). Use of scheduled nutritional insulin increased from 40% to 75% (odds ratio 4.5; 95% CI, 2.0-9.9) and adjusted length of stay decreased by 25% (95% CI, 9%-44%). Daily insulin adjustment did not improve, nor did glucose control beyond hospital day 3. CONCLUSIONS: This multifaceted intervention, which was easy to implement and required minimal resources, was associated with improvements in both insulin ordering practices and glycemic control among non-ICU medical patients.

Hypoglycemia and clinical outcomes in patients with diabetes hospitalized in the general ward.

OBJECTIVE: Hypoglycemia is associated with adverse outcomes in mixed populations of patients in intensive care units. It is not known whether the same risks exist for diabetic patients who are less severely ill. In this study, we aimed to determine whether hypoglycemic episodes are associated with higher mortality in diabetic patients hospitalized in the general ward. RESEARCH DESIGN AND METHODS: This retrospective cohort study analyzed 4,368 admissions of 2,582 patients with diabetes hospitalized in the general ward of a teaching hospital between January 2003 and August 2004. The associations between the number and severity of hypoglycemic (

Improving glycemic control in medical inpatients: a pilot study.

BACKGROUND: Inpatient hyperglycemia is associated with poor patient outcomes. Current guidelines recommend that in an inpatient non-ICU setting there be treatment to achieve a glucose level below 180 mg/dL. METHODS: Objectives of this prospective quality-improvement pilot study were to implement a subcutaneous insulin protocol on a general medicine service, to identify barriers to implementation, and to determine the effect of this protocol on glycemic control. Eighty-nine patients with a preexisting diagnosis of type 2 diabetes or inpatient hyperglycemia were eligible. Study outcomes included resident acceptance of the protocol, insulin-ordering practices, and mean rate of hyperglycemia (glucose > 180 mg/dL) per person. Results were compared with those of a previously conducted observational study. RESULTS: Residents agreed to use the protocol in 56% of cases. Reasons for declining the protocol included severity of a patient's other disease states, desire to titrate oral medications, and fear of hypoglycemia. Basal and nutritional insulin were prescribed more often in the pilot group compared with at baseline (64% vs. 49% for basal, P = .05; 13% vs. 0% for nutritional, P < .001). Basal insulin was started after the first full hospital day in 42% of patients, and only one-third of patients with any hypo- or hyperglycemia had any subsequent changes in their insulin orders. The mean rate of hyperglycemia was not significantly different between groups (31.6% of measurements per patient vs. 33.3%, P = .85). CONCLUSIONS: Adherence to a new inpatient subcutaneous insulin protocol was fair. Barriers included fear of hypoglycemia, delays in starting basal insulin, and clinical inertia. Quality improvement efforts likely need to target these barriers to successfully improve inpatient glycemic control.

Identification of documented medication non-adherence in physician notes.

Medication non-adherence is common and the physicians awareness of it may be an important factor in clinical decision making. Few sources of data on physician awareness of medication non-adherence are available. We have designed an algorithm to identify documentation of medication non-adherence in the text of physician notes. The algorithm recognizes eight semantic classes of documentation of medication non-adherence. We evaluated the algorithm against manual ratings of 200 randomly selected notes of hypertensive patients. The algorithm detected 89% of the notes with documented medication non-adherence with specificity of 84.7% and positive predictive value of 80.2%. In a larger dataset of 1,000 documents, notes that documented medication non-adherence were more likely to report significantly elevated systolic (15.3% vs. 9.0%; p = 0.002) and diastolic (4.1% vs. 1.9%; p = 0.03) blood pressure. This novel clinically validated tool expands the range of information on medication non-adherence available to researchers.

Relationship of physician volume with process measures and outcomes in diabetes.

OBJECTIVE: The volume of patients cared for by an individual physician (physician volume) has been linked to improved outcomes for a number of conditions. It is not known whether a similar association exists for treatment of diabetes. In this study we aimed to determine whether physician volume is associated with improved process measures and outcomes in diabetes care. RESEARCH DESIGN AND METHODS: This retrospective cohort study analyzed electronic medical records data for 7,120 patients with diabetes treated by 368 primary care physicians at practices affiliated with two large academic hospitals. The associations between physician volume of diabetic patients (diabetes volume) and annual A1C and LDL testing, as well as blood pressure, A1C, and LDL levels, were evaluated. RESULTS: In multivariable analysis, absolute diabetes volume was linked to decreased odds of A1C testing (4% less for each additional patient seen; P = 0.05), and relative diabetes volume (fraction of the total patients seen who had diabetes) was associated with decreased odds of both A1C (25% less for every 10% increase in the number of diabetic patients seen annually; P = 0.03) and LDL testing (20% less for every 10% increase in the number of diabetic patients; P < 0.001). Physician volume was not significantly associated with the odds of blood pressure, A1C, or LDL control at the end of the study. CONCLUSIONS: Higher physician volume in care of diabetic patients is associated with decreased adherence to surveillance guidelines and no measurable difference in treatment outcomes.

Computational analysis of non-adherence and non-attendance using the text of narrative physician notes in the electronic medical record.

Non-adherence to physician recommendations is common and is thought to lead to poor clinical outcomes. However, no techniques exist for a large-scale assessment of this phenomenon. We evaluated a computational approach that quantifies patient non-adherence from an analysis of the text of physician notes. Index of non-adherence (INA) was computed based on the number of non-adherence word tags detected in physician notes. INA was evaluated by comparing the results to a manual patient record review at the individual sentence and patient level. The relationship between INA and frequency of Emergency Department visits was determined. The positive predictive value of identification of individual non-adherence word tags was 93.3%. The Pearson correlation coefficient between the INA and the number of documented instances of non-adherence identified by manual review was 0.62. The frequency of ED visits was more than twice as high for patients with INA in the highest quartile (least adherent) than for patients with INA in the lowest (most adherent) quartile (p < 0.0001). We have described the design and evaluation of a novel approach that allows quantification of patient non-adherence with physician recommendations through an analysis of physician notes. This approach has been validated at several levels and demonstrated to correlate with clinical outcomes.