Interrater reliability and ease of use of the High-Alert Medication Stratification Tool‒Revised Prospective (HAMST-R PRO): A prospective, multisite study.

OBJECTIVE: The High-Alert Medication Stratification Tool‒Revised (HAMST-R) was originally designed to standardize the identification of high-alert medications (HAMs) according to safety risk. The primary objective of this multisite study was to assess interrater reliability of the HAMST-R PRO, a version of the tool designed to prospectively evaluate safety risk of medications during evaluation for formulary addition. METHODS: HAMST-R was designed as an objective tool to evaluate HAMs at a single site during the HAMST-R phase I study. Phase II of the study demonstrated the validity of the tool in a multisite, national study. In this third study, 11 medication safety experts from 8 health systems across the United States and 1 in Canada facilitated evaluation of medications prospectively with the HAMST-R PRO during the formulary review process for 27 medications. At each site, at least 5 individuals were asked to review each medication. Interrater reliability was evaluated using Kendall's coefficient of concordance. Ease of use was determined by participant interviews. RESULTS: Overall interrater reliability for HAMST-R PRO was found to be 0.76 (P < 0.001) across all sites, indicating substantial agreement between users. Interrater reliability among individual sites ranged from 0.52 to 0.82 (P < 0.05 for all sites). CONCLUSION: Interrater reliability of HAMST-R PRO is substantial, indicating consistency and agreement among pharmacists utilizing this tool to evaluate safety risk of medications before their addition to a health-system formulary. This information can be used to identify potential interventions for each step of the medication-use process that institutions may implement to decrease a medication's potential safety risk.

High-Alert Medication Stratification Tool-Revised Phase II: A Multisite Study Examining the Validity, Interrater Reliability, and Ease of Use of the High-Alert Medication Stratification Tool-Revised.

OBJECTIVES: The primary objective of this multisite study, High-Alert Medication Stratification Tool-Revised (HAMST-R) phase II, was to assess the content validity of HAMST-R. Secondary outcomes included interrater reliability and ease of use. METHODS: HAMST-R was designed as an objective tool to evaluate high-alert medications (HAMs) at a single site during HAMST-R Phase I. Medication safety experts from 7 health systems across the United States volunteered to participate in this phase II study. Participants completed a demographic survey, oversaw evaluation of 47 HAMs and 35 non-HAMs using HAMST-R, and submitted scores for each medication evaluated. In addition, participants rated each question of HAMST-R on its relevance to assess a medication's safety risk, measured as scale-content validity index. Positive and negative predictive values were evaluated in a post hoc analysis. Interrater reliability was evaluated using the Kendall coefficient of concordance (K), and ease of use was assessed using a mixed-methods approach. RESULTS: Scale-content validity index was 0.80, indicating that the tool was valid. Positive predictive value was 90.5% (95% confidence interval, 87.2%-93.0%), and negative predictive value was 98.2% (95% confidence interval, 95.4%-99.3%). A score of 4 or more differentiated between HAMs and non-HAMs, confirming phase I findings. K was 0.56, indicating moderate agreement. Participants confirmed that the tool was easy to use and plan to incorporate the tool into HAM policies and procedures, formulary review, and safety strategy implementation. CONCLUSIONS: HAMST-R is a valid, objective, and easy to use method that institutions may implement to evaluate a medication's potential safety risk.

High-Alert Medication Stratification Tool-Revised: An Exploratory Study of an Objective, Standardized Medication Safety Tool.

OBJECTIVE: To develop an objective tool designed to standardize the identification of high-alert medications (HAMs) according to patient safety risk. METHODS: Medications were evaluated using the High-Alert Medication Stratification Tool (HAMST). Tool revision occurred through assessing medications on an organization-approved HAM list and comparing scores with control medications not included on the list. Because of variations in HAMST interpretation by end users in interdisciplinary committees, a revision of the scoring tool was completed to create the High-Alert Medication Stratification Tool-Revised (HAMST-R), and the assessment was repeated. Both tools range from 0 to 10, with 10 describing agents with highest risk. RESULTS: The median (interquartile range [IQR]) initial HAM (n = 44) score using HAMST was 6 (5-7). The median (IQR) control (n = 45) score was 1 (0-2). Using the modified tool, HAMST-R, the median (IQR) HAM score was 4 (4-6) versus 1 (0-1) for controls. Scores for HAMs were significantly higher than controls using both tools (P < 0.001). A HAMST-R score of 4 or higher defines medications as high alert, as this score includes 75% of HAMs and no controls. CONCLUSIONS: Through this exploratory study, clarification of the tool was required to increase its concurrent validity, interrater reliability, and implementation among other health systems. The revised tool, HAMST-R, is a newly developed, objective tool for standardized identification of HAMs. The tool may also be used for prospective identification of medications as high risk to patient safety during formulary review.

The Impact of Changes to an Electronic Admission Order Set on Prescribing and Clinical Outcomes in the Intensive Care Unit.

BACKGROUND: Implementation of disease-specific order sets has improved compliance with standards of care for a variety of diseases. Evidence of the impact admission order sets can have on care is limited. OBJECTIVE: The main purpose of this article is to evaluate the impact of changes made to an electronic critical care admission order set on provider prescribing patterns and clinical outcomes. METHODS: A retrospective, observational before-and-after exploratory study was performed on adult patients admitted to the medical intensive care unit using the Inpatient Critical Care Admission Order Set. The primary outcome measure was the percentage change in the number of orders for scheduled acetaminophen, a histamine-2 receptor antagonist (H2RA), and lactated ringers at admission before implementation of the revised order set compared with after implementation. Secondary outcomes assessed clinical impact of changes made to the order set. RESULTS: The addition of a different dosing strategy for a medication already available on the order set (scheduled acetaminophen vs. as needed acetaminophen) had no impact on physician prescribing (0 vs. 0%, p = 1.000). The addition of a new medication class (an H2RA) to the order set significantly increased the number of patients prescribed an H2RA for stress ulcer prophylaxis (0 vs. 20%, p < 0.001). Rearranging the list of maintenance intravenous fluids to make lactated ringers the first fluid option in place of normal saline significantly decreased the number of orders for lactated ringers (17 vs. 4%, p = 0.005). The order set changes had no significant impact on clinical outcomes such as incidence of transaminitis, gastrointestinal bleed, and acute kidney injury. CONCLUSION: Making changes to an admission order set can impact provider prescribing patterns. The type of change made to the order set, in addition to the specific medication changed, may have an effect on how influential the changes are on prescribing patterns.