User-centred design for machine learning in health care: a case study from care management.

OBJECTIVES: Few machine learning (ML) models are successfully deployed in clinical practice. One of the common pitfalls across the field is inappropriate problem formulation: designing ML to fit the data rather than to address a real-world clinical pain point. METHODS: We introduce a practical toolkit for user-centred design consisting of four questions covering: (1) solvable pain points, (2) the unique value of ML (eg, automation and augmentation), (3) the actionability pathway and (4) the model's reward function. This toolkit was implemented in a series of six participatory design workshops with care managers in an academic medical centre. RESULTS: Pain points amenable to ML solutions included outpatient risk stratification and risk factor identification. The endpoint definitions, triggering frequency and evaluation metrics of the proposed risk scoring model were directly influenced by care manager workflows and real-world constraints. CONCLUSIONS: Integrating user-centred design early in the ML life cycle is key for configuring models in a clinically actionable way. This toolkit can guide problem selection and influence choices about the technical setup of the ML problem.

Conversion from Vancomycin Trough Concentration-Guided Dosing to Area Under the Curve-Guided Dosing Using Two Sample Measurements in Adults: Implementation at an Academic Medical Center.

STUDY OBJECTIVE: The optimal pharmacodynamic parameter for the prediction of efficacy of vancomycin is the area under the concentration-time curve (AUC), and current published data indicate that dosing based on vancomycin trough concentrations is an inaccurate substitute. In this study, our objective was to compare the achievement of therapeutic target attainment after switching from a trough-based to an AUC-based dosing strategy as a part of our institution's vancomycin-per-pharmacy protocol. DESIGN: Prospective observational quality assurance study. SETTING: Academic medical center. PATIENTS: A total of 296 hospitalized adults who received vancomycin and monitoring under our institution's vancomycin-per-pharmacy protocol were included in the analysis. The preimplementation retrospective comparison group consisted of 179 patients in whom vancomycin was initiated using a trough-based dosing strategy between November 22, 2017, and January 22, 2018. The postimplementation group included 117 patients in whom vancomycin was initiated using an AUC-based dosing strategy using two-point sampling between June 19, 2018, and July 19, 2018, after hospital-wide implementation of this protocol on June 19, 2018. MEASUREMENTS AND MAIN RESULTS: AUC values were calculated from two vancomycin concentrations (peak and trough). The primary outcome was achievement of therapeutic AUC values (400-800 mg·hr/L) in the postimplementation group or therapeutic trough level values (10-20 mg/L) in the preimplementation group. Only 98 (55%) of 179 initial trough values were therapeutic in the preimplementation group (trough-only dosing method) versus 86 (73.5%) of 117 initial AUC values in the postimplementation group (AUC-based dosing method) (p=0.0014). A lower proportion of supratherapeutic AUC values was observed in the postimplementation group compared with supratherapeutic trough concentrations in the preimplementation group (1.7% vs 18%, p<0.0001). Overall, 62% of patients with initially therapeutic AUC values had subsequent trough value increases of 25% or greater, occurring at a median of 6 days of vancomycin therapy. Nephrotoxicity occurred in 11% of patients in the preimplementation versus 9.4% in the postimplementation group (p=0.70). CONCLUSION: Compared with a trough concentration-based dosing strategy, AUC-based dosing using two-point sampling improved therapeutic target attainment. Implementation is feasible at any hospital that performs vancomycin peak concentration testing and is a workable alternative to using Bayesian software for estimating AUC. This approach should also be directly compared with AUC-based dosing using Bayesian software.