Relating process and outcome metrics for meaningful and interpretable cannulation skill assessment: A machine learning paradigm.

BACKGROUND AND OBJECTIVES: The quality of healthcare delivery depends directly on the skills of clinicians. For patients on hemodialysis, medical errors or injuries caused during cannulation can lead to adverse outcomes, including potential death. To promote objective skill assessment and effective training, we present a machine learning approach, which utilizes a highly-sensorized cannulation simulator and a set of objective process and outcome metrics. METHODS: In this study, 52 clinicians were recruited to perform a set of pre-defined cannulation tasks on the simulator. Based on data collected by sensors during their task performance, the feature space was then constructed based on force, motion, and infrared sensor data. Following this, three machine learning models- support vector machine (SVM), support vector regression (SVR), and elastic net (EN)- were constructed to relate the feature space to objective outcome metrics. Our models utilize classification based on the conventional skill classification labels as well as a new method that represents skill on a continuum. RESULTS: With less than 5% of trials misplaced by two classes, the SVM model was effective in predicting skill based on the feature space. In addition, the SVR model effectively places both skill and outcome on a fine-grained continuum (versus discrete divisions) that is representative of reality. As importantly, the elastic net model enabled the identification of a set of process metrics that highly impact outcomes of the cannulation task, including smoothness of motion, needle angles, and pinch forces. CONCLUSIONS: The proposed cannulation simulator, paired with machine learning assessment, demonstrates definite advantages over current cannulation training practices. The methods presented here can be adopted to drastically increase the effectiveness of skill assessment and training, thereby potentially improving clinical outcomes of hemodialysis treatment.

Measuring Cannulation Skills for Hemodialysis: Objective Versus Subjective Assessment.

Lack of cannulation skill during hemodialysis treatments results in poor clinical outcomes due to infiltration and other cannulation-related trauma. Unfortunately, training of patient care technicians and nurses, specifically on the "technical" aspects of cannulation, has traditionally not received much attention. Simulators have been successfully deployed in many medical specialties for assessment and training of clinical skills. However, simulators have not been as widely used in nursing, especially in the context of training clinical personnel in the dialysis unit. We designed a state-of-the-art simulator for quantifying skill for hemodialysis cannulation. In this study, 52 nurses and patient care technicians with varying levels of clinical experience performed 16 cannulations on the simulator with different fistula properties. We formulated a composite metric for objectively measuring overall success of cannulation and compared this metric with subjective assessment by experts. In addition, we examined if years of clinical experience correlated with objective and subjective scores for cannulation skill. Results indicated that, while subjective and objective metrics generally correlated with each other, the objective metric was more precise and better suited for quantifying cannulation skill. Further, the simulator-based objective metric provides several advantages over subjective ratings, including providing fine-grained assessment of skill, consistency in measurement unaffected by subjective biases, and basing assessment on a more complete evaluation of performance. Years of clinical experience, however, demonstrated little correlation with either method of skill assessment. The methods presented for cannulation skill assessment in this study, if widely applied, could result in improved cannulation skill among our PCTs and nurses, which could positively impact patient outcomes in a tangible way.