An extended protocol for usability validation of medical devices: Research design and reference model.

This paper proposes and demonstrates an extended protocol for usability validation testing of medical devices. A review of currently used methods for the usability evaluation of medical devices revealed two main shortcomings. Firstly, the lack of methods to closely trace the interaction sequences and derive performance measures. Secondly, a prevailing focus on cross-sectional validation studies, ignoring the issues of learnability and training. The U.S. Federal Drug and Food Administration's recent proposal for a validation testing protocol for medical devices is then extended to address these shortcomings: (1) a novel process measure 'normative path deviations' is introduced that is useful for both quantitative and qualitative usability studies and (2) a longitudinal, completely within-subject study design is presented that assesses learnability, training effects and allows analysis of diversity of users. A reference regression model is introduced to analyze data from this and similar studies, drawing upon generalized linear mixed-effects models and a Bayesian estimation approach. The extended protocol is implemented and demonstrated in a study comparing a novel syringe infusion pump prototype to an existing design with a sample of 25 healthcare professionals. Strong performance differences between designs were observed with a variety of usability measures, as well as varying training-on-the-job effects. We discuss our findings with regard to validation testing guidelines, reflect on the extensions and discuss the perspectives they add to the validation process.

Network measures for characterising team adaptation processes.

The aim of this study was to advance the conceptualisation of team adaptation by applying social network analysis (SNA) measures in a field study of a paediatric cardiac surgical team adapting to changes in task complexity and ongoing dynamic complexity. Forty surgical procedures were observed by trained human factors researchers, and communication processes amongst team members were recorded. Focusing on who talked to whom, team communication structures, in response to changing task demands, were characterised by various network measures. Results showed that in complex procedures, the communication patterns were more decentralised and flatter. Also, in critical transition phases of the procedure, communication was characterised by higher information sharing and participation. We discuss implications for team adaptation theory and teamwork observation methods. PRACTITIONER SUMMARY: The reasons for this study were to advance our conceptualisation of team adaptation processes and to further quantify team observation methods. We found that the surgical team studied adapted to complexity of surgical procedures by adopting flatter communication patterns. We quantified team observation methods by applying SNA techniques.

With how many users should you test a medical infusion pump? Sampling strategies for usability tests on high-risk systems.

Usability testing is recognized as an effective means to improve the usability of medical devices and prevent harm for patients and users. Effectiveness of problem discovery in usability testing strongly depends on size and representativeness of the sample. We introduce the late control strategy, which is to continuously monitor effectiveness of a study towards a preset target. A statistical model, the LNB(zt) model, is presented, supporting the late control strategy. We report on a case study, where a prototype medical infusion pump underwent a usability test with 34 users. On the data obtained in this study, the LNB(zt) model is evaluated and compared against earlier prediction models. The LNB(zt) model fits the data much better than previously suggested approaches and improves prediction. We measure the effectiveness of problem identification, and observe that it is lower than is suggested by much of the literature. Larger sample sizes seem to be in order. In addition, the testing process showed high levels of uncertainty and volatility at small to moderate sample sizes, partly due to users' individual differences. In reaction, we propose the idiosyncrasy score as a means to obtain representative samples. Statistical programs are provided to assist practitioners and researchers in applying the late control strategy.

Seizure duration predicts postictal electroencephalographic recovery after electroconvulsive therapy-induced seizures.

OBJECTIVE: We aim to provide a quantitative description of the relation between seizure duration and the postictal state using features extracted from the postictal electroencephalogram (EEG). METHODS: Thirty patients with major depressive disorder treated with electroconvulsive therapy (ECT) were studied with continuous EEG before, during, and after ECT-induced seizures. EEG recovery was quantified as the spectral difference between postictal and baseline EEG using the temporal brain symmetry index (BSI). The postictal temporal EEG evolution was modeled with a single exponential. The parameters of the model, including the time constant τ, describe the change and speed of postictal EEG recovery. The change from baseline EEG at t = 60 minutes post-seizure (ΔBSI) was calculated from the exponential fit. Postictal clinical reorientation time (ROT) was clinically established. A multivariate generalized multi-level Bayesian model was estimated with seizure duration and ROT as predictors of τ and ΔBSI. RESULTS: EEG features of 290 seizures and postictal states were used for analyses. The model faithfully described the dynamics of the postictal EEG in nearly all patients. Seizure duration was associated with the recovery time constant, τ, and ΔBSI. ROT was associated with τ, but not with ΔBSI. CONCLUSIONS: Longer seizures are associated with slower postictal EEG recovery and more enduring EEG changes compared to baseline. SIGNIFICANCE: Quantitative EEG allows objective assessment of the postictal state.