Adopting human factors in early phase and experimental medicine research: A nested pilot study observing controlled human infection with SARS-CoV-2.

AIMS: The influence of human factors on safety in healthcare settings is well established, with targeted interventions reducing risk and enhancing team performance. In experimental and early phase clinical research participant safety is paramount and safeguarded by guidelines, protocolized care and staff training; however, the real-world interaction and implementation of these risk-mitigating measures has never been subjected to formal system-based assessment. METHODS: Independent structured observations, systematic review of study documents, and interviews and focus groups were used to collate data on three key tasks undertaken in a clinical research facility (CRF) during a SARS CoV-2 controlled human infection model (CHIM) study. The Systems Engineering Initiative for Patient Safety (SEIPS) was employed to analyse and categorize findings, and develop recommendations for safety interventions. RESULTS: High levels of team functioning and a clear focus on participant safety were evident throughout the study. Despite this, latent risks in both study-specific and CRF work systems were identified in all four SEIPS domains (people, environment, tasks and tools). Fourteen actionable recommendations were generated collaboratively. These included inter-organization and inter-study standardization, optimized checklists for safety critical tasks, and use of simulation for team training and exploration of work systems. CONCLUSIONS: This pioneering application of human factors techniques to analyse work systems during the conduct of research in a CRF revealed risks unidentified by routine review and appraisal, and despite international guideline adherence. SEIPS may aid categorization of system problems and the formulation of recommendations that reduce risk and mitigate potential harm applicable across a trials portfolio.

Patient Harm and Institutional Avoidability of Out-of-Hours Discharge From Intensive Care: An Analysis Using Mixed Methods.

OBJECTIVES: Out-of-hours discharge from ICU to the ward is associated with increased in-hospital mortality and ICU readmission. Little is known about why this occurs. We map the discharge process and describe the consequences of out-of-hours discharge to inform practice changes to reduce the impact of discharge at night. DESIGN: This study was part of the REcovery FoLlowing intensive CarE Treatment mixed methods study. We defined out-of-hours discharge as 16:00 to 07:59 hours. We undertook 20 in-depth case record reviews where in-hospital death after ICU discharge had been judged "probably avoidable" in previous retrospective structured judgment reviews, and 20 where patients survived. We conducted semistructured interviews with 55 patients, family members, and staff with experience of ICU discharge processes. These, along with a stakeholder focus group, informed ICU discharge process mapping using the human factors-based functional analysis resonance method. SETTING: Three U.K. National Health Service hospitals, chosen to represent different hospital settings. SUBJECTS: Patients discharged from ICU, their families, and staff involved in their care. INTERVENTIONS: None. MEASUREMENTS AND MAIN RESULTS: Out-of-hours discharge was common. Patients and staff described out-of-hours discharge as unsafe due to a reduction in staffing and skill mix at night. Patients discharged out-of-hours were commonly discharged prematurely, had inadequate handover, were physiologically unstable, and did not have deterioration recognized or escalated appropriately. We identified five interdependent function keys to facilitating timely ICU discharge: multidisciplinary team decision for discharge, patient prepared for discharge, bed meeting, bed manager allocation of beds, and ward bed made available. CONCLUSIONS: We identified significant limitations in out-of-hours care provision following overnight discharge from ICU. Transfer to the ward before 16:00 should be facilitated where possible. Our work highlights changes to help make day time discharge more likely. Where discharge after 16:00 is unavoidable, support systems should be implemented to ensure the safety of patients discharged from ICU at night.

Development and evaluation of an electronic hospital referral system: a human factors approach.

Coordinating care across hospitals has been identified as a patient safety risk as referrals are often paper-based and poorly documented. Electronic referral systems have the potential to improve the situation but can fail to gain uptake. We applied a human factors/ergonomics (HFE) approach to place analysis of local workflow and user engagement central to the development of a new regional electronic referral system. The intervention was evaluated with a before-and-after study. Referral quality improved, referrals containing sufficient clinical information for continuation of care increased from 36.9% to 83.5% and completeness of referral information significantly improved. There was a 35.7% reduction in the number of calls to the on-call specialist, and the mean period between admission and surgery for expedited transfers was reduced. Applying HFE informed design with use-based evidence; the system maintains sustained uptake three years after implementation. Reliable recording of information translates to better patient safety during inter-hospital transitions. Practitioners summary: This study developed, implemented and evaluated a clinical referral system using a human factors approach. Process analysis and usability studies were used to inform the application requirements and design. Region-wide implementation in hospitals resulted in the improved quality and completeness of clinical referral information and efficiencies in the referral process.

Combining Systems and Teamwork Approaches to Enhance the Effectiveness of Safety Improvement Interventions in Surgery: The Safer Delivery of Surgical Services (S3) Program.

IMPORTANCE: Patient safety improvement interventions usually address either work systems or team culture. We do not know which is more effective, or whether combining approaches is beneficial. OBJECTIVE: To compare improvement in surgical team performance after interventions addressing teamwork culture, work systems, or both. DESIGN: Suite of 5 identical controlled before-after intervention studies, with preplanned analysis of pooled data for indirect comparisons of strategies. SETTING: Operating theatres in 5 UK hospitals performing elective orthopedic, plastic, or vascular surgery PARTICIPANTS:: All operating theatres staff, including surgeons, nurses, anaesthetists, and others INTERVENTIONS:: 4-month safety improvement interventions, using teamwork training (TT), systems redesign and standardization (SOP), Lean quality improvement, SOP + TT combination, or Lean + TT combination. MAIN OUTCOMES AND MEASURES: Team technical and nontechnical performance and World Health Organization (WHO) checklist compliance, measured for 3 months before and after intervention using validated scales. Pooled data analysis of before-after change in active and control groups, comparing combined versus single and systems versus teamwork interventions, using 2-way ANOVA. RESULTS: We studied 453 operations, (255 intervention, 198 control). TT improved nontechnical skills and WHO compliance (P < 0.001), but not technical performance; systems interventions (Lean & SOP, 2 & 3) improved nontechnical skills and technical performance (P < 0.001) but improved WHO compliance less. Combined interventions (4 & 5) improved all performance measures except WHO time-out attempts, whereas single approaches (1 & 2 & 3) improved WHO compliance less (P < 0.001) and failed to improve technical performance. CONCLUSIONS & RELEVANCE: Safety interventions combining teamwork training and systems rationalization are more effective than those adopting either approach alone. This has important implications for safety improvement strategies in hospitals.

Intentional Rounding: a staff-led quality improvement intervention in the prevention of patient falls.

AIMS AND OBJECTIVES: This study designed and evaluated the use of a specific implementation strategy to deliver a nursing staff-led Intentional Rounding intervention to reduce inpatient falls. BACKGROUND: Patient falls are a common cause of harm during hospital treatment. Intentional Rounding has been proposed as a potential strategy for prevention, but has not received much objective evaluation. Previous work has suggested that logical interventions to improve patient care require an integrated implementation strategy, using teamwork training and systems improvement training, to instigate positive change and improvement. METHODS: Customised Intentional Rounding was implemented and evaluated as part of a staff-led quality improvement intervention to reduce falls on a neuroscience ward. Intentional Rounding was instigated using a prespecified implementation strategy, which comprised of: (1) engagement and communication activities, (2) teamwork and systems improvement training, (3) support and coaching and (4) iterative Plan-Do-Check-Act cycles. Process (compliance with hourly visiting to patients by staff) and outcome (incidence of falls) measures were recorded pre- and postintervention. Falls measured on the active ward were compared with incidence of falls in 50 wards across the rest of the same Trust. RESULTS: There was a 50% reduction in patient falls on the active ward vs. a minimal increase across the rest of the Trust (3·48%). Customised Intentional Rounding, designed by staff specifically for the context, appeared to be effective in reducing patient falls. CONCLUSIONS: Improvement programmes based on integrating teamwork training and staff-led systems redesign, together with a preplanned implementation strategy, can deliver effective change and improvement. RELEVANCE TO CLINICAL PRACTICE: This study demonstrates, through the implementation of a specific strategy, an effective improvement intervention to reduce patient falls. It provides insight into the effective design and practical implementation of integrated improvement programmes to reduce risk to patients at the frontline.

The Safer Delivery of Surgical Services Program (S3): Explaining Its Differential Effectiveness and Exploring Implications for Improving Quality in Complex Systems.

OBJECTIVE: To analyze the challenges encountered during surgical quality improvement interventions, and explain the relative success of different intervention strategies. SUMMARY BACKGROUND DATA: Understanding why and how interventions work is vital for developing improvement science. The S3 Program of studies tested whether combining interventions addressing culture and system was more likely to result in improvement than either approach alone. Quantitative results supported this theory. This qualitative study investigates why this happened, what aspects of the interventions and their implementation most affected improvement, and the implications for similar programs. METHODS: Semistructured interviews were conducted with hospital staff (23) and research team members (11) involved in S3 studies. Analysis was based on the constant comparative method, with coding conducted concurrently with data collection. Themes were identified and developed in relation to the program theory behind S3. RESULTS: The superior performance of combined intervention over single intervention arms appeared related to greater awareness and ability to act, supporting the S3 hypothesis. However, we also noted unforeseen differences in implementation that seemed to amplify this difference. The greater ambition and more sophisticated approach in combined intervention arms resulted in requests for more intensive expert support, which seemed crucial in their success. The contextual challenges encountered have potential implications for the replicability and sustainability of the approach. CONCLUSIONS: Our findings support the S3 hypothesis, triangulating with quantitative results and providing an explanatory account of the causal relationship between interventions and outcomes. They also highlight the importance of implementation strategies, and of factors outside the control of program designers.

SEND: a system for electronic notification and documentation of vital sign observations.

BACKGROUND: Recognising the limitations of a paper-based approach to documenting vital sign observations and responding to national clinical guidelines, we have explored the use of an electronic solution that could improve the quality and safety of patient care. We have developed a system for recording vital sign observations at the bedside, automatically calculating an Early Warning Score, and saving data such that it is accessible to all relevant clinicians within a hospital trust. We have studied current clinical practice of using paper observation charts, and attempted to streamline the process. We describe our user-focussed design process, and present the key design decisions prior to describing the system in greater detail. RESULTS: The system has been deployed in three pilot clinical areas over a period of 9 months. During this time, vital sign observations were recorded electronically using our system. Analysis of the number of observations recorded (21,316 observations) and the number of active users (111 users) confirmed that the system is being used for routine clinical observations. Feedback from clinical end-users was collected to assess user acceptance of the system. This resulted in a System Usability Scale score of 77.8, indicating high user acceptability. CONCLUSIONS: Our system has been successfully piloted, and is in the process of full implementation throughout adult inpatient clinical areas in the Oxford University Hospitals. Whilst our results demonstrate qualitative acceptance of the system, its quantitative effect on clinical care is yet to be evaluated.

A survey of expert opinion on the effects of occupational exposures to trunk rotation and whole-body vibration.

We present a review of current expert opinion on the effects of combined exposures to trunk rotation and whole-body vibration (WBV), commonly experienced by operators of agricultural machinery. We evaluate the level of agreement between academic experts in the field of ergonomics, human response to WBV and agricultural operators, on the effects of exposure to WBV and trunk rotation. A total of 83 individuals responded to the paper-based questionnaire, which included questions on risk levels from individual and combined exposures, discomfort development, exposure duration limits and tasks within agriculture. The results showed that all groups considered exposure to WBV and trunk rotation as risk factors for the development of back pain. The experts were not in consensus regarding acceptable exposure durations, areas of discomfort experienced or recommendations for cab developments.

Managing Complexity: Black Older Adults With Multimorbidity.

BACKGROUND AND OBJECTIVES: Black older adults have higher rates of multimorbidity and receive less effective multimorbidity support than their white counterparts. Yet little is known about the experiences of Black older adults with multimorbidity that may be at the heart of those disparities and which are central to interventions and improving care for this population. In this study, we aimed to conceptualize the multimorbidity management (MM) experience for Black older adults. RESEARCH DESIGN AND METHODS: As part of a larger study on Black older adults' multimorbidity and physician empathy, we conducted in-depth qualitative interviews with 30 Black older adults living in a large midwestern city in the United States aged 65 years and older with self-reported multimorbidity. We used grounded theory analysis to distill findings into a core conceptual category as well as component domains and dimensions. RESULTS: "Managing complexity" emerged as the core category to describe MM in our sample. Managing complexity included domains of "social context," "daily logistics," "care time," and "care roles." DISCUSSION AND IMPLICATIONS: We discuss how managing complexity is distinct from patient complexity and how it is related to cumulative inequality and precarity. Study findings have potential implications for intervention around provider education and empathy as well as for enabling agency of Black older adults with MM.

Reporting guideline for the early-stage clinical evaluation of decision support systems driven by artificial intelligence: DECIDE-AI.

A growing number of artificial intelligence (AI)-based clinical decision support systems are showing promising performance in preclinical, in silico evaluation, but few have yet demonstrated real benefit to patient care. Early-stage clinical evaluation is important to assess an AI system's actual clinical performance at small scale, ensure its safety, evaluate the human factors surrounding its use and pave the way to further large-scale trials. However, the reporting of these early studies remains inadequate. The present statement provides a multi-stakeholder, consensus-based reporting guideline for the Developmental and Exploratory Clinical Investigations of DEcision support systems driven by Artificial Intelligence (DECIDE-AI). We conducted a two-round, modified Delphi process to collect and analyze expert opinion on the reporting of early clinical evaluation of AI systems. Experts were recruited from 20 pre-defined stakeholder categories. The final composition and wording of the guideline was determined at a virtual consensus meeting. The checklist and the Explanation & Elaboration (E&E) sections were refined based on feedback from a qualitative evaluation process. In total, 123 experts participated in the first round of Delphi, 138 in the second round, 16 in the consensus meeting and 16 in the qualitative evaluation. The DECIDE-AI reporting guideline comprises 17 AI-specific reporting items (made of 28 subitems) and ten generic reporting items, with an E&E paragraph provided for each. Through consultation and consensus with a range of stakeholders, we developed a guideline comprising key items that should be reported in early-stage clinical studies of AI-based decision support systems in healthcare. By providing an actionable checklist of minimal reporting items, the DECIDE-AI guideline will facilitate the appraisal of these studies and replicability of their findings.

Systematic review of applied usability metrics within usability evaluation methods for hospital electronic healthcare record systems: Metrics and Evaluation Methods for eHealth Systems.

BACKGROUND AND OBJECTIVES: Electronic healthcare records have become central to patient care. Evaluation of new systems include a variety of usability evaluation methods or usability metrics (often referred to interchangeably as usability components or usability attributes). This study reviews the breadth of usability evaluation methods, metrics, and associated measurement techniques that have been reported to assess systems designed for hospital staff to assess inpatient clinical condition. METHODS: Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) methodology, we searched Medline, EMBASE, CINAHL, Cochrane Database of Systematic Reviews, and Open Grey from 1986 to 2019. For included studies, we recorded usability evaluation methods or usability metrics as appropriate, and any measurement techniques applied to illustrate these. We classified and described all usability evaluation methods, usability metrics, and measurement techniques. Study quality was evaluated using a modified Downs and Black checklist. RESULTS: The search identified 1336 studies. After abstract screening, 130 full texts were reviewed. In the 51 included studies 11 distinct usability evaluation methods were identified. Within these usability evaluation methods, seven usability metrics were reported. The most common metrics were ISO9241-11 and Nielsen's components. An additional "usefulness" metric was reported in almost 40% of included studies. We identified 70 measurement techniques used to evaluate systems. Overall study quality was reflected in a mean modified Downs and Black checklist score of 6.8/10 (range 1-9) 33% studies classified as "high-quality" (scoring eight or higher), 51% studies "moderate-quality" (scoring 6-7), and the remaining 16% (scoring below five) were "low-quality." CONCLUSION: There is little consistency within the field of electronic health record systems evaluation. This review highlights the variability within usability methods, metrics, and reporting. Standardized processes may improve evaluation and comparison electronic health record systems and improve their development and implementation.

A single-tube quantitative assay for mRNA levels of hormonal and growth factor receptors in breast cancer specimens.

Knowledge of estrogen receptor (ER) and progesterone receptor (PR) status has been critical in the evolution of modern targeted therapy of breast cancer and remains essential for making informed therapeutic decisions. Recently, growth factor receptor HER2/neu (ERBB2) status has made it possible to provide another form of targeted therapy linked to the overexpression of this protein. Presently, pathologists determine the receptor status in formalin-fixed, paraffin-embedded sections using subjective, semiquantitative immunohistochemistry (IHC) assays and quantitative fluorescence in situ hybridization for HER2. We developed a single-tube multiplex TaqMan (mERPR+HER2) assay to quantitate mRNA levels of ER, PR, HER2, and two housekeeping genes for breast cancer formalin-fixed, paraffin-embedded sections. Using data from the discovery sample sets, we evaluated IHC-status-dependent cutoff-point and IHC-status-independent clustering methods for the classification of receptor status and then validated these results with independent sample sets. Compared with IHC-status, the accuracies of the mERPR+HER2 assay with the cutoff-point classification method were 0.98 (95% CI: 0.97-1.00), 0.92 (95% CI: 0.88-0.95), and 0.97 (95% CI: 0.95-0.99) for ER, PR, and HER2, respectively, for the validation sets. Furthermore, the areas under the receiver operating-characteristic curves were 0.997 (95% CI: 0.994-1.000), 0.967 (95% CI: 0.949-0.985), and 0.968 (95% CI: 0.915-1.000) for ER, PR, and HER2, respectively. This multiplex assay provides a sensitive and reliable method to quantitate hormonal and growth factor receptors.

Retrospective Attention in Short-Term Memory Has a Lasting Effect on Long-Term Memory Across Age.

OBJECTIVES: Declines in both short- and long-term memory are typical of healthy aging. Recent findings suggest that retrodictive attentional cues ("retro-cues") that indicate the location of to-be-probed items in short-term memory (STM) have a lasting impact on long-term memory (LTM) performance in young adults. Whether older adults can also use retro-cues to facilitate both STM and LTM is unknown. METHOD: Young and older adults performed a visual STM task in which spatially informative retro-cues or noninformative neutral-cues were presented during STM maintenance of real-world objects. We tested participants' memory at both STM and LTM delays for objects that were previously cued with retrodictive or neutral-cues during STM order to measure the lasting impact of retrospective attention on LTM. RESULTS: Older adults showed reduced STM and LTM capacity compared to young adults. However, they showed similar magnitude retro-cue memory benefits as young adults at both STM and LTM delays. DISCUSSION: To the best of our knowledge, this is the first study to investigate whether retro-cues in STM facilitate the encoding of objects into LTM such that they are more likely to be subsequently retrieved by older adults. Our results support the idea that retrospective attention can be an effective means by which older adults can improve their STM and LTM performance, even in the context of reduced memory capacity.

Protocol for a mixed-methods exploratory investigation of care following intensive care discharge: the REFLECT study.

INTRODUCTION: A substantial number of patients discharged from intensive care units (ICUs) subsequently die without leaving hospital. It is unclear how many of these deaths are preventable. Ward-based management following discharge from ICU is an area that patients and healthcare staff are concerned about. The primary aim of REFLECT (Recovery Following Intensive Care Treatment) is to develop an intervention plan to reduce in-hospital mortality rates in patients who have been discharged from ICU. METHODS AND ANALYSIS: REFLECT is a multicentre mixed-methods exploratory study examining ward care delivery to adult patients discharged from ICU. The study will be made up of four substudies. Medical notes of patients who were discharged from ICU and subsequently died will be examined using a retrospective case records review (RCRR) technique. Patients and their relatives will be interviewed about their post-ICU care, including relatives of patients who died in hospital following ICU discharge. Staff involved in the care of patients post-ICU discharge will be interviewed about the care of this patient group. The medical records of patients who survived their post-ICU stay will also be reviewed using the RCRR technique. The analyses of the substudies will be both descriptive and use a modified grounded theory approach to identify emerging themes. The evidence generated in these four substudies will form the basis of the intervention development, which will take place through stakeholder and clinical expert meetings. ETHICS AND DISSEMINATION: Ethical approval has been obtained through the Wales Research and Ethics Committee 4 (17/WA/0107). We aim to disseminate the findings through international conferences, international peer-reviewed journals and social media. TRIAL REGISTRATION NUMBER: ISRCTN14658054.

Risk estimation of distant metastasis in node-negative, estrogen receptor-positive breast cancer patients using an RT-PCR based prognostic expression signature.

BACKGROUND: Given the large number of genes purported to be prognostic for breast cancer, it would be optimal if the genes identified are not confounded by the continuously changing systemic therapies. The aim of this study was to discover and validate a breast cancer prognostic expression signature for distant metastasis in untreated, early stage, lymph node-negative (N-) estrogen receptor-positive (ER+) patients with extensive follow-up times. METHODS: 197 genes previously associated with metastasis and ER status were profiled from 142 untreated breast cancer subjects. A "metastasis score" (MS) representing fourteen differentially expressed genes was developed and evaluated for its association with distant-metastasis-free survival (DMFS). Categorical risk classification was established from the continuous MS and further evaluated on an independent set of 279 untreated subjects. A third set of 45 subjects was tested to determine the prognostic performance of the MS in tamoxifen-treated women. RESULTS: A 14-gene signature was found to be significantly associated (p < 0.05) with distant metastasis in a training set and subsequently in an independent validation set. In the validation set, the hazard ratios (HR) of the high risk compared to low risk groups were 4.02 (95% CI 1.91-8.44) for the endpoint of DMFS and 1.97 (95% CI 1.28 to 3.04) for overall survival after adjustment for age, tumor size and grade. The low and high MS risk groups had 10-year estimates (95% CI) of 96% (90-99%) and 72% (64-78%) respectively, for DMFS and 91% (84-95%) and 68% (61-75%), respectively for overall survival. Performance characteristics of the signature in the two sets were similar. Ki-67 labeling index (LI) was predictive for recurrent disease in the training set, but lost significance after adjustment for the expression signature. In a study of tamoxifen-treated patients, the HR for DMFS in high compared to low risk groups was 3.61 (95% CI 0.86-15.14). CONCLUSION: The 14-gene signature is significantly associated with risk of distant metastasis. The signature has a predominance of proliferation genes which have prognostic significance above that of Ki-67 LI and may aid in prioritizing future mechanistic studies and therapeutic interventions.

Improving emergency surgical care for patients with right iliac fossa pain at a regional scale: A quality improvement study using the Supported Champions implementation strategy.

INTRODUCTION: Methods to improve clinical systems safety suffer from significant difficulties in implementation and scaling up. We used an upscaling implementation strategy entitled Supported Champions in a quality and safety improvement programme for emergency surgery at regional level, focusing on patients with right iliac fossa pain. METHODS: A before-after study was conducted across four acute NHS Trusts: A 6 month intervention phase was preceded and followed by 3 months of data collection. An established Human Factors intervention was led at each Trust by a small group of staff selected as Champions. Champions received training in teamwork and systems improvement and were supported by Human Factors experts. The primary improvement aim was to expedite surgery for patients with sepsis, using Royal College of Surgeons emergency surgery guidelines as the measure. Additional outcomes studied included length of inpatient stay and 30-day readmission rates. RESULTS: Breaches of RCS urgency guidelines decreased markedly from 13.7% of operated patients pre-intervention to 3.5% post-intervention (p = 0.000). Mean time from booking to incision decreased in three of the four sites, whilst median length of stay increased in 3 of 4. Overall 30-day readmission rate remained stable (7.84% pre-intervention versus 7.31% post-intervention, p = 0.959). DISCUSSION: The Supported Champions model allowed all surgical teams to reduce delay for septic patients by more than 50%, using distinct Quality Improvement strategies to address local issues. Improvement was implemented in 4 diverse settings with a quarter of the level of expert input previously used in a single hospital.

Usability evaluation methods employed to assess information visualisations of electronically stored patient data for clinical use: a protocol for a systematic review.

BACKGROUND: The use of electronic records in healthcare is increasing. To avoid errors, it is essential that the data displays used by these systems are usable: efficient, effective and satisfying. A wide variety of display techniques are used to present clinical data, but the best methods to assess the usability of these techniques have not been determined. This systematic review will answer the question: What methods are employed to assess the usability of electronic visualisations of patient data for clinical use? The results of this systematic review will then be used to inform best assessment and design practice. METHODS: MEDLINE, EMBASE, CINAHL, OpenGrey, and the Cochrane Database of Systematic Reviews will be searched for original studies related to the usability of electronic information visualisations of patient data for clinical use. Reference lists of eligible studies and relevant reviews will be explored to identify further eligible studies. DISCUSSION: This systematic review will identify methods used to assess the usability of electronic information visualisations of patient data for clinical use. We will summarise the similarities and differences between the methods found. Our results will inform best practice when developing new user interfaces to display electronic patient data for clinical use. TRIAL REGISTRATION: PROSPERO CRD42016041604.