Testing the reliability of accident analysis methods: a comparison of AcciMap, STAMP-CAST and AcciNet.

Accident analysis methods are used to model the multifactorial cause of adverse incidents. Methods such as AcciMap, STAMP-CAST and recently AcciNet, are systemic approaches that support the identification of safety interventions across sociotechnical system levels. Despite their growing popularity, little is known about how reliable systems-based methods are when used to describe, model and classify contributory factors and relationships. Here, we conducted an intra-rater and inter-rater reliability assessment of AcciMap, STAMP-CAST and AcciNet using the Signal Detection Theory (SDT) paradigm. A total of 180 hours' worth of analyses across 360 comparisons were performed by 30 expert analysts. Findings revealed that all three methods produced a weak to moderate positive correlation coefficient, however the inter-rater reliability of STAMP-CAST was significantly higher compared to AcciMap and AcciNet. No statistically significant or practically meaningful differences were found between methods in the overall intra-rater reliability analyses. Implications and future research directions are discussed.

Practitioners who undertake accident analysis within their organisations should consider the use of STAMP-CAST due to the significantly higher inter-rater reliability findings obtained in this study compared to AcciMap and AcciNet, particularly if they tend to work alone and/or part of relatively small teams.

The Chartered Institute of Ergonomics and Human Factors at 75: perspectives on contemporary challenges and future directions for Ergonomics and Human Factors.

As the UK's Chartered Institute of Ergonomics and Human Factors (CIEHF) celebrates its 75th anniversary, it is worth reflecting on our discipline's contribution, current state, and critical future endeavours. We present the perspectives of 18 EHF professionals who were asked to respond to five questions regarding the impact of EHF, contemporary challenges, and future directions. Co-authors were in agreement that EHF's impact has been only limited to date and that critical issues require resolution, such as increasing the number of suitably qualified practitioners, resolving the research-practice gap, and increasing awareness of EHF and its benefits. Frequently discussed future directions include advanced emerging technologies such as artificial intelligence, the development of new EHF methods, and enhancing the quality and reach of education and training. The majority felt there will be a need for EHF in 75 years; however, many noted that our methods will need to adapt to meet new needs.Practitioner statement: This article provides the perspectives of 18 Ergonomics and Human Factors (EHF) professionals on the impact of EHF, contemporary challenges and critical future directions, and changes that are necessary to ensure EHF remains relevant in future. As such, it provides important guidance on future EHF research and practice.

Towards a unified model of accident causation: refining and validating the systems thinking safety tenets.

The systems thinking tenets were developed based on a synthesis of contemporary accident causation theory, models and approaches and encapsulate 15 features of complex systems that interact to create both safety and adverse events. Whilst initial testing provided supportive evidence, the tenets have not yet been subject to formal validation. This article presents the findings from a three-round Delphi study undertaken to refine and validate the tenets and assess their suitability for inclusion in a unified model of accident causation. Participants with expertise in accident causation and systems thinking provided feedback on the tenets and associated definitions until an acceptable level of consensus was achieved. The results reduced the original 15 tenets to 14 and 10 were identified as important to include in unified model of accident causation. The refined systems thinking tenets are presented along with future research directions designed to facilitate their use in safety practice.Practitioner summary: This article presents a refined and validated set of systems thinking tenets which describe features of complex systems that interact to create adverse events. The tenets can be used by practitioners to proactively identify safety leading indicators and contributory factors during adverse event analysis.

'Over to you': using a STAMP control structure analysis to probe deeper into the control of UK road safety at a municipal level - the case of Cambridgeshire.

The UK has seen little progress in reducing road death over the last decade and as a result, the government has been criticised by industry stakeholders for a lack of leadership, including the removal of national targets in 2011 and the devolution of powers to a municipal level. The aim of this paper is to understand how decision-making at a municipal level takes place from a systems perspective, using the case study of Cambridgeshire. Actors involved were mapped using a STAMP control structure analysis and highlighted a key role for formal and informal partnerships between local and national government agencies and non-government organisations at the same level in the control structure. The changing international context of the model for the UK is also discussed in relation to the UK's withdrawal from the European Union and provides a useful tool for future analysis of its effect on policy and decision-making. Practitioner summary: This paper uses a STAMP control structure analysis to understand how decision-making at a municipal level takes place from a systems perspective, using the case study of Cambridgeshire. It highlights a key role for formal and informal partnerships between organisations at the same level in the control structure. Abbreviations: STAMP: System Theoretic Accident Model and Processes; STAMP-CAST: Systems-Theoretic Accident Model and Processes - Causal Analysis using Systems Theory; FRAM: Functional Resonance Analysis Method; HFACS: Human Factors Analysis and Classification Scheme; NGO: Non-Government organisation; iRAP: International Road Assessment Programme; EuroRAP: European Road Assessment Programme; NCAP: New Car Assessment Programme; CCG: Clinical Commissioning Group; GP: General Practitioner; PPE: Personal Protective Equipment; SD: standard deviation; Beds: Bedfordshire; Cambs: Cambridgeshire; Herts: Hertfordshire; Pboro: Peterborough.

Testing the reliability and validity of risk assessment methods in Human Factors and Ergonomics.

There is growing interest in the use of systems-based risk assessment methods in Human Factors and Ergonomics (HFE). The purpose of this study was to test the intra-rater reliability and criterion-referenced concurrent validity of three systems-based risk assessment approaches: (i) the Systems-Theoretic Process Analysis (STPA) method; (ii) the Event Analysis of Systemic Teamwork Broken Links (EAST-BL) method; and, (iii) the Network Hazard Analysis and Risk Management System (Net-HARMS) method. Reliability and validity measures were obtained using the Signal Detection Theory (SDT) paradigm. Whilst STPA identified the highest number of risks, the findings indicate a weak to moderate level of reliability and validity for STPA, EAST-BL and Net-HARMS. There were no statistically significant differences between the methods across analyses. The results suggest that there is merit to the continued use of systems-based risk assessment methods following a series of methodological extensions that aim to enhance the reliability and validity of future applications. Practitioner summary The three risk assessment methods produced weak to moderate levels of stability and accuracy regarding their capability to predict risks. There is a pressing need to further test the reliability and validity of safety methods in Human Factors and Ergonomics.

Complexity theory in accident causation: using AcciMap to identify the systems thinking tenets in 11 catastrophes.

The quest to explain and understand the cause of accidents is both ever-present and ongoing amongst the safety science community. In an attempt to advance the theory and science of accident causation, researchers have recently formalised a set of '15 systems thinking tenets' that cover the conditions and characteristics of work systems that are believed to contribute to the cause of accidents. The purpose of this study was to attempt to identify the systems thinking tenets across a range of different systems and accidents using the Accident Mapping (AcciMap) method. The findings suggest that the tenets can be attributed to play a role in accident causation, however as a result of this process, the capability of AcciMap has been brought into question. Implications and directions for future research are described. Practitioner Summary: This study is an extension of previous work that suggested there was a need to test for the 'systems thinking tenets of accident causation' in a multi-incident dataset. We used AcciMap to evaluate whether it has the capability to support ongoing accident analysis activities in ergonomics research.

Human factors and ergonomics systems-based tools for understanding and addressing global problems of the twenty-first century.

Sustainability is a systems problem with humans as integral elements of the system. However, sustainability problems usually have a broader scope than socio-technical systems and therefore, require additional considerations. This requires a fuller integration of complex systems understanding into the systems analysis toolset currently available to human factors and ergonomics. In this paper, we outline these complex systems requirements necessary to tackle global problems such as sustainability and then assess how three common systems analysis tools (i.e. Accimap, System Theoretic Accident Mapping and Processes, and Cognitive Work Analysis) stand up against these revised criteria. This assessment is then further explored through applying two of these tools (i.e. Accimap and System Theoretic Accident Mapping and Processes) to a transnational food integrity system problem. This case study shows that no single systems analysis method can be used in isolation to help identify key insights for intervention and that new methods may need to be developed or existing methods need to be adapted to understand these dynamic, adaptive systems. The implications for the further development of systems analysis tools are discussed. Practitioner summary: We assess the applicability of existing human factors and ergonomics systems-analysis tools for examining global problems and for identifying points to intervene in these systems. We comment on what extensions and further work will be required to enable human factors and ergonomics to intervene effectively. Abbreviations: HFE: human factors and ergonomics; CO2: carbon dioxide; CO: carbon monoxide; O3: ozone; SSoS: sustainable system-of-systems; BSE: bovine spongiform encephalopathy; STAMP: systems-theoretic accident model; CWA: cognitive work analysis; WDA: work domain analysis; ConTA: control task analysis; StrA: strategies analysis; SOCA: social and organisation cooperation analysis; SOCA-CAT: social and organisation cooperation analysis contextual analysis template; SOCA-DL: social and organisation cooperation analysis decision ladder; WOP: work organisation possibilities; FRAM: functional resonance analysis method; US FDA: United States Food and Drug Administration; UK: United Kingdom; NET-HARMS: networked hazard analysis and risk management system; PreMiSTS: predicting malfunctions in socio-technical systems.

State of Science: ergonomics and global issues.

In his 1993 IEA keynote address, Neville Moray urged the ergonomics discipline to face up to the global problems facing humanity and consider how ergonomics might help find some of the solutions. In this State of Science article we critically evaluate what the ergonomics discipline has achieved in the last two and a half decades to help create a secure future for humanity. Moray's challenges for ergonomics included deriving a value structure that moves us beyond a Westernised view of worker-organisation-technology fit, taking a multidisciplinary approach which engages with other social and biological sciences, considering the gross cross-cultural factors that determine how different societies function, paying more attention to mindful consumption, and embracing the complexity of our interconnected world. This article takes a socio-historical approach by considering the factors that influence what has been achieved since Moray's keynote address. We conclude with our own set of predictions for the future and priorities for addressing the challenges that we are likely to face. Practitioner Summary: We critically reflect on what has been achieved by the ergonomics profession in addressing the global challenges raised by Moray's 1993 keynote address to the International Ergonomics Association. Apart from healthcare, the response has largely been weak and disorganised. We make suggestions for priority research and practice that is required to facilitate a sustainable future for humanity.

Defining the methodological challenges and opportunities for an effective science of sociotechnical systems and safety.

An important part of the application of sociotechnical systems theory (STS) is the development of methods, tools and techniques to assess human factors and ergonomics workplace requirements. We focus in this paper on describing and evaluating current STS methods for workplace safety, as well as outlining a set of six case studies covering the application of these methods to a range of safety contexts. We also describe an evaluation of the methods in terms of ratings of their ability to address a set of theoretical and practical questions (e.g. the degree to which methods capture static/dynamic aspects of tasks and interactions between system levels). The outcomes from the evaluation highlight a set of gaps relating to the coverage and applicability of current methods for STS and safety (e.g. coverage of external influences on system functioning; method usability). The final sections of the paper describe a set of future challenges, as well as some practical suggestions for tackling these. PRACTITIONER SUMMARY: We provide an up-to-date review of STS methods, a set of case studies illustrating their use and an evaluation of their strengths and weaknesses. The paper concludes with a 'roadmap' for future work.

Sociotechnical approaches to workplace safety: Research needs and opportunities.

The sociotechnical systems perspective offers intriguing and potentially valuable insights into problems associated with workplace safety. While formal sociotechnical systems thinking originated in the 1950s, its application to the analysis and design of sustainable, safe working environments has not been fully developed. To that end, a Hopkinton Conference was organised to review and summarise the state of knowledge in the area and to identify research priorities. A group of 26 international experts produced collaborative articles for this special issue of Ergonomics, and each focused on examining a key conceptual, methodological and/or theoretical issue associated with sociotechnical systems and safety. In this concluding paper, we describe the major conference themes and recommendations. These are organised into six topic areas: (1) Concepts, definitions and frameworks, (2) defining research methodologies, (3) modelling and simulation, (4) communications and decision-making, (5) sociotechnical attributes of safe and unsafe systems and (6) potential future research directions for sociotechnical systems research. PRACTITIONER SUMMARY: Sociotechnical complexity, a characteristic of many contemporary work environments, presents potential safety risks that traditional approaches to workplace safety may not adequately address. In this paper, we summarise the investigations of a group of international researchers into questions associated with the application of sociotechnical systems thinking to improve worker safety.

The use of Bayesian Belief Networks (BBNs) to probe deeper into railway safety management systems - Two studies from Great Britain and Italy.

The importance of Safety Management Systems (SMS) to the railway industry is underlined by the fact that all organisations operating on UK railways are required by law to have one. Analysing SMSs can provide a reliable systemic tool to identify hazards and weaknesses within complex systems like the railway, making it possible to significantly increase safety, reducing the odds of near misses and accidents. However, there is little empirical research evidence to determine the impact on safety of a structured SMS. The current paper describes two studies which use Bayesian Belief Networks (BBN) to conceptualise SMSs and their impact on front-line performance. The paper presents the usefulness of BBNs to compare complex systems and reconcile cultural differences within the railway industry, identifying factors that are deemed vital within Italy and Britain. The two studies allowed us to identify the most influential factors within a SMS and how they interact with each other, as well as the strength of the identified relationships. A BBN is particularly useful in estimating how changing some of the node states (e.g., by making safety leadership present) affected the other factors. The current study showed that safety leadership has an impact on the SMSs of the British and Italian railway industries.

Bottom-up and middle-out approaches to electronic patient information systems: a focus on healthcare pathways.

BACKGROUND: A study is reported that examines the use of electronic health record (EHR) systems in two UK local health communities. OBJECTIVE: These systems were developed locally and the aim of the study was to explore how well they were supporting the coordination of care along healthcare pathways that cross the organisational boundaries between the agencies delivering health care. RESULTS: The paper presents the findings for two healthcare pathways; the Stroke Pathway and a pathway for the care of the frail elderly in their own homes. All the pathways examined involved multiple agencies and many locally tailored EHR systems are in use to aid the coordination of care. However, the ability to share electronic patient information along the pathways was patchy. The development of systems that did enable effective sharing of information was characterised by sociotechnical system development, i.e. associating the technical development with process changes and organisational changes, with local development teams that drew on all the relevant agencies in the local health community and on evolutionary development, as experience grew of the benefits that EHR systems could deliver. CONCLUSIONS: The study concludes that whilst there may be a role for a national IT strategy, for example, to set standards for systems procurement that facilitate data interchange, most systems development work needs to be done at a 'middle-out' level in the local health community, where joint planning between healthcare agencies can occur, and at the local healthcare pathway level where systems can be matched to specific needs for information sharing.

World War II and other historical influences on the formation of the Ergonomics Research Society.

Little has been written about wartime ergonomics and the role this played in prompting the need for a society dedicated to ergonomics within the UK, namely the formation of the Ergonomics Research Society (ERS) in early 1950. This article aims to fill this gap in our understanding of the history of ergonomics in the UK and provide further details of the types of research undertaken by wartime research groups and committees such as the Institute of Aviation Medicine, Medical Research Council Applied Psychology Unit and the Flying Personnel Research Committee. In addition, the role of societal developments such as wartime links with the USA, the post-war drive to increase productivity and collaboration with industry and the recommendations of government committees in stimulating the work of the ERS are described in detail. This article also offers some reflection on present-day ergonomics in the UK and how this contrasts with the past. STATEMENT OF RELEVANCE: This article will provide practitioners with a historical perspective on the development of ergonomics from its roots in the Second World War. These developments shed light on current trends and challenges within the discipline as a whole.

Facilitators and barriers of care transitions - Comparing the perspectives of hospital and community healthcare staff.

As a patient moves from hospital back home to receive community-based care, quality of care and patient safety are often put at risk. This study aimed to analyse the discharge process to identify and compare the barriers and facilitators within the context of the system in which they occur, from the perspectives of both hospital and community healthcare staff. The results were derived from the analysis of 348 incident reports, the observation of five discharge planning meetings with hospital staff, three focus groups with hospital staff, and six focus groups with community healthcare staff. Five themes representative of the barriers and four themes representative of the facilitators for this process were identified from both hospital and community healthcare staff's perspective. These were then discussed in the context of the subsystem, hospital or community healthcare setting, in which they occur.

Systems thinking-based risk assessment methods applied to sports performance: A comparison of STPA, EAST-BL, and Net-HARMS in the context of elite women's road cycling.

There is increasing interest in applying systems Human Factors and Ergonomics (HFE) methods in sport. Risk assessment (RA) methods can be used identify risks which may impact the performance of individual athletes, teams, and overall sports systems; however, they have not yet been tested in sport. This study sets out to apply and compare three systems thinking-based RA methods in the context of elite sports performance and report on the frequency and types of the risks identified. The Systems-Theoretic Process Analysis (STPA) method, the Event Analysis of Systemic Teamwork Broken Links (EAST-BL) method, and the Networked Hazard Analysis and Risk Management System (Net-HARMS) method were applied to elite women's road cycling to identify all the credible risks that could degrade optimal team performance. The findings demonstrate that all three methods appear to provide useful results in a context other than safety, and that multiple risks threatening the performance of the cycling team were identified. Whilst the frequency and types of risks differed across the methods applied, there are additional theoretical, methodological, and practical implications to be considered prior to the selection and use of systems thinking-based RA approaches. Recommendations and directions for future HFE and sports science research are discussed.