Testing the reliability of accident analysis methods: a comparison of AcciMap, STAMP-CAST and AcciNet.

Accident analysis methods are used to model the multifactorial cause of adverse incidents. Methods such as AcciMap, STAMP-CAST and recently AcciNet, are systemic approaches that support the identification of safety interventions across sociotechnical system levels. Despite their growing popularity, little is known about how reliable systems-based methods are when used to describe, model and classify contributory factors and relationships. Here, we conducted an intra-rater and inter-rater reliability assessment of AcciMap, STAMP-CAST and AcciNet using the Signal Detection Theory (SDT) paradigm. A total of 180 hours' worth of analyses across 360 comparisons were performed by 30 expert analysts. Findings revealed that all three methods produced a weak to moderate positive correlation coefficient, however the inter-rater reliability of STAMP-CAST was significantly higher compared to AcciMap and AcciNet. No statistically significant or practically meaningful differences were found between methods in the overall intra-rater reliability analyses. Implications and future research directions are discussed.

Practitioners who undertake accident analysis within their organisations should consider the use of STAMP-CAST due to the significantly higher inter-rater reliability findings obtained in this study compared to AcciMap and AcciNet, particularly if they tend to work alone and/or part of relatively small teams.

Exploring improvisations in road safety in a low-income setting.

Road collision types repeat themselves, especially in low- and middle-income countries (LMICs), where countermeasures are often improvised and implemented with little planning. At the Shahbag intersection in Dhaka, Bangladesh, speed bumps were quickly constructed at the exit of the intersection as an improvised road safety measure following the occurrence of a fatal collision, which eventually contributed to another collision between a truck and a car. The events influencing the improvisation decision, and that action's consequences, have been analysed using the Impromap methodology, a variation of the Accimap approach that focusses specifically on improvisation. The applicability of the Impromap as a systems-based approach to the road safety domain is assessed using the predictions described in Rasmussen's risk management framework, and corresponding countermeasures are proposed. The analysis shows that improvisation in the road safety domain is undesirable irrespective of the economic setting as it is likely to eventually contribute to secondary collisions.Practitioner summary: In this paper, the events influencing the improvisation decision following a road crash, and that action's consequences, have been analysed using the Impromap methodology. The applicability of Impromap as a systems-based approach in road safety domain is assessed using the Rasmussen's risk management framework-based predictions, and corresponding countermeasures are proposed.

The effect of information integration on team communication in a simulated submarine control room task.

Submarine control rooms are characterised by dedicated individual roles for information types (e.g. Sonar operator processes sound energy), with individuals verbally reporting the information that they receive to other team members to help resolve uncertainty in the operational environment (low information integration). We compared this work design with one that ensured critical information was more readily available to all team members (high information integration). We used the Event Analysis of Systemic Teamwork (EAST) method to analyse task, information, and social networks for novice teams operating within a simulated submarine control room under low versus high information integration. Integration impacted team member centrality (importance relative to other operators) and the nature of information shared. Team members with greater centrality reported higher workload. Higher integration across consoles altered how team members interacted and their relative status, the information shared, and how workload was distributed. However, overall network structures remained intact.

Wider integration (distribution) of information within teams in a simulated submarine control room altered the content of the information shared between team members and the centrality and workload of team members. Practitioners must consider how to integrate information in sociotechnical systems such that information traditionally held by specialist positions can be distributed within teams to benefit team performance and other outcomes.

The Chartered Institute of Ergonomics and Human Factors at 75: perspectives on contemporary challenges and future directions for Ergonomics and Human Factors.

As the UK's Chartered Institute of Ergonomics and Human Factors (CIEHF) celebrates its 75th anniversary, it is worth reflecting on our discipline's contribution, current state, and critical future endeavours. We present the perspectives of 18 EHF professionals who were asked to respond to five questions regarding the impact of EHF, contemporary challenges, and future directions. Co-authors were in agreement that EHF's impact has been only limited to date and that critical issues require resolution, such as increasing the number of suitably qualified practitioners, resolving the research-practice gap, and increasing awareness of EHF and its benefits. Frequently discussed future directions include advanced emerging technologies such as artificial intelligence, the development of new EHF methods, and enhancing the quality and reach of education and training. The majority felt there will be a need for EHF in 75 years; however, many noted that our methods will need to adapt to meet new needs.Practitioner statement: This article provides the perspectives of 18 Ergonomics and Human Factors (EHF) professionals on the impact of EHF, contemporary challenges and critical future directions, and changes that are necessary to ensure EHF remains relevant in future. As such, it provides important guidance on future EHF research and practice.

To automate or not to automate: advocating the 'cliff-edge' principle.

We reflect briefly on the last forty years or so of ergonomics and human factors research in automation, observing that many of the issues being discussed today are the same as all those decades ago. In this paper, we explicate one of the key arguments regarding the application of automation in complex safety-critical domains, which proposes restraining the capabilities of automation technology until it is able to fully and completely take over the task at hand. We call this the 'cliff-edge' principle of automation design. Instead, we espouse a use for the technology in a more problem-driven, human-centred way. These are not entirely new ideas and such a philosophy is already gaining traction in ergonomics and human factors. The point is that in a given system, tasks should be controlled either by human or by automation; anything in between only causes problems for system performance.

Human factors problems with automation have been with us for over forty years, and have changed little in that time. This brief review shows a groundswell of opinion that points to what we call the cliff-edge automation principle ­ restraining the full capabilities of technology until it is ready to fully and completely take over the task. This approach improves human performance in the system by keeping the person in the loop and in control. Researchers and practitioners in ergonomics and human factors should continue to push this message to the designers and manufacturers of automated systems.

Towards a unified model of accident causation: refining and validating the systems thinking safety tenets.

The systems thinking tenets were developed based on a synthesis of contemporary accident causation theory, models and approaches and encapsulate 15 features of complex systems that interact to create both safety and adverse events. Whilst initial testing provided supportive evidence, the tenets have not yet been subject to formal validation. This article presents the findings from a three-round Delphi study undertaken to refine and validate the tenets and assess their suitability for inclusion in a unified model of accident causation. Participants with expertise in accident causation and systems thinking provided feedback on the tenets and associated definitions until an acceptable level of consensus was achieved. The results reduced the original 15 tenets to 14 and 10 were identified as important to include in unified model of accident causation. The refined systems thinking tenets are presented along with future research directions designed to facilitate their use in safety practice.Practitioner summary: This article presents a refined and validated set of systems thinking tenets which describe features of complex systems that interact to create adverse events. The tenets can be used by practitioners to proactively identify safety leading indicators and contributory factors during adverse event analysis.

Testing the reliability and validity of risk assessment methods in Human Factors and Ergonomics.

There is growing interest in the use of systems-based risk assessment methods in Human Factors and Ergonomics (HFE). The purpose of this study was to test the intra-rater reliability and criterion-referenced concurrent validity of three systems-based risk assessment approaches: (i) the Systems-Theoretic Process Analysis (STPA) method; (ii) the Event Analysis of Systemic Teamwork Broken Links (EAST-BL) method; and, (iii) the Network Hazard Analysis and Risk Management System (Net-HARMS) method. Reliability and validity measures were obtained using the Signal Detection Theory (SDT) paradigm. Whilst STPA identified the highest number of risks, the findings indicate a weak to moderate level of reliability and validity for STPA, EAST-BL and Net-HARMS. There were no statistically significant differences between the methods across analyses. The results suggest that there is merit to the continued use of systems-based risk assessment methods following a series of methodological extensions that aim to enhance the reliability and validity of future applications. Practitioner summary The three risk assessment methods produced weak to moderate levels of stability and accuracy regarding their capability to predict risks. There is a pressing need to further test the reliability and validity of safety methods in Human Factors and Ergonomics.

The quest for the ring: a case study of a new submarine control room configuration.

Submarine control room layouts have remained similar across decades of operation, despite the introduction of new technologies that allow for the co-location of the sound and control room. Operation of an inwards, rather than outwards, facing ring control room configuration was examined in three scenarios: Return to Periscope Depth, Inshore Operations, and Dived Tracking. A case study design employed a serving team of qualified submariners participated in all three scenarios with high and low demand. Communications and activities in the control room were recorded and analysed using the Event Analysis of Systemic Teamwork (EAST) method. EAST models collaborative teamwork through social, information, and task networks. The results from the ring configuration were compared to an outward facing baseline of the contemporary control room layout with a separate sound and control room. The ring control room configuration increased communications between operators, leading to a more information exchanged, and more tasks completed. Practitioner summary: Control room design on submarines, and other domains, has traditionally been outward facing with supervisory staff looking over the shoulders of their subordinates. In this paper, and inward looking control room design was explored, with subordinate staff facing their supervisors. This design resulted in more information exchange and productive work.

Predicting and mitigating failures on the flight deck: an aircraft engine bird strike scenario.

Engine damage as a consequence of foreign object debris (FOD) during flight is frequently caused by birds. One approach to minimising disruption caused by this damage is to provide flight crew with accurate information relating to the continuing operational status of the aircraft's engines. Before designing such avionic systems however, understanding of current procedures is needed. Hierarchical Task Analysis (HTA) and Systematic Human Error Reduction and Prediction Approach (SHERPA) were used to identify potential failures that flight crew may make when managing an engine bird strike. Workshops with commercial pilots generated insights into current practice and a commercial pilot SME reviewed outputs for accuracy. Over 200 potential failures were identified, most commonly related to communication. Remedial measures, considering future avionic systems, are proposed to mitigate identified failures. This analysis provides a starting point for future design concepts for assisting flight crew in dealing with engine malfunction due to FOD strikes. Practitioner summary: Hierarchical Task Analysis was conducted to show all tasks involved in dealing with an in-flight aircraft engine bird strike. Systematic Human Error Reduction and Prediction Approach analysis was performed and over 200 possible failures were identified when managing this event. Remedial measures are proposed to help mitigate possible failures.

State of science: models and methods for understanding and enhancing teams and teamwork in complex sociotechnical systems.

This state of the science review brings together the disparate literature of effective strategies for enhancing and accelerating team performance. The review evaluates and synthesises models and proposes recommended avenues for future research. The two major models of the Input-Mediator-Output-Input (IMOI) framework and the Big Five dimensions of teamwork were reviewed and both will need significant development for application to future teams comprising non-human agents. Research suggests that a multi-method approach is appropriate for team measurements, such as the integration of methods from self-report, observer ratings, event-based measurement and automated recordings. Simulations are recommended as the most effective team-based training interventions. The impact of new technology and autonomous agents is discussed with respect to the changing nature of teamwork. In particular, whether existing teamwork models and measures are suitable to support the design, operation and evaluation of human-nonhuman teams of the future. Practitioner summary: This review recommends a multi-method approach to the measurement and evaluation of teamwork. Team models will need to be adapted to describe interaction with non-human agents, which is what the future is most likely to hold. The most effective team training interventions use simulation-based approaches.

Predicting Design-Induced Error on the Flight Deck : An Aircraft Engine Oil Leak Scenario.

OBJECTIVE: To explore the types of errors that commercial pilots may make when trying to resolve a suspected engine oil leak using the interfaces currently available. BACKGROUND: The decisions that pilots make often have to be made quickly and under time pressure, with the emphasis on avoiding critical situations from arising. To make the correct decisions, it is vital that pilots have accurate and up-to-date information available. However, interaction with flight deck interfaces may lead to error if they are not effectively designed. METHOD: A hierarchical task analysis was conducted using evidence from pilot interview data to understand the pilots' typical response to a suspected engine oil leak scenario. This was used as the primary input into the Systematic Human Error Reduction and Prediction Approach (SHERPA). RESULTS: A total of 108 possible errors were identified. The most common error type was a retrieval error, in which flight crews may retrieve the wrong information about the engine. A number of remedial measures are proposed to try and overcome such issues. CONCLUSION: This analysis provides an initial starting point for identifying potential future design ideas that can assist the pilots in dealing with oil leaks. APPLICATION: This work has identified the value of applying human error identification methodologies to the assessment of current flight deck processes surrounding engine oil leaks. The method presented permits the operational analysis of possible errors on the flight deck and facilitates the proposition of remedial measures to implement technological innovations that can mitigate error.

Thinking aloud on the road: Thematic differences in the experiences of drivers, cyclists, and motorcyclists.

This study takes a qualitative approach to exploring the experiences (and differences therein) of individuals using either their car, bicycle, or motorcycle to navigate a ∼10.5 km urban route in a provincial UK city, with the aim of contributing to our understanding of the needs and requirements of different road users. Forty-six individuals provided concurrent verbal reports, using the 'think aloud' method, whilst using their vehicle to navigate the test route, the transcripts of which were subjected to a theory-agnostic, inductive, thematic analysis. A number of group differences were observed, revealing (among other factors) the importance of road surface quality to cyclists, the focus on vigilant observation in motorcyclists, and the heightened emotionality experienced by both two-wheeled groups, particularly those on bicycles. This affective component has, as yet, been under-explored in the academic domain and under-utilised in road transport policy and strategy; this is discussed, with attention drawn to the cyclists' greater tendency to make negatively valenced value judgements. Results are also discussed in terms of the potential to improve road users' experiences, foster inter-group empathy and understanding, and encourage a shift in mobility towards more sustainable modes.

Adjusting the need for speed: assessment of a visual interface to reduce fuel use.

Previous research has identified that fuel consumption and emissions can be considerably reduced if drivers engage in eco-driving behaviours. However, the literature suggests that individuals struggle to maintain eco-driving behaviours without support. This paper evaluates an in-vehicle visual interface system designed to support eco-driving through recommendations based on both feedforward and feedback information. A simulator study explored participants' fuel usage, driving style, and cognitive workload driving normally, when eco-driving without assistance and when using a visual interface. Improvements in fuel-efficiency were observed for both assisted (8.5%) and unassisted eco-driving (11%), however unassisted eco-driving also induced a significantly greater rating of self-reported effort. In contrast, using the visual interface did not induce the same increase of reported effort compared to everyday driving, but itself did not differ from unassisted driving. Results hold positive implications for the use of feedforward in-vehicle interfaces to improve fuel efficiency. Accordingly, directions are suggested for future research. Practitioner Summary: Results from a simulator study comparing fuel usage from normal driving, engaging in unassisted eco-driving, or using a novel speed advisory interface, designed to reduce fuel use, are presented. Whilst both unassisted and assisted eco-driving reduced fuel use, assisted eco-driving did not induce workload changes, unlike unassisted eco-driving. Abbreviations: CO-2: carbon dioxide; NASA-TLX: NASA task load index; RMS: root-mean-square; MD: mean difference.

Complexity theory in accident causation: using AcciMap to identify the systems thinking tenets in 11 catastrophes.

The quest to explain and understand the cause of accidents is both ever-present and ongoing amongst the safety science community. In an attempt to advance the theory and science of accident causation, researchers have recently formalised a set of '15 systems thinking tenets' that cover the conditions and characteristics of work systems that are believed to contribute to the cause of accidents. The purpose of this study was to attempt to identify the systems thinking tenets across a range of different systems and accidents using the Accident Mapping (AcciMap) method. The findings suggest that the tenets can be attributed to play a role in accident causation, however as a result of this process, the capability of AcciMap has been brought into question. Implications and directions for future research are described. Practitioner Summary: This study is an extension of previous work that suggested there was a need to test for the 'systems thinking tenets of accident causation' in a multi-incident dataset. We used AcciMap to evaluate whether it has the capability to support ongoing accident analysis activities in ergonomics research.

Resolving the differences between system development and system operation using STAMP: a road safety case study in a low-income setting.

Road safety strategies adopted worldwide have made significant progress in reducing road trauma, but have stagnated more recently. The situation in low- and middle-income countries is even worse with no significant decrease in fatality rates. Safety researchers have argued that adopting sociotechnical systems approach is necessary to make significant advancements and improvements. The aim of this study was to develop a control structure model of the Bangladesh road safety system by identifying the actors and organisations involved across the system. Expert stakeholders were identified and interviewed, and relevant information was gathered in order to generate the Systems Theoretic Accident Model and Process control structure model. Throughout the analysis of this model, differences in the control and feedback mechanisms of the system were identified, and road safety intervention recommendations were made. Future research should also predict potential risks within the system and propose proactive and preventative countermeasures. Practitioner summary: In this article, a Systems Theoretic Accident Model and Process control structure model of the Bangladesh road safety system is developed, and the involved actors are identified. Based on interviews and workshops with expert stakeholders, differences in the controls and feedback mechanisms in the system were identified, and road safety intervention recommendations were made. Abbreviations: BUET: Bangladesh University of Engineering and Technology.

Better together? Investigating new control room configurations and reduced crew size in submarine command and control.

The separation of the sound and control rooms in Royal Navy submarines seems to be artefactually reducing the effectiveness of information transition and the overall productivity of the team. A proposed integrated sound and control room was tested in three scenarios: Return to Periscope Depth (RTPD), Inshore Operations (INSO) and Dived Tracking (DT). The activities and communications of a team of serving submariners were recorded in a control room, in a single case study design, comparing co-location and reduced crewing with a baseline of the separate sound and control room configurations that is representative of current submarines. The Event Analysis of Systemic Teamwork (EAST) method was used to examine changes in social, information and task networks. In general terms, the co-location of the submariner team led to more efficient communication and completion of tasks. Reducing the crew was more challenging in the higher demand scenarios. Practitioner Summary: There are constraints acting on control rooms, both in terms of physical space and crew size. This study compared conventional control room with co-location and reduced crew in turn. Teamwork improved in the collocated control room but the reduced crew struggled most under conditions of high demand. Abbreviations: DT: dived tracking; EAST: event analysis for systemic teamwork; H: high; INSO: inshore operations; L: low; OOW: office of the watch; OpsO: operations officer; Peri: periscope operator; RTPD: return to periscope depth; RN: royal navy; SoC: sonar controller; SoP: sonar operator; TMA: time-motion analyst.

Representing two road traffic collisions in one Accimap: highlighting the importance of emergency response and enforcement in a low-income country.

Seemingly erratic pedestrian crossing has become a major source of vehicle-pedestrian collisions on highways in Bangladesh, and across other low- and middle-income countries (LMICs). In this article, we approach the challenge from a sociotechnical systems perspective by using the Accimap method to analyse a pair of time-separated yet interconnected road traffic collisions. The first event involved a truck colliding with a road divider; in the second, fatal incident, a bus hit a university student. The traditional-style investigation conducted immediately after the collision apportioned blame to end users, that is, drivers and pedestrian; however, application of sociotechnical systems thinking revealed the contribution from lack of emergency response and enforcement among many other important factors. Results and recommendations are discussed in terms of reducing the chance and severity of such collisions across LMICs, and in terms of the need to look beyond the end-user, a focus that remains dominant in such settings. Practitioner summary: This paper applies sociotechnical systems thinking to pedestrian safety in Bangladesh by analysing two inter-connected road traffic collisions using a single Accimap. The findings emphasise the importance of implementing road safety interventions that target all system levels, and draw attention to the importance of post-collision response in low-income settings. Abbreviation: BUET: Bangladesh University of Engineering and Technology.

Modelling distributed crewing in commercial aircraft with STAMP for a rapid decompression hazard.

Changes to crewing configurations in commercial airlines are likely as a means of reducing operating costs. To consider the safety implications for a distributed crewing configuration, system theoretic accident model and processes (STAMP) was applied to a rapid decompression hazard. High level control structures for current operations and distributed crewing are presented. The CONOPS generated by STAMP-STPA for distributed crewing, and design constraints associated with unsafe control actions (UCAs) are offered to progress in the route to certification for distributed crewing, and improve safety in current operations. Control loops between stakeholders were created using system-theoretic process analysis (STPA). The factors leading to the Helios 255 incident demonstrated the redundancy that a ground station could offer without the risk of hypoxia, during a decompression incident. STPA analysis also highlighted initial UCAs that could occur within the hypothetical distributed crewing configuration, prompting consideration of design constraints and new CONOPS for ground station design. Practitioner Summary: SPO in commercial aircraft is likely as a means to reduce costs. This paper makes a case for distributed crewing using STAMP-STPA. Comparing current operations with a distributed crewing configuration, the redundancy offered by a ground station is demonstrated. Design constraints and new CONOPs for distributed crewing, and current operations are proposed.

Use of Highways in the Sky and a virtual pad for landing Head Up Display symbology to enable improved helicopter pilots situation awareness and workload in degraded visual conditions.

Flight within degraded visual conditions is a great challenge to pilots of rotary-wing craft. Environmental cues typically used to guide interpretation of speed, location and approach can become obscured, forcing the pilots to rely on data available from in-cockpit instrumentation. To ease the task of flight during degraded visual conditions, pilots require easy access to flight critical information. The current study examined the effect of 'Highways in the Sky' symbology and a conformal virtual pad for landing presented using a Head Up Display (HUD) on pilots' workload and situation awareness for both clear and degraded conditions across a series of simulated rotary-wing approach and landings. Results suggest that access to the HUD lead to significant improvements to pilots' situation awareness, especially within degraded visual conditions. Importantly, access to the HUD facilitated pilot awareness in all conditions. Results are discussed in terms of future HUD development. Practitioner Summary: This paper explores the use of a novel Heads Up Display, to facilitate rotary-wing pilots' situation awareness and workload for simulated flights in both clear and degraded visual conditions. Results suggest that access to HUD facilitated pilots' situation awareness, especially when flying in degraded conditions.

Using the Event Analysis of Systemic Teamwork (EAST) broken-links approach to understand vulnerabilities to disruption in a darknet market.

Darknet markets provide an anonymous, online platform for users to trade illicit drugs, fraudulent identity data, and other commodities. Although law enforcement agencies have been successful in seising many markets, the Darknet is an agile and dynamic environment and market activities often persist and emerge in a new form. Given this constantly changing environment, new ways of disrupting darknet markets are required. This study used Event Analysis of Systemic Teamwork (EAST) to analyse market activity and understand vulnerabilities to disruption. This involved using the EAST broken-links approach to assess the effects of compromising the transmission of information between tasks and between agents. The analysis identified critical vulnerabilities in the system, which included information involved in registering, depositing funds, communicating listing details to buyers, and communicating dispute resolution messages. This study indicates that systems ergonomics methods-in particular, EAST-can provide insight into system vulnerabilities that might be targeted for disruption. Practitioner summary This study provides a conceptualisation of the processes, people, structures, and information involved in the buying and selling of goods on a darknet market. Law enforcement agencies may use broken-links analyses to systematically consider the effects of their interventions.