Using cognitive work analysis to identify competencies for human factors and ergonomics practitioners.

While several competency frameworks have been proposed for Human Factors and Ergonomics (HFE) practitioners, these are not commonly based on structured analyses. The aim of this research was to develop a sociotechnical systems model of the HFE practitioner role in Australia and identify the competencies required to fulfil the role. Study One applied the Work Domain Analysis phase of cognitive work analysis (CWA) to model the HFE practitioner role. Model refinement was undertaken with seven subject matter experts. In Study Two, the model was used to elicit the competencies (knowledge, skills, abilities, other characteristics) required for successful performance, via a survey of 28 HFE practitioners. Most competencies related to skills (i.e. communication skills) and knowledge (i.e. domain knowledge). Gaps in competencies were also identified, linked to a lack of HFE education pathways in Australia. The findings have practical utility for designing HFE practitioner roles and educational programs. Practitioner summary: Cognitive work analysis provided a structured analysis of the role of the Human Factors and Ergonomics (HFE) practitioner and to support the identification of competencies. The results suggest that HFE practitioners are generalists rather than specialists and have implications for the job design and education of HFE practitioners. Abbreviations: BCPE: board of certification in professional ergonomics; CIEHF: chartered institute of ergonomics and human factors; CWA: cognitive work analysis; HFE: human factors and ergonomics; HFESA: human factors and ergonomics society of Australia; IEA: International Ergonomics Association; KSAO: knowledge, skills, abilities and other characteristics; O*NET: occupation information network; SME: subject matter expert; SRK: skills, rules, knowledge; UK: United Kingdom; USA: United States of America; WDA: work domain analysis.

Applying a systems thinking lens to injury causation in the outdoors: Evidence collected during 3 years of the Understanding and Preventing Led Outdoor Accidents Data System.

INTRODUCTION: This article presents a detailed systems analysis of injury incidents from 35 Australian led outdoor activity organisations between 2014 to 2017. METHOD: Injury incident reports were collected using a specific led outdoor activity incident reporting system known as UPLOADS (Understanding and Preventing Led Outdoor Accidents Data System). RESULTS: In total, 1367 people sustained injuries from across 20 different activities, with an injury rate of 1.9 injured people per 1000 participants over the three-year period. A total of 2234 contributory factors from multiple levels of the led outdoor activity system were identified from the incident reports, and 361 relationships were identified between contributory factors. DISCUSSION: This systems analysis of injury incidents demonstrates that it is not only factors within the immediate context of the incident (Participants, Environment, Equipment) but factors from across multiple systemic levels that contributes to injury incidents (Schools, Parents, Activity centre management). Prevention efforts should focus on addressing the whole network of contributing factors and not only the prominent factors at the lower system levels within the immediate context of the injury incident occurrences.

Evaluation of construct and criterion-referenced validity of a systems-thinking based near miss reporting form.

The validity of methods is an ongoing issue in ergonomics. Inconsistent definitions and approaches to evaluation exacerbate this challenge. In this study, the construct and criterion-referenced validity of a new near miss reporting form was evaluated to determine the extent to which it comprehensively captures near miss incidents and is aligned with the systems thinking approach to accident causation. Interview data were used as the reference standard in the evaluation. Using signal detection theory (SDT), a high average hit rate (HR), predictive value (PV) and sensitivity index (SI) were found, with an almost perfect ranking for the index of concordance. The findings show that the reporting form has strong construct and criterion-referenced validity. It is proposed that the approach used in this study could be used by researchers and practitioners when testing the validity of incident data collection tools. Practitioner summary: The validity of methods is a key issue in ergonomics. In this study, we test the validity of a near miss reporting form using interview data as a standard. This approach could be used by practitioners when testing the validity of other ergonomics methods.

Interaction-centred design: an end user evaluation of road intersection concepts developed using the cognitive work analysis design toolkit (CWA-DT).

Crashes at intersections represent an important road safety problem. Interactions between different road user types, such as between vehicles and vulnerable road users, are a particular concern. It has been suggested driver-centric road design plays a role in crashes. A multi-road user evaluation of three novel intersection designs is described. The designs were generated using the Cognitive Work Analysis Design Toolkit, underpinned by sociotechnical systems theory. The desktop evaluation involved drivers, motorcyclists, cyclists and pedestrians rating the design concepts against alignment with design goals, sociotechnical systems theory and usability, and providing feedback on the positive and negative aspects. Two concepts received more positive ratings and feedback in comparison to a concept that provided more user autonomy. The evaluation results also highlight clear differences in needs across road user groups. The design and evaluation process demonstrates how sociotechnical systems values and principles can be applied in the design of public spaces. Practitioner Summary: This study involved a participatory evaluation of novel road intersection designs, based on sociotechnical systems theory. The results identified important differences in needs and preferences across road user groups and demonstrate the value of sociotechnical systems theory and user participation in road transport design and evaluation processes. Abbreviations: CWA-DT: cognitive work analysis design toolkit; WDA: work domain analysis; SUS: system usability scale.

Accounting for memes in sociotechnical systems: extending the abstraction hierarchy to consider cognitive objects.

Work domain analysis (WDA) is used to model the functional structure of sociotechnical systems (STS) through the abstraction hierarchy (AH). By identifying objects, processes, functions and measures that support system purposes, WDA reveals constraints within the system. Traditionally, the AH describes system elements at the lowest level of abstraction as physical objects. Multiple analyses of complex systems reveal that many include objects that exist only at a conceptual level. This paper argues that, by extending the AH to include cognitive objects, the analytical power of WDA is extended, and novel areas of application are enabled. Three case studies are used to demonstrate the role that cognitive objects play within STS. It is concluded that cognitive objects are a valid construct that offer a significant enhancement of WDA and enable its application to some of the world's most pressing problems. Implications for future applications of WDA and the AH are discussed. Practitioner summary: Some sociotechnical systems include memes as part of their functional structure. Three case studies were used to evaluate the utility of introducing cognitive objects alongside physical ones in work domain analysis, the first phase of cognitive work analysis. Including cognitive objects increases the scope and accuracy of work domain analysis.

What went right? An analysis of the protective factors in aviation near misses.

Learning from successful safety outcomes, or what went right, is an important emerging component of maintaining safe systems. Accordingly, there are increasing calls to study normal performance in near misses as a part of safety management activities. Despite this, there is limited guidance on how to accomplish this in practice. This article presents a study in which using Rasmussen's risk management framework to analyse 16 serious incidents from the aviation domain. The findings show that a network of protective factors prevents accidents with factors identified across the sociotechnical system. These protective networks share many properties with those identified in accidents. The article demonstrates that is possible to identify these networks of protective factors from incident investigation reports. The theoretical implications of these results and future research opportunities are discussed. Practitioner Statement: The analysis of near misses is an important part of safety management activities. This article demonstrates that Rasmussen?s risk management framework can be used to identify networks of protective factors which prevent accidents. Safety practitioners can use the framework described to discover and support the system-wide networks of protective factors.

Distributed improvisation: a systems perspective of improvisation 'epics' by led outdoor activity leaders.

Improvisation represents the spontaneous and real-time conception and execution of a novel response to an unanticipated situation. In order to benefit from the positive safety potential of this phenomenon, it is necessary to understand what influences its appropriateness and effectiveness. This study has applied the system-based methodology Impromaps to analysing accounts of improvisation aimed at mitigating adverse safety outcomes. These accounts were obtained from led outdoor activity (LOA) leaders through critical decision method interviews. Influencing factors and interactions have been identified across all system levels. The factors most influential to leaders' ability to improvise are 'Policy, procedures and rules', 'Organisation culture', 'Training', 'Role responsibilities', 'Communication/instruction/demonstration', 'Situation awareness', 'Leader experience', 'Mental simulation', 'Equipment, clothing & PPE' and 'Terrain/physical environment'. To enhance the likelihood of effective, appropriate improvisation, LOA providers are recommended to focus on higher level factors over which they are able to exert greater control. Practitioner Summary: To enhance resilience in safety-critical situations, organisations need to understand what influences appropriate, effective improvisation. To elucidate this, the Impromaps methodology is applied to in-depth interview data. The Impromap affords a graphical depiction of the influencing factors and interactions across the system, providing a basis for the development of interventions.

A knock to the system: A new sociotechnical systems approach to sport-related concussion.

Broader contextual factors that influence concussion management have tended to be overlooked. To address this, the present study used a sociotechnical systems approach to identify perceived responsibilities and applied strategies for three domains of concussion management (i.e., prevention, identification and treatment). Participants were 118 members of the community rugby union system in Australia (69.2% male). Participants from throughout the rugby system (e.g., players, parents, coaches, club management) were asked open-ended questions about their perceived responsibilities and the strategies they use for concussion management. It was found that (a) proper training, technique correction and education were recurrent prevention themes; (b) the majority of key stakeholders felt that they could consistently identify concussion; however, medical aids (medics) were the only system actors who stated a responsibility to use standardised concussion assessment measures and (c) less than one third of the respondents indicated their involvement in treating concussion. This study identifies specific junctures in the system that prevents effective concussion management strategies. A sociotechnical systems approach improves the understanding of concussion prevention, and management beliefs and behaviours.

Defining disaster resilience: comparisons from key stakeholders involved in emergency management in Victoria, Australia.

Three years after the introduction of the National Strategy for Disaster Resilience there remains no unanimously adopted definition of disaster resilience within Australia's emergency management sector. The aim of this study is to determine what the concept means to key stakeholders in the emergency management sector in the Australian State of Victoria, and how these conceptualisations overlap and diverge. Via an online survey, 113 people were asked how they define disaster resilience in their work in the emergency management sector. A data mining software tool, Leximancer, was employed to uncover the relationships between the definitions provided. The findings show that stakeholders see resilience as an 'ability' that encompasses emergency management activities and personal responsibility. However, the findings also highlight some possible points of conflict between stakeholders. In addition, the paper outlines and discusses a number of potential consequences for the implementation and the success of the resilience-based approach in Australia.

Is it really good to talk? Testing the impact of providing concurrent verbal protocols on driving performance.

Questions have been raised regarding the impact that providing concurrent verbal protocols has on task performance in various settings; however, there has been little empirical testing of this in road transport. The aim of this study was to examine the impact of providing concurrent verbal protocols on driving performance. Participants drove an instrumented vehicle around a set route, twice whilst providing a concurrent verbal protocol, and twice without. A comparison revealed no differences in behaviour related to speed, braking and steering wheel angle when driving mid-block, but a significant difference in aspects of braking and acceleration at roundabouts. When not providing a verbal protocol, participants were found to brake harder on approach to a roundabout and accelerate more heavily coming out of roundabouts. It is concluded that providing verbal protocols may have a positive effect on braking and accelerating. Practical implications related to driver training and future research are discussed. Practitioner Summary: Verbal protocol analysis is used by ergonomists to understand aspects of cognition and decision-making during complex tasks such as driving and control room operation. This study examines the impact that it has on driving performance, providing evidence to support its continued use in ergonomics applications.

Fitting methods to paradigms: are ergonomics methods fit for systems thinking?

The issues being tackled within ergonomics problem spaces are shifting. Although existing paradigms appear relevant for modern day systems, it is worth questioning whether our methods are. This paper asks whether the complexities of systems thinking, a currently ubiquitous ergonomics paradigm, are outpacing the capabilities of our methodological toolkit. This is achieved through examining the contemporary ergonomics problem space and the extent to which ergonomics methods can meet the challenges posed. Specifically, five key areas within the ergonomics paradigm of systems thinking are focused on: normal performance as a cause of accidents, accident prediction, system migration, systems concepts and ergonomics in design. The methods available for pursuing each line of inquiry are discussed, along with their ability to respond to key requirements. In doing so, a series of new methodological requirements and capabilities are identified. It is argued that further methodological development is required to provide researchers and practitioners with appropriate tools to explore both contemporary and future problems. Practitioner Summary: Ergonomics methods are the cornerstone of our discipline. This paper examines whether our current methodological toolkit is fit for purpose given the changing nature of ergonomics problems. The findings provide key research and practice requirements for methodological development.

Causal factors of hot air ballooning incidents: identification, frequency, and potential impact.

BACKGROUND: Hot air ballooning incidents are relatively rare; however, when they do occur they are likely to result in a fatality or serious injury. Human error is commonly attributed as the cause of hot air ballooning incidents; however, error in itself is not an explanation for safety failures. This research aims to identify and establish the relative importance of factors contributing toward hot air ballooning incidents. METHODS: Twenty-two Australian Ballooning Federation (ABF) incident reports were thematically coded using a bottom-up approach to identify causal factors. Subsequently, 69 balloonists (mean 19.51 yr experience) participated in a survey to identify additional causal factors and rate (out of 7) the perceived frequency and potential impact to ballooning operations of each of the previously identified causal factors. Perceived associated risk was calculated by multiplying mean perceived frequency and impact ratings. RESULTS: Incident report coding identified 54 causal factors within 9 higher level areas: Attributes, Crew Resource Management, Equipment, Errors, Instructors, Organizational, Physical Environment, Regulatory Body, and Violations. Overall, 'weather', 'inexperience,' and 'poor/inappropriate decisions' were rated as having greatest perceived associated risk. DISCUSSION: Although errors were nominated as a prominent cause of hot air ballooning incidents, physical environment and personal attributes are also particularly important for safe hot air ballooning operations. In identifying a range of causal factors, the areas of weakness surrounding ballooning operations have been defined; it is hoped that targeted safety and training strategies can now be put into place to remove these contributing factors and reduce the chance of pilot error.

Understanding complexity in a safety critical setting: A systems approach to medication administration.

'Medication errors' are a significant concern and are associated with a higher incidence of adverse events and unintentional patient harm than any other aspect of healthcare. While much research has focused on adverse medication errors, limited studies have specifically examined 'normal' medication delivery performance and the interactions between tasks, agents, and information within the medication administration system. This article describes a study that applied the Event Analysis of Systemic Teamwork (EAST) model to study the hospital medication administration system to identify opportunities to optimise performance and patient safety. Key findings of this study demonstrate that this is a highly complex system, comprising many social agents and a relatively closely linked series of tasks and information. However, most of the workload relies on a small proportion of healthcare professionals. Significantly, the patient has a minimal role in the medication administration system during their hospital stay. The research has shown that this approach enables mapping networks and their interdependencies to optimise the system as a whole rather than its parts in isolation.

The impact of on-road motion on BMS touch screen device operation.

This study investigates the effect of vehicle motion on performance, usability and workload for a touch screen in-vehicle Battle Management System (BMS). Participants performed a series of battle management tasks while a vehicle was driven over sealed (characteristic of 'normal' vehicle motion) and unsealed (characteristic of 'high' vehicle motion) roads. The results indicate that unsealed road conditions impair the performance of information input tasks (tasks that require the user to enter information, e.g. text entry) but not information extraction tasks (tasks that require the user to retrieve information from the system, e.g. reading coordinates). Participants rated workload as higher and the system as less usable on the unsealed road. In closing, the implications for in-vehicle touch screen design and use in both military and civilian driving contexts are discussed. Practitioner Summary: The effect of motion on interacting with in-vehicle touch screen devices remains largely unexplored. This study examines the effect of different levels of vehicle motion on the use of a BMS. Using the system under off-road conditions had a detrimental impact on workload, performance and usability.

Applying AcciMap to test the common cause hypothesis using aviation near misses.

The common cause hypothesis, as applied here, proposes that similar networks of influencing factors may contribute to both adverse outcomes and near misses. This hypothesis has not been evaluated using a systems-thinking perspective. The aims of this study are to evaluate whether networks of contributory and protective factors exist within aviation serious near miss reports and to determine if the common cause hypothesis is applicable in this context. Sixteen incident reports from French civil aviation crash investigation bureau were analysed using the AcciMap method. Contributory and protective factors, and relationships between both were identified via coding of the reports. The results indicate that considering protective factors support a richer picture of incidents and provide support for the common cause hypothesis as measured by similar mean factor volume and sociotechnical levels for both contributory and protective factors. However, the findings also show the direction of relationships among protective and contributory factors may be indicative of a difference among adverse outcomes, near misses, and normal work. Future research should consider how a network of relationships may impact on the common contributory and protective factors found in near misses.

Communication in the workplace: Defining the conversations of supervisors.

BACKGROUND: Communications plays a central role in promoting the health and wellbeing of workers. Although much literature has shown the positive benefits of safety communication in the workplace, research has yet to explore the nature of these communication practices within supervisor-worker relationships. This study overcomes this gap in the literature through objectively monitoring communication within the daily working lives of work-group supervisors in one organization. AIMS: The aims of the research were to: (a) categorize communication in the workplace into three categories, namely task-related communication, relationship-related communication, and safety-related communication; and (b) explore the frequency of these dialogs. METHOD: We periodically recorded brief snippets of ambient (acoustic) sounds in supervisors' workplace environment by using an Electronically Activated Recorder (EAR). The EAR was run on an Apple iPod, with an application downloaded for free on iTunes (i.e., iEAR). The EAR was programmed to record for 30 s every three minutes for eight working hours a day of a five-day working week. RESULTS: A total of 12.38 h of acoustic sounds from five workgroup supervisors was useable for coding. The results found examples of task-related (productivity, efficiency, workflow, and human resources) communication, as well as relationship-related (greetings, personal life discussions, workplace relations), and safety-related communication. We also found that the majority of the communication recorded was task-related communication compared with relationship-related and safety-related communication. Practical applications: This research provides preliminary insights into communication practices in the workplace and avenues for future research.

End-user experiences with two incident and injury reporting systems designed for led outdoor activities - challenges for implementation of future data systems.

BACKGROUND: Injury and incident (near miss) prevention is heavily dependent upon robust and high-quality data systems. Evaluations of surveillance systems designed to report factors associated with incidents and injuries are essential to understand their value, as well as to improve their performance and efficiency. Despite, this there have been few such evaluations published in the peer-review literature. METHODS: The attitudes and experiences of industry representatives who used one of two variants of an incident and injury surveillance system to collect injury and incident data for the led outdoor activity setting were obtained through an online self-report survey following a 12-month trial. Survey respondents were 18 representatives of 33 organisations who were users of a comprehensive incident reporting and surveillance system - the Understanding and Preventing Led Outdoor Accidents Data System Software Tool (UPLOADS-ST) - and six out of 11 users of a modified system (UPLOADS-Lite). The survey collected information on user experiences in relation to system training, accessibility, ease of use, security, feedback and perceived value to the sector of collating and reporting data across organisations. FINDINGS: Only four UPLOADS-ST responding users found the system easy to use and just three considered entering incident reports to be easy. However, many considered the training on reporting incidents to be sufficient and that the incident reports contained relevant details. Fewer than half of respondents (seven for UPLOADS-ST, three for UPLOADS-Lite) believed entering data was a good use of staff time and resources. Nonetheless, a majority of respondents (seven for UPLOADS-ST, five for UPLOADS-Lite) found the reporting format easy to read and felt the information provided was useful for their organisation. CONCLUSIONS: Usability barriers to incident reporting were identified, particularly for UPLOADS-ST, including time constraints and user friendliness. The majority of users believed aggregating and reporting incident and injury data across organisations would be of value in making the led outdoor activity sector safer. Improving the utility of the surveillance systems will assist in ensuring their sustainability in the led outdoor activity sector.

Sociotechnical systems as a framework for regulatory system design and evaluation: Using Work Domain Analysis to examine a new regulatory system.

Like most work systems, the domain of adventure activities has seen a series of serious incidents and subsequent calls to improve regulation. Safety regulation systems aim to promote safety and reduce accidents. However, there is scant evidence they have led to improved safety outcomes. In fact there is some evidence that the poor integration of regulatory system components has led to adverse safety outcomes in some contexts. Despite this, there is an absence of methods for evaluating regulatory and compliance systems. This article argues that sociotechnical systems theory and methods provide a suitable framework for evaluating regulatory systems. This is demonstrated through an analysis of a recently introduced set of adventure activity regulations. Work Domain Analysis (WDA) was used to describe the regulatory system in terms of its functional purposes, values and priority measures, purpose-related functions, object-related processes and cognitive objects. This allowed judgement to be made on the nature of the new regulatory system and on the constraints that may impact its efficacy following implementation. Importantly, the analysis suggests that the new system's functional purpose of ensuring safe activities is not fully supported in terms of the functions and objects available to fulfil them. Potential improvements to the design of the system are discussed along with the implications for regulatory system design and evaluation across the safety critical domains generally.

A systems approach to understanding the identification and treatment of sport-related concussion in community rugby union.

AIM: The aim of the present study was to utilise a systems thinking approach to explore the perceived responsibilities for identifying and treating concussion held by different actors across the community rugby system (e.g., players, coaches, parents, medics, referees, and management), as well as their role-specific concussion management strategies. METHODS: A systems approach was taken to assess what different stakeholders within rugby systems perceive their roles to be regarding concussion identification and treatment. Through an online survey, 118 members of the amateur (community) rugby union system were asked about their role-specific concussion management responsibilities and strategies. Respondents included players, parents, medics, coaches, club managers, administrators, and volunteers. RESULTS: The majority of respondents indicated that they were able to identify the symptoms of rugby-related concussion, however, only medics stated their responsibility to use formal concussion assessments (e.g., SCAT2). A smaller number of the respondents indicated that they were involved in treating concussion within their current role/s (majority of which were medics). CONCLUSIONS: This study illustrated that the current challenges in the identification and treatment of rugby-related concussion in community sport may be due to role/responsibility confusion and possible overreliance on field-side medics. These findings offer insight into the possible limitations of the current concussion management guidelines and may offer empirically based direction for future revisions.