Test procedure for the evaluation of partially automated driving HMI including driver monitoring systems in driving simulation.

The proposed test procedure presents an approach for the evaluation of the usability of partial automated driving HMI including driver monitoring systems in driving simulation. This procedure is based on a definition of requirements that a Level 2 HMI and its included driver monitoring system must fulfill in order to guarantee that the drivers understand their responsibilities of continuously monitoring the driving environment and the status of the partial automated driving system. These requirements are used to define the evaluation criteria that have to be validated in the test as well as the use cases in which these criteria can be assessed. The result is a detailed and comprehensive test guide including the specification of the test drives, the necessary instructions, the test environment and the recruiting criteria for the test sample.•Evaluation of usability aspects of level 2 automated driving HMI including driver monitoring systems•Based on the definition of requirements for L2 HMI•Test guide including the definition of use cases, evaluation criteria and testing conditions in driving simulation.

PADA-AD Test - a standardized test method for assessing takeover performance of drivers during automated driving.

The presented method describes a standardized test procedure for the evaluation of takeover performance of drivers during automated driving. It was primarily developed to be used for evaluating Level 3 systems (conditional automated driving). It should be applied in a driving simulator environment during the development phase of a system. The method consists of a test course on a three-lane highway with 12 test scenarios where the driver repeatedly has to take over control from the automated system. Each scenario requires the driver to build up an adequate situation awareness in order to take the decision for the correct action. The method was explicitly designed to map the four relevant steps in the takeover process of automated driving: Perception - Awareness - Decision - Action and is therefore called PADA-AD Test for automated driving. The method description contains guidelines with regard to the specification of the test course and the included test scenarios, the design and duration of the automated drives, the non-driving related task to be performed during the automated drives, the instructions to be given to the subjects and finally the measures for evaluating takeover performance of the drivers. • A test procedure for the evaluation of takeover performance of drivers during automated driving was developed for usage in a driving simulator during the development phase of a system/HMI •The test course enables the assessment of the driver's takeover performance in various test scenarios including higher cognitive processes•The method is highly standardized and thus replicable through use of a predetermined test course with clearly defined scenarios, reduced environmental conditions and "popping up" of situational elements.

Test procedure for evaluating the human-machine interface of vehicles with automated driving systems.

Objective: The human-machine interface (HMI) is a crucial part of every automated driving system (ADS). In the near future, it is likely that-depending on the operational design domain (ODD)-different levels of automation will be available within the same vehicle. The capabilities of a given automation level as well as the operator's responsibilities must be communicated in an appropriate way. To date, however, there are no agreed-upon evaluation methods that can be used by human factors practitioners as well as researchers to test this. Methods: We developed an iterative test procedure that can be applied during the product development cycle of ADS. The test procedure is specifically designed to evaluate whether minimum requirements as proposed in NHTSA's automated vehicle policy are met. Results: The proposed evaluation protocol includes (a) a method to identify relevant use cases for testing on the basis of all theoretically possible steady states and mode transitions of a given ADS; (b) an expert-based heuristic assessment to evaluate whether the HMI complies with applicable norms, standards, and best practices; and (c) an empirical evaluation of ADS HMIs using a standardized design for user studies and performance metrics. Conclusions: Each can be used as a stand-alone method or in combination to generate objective, reliable, and valid evaluations of HMIs, focusing on whether they meet minimum requirements. However, we also emphasize that other evaluation aspects such as controllability, misuse, and acceptance are not within the scope of the evaluation protocol.

Expert-based controllability assessment of control transitions from automated to manual driving.

Up to a level of full vehicle automation, drivers will have to be available as a fallback level and take back manual control of the vehicle in case of system limits or failures. Before introducing automated vehicles to the consumer market, the controllability of these control transitions has to be demonstrated. This paper presents a novel procedure for an expert-based controllability assessment of control transitions from automated to manual driving. A standardized rating scheme is developed that allows trained raters to integrate different aspects of driving performance during control transitions (e.g., quality of lateral and longitudinal control, adequateness of signalling to other road users, etc.) into one global controllability measure based on video material of the driving situation. The method is adapted from an existing assessment procedure that has been successfully applied to assess the criticality of driving situations in manual driving conditions (e.g., assessment of substance-induced impairments, assessment of fitness-to-drive of novice drivers). This paper presents the rating procedure, including instructions of how to code relevant qualities of the drivers' performance with accompanying video-demonstrations, and material used for rater training. •A rating procedure for an expert-based controllability assessment of control transitions from automated to manual driving based on observation of video material was adapted from an existing method used in studies on manual driving.•The advantage of this method consists in an integration of different dimensions of driving performance (e.g., operational and tactical driving behaviour, criticality of the situation) into one global controllability measure.•The method allows an assessment and comparison of diverse take-over scenarios, detached from driver performance variables.•The accompanying video-based training material allows reproducible and reliable execution of the rating procedure.

Effects of Non-Driving Related Task Modalities on Takeover Performance in Highly Automated Driving.

OBJECTIVE: Aim of the study was to evaluate the impact of different non-driving related tasks (NDR tasks) on takeover performance in highly automated driving. BACKGROUND: During highly automated driving, it is allowed to engage in NDR tasks temporarily. However, drivers must be able to take over control when reaching a system limit. There is evidence that the type of NDR task has an impact on takeover performance, but little is known about the specific task characteristics that account for performance decrements. METHOD: Thirty participants drove in a simulator using a highly automated driving system. Each participant faced five critical takeover situations. Based on assumptions of Wickens's multiple resource theory, stimulus and response modalities of a prototypical NDR task were systematically manipulated. Additionally, in one experimental group, the task was locked out simultaneously with the takeover request. RESULTS: Task modalities had significant effects on several measures of takeover performance. A visual-manual texting task degraded performance the most, particularly when performed handheld. In contrast, takeover performance with an auditory-vocal task was comparable to a baseline without any task. Task lockout was associated with faster hands-on-wheel times but not altered brake response times. CONCLUSION: Results showed that NDR task modalities are relevant factors for takeover performance. An NDR task lockout was highly accepted by the drivers and showed moderate benefits for the first takeover reaction. APPLICATION: Knowledge about the impact of NDR task characteristics is an enabler for adaptive takeover concepts. In addition, it might help regulators to make decisions on allowed NDR tasks during automated driving.