A response time model of the three-choice Mnemonic Similarity Task provides stable, mechanistically interpretable individual-difference measures.

Introduction: The Mnemonic Similarity Task (MST) is a widely used measure of individual tendency to discern small differences between remembered and presently presented stimuli. Significant work has established this measure as a reliable index of neurological and cognitive dysfunction and decline. However, questions remain about the neural and psychological mechanisms that support performance in the task. Methods: Here, we provide new insights into these questions by fitting seven previously-collected MST datasets (total N = 519), adapting a three-choice evidence accumulation model (the Linear Ballistic Accumulator). The model decomposes choices into automatic and deliberative components. Results: We show that these decomposed processes both contribute to the standard measure of behavior in this task, as well as capturing individual variation in this measure across the lifespan. We also exploit a delayed test/re-test manipulation in one of the experiments to show that model parameters exhibit improved stability, relative to the standard metric, across a 1 week delay. Finally, we apply the model to a resting-state fMRI dataset, finding that only the deliberative component corresponds to off-task co-activation in networks associated with long-term, episodic memory. Discussion: Taken together, these findings establish a novel mechanistic decomposition of MST behavior and help to constrain theories about the cognitive processes that support performance in the task.

Multimodal Data Fusion to Detect Preknowledge Test-Taking Behavior Using Machine Learning.

In various fields, including college admission, medical board certifications, and military recruitment, high-stakes decisions are frequently made based on scores obtained from large-scale assessments. These decisions necessitate precise and reliable scores that enable valid inferences to be drawn about test-takers. However, the ability of such tests to provide reliable, accurate inference on a test-taker's performance could be jeopardized by aberrant test-taking practices, for instance, practicing real items prior to the test. As a result, it is crucial for administrators of such assessments to develop strategies that detect potential aberrant test-takers after data collection. The aim of this study is to explore the implementation of machine learning methods in combination with multimodal data fusion strategies that integrate bio-information technology, such as eye-tracking, and psychometric measures, including response times and item responses, to detect aberrant test-taking behaviors in technology-assisted remote testing settings.

Associated Factors of Patients' Survival in Out of Hospital Cardiac Arrest; a Cross-sectional Study.

Introduction: Chinese populations have an increasingly high prevalence of cardiac arrest. This study aimed to investigate the prehospital associated factors of survival to hospital admission and discharge among out-of-hospital cardiac arrest (OHCA) adult cases in Macao Special Administrative Region (SAR), China. Methods: Baseline characteristics as well as prehospital factors of OHCA patients were collected from publicly accessible medical records and Macao Fire Services Bureau, China. Demographic and other prehospital OHCA characteristics of patients who survived to hospital admission and discharge were analyzed using multivariate logistic regression analysis. Results: A total of 904 cases with a mean age of 74.2±17.3 (range: 18-106) years were included (78%>65 years, 62% male). Initial shockable cardiac rhythm was the strongest predictor for survival to both hospital admission (OR=3.57, 95% CI: 2.26-5.63; p<0.001) and discharge (OR=12.40, 95% CI: 5.70-26.96; p<0.001). Being male (OR=1.63, 95% CI:1.08-2.46; p =0.021) and the lower emergency medical service (EMS) response time (OR=1.62, 95% CI: 1.12-2.34; p =0.010) were also associated with a 2-fold association with survival to hospital admission. In addition, access to prehospital defibrillation (OR=4.25, 95% CI: 1.78-10.12; p <0.001) had a 4-fold association with survival to hospital discharge. None of these associations substantively increased with age. Conclusion: The major OHCA predictors of survival were initial shockable cardiac rhythm, being male, lower EMS response time, and access to prehospital defibrillation. These findings indicate a need for increased public awareness and more education.

Behavioral signatures of the rapid recruitment of long-term memory to overcome working memory capacity limits.

Working- and long-term memory are often studied in isolation. To better understand the specific limitations of working memory, effort is made to reduce the potential influence of long-term memory on performance in working memory tasks (e.g., asking participants to remember artificial, abstract items rather than familiar real-world objects). However, in everyday life we use working- and long-term memory in tandem. Here, our goal was to characterize how long-term memory can be recruited to circumvent capacity limits in a typical visual working memory task (i.e., remembering colored squares). Prior work has shown that incidental repetitions of working memory arrays often do not improve visual working memory performance - even after dozens of incidental repetitions, working memory performance often shows no improvement for repeated arrays. Here, we used a whole-report working memory task with explicit rather than incidental repetitions of arrays. In contrast to prior work with incidental repetitions, in two behavioral experiments we found that explicit repetitions of arrays yielded robust improvement to working memory performance, even after a single repetition. Participants performed above chance at recognizing repeated arrays in a later long-term memory test, consistent with the idea that long-term memory was used to rapidly improve performance across array repetitions. Finally, we analyzed inter-item response times and we found a response time signature of chunk formation that only emerged after the array was repeated (inter-response time slowing after two to three items); thus, inter-item response times may be useful for examining the coordinated interaction of visual working and long-term memory in future work.

Gestures speed up responses to questions.

Most language use occurs in face-to-face conversation, which involves rapid turn-taking. Seeing communicative bodily signals in addition to hearing speech may facilitate such fast responding. We tested whether this holds for co-speech hand gestures by investigating whether these gestures speed up button press responses to questions. Sixty native speakers of Dutch viewed videos in which an actress asked yes/no-questions, either with or without a corresponding iconic hand gesture. Participants answered the questions as quickly and accurately as possible via button press. Gestures did not impact response accuracy, but crucially, gestures sped up responses, suggesting that response planning may be finished earlier when gestures are seen. How much gestures sped up responses was not related to their timing in the question or their timing with respect to the corresponding information in speech. Overall, these results are in line with the idea that multimodality may facilitate fast responding during face-to-face conversation.

Connecting process models to response times through Bayesian hierarchical regression analysis.

Process models specify a series of mental operations necessary to complete a task. We demonstrate how to use process models to analyze response-time data and obtain parameter estimates that have a clear psychological interpretation. A prerequisite for our analysis is a process model that generates a count of elementary information processing steps (EIP steps) for each trial of an experiment. We can estimate the duration of an EIP step by assuming that every EIP step is of random duration, modeled as draws from a gamma distribution. A natural effect of summing several random EIP steps is that the expected spread of the overall response time increases with a higher EIP step count. With modern probabilistic programming tools, it becomes relatively easy to fit Bayesian hierarchical models to data and thus estimate the duration of a step for each individual participant. We present two examples in this paper: The first example is children's performance on simple addition tasks, where the response time is often well predicted by the smaller of the two addends. The second example is response times in a Sudoku task. Here, the process model contains some random decisions and the EIP step count thus becomes latent. We show how our EIP regression model can be extended to such a case. We believe this approach can be used to bridge the gap between classical cognitive modeling and statistical inference and will be easily applicable to many use cases.

Refining ambulance clinical response models: The impact on ambulance response and emergency department presentations.

OBJECTIVE: The ambulance service in Victoria, Australia implemented a revised clinical response model (CRM) in 2016 which was designed to increase the diversion of low-acuity Triple Zero (000) calls to secondary telephone triage and reduce emergency ambulance dispatches. The present study evaluates the influence of the revised CRM on emergency ambulance response times and ED presentations. METHODS: A retrospective study of emergency calls for ambulance between 1 January 2015 and 31 December 2018. Ambulance data were linked with ED presentations occurring up to 48 h after contact. Interrupted time series analyses were used to evaluate the impact of the revised CRM. RESULTS: A total of 2 365 529 calls were included. The proportion allocated a Code 1 (time-critical, lights/sirens) dispatch decreased from 56.6 to 41.0% after implementation of the revised CRM. The proportion of calls not receiving an emergency ambulance increased from 10.4 to 19.6%. Interrupted time series analyses demonstrated an improvement in Code 1 cases attended within 15 min (Key Performance Indicator). However, for patients with out-of-hospital cardiac arrest or requiring lights and sirens transport to hospital, there was no improvement in response time performance. By the end of the study period, there was also no difference in the proportion of callers presenting to ED when compared with the estimated proportion assuming the revised CRM had not been implemented. CONCLUSION: The revised CRM was associated with improved Code 1 response time performance. However, there was no improvement in response times for high acuity patients, and no change in the proportion of callers presenting to ED.

A Model-Based Approach to the Disentanglement and Differential Treatment of Engaged and Disengaged Item Omissions.

Item omissions in large-scale assessments may occur for various reasons, ranging from disengagement to not being capable of solving the item and giving up. Current response-time-based classification approaches allow researchers to implement different treatments of item omissions presumably going back to different mechanisms. These approaches, however, are limited in that they require a clear-cut decision on the underlying missingness mechanism and do not allow to take the uncertainty in classification into account. We present a response-time-based model-based mixture modeling approach that overcomes this limitation. The approach (a) facilitates disentangling item omissions stemming from disengagement from those going back to solution behavior, (b) considers the uncertainty in omission classification, (c) allows for omission mechanisms to vary on the item-by-examinee level, (d) supports investigating person and item characteristics associated with different types of omission behavior, and (e) gives researchers flexibility in deciding on how to handle different types of omissions. The approach exhibits good parameter recovery under realistic research conditions. We illustrate the approach on data from the Programme for the International Assessment of Adult Competencies 2012 and compare it against previous classification approaches for item omissions.

Measuring Cognitive Function and Cognitive Decline with Response Time Data in NSHAP.

OBJECTIVES: Scholarly, clinical, and policy interest in cognitive function has grown over the last several decades in part due to large increases in Alzheimer's Disease and related dementias as populations age. However, adequate measures of cognitive function have not been available in many research data sets. We argue that a wealth of previously unexploited survey data exists to model cognition and cognitive decline. METHODS: We use metadata of the time it takes older respondents in the National Social Life, Health and Aging Survey, which we label response times (RT), to answer questions in a standard cognitive assessment. We compare several measures of RT to a survey-adapted form of the Montreal Cognitive Assessment (MoCA). RESULTS: We show that RT predict both concurrent and future MoCA scores. Our results show that longer and more varied RT at baseline predict lower MoCA scores five year later, net of baseline scores and controls. We also show that the effect of RT measures on predicting current MoCA differ for individuals of different races and ages, but are not different by gender. DISCUSSION: Our paper demonstrates that RT constitute a separate powerful measure of cognitive functioning. RT may be remarkably useful both to clinicians and social scientists because they can increase accuracy of cognitive assessment without increasing the time it takes to administer the assessment.

Comparisons of Self-Report With Objective Measurements Suggest Faster Responding but Little Change in Response Quality Over Time in Ecological Momentary Assessment Studies.

Response times (RTs) to ecological momentary assessment (EMA) items often decrease after repeated EMA administration, but whether this is accompanied by lower response quality requires investigation. We examined the relationship between EMA item RTs and EMA response quality. In one data set, declining response quality was operationalized as decreasing correspondence over time between subjective and objective measures of blood glucose taken at the same time. In a second EMA study data set, declining response quality was operationalized as decreasing correspondence between subjective ratings of memory test performance and objective memory test scores. We assumed that measurement error in the objective measures did not increase across time, meaning that decreasing correspondence across days within a person could be attributed to lower response quality. RTs to EMA items decreased across study days, while no decrements in the mean response quality were observed. Decreasing EMA item RTs across study days did not appear problematic overall.

A Mixture Fluency model using responses and response times with cognitive diagnosis model framework.

Random guessing behaviors are frequently observed in low-stakes assessments, often attributed to factors such as test-takers lacking motivation or experiencing time constraints and fatigue. Existing research suggests that responses stemming from random guessing behaviors introduce biases into the constructs and relationships of interest. This is particularly problematic when estimating the relationship between speed and ability. This study introduces a Mixture Fluency model designed to account for random guessing behaviors while utilizing valid response accuracy and response time to uncover students' latent attribute profiles. The model directly addresses a limitation present in the Fluency cognitive diagnostic model (Wang & Chen, Psychometrika, 85, 600-629, (2020), which assumes that test-takers consistently employ solution behaviors when answering questions. To investigate the effectiveness of the proposed Mixture Fluency model, we conducted a simulation study encompassing various simulation conditions. Results from this study not only confirm the model's ability to detect potential random guessing behaviors but also demonstrate its capacity to enhance the inference of targeted latent constructs within the assessment. Additionally, we showcase the practical utility of the proposed model through an application to real data.

Conditional Dependence across Slow and Fast Item Responses: With a Latent Space Item Response Modeling Approach.

There recently have been many studies examining conditional dependence between response accuracy and response times in cognitive tests. While most previous research has focused on revealing a general pattern of conditional dependence for all respondents and items, it is plausible that the pattern may vary across respondents and items. In this paper, we attend to its potential heterogeneity and examine the item and person specificities involved in the conditional dependence between item responses and response times. To this end, we use a latent space item response theory (LSIRT) approach with an interaction map that visualizes conditional dependence in response data in the form of item-respondent interactions. We incorporate response time information into the interaction map by applying LSIRT models to slow and fast item responses. Through empirical illustrations with three cognitive test datasets, we confirm the presence and patterns of conditional dependence between item responses and response times, a result consistent with previous studies. Our results further illustrate the heterogeneity in the conditional dependence across respondents, which provides insights into understanding individuals' underlying item-solving processes in cognitive tests. Some practical implications of the results and the use of interaction maps in cognitive tests are discussed.

Adopting Stimulus Detection Tasks for Cognitive Workload Assessment: Some Considerations.

OBJECTIVE: This article tackles the issue of correct data interpretation when using stimulus detection tasks for determining the operator's workload. BACKGROUND: Stimulus detection tasks are a relative simple and inexpensive means of measuring the operator's state. While stimulus detection tasks may be better geared to measure conditions of high workload, adopting this approach for the assessment of low workload may be more problematic. METHOD: This mini-review details the use of common stimulus detection tasks and their contributions to the Human Factors practice. It also borrows from the conceptual framework of the inverted-U shape model to discuss the issue of data interpretation. RESULTS: The evidence being discussed here highlights a clear limitation of stimulus detection task paradigms. CONCLUSION: There is an inherent risk in using a unidimensional tool like stimulus detection tasks as the primary source of information for determining the operator's psychophysiological state. APPLICATION: Two recommendations are put forward to Human Factors researchers and practitioners dealing with the interpretation conundrum of dealing with stimulus detection tasks.

Response times in rural areas for emergency medical services, fire and rescue services and voluntary first responders during out-of-hospital cardiac arrests.

Aim: To increase survival in out-of-hospital cardiac arrests (OHCA), great efforts are made to improve the number of voluntary first responders (VFR). However, evidence of the potential utility of such efforts is sparse, especially in rural areas. Therefore, the aim was to describe and compare response times for emergency medical services (EMS), fire and rescue services (FRS), and VFR during OHCA in relation to population density. Methods: This observational and comparative study was based on data including positions and time stamps for VFR and response times for EMS and FRS in a region in southern Sweden. Results: In total, 285 OHCAs between 1 July 2020 and 31 December 2021 were analysed. VFR had the shortest median response time in comparison to EMS and FRS in all studied population densities. The overall median (Q1-Q3) time gain for VFR was 03:07 (01:39-05:41) minutes. A small proportion (19.2%) of alerted VFR accepted the assignments. This is most problematic in rural and sub-rural areas, where there were low numbers of alerted VFR. Also, FRS had shorter response time than EMS in all studied population densities except in urban areas. Conclusion: The differences found in median response times between rural and urban areas are worrisome from an equality perspective. More focus should be placed on recruiting VFR, especially in rural areas since VFR can potentially contribute to saving more lives. Also, since FRS has a shorter response time than EMS in rural, sub-rural, and sub-urban areas, FRS should be dispatched more frequently.

Using item scores and response times in person-fit assessment.

The use of joint models for item scores and response times is becoming increasingly popular in educational and psychological testing. In this paper, we propose two new person-fit statistics for such models in order to detect aberrant behaviour. The first statistic is computed by combining two existing person-fit statistics: one for the item scores, and one for the item response times. The second statistic is computed directly using the likelihood function of the joint model. Using detailed simulations, we show that the empirical null distributions of the new statistics are very close to the theoretical null distributions, and that the new statistics tend to be more powerful than several existing statistics for item scores and/or response times. A real data example is also provided using data from a licensure examination.

Longitudinal joint modeling for assessing parallel interactive development of latent ability and processing speed using responses and response times.

To measure the parallel interactive development of latent ability and processing speed using longitudinal item response accuracy (RA) and longitudinal response time (RT) data, we proposed three longitudinal joint modeling approaches from the structural equation modeling perspective, namely unstructured-covariance-matrix-based longitudinal joint modeling, latent growth curve-based longitudinal joint modeling, and autoregressive cross-lagged longitudinal joint modeling. The proposed modeling approaches can not only provide the developmental trajectories of latent ability and processing speed individually, but also exploit the relationship between the change in latent ability and processing speed through the across-time relationships of these two constructs. The results of two empirical studies indicate that (1) all three models are practically applicable and have highly consistent conclusions in terms of the changes in ability and speed in the analysis of the same data set, and (2) additional analysis of the RT data and acquisition of individual processing speed measurements can reveal the parallel interactive development phenomena that are difficult to detect using RA data alone. Furthermore, the results of our simulation study demonstrate that the proposed Bayesian Markov chain Monte Carlo estimation algorithm can ensure accurate model parameter recovery for all three proposed longitudinal joint models. Finally, the implications of our findings are discussed from the research and practice perspectives.

A sequential Bayesian changepoint detection procedure for aberrant behaviours in computerized testing.

Changepoints are abrupt variations in a sequence of data in statistical inference. In educational and psychological assessments, it is essential to properly differentiate examinees' aberrant behaviours from solution behaviour to ensure test reliability and validity. In this paper, we propose a sequential Bayesian changepoint detection algorithm to monitor the locations of changepoints for response times in real time and, subsequently, further identify types of aberrant behaviours in conjunction with response patterns. Two simulation studies were conducted to investigate the efficiency and accuracy of the proposed detection procedure in terms of identifying one or multiple changepoints at different locations. In addition to manipulating the number and locations of changepoints, two types of aberrant behaviours were also considered: rapid guessing behaviour and cheating behaviour. Simulation results indicate that ability estimates could be improved after removing responses from aberrant behaviours identified by our approach. Two empirical examples were analysed to illustrate the application of the proposed sequential Bayesian changepoint detection procedure.

Information-Theoretic Neural Decoding Reproduces Several Laws of Human Behavior.

Human response times conform to several regularities including the Hick-Hyman law, the power law of practice, speed-accuracy trade-offs, and the Stroop effect. Each of these has been thoroughly modeled in isolation, but no account describes these phenomena as predictions of a unified framework. We provide such a framework and show that the phenomena arise as decoding times in a simple neural rate code with an entropy stopping threshold. Whereas traditional information-theoretic encoding systems exploit task statistics to optimize encoding strategies, we move this optimization to the decoder, treating it as a Bayesian ideal observer that can track transmission statistics as prior information during decoding. Our approach allays prominent concerns that applying information-theoretic perspectives to modeling brain and behavior requires complex encoding schemes that are incommensurate with neural encoding.

Detecting Preknowledge Cheating via Innovative Measures: A Mixture Hierarchical Model for Jointly Modeling Item Responses, Response Times, and Visual Fixation Counts.

Preknowledge cheating jeopardizes the validity of inferences based on test results. Many methods have been developed to detect preknowledge cheating by jointly analyzing item responses and response times. Gaze fixations, an essential eye-tracker measure, can be utilized to help detect aberrant testing behavior with improved accuracy beyond using product and process data types in isolation. As such, this study proposes a mixture hierarchical model that integrates item responses, response times, and visual fixation counts collected from an eye-tracker (a) to detect aberrant test takers who have different levels of preknowledge and (b) to account for nuances in behavioral patterns between normally-behaved and aberrant examinees. A Bayesian approach to estimating model parameters is carried out via an MCMC algorithm. Finally, the proposed model is applied to experimental data to illustrate how the model can be used to identify test takers having preknowledge on the test items.

The hazards of dealing with response time outliers.

The presence of outliers in response times can affect statistical analyses and lead to incorrect interpretation of the outcome of a study. Therefore, it is a widely accepted practice to try to minimize the effect of outliers by preprocessing the raw data. There exist numerous methods for handling outliers and researchers are free to choose among them. In this article, we use computer simulations to show that serious problems arise from this flexibility. Choosing between alternative ways for handling outliers can result in the inflation of p-values and the distortion of confidence intervals and measures of effect size. Using Bayesian parameter estimation and probability distributions with heavier tails eliminates the need to deal with response times outliers, but at the expense of opening another source of flexibility.