Assessing approaches to learning with nonparametric multidimensional scaling.

This article reports on a trace-based assessment of approaches to learning used by middle school aged children who interacted with NASA Mars Mission science, technology, engineering and mathematics (STEM) games in Whyville, an online game environment with 8 million registered young learners. The learning objectives of two games included awareness and knowledge of NASA missions, developing knowledge and skills of measurement and scaling, applying measurement for planetary comparisons in the solar system. Trace data from 1361 interactions were analysed with nonparametric multidimensional scaling methods, which permitted visual examination and statistical validation, and provided an example and proof of concept for the multidimensional scaling approach to analysis of time-based behavioural data from a game or simulation. Differences in approach to learning were found illustrating the potential value of the methodology to curriculum and game-based learning designers as well as other creators of online STEM content for pre-college youth. The theoretical framework of the method and analysis makes use of the Epistemic Network Analysis toolkit as a post hoc data exploration platform, and the discussion centres on issues of semantic interpretation of interaction end-states and the application of evidence centred design in post hoc analysis.

Evidence centered design framework and dynamic bayesian network for modeling learning progression in online assessment system.

An overarching mission of the educational assessment community today is strengthening the connection between assessment and learning. To support this effort, researchers draw variously on developments across technology, analytic methods, assessment design frameworks, research in learning domains, and cognitive, social, and situated psychology. The study lays out the connection among three such developments, namely learning progressions, evidence-centered assessment design (ECD), and dynamic Bayesian modeling for measuring students' advancement along learning progression in a substantive domain. Their conjunction can be applied in both formative and summative assessment uses. In addition, this study conducted an application study in domain of beginning computer network engineering for illustrating the ideas with data drawn from the Cisco Networking Academy's online assessment system.

Applying Evidence-Centered Design to Measure Psychological Resilience: The Development and Preliminary Validation of a Novel Simulation-Based Assessment Methodology.

Modern technologies have enabled the development of dynamic game- and simulation-based assessments to measure psychological constructs. This has highlighted their potential for supplementing other assessment modalities, such as self-report. This study describes the development, design, and preliminary validation of a simulation-based assessment methodology to measure psychological resilience-an important construct for multiple life domains. The design was guided by theories of resilience, and principles of evidence-centered design and stealth assessment. The system analyzed log files from a simulated task to derive individual trajectories in response to stressors. Using slope analyses, these trajectories were indicative of four types of responses to stressors: thriving, recovery, surviving, and succumbing. Using Machine Learning, the trajectories were predictive of self-reported resilience (Connor-Davidson Resilience Scale) with high accuracy, supporting construct validity of the simulation-based assessment. These findings add to the growing evidence supporting the utility of gamified assessment of psychological constructs. Importantly, these findings address theoretical debates about the construct of resilience, adding to its theory, supporting the combination of the "trait" and "process" approaches to its operationalization.

A Novel Integrating Virtual Reality Approach for the Assessment of the Attachment Behavioral System.

Virtual reality (VR) technology represents a novel and powerful tool for behavioral research in psychological assessment. VR provides simulated experiences able to create the sensation of undergoing real situations. Users become active participants in the virtual environment seeing, hearing, feeling, and actuating as if they were in the real world. Currently, the most psychological VR applications concern the treatment of various mental disorders but not the assessment, that it is mainly based on paper and pencil tests. The observation of behaviors is costly, labor-intensive, and it is hard to create social situations in laboratory settings, even if the observation of actual behaviors could be particularly informative. In this framework, social stressful experiences can activate various behaviors of attachment for a significant person that can help to control and soothe them to promote individual's well-being. Social support seeking, physical proximity, and positive and negative behaviors represent the main attachment behaviors that people can carry out during experiences of distress. We proposed VR as a novel integrating approach to measure real attachment behaviors. The first studies on attachment behavioral system by VR showed the potentiality of this approach. To improve the assessment during the VR experience, we proposed virtual stealth assessment (VSA) as a new method. VSA could represent a valid and novel technique to measure various psychological attributes in real-time during the virtual experience. The possible use of this method in psychology could be to generate a more complete, exhaustive, and accurate individual's psychological evaluation.