METACOGNITIVE SCAFFOLDS IMPROVE SELF-JUDGMENTS OF ACCURACY IN A MEDICAL INTELLIGENT TUTORING SYSTEM.

In this study, we examined the effect of two metacognitive scaffolds on the accuracy of confidence judgments made while diagnosing dermatopathology slides in SlideTutor. Thirty-one (N = 31) first- to fourth-year pathology and dermatology residents were randomly assigned to one of the two scaffolding conditions. The cases used in this study were selected from the domain of Nodular and Diffuse Dermatitides. Both groups worked with a version of SlideTutor that provided immediate feedback on their actions for two hours before proceeding to solve cases in either the Considering Alternatives or Playback condition. No immediate feedback was provided on actions performed by participants in the scaffolding mode. Measurements included learning gains (pre-test and post-test), as well as metacognitive performance, including Goodman-Kruskal Gamma correlation, bias, and discrimination. Results showed that participants in both conditions improved significantly in terms of their diagnostic scores from pre-test to post-test. More importantly, participants in the Considering Alternatives condition outperformed those in the Playback condition in the accuracy of their confidence judgments and the discrimination of the correctness of their assertions while solving cases. The results suggested that presenting participants with their diagnostic decision paths and highlighting correct and incorrect paths helps them to become more metacognitively accurate in their confidence judgments.

Investigating the Impact of Augmented Reality and BIM on Retrofitting Training for Non-Experts.

Augmented Reality (AR) tools have shown significant potential in providing on-site visualization of Building Information Modeling (BIM) data and models for supporting construction evaluation, inspection, and guidance. Retrofitting existing buildings, however, remains a challenging task requiring more innovative solutions to successfully integrate AR and BIM. This study aims to investigate the impact of AR+BIM technology on the retrofitting training process and assess the potential for future on-site usage. We conducted a study with 64 non-expert participants, who were asked to perform a common retrofitting procedure of an electrical outlet installation using either an AR+BIM system or a standard printed blueprint documentation set. Our findings indicate that AR+BIM reduced task time significantly and improved performance consistency across participants, while also decreasing the physical and cognitive demands of the training. This study provides a foundation for augmenting future retrofitting construction research that can extend the use of [Formula: see text] technology, thus facilitating more efficient retrofitting of existing buildings. A video presentation of this article and all supplemental materials are available at https://github.com/DesignLabUCF/SENSEable_RetrofittingTraining.

THNN - A Neural Network Model for Telehealth Data Incompleteness Prediction.

In modern-day medical practices, practitioners and physicians are adapting to new technologies and utilizing new methods of communication with patients. Telemedicine, or telehealth, is one of the newest innovations in medical technology, enabling practitioners to communicate with their patients over the phone, video conferencing, or chat. However, clinical data and sentiments/attitudes are often not reflected in the practitioner's analysis and diagnosis of the patients they serve. As a solution to the problem of data incompleteness in telehealth, THNN allows medical practices to accommodate for possible missing or incomplete data and provide a greater quality of care overall. Through an ensemble of Natural Language Processing (NLP) and AI-enabled systems, THNN produces sentiment and incompleteness mapping to provide seamless results.Clinical relevance- The method presented utilizes telehealth natural language data to process the sentiments of patients and the incompleteness found in the conversations, increasing the possibility of improved healthcare outcomes.

Identifying the effects of scaffolding on learners' temporal deployment of self-regulated learning operations during game-based learning using multimodal data.

Introduction: Self-regulated learning (SRL), or learners' ability to monitor and change their own cognitive, affective, metacognitive, and motivational processes, encompasses several operations that should be deployed during learning including Searching, Monitoring, Assembling, Rehearsing, and Translating (SMART). Scaffolds are needed within GBLEs to both increase learning outcomes and promote the accurate and efficient use of SRL SMART operations. This study aims to examine how restricted agency (i.e., control over one's actions) can be used to scaffold learners' SMART operations as they learn about microbiology with Crystal Island, a game-based learning environment. Methods: Undergraduate students (N = 94) were randomly assigned to one of two conditions: (1) Full Agency, where participants were able to make their own decisions about which actions they could take; and (2) Partial Agency, where participants were required to follow a pre-defined path that dictated the order in which buildings were visited, restricting one's control. As participants played Crystal Island, participants' multimodal data (i.e., log files, eye tracking) were collected to identify instances where participants deployed SMART operations. Results: Results from this study support restricted agency as a successful scaffold of both learning outcomes and SRL SMART operations, where learners who were scaffolded demonstrated more efficient and accurate use of SMART operations. Discussion: This study provides implications for future scaffolds to better support SRL SMART operations during learning and discussions for future directions for future studies scaffolding SRL during game-based learning.

Capturing Sequences of Learners' Self-Regulatory Interactions With Instructional Material During Game-Based Learning Using Auto-Recurrence Quantification Analysis.

Undergraduate students (N = 82) learned about microbiology with Crystal Island, a game-based learning environment (GBLE), which required participants to interact with instructional materials (i.e., books and research articles, non-player character [NPC] dialogue, posters) spread throughout the game. Participants were randomly assigned to one of two conditions: full agency, where they had complete control over their actions, and partial agency, where they were required to complete an ordered play-through of Crystal Island. As participants learned with Crystal Island, log-file and eye-tracking time series data were collected to pinpoint instances when participants interacted with instructional materials. Hierarchical linear growth models indicated relationships between eye gaze dwell time and (1) the type of representation a learner gathered information from (i.e., large sections of text, poster, or dialogue); (2) the ability of the learner to distinguish relevant from irrelevant information; (3) learning gains; and (4) agency. Auto-recurrence quantification analysis (aRQA) revealed the degree to which repetitive sequences of interactions with instructional material were random or predictable. Through hierarchical modeling, analyses suggested that greater dwell times and learning gains were associated with more predictable sequences of interaction with instructional materials. Results from hierarchical clustering found that participants with restricted agency and more recurrent action sequences had greater learning gains. Implications are provided for how learning unfolds over learners' time in game using a non-linear dynamical systems analysis and the extent to which it can be supported within GBLEs to design advanced learning technologies to scaffold self-regulation during game play.

Lessons Learned and Future Directions of MetaTutor: Leveraging Multichannel Data to Scaffold Self-Regulated Learning With an Intelligent Tutoring System.

Self-regulated learning (SRL) is critical for learning across tasks, domains, and contexts. Despite its importance, research shows that not all learners are equally skilled at accurately and dynamically monitoring and regulating their self-regulatory processes. Therefore, learning technologies, such as intelligent tutoring systems (ITSs), have been designed to measure and foster SRL. This paper presents an overview of over 10 years of research on SRL with MetaTutor, a hypermedia-based ITS designed to scaffold college students' SRL while they learn about the human circulatory system. MetaTutor's architecture and instructional features are designed based on models of SRL, empirical evidence on human and computerized tutoring principles of multimedia learning, Artificial Intelligence (AI) in educational systems for metacognition and SRL, and research on SRL from our team and that of other researchers. We present MetaTutor followed by a synthesis of key research findings on the effectiveness of various versions of the system (e.g., adaptive scaffolding vs. no scaffolding of self-regulatory behavior) on learning outcomes. First, we focus on findings from self-reports, learning outcomes, and multimodal data (e.g., log files, eye tracking, facial expressions of emotion, screen recordings) and their contributions to our understanding of SRL with an ITS. Second, we elaborate on the role of embedded pedagogical agents (PAs) as external regulators designed to scaffold learners' cognitive and metacognitive SRL strategy use. Third, we highlight and elaborate on the contributions of multimodal data in measuring and understanding the role of cognitive, affective, metacognitive, and motivational (CAMM) processes. Additionally, we unpack some of the challenges these data pose for designing real-time instructional interventions that scaffold SRL. Fourth, we present existing theoretical, methodological, and analytical challenges and briefly discuss lessons learned and open challenges.

Factors affecting feeling-of-knowing in a medical intelligent tutoring system: the role of immediate feedback as a metacognitive scaffold.

Previous studies in our laboratory have shown the benefits of immediate feedback on cognitive performance for pathology residents using an intelligent tutoring system (ITS) in pathology. In this study, we examined the effect of immediate feedback on metacognitive performance, and investigated whether other metacognitive scaffolds will support metacognitive gains when immediate feedback is faded. Twenty-three participants were randomized into intervention and control groups. For both groups, periods working with the ITS under varying conditions were alternated with independent computer-based assessments. On day 1, a within-subjects design was used to evaluate the effect of immediate feedback on cognitive and metacognitive performance. On day 2, a between-subjects design was used to compare the use of other metacognitive scaffolds (intervention group) against no metacognitive scaffolds (control group) on cognitive and metacognitive performance, as immediate feedback was faded. Measurements included learning gains (a measure of cognitive performance), as well as several measures of metacognitive performance, including Goodman-Kruskal gamma correlation (G), bias, and discrimination. For the intervention group, we also computed metacognitive measures during tutoring sessions. Results showed that immediate feedback in an intelligent tutoring system had a statistically significant positive effect on learning gains, G and discrimination. Removal of immediate feedback was associated with decreasing metacognitive performance, and this decline was not prevented when students used a version of the tutoring system that provided other metacognitive scaffolds. Results obtained directly from the ITS suggest that other metacognitive scaffolds do have a positive effect on G and discrimination, as immediate feedback is faded. We conclude that immediate feedback had a positive effect on both metacognitive and cognitive gains in a medical tutoring system. Other metacognitive scaffolds were not sufficient to replace immediate feedback in this study. However, results obtained directly from the tutoring system are not consistent with results obtained from assessments. In order to facilitate transfer to real-world tasks, further research will be needed to determine the optimum methods for supporting metacognition as immediate feedback is faded.

Multimodal Protocol for Assessing Metacognition and Self-Regulation in Adults with Learning Difficulties.

Learning disabilities (LDs) encompass disorders of those who have difficulty learning and using academic skills, exhibiting performance below expectations for their chronological age in the areas of reading, writing, and/or mathematics. Each of the disorders making up the LDs involve different deficits; however, some commonalities can be found within that heterogeneity, such in terms of learning self-regulation and metacognition. Unlike in early ages and later educational levels, there are hardly any evidence-based evaluation protocols for adults with LDs. LDs influence academic performance but also have serious consequences in professional, social, and family contexts. In response to this, the current work proposes a multimodal evaluation protocol focused on metacognitive, self-regulation of learning, and emotional processes, which make up the basis of the difficulties in adults with LDs. The assessment is carried out through analysis of the on-line learning process using a variety methods, techniques, and sensors (e.g., eye tracking, facial expressions of emotion, physiological responses, concurrent verbalizations, log files, screen recordings of human-machine interactions) and off-line methods (e.g., questionnaires, interviews, and self-report measures). This theoretically-driven and empirically-based guideline aims to provide an accurate assessment of LDs in adulthood in order to design effective prevention and intervention proposals.

Multiple Negative Emotions During Learning With Digital Learning Environments - Evidence on Their Detrimental Effect on Learning From Two Methodological Approaches.

Emotions are a core factor of learning. Studies have shown that multiple emotions are co-experienced during learning and have a significant impact on learning outcomes. The present study investigated the importance of multiple, co-occurring emotions during learning about human biology with MetaTutor, a hypermedia-based tutoring system. Person-centered as well as variable-centered approaches of cluster analyses were used to identify emotion clusters. The person-centered clustering analyses indicated three emotion profiles: a positive, negative and neutral profile. Students with a negative profile learned less than those with other profiles and also reported less usage of emotion regulation strategies. Emotion patterns identified through spectral co-clustering confirmed these results. Throughout the learning activity, emotions built a stable correlational structure of a positive, a negative, a neutral and a boredom emotion pattern. Positive emotion pattern scores before the learning activity and negative emotion pattern scores during the learning activity predicted learning, but not consistently. These results reveal the importance of negative emotions during learning with MetaTutor. Potential moderating factors and implications for the design and development of educational interventions that target emotions and emotion regulation with digital learning environments are discussed.