Metacognitive judgments during visuomotor learning reflect the integration of error history.

People form metacognitive representations of their own abilities across a range of tasks. How these representations are influenced by errors during learning is poorly understood. Here, we ask how metacognitive confidence judgments of performance during motor learning are shaped by the learner's recent history of errors. Across four motor learning experiments, our computational modeling approach demonstrated that people's confidence judgments are best explained by a recency-weighted averaging of visually observed errors. Moreover, in the formation of these confidence estimates, people appear to reweight observed motor errors according to a subjective cost function. Confidence judgments were adaptive, incorporating recent motor errors in a manner that was sensitive to the volatility of the learning environment, integrating a shallower history when the environment was more volatile. Finally, confidence tracked motor errors in the context of both implicit and explicit motor learning but only showed evidence of influencing behavior in the latter. Our study thus provides a novel descriptive model that successfully approximates the dynamics of metacognitive judgments during motor learning.NEW & NOTEWORTHY This study examined how, during visuomotor learning, people's confidence in their performance is shaped by their recent history of errors. Using computational modeling, we found that confidence incorporated recent error history, tracked subjective error costs, was sensitive to environmental volatility, and in some contexts may influence learning. Together, these results provide a novel model of metacognitive judgments during motor learning that could be applied to future computational and neural studies at the interface of higher-order cognition and motor control.

The Role of Semantic Associations as a Metacognitive Cue in Creative Idea Generation.

Is my idea creative? This question directs investing in companies and choosing a research agenda. Following previous research, we focus on the originality of ideas and consider their association with self-assessments of idea generators regarding their own originality. We operationalize the originality score as the frequency (%) of each idea within a sample of participants and originality judgment as the self-assessment of this frequency. Initial evidence suggests that originality scores and originality judgments are produced by separate processes. As a result, originality judgments are prone to biases. So far, heuristic cues that lead to such biases are hardly known. We used methods from computational linguistics to examine the semantic distance as a potential heuristic cue underlying originality judgments. We examined the extent to which the semantic distance would contribute additional explanatory value in predicting originality scores and originality judgments, above and beyond cues known from previous research. In Experiment 1, we re-analyzed previous data that compared originality scores and originality judgments after adding the semantic distance of the generated ideas from the stimuli. We found that the semantic distance contributed to the gap between originality scores and originality judgments. In Experiment 2, we manipulated the examples given in task instructions to prime participants with two levels of idea originality and two levels of semantic distance. We replicated Experiment 1 in finding the semantic distance as a biasing factor for originality judgments. In addition, we found differences among the conditions in the extent of the bias. This study highlights the semantic distance as an unacknowledged metacognitive cue and demonstrates its biasing power for originality judgments.

Assessing Metacognitive Regulation during Problem Solving: A Comparison of Three Measures.

Metacognition is hypothesized to play a central role in problem solving and self-regulated learning. Various measures have been developed to assess metacognitive regulation, including survey items in questionnaires, verbal protocols, and metacognitive judgments. However, few studies have examined whether these measures assess the same metacognitive skills or are related to the same learning outcomes. To explore these questions, we investigated the relations between three metacognitive regulation measures given at various points during a learning activity and subsequent test. Verbal protocols were collected during the learning activity, questionnaire responses were collected after the learning tasks but before the test, and judgments of knowing (JOKs) were collected during the test. We found that the number of evaluation statements as measured via verbal protocols was positively associated with students' responses on the control/debugging and evaluation components of the questionnaire. There were also two other positive trends. However, the number of monitoring statements was negatively associated with students' responses on the monitoring component of the questionnaire and their JOKs on the later test. Each measure was also related to some aspect of performance, but the particular metacognitive skill, the direction of the effect, and the type of learning outcome differed across the measures. These results highlight the heterogeneity of outcomes across the measures, with each having different affordances and constraints for use in research and educational practice.

Does Performance Calibration Generalize Across Sport Tasks? A Multiexperiment Study in Physical Education.

Three experiments examined students' calibration in physical education in relation to task characteristics. Participants in the 3 experiments were 388 students. Calibration accuracy and bias were calculated based on students' predicted and actual performance in tests including variations of a sport task (basketball shooting) and tasks from different sports (basketball and soccer). An overconfidence effect was found in all experiments, and evidence regarding the hard-easy effect emerged. High compared with low performers were more accurate, and some variations with respect to gender also emerged. The magnitude of calibration error was similar across tasks, whereas approximately half of the students were consistent in the direction of calibration (most of them were overestimators). Results are discussed with reference to theoretical and empirical evidence associated with performance calibration and self-regulated learning in physical education. Methodological issues, practical implications, and future directions are also discussed.

Individual consistency in the accuracy and distribution of confidence judgments.

We examine which aspects of the confidence distributions - its shape, its bias toward higher or lower values, and its ability to distinguish correct from erred trials - are idiosyncratic of the who (individual specificity), the when (variability across days) and the what (task specificity). Measuring confidence across different sessions of four different perceptual tasks we show that: (1) Confidence distributions are virtually identical when measured in different days for the same subject and the same task, constituting a subjective fingerprint, (2) The capacity of confidence reports to distinguish correct from incorrect responses is only modestly (but significantly) correlated when compared across tasks, (3) Confidence distributions are very similar for tasks that involve different sensory modalities but have similar structure, (4) Confidence accuracy is independent of the mean and width of the confidence distribution, (5) The mean of the confidence distribution (an individual's confidence bias) constitutes the most efficient indicator to infer a subject's identity from confidence reports and (6) Confidence bias measured in simple perceptual decisions correlates with an individual's optimism bias measured with standard questionnaire.

METACOGNITIVE SCAFFOLDS IMPROVE SELF-JUDGMENTS OF ACCURACY IN A MEDICAL INTELLIGENT TUTORING SYSTEM.

In this study, we examined the effect of two metacognitive scaffolds on the accuracy of confidence judgments made while diagnosing dermatopathology slides in SlideTutor. Thirty-one (N = 31) first- to fourth-year pathology and dermatology residents were randomly assigned to one of the two scaffolding conditions. The cases used in this study were selected from the domain of Nodular and Diffuse Dermatitides. Both groups worked with a version of SlideTutor that provided immediate feedback on their actions for two hours before proceeding to solve cases in either the Considering Alternatives or Playback condition. No immediate feedback was provided on actions performed by participants in the scaffolding mode. Measurements included learning gains (pre-test and post-test), as well as metacognitive performance, including Goodman-Kruskal Gamma correlation, bias, and discrimination. Results showed that participants in both conditions improved significantly in terms of their diagnostic scores from pre-test to post-test. More importantly, participants in the Considering Alternatives condition outperformed those in the Playback condition in the accuracy of their confidence judgments and the discrimination of the correctness of their assertions while solving cases. The results suggested that presenting participants with their diagnostic decision paths and highlighting correct and incorrect paths helps them to become more metacognitively accurate in their confidence judgments.