Patterns of Domain-Specific Learning Among Medical Undergraduate Students in Relation to Confidence in Their Physiology Knowledge: Insights From a Pre-post Study.

RESEARCH FOCUS: The promotion of domain-specific knowledge is a central goal of higher education and, in the field of medicine, it is particularly essential to promote global health. Domain-specific knowledge on its own is not exhaustive; confidence regarding the factual truth of this knowledge content is also required. An increase in both knowledge and confidence is considered a necessary prerequisite for making professional decisions in the clinical context. Especially the knowledge of human physiology is fundamental and simultaneously critical to medical decision-making. However, numerous studies have shown difficulties in understanding and misconceptions in this area of knowledge. Therefore, we investigate (i) how preclinical medical students acquire knowledge in physiology over the course of their studies and simultaneously gain confidence in the correctness of this knowledge as well as (ii) the interrelations between these variables, and (iii) how they affect the development of domain-specific knowledge. METHOD: In a pre-post study, 169 medical students' development of physiology knowledge and their confidence related to this knowledge were assessed via paper-pencil questionnaires before and after attending physiology seminars for one semester. Data from a longitudinal sample of n = 97 students were analyzed using mean comparisons, regression analyses, and latent class analyses (LCAs). In addition, four types of item responses were formed based on confidence and correctness in the knowledge test. RESULTS: We found a significant and large increase in the students' physiology knowledge, with task-related confidence being the strongest predictor (apart from learning motivation). Moreover, a significantly higher level of confidence at t2 was confirmed, with the level of prior confidence being a strong predictor (apart from knowledge at t2). Furthermore, based on the students' development of knowledge and confidence levels between measurement points, three empirically distinct groups were distinguished: knowledge gainers, confidence gainers, and overall gainers. The students whose confidence in incorrect knowledge increased constituted one particularly striking group. Therefore, the training of both knowledge and the ability to critically reflect on one's knowledge and skills as well as an assessment of their development in education is required, especially in professions such as medicine, where knowledge-based decisions made with confidence are of vital importance.

Epistemic Network Analyses of Economics Students' Graph Understanding: An Eye-Tracking Study.

Learning to solve graph tasks is one of the key prerequisites of acquiring domain-specific knowledge in most study domains. Analyses of graph understanding often use eye-tracking and focus on analyzing how much time students spend gazing at particular areas of a graph-Areas of Interest (AOIs). To gain a deeper insight into students' task-solving process, we argue that the gaze shifts between students' fixations on different AOIs (so-termed transitions) also need to be included in holistic analyses of graph understanding that consider the importance of transitions for the task-solving process. Thus, we introduced Epistemic Network Analysis (ENA) as a novel approach to analyze eye-tracking data of 23 university students who solved eight multiple-choice graph tasks in physics and economics. ENA is a method for quantifying, visualizing, and interpreting network data allowing a weighted analysis of the gaze patterns of both correct and incorrect graph task solvers considering the interrelations between fixations and transitions. After an analysis of the differences in the number of fixations and the number of single transitions between correct and incorrect solvers, we conducted an ENA for each task. We demonstrate that an isolated analysis of fixations and transitions provides only a limited insight into graph solving behavior. In contrast, ENA identifies differences between the gaze patterns of students who solved the graph tasks correctly and incorrectly across the multiple graph tasks. For instance, incorrect solvers shifted their gaze from the graph to the x-axis and from the question to the graph comparatively more often than correct solvers. The results indicate that incorrect solvers often have problems transferring textual information into graphical information and rely more on partly irrelevant parts of a graph. Finally, we discuss how the findings can be used to design experimental studies and for innovative instructional procedures in higher education.

The Role of Students' Beliefs When Critically Reasoning From Multiple Contradictory Sources of Information in Performance Assessments.

Critical reasoning (CR) when confronted with contradictory information from multiple sources is a crucial ability in a knowledge-based society and digital world. Using information without critically reflecting on the content and its quality may lead to the acceptance of information based on unwarranted claims. Previous personal beliefs are assumed to play a decisive role when it comes to critically differentiating between assertions and claims and warranted knowledge and facts. The role of generic epistemic beliefs on critical stance and attitude in reflectively dealing with information is well researched. Relatively few studies however, have been conducted on the influence of domain-specific beliefs, i.e., beliefs in relation to specific content encountered in a piece of information or task, on the reasoning process, and on how these beliefs may affect decision-making processes. This study focuses on students' task- and topic-related beliefs that may influence their reasoning when dealing with multiple and partly contradictory sources of information. To validly assess CR among university students, we used a newly developed computer-based performance assessment in which the students were confronted with an authentic task which contains theoretically defined psychological stimuli for measuring CR. To investigate the particular role of task- and topic-related beliefs on CR, a purposeful sample of 30 university students took part in a performance assessment and then were interviewed immediately afterward. In the semi-structured cognitive interviews, the participants' task-related beliefs were assessed. Based on qualitative analyses of the interview transcripts, three distinct profiles of decision-making among students have been identified. More specifically, the different types of students' beliefs and attitudes derived from the cognitive interview data suggest their influence on information processing, reasoning approaches and decision-making. The results indicated that the students' beliefs had an influence on their selection, critical evaluation and use of information as well as on their reasoning processes and final decisions.

Changes in Students' Understanding of and Visual Attention on Digitally Represented Graphs Across Two Domains in Higher Education: A Postreplication Study.

Domain-specific understanding of digitally represented graphs is necessary for successful learning within and across domains in higher education. Two recent studies conducted a cross-sectional analysis of graph understanding in different contexts (physics and finance), task concepts, and question types among students of physics, psychology, and economics. However, neither changes in graph processing nor changes in test scores over the course of one semester have been sufficiently researched so far. This eye-tracking replication study with a pretest-posttest design examines and contrasts changes in physics and economics students' understanding of linear physics and finance graphs. It analyzes the relations between changes in students' gaze behavior regarding relevant graph areas, scores, and self-reported task-related confidence. The results indicate domain-specific, context- and concept-related differences in the development of graph understanding over the first semester, as well as its successful transferability across the different contexts and concepts. Specifically, we discovered a tendency of physics students to develop a task-independent overconfidence in the graph understanding during the first semester.

Undergraduate Students' Critical Online Reasoning-Process Mining Analysis.

To successfully learn using open Internet resources, students must be able to critically search, evaluate and select online information, and verify sources. Defined as critical online reasoning (COR), this construct is operationalized on two levels in our study: (1) the student level using the newly developed Critical Online Reasoning Assessment (CORA), and (2) the online information processing level using event log data, including gaze durations and fixations. The written responses of 32 students for one CORA task were scored by three independent raters. The resulting score was operationalized as "task performance," whereas the gaze fixations and durations were defined as indicators of "process performance." Following a person-oriented approach, we conducted a process mining (PM) analysis, as well as a latent class analysis (LCA) to test whether-following the dual-process theory-the undergraduates could be distinguished into two groups based on both their process and task performance. Using PM, the process performance of all 32 students was visualized and compared, indicating two distinct response process patterns. One group of students (11), defined as "strategic information processers," processed online information more comprehensively, as well as more efficiently, which was also reflected in their higher task scores. In contrast, the distributions of the process performance variables for the other group (21), defined as "avoidance information processers," indicated a poorer process performance, which was also reflected in their lower task scores. In the LCA, where two student groups were empirically distinguished by combining the process performance indicators and the task score as a joint discriminant criterion, we confirmed these two COR profiles, which were reflected in high vs. low process and task performances. The estimated parameters indicated that high-performing students were significantly more efficient at conducting strategic information processing, as reflected in their higher process performance. These findings are so far based on quantitative analyses using event log data. To enable a more differentiated analysis of students' visual attention dynamics, more in-depth qualitative research of the identified student profiles in terms of COR will be required.

Performance assessment to investigate the domain specificity of instructional skills among pre-service and in-service teachers of mathematics and economics.

BACKGROUND: Key elements of instructional quality include the teacher's ability to immediately react in domain-specific classroom situations. Such skills - defined as action-related skills - can only be validly assessed using authentic representations of real-life teaching practice. However, research has not yet explained how teachers apply domain-specific knowledge for teaching and to what extent action-related skills are transferable from one domain to another. AIMS: Our study aims to examine (1) the relationship between action-related skills, content knowledge, and pedagogical content knowledge, and (2) the domain specificity of action-related skills of (prospective) teachers in the two domains of mathematics and economics. SAMPLE(S): We examined German pre-service and in-service teachers of mathematics (N = 239) and economics (N = 321), including n = 96 (prospective) teachers who teach both subjects. METHODS: Action-related skills in mathematics and economics were measured using video-based performance assessments. Content knowledge and pedagogical content knowledge were assessed using established paper-pencil tests. Correlation analyses, linear regressions, and a path model were applied. RESULTS: In mathematics and economics, we find a similar pattern of moderate correlations between action-related skills, content knowledge, and pedagogical content knowledge. Moreover, a significant correlation between action-related skills in mathematics and economics can be explained almost entirely by underlying relations between content knowledge and pedagogical content knowledge in both domains. CONCLUSIONS: Our findings suggest that action-related skills empirically differ from domain-specific knowledge and should be considered as domain-specific constructs. This indicates that teacher education should not only focus on domain-specific teacher knowledge, but may also provide learning opportunities for action-related skills in each domain.

On the complementarity of holistic and analytic approaches to performance assessment scoring.

BACKGROUND: A holistic approach to performance assessment recognizes the theoretical complexity of multifaceted critical thinking (CT), a key objective of higher education. However, issues related to reliability, interpretation, and use arise with this approach. AIMS AND METHOD: Therefore, we take an analytic approach to scoring students' written responses on a performance assessment. We focus on the complementarity of holistic and analytic approaches and on whether theoretically developed analytical scoring rubrics can produce sub-scores that may measure the 'whole' performance in a holistic assessment. SAMPLE: We use data from the Wind Turbines performance assessment, developed in the iPAL project this study where 55 students at a German university participated. RESULTS: The (sub)scores generated from the scoring scheme empirically reproduced the theoretically assumed structure of CT, with valid and reliable scores in a three-dimensional model. The proposed interpretation of CT as assessed with a performance assessment and measured by the rating scheme was supported preliminarily. CONCLUSION: Our results support the complementarity of holistic and analytic approaches to assessing CT. When combined, they provide interpretable scores for a complex, multifaceted construct useful in diagnostic contexts.