Learning how to learn using simulation: Unpacking disguised feedback using a qualitative analysis of doctors' telephone talk.

INTRODUCTION: Telephone talk between clinicians represents a substantial workplace activity in postgraduate clinical education, yet junior doctors receive little training in goal-directed, professional telephone communication. AIM: To assess educational needs for telephone talk and develop a simulation-based educational intervention. METHODS: Thematic analysis of 17 semi-structured interviews with doctors-in-training from various training levels and specialties. RESULTS: We identified essential elements to incorporate into simulation-based telephone talk, including common challenging situations for junior doctors as well as explicit and informal aspects that promote learning. These elements have implications for both junior doctors and clinical supervisors, including: (a) explicit teaching and feedback practices and (b) informal conversational interruptions and questions. The latter serve as "disguised" feedback, which aligns with recent conceptualizations of feedback as "performance relevant information". CONCLUSIONS: In addition to preparing clinical supervisors to support learning through telephone talk, we propose several potential educational strategies: (a) embedding telephone communication skills throughout simulation activities and (b) developing stand-alone curricular elements to sensitize junior doctors to "disguised" feedback during telephone talk as a mechanism to augment future workplace learning, i.e. 'learning how to learn' through simulation.

Evaluating real-patient learning in medical education - Hungarian validation of the Manchester Clinical Placement Index.

Introduction: The Manchester Clinical Placement Index (MCPI) is an instrument to measure medical undergraduates' real-patient learning in communities of practice both in hospital and in GP placements. Its suitability to evaluate the quality of placement learning environments has been validated in an English-language context; however, there is a lack of evidence for its applicability in other languages. Our aim was to thoroughly explore the factor structure and the key psychometric properties of the Hungarian language version. Methods: MCPI is an 8-item, mixed-method instrument which evaluates the quality of clinical placements as represented by the leadership, reception, supportiveness, facilities and organization of the placement (learning environment) as well as instruction, observation and feedback (training) on 7-point Likert scales with options for free-text comments on the strengths and weaknesses of the given placement on any of the items. We collected data online from medical students in their preclinical (1st, 2nd) as well as clinical years (4th, 5th) in a cross-sectional design in the academic years 2019-2020 and 2021-2022, by the end of their clinical placements. Our sample comprises data from 748 medical students. Exploratory and confirmatory factor analyses were performed, and higher-order factors were tested. Results: Although a bifactor model gave the best model fit (RMSEA = 0.024, CFI = 0.999, and TLI = 0.998), a high explained common variance (ECV = 0.82) and reliability coefficients (ωH = 0.87) for the general factor suggested that the Hungarian version of the MCPI could be considered unidimensional. Individual application of either of the subscales was not supported statistically due to their low reliabilities. Discussion: The Hungarian language version of MCPI proved to be a valid unidimensional instrument to measure the quality of undergraduate medical placements. The previously reported subscales were not robust enough, in the Hungarian context, to distinguish, statistically, the quality of learning environments from the training provided within those environments. This does not, however, preclude formative use of the subscales for quality improvement purposes.

Feedback data sources that inform physician self-assessment.

BACKGROUND: Self-assessment is a process of interpreting data about one's performance and comparing it to explicit or implicit standards. AIM: To examine the external data sources physicians used to monitor themselves. METHODS: Focus groups were conducted with physicians who participated in three practice improvement activities: a multisource feedback program; a program providing patient and chart audit data; and practice-based learning groups. We used grounded theory strategies to understand the external sources that stimulated self-assessment and how they worked. RESULTS: Data from seven focus groups (49 physicians) were analyzed. Physicians used information from structured programs, other educational activities, professional colleagues, and patients. Data were of varying quality, often from non-formal sources with implicit (not explicit) standards. Mandatory programs elicited variable responses, whereas data and activities the physicians selected themselves were more likely to be accepted. Physicians used the information to create a reference point against which they could weigh their performance using it variably depending on their personal interpretation of its accuracy, application, and utility. CONCLUSIONS: Physicians use and interpret data and standards of varying quality to inform self-assessment. Physicians may benefit from regular and routine feedback and guidance on how to seek out data for self-assessment.

Conducting a best evidence systematic review. Part 1: From idea to data coding. BEME Guide No. 13.

This paper outlines the essential aspects of conducting a systematic review of an educational topic beginning with the work needed once an initial idea for a review topic has been suggested through to the stage when all data from the selected primary studies has been coded. It draws extensively on the wisdom and experience of those who have undertaken systematic reviews of professional education, including Best Evidence Medical Education systematic reviews. Material from completed reviews is used to illustrate the practical application of the review processes discussed. The paper provides practical help to new review groups and contributes to the debate about ways of obtaining evidence (and what sort of evidence) to inform policy and practice in education.

Tensions in informed self-assessment: how the desire for feedback and reticence to collect and use it can conflict.

PURPOSE: Informed self-assessment describes the set of processes through which individuals use external and internal data to generate an appraisal of their own abilities. The purpose of this project was to explore the tensions described by learners and professionals when informing their self-assessments of clinical performance. METHOD: This 2008 qualitative study was guided by principles of grounded theory. Eight programs in five countries across undergraduate, postgraduate, and continuing medical education were purposively sampled. Seventeen focus groups were held (134 participants). Detailed analyses were conducted iteratively to understand themes and relationships. RESULTS: Participants experienced multiple tensions in informed self-assessment. Three categories of tensions emerged: within people (e.g., wanting feedback, yet fearing disconfirming feedback), between people (e.g., providing genuine feedback yet wanting to preserve relationships), and in the learning/practice environment (e.g., engaging in authentic self-assessment activities versus "playing the evaluation game"). Tensions were ongoing, contextual, and dynamic; they prevailed across participant groups, infusing all components of informed self-assessment. They also were present in varied contexts and at all levels of learners and practicing physicians. CONCLUSIONS: Multiple tensions, requiring ongoing negotiation and renegotiation, are inherent in informed self-assessment. Tensions are both intraindividual and interindividual and they are culturally situated, reflecting both professional and institutional influences. Social learning theories (social cognitive theory) and sociocultural theories of learning (situated learning and communities of practice) may inform our understanding and interpretation of the study findings. The findings suggest that educational interventions should be directed at individual, collective, and institutional cultural levels. Implications for practice are presented.

The processes and dimensions of informed self-assessment: a conceptual model.

PURPOSE: To determine how learners and physicians engaged in various structured interventions to inform self-assessment, how they perceived and used self-assessment in clinical learning and practice, and the components and processes comprising informed self-assessment and factors that influence these. METHOD: This was a qualitative study guided by principles of grounded theory. Using purposive sampling, eight programs were selected in Canada, the United States, the United Kingdom, the Netherlands, and Belgium, representing low, medium, and high degrees of structure/rigor in self-assessment activities. In 2008, 17 focus groups were conducted with 134 participants (53 undergraduate learners, 32 postgraduate learners, 49 physicians). Focus-group transcripts were analyzed interactively and iteratively by the research team to identify themes and compare and confirm findings. RESULTS: Informed self-assessment appeared as a flexible, dynamic process of accessing, interpreting, and responding to varied external and internal data. It was characterized by multiple tensions arising from complex interactions among competing internal and external data and multiple influencing conditions. The complex process was evident across the continuum of medical education and practice. A conceptual model of informed self-assessment emerged. CONCLUSIONS: Central challenges to informing self-assessment are the dynamic interrelationships and underlying tensions among the components comprising self-assessment. Realizing this increases understanding of why self-assessment accuracy seems frequently unreliable. Findings suggest the need for attention to the varied influencing conditions and inherent tensions to progress in understanding self-assessment, how it is informed, and its role in self-directed learning and professional self-regulation. Informed self-assessment is a multidimensional, complex construct requiring further research.