The identification of requirements for competency development during work-integrated learning in healthcare education.

BACKGROUND: Work-integrated learning (WIL) is widely accepted and necessary to attain the essential competencies healthcare students need at their future workplaces. Yet, competency-based education (CBE) remains complex. There often is a focus on daily practice during WIL. Hereby, continuous competency development is at stake. Moreover, the fact that competencies need to continuously develop is often neglected. OBJECTIVES: To ultimately contribute to the optimization of CBE in healthcare education, this study aimed at examining how competency development during WIL in healthcare education could be optimized, before and after graduation. METHODS: Fourteen semi-structured interviews with 16 experts in competency development and WIL were carried out. Eight healthcare disciplines were included namely associate degree nursing, audiology, family medicine, nursing (bachelor), occupational therapy, podiatry, pediatrics, and speech therapy. Moreover, two independent experts outside the healthcare domain were included to broaden the perspectives on competency development. A qualitative research approach was used based on an inductive thematic analysis using Nvivo12© where 'in vivo' codes were clustered as sub-themes and themes. RESULTS: The analysis revealed eight types of requirements for effective and continuous competency development, namely requirements in the context of (1) competency frameworks, (2) reflection and feedback, (3) assessment, (4) the continuity of competency development, (5) mentor involvement, (6) ePortfolios, (7) competency development visualizations, and (8) competency development after graduation. It was noteworthy that certain requirements were fulfilled in one educational program whereas they were absent in another. This emphasizes the large differences in how competence-based education is taking shape in different educational programs and internship contexts. Nevertheless, all educational programs seemed to recognize the importance of ongoing competency development. CONCLUSION: The results of this study indicate that identifying and meeting the requirements for effective and continuous competency development is essential to optimize competency development during practice in healthcare education.

Perceptions and experiences of primary healthcare providers toward interprofessional collaboration in chronic disease management in Hue, Vietnam.

In Vietnam, each primary care community health center (CHC) consists of a multi-professional team with six disciplines, including a physician, assistant physician, nurse, pharmacist, midwife, and Vietnamese traditional physician, who are able to meet the majority of patient's needs at the primary care level. How they collaborate, especially in chronic disease management (CDM), is still limited described in the literature. This study aims to gain insight into the perceptions and the experiences of primary health care providers (PHCPs) toward interprofessional collaboration (IPC) in CDM in CHCs in (Hue) Vietnam. A qualitative study of descriptive phenomenology was performed using two focus groups and 15 semi-structured interviews with PHCPs from six professions relevant to CDM in CHCs. The data were analyzed using NVivo 12.0 with a thematic analysis method by a multiprofessional research team. From the analysis, the data were classified into three main themes: "lack of collaborative practice," "knowledge," and "facilitators and barriers to IPC." This study provided evidence of the awareness that actual collaboration in daily care is fragmentarily organized and that PHCPs try to finish their tasks within their profession. PHCPs work multiprofessionally and lack shared decision-making in patient-centered care. There is a need to develop an interprofessional education program and training to address these deficiencies in the Vietnamese context to improve interprofessional collaboration in health care.

From bones to bytes: Do manipulable 3D models have added value in osteology education compared to static images?

BACKGROUND: Over the past few years, anatomy education has been revolutionized through digital media, resulting in innovative computer-based 3D models to supplement or even replace traditional learning materials. However, the added value of these models in terms of learning performance remains unclear. Multiple mechanisms may contribute to the inconclusive findings. This study focusses on the impact of active manipulation on learning performance and the influence that posttest design features may have on the outcome measurement. METHODS: Participants were randomly assigned to one of two research conditions: studying on the base of a computer-based manipulable pelvic bone model versus online static images of the same model. Pretests focused on students' baseline anatomy knowledge and spatial ability. Three knowledge posttests were administered: a test based on a physical pelvic bone model, and two computer-based tests based on static images and a manipulable model. Mental effort was measured with the Paas mental effort rating scale. RESULTS: In the static images-based posttest, significantly higher knowledge scores were attained by participants studying in the static images research condition (p = 0.043). No other significant knowledge-related differences could be observed. In the manipulable model-based posttest, spatial ability rather than the research condition seemed to have an influential role on the outcome scores (r = 0.18, p = 0.049). Mental effort scores reflected no difference between both research conditions. CONCLUSION: The research results are counter-intuitive, especially because no significant differences were found in the physical model-based posttest in students who studied with the manipulable model. Explaining the results builds on differences in anatomical models requiring less or more active manipulation to process spatial information. The pelvic bone manipulable model, and by extension osteology models, might be insufficiently complex to provide added value compared with static images. Moreover, the posttest modality should be chosen with care since spatial ability rather than anatomy knowledge may be measured.

When theory beats practice: the implementation of competency-based education at healthcare workplaces : Focus group interviews with students, mentors, and educators of six healthcare disciplines.

BACKGROUND: Work-integrated learning constitutes a large part of current healthcare education. During the last decades, a competency-based educational (CBE) approach has been introduced to reduce the theory-practice gap and to promote continuous competency development. Different frameworks and models have been developed to support CBE implementation in practice. Although CBE is now well-established, implementation at healthcare workplaces remains complex and controversial. This study aims to explore how students, mentors, and educators from different healthcare disciplines perceive the implementation of CBE at the workplace. The six-step model of Embo et al. (2015) was used as a base: (1) competency selection, (2) formulating learning goals, (3) self-monitoring performance, (4) self-assessing competency development, (5) summative assessment of individual competencies, and (6) summative assessment of global professional competence. METHODS: Three semi-structured focus group interviews were conducted with (1) five students, (2) five mentors, and (3) five educators. We recruited participants from six different educational programs: audiology, midwifery, nursing (associate degree and bachelor), occupational therapy, or speech therapy. We used thematic analysis combining an inductive and deductive approach. RESULTS: An overview of the predefined competencies was hard to find which complicated CBE implementation and resulted in a lack of consistency between the steps; e.g., the link between the selection of relevant competencies (step 1) and the formulation of learning goals based on these selected competencies (step 2) was absent. Furthermore, the analysis of the data helped identifying seven barriers for CBE implementation: (1) a gap between the educational program and the workplace, (2) a lacking overview of predefined competencies, (3) a major focus on technical competencies at the expense of generic competencies, (4) weak formulation of the learning goals, (5) obstacles related to reflection, (6) low feedback quality, and (7) perceived subjectivity of the assessment approach. CONCLUSION: The present barriers to CBE implementation lead to a fragmentation of current work-integrated learning. In this way, theory beats practice when it comes to CBE implementation as the theory of CBE is not effectively implemented. However, the identification of these barriers might help to find solutions to optimize CBE implementation. Future research seems critical to optimize CBE so that theory can meet practice and the opportunities of CBE optimize healthcare education.

Comparison of students' readiness from six health education programs for interprofessional learning in Vietnam: a cross-sectional study.

BACKGROUND: Interprofessional education (IPE) is expected to help prepare undergraduate health profession students to collaborate with other healthcare professionals in realising quality of care. Studies stress the necessity of students' readiness for interprofessional learning (IPL) in view of designing IPE programs. The present study aims to determine students' IPL-readiness and looks at related differences in students enrolled in different programs and at different phases in their educational program. METHODS: A cross-sectional survey study was set up among 1139 students from six health programs at HueUMP, using the Readiness for Interprofessional Learning Scale (RIPLS). Statistical analysis was performed using Kruskal-Wallis H and Mann-Whitney U tests. RESULTS: The overall mean RIPLS score was 68.89. RIPLS scores significantly differed between programs and between phases in the educational programs. Medical students presented a lower readiness level for IPL than students from other programs. In contrast to a significant increase in RIPLS scores of students in the clinical phase in Vietnamese traditional medicine, medicine, and pharmacy, a decrease in RIPLS scores was observed in students in the clinical phase in odonto-stomatology. CONCLUSIONS: The differences could be related to differences in educational programs and the study phases in a particular program. These results offer insights to direct the design and implementation of IPE in health education curricula and especially underscore the need to provide IPE throughout the curriculum.

Development of an integrated competency framework for postgraduate paediatric training: a Delphi study.

Competency-based education (CBE) has transformed medical training during the last decades. In Flanders (Belgium), multiple competency frameworks are being used concurrently guiding paediatric postgraduate CBE. This study aimed to merge these frameworks into an integrated competency framework for postgraduate paediatric training. In a first phase, these frameworks were scrutinized and merged into one using the Canadian Medical Education Directives for Specialists (CanMEDS) framework as a comprehensive basis. Thereafter, the resulting unified competency framework was validated using a Delphi study with three consecutive rounds. All competencies (n = 95) were scored as relevant in the first round, and twelve competencies were adjusted in the second round. After the third round, all competencies were validated for inclusion. Nevertheless, differences in the setting in which a paediatrician may work make it difficult to apply a general framework, as not all competencies are equally relevant, applicable, or suitable for evaluation in every clinical setting. These challenges call for a clear description of the competencies to guide curriculum planning, and to provide a fitting workplace context and learning opportunities.Conclusion: A competency framework for paediatric post-graduate training was developed by combining three existing frameworks, and was validated through a Delphi study. This competency framework can be used in setting the goals for workplace learning during paediatric training. What is Known: •Benefits of competency-based education and its underlying competency frameworks have been described in the literature. •A single and comprehensive competency framework can facilitate training, assessment, and certification. What is New: •Three existing frameworks were merged into one integrated framework for paediatric postgraduate education, which was then adjusted and approved by an expert panel. •Differences in the working environment might explain how relevant a competency is perceived.

An online Delphi study to investigate the completeness of the CanMEDS Roles and the relevance, formulation, and measurability of their key competencies within eight healthcare disciplines in Flanders.

BACKGROUND: Several competency frameworks are being developed to support competency-based education (CBE). In medical education, extensive literature exists about validated competency frameworks for example, the CanMEDS competency framework. In contrast, comparable literature is limited in nursing, midwifery, and allied health disciplines. Therefore, this study aims to investigate (1) the completeness of the CanMEDS Roles, and (2) the relevance, formulation, and measurability of the CanMEDS key competencies in nursing, midwifery, and allied health disciplines. If the competency framework is validated in different educational programs, opportunities to support CBE and interprofessional education/collaboration can be created. METHODS: A three-round online Delphi study was conducted with respectively 42, 37, and 35 experts rating the Roles (n = 7) and key competencies (n = 27). These experts came from non-university healthcare disciplines in Flanders (Belgium): audiology, dental hygiene, midwifery, nursing, occupational therapy, podiatry, and speech therapy. Experts answered with yes/no (Roles) or on a Likert-type scale (key competencies). Agreement percentages were analyzed quantitatively whereby consensus was attained when 70% or more of the experts scored positively. In round one, experts could also add remarks which were qualitatively analyzed using inductive content analysis. RESULTS: After round one, there was consensus about the completeness of all the Roles, the relevance of 25, the formulation of 24, and the measurability of eight key competencies. Afterwards, key competencies were clarified or modified based on experts' remarks by adding context-specific information and acknowledging the developmental aspect of key competencies. After round two, no additional key competencies were validated for the relevance criterion, two additional key competencies were validated for the formulation criterion, and 16 additional key competencies were validated for the measurability criterion. After adding enabling competencies in round three, consensus was reached about the measurability of one additional key competency resulting in the validation of the complete CanMEDS competency framework except for the measurability of two key competencies. CONCLUSIONS: The CanMEDS competency framework can be seen as a grounding for competency-based healthcare education. Future research could build on the findings and focus on validating the enabling competencies in nursing, midwifery, and allied health disciplines possibly improving the measurability of key competencies.

Care competencies in adolescents: Development of a new measure and relationships with well-being.

Worldwide, we observe an increasing focus on fostering well-being in adolescents. This is reflected in growing research in the field of care. However, empirical research is lacking that focuses on the relationship between components of care: receiving care, self-care and extending care. This lack of research is mirrored in the lack of valid measures for assessing care competencies in educational contexts. The present research, therefore, has four goals: 1) to create a valid and reliable scale assessing levels of care competencies based on the multifaceted and multidimensional concept of care; 2) to explore the relationship between socio-demographic characteristics and care competencies in adolescents; 3) to examine the psychological outcomes associated with different levels of care competencies in adolescents; 4) to explore the interaction of the three care dimensions in predicting adolescents' well-being. Based on data from 742 adolescents, a reliable and valid scale could be developed with six subscales: Receiving care, self-care and extending care, with each of them falling apart in care competencies and care failures. Gender, age and academic achievement were related to care competencies/failures. Findings support the link between adolescents' well-being and care competencies/failures. These results have implications for promoting adolescents' well-being through school-based care-cultivation programs.

Development of Surgical Competence in Veterinary Students Using a Flipped Classroom Approach.

Clinical skills laboratory (CSL) training was recently introduced in the renewed veterinary curriculum at Ghent University, using models and simulators for teaching practical skills. However, time in the CSL is restricted due to the large number of students combined with limited availability of personnel. Therefore, a flipped classroom (FC) model was introduced to maximize learning experiences. The goal of the present study was to evaluate the effect of flipped classroom CSL training on students' self-efficacy and practical surgical skills. Flipped classroom CSL training was implemented for the third-year pre-clinical students (n = 196) in the 6-year veterinary medicine program. Prior to CSL sessions, students studied online 'learning paths,' including text, pictures, videos of the skills, links to background information, a forum, and a compulsory pre-class quiz. A pre- and post-test were administered before and after flipped classroom CSL training. The tests consisted of a self-efficacy scale consisting of 20 items and an objective structured clinical examination (OSCE) test of surgical skills performance. Flipped classroom CSL training resulted in significantly higher self-efficacy (score/100, pre-test 55 ± 14 vs. post-test 83 ± 8, p< .001) and surgical skills performance (score/20, pre-test 5 ± 3 vs. post-test 17 ± 3, p< .001). In conclusion, this study demonstrated the feasibility and value of implementing a flipped classroom approach in combination with CSL training.

Decreasing Risky Behavior on Social Network Sites: The Impact of Parental Involvement in Secondary Education Interventions.

Teenagers face significant risks when using increasingly popular social network sites. Prevention and intervention efforts to raise awareness about these risks and to change risky behavior (so-called "e-safety" interventions) are essential for the wellbeing of these minors. However, several studies have revealed that while school interventions often affect awareness, they have only a limited impact on pupils' unsafe behavior. Utilizing the Theory of Planned Behavior and theories about parental involvement, we hypothesized that involving parents in an e-safety intervention would positively influence pupils' intentions and behavior. In a quasi-experimental study with pre- and post-test measures involving 207 pupils in secondary education, we compared the impact of an intervention without parental involvement with one that included active parental involvement by means of a homework task. We found that whereas parental involvement was not necessary to improve the intervention's impact on risk awareness, it did change intentions to engage in certain unsafe behavior, such as posting personal and sexual information on the profile page of a social network site, and in reducing existing problematic behavior. This beneficial impact was particularly evident for boys. These findings suggest that developing prevention campaigns with active parental involvement is well worth the effort. Researchers and developers should therefore focus on other efficient strategies to involve parents.

Anatomical knowledge enhancement through echocardiography and videos, with a spotlight on cognitive load, self-efficacy, and motivation.

In recent years, there has been a growing recognition of the importance of integrating ultrasound into undergraduate medical curricula. However, empirical evidence is lacking as to its effect on anatomy learning and related student cognition. Therefore, the present study compared the impact of echocardiography-based instruction with narrated videos on students' understanding of anatomical relationships, as well as the interaction with students' autonomous motivation, self-efficacy beliefs, mental load, and attitudes. Second-year medical students were given the opportunity to enroll in a supplementary booster course about cardiac anatomy. On the base of a randomized controlled trial with a cross-over design, we studied the effect of taking this course on spatial anatomical knowledge. After completing a pre-test (T0), students were allocated randomly to either the echocardiography-based teaching condition (cohort A) or the narrated anatomy video condition (cohort B). Next, participants were crossed over to the alternative intervention. Immediately after each phase in the intervention, students were asked to rate their mental load. Additionally, a spatial anatomical knowledge test, an autonomous motivation scale, and a self-efficacy scale were administered before (T0) and after the first intervention (T1) and at the end of the study (T2). Finally, each student completed a perception-based survey. The study design allowed a comparative evaluation of both interventions at T1, while the cross-over design facilitated the assessment of the most optimal sequencing in the interventions at T2. A total of 206 students participated (cohort A: n = 99, cohort B: n = 107). At T1, no significant differences in the knowledge test and the autonomous motivation scale were observed between cohorts A and B. However, cohort A showed higher self-efficacy beliefs compared to cohort B (p = 0.043). Moreover, cohort A reported higher levels of perceived mental load (p < 0.001). At T2, the results showed that neither sequence of interventions resulted in significant differences in anatomy scores, autonomous motivation, or self-efficacy. However, a significant difference in mental load was found again, with students in cohort B reporting a higher level of mental load (p < 0.001). Finally, based on the perception-based survey, students reported favorably on the echocardiography experience. In conclusion, the hands-on echocardiography sessions were highly appreciated by the medical students. After participating in the ultrasound sessions, they reported higher levels of self-efficacy beliefs compared to the video-based condition. However, despite embodied cognition principles, students in the echocardiography condition did not outperform students in the narrated anatomy video condition. The reported levels of mental load in the ultrasound condition could explain these findings.

A study of the impact of an interprofessional education module in Vietnam on students' readiness and competencies.

INTRODUCTION: The literature puts forward a range of challenges of interprofessional education (IPE) related to its planning, initiation, implementation, and especially to IPE assessment. The present study aims to map changes in students' readiness and interprofessional collaboration competence (IPCC) in implementing an innovative IPE module. Potential differences in impact related to the health education programs and IPCC scores resulting from self-, peer-, and tutor assessments will also be analysed. METHODS: A pre-post design was adopted. The student's readiness for interprofessional learning was assessed using the Readiness for Interprofessional Learning Scale, and the student's IPCC score was calculated based on self-, peer-, and tutor assessments with the interprofessional collaborator assessment rubric. RESULTS: Students' mean post-test readiness scores and mean post-test IPCC scores were significantly higher than the total and subscales/domain pre-test scores (p<0.01). No significant within-subject differences were observed in students' readiness total or subscale scores when comparing health educational programs. However, significant differences were observed in students' mean total IPCC scores between programs (p<0.01). Significant differences in students' average IPCC scores were found when comparing self-, peer- and tutor assessment scores in six domains (p<0.01). Also, significant correlations between peer and tutor assessment scores were observed (p<0.01). CONCLUSION: The IPE module, designed and implemented to focus on patient-centred practice within a primary care context, positively impacted students' readiness and IPCC development. These results offer insights to expand the implementation of the IPE module to all health educational programs.

A novel Medical Achievement Self-efficacy Scale (MASS): a valid and reliable tool.

In search for an instrument to measure overall curriculum impact, we developed a Medical Achievement Self-efficacy Scale (MASS) and presented it to medical students enrolled in the different years of the integrated Ghent curriculum. The research aim was to study the validity and reliability of this new scale. MASS items were constructed based on the end terms of the Ghent curriculum, as it is related to the general competency frameworks of CanMEDs and the Five-star Doctor. The scale includes at least two items for each CanMEDS competency domain. Items were examined by seven experts in view of content and face validity. This resulted in an MASS version, containing 18 items, to be rated on a five-point Likert scale. This version was piloted on 94 undergraduate medical students enrolled at the Catholic University of Leuven. The final version was presented to 1066 undergraduate medical students enrolled at Ghent University. Reliability of the MASS scale was high (α=0.89). As expected, self-efficacy scores increased significantly over the years (F=39.11, p<0.001). In view of determining predictive validity, regression analysis was carried out to predict students' academic achievement from self-efficacy scores. As expected, MASS scores significantly predicted Maastricht Progress Test scores (F=108.18, p<0.001).

The Impact of School Strategies and the Home Environment on Home Learning Experiences During the COVID-19 Pandemic in Children With and Without Developmental Disorders.

Using the Opportunity-Propensity Model (Byrnes in Dev Rev 56:100911, 2020; Byrnes & Miller in Contemp Educ Psychol 32(4);599-629, 2007), the current study investigated which factors helped predicting children's home learning experiences during the COVID-19 pandemic, thereby examining differences between children with (DD; n = 779) and without (TD; n = 1443) developmental disorders. MANCOVA results indicated more negative experiences for DD children and their parents. SEM-results revealed the alignment between different teachers and autonomous motivation in children as the most important predictors for the outcome variables. Less predictors were significant for DD as compared to TD children which suggests other factors are at play in the DD group. Limitations, strengths and suggestions for future research are being discussed, together with some implications for classroom practices and remote learning approaches.

Ultrasound versus videos: A comparative study on the effectiveness of musculoskeletal anatomy education and student cognition.

Ultrasound imaging is a dynamic imaging technique that uses high-frequency sound waves to capture live images of the structures beneath the skin. In addition to its growing use in diagnosis and interventions, ultrasound imaging has the potential to reinforce concepts in the undergraduate medical curriculum. However, research assessing the impact of ultrasound on anatomy learning and student cognition is scarce. The purpose of this study was to compare the impact of ultrasound-based instruction versus narrated videos on students' understanding of anatomical relationships, as well as the role of intrinsic motivation, self-efficacy beliefs, and students' attitudes in this process. A booster course on anterior leg and wrist anatomy was offered to second-year medical students. A randomized controlled trial with a cross-over design allocated students to either an ultrasound-based teaching condition (cohort A) or a narrated anatomy video condition (cohort B). Next, participants were crossed to the alternative intervention. At the start of the study (T0), baseline anatomy knowledge, intrinsic motivation, self-efficacy beliefs, and spatial ability were measured. After the first intervention (T1) and at the end of the study (T2), both cohorts were administered an anatomy test, an intrinsic motivation scale, and a self-efficacy scale. In addition, each student was asked to fill out a perception survey after the ultrasound intervention. Finally, building on the cross-over design, the most optimal sequence of interventions was examined. A total of 181 students participated (cohort A: n = 82, cohort B: n = 99). Both cohorts performed comparably on the baseline anatomy knowledge test, spatial ability test, intrinsic motivation, and self-efficacy scale. At T1, cohort B outperformed cohort A on the anatomy test (p = 0.019), although only a small effect size could be detected (Cohen's d = 0.34). Intrinsic motivation and self-efficacy of both cohorts were similar at T1. At T2, the anatomy test, intrinsic motivation, and self-efficacy scale did not reflect an effect after studying either sequence of the interventions. Students reported favorably about the ultrasound experience, but also mentioned a steep learning curve. Medical students found the hands-on ultrasound sessions to be valuable, increasing their interest in musculoskeletal anatomy and ultrasound imaging. However, the addition of ultrasound did not result in superior spatial anatomy understanding compared to watching anatomy videos. In addition, ultrasound teaching did not have a major effect on student cognition. Ultrasound-based teaching of musculoskeletal anatomy is regarded as difficult to learn, and therefore it is hypothesized that too high levels of cognitive load might explain the presented results.

A Qualitative Textual Analysis of Feedback Comments in ePortfolios: Quality and Alignment with the CanMEDS Roles.

Introduction: Competency-based education requires high-quality feedback to guide students' acquisition of competencies. Sound assessment and feedback systems, such as ePortfolios, are needed to facilitate seeking and giving feedback during clinical placements. However, it is unclear whether the written feedback comments in ePortfolios are of high quality and aligned with the current competency focus. Therefore, this study investigates the quality of written feedback comments in ePortfolios of healthcare students, as well as how these feedback comments align with the CanMEDS roles. Methods: A qualitative textual analysis was conducted. 2,349 written feedback comments retrieved from the ePortfolios of 149 healthcare students (specialist medicine, general practice, occupational therapy, speech therapy and midwifery) were analysed retrospectively using deductive content analysis. Two structured categorisation matrices, one based on four literature-derived feedback quality criteria (performance, judgment, elaboration and improvement) and another one on the seven CanMEDS roles (Medical Expert, Communicator, Collaborator, Leader, Health Advocate, Scholar and Professional), guided the analysis. Results: The minority of the feedback comments (n = 352; 14.9%) could be considered of high quality because they met all four quality criteria. Most feedback comments were of moderate quality and met only two to three quality criteria. Regarding the CanMEDS roles, the Medical Expert role was most frequently represented in the feedback comments, as opposed to the roles Leader and Health Advocate. Discussion: The results highlighted that providing high-quality feedback is challenging. To respond to these challenges, it is recommended to set up individual and continuous feedback training.

Automating the Identification of Feedback Quality Criteria and the CanMEDS Roles in Written Feedback Comments Using Natural Language Processing.

Introduction: Manually analysing the quality of large amounts of written feedback comments is time-consuming and demands extensive resources and human effort. Therefore, this study aimed to explore whether a state-of-the-art large language model (LLM) could be fine-tuned to identify the presence of four literature-derived feedback quality criteria (performance, judgment, elaboration and improvement) and the seven CanMEDS roles (Medical Expert, Communicator, Collaborator, Leader, Health Advocate, Scholar and Professional) in written feedback comments. Methods: A set of 2,349 labelled feedback comments of five healthcare educational programs in Flanders (Belgium) (specialistic medicine, general practice, midwifery, speech therapy and occupational therapy) was split into 12,452 sentences to create two datasets for the machine learning analysis. The Dutch BERT models BERTje and RobBERT were used to train four multiclass-multilabel classification models: two to identify the four feedback quality criteria and two to identify the seven CanMEDS roles. Results: The classification models trained with BERTje and RobBERT to predict the presence of the four feedback quality criteria attained macro average F1-scores of 0.73 and 0.76, respectively. The F1-score of the model predicting the presence of the CanMEDS roles trained with BERTje was 0.71 and 0.72 with RobBERT. Discussion: The results showed that a state-of-the-art LLM is able to identify the presence of the four feedback quality criteria and the CanMEDS roles in written feedback comments. This implies that the quality analysis of written feedback comments can be automated using an LLM, leading to savings of time and resources.

Impact of three alternative consultation training formats on self-efficacy and consultation skills of medical students.

BACKGROUND: Conducting a consultation is a core competence of medical professionals. Consultation training of medical students centers on clinical, communication, reasoning and reflection skills. The training incorporates practice with a standardized simulated patient and supervising physician, to prepare for real patient encounters. To meet the request for more training, while dealing with an increasing student population and limited staff availability, alternative formats of consultation training were developed and evaluated. AIM: To investigate the impact of three consultation training formats on students' self-efficacy beliefs and their consultation skills acquisition. The three formats comprised (1) traditional training with supervising physician, (2) autonomous training with feedback from simulated patients and peers, without direct supervision and (3) online training based on video fragments and answering guiding questions. METHODS: A quasi-experimental pre/posttest study was set up, with random assignment of students to a training condition. The differential impact was tested on two dependent measures: self-efficacy and consultation performance. Self-efficacy was tested with a nine-item scale and the cognitive component of consultation performance was tested on the base of responses to a standardized video case. RESULTS: The autonomous training has a significant positive effect on students' self-efficacy (p=0.016). The traditional training and the online training did only positively influence the cognitive component of the consultation competence (p<0.001 and p=0.003). CONCLUSIONS: Each consultation training contributes to the learning process in a different way. In order to achieve optimum learning effects, medical educators should be aware of the particular impact of specific trainings on the cognitive and motivational side of skills and pursue a balanced mixture of instructional formats.

Assessing basic life support skills without an instructor: is it possible?

BACKGROUND: Current methods to assess Basic Life Support skills (BLS; chest compressions and ventilations) require the presence of an instructor. This is time-consuming and comports instructor bias. Since BLS skills testing is a routine activity, it is potentially suitable for automation. We developed a fully automated BLS testing station without instructor by using innovative software linked to a training manikin. The goal of our study was to investigate the feasibility of adequate testing (effectiveness) within the shortest period of time (efficiency). METHODS: As part of a randomised controlled trial investigating different compression depth training strategies, 184 medicine students received an individual appointment for a retention test six months after training. An interactive FlashTM (Adobe Systems Inc., USA) user interface was developed, to guide the students through the testing procedure after login, while Skills StationTM software (Laerdal Medical, Norway) automatically recorded compressions and ventilations and their duration ("time on task"). In a subgroup of 29 students the room entrance and exit time was registered to assess efficiency. To obtain a qualitative insight of the effectiveness, student's perceptions about the instructional organisation and about the usability of the fully automated testing station were surveyed. RESULTS: During testing there was incomplete data registration in two students and one student performed compressions only. The average time on task for the remaining 181 students was three minutes (SD 0.5). In the subgroup, the average overall time spent in the testing station was 7.5 minutes (SD 1.4). Mean scores were 5.3/6 (SD 0.5, range 4.0-6.0) for instructional organisation and 5.0/6 (SD 0.61, range 3.1-6.0) for usability. Students highly appreciated the automated testing procedure. CONCLUSIONS: Our automated testing station was an effective and efficient method to assess BLS skills in medicine students. Instructional organisation and usability were judged to be very good. This method enables future formative assessment and certification procedures to be carried out without instructor involvement. TRIAL REGISTRATION: B67020097543.