Implementation of a programmatic assessment model in radiation oncology medical physics training.

PURPOSE: In 2019, a formal review and update of the current training program for medical physics residents/registrars in Australasia was conducted. The purpose of this was to ensure the program met current local clinical and technological requirements, to improve standardization of training across Australia and New Zealand and generate a dynamic curriculum and programmatic assessment model. METHODS: A four-phase project was initiated, including a consultant desktop review of the current program and stakeholder consultation. Overarching program outcomes on which to base the training model were developed, with content experts used to update the scientific content. Finally, assessment specialists reviewed a range of assessment models to determine appropriate assessment methods for each learning outcome, creating a model of programmatic assessment. RESULTS: The first phase identified a need for increased standardized assessment incorporating programmatic assessment. Seven clear program outcome statements were generated and used to guide and underpin the new curriculum framework. The curriculum was expanded from the previous version to include emerging technologies, while removing previous duplication. Finally, a range of proposed assessments for learning outcomes in the curriculum were generated into the programmatic assessment model. These new assessment methods were structured to incorporate rubric scoring to provide meaningful feedback. CONCLUSIONS: An updated training program for Radiation Oncology Medial Physics registrars/residents was released in Australasia. Scientific content from a previous program was used as a foundation and revised for currency with the ability to accommodate a dynamic curriculum model. A programmatic model of assessment was created after comprehensive review and consultation. This new model of assessment provides more structured, ongoing assessment throughout the training period. It contains allowances for local bespoke assessment, and guidance for supervisors by the provision of marking templates and rubrics.

One-shot skill assessment in high-stakes domains with limited data via meta learning.

Deep Learning (DL) has achieved robust competency assessment in various high-stakes fields. However, the applicability of DL models is often hampered by their substantial data requirements and confinement to specific training domains. This prevents them from transitioning to new tasks where data is scarce. Therefore, domain adaptation emerges as a critical element for the practical implementation of DL in real-world scenarios. Herein, we introduce A-VBANet, a novel meta-learning model capable of delivering domain-agnostic skill assessment via one-shot learning. Our methodology has been tested by assessing surgical skills on five laparoscopic and robotic simulators and real-life laparoscopic cholecystectomy. Our model successfully adapted with accuracies up to 99.5 % in one-shot and 99.9 % in few-shot settings for simulated tasks and 89.7 % for laparoscopic cholecystectomy. This study marks the first instance of a domain-agnostic methodology for skill assessment in critical fields setting a precedent for the broad application of DL across diverse real-life domains with limited data.

Development of a validated assessment tool for medical students using simulated patients: an 8-year panel survey.

BACKGROUND: The use of simulated patients (SPs) to assess medical students' clinical performance is gaining prominence, underscored by patient safety perspective. However, few reports have investigated the validity of such assessment. Here, we examined the validity and reliability of an assessment tool that serves as a standardized tool for SPs to assess medical students' medical interview. METHODS: This longitudinal survey was conducted at Keio University School of Medicine in Japan from 2014 to 2021. To establish content validity, the simulated patient assessment tool (SPAT) was developed by several medical education specialists from 2008 to 2013. A cohort of 36 SPs assessed the performance of 831 medical students in clinical practice medical interview sessions from April 2014 to December 2021. The assessment's internal structure was analyzed using descriptive statistics (maximum, minimum, median, mean, and standard deviation) for the SPAT's 13 item total scores. Structural validity was examined with exploratory factor analysis, and internal consistency with Cronbach's alpha coefficients. The mean SPAT total scores across different SPs and scenarios were compared using one way analysis of variance (ANOVA). Convergent validity was determined by correlating SPAT with the post-clinical clerkship obstructive structured clinical examination (post-CC OSCE) total scores using Pearson's correlation coefficient. RESULTS: Of the 831 assessment sheets, 36 with missing values were excluded, leaving 795 for analysis. Thirty-five SPs, excluding one SP who quit in 2014, completed 795 assessments, for a response rate of 95.6%. Exploratory factor analysis revealed two factors, communication and physician performance. The overall Cronbach's alpha coefficient was 0.929. Significant differences in SPAT total scores were observed across SPs and scenarios via one-way ANOVA. A moderate correlation (r =.212, p <.05) was found between SPAT and post-CC OSCE total scores, indicating convergent validity. CONCLUSIONS: Evidence for the validity of SPAT was examined. These findings may be useful in the standardization of SP assessment of the scenario-based clinical performance of medical students.

Developing a competency assessment framework for medical laboratory technologists in primary healthcare settings in India.

Medical Laboratory Technologists play a significant role in delivering quality laboratory Services. The competency assessment of MLTs is a critical driver for enhancing primary healthcare performance. While several countries have developed competency frameworks for MLTs in primary care, such frameworks are lacking in the Indian context. This study aimed to create a competency assessment framework to assess the competencies of MLTs (Medical Laboratory Technologists) working in Indian public primary healthcare facilities. The research followed a five-step process, starting with a review of existing literature on MLTs' competencies in primary healthcare. Expert consultations were then conducted to establish a consensus on these competencies. Following this, assessment tools were developed based on the literature review and expert input. Another round of expert consultations was held to ensure agreement on the assessment tools. Finally, the developed tools were tested in a public primary healthcare facility. The literature review identified 86 competencies across 11 domains: safe work practices, data/ sample collection, specimen preparation equipment instruments and regiments, assessment and analysis, recording and reporting, infection control, quality management, critical thinking, communication and interaction, and professional practice. Expert consultations resulted in the consensus on ninety-five competencies in ten domains of MLTs in primary healthcare settings. Competencies for each domain were discussed and agreed upon. A competency assessment tool was finalized after unanimous agreement among experts. The competency assessment tool was later finalized after pre-testing on MLTs in a clinical laboratory part of a public primary health care facility. This study successfully developed a competency assessment framework for in-service MLTs in Indian public primary healthcare settings. The framework encompasses ninety-five competencies covering ten domains of MLT responsibilities. It provides a comprehensive tool for assessing MLT's competencies and identifying competency gaps. The framework can be used to capacitate MLTs, improve their performance in primary healthcare settings, and enhance the delivery of healthcare services in India. It bridges a critical gap in the existing literature and can aid as a valuable resource for policymakers, educators, and healthcare professionals involved in practicing medical laboratory Services in primary healthcare settings.

Mastering your fellowship: Part 2, 2024.

The 'Mastering your Fellowship' series provides examples of the question format encountered in the written and clinical examinations, Part A of the Fellowship of the College of Family Physicians of South Africa (FCFP [SA]) examination. The series is aimed at helping family medicine registrars prepare for this examination.

Mastering your fellowship: Part 3, 2024.

The series 'Mastering your Fellowship' provides examples of the question formats encountered in the written and clinical examinations, Part A of the Fellowship of the College of Family Physicians of South Africa (FCFP [SA]) examination. The series aims to help family medicine registrars (and supervisors) prepare for this examination.

Attitudes toward communication skills with learner needs assessment within radiology residency programs in China: a cross-sectional survey.

BACKGROUND: Communication skills (CS) represent a core competency in radiology residency training. However, no structured curriculum exists to train radiology residents in CS in China. The aim of this study was to evaluate the status and prevalence of doctor-patient communication training among radiology residents in nine Chinese accredited radiology residency training programs and to determine whether there is a perceived need for a formalized curriculum in this field. METHODS: We administered a cross-sectional online survey to radiology residents involved in CS training at nine standard residency training programs in China. The questionnaire developed for this study included CS training status, residents' demographics, attitudes toward CS training, communication needs, and barriers. Residents' attitudes toward CS training were measured with the Communication Skills Attitude Scale (CSAS) and its subscales, a positive attitude scale (PAS) and negative attitude scale (NAS). RESULTS: A total of 133 (48.36%) residents participated in the survey. The mean total scores on the two dimensions of the CSAS were 47.61 ± 9.35 in the PAS and 36.34 ± 7.75 in the NAS. Factors found to be significantly associated with the PAS included receiving previous training in CS, medical ethics, or humanities and the doctor's attire. We found that first-year residents and poor personal CS were the most influential factors on the NAS. Only 58.65% of participants reported having previously received CS training during medical school, and 72.93% of respondents reported failure in at least one difficult communication during their residency rotation. Most of those surveyed agreed that CS can be learned through courses and were interested in CS training. Some of the most common barriers to implementing formal CS training were a lack of time, no standardized curriculum, and a lack of materials and faculty expertise. CONCLUSIONS: Most residents had a very positive attitude toward CS training and would value further training, despite the limited formal CS training for radiology residents in China. Future efforts should be made to establish and promote a standard and targeted CS curriculum for Chinese radiology residents.

Evaluating Effectiveness of an Online LGBTQIA+ Health Course for Medical Students.

BACKGROUND AND OBJECTIVES: Despite the increasing number of sexual and gender minority (SGM) patients in the United States and designation by the National Institutes of Health as a population with health disparities, available tools are lacking to train medical students on appropriate care for this population. Therefore, we developed and implemented a novel, self-directed, 2-week online elective for undergraduate medical students. The objective of our study was to evaluate the effectiveness of this course in increasing medical students' competency and confidence in caring for SGM patients. METHODS: We developed the curriculum using Kern's six-step model for curriculum development. We created anonymous pre- and postcourse surveys using the standardized Lesbian, Gay, Bisexual, Transgender Development of Clinical Skills Survey (LGBT-DOCSS) questionnaire to assess cultural competence, as well as a 5-point Likert-scored survey to assess self-perceived confidence in the care of SGM patients. We tested the statistical significance in pre- and postsurvey scores via paired sample t tests in R (R Project for Statistical Computing). RESULTS: We found statistically significant increases in the LGBT-DOCSS categories of clinical preparedness (P<.001), basic knowledge (P<.001), overall competency (P<.001), and self-perceived confidence in caring for SGM patients (P<.001, N=33). CONCLUSIONS: The course represents an effective solution for increasing medical students' self-perceived competence and confidence in caring for SGM patients. The flexibility and ease of the online format may be appealing to both students and institutions, and ultimately can serve to increase access to crucial content that is largely absent from current undergraduate medical education. Future evaluation efforts will be required to determine whether the course impacts long-term behavioral changes and outcomes.

Objective structured clinical examination to teach competency in planetary health care and management - a prospective observational study.

BACKGROUND: Health professionals are increasingly called upon and willing to engage in planetary health care and management. However, so far, this topic is rarely covered in medical curricula. As the need for professional communication is particularly high in this subject area, this study aimed to evaluate whether the objective structured clinical examination (OSCE) could be used as an accompanying teaching tool. METHODS: During the winter semester 2022/2023, 20 third- and fifth-year medical students voluntarily participated in a self-directed online course, three workshops, and a formal eight-station OSCE on planetary health care and management. Each examinee was also charged alternatingly as a shadower with the role of providing feedback. Experienced examiners rated students' performance using a scoring system supported by tablet computers. Examiners and shadowers provided timely feedback on candidates` performance in the OSCE. Immediately after the OSCE, students were asked about their experience using a nine-point Likert-scale survey and a videotaped group interview. Quantitative analysis included the presentation of the proportional distribution of student responses to the survey and of box plots showing percentages of maximum scores for the OSCE performance. The student group interview was analyzed qualitatively. RESULTS: Depending on the sub-theme, 60% -100% of students rated the subject of planetary health as likely to be useful in their professional lives. Similar proportions (57%-100%) were in favour of integrating planetary health into required courses. Students perceived learning success from OSCE experience and feedback as higher compared to that from online courses and workshops. Even shadowers learned from observation and feedback discussions. Examiners assessed students' OSCE performance at a median of 80% (interquartile range: 83%-77%) of the maximum score. CONCLUSIONS: OSCE can be used as an accompanying teaching tool for advanced students on the topic of planetary health care and management. It supports learning outcomes, particularly in terms of communication skills to sensitise and empower dialogue partners, and to initiate adaptation steps at the level of individual patients and local communities.

Development of an integrated milestone assessment tool across multiple early-adopter programs for breaking bad news: a pilot project.

BACKGROUND: The transition of the Accreditation Council for Graduate Medical Education (ACGME) to milestone assessment creates opportunities for collaboration and shared assessments across graduate medical programs. Breaking bad news is an essential communication skill that is a common milestone across almost every medical specialty. The purpose of this study was to develop and pilot an integrated milestone assessment (IMA) tool for breaking bad news using ACGME milestone criteria and to compare the IMA tool with the existing SPIKES protocol. METHODS: The IMA tool was created using sub-anchors in professionalism and interpersonal communication skills that are applicable to every specialty and to the ability to break bad news. Two cases of breaking bad news, designed to be "easy" and "intermediate" in difficulty, were used to assess basic skills in breaking bad news in first-year medical residents from six residency specialties. Eight standardized patients were trained to portray the cases in sessions held in November 2013 and May 2014. Standardized patients completed an assessment checklist to evaluate each resident's performance in breaking bad news based on their use of the SPIKES protocol and IMA tool. Residents answered post-encounter questions about their training and comfort in breaking bad news. The association between SPIKES and IMA scores was investigated by simple linear regression models and Spearman rank correlations. RESULTS: There were 136 eligible medical residents: 108 (79.4%) participated in the first session and 97 (71.3%) participated in the second session, with 96 (70.6%) residents participating in both sessions. Overall, we were able to identify residents that performed at both extremes of the assessment criteria using the integrated milestone assessment (IMA) and the SPIKES protocol. Interestingly, residents rated themselves below "comfortable" on average. CONCLUSION: We developed an integrated milestone assessment (IMA) that was better than the SPIKES protocol at assessing the skill of breaking bad news. This collaborative assessment tool can be used as supplement tool in the era of milestone transformation. We aim assess our tool in other specialties and institutions, as well as assess other shared milestones across specialties.

Development and assessment of a loop ligation simulator for laparoscopic appendectomy.

OBJECTIVE: Loop ligation of the appendix is a challenging surgical skill and well suited to be trained in a simulator. We aimed to develop an affordable and easy-to-build simulator and test its training effect. DESIGN AND PARTICIPANTS: Different materials were tested, and the best training modality was identified by researching the literature. The developed simulator training was tested on 20 surgical novices. RESULTS: A video was produced including an instruction on how to build the simulator and a step-by-step tuition on how to ligate the appendix. The Peyton approach was utilized to guide learners. Training with the simulator leads to reliable skill acquisition. All participants improved significantly in completing the task successfully during the structured learning. CONCLUSION: We succeeded in developing a simulator for loop ligation of the appendix during laparoscopic appendectomy. Participants significantly improve in handling the loops. The transferability of the skill learned during simulation to the operating room will be subject of a follow-up study.

Effectiveness of Using a Self-Directed Learning Program to Teach Physical Examination and Health Assessment Skills: A Quasi-Experimental Study.

BACKGROUND: Conducting health assessments and physical examinations are essential skills for independent practice. Exploring how to teach these skills effectively is essential. PURPOSE: This study was designed to validate the effectiveness of a self-directed learning program in improving nursing student outcomes in terms of their learning health assessment and physical examination skills. METHOD: A quasi-experimental design was used, and second-year nursing students enrolled in a health assessment and physical examination course at a university of science and technology in central Taiwan were recruited using purposive sampling. Both the control and experimental groups were taught using a traditional teaching method. The experimental group additionally participated in a self-directed learning program. Structured questionnaires, including the Chinese version of the Self-Directed Learning Readiness Scale, Scales of Motivation and Learning Strategies, Interpersonal Skills Assessment Tool, the Course Satisfaction Evaluation Scale, and the Perception of Health Assessment and Physical Examination Competence Scale, were used to collect data at three time points. RESULTS: After controlling for the effects of pretest scores, after the intervention, the experimental group achieved significantly higher scores than the control group for self-directed learning (creative learning and love of learning), learning motivation (goal orientation, work value, expected success, and test anxiety), and cognitive strategies (total score, elaboration strategy, recitation strategy, and monitoring strategy). In addition, the health assessment exercise improved interpersonal and communication skills, and learning satisfaction was significantly higher in the experimental group than the control group. CONCLUSIONS/IMPLICATIONS FOR PRACTICE: A physical examination and health assessment course designed using the self-directed learning framework can effectively improve student competence in related skills to enhance their ability to assess patient health problems in clinical settings. This study presents an alternative approach to teaching health assessment and physical examination courses and validates the positive effect of this approach on student learning outcomes.

Improving Adolescent Psychosocial Assessment through Standardized Patient Simulation: An Interdisciplinary Quality Improvement Initiative.

Adolescent suicide and mental illness have increased at alarming rates. Healthcare professionals report a lack of skill and confidence in obtaining adolescent histories and managing confidential care due to limited training in residency. Nursing professional development practitioners face challenges of adequately preparing interdisciplinary healthcare providers to assess, identify, and intervene at all points of contact with adolescents. To increase the confidence in clinical communication skills and clinical competency, and to increase the number of social work referrals related to modifiable risk factors for adolescent patients, a Texas pediatric tertiary care center utilized standardized patient (SP) methodology to supplement traditional clinical experiences with communication-focused education based on the Home, Education, Eating, Activities, Drugs, Sexuality, Suicidality, and Safety (HEEADSSS) interviewing. This quality improvement (QI) pilot demonstrated the benefits of utilizing standardized patient methodology in communication-focused education based on the HEEADSSS interviewing. Following the SP simulations, confidence in clinical communication skills increased by 13%, clinical competency in performing comprehensive psychosocial interviews increased by 11%, use of HEEADSSS increased by 64%, and social work referrals increased by 89%. This interdisciplinary SP interviewing simulation pilot was beneficial in improving the 36 physician and nursing residents' ability to conduct psychosocial assessments for risk factors of suicidality among adolescents.

Assessment and application of non-technical skills in robotic-assisted surgery: a systematic review.

BACKGROUND: Undeniably, robotic-assisted surgery (RAS) has become very popular in recent decades, but it has introduced challenges to the workflow of the surgical team. Non-technical skills (NTS) have received less emphasis than technical skills in training and assessment. The systematic review aimed to update the evidence on the role of NTS in robotic surgery, specifically focusing on evaluating assessment tools and their utilisation in training and surgical education in robotic surgery. METHODS: A systematic literature search of PubMed, PsycINFO, MEDLINE, and EMBASE was conducted to identify primary articles on NTS in RAS. Messick's validity framework and the Modified Medical Education Research Study Quality Instrument were utilised to evaluate the quality of the validity evidence of the abstracted articles. RESULTS: Seventeen studies were eligible for the final analysis. Communication, environmental factors, anticipation and teamwork were key NTS for RAS. Team-related factors such as ambient noise and chatter, inconveniences due to repeated requests during the procedure and constraints due to poor design of the operating room may harm patient safety during RAS. Three novel rater-based scoring systems and one sensor-based method for assessing NTS in RAS were identified. Anticipation by the team to predict and execute the next move before an explicit verbal command improved the surgeon's situational awareness. CONCLUSION: This systematic review highlighted the paucity of reporting on non-technical skills in robotic surgery with only three bespoke objective assessment tools being identified. Communication, environmental factors, anticipation, and teamwork are the key non-technical skills reported in robotic surgery, and further research is required to investigate their benefits to improve patient safety during robotic surgery.

Developing and validating the Unhurried Conversations Assessment Tool (UCAT).

OBJECTIVE: Given the importance of unhurried conversations for providing careful and kind care, we sought to create, test, and validate the Unhurried Conversations Assessment Tool (UCAT) for assessing the unhurriedness of patient-clinician consultations. METHODS: In the first two phases, the unhurried conversation dimensions were identified and transformed into an assessment tool. In the third phase, two independent raters used UCAT to evaluate the unhurriedness of 100 randomly selected consultations from 184 videos recorded for a large research trial. UCAT's psychometric properties were evaluated using this data. RESULTS: UCAT demonstrates content validity based on the literature and expert review. EFA and reliability analyses confirm its construct validity and internal consistency. The seven formative dimensions account for 89.93% of the variance in unhurriedness, each displaying excellent internal consistency (α > 0.90). Inter-rater agreement for the overall assessment item was fair (ICC = 0.59), with individual dimension ICCs ranging from 0.26 (poor) to 0.95 (excellent). CONCLUSION: UCAT components comprehensively assess the unhurriedness of consultations. The tool exhibits content and construct validity and can be used reliably. PRACTICE IMPLICATIONS: UCAT's design and psychometric properties make it a practical and efficient tool. Clinicians can use it for self-evaluations and training to foster unhurried conversations.

Train the paediatric colonoscopy trainer course: Assessment of perceived value in improving colonoscopy teaching practice.

OBJECTIVES: Endoscopy teaching practice is variable, which inevitably affects the training provided. There is only one Train the Paediatric Colonoscopy Trainer (TPCT) course in the UK. Informal feedback has been positive, but its practical value has never been formally assessed. We aim to assess the practical value of the TPCT course and how attendees perceive their teaching practice compared to nonattendees. METHODS: A questionnaire based on the TPCT course learning aims and objectives was distributed to two groups of consultant paediatric gastroenterologists who teach colonoscopy in the UK; those who had attended the course (participants) and those who had not (controls). RESULTS: The 41 completed responses were received. Overall, responses indicated participants of the TPCT course rated their confidence and knowledge in teaching practices as higher than controls (4.27 vs. 3.56 p = < 0.001). There was a statistically significant difference in all areas: set (4.21 vs. 3.71 p = 0.011), dialogue (4.29 vs. 3.55 p = < 0.001) and closure (4.37 vs. 3.6 p = < 0.001) with those who attended the TPCT course giving higher ratings. There was evidence of increased understanding of key concepts such as using standardised language, conscious competence, dual task interference and performance enhancing feedback. CONCLUSION: Attending a TPCT course results in a higher perceived level of knowledge in fundamental teaching principles and confidence in colonoscopy teaching skills.