Validity evidence and psychometric evaluation of a socially accountable health index for health professions schools.

There is an expectation that health professions schools respond to priority societal health needs. This expectation is largely based on the underlying assumption that schools are aware of the priority needs in their communities. This paper demonstrates how open-access, pan-national health data can be used to create a reliable health index to assist schools in identifying societal needs and advance social accountability in health professions education. Using open-access data, a psychometric evaluation was conducted to examine the reliability and validity of the Canadian Health Indicators Framework (CHIF) conceptual model. A non-linear confirmatory factor analysis (CFA) on 67 health indicators, at the health-region level (n = 97) was used to assess the model fit of the hypothesized 10-factor model. Reliability analysis using McDonald's Omega were conducted, followed by Pearson's correlation coefficients. Findings from the non-linear CFA rejected the original conceptual model structure of the CHIF. Exploratory post hoc analyses were conducted using modification indices and parameter constraints to improve model fit. A final 5-factor multidimensional model demonstrated superior fit, reducing the number of indicators from 67 to 32. The 5-factors included: Health Conditions (8-indicators); Health Functions (6-indicators); Deaths (5-indicators); Non-Medical Health Determinants (7-indicators); and Community & Health System Characteristics (6-indicators). All factor loadings were statistically significant (p < 0.001) and demonstrated excellent internal consistency ( ω >0.95). Many schools struggle to identify and measure socially accountable outcomes. The process highlighted in this paper and the indices developed serve as starting points to allow schools to leverage open-access data as an initial step in identifying societal needs.

Data sharing and big data in health professions education: Ottawa consensus statement and recommendations for scholarship.

Changes in digital technology, increasing volume of data collection, and advances in methods have the potential to unleash the value of big data generated through the education of health professionals. Coupled with this potential are legitimate concerns about how data can be used or misused in ways that limit autonomy, equity, or harm stakeholders. This consensus statement is intended to address these issues by foregrounding the ethical imperatives for engaging with big data as well as the potential risks and challenges. Recognizing the wide and ever evolving scope of big data scholarship, we focus on foundational issues for framing and engaging in research. We ground our recommendations in the context of big data created through data sharing across and within the stages of the continuum of the education and training of health professionals. Ultimately, the goal of this statement is to support a culture of trust and quality for big data research to deliver on its promises for health professions education (HPE) and the health of society. Based on expert consensus and review of the literature, we report 19 recommendations in (1) framing scholarship and research through research, (2) considering unique ethical practices, (3) governance of data sharing collaborations that engage stakeholders, (4) data sharing processes best practices, (5) the importance of knowledge translation, and (6) advancing the quality of scholarship through multidisciplinary collaboration. The recommendations were modified and refined based on feedback from the 2022 Ottawa Conference attendees and subsequent public engagement. Adoption of these recommendations can help HPE scholars share data ethically and engage in high impact big data scholarship, which in turn can help the field meet the ultimate goal: high-quality education that leads to high-quality healthcare.

Global Perceptions on Social Accountability and Outcomes: A Survey of Medical Schools.

Phenomenon: Social accountability has become a universal component in medical education. However, medical schools have little guidance for operationalizing and applying this concept in practice. This study explored institutional practices and administrative perceptions of social accountability in medical education. Approach: An online survey was distributed to a purposeful sample of English-speaking undergraduate medical school deans and program directors/leads from 245 institutions in 14 countries. The survey comprised of 38-items related to program mission statements, admission processes, curricular content, and educational outcomes. Survey items were developed using previous literature and categorized using a context-input-process-products (CIPP) evaluation model. Exploratory Factor Analysis (EFA) was used to assess the inter-relationship among survey items. Reliability and internal consistency of items were evaluated using McDonald's Omega. Findings: Results from 81 medical schools in 14 countries collected between February and June 2020 are presented. Institutional commonalities of social accountability were observed. However, our findings suggest programs focus predominately on educational inputs and processes, and not necessarily on outcomes. Findings from our EFA demonstrated excellent internal consistency and reliability. Four-factors were extracted: (1) selection and recruitment; (2) institutional mandates; (3) institutional activities; and (4) community awareness, accounting for 71% of the variance. McDonald's Omega reliability estimates for subscales ranged from 0.80-0.87. Insights: This study identified common practices of social accountability. While many medical schools expressed an institutional commitment to social accountability, their effects on the community remain unknown and not evaluated. Overall, this paper offers programs and educators a psychometrically supported tool to aid in the operationalization and reliability of evaluating social accountability.

What is Transferred and How Much is Retained? A Simulation Study of Complex Surgical Skills.

INTRODUCTION: Studies indicate that learning surgical skills on low-fidelity models is equally beneficial to learning on high-fidelity models in terms of skills retention and transfer. However, it is unclear how low-fidelity simulation training impacts retention and transfer in novice learners, particularly on complex surgical tasks that incorporate multiple challenging skills. This study explores the capacity of complete novices to learn and transfer complex surgical skills from a low-fidelity model to a high-fidelity simulation after a delay. METHODS: Task-naïve medical and nonmedical undergraduate students (n = 62) participated in a three-phase prospective double-arm randomized (2:1) experimental study. Participants completed two skills training sessions (end-to-side anastomosis) on a low-fidelity bench model. After a 4-week delay, participants completed the task again either using the low-fidelity model or a high-fidelity model (cadaver) and were assessed using a validated checklist. RESULTS: There was a significant time × fidelity group interaction (P = 0.004). Simple effects analysis indicated the high-fidelity group (Mdiff = 4.18, P < 0.001) performed significantly worse (P = 0.003) in phase 3 relative to phase 2 compared to the low-fidelity group (Mdiff = 0.75, P = 0.39). Post hoc logistic regression analysis indicated that radial suturing technique and economy of motion skills were less likely to be completed correctly for those in the high-fidelity group. CONCLUSIONS: These findings suggest that for novice populations, relying on low-fidelity simulation training as a source of teaching complex skills may not provide a reliable transfer to high-fidelity models and in turn clinical settings.

"Doing it Right" Overnight: a Multi-perspective Qualitative Study Exploring Senior Medical Resident Overnight Call.

BACKGROUND: Competency-based medical education (CBME) requires the development of workplace-based assessment tools that are grounded in authentic clinical work. Developing such tools, however, requires a deep understanding of the underlying facets of the competencies being assessed. Gaining this understanding remains challenging in contexts where performance is not readily visible to supervisors such as the senior medical resident (SMR) on-call role in internal medicine. OBJECTIVE: This study draws on the perspectives of healthcare professionals with whom the SMR interacts with overnight to generate insights into the different components of on-call SMR practice and the range of ways SMRs effectively and less effectively enact these. APPROACH: We used a constructivist grounded theory (CGT) approach to examine variation in how on-call SMRs carry out their role overnight. PARTICIPANTS: Six medical students, five junior residents, five internal medicine attending physicians, five emergency physicians, and three emergency nurses conducted observations of their on-call interactions with SMRs. Participants were then interviewed and asked to elaborate on their observations as well as provide comparative reflections on the practices of past SMRs they worked with. KEY RESULTS: Strong collaboration and organizational skills were identified as critical components to effectively being the on-call SMR. Perceived weaker SMRs, while potentially also having issues with clinical skills, stood out more when they could not effectively manage the realities of collaboration in a busy workplace. CONCLUSION: What consistently differentiated a perceived effective SMR from a less effective SMR was someone who was equipped to manage the realities of interprofessional collaboration in a busy workplace. Our study invites medical educators to consider what residents, particularly those in more complex roles, need to receive feedback on to support their development as physicians. It is our intention that the findings be used to inform the ways programs approach teaching, assessment, and the provision of feedback.

Journey into uncertainty: Medical students' experiences and perceptions of failure.

CONTEXT: Having succeeded in being selected for medical school, medical students are not always familiar with failure and yet they are expected to graduate prepared to effectively function in the failure-burdened arena of clinical medicine. Lacking in the developing literature on learners and failure is an exploration of how this transformation is accomplished. The purpose of this study was to examine how medical students perceive and experience failure during their medical school training. METHOD: We used a qualitative description methodology to probe the failure experiences of medical students attending a Canadian medical school. Participants were provided with the broad definition of failure used in this research: 'deviation from expected and desired results.'In total, 12 students were sampled, three from each of the 4 years of study, and participated in individual, semi-structured interviews that were analysed using thematic analysis to identify and describe core themes. RESULTS: At the start of medical school, students admitted limited experience with failure; their early descriptions were self-centred and binary. Personal stories recounted by preceptors encouraged students and helped them understand that physicians are human and that failure is inevitable. Students felt relatively protected from failures that could impact patients. Both witnessing and participating in a failure event were distressing and sometimes at odds with their expectations. Students expressed a desire to talk about the experience. CONCLUSIONS: Medical students described examples of experiencing failure during medical school that transported them from the more certain black and white beginnings of their classroom into the uncertain shades of grey of clinical medicine. What the participants heard, saw and experienced suggests opportunities for classroom teachers to better prepare pre-clinical students for the inevitability of failure in clinical medicine and opportunities for clinical teachers to engage in open, inclusive conversations surrounding failures that occur on their watch.

The erosion of ambiguity tolerance and sustainment of perfectionism in undergraduate medical training: results from multiple samplings of a single cohort.

BACKGROUND: Medicine is a field that is simultaneously factual and ambiguous. Medical students have their first exposure to full time clinical practice during clerkship. While studies have examined medical trainees' tolerance of ambiguity (TOA), the extent to which TOA is affected by clinical experiences and its association with perfectionism is unknown. The aim of this study was to evaluate the effect of clerkship experience on TOA and perfectionism in medical students. METHODS: This was a multiple sampling, single cohort study of students in their first year of clinical clerkship which is comprised of 6 core rotations. Consenting students completed an online anonymous survey assessing their tolerance of ambiguity (TOA) and perfectionism in their first (pre) and last (post) 12 weeks of their clinical clerkship year. Tolerance of Ambiguity in Medical Students and Doctors (TAMSAD) and The Big Three perfectionism scale-short form (BTPS-SF) were used to assess TOA and perfectionism respectively. Pre-Post mean comparisons of TOA and perfectionism were assessed via t-tests. RESULTS: From a cohort of 174 clinical clerkship students, 51 students responded to pre-survey, 62 responded to post-survey. Clerkship was associated with a significant decrease in TOA (p < 0.00) with mean pre-TOA scores of 59.57 and post TOA of 43.8. Perfectionism scores were not significantly different over time (p > 0.05). There was a moderate inverse correlation between TOA and perfectionism before clerkship (r = 0.32) that increased slightly after clerkship (r = 0.39). Those preferring primary care specialties had significantly lower rigid and total perfectionism scores in pre-clerkship than those choosing other specialties, but this difference was not found post-clerkship. CONCLUSION: Exposure to clerkship decreased TOA while perfectionism remained stable in medical students. These results were not expected as exposure has been previously shown to increase TOA. The frequency of rotation changes maintaining a cycle of anxiety may be an underlying factor accounting for these results. Overall these results require further investigation to better characterize the role of clinical exposure on TOA.

Some assembly required: tracing the interpretative work of Clinical Competency Committees.

OBJECTIVES: This qualitative study describes the social processes of evidence interpretation employed by Clinical Competency Committees (CCCs), explicating how they interpret, grapple with and weigh assessment data. METHODS: Over 8 months, two researchers observed 10 CCC meetings across four postgraduate programmes at a Canadian medical school, spanning over 25 hours and 100 individual decisions. After each CCC meeting, a semi-structured interview was conducted with one member. Following constructivist grounded theory methodology, data collection and inductive analysis were conducted iteratively. RESULTS: Members of the CCCs held an assumption that they would be presented with high-quality assessment data that would enable them to make systematic and transparent decisions. This assumption was frequently challenged by the discovery of what we have termed 'problematic evidence' (evidence that CCC members struggled to meaningful interpret) within the catalogue of learner data. When CCCs were confronted with 'problematic evidence', they engaged in lengthy, effortful discussions aided by contextual data in order to make meaning of the evidence in question. This process of effortful discussion enabled CCCs to arrive at progression decisions that were informed by, rather than ignored, problematic evidence. CONCLUSIONS: Small groups involved in the review of trainee assessment data should be prepared to encounter evidence that is uncertain, absent, incomplete, or otherwise difficult to interpret, and should openly discuss strategies for addressing these challenges. The answer to the problem of effortful processes of data interpretation and problematic evidence is not as simple as generating more data with strong psychometric properties. Rather, it involves grappling with the discrepancies between our interpretive frameworks and the inescapably subjective nature of assessment data and judgement.

What trainees grapple with: a study of threshold concepts on the medicine ward.

OBJECTIVES: Socialisation theories of professional identity formation (PIF) consider clinical rotations to be critically intense transformative experiences. However, few studies have explored what trainees grapple with during these transformative experiences or their influence on performance. Applying a threshold concepts (TCs) lens, this study investigates and documents 'troublesome' and 'transformative' concepts that junior trainees may encounter during a clinical rotation. Insights gained are essential for supporting trainee development. METHODS: Constructivist grounded theory was used to guide the collection and analysis of data for this two-phase study. Phase 1 involved direct observation and field interviews with 17 junior trainees over two observation periods and phase 2 involved in-depth interviews with 13 attending physicians. The theory of TCs was used as a sensitising concept. RESULTS: In total, nine TCs were identified and thematically grouped under the headings: Developing as a Professional, Providing Patient Care and Working Collectively. Across the interviewed attending physicians, there appeared to be a shared understanding of TCs strong trainees had crossed and weaker trainees struggled with. Observational and field interview data suggested that individual trainee actions were strongly influenced by the identified TC and whether or not the trainee appeared to have crossed any given threshold. Moreover, individual clinical practices could be influenced by more than one TC. Trainees were also observed to vary in the thresholds they had already crossed or struggled with. CONCLUSIONS: The identified TCs offer important insights into the relationship between trainee actions and how they conceptualise practice. At their heart, many appeared to represent ideals of practice that trainees should incorporate into their developing identities as they explore what it means to be a physician. Future research should explore how to incorporate TCs into assessment and the support of trainee development.

Considering the interdependence of clinical performance: implications for assessment and entrustment.

INTRODUCTION: Our ability to assess independent trainee performance is a key element of competency-based medical education (CBME). In workplace-based clinical settings, however, the performance of a trainee can be deeply entangled with others on the team. This presents a fundamental challenge, given the need to assess and entrust trainees based on the evolution of their independent clinical performance. The purpose of this study, therefore, was to understand what faculty members and senior postgraduate trainees believe constitutes independent performance in a variety of clinical specialty contexts. METHODS: Following constructivist grounded theory, and using both purposive and theoretical sampling, we conducted individual interviews with 11 clinical teaching faculty members and 10 senior trainees (postgraduate year 4/5) across 12 postgraduate specialties. Constant comparative inductive analysis was conducted. Return of findings was also carried out using one-to-one sessions with key informants and public presentations. RESULTS: Although some independent performances were described, participants spoke mostly about the exceptions to and disclaimers about these, elaborating their sense of the interdependence of trainee performances. Our analysis of these interdependence patterns identified multiple configurations of coupling, with the dominant being coupling of trainee and supervisor performance. We consider how the concept of coupling could advance workplace-based assessment efforts by supporting models that account for the collective dimensions of clinical performance. CONCLUSION: These findings call into question the assumption of independent performance, and offer an important step toward measuring coupled performance. An understanding of coupling can help both to better distinguish independent and interdependent performances, and to consider revising workplace-based assessment approaches for CBME.

From good to excellent: Improving clinical departments' learning climate in residency training.

INTRODUCTION: The improvement of clinical departments' learning climate is central to achieving high-quality residency training and patient care. However, improving the learning climate can be challenging given its complexity as a multi-dimensional construct. Distinct representations of the dimensions might create different learning climate groups across departments and may require varying efforts to achieve improvement. Therefore, this study investigated: (1) whether distinct learning climate groups could be identified and (2) whether contextual factors could explain variation in departments' learning climate performance. METHODS: This study included departments that used the Dutch Residency Educational Climate Test (D-RECT) through a web-based system in 2014-2015. Latent profile analysis was used to identify learning climate groups and multilevel modeling to predict clinical departments' learning climate performance. RESULTS: The study included 1730 resident evaluations. Departments were classified into one of the four learning climate groups: substandard, adequate, good and excellent performers. The teaching status of the hospital, departments' average teaching performance and percentage of time spent on educational activities by faculty-predicted departments' learning climate performance. DISCUSSION: Clinical departments can be successfully classified into informative learning climate groups. Ideally, given informative climate grouping with potential for cross learning, the departments could embark on targeted performance improvement.

Peer assessment in the objective structured clinical examination: A scoping review.

BACKGROUND: The objective structured clinical examination (OSCE), originally designed with experts assessing trainees' competence, is more frequently employed with an element of peer assessment and feedback. Although peer assessment in higher education has been studied, its role in OSCEs has not reviewed. AIMS: The aim of this study is to conduct a scoping review and explore the role of peer assessment and feedback in the OSCE. METHODS: Electronic database and hand searching yielded 507 articles. Twenty-one full records were screened, of which 13 were included in the review. Two independent reviewers completed each step of the review. RESULTS: Peer-based OSCEs are used to assess students' accuracy in assessing OSCE performance and to promote learning. Peer examiners (PE) tend to award better global ratings and variable checklist ratings compared to faculty and provide high-quality feedback. Participating in these OSCEs is perceived as beneficial for learning. CONCLUSIONS: Peer assessment and feedback can be used to gauge PE reliability and promote learning. Teachers using these OSCEs must use methodology which fits their purpose. Competency-based education calls for diversification of assessment practices and asks how assessment impacts learning; the peer-based OSCE responds to these demands and will become an important practice in health professions education.

In the minds of OSCE examiners: uncovering hidden assumptions.

The Objective Structured Clinical Exam (OSCE) is a widely used method of assessment in medical education. Rater cognition has become an important area of inquiry in the medical education assessment literature generally, and in the OSCE literature specifically, because of concerns about potential compromises of validity. In this study, a novel approach to mixed methods that combined Ordinal Logistic Hierarchical Linear Modeling and cognitive interviews was used to gain insights about what examiners were thinking during an OSCE. This study is based on data from the 2010 to 2014 administrations of the Clinician Assessment for Practice Program OSCE for International Medical Graduates (IMGs) in Nova Scotia. An IMG is a physician trained outside of Canada who was a licensed practitioner in a different country. The quantitative data were examined alongside four follow-up cognitive interviews of examiners conducted after the 2014 administration. The quantitative results show that competencies of (1) Investigation and Management and (2) Counseling were highly predictive of the Overall Global score. These competencies were also described in the cognitive interviews as the most salient parts of OSCE. Examiners also found Communication Skills and Professional Behavior to be relevant but the quantitative results revealed these to be less predictive of the Overall Global score. The interviews also reveal that there is a tacit sequence by which IMGs are expected to proceed in an OSCE, starting with more basic competencies such as History Taking and building up to Investigation Management and Counseling. The combined results confirm that a hidden pattern exists with respect to how examiners rate candidates. This study has potential implications for research into rater cognition, and the design and scoring of practice-ready OSCEs.

Using Resident-Sensitive Quality Measures Derived From Electronic Health Record Data to Assess Residents' Performance in Pediatric Emergency Medicine.

PURPOSE: Traditional quality metrics do not adequately represent the clinical work done by residents and, thus, cannot be used to link residency training to health care quality. This study aimed to determine whether electronic health record (EHR) data can be used to meaningfully assess residents' clinical performance in pediatric emergency medicine using resident-sensitive quality measures (RSQMs). METHOD: EHR data for asthma and bronchiolitis RSQMs from Cincinnati Children's Hospital Medical Center, a quaternary children's hospital, between July 1, 2017, and June 30, 2019, were analyzed by ranking residents based on composite scores calculated using raw, unadjusted, and case-mix adjusted latent score models, with lower percentiles indicating a lower quality of care and performance. Reliability and associations between the scores produced by the 3 scoring models were compared. Resident and patient characteristics associated with performance in the highest and lowest tertiles and changes in residents' rank after case-mix adjustments were also identified. RESULTS: 274 residents and 1,891 individual encounters of bronchiolitis patients aged 0-1 as well as 270 residents and 1,752 individual encounters of asthmatic patients aged 2-21 were included in the analysis. The minimum reliability requirement to create a composite score was met for asthma data (α = 0.77), but not bronchiolitis (α = 0.17). The asthma composite scores showed high correlations ( r = 0.90-0.99) between raw, latent, and adjusted composite scores. After case-mix adjustments, residents' absolute percentile rank shifted on average 10 percentiles. Residents who dropped by 10 or more percentiles were likely to be more junior, saw fewer patients, cared for less acute and younger patients, or had patients with a longer emergency department stay. CONCLUSIONS: For some clinical areas, it is possible to use EHR data, adjusted for patient complexity, to meaningfully assess residents' clinical performance and identify opportunities for quality improvement.

Chinese university students' conceptions of feedback and the relationships with self-regulated learning, self-efficacy, and English language achievement.

In China, under the influence of examination-driven culture and teacher-centered ways of learning, students' self-regulated learning (SRL) capabilities, self-efficacy, and actual English proficiency are greatly hindered. Given this situation, the Chinese Ministry of Education has promulgated the use of formative assessment in the College English curriculum at the tertiary level since 2004. Feedback, as an integrated part of formative assessment, takes up the largest proportion of the Chinese College English classroom assessment and intends to facilitate SRL and learning. However, whether feedback could facilitate students' SRL and learning has not been fully investigated in this context in China. Therefore, this study first explored how students self-reported their conceptions of feedback, SRL, and self-efficacy, and second, the relationships among these constructs and their English language achievement in the College English course. A questionnaire was used to collect data on students' conceptions of feedback, SRL, self-efficacy, and self-perceived English language proficiency. Their English test scores as an indicator of English language achievement were also collected. A total of 538 participants from a university in Northern China participated in this study. Data were analyzed using descriptive statistics, exploratory factor analyses, Pearson correlation analyses, and multiple regression analyses. The results found that Chinese students from the College English course reported a high level of conceptions of teacher and peer feedback, SRL, and self-efficacy, yet a low level of Teacher/Peer Feedback Ignored. For the relationships among these variables, students' conceptions of feedback contributed to SRL and self-efficacy. Besides, self-efficacy was found to be the strongest predictor for self-perceived English language proficiency and standardized English test scores, both indicators for English language achievement. From the theoretical perspective, this study addressed the research gap in the literature by examining four constructs together, that is, students' conceptions of feedback, SRL, self-efficacy, and English language achievement within a university context in China. From the pedagogical angle, the results can also support teachers in their feedback practices to facilitate students' SRL, self-efficacy, and learning.

The Interactive Relationship Between Instructor Perceptions and Learner Personality on Surgical Skills Performance.

OBJECTIVE: The objective of this study was to examine the association between learner personality and capacity to be trained (i.e., performance improvement) on a surgical task, and how instructor perceptions of the learners' capacity to be trained interact with learner personality and performance during training and feedback. There is meaningful heterogeneity in the degree of learner surgical skills acquisition despite receiving the same amount of training. While learner personality may independently contribute to skill acquisition, the instructor-learner feedback process is also important to consider. To better understand this interpersonal relationship, it is necessary to also consider instructor factors (i.e., perceptions), and how this may contribute to learner variability in skills training. DESIGN: This exploratory study employed a prospective two-phase design. Medical and non-medical undergraduate students (N = 62) completed measures of personality and participated in two 20-minute training sessions with expert feedback 2 weeks apart, performing an end-to-side anastomosis on a low-fidelity model. Learner performance and instructors' perceptions of a learner's capacity to be trained were assessed. PARTICIPANTS: Sixty-two medical and non-medical undergraduate students. RESULTS: There was a significant interaction between learner Extraversion and instructor's perceptions of learner capacity to be trained. Higher learner Extraversion was only associated with an increase in performance improvement for those who were considered trainable (OR = 4.83, p = 0.017). Post hoc analysis revealed a significant difference in the amount of feedback provided to participants who were considered trainable (M = 9.45) versus not trainable (M = 16.48). CONCLUSIONS: This study highlights the importance of both individual learner factors and instructor perceptions on surgical skill acquisition.

Factors Associated with a Label of Failure to Cope in Older Medical Inpatients: a Case-Control Study.

BACKGROUND: The term failure to cope (FTC) is often used to dismissively describe hospitalized older adults. The purpose of this study was to identify the factors associated with receiving a label of FTC. METHODS: Age-matched, case-control study with electronic and paper chart review identifying patient characteristics and admission details. RESULTS: One hundred eighty-five patients 70 years of age or older admitted to a general medicine team over two years: 99 patients with the label of FTC and 86 controls. No patients labelled with FTC came from long-term care. Characteristics associated with a label of FTC included living alone (aOR 3.8, 95% CI 1.9-7.8), falls (aOR 3.8, 95% CI 1.9-7.8), rehospitalization (aOR 3.6, 95% CI 1.7-8.0), and living in an independent dwelling (aOR 2.4, 95% CI 1.0-5.5). A higher number of chronic medications was associated with a lower likelihood of being labelled with FTC (aOR 0.9, 95% CI 0.8-1.0). CONCLUSIONS: The results suggest that FTC is a label based predominantly on social factors and has no role in a medical assessment. The patient's home setting was the key factor in being labelled with FTC, most medical factors did not play a significant role, and a pervasive language of blame was present.

Feedback Credibility in Healthcare Education: a Systematic Review and Synthesis.

PURPOSE: The purpose of this study was to systematically review and synthesize factors that influence learners' perceptions of credibility when feedback is provided by an authority figure in a healthcare environment. METHODS: This study reviewed literature from medicine, psychology, and education using systematic review and qualitative synthesis methods. In a multi-step process, major electronic bibliographic databases were searched for relevant studies until October 2020. RESULTS: The search identified 9216 articles. A total of 134 abstracts underwent full-text review. Of these, 22 articles met inclusion criteria. The studies were heterogenous and the majority utilized a qualitative design with interviews and focus groups. A few studies employed mixed methodology (n = 2) and two studies used a quantitative design. Four main themes were identified: feedback characteristics, context of feedback, source credibility, and recipient characteristics. CONCLUSION: As programs implement major educational change initiatives to create more formative assessment practices, feedback will become even more crucial. The four main themes identified are important factors that contribute to the perception of feedback credibility. While the factors are described independently, they may be viewed as interrelated and the association between these factors and feedback may be driven more by learning culture than each characteristic. SUPPLEMENTARY INFORMATION: The online version contains supplementary material available at 10.1007/s40670-020-01167-w.

Unusual suspects: Real-time physiological evaluation of stressors during laparoscopic donor nephrectomy.

INTRODUCTION: The purpose of this study was to document the variability of faculty surgeon electrodermal activity (EDA) peaks during laparoscopic donor nephrectomy (LDN) to determine the effect of case difficulty and learner expertise on the stress response. METHODS: EDA for a single faculty surgeon was captured over 15 LDN cases using an Empatica E4 wristband. During each case, one of three transplant fellows (novice, intermediate, or expert level LDN expertise) participated. Difficulty was rated preoperatively as "low/moderate/high" by the faculty. EDA peaks were collected and analyzed; the frequency and magnitude of EDA peaks, case difficulty, and fellow expertise were compared using a two-way factorial ANOVA. RESULTS: The main effects of learner expertise (F[2, 308]=11.27, p<0.001) and difficulty rating (F[2, 414]=15.13, p<0.001) were significant. The interaction between difficulty and expertise on faculty EDA peaks was also significant (F[3, 391]=14.29, p<0.001). The novice fellow resulted in higher faculty EDA levels compared to intermediate and expert fellows on low-difficulty cases, but not moderate- or high-difficulty cases. CONCLUSIONS: This is the first report examining faculty surgeon EDA across cases of varying difficulty and varying learner expertise during a high-stakes operation. EDA levels were inversely proportional to the expertise of the learner and case difficulty, suggestive of a significant impact of learner autonomy on faculty stress response.

Working Towards Competence: A Novel Application of Borderline Regression to a Task-Specific Checklist for Technical Skills in Novices.

OBJECTIVE: Demonstrated competence through frequent assessment is an expected goal for progressive development in competency-based medical education curricula. The Objective Structured Assessment of Technical Skill (OSATS) is considered a valid method of formative assessment, but in few instances have standards been set for determining competence. The present study used borderline regression methods to examine standard setting of performance on a complex technical task with novices assessed using an OSATS checklist. METHODS: This was a single institution prospective single arm experimental design study. Participants were 58 non-medical undergraduate students with no previous surgical experience, who observed a computer-based training module on end-to-side vascular anastomosis. Subsequently, participants were provided two 20-minute training sessions, two weeks apart where they received expert feedback whilst performing the task on a low-fidelity model. After each training session, participants completed the task unaided. Sessions were recorded and assessed using an OSATS checklist retrospectively by experts. RESULTS: Paired t-test analyses indicate that for both the checklist total score (t(52) = 8.05, p < 0.001) and the global rating score (t(53) = 8.15, p < 0.001), individuals performed significantly better in Phase 2. Borderline regression analyses indicated that in Phase 1 (R2 = .60) and Phase 2 (R2 = .75), the OSATS checklist could adequately capture variation in performance in novices. Further, the checklist could reliably classify novices at three of the five global rating performance levels. Pass rates determined by regression equations improved from Phase 1 to Phase 2 on all global rating levels. CONCLUSIONS: With the increasing focus on competency-based medical education, it is imperative that training programs have the capacity to accurately assess outcomes and set minimum performance standards. Borderline regression methods can accurately differentiate novice learners of varying performance levels before and after training on a complex technical skill task using an OSATS checklist.