RESUMEN
Competency-based medical education (CBME) is a patient-centered and learner-focused approach to education where curricula are delivered in a manner tailored to the individuals' learning needs, and assessment focuses on ensuring trainees achieve requisite and clearly specified learning outcomes. Despite calls to focus assessment on what matters for patients. In this article, the authors explore one aspect of this next era: the use of electronic health record clinical performance indicators, such as Resident-Sensitive Quality Measures (RSQMs) and TRainee Attributable and Automatable Care Evaluations in Real-time (TRACERs), for learner assessment. They elaborate on both the promise and the potential limitations of using such measures in a program of learner assessment.
RESUMEN
ABSTRACT: In the United States, initial board certification remains focused on a high-stakes knowledge examination after completion of training. A more contemporary view supports a program of assessment that includes multiple types and sources of data with an emphasis on direct workplace observation to get the best picture of an individual's performance. In this article, the authors reimagine initial certification as a continuous assessment for learning that begins in residency, focuses on both knowledge acquisition and its application, and interdigitates intentionally with the first cycle of maintenance of certification to advance learning and smooth the transition from training to practice. A more expanded view of competence, as a 3-layered construct (canonical, contextual, and personalized), supports this proposal. Canonical competence (context-independent knowledge)-best assessed through examinations of knowledge-is most heavily weighted and assessed during medical school but remains critical throughout one's career. Contextual competence (context-dependent knowledge) is best assessed in the workplace and is key during residency and fellowship as trainees navigate a myriad of clinical work environments. Personalized competence, representing the totality of one's unique areas of expertise, is best demonstrated during the practice years when deliberate practice experience supports the growth of personalized expertise and discoveries that advance the field. Shifting initial board certification from relying on an anxiety-provoking, high-stakes, often single moment in time standardized examination to a nuanced approach that is part of an individual trainee's program of assessment offers a more just and robust decision about readiness for unsupervised practice. Such a model would also sow the seeds of meaningful individualization of learning needs that begins in training, continues through practice, and lays the foundation for improving the quality of care for patients within a given practice as well as shifting the current contiguous phases of training and practice into a true continuum.
RESUMEN
Purpose: To acquire comments on pediatric dentistry entrustable professional activities (EPAs) from pediatric dentistry residency program directors (PDs). Methods: An electronic survey invited PDs to evaluate 16 previously developed EPAs on whether they were critical to patient safety, resident education, or both. PDs were asked to evaluate a fully developed EPA to assess structure and clarity and describe barriers to EPA. Descriptive statistics were completed. Results: Forty-one of 103 PDs completed the entire survey. Eighty-five percent (36 of 42) of PDs believed EPAs are critical to pediatric dentistry education, and 81 percent (34 of 42) believed EPAs are critical to patient safety. Eighty-one percent of PDs would likely use EPAs when available. Seventy-five percent (31 of 41) of PDs reported that they have had a resident who would have benefited from a longer duration of training. Conclusions: The majority of pediatric dentistry residency program director participants surveyed reported that entrustable professional activities are critical to patient safety and resident education. EPAs may be a valuable option for assessing residents' readiness for graduation.
Asunto(s)
Actitud del Personal de Salud , Internado y Residencia , Odontología Pediátrica , Odontología Pediátrica/educación , Humanos , Encuestas y Cuestionarios , Competencia Clínica , Seguridad del PacienteRESUMEN
OBJECTIVES: To compare level of supervision (LOS) ratings of graduating pediatric residents with their assessments as fellows for the five Entrustable Professional Activities (EPAs) common to general pediatrics and the subspecialties and to determine if the difference between ratings from residency to fellowship is less for the QI and Practice Management EPAs, since the skills needed to perform these may be less context-dependent. METHODS: We compared ratings of graduating residents with their assessments as fellows using LOS data from two sequential EPA studies. RESULTS: There were 65 ratings from 41 residents at the first fellow assessment. At graduation, most residents needed little to no supervision for all EPAs with 94% (61/65) of ratings level four or five. In contrast, only 5/65 (8%) of the first fellow assessments were level four or five. The ratings difference for the QI and Practice Management EPAs was similar to the others. CONCLUSIONS: LOS ratings for the EPAs common to generalists and subspecialists reset as residents become fellows. There was no evidence that the QI and Practice Management EPAs are less context-dependent. This study provides additional validity evidence for using these LOS scales to assess trainees in pediatric residency and fellowship.
Asunto(s)
Competencia Clínica , Becas , Internado y Residencia , Pediatría , Humanos , Pediatría/educación , Educación de Postgrado en Medicina , Masculino , FemeninoRESUMEN
Changes in digital technology, increasing volume of data collection, and advances in methods have the potential to unleash the value of big data generated through the education of health professionals. Coupled with this potential are legitimate concerns about how data can be used or misused in ways that limit autonomy, equity, or harm stakeholders. This consensus statement is intended to address these issues by foregrounding the ethical imperatives for engaging with big data as well as the potential risks and challenges. Recognizing the wide and ever evolving scope of big data scholarship, we focus on foundational issues for framing and engaging in research. We ground our recommendations in the context of big data created through data sharing across and within the stages of the continuum of the education and training of health professionals. Ultimately, the goal of this statement is to support a culture of trust and quality for big data research to deliver on its promises for health professions education (HPE) and the health of society. Based on expert consensus and review of the literature, we report 19 recommendations in (1) framing scholarship and research through research, (2) considering unique ethical practices, (3) governance of data sharing collaborations that engage stakeholders, (4) data sharing processes best practices, (5) the importance of knowledge translation, and (6) advancing the quality of scholarship through multidisciplinary collaboration. The recommendations were modified and refined based on feedback from the 2022 Ottawa Conference attendees and subsequent public engagement. Adoption of these recommendations can help HPE scholars share data ethically and engage in high impact big data scholarship, which in turn can help the field meet the ultimate goal: high-quality education that leads to high-quality healthcare.
Asunto(s)
Macrodatos , Empleos en Salud , Difusión de la Información , Humanos , Empleos en Salud/educación , ConsensoRESUMEN
INTRODUCTION: Sepsis is a leading cause of pediatric mortality. While there has been significant effort toward improving adherence to evidence-based care, gaps remain. Immersive multiuser virtual reality (MUVR) simulation may be an approach to enhance provider clinical competency and situation awareness for sepsis. METHODS: A prospective, observational pilot of an interprofessional MUVR simulation assessing a decompensating patient from sepsis was conducted from January to June 2021. The study objective was to establish validity and acceptability evidence for the platform by assessing differences in sepsis recognition between experienced and novice participants. Interprofessional teams assessed and managed a patient together in the same VR experience with the primary outcome of time to recognition of sepsis utilizing the Situation Awareness Global Assessment Technique analyzed using a logistic regression model. Secondary outcomes were perceived clinical accuracy, relevancy to practice, and side effects experienced. RESULTS: Seventy-two simulations included 144 participants. The cumulative odds ratio of recognizing sepsis at 2 min into the simulation in comparison to later time points by experienced versus novice providers were significantly higher with a cumulative odds ratio of 3.70 (95% confidence interval: 1.15-9.07, p = .004). Participants agreed that the simulation was clinically accurate (98.6%) and will impact their practice (81.1%), with a high degree of immersion (95.7%-99.3%), and the majority of side effects were perceived as mild (70.4%-81.4%). CONCLUSIONS: Our novel MUVR simulation demonstrated significant differences in sepsis recognition between experienced and novice participants. This validity evidence along with the data on the simulation's acceptability supports expanded use in training and assessment.
Asunto(s)
Sepsis , Realidad Virtual , Niño , Humanos , Concienciación , Simulación por Computador , Estudios Prospectivos , Sepsis/diagnóstico , Sepsis/terapia , Proyectos PilotoRESUMEN
Assessment in medical education has evolved through a sequence of eras each centering on distinct views and values. These eras include measurement (e.g., knowledge exams, objective structured clinical examinations), then judgments (e.g., workplace-based assessments, entrustable professional activities), and most recently systems or programmatic assessment, where over time multiple types and sources of data are collected and combined by competency committees to ensure individual learners are ready to progress to the next stage in their training. Significantly less attention has been paid to the social context of assessment, which has led to an overall erosion of trust in assessment by a variety of stakeholders including learners and frontline assessors. To meaningfully move forward, the authors assert that the reestablishment of trust should be foundational to the next era of assessment. In our actions and interventions, it is imperative that medical education leaders address and build trust in assessment at a systems level. To that end, the authors first review tenets on the social contextualization of assessment and its linkage to trust and discuss consequences should the current state of low trust continue. The authors then posit that trusting and trustworthy relationships can exist at individual as well as organizational and systems levels. Finally, the authors propose a framework to build trust at multiple levels in a future assessment system; one that invites and supports professional and human growth and has the potential to position assessment as a fundamental component of renegotiating the social contract between medical education and the health of the public.
Asunto(s)
Curriculum , Educación Médica , Humanos , Educación Basada en Competencias , Lugar de Trabajo , ConfianzaRESUMEN
INTRODUCTION: Health professions education (HPE) has adopted the conceptualization of validity as an argument. However, the theoretical and practical aspects of how validity arguments should be developed, used and evaluated in HPE have not been deeply explored. Articulating the argumentation theory undergirding validity and validation can help HPE better operationalise validity as an argument. To better understand this, the authors explored how HPE validity scholars conceptualise assessment validity arguments and argumentation, seeking to understand potential consequences of these views on validation practices. METHODS: The authors used critical case sampling to identify HPE assessment validity experts in three ways: (1) participation in a prominent validity research group, (2) appearing in a bibliometric study of HPE validity publications and (3) authorship of recent HPE validity literature. Qualitative semi-structured interviews were conducted with 16 experts in HPE assessment validity from four different countries. The authors used reflexive thematic analysis to develop themes relevant to their research question. RESULTS: The authors developed three themes grounded in participants' responses: (1) In theory, HPE validity is a social and situated argument. (2) In practice, the absence of audience and evaluation stymies the social nature of HPE validity. (3) Lack of validity argumentation creates and maintains power differentials within HPE. Participants articulated that current HPE validation practices are rooted in post-positivist epistemology when they should be situated (i.e. context-dependent), audience-centric and inclusive. DISCUSSION: When discussing validity argumentation in theory, participants' descriptions reflect an interpretivist lens for evaluation that is misaligned with real-world validity practices. This misalignment likely arises from HPE's adoption of "validity as an argument" as a slogan, without integrating theoretical and practical principles of argumentation theory.
Asunto(s)
Empleos en Salud , Humanos , Reproducibilidad de los Resultados , Empleos en Salud/educación , Investigación CualitativaRESUMEN
High-value care is what patients deserve and what healthcare professionals should deliver. However, it is not what happens much of the time. Quality improvement master Dr. Don Berwick argued more than two decades ago that American healthcare needs an escape fire, which is a new way of seeing and acting in a crisis situation. While coined in the U.S. context, the analogy applies in other Western healthcare contexts as well. Therefore, in this paper, the authors revisit Berwick's analogy, arguing that medical education can, and should, provide the spark for such an escape fire across the globe. They assert that medical education can achieve this by fully embracing competency-based medical education (CBME) as a way to place medicine's focus on the patient. CBME targets training outcomes that prepare graduates to optimize patient care. The authors use the escape fire analogy to argue that medical educators must drop long-held approaches and tools; treat CBME implementation as an adaptive challenge rather than a technical fix; demand genuine, rich discussions and engagement about the path forward; and, above all, center the patient in all they do.
Asunto(s)
Educación Basada en Competencias , Educación Médica , Humanos , Personal de Salud , Atención a la Salud , Instituciones de SaludRESUMEN
ABSTRACT: In this commentary, the authors explore the tension of balancing high performance standards in medical education with the acceptability of those standards to stakeholders (e.g., learners and patients). The authors then offer a lens through which this tension might be considered and ways forward that focus on both patient outcomes and learner needs.In examining this phenomenon, the authors argue that high performance standards are often necessary. Societal accountability is key to medical education, with the public demanding that training programs prepare physicians to provide high-quality care. Medical schools and residency programs, therefore, require rigorous standards to ensure graduates are ready to care for patients. At the same time, learners' experience is important to consider. Making sure that performance standards are acceptable to stakeholders supports the validity of assessment decisions.Equity should also be central to program evaluation and validity arguments when considering performance standards. Currently, learners across the continuum are variably prepared for the next phase in training and often face inequities in resource availability to meet high passing standards, which may lead to learner attrition. Many students who face these inequities come from underrepresented or disadvantaged backgrounds and are essential to ensuring a diverse medical workforce to meet the needs of patients and society. When these students struggle, it contributes to the leaky pipeline of more socioeconomically and racially diverse applicants.The authors posit that 4 key factors can balance the tension between high performance standards and stakeholder acceptability: standards that are acceptable and defensible, progression that is time variable, requisite support structures that are uniquely tailored for each learner, and assessment systems that are equitably designed.
Asunto(s)
Química Orgánica , Educación Médica , Humanos , Estudiantes , Evaluación de Programas y Proyectos de Salud , Personal de SaludRESUMEN
ABSTRACT: Competency-based medical education (CBME) depends on effective programs of assessment to achieve the desired outcomes and goals of training. Residency programs must be able to defend clinical competency committee (CCC) group decisions about learner readiness for practice, including decisions about time-variable resident promotion and graduation. In this article, the authors describe why CCC group decision-making processes should be supported by theory and review 3 theories they used in designing their group processes: social decision scheme theory, functional theory, and wisdom of crowds. They describe how these theories were applied in a competency-based, time-variable training pilot-Transitioning in Internal Medicine Education Leveraging Entrustment Scores Synthesis (TIMELESS) at the University of Cincinnati internal medicine residency program in 2020-2022-to increase the defensibility of their CCC group decision-making. This work serves as an example of how use of theory can bolster validity arguments supporting group decisions about resident readiness for practice.
Asunto(s)
Educación de Postgrado en Medicina , Internado y Residencia , Humanos , Competencia Clínica , Toma de Decisiones , Disentimientos y Disputas , Educación Basada en CompetenciasRESUMEN
BACKGROUND: Consults are increasingly frequent in inpatient pediatric services. Consult interactions between trainees provide a rich opportunity for teaching and learning. What constitutes meaningful teaching interactions with trainees has not been described. OBJECTIVE: Explore how consulting fellows and residents define "meaningful teaching interactions" associated with inpatient consult interactions. METHODS: Four focus groups were conducted with 21 pediatric trainees (11 subspecialty fellows and 10 residents) at one institution. Transcriptions were analyzed using thematic analysis to inductively create categories and themes. RESULTS: Five factors define meaningful teaching interactions: (1) Relevance; (2) Quick Hits; (3) Vibe; (4) Face-to-face Interactions; and (5) Timing and Busyness. Meaningful content was described as relevant to current or future patient care. Residents valued content that would enable them to explain the reasoning behind recommendations and think through the next steps. Trainees highlighted brief clinical pearls as superior to longer teaching sessions. The "vibe" between resident and fellow was described as a prerequisite to meaningful teaching and included aspects of interest, receptivity, tone, and attitude. Face-to-face interactions were preferred by many trainees, from initial consults to seeing patients or co-rounding. Timing and workload reflected discordant schedules, including time of day and week, but setting a planned time for teaching was beneficial. CONCLUSION: Relevant, bite-sized educational content combined with a good vibe and optimal timing creates a context in which consult fellows can foster meaningful teaching opportunities for residents.
Asunto(s)
Internado y Residencia , Humanos , Niño , Educación de Postgrado en Medicina , Aprendizaje , Grupos Focales , Derivación y Consulta , EnseñanzaRESUMEN
ABSTRACT: Previous eras of assessment in medical education have been defined by how assessment is done, from knowledge exams popularized in the 1960s to the emergence of work-based assessment in the 1990s to current efforts to integrate multiple types and sources of performance data through programmatic assessment. Each of these eras was a response to why assessment was performed (e.g., assessing medical knowledge with exams; assessing communication, professionalism, and systems competencies with work-based assessment). Despite the evolution of assessment eras, current evidence highlights the graduation of trainees with foundational gaps in the ability to provide high-quality care to patients presenting with common problems, and training program leaders report they graduate trainees they would not trust to care for themselves or their loved ones. In this article, the authors argue that the next era of assessment should be defined by why assessment is done: to ensure high-quality, equitable care. Assessment should place focus on demanding graduates possess the knowledge, skills, attitudes, and adaptive expertise to meet the needs of all patients and ensuring that graduates are able to do this in an equitable fashion. The authors explore 2 patient-focused assessment approaches that could help realize the promise of this envisioned era: entrustable professional activities (EPAs) and resident sensitive quality measures (RSQMs)/TRainee Attributable and Automatable Care Evaluations in Real-time (TRACERs). These examples illustrate how the envisioned next era of assessment can leverage existing and new data to provide precision education assessment that focuses on providing formative and summative feedback to trainees in a manner that seeks to ensure their learning outcomes prepare them to ensure high-quality, equitable patient outcomes.
Asunto(s)
Internado y Residencia , Calidad de la Atención de Salud , Humanos , Curriculum , Educación Basada en Competencias , Atención al Paciente , Competencia Clínica , Educación de Postgrado en MedicinaRESUMEN
ABSTRACT: Precision education (PE) leverages longitudinal data and analytics to tailor educational interventions to improve patient, learner, and system-level outcomes. At present, few programs in medical education can accomplish this goal as they must develop new data streams transformed by analytics to drive trainee learning and program improvement. Other professions, such as Major League Baseball (MLB), have already developed extremely sophisticated approaches to gathering large volumes of precise data points to inform assessment of individual performance.In this perspective, the authors argue that medical education-whose entry into precision assessment is fairly nascent-can look to MLB to learn the possibilities and pitfalls of precision assessment strategies. They describe 3 epochs of player assessment in MLB: observation, analytics (sabermetrics), and technology (Statcast). The longest tenured approach, observation, relies on scouting and expert opinion. Sabermetrics brought new approaches to analyzing existing data in a way that better predicted which players would help the team win. Statcast created precise, granular data about highly attributable elements of player performance while helping to account for nonplayer factors that confound assessment such as weather, ballpark dimensions, and the performance of other players. Medical education is progressing through similar epochs marked by workplace-based assessment, learning analytics, and novel measurement technologies. The authors explore how medical education can leverage intersectional concepts of MLB player and medical trainee assessment to inform present and future directions of PE.
Asunto(s)
Béisbol , Educación Médica , Humanos , Escolaridad , Lugar de TrabajoRESUMEN
INTRODUCTION: The real-world mechanisms underlying prospective entrustment decision making (PEDM) by entrustment or clinical competency committees (E/CCCs) are poorly understood. To advance understanding in this area, the authors conducted a realist synthesis of the published literature to address the following research question: In E/CCC efforts to make defensible prospective entrustment decisions (PEDs), what works, for whom, under what circumstances and why? METHODS: Realist work seeks to understand the contexts (C), mechanisms (M) and outcomes (O) that explain how and why things work (or do not). In the authors' study, contexts included individual E/CCC members, E/CCC structures and processes, and training programmes. The outcome (i.e. desired outcome) was a PED. Mechanisms were a substantial focus of the analysis and informed the core findings. To define a final corpus of 52 included papers, the authors searched four databases, screened all results from those searches and performed a full-text review of a subset of screened papers. Data extraction focused on developing context-mechanism-outcome configurations from the papers, which were used to create a theory for how PEDM leads to PEDs. RESULTS: PEDM is often driven by default (non-deliberate) decision making rather than a deliberate process of deciding whether a trainee should be entrusted or not. When defaulting, some E/CCCs find red flags that sometimes lead to being more deliberate with decision making. E/CCCs that seek to be deliberate describe PEDM that can be effortful (when data are insufficient or incongruent) or effortless (when data are robust and tell a congruent story about a trainee). Both information about trainee trustworthiness and the sufficiency of data about trainee performance influence PEDM. Several moderators influence what is considered to be sufficient data, how trustworthiness data are viewed and how PEDM is carried out. These include perceived consequences and associated risks, E/CCC member trust propensity, E/CCC member personal knowledge of and experience with trainees and E/CCC structures and processes. DISCUSSION: PEDM is rarely deliberate but should be. Data about trainee trustworthiness are foundational to making PEDs. Bias, equity and fairness are nearly absent from the papers in this synthesis, and future efforts must seek to advance understanding and practice regarding the roles of bias, equity and fairness in PEDM.