Mobile technologies to support workplace-based assessment for entrustment decisions: Guidelines for programs and educators: AMEE Guide No. 154.

With the rise of competency-based medical education and workplace-based assessment (WBA) since the turn of the century, much has been written about methods of assessment. Direct observation and other sources of information have become standard in many clinical programs. Entrustable professional activities (EPAs) have also become a central focus of assessment in the clinical workplace. Paper and pencil (one of the earliest mobile technologies!) to document observations have become almost obsolete with the advent of digital technology. Typically, clinical supervisors are asked to document assessment ratings using forms on computers. However, accessing these forms can be cumbersome and is not easily integrated into existing clinical workflows. With a call for more frequent documentation, this practice is hardly sustainable, and mobile technology is quickly becoming indispensable. Documentation of learner performance at the point of care merges WBA with patient care and WBA increasingly uses smartphone applications for this purpose.This AMEE Guide was developed to support institutions and programs who wish to use mobile technology to implement EPA-based assessment and, more generally, any type of workplace-based assessment. It covers backgrounds of WBA, EPAs and entrustment decision-making, provides guidance for choosing or developing mobile technology, discusses challenges and describes best practices.

Instructional Design Strategies for Teaching the Mental Status Examination and Psychiatric Interview: a Scoping Review.

OBJECTIVE: The psychiatric mental status examination is a fundamental aspect of the psychiatric clinical interview. However, despite its importance, little emphasis has been given to evidence-based instructional design. Therefore, this review summarizes the literature from an instructional design perspective with the aim of uncovering design strategies that have been used for teaching the psychiatric interview and mental status examination to health professionals. METHODS: The authors conducted a scoping review. Multiple databases, reference lists, and the gray literature were searched for relevant publications across educational levels and professions. A cognitive task analysis and an instructional design framework was used to summarize and chart the findings. RESULTS: A total of 61 articles from 17 countries in six disciplines and three educational levels were identified for data extraction and analysis. Most studies were from the USA, presented as educational case reports, and carried out in undergraduate education in the field of psychiatry. Few articles described the instructional rationale for their curriculum. None of the studies compared the effectiveness of different instructional design components. Reported learning activities for each task domain (knowledge, skills, and attitudes) and for each step of an instructional design process were charted. Most articles reported the use of introductory seminars or lectures in combination with digital learning material (videos and virtual patients in more recent publications) and role-play exercises. CONCLUSIONS: Educators in psychiatry should consider all task domains of the psychiatric interview and mental status examination. Currently, there is a lack of empirical research on expertise acquisition and use of instructional design frameworks in this context.

Evidence for validity for the Cognitive Load Inventory for Handoffs.

CONTEXT: Patient handovers remain a significant patient safety challenge. Cognitive load theory (CLT) can be used to identify the cognitive mechanisms for handover errors. The ability to measure cognitive load types during handovers could drive the development of more effective curricula and protocols. No such measure currently exists. METHODS: The authors developed the Cognitive Load Inventory for Handoffs (CLIH) using a multi-step process, including expert interviews to enhance content validity and talk-alouds to optimise response process validity. The final version contained 28 items. From January to March 2019, we administered a cross-sectional survey to 1807 residents and fellows from a large health care system in the USA. Participants completed the CLIH following a handover. Exploratory factor analysis of data from one-third of respondents identified high-performing items; confirmatory factor analysis of data from the remaining sample assessed model fit. Model fit was evaluated using the comparative fit index (CFI) (>0.90), Tucker-Lewis index (TFI) (>0.80), standardised root mean square residual (SRMR) (<0.08) and root mean square of error of approximation (RMSEA) (<0.08). RESULTS: Participants included 693 trainees (38.4%) (231 in the exploratory study and 462 in the confirmatory study). Eleven items were removed during exploratory factor analysis. Confirmatory factor analysis of the 16 remaining items (five for intrinsic load, seven for extraneous load and four for germane load) supported a three-factor model and met criteria for good model fit: the CFI was 0.95, TFI was 0.93, RMSEA was 0.074 and SRMR was 0.07. The factor structure was comparable for gender and role. Intrinsic, extraneous and germane load scales had high internal consistency. With one exception, scale scores were associated, as hypothesised, with postgraduate level and clinical setting. CONCLUSIONS: The CLIH measures three types of cognitive load during patient handovers. Evidencefor validity is provided for the CLIH's content, response process, internal structure and association with other variables. This instrument can be used to determine the relative drivers of cognitive load during handovers in order to optimize handover instruction and protocols.

The medical pause: Importance, processes and training.

Research has shown that taking 'timeouts' in medical practice improves performance and patient safety. However, the benefits of taking timeouts, or pausing, are not sufficiently acknowledged in workplaces and training programmes. To promote this acknowledgement, we suggest a systematic conceptualisation of the medical pause, focusing on its importance, processes and implementation in training programmes. By employing insights from educational and cognitive psychology, we first identified pausing as an important skill to interrupt negative momentum and bolster learning. Subsequently, we categorised constituent cognitive processes for pausing skills into two phases: the decision-making phase (determining when and how to take pauses) and the executive phase (applying relaxation or reflection during pauses). We present a model that describes how relaxation and reflection during pauses can optimise cognitive load in performance. Several strategies to implement pause training in medical curricula are proposed: intertwining pause training with training of primary skills, providing second-order scaffolding through shared control and employing auxiliary tools such as computer-based simulations with a pause function.

Exploring the relationship between emotion and cognitive load types during patient handovers.

Cognitive Load Theory has emerged as an important approach to improving instruction in the health professions workplace, including patient handovers. At the same time, there is growing recognition that emotion influences learning through numerous cognitive processes including motivation, attention, working memory, and long-term memory. This study explores how emotion influences the cognitive load experienced by trainees performing patient handovers. From January to March 2019, 693 (38.7%) of 1807 residents and fellows from a 24-hospital health system in New York city completed a survey after performing a handover. Participants rated their emotional state and cognitive load. The survey included questions about features of the learner, task, and instructional environment. The authors used factor analysis to identify the core dimensions of emotion. Regression analyses explored the relationship between the emotion factors and cognitive load types. Two emotion dimensions were identified representing invigoration and tranquility. In regression analyses, higher levels of invigoration, tranquility, and their interaction were independently associated with lower intrinsic load and extraneous load. The interaction of invigoration and tranquility predicted lower germane load. The addition of the emotion variables to multivariate models including other predictors of cognitive load types significantly increased the amount of variance explained. The study provides a model for measuring emotions in workplace learning. Because emotion appears to have a significant influence on cognitive load types, instructional designers should consider strategies that help trainees regulate emotion in order to reduce cognitive load and improve learning and performance.

Questioning medical competence: Should the Covid-19 crisis affect the goals of medical education?

The COVID-19 pandemic has disrupted many societal institutions, including health care and education. Although the pandemic's impact was initially assumed to be temporary, there is growing conviction that medical education might change more permanently. The International Competency-based Medical Education (ICBME) collaborators, scholars devoted to improving physician training, deliberated how the pandemic raises questions about medical competence. We formulated 12 broad-reaching issues for discussion, grouped into micro-, meso-, and macro-level questions. At the individual micro level, we ask questions about adaptability, coping with uncertainty, and the value and limitations of clinical courage. At the institutional meso level, we question whether curricula could include more than core entrustable professional activities (EPAs) and focus on individualized, dynamic, and adaptable portfolios of EPAs that, at any moment, reflect current competence and preparedness for disasters. At the regulatory and societal macro level, should conditions for licensing be reconsidered? Should rules of liability be adapted to match the need for rapid redeployment? We do not propose a blueprint for the future of medical training but rather aim to provoke discussions needed to build a workforce that is competent to cope with future health care crises.

Psychiatry Clinical Skills Evaluation: a Multisite Study of Validity.

OBJECTIVE: Since 2007, the American Board of Psychiatry and Neurology (ABPN) has required that residency programs conduct a specific clinical skills evaluation (CSE) of physician-patient interaction, psychiatric interview and mental status examination, and case presentation on a directly observed patient interview as a prerequisite for certification. The authors examined a multisite database of CSE assessments to investigate the validity of the evaluation. METHODS: The authors collected 1156 CSE assessments from 4 residency programs conducted over a 6-year period, compared scoring patterns among the programs, score improvement over 4 years of residency, time and number of CSEs required to meet ABPN requirements, and patterns of scoring for individual faculty evaluators. RESULTS: The distribution of scores within each of the 4 programs showed similar, but nonidentical patterns. The number of CSEs required to meet the ABPN standards (3.5) and the point in training at which this was completed (late PGY-2) were the same in all programs. CSE scores were highly correlated with year of training but were not correlated with performance on an unrelated cognitive examination. Individual faculty members tended to stay within a moderate range of scores over multiple residents, partially attributable to year of training. CONCLUSIONS: Taken together, these findings support the validity of the CSE as a measure of residents' clinical skills in the specified areas and demonstrate a moderate-high degree of consistency in the scoring of the CSE across these 4 programs.

Overcoming the Challenges of Direct Observation and Feedback Programs: A Qualitative Exploration of Resident and Faculty Experiences.

Problem: Prior studies have reported significant negative attitudes amongst both faculty and residents toward direct observation and feedback. Numerous contributing factors have been identified, including insufficient time for direct observation and feedback, poorly understood purpose, inadequate training, disbelief in the formative intent, inauthentic resident-patient clinical interactions, undermining of resident autonomy, lack of trust between the faculty-resident dyad, and low-quality feedback information that lacks credibility. Strategies are urgently needed to overcome these challenges and more effectively engage faculty and residents in direct observation and feedback. Otherwise, the primary goals of supporting both formative and summative assessment will not be realized and the viability of competency-based medical education will be threatened. Intervention: Toward this end, recent studies have recommended numerous strategies to overcome these barriers: protected time for direct observation and feedback; ongoing faculty and resident training on goals and bidirectional, co-constructed feedback; repeated direct observations and feedback within a longitudinal resident-supervisor relationship; utilization of assessment tools with evidence for validity; and monitoring for engagement. Given the complexity of the problem, it is likely that bundling multiple strategies together will be necessary to overcome the challenges. The Direct Observation Structured Feedback Program (DOSFP) incorporated many of the recommended features, including protected time for direct observation and feedback within longitudinal faculty-resident relationships. Using a qualitative thematic approach the authors conducted semi-structured interviews, during February and March, 2019, with 10 supervisors and ten residents. Participants were asked to reflect on their experiences. Interview guide questions explored key themes from the literature on direct observation and feedback. Transcripts were anonymized. Two authors independently and iteratively coded the transcripts. Coding was theory-driven and differences were discussed until consensus was reached. The authors then explored the relationships between the codes and used a semantic approach to construct themes. Context: The DOSFP was implemented in a psychiatry continuity clinic for second and third year residents. Impact: Faculty and residents were aligned around the goals. They both perceived the DOSFP as focused on growth rather than judgment even though residents understood that the feedback had both formative and summative purposes. The DOSFP facilitated educational alliances characterized by trust and respect. With repeated practice within a longitudinal relationship, trainees dropped the performance orientation and described their interactions with patients as authentic. Residents generally perceived the feedback as credible, described feedback quality as high, and valued the two-way conversation. However, when receiving feedback with which they did not agree, residents demurred or, at most, would ask a clarifying question, but then internally discounted the feedback. Lessons Learned: Direct observation and structured feedback programs that bundle recent recommendations may overcome many of the challenges identified by previous research. Yet, residents discounted disagreeable feedback, illustrating a significant limitation and the need for other strategies that help residents reconcile conflict between external data and one's self-appraisal.

Exploring Residents' Experience of Career Development Scholarship Tracks: A Qualitative Case Study Using Social Cognitive Career Theory.

Problem: Trainees enter graduate medical education with professional aspirations that often extend beyond the role of clinician to roles such as educator, innovator, leader, advocate, or researcher. Many residency programs have implemented academic tracks to support career development in these areas. With the exception of research tracks, these tracks generally do not include significant longitudinal protected time and often rely upon 'extra-curricular' effort and possess insufficient structure, mentorship, and accountability. Most prior studies of non-research scholarship tracks have not been theory driven and do not explore in depth the experience of residents who participate. Approach: To address this gap in the literature, we conducted a qualitative case study informed by Social Cognitive Career Theory to explore the professional identity development of residents who participated in a non-research scholarship track that incorporates recommended best practices. The track, Pathways to Expertise Program, incorporates features of successful research tracks: protected time, longitudinal experience, mentorship, platforms for recognition, and accountability. Participants from the first three cohorts were interviewed at the time of their graduation (2017-2019). Semi-structured interviews were conducted, transcribed, and independently coded. Social Cognitive Career Theory informed the organization of codes into themes. Context: The Pathways to Expertise Program was implemented in a psychiatry residency training program in a large urban academic teaching hospital. Impact: Fifteen residents entered Pathways to Expertise Program during the study period and all 15 participated in the study. Fourteen completed the program and presented their projects at the department grand rounds. For dissemination, 12 presented their project findings at one or more national meetings in the form of a poster (20 distributed across 11 residents), workshop (six distributed across four residents), or presentation (two across two residents). Six residents accounted for a total of seven first author publications in peer reviewed journals. All participants described how their self-efficacy increased as a result of new skills (e.g., content, methodology, and scientific communication), mentorship (e.g., content and process guidance), peer and broader support (e.g., small group supervision), persuasive communications (e.g., recognition both locally and nationally), and positive emotional reactions (e.g., triumph). The residents also described expecting compelling benefits (e.g., stronger application for fellowship and expanded career opportunities). Participants indicated that the experience influenced their career goals and how they perceived their professional identities. Lessons Learned: These findings suggest that a longitudinal academic track that incorporates features of successful research tracks (protected time, mentorship, peer support, and accountability for deliverables) can be instrumental in forming and maturing professional identities for non-clinical roles. These tracks can accomplish several important goals, including enhancing resilience via identity formation around passion and purpose and meeting society's need for physicians who are engaged in inquiry and innovation. Implications for the design of academic tracks in general are explored.

Trainee perception of cognitive load during observed faculty staff teaching of procedural skills.

OBJECTIVES: Although teachers impact learners' cognitive load, how specific teaching activities affect intrinsic, germane and extraneous load during procedural skills training is unknown. We sought to characterise teaching activities used in the exemplar procedural setting of colonoscopy, and to explore how they were enacted and how learners perceived them as affecting intrinsic, germane and extraneous cognitive load. METHODS: We observed 10 colonoscopies performed by eight different gastroenterology fellows and supervised by 10 different attending physicians at two hospitals, and recorded the teaching activities observed, as well as details of when they were used and how they were enacted. After the colonoscopy, each fellow completed the Cognitive Load Inventory for Colonoscopy to quantify intrinsic, germane and extraneous load. We then interviewed each fellow to determine how he or she perceived teaching as affecting cognitive load. Qualitative data were subjected to content analysis. Instances of germane load-promoting activities were correlated with measured germane load. RESULTS: We observed 515 instances of teaching activities. The intensity of teaching varied substantially, ranging from 0.7 to 3.3 activities per minute, as did the pattern of teaching activities used by different attending physicians. Little teaching occurred immediately before or after a procedure. Fellows usually perceived teaching as affecting cognitive load in ways that promoted learning, particularly by reducing intrinsic load and increasing germane load. Fellows strongly perceived that the provision of autonomy promoted germane load. Conversely, fellows perceived that excessive teaching increased extraneous load. Instances of germane load-promoting teaching activities correlated moderately with measured germane load. CONCLUSIONS: Teaching in the exemplar procedural setting of colonoscopy affected learners' cognitive load in mostly beneficial ways, yet even 'good' teaching activities had detrimental effects when used excessively. Teachers of procedures should consider learner experience, task complexity and environmental factors to modulate the modality, content and intensity of teaching to promote balanced cognitive load and learning. Teaching more reservedly during the procedure and taking advantage of pre- and post-procedure opportunities may help.

Cognitive load theory for training health professionals in the workplace: A BEME review of studies among diverse professions: BEME Guide No. 53.

AIM: Cognitive load theory (CLT) is of increasing interest to health professions education researchers. CLT has intuitive applicability to workplace settings, yet how CLT should inform teaching, learning, and research in health professions workplaces is unclear. METHOD: To map the existing literature, we performed a scoping review of studies involving cognitive load, mental effort and/or mental workload in professional workplace settings within and outside of the health professions. We included actual and simulated workplaces and workplace tasks. RESULT: Searching eight databases, we identified 4571 citations, of which 116 met inclusion criteria. Studies were most often quantitative. Methods to measure cognitive load included psychometric, physiologic, and secondary task approaches. Few covariates of cognitive load or performance were studied. Overall cognitive load and intrinsic load were consistently negatively associated with the level of experience and performance. Studies consistently found distractions and other aspects of workplace environments as contributing to extraneous load. Studies outside the health professions documented similar findings to those within the health professions, supporting relevance of CLT to workplace learning. CONCLUSION: The authors discuss implications for workplace teaching, curricular design, learning environment, and metacognition. To advance workplace learning, the authors suggest future CLT research should address higher-level questions and integrate other learning frameworks.

Evidence for the Validity of the Psychopharmacotherapy-Structured Clinical Observation Tool: Results of a Factor and Time Series Analysis.

OBJECTIVE: The Psychopharmacotherapy-Structured Clinical Observation (P-SCO) is a direct observation tool designed to assess resident performance of a medication visit. This study examines two dimensions of validity for the P-SCO: internal structure and how scores correlate with another variable associated with competence (experience). METHODS: The faculty completed 601 P-SCOs over 4 years. Multilevel exploratory factor analysis was performed with minimum thresholds for eigenvalue (≥ 1.0) and proportion of variance explained (≥ 5.0%). Internal reliability was assessed with Cronbach alpha. To examine how scores changed with experience, mean ratings (1-4 scale) were calculated for each factor by quarter of the academic year. Separate linear mixed models were also performed. RESULTS: The analysis yielded three factors that explained 50% of the variance and demonstrated high internal reliability: affective tasks (alpha = 0.90), cognitive tasks (alpha = 0.84), and hard tasks (alpha = 0.74). Items within "hard tasks" were assessment of substance use, violence risk, and adherence, and inquiry about interactions with other providers. Monitoring adverse effects did not load on the hard task factor but also had overall low mean ratings. Compared to the first quarter, fourth quarter scores for affective tasks (b = 0.54, p < 0.01) and hard tasks (b = 0.46, p = 0.02) were significantly improved while cognitive tasks had a non-significant increase. For the hard tasks, the proportion of residents with a low mean rating improved but was still over 30% during the fourth quarter. CONCLUSIONS: The results provide evidence for the validity of the P-SCO with respect to its internal structure and how scores correlate with experience. Curricular implications are explored, especially for the tasks that were hard to learn.

Performance Assessment of Pharmacotherapy: Results from a Content Validity Survey of the Psychopharmacotherapy-Structured Clinical Observation (P-SCO) Tool.

OBJECTIVE: The Psychopharmacotherapy-Structured Clinical Observation (P-SCO) tool is designed to assess performance of a medication management visit and to enhance feedback. Prior research indicated that the P-SCO was feasible to implement in a resident clinic and generated behaviorally specific, high-quality feedback. This research also highlighted problems with some of the instrument's items. This study seeks to improve the items. METHODS: The authors initially revised the P-SCO items based on the problems identified by a prior study. Next, these items were iteratively modified by experts in clinical pharmacotherapy and educational assessment. Forty-five items emerged. Finally, faculty attending an annual department education retreat rated each item on its relevance (4-point scale) and provided comments on how the item might be revised. For final inclusion, an item must have met a quantitative threshold (i.e., content validity index equal to or greater than 0.8 and the lower end of the asymmetric confidence interval equal to or greater than 3.0) and received comments that were supportive. RESULTS: Forty-one of the 45 items had strong quantitative support. However, the comments endorsed lumping a number of items in order to decrease overlap between items and to shorten the instrument. This process resulted in the further elimination of 15 items. CONCLUSIONS: The revised 26-item P-SCO builds upon prior evidence of feasibility and utility and now possesses additional evidence of content validity. The use of the tool should enhance feedback and improve the capacity of educational programs to assess performance.

Year-End Resident Clinic Handoffs: Narrative Review and Recommendations for Improvement.

BACKGROUND: Year-end clinic handoffs in resident continuity clinics are an important patient safety issue. METHODS: Intervention articles addressing the year-end resident clinic handoff were identified in a targeted literature search. These articles were reviewed and abstracted to summarize the current literature. On the basis of these reviews and consensus expert opinion, recommendations to improve year-end clinic handoffs were developed. RESULTS: Of 23 identified articles, 10 intervention articles in the fields of internal medicine, internal medicine-pediatrics, psychiatry, and family medicine were ultimately included. The additional 13 nonintervention studies were used as background material. There were 12 clinic handoff recommendations for improvement: (1) focus on patients most at risk during the handoff, (2) educate residents, (3) consider balancing caseloads for the residents, (4) prepare patients for the handoff and perform patient-centered outreach, (5) standardize a written method of sign-out and require verbal communication for a subset of patients, (6) use a standardized template or technology solution for the handoff, (7) identify specific tasks that require follow-up, (8) enhance attending supervision during the handoff, (9) make patient assignments clear after the handoff, (10) have patients establish care with the new provider as soon as possible after the handoff, (11) establish care with telephone contact prior to the first visit, (12) perform safety audits to ensure that sign-out occurs, patients receive appointments, no-shows are rescheduled, and task follow-up is completed. CONCLUSION: There is emerging evidence for interventions to improve year-end resident clinic handoffs, and the recommendations provided are a starting point to guide training programs.

Assessing Residents' Confidence in the Context of Pharmacotherapy Competence.

OBJECTIVE: We aimed to determine whether residents' confidence initiating medications increased with the number of times they prescribed individual medications and to quantify the relationship between prescription frequency and gains in confidence. METHODS: From July 2011 to June 2014, PGY-3 residents completed a survey of confidence levels at their psychopharmacology clinic orientation and then again 12 months later. The Emory Healthcare electronic medical record was used to identify all medications prescribed by each resident during their 12-month rotation and the frequency of these prescriptions. RESULTS: Confidence in initiating treatment with all medicines/medication classes increased over the 12-month period. For three of the medication classes for which residents indicated they were least confident at orientation, the number of prescriptions written during the year was significantly associated with an increase in confidence. CONCLUSIONS: Measuring resident confidence is a relevant and achievable outcome and provides data for educators regarding the amount of experience needed to increase confidence.

Measuring cognitive load during procedural skills training with colonoscopy as an exemplar.

OBJECTIVES: Few studies have investigated cognitive factors affecting learning of procedural skills in medical education. Cognitive load theory, which focuses on working memory, is highly relevant, but methods for measuring cognitive load during procedural training are not well understood. Using colonoscopy as an exemplar, we used cognitive load theory to develop a self-report instrument to measure three types of cognitive load (intrinsic, extraneous and germane load) and to provide evidence for instrument validity. METHODS: We developed the instrument (the Cognitive Load Inventory for Colonoscopy [CLIC]) using a multi-step process. It included 19 items measuring three types of cognitive load, three global rating items and demographics. We then conducted a cross-sectional survey that was administered electronically to 1061 gastroenterology trainees in the USA. Participants completed the CLIC following a colonoscopy. The two study phases (exploratory and confirmatory) each lasted for 10 weeks during the 2014-2015 academic year. Exploratory factor analysis determined the most parsimonious factor structure; confirmatory factor analysis assessed model fit. Composite measures of intrinsic, extraneous and germane load were compared across years of training and with global rating items. RESULTS: A total of 477 (45.0%) invitees participated (116 in the exploratory study and 361 in the confirmatory study) in 154 (95.1%) training programmes. Demographics were similar to national data from the USA. The most parsimonious factor structure included three factors reflecting the three types of cognitive load. Confirmatory factor analysis verified that a three-factor model was the best fit. Intrinsic, extraneous and germane load items had high internal consistency (Cronbach's alpha 0.90, 0.87 and 0.96, respectively) and correlated as expected with year in training and global assessment of cognitive load. CONCLUSIONS: The CLIC measures three types of cognitive load during colonoscopy training. Evidence of validity is provided. Although CLIC items relate to colonoscopy, the development process we detail can be used to adapt the instrument for use in other learning settings in medical education.

Influence of learner knowledge and case complexity on handover accuracy and cognitive load: results from a simulation study.

CONTEXT: The handover represents a high-risk event in which errors are common and lead to patient harm. A better understanding of the cognitive mechanisms of handover errors is essential to improving handover education and practice. OBJECTIVES: This paper reports on an experiment conducted to study the effects of learner knowledge, case complexity (i.e. cases with or without a clear diagnosis) and their interaction on handover accuracy and cognitive load. METHODS: Participants were 52 Dutch medical students in Years 2 and 6. The experiment employed a repeated-measures design with two explanatory variables: case complexity (simple or complex) as the within-subject variable, and learner knowledge (as indicated by illness script maturity) as the between-subject covariate. The dependent variables were handover accuracy and cognitive load. Each participant performed a total of four simulated handovers involving two simple cases and two complex cases. RESULTS: Higher illness script maturity predicted increased handover accuracy (p < 0.001) and lower cognitive load (p = 0.007). Case complexity did not independently affect either outcome. For handover accuracy, there was no interaction between case complexity and illness script maturity. For cognitive load, there was an interaction effect between illness script maturity and case complexity, indicating that more mature illness scripts reduced cognitive load less in complex cases than in simple cases. CONCLUSIONS: Students with more mature illness scripts performed more accurate handovers and experienced lower cognitive load. For cognitive load, these effects were more pronounced in simple than complex cases. If replicated, these findings suggest that handover curricula and protocols should provide support that varies according to the knowledge of the trainee.

Unpacking the Complexity of Patient Handoffs Through the Lens of Cognitive Load Theory.

ISSUE: The transfer of a patient from one clinician to another is a high-risk event. Errors are common and lead to patient harm. More effective methods for learning how to give and receive sign-out is an important public health priority. EVIDENCE: Performing a handoff is a complex task. Trainees must simultaneously apply and integrate clinical, communication, and systems skills into one time-limited and highly constrained activity. The task demands can easily exceed the information-processing capacity of the trainee, resulting in impaired learning and performance. Appreciating the limits of working memory can help identify the challenges that instructional techniques and research must then address. Cognitive load theory (CLT) identifies three types of load that impact working memory: intrinsic (task-essential), extraneous (not essential to task), and germane (learning related). The authors generated a list of factors that affect a trainee's learning and performance of a handoff based on CLT. The list was revised based on feedback from experts in medical education and in handoffs. By consensus, the authors associated each factor with the type of cognitive load it primarily effects. The authors used this analysis to build a conceptual model of handoffs through the lens of CLT. IMPLICATIONS: The resulting conceptual model unpacks the complexity of handoffs and identifies testable hypotheses for educational research and instructional design. The model identifies features of a handoff that drive extraneous, intrinsic, and germane load for both the sender and the receiver. The model highlights the importance of reducing extraneous load, matching intrinsic load to the developmental stage of the learner and optimizing germane load. Specific CLT-informed instructional techniques for handoffs are explored. Intrinsic and germane load are especially important to address and include factors such as knowledge of the learner, number of patients, time constraints, clinical uncertainties, overall patient/panel complexity, interacting comorbidities or therapeutics, experience or specialty gradients between the sender and receiver, the maturity of the evidence base for the patient's disease, and the use of metacognitive techniques. Research that identifies which cognitive load factors most significantly affect the learning and performance of handoffs can lead to novel, contextually adapted instructional techniques and handoff protocols. The application of CLT to handoffs may also help with the further development of CLT as a learning theory.

The Systems SOAP Note: A Systems Learning Tool.

OBJECTIVE: Systems-based practice (SBP) is the only Accreditation Council for Graduate Medical Education (ACGME) competency concerned with public health and is relatively neglected in residency curricula. A tool was developed and pilot-tested to improve SBP learning on inpatient psychiatry rotations. METHODS: A four-step approach was used: (1) literature review, (2) expert consultation, (3) tool development, and (4) pilot testing on four cases and evaluation for completion time and preliminary efficacy. RESULTS: Out of 51 SBP articles, six (12%) focused on psychiatric residency programs, and none had a practical SBP learning tool. The "systems SOAP (subjective, objective, assessment, plan) note" (S-SOAP) was structured after a clinical SOAP note and was easy to use (mean completion time = 60 min), and residents self-reported more insight into systems issues. CONCLUSIONS: The S-SOAP tool was effectively integrated into clinical experience and provided insight into systemic complexities. Future research should assess SBP knowledge acquisition after the use of such tools.