How Teachers Can Promote Assessment for Learning During All Phases of Learning.

Assessment for Learning (AFL) includes all educational activities designed to improve teaching and learning through gathering, sharing, and acting on information. AFL expands on the concept of formative assessment-which focuses mainly on an in-the-moment assessment activity-to include all processes that facilitate teachers and learners actively seeking and interpreting evidence to guide learning. AFL involves teachers and learners as partners and uses evidence to identify what the learner needs to learn (planning), review where the learner is in their learning (observing), and strategize how to maximize learning (supporting). 1 We provide the following guidelines for teachers to implement AFL within these 3 phases of a learning activity. Teachers should tailor their choice of AFL strategies to suit their relationship with the learner and learning environment context, aiming to support the development of self-regulated learning and metacognitive skills.

Development of Health Professions Education Competencies for a National Fellowship Program: A Group Consensus Process.

ABSTRACT: Health professions educators need knowledge, skills, and attitudes to provide high-quality education within dynamic clinical learning environments. Although postgraduate training opportunities in health professions education (HPE) have increased significantly, no shared competencies exist across the field. This article describes the systematic development of postgraduate HPE competencies for the Health Professions Education, Evaluation, and Research (HPEER) Advanced Fellowship, a 2-year, interprofessional, post-master's degree and postdoctoral HPE training program funded through the Department of Veterans Affairs' Office of Academic Affiliations. Using a modified RAND/University of California at Los Angeles Appropriateness Method, the authors developed competencies from March 2021 to August 2021 that were informed by current practices and standards in HPE. Literature reviews were conducted of published literature from November 2020 to February 2021 and gray literature from February to March 2021, identifying 78 and 274 HPE training competencies, respectively. These competencies were combined with 71 competencies submitted by program faculty from 6 HPEER fellowship sites, resulting in 423 initial competencies. Competencies were organized into 6 primary domains and condensed by deleting redundant items and combining overlapping concepts. Eight subject matter experts completed 3 Delphi surveys and met during 2 nominal group technique meetings. The final 25 competencies spanned 6 domains: teaching methods and learning theories; educational assessment and program evaluation; educational research methods; diversity, equity, and inclusion; interprofessional practice and team science; and leadership and management. In addition to informing the national HPEER curriculum, program evaluation, and learner assessment, these advanced competencies describe the knowledge, skills, and attitudes health professions educators need to provide high-quality education with an emphasis on the global and societal impact of HPE. These competencies represent a step toward leveraging HPE expertise to establish competencies to drive HPE program changes. Other programs should report their competencies and competency development processes to spur further dialog and progress.

"Finding My Piece in That Puzzle": A Qualitative Study Exploring How Medical Students at Four U.S. Schools Envision Their Future Professional Identity in Relation to Health Systems.

PURPOSE: Health systems science (HSS) curricula equip future physicians to improve patient, population, and health systems outcomes (i.e., to become "systems citizens"), but the degree to which medical students internalize this conception of the physician role remains unclear. This study aimed to explore how students envision their future professional identity in relation to the system and identify experiences relevant to this aspect of identity formation. METHOD: Between December 2018 and September 2019, authors interviewed 48 students at 4 U.S. medical schools with HSS curricula. Semistructured interviews were audiorecorded, transcribed, and analyzed iteratively using inductive thematic analysis. Interview questions explored how students understood the health system, systems-related activities they envisioned as future physicians, and experiences and considerations shaping their perspectives. RESULTS: Most students anticipated enacting one or more systems-related roles as a future physician, categorized as "bottom-up" efforts enacted at a patient or community level (humanist, connector, steward) or "top-down" efforts enacted at a system or policy level (system improver, system scholar, policy advocate). Corresponding activities included attending to social determinants of health or serving medically underserved populations, connecting patients with team members to address systems-related barriers, stewarding health care resources, conducting quality improvement projects, researching/teaching systems topics, and advocating for policy change. Students attributed systems-related aspirations to experiences beyond HSS curricula (e.g., low-income background; work or volunteer experience; undergraduate studies; exposure to systems challenges affecting patients; supportive classmates, faculty, and institutional culture). Students also described future-oriented considerations promoting or undermining identification with systems-related roles (responsibility, affinity, ability, efficacy, priority, reality, consequences). CONCLUSIONS: This study illuminates systems-related roles medical students at 4 schools with HSS curricula envisioned as part of their future physician identity and highlights past/present experiences and future-oriented considerations shaping identification with such roles. These findings inform practical strategies to support professional identity formation inclusive of systems engagement.

Beyond Checking Boxes: Exploring Tensions With Use of a Workplace-Based Assessment Tool for Formative Assessment in Clerkships.

PURPOSE: To understand the role of a workplace-based assessment (WBA) tool in facilitating feedback for medical students, this study explored changes and tensions in a clerkship feedback activity system through the lens of cultural historical activity theory (CHAT) over 2 years of tool implementation. METHOD: This qualitative study uses CHAT to explore WBA use in core clerkships by identifying feedback activity system elements (e.g., community, tools, rules, objects) and tensions among these elements. University of California, San Francisco core clerkship students were invited to participate in semistructured interviews eliciting experience with a WBA tool intended to enhance direct observation and feedback in year 1 (2019) and year 2 (2020) of implementation. In year 1, the WBA tool required supervisor completion in the school's evaluation system on a computer. In year 2, both students and supervisors had WBA completion abilities and could access the form via a smartphone separate from the school's evaluation system. RESULTS: Thirty-five students participated in interviews. The authors identified tensions that shifted with time and tool iterations. Year 1 students described tensions related to cumbersome tool design, fear of burdening supervisors, confusion over WBA purpose, WBA as checking boxes, and WBA usefulness depending on clerkship context and culture. Students perceived dissatisfaction with the year 1 tool version among peers and supervisors. The year 2 mobile-based tool and student completion capabilities helped to reduce many of the tensions noted in year 1. Students expressed wider WBA acceptance among peers and supervisors in year 2 and reported understanding WBA to be for low-stakes feedback, thereby supporting formative assessment for learning. CONCLUSIONS: Using CHAT to explore changes in a feedback activity system with WBA tool iterations revealed elements important to WBA implementation, including designing technology for tool efficiency and affording students autonomy to document feedback with WBAs.

Are You Thinking What I'm Thinking? Exploring Response Process Validity Evidence for a Workplace-based Assessment for Operative Feedback.

OBJECTIVE: Workplace-based assessments (WBAs) are used in multiple surgical specialties to facilitate feedback to residents as a form of formative assessment. The validity evidence to support this purpose is limited and has yet to include investigations of how users interpret the assessment and make rating decisions (response processes). This study aimed to explore the validity evidence based on response processes for a WBA in surgery. DESIGN: Semi-structured interviews explored the reasonings and strategies used when answering questions in a surgical WBA, the System for Improving and Measuring Procedural Learning (SIMPL). Interview questions probed the interpretation of the three assessment questions and their respective answer categories (level of autonomy, operative performance, case complexity). Researchers analyzed transcripts using directed qualitative content analysis to generate themes. SETTING: Single tertiary academic medical center. PARTICIPANTS: Eight residents and 13 faculty within the Department of Otolaryngology-Head and Neck Surgery participating in a 6-month pilot of SIMPL. RESULTS: We identified four overarching themes that that characterized faculty and resident response processes while completing SIMPL: (1) Faculty and resident users had similar content-level interpretations of the questions and corresponding answer choices; (2) Users employed a variety of cognitive, behavioral, and emotional processes to make rating decisions; (3) Contextual factors influenced ratings; and (4) Tensions during interpretation contributed to rating uncertainty. CONCLUSIONS: Response processes are a key source of evidence to support the validity for the formative use of WBAs. Evaluating response process evidence should go beyond basic content-level analysis as contextual factors and tensions that arise during interpretation also play a large role in rating decisions. Additional work and a continued critical lens are needed to ensure that WBAs can truly meet the needs for formative assessment.

Exploring perspectives on health professions education scholarship units from sub-Saharan Africa.

INTRODUCTION: There has been a marked increase in institutional structures developed to support health professions education scholarship recently. These health professions education scholarship units (HPESUs) engage in a diverse range of activities. Previous work provided insight into factors that influence the functioning of such units, but data from European, Asian, Latin American, and African contexts was absent, potentially leading to a single world-view informing international standards for HPESUs. This aim of this study was to explore perspectives from sub-Saharan Africa (SSA) in response to this omission. METHODS: Situated within an interpretivist paradigm, the research team conducted semi-structured interviews with nine HPESU leaders in SSA, exploring how participants experienced and understood the functioning of their units. Despite efforts to have representation from across the region, most participants were from South Africa. The researchers analysed data thematically using the theory of institutional logics as an analytical frame. RESULTS: Several aspects of the HPESUs aligned with the previously identified logics of academic research, service and teaching; and of a cohesive education continuum. By contrast, leaders described financial sustainability as a more prominent logic than financial accountability. DISCUSSION: The similarities identified in this study may reflect isomorphism-a process which sees institutions within a similar field becoming more alike, particularly as newer institutions seek to acquire legitimacy within that field. An important caveat, however, is that isomorphism tends to occur across similar institutional contexts, which was not the case in this study. Understanding these differences is key as these HPESUs move to foster scholarship that can respond to the region's unique context.

Boyer and Beyond: An Interview Study of Health Professions Education Scholarship Units in the United States and a Synthetic Framework for Scholarship at the Unit Level.

PURPOSE: Health professions education scholarship units (HPESUs) in the United States are large in number and diverse in purpose, activities, and contributions. Although each of these units shares a commitment to scholarship, there is no synthetic framework to accurately represent and evaluate their activities and contributions. This study aimed to provide such a framework. METHOD: The authors examined data collected from 11 U.S. HPESU directors. Interviews occurred between April 2015 and February 2016. The research team used a combination of deductive and inductive qualitative techniques to analyze the interview transcripts. The deductive portion drew on Boyer's four-part framework of scholarship; the inductive portion produced a new conceptualization of scholarship at the HPESU level. RESULTS: The scholarly activities of HPESUs generally align with Boyer's four types of scholarship-discovery, integration, application, and teaching. However, this categorization fails to capture the interconnectedness and variety of purposes served by these activities. Both are important when considering how best to represent the scholarly contributions made by HPESUs. From their analysis of interviews, the authors developed a three-part framework characterizing HPESU scholarly activities: supporting a scholarly approach to education, supporting educational scholarship within the institution, and supporting HPESU members' scholarship. CONCLUSIONS: The authors contend that the three-part, unit-level framework for scholarship constructed in this study brings clarity and understanding to the purpose, activities, and contributions made by HPESUs in the United States. The proposed framework may allow unit directors to better justify and advocate for the resources needed to further promote the work of HPESUs.

Commentary: Racism and Bias in Health Professions Education: How Educators, Faculty Developers, and Researchers Can Make a Difference.

The Research in Medical Education (RIME) Program Planning Committee is committed to advancing scholarship in and promoting dialogue about the critical issues of racism and bias in health professions education (HPE). From the call for studies focused on underrepresented learners and faculty in medicine to the invited 2016 RIME plenary address by Dr. Camara Jones, the committee strongly believes that dismantling racism is critical to the future of HPE.The evidence is glaring: Dramatic racial and ethnic health disparities persist in the United States, people of color remain deeply underrepresented in medical school and academic health systems as faculty, learner experiences across the medical education continuum are fraught with bias, and current approaches to teaching perpetuate stereotypes and insufficiently challenge structural inequities. To achieve racial justice in HPE, academic medicine must commit to leveraging positions of influence and contributing from these positions. In this Commentary, the authors consider three roles (educator, faculty developer, and researcher) represented by the community of scholars and pose potential research questions as well as suggestions for advancing educational research relevant to eliminating racism and bias in HPE.

Exploring the institutional logics of health professions education scholarship units.

CONTEXT: Although health professions education scholarship units (HPESUs) share a commitment to the production and dissemination of rigorous educational practices and research, they are situated in many different contexts and have a wide range of structures and functions. OBJECTIVES: In this study, the authors explore the institutional logics common across HPESUs, and how these logics influence the organisation and activities of HPESUs. METHODS: The authors analysed interviews with HPESU leaders in Canada (n = 12), Australia (n = 21), New Zealand (n = 3) and the USA (n = 11). Using an iterative process, they engaged in inductive and deductive analyses to identify institutional logics across all participating HPESUs. They explored the contextual factors that influence how these institutional logics impact each HPESU's structure and function. RESULTS: Participants identified three institutional logics influencing the organisational structure and functions of an HPESU: (i) the logic of financial accountability; (ii) the logic of a cohesive education continuum, and (iii) the logic of academic research, service and teaching. Although most HPESUs embodied all three logics, the power of the logics varied among units. The relative power of each logic influenced leaders' decisions about how members of the unit allocate their time, and what kinds of scholarly contribution and product are valued by the HPESU. CONCLUSIONS: Identifying the configuration of these three logics within and across HPESUs provides insights into the reasons why individual units are structured and function in particular ways. Having a common language in which to discuss these logics can enhance transparency, facilitate evaluation, and help leaders select appropriate indicators of HPESU success.

Health Professions Education Scholarship Unit Leaders as Institutional Entrepreneurs.

PURPOSE: Health professions education scholarship units (HPESUs) are organizational structures within which a group is substantively engaged in health professions education scholarship. Little research investigates the strategies employed by HPESU administrative leaders to secure and maintain HPESU success. Using institutional entrepreneurship as a theoretical lens, this study asks: Do HPESU administrative leaders act as institutional entrepreneurs (IEs)? METHOD: This study recontextualizes two preexisting qualitative datasets that comprised interviews with leaders in health professions education in Canada (2011-2012) and Australia and New Zealand (2013-1014). Two researchers iteratively analyzed the data using the institutional entrepreneurship construct until consensus was achieved. A third investigator independently reviewed and contributed to the recontextualized analyses. A summary of the analyses was shared with all authors, and their feedback was incorporated into the final interpretations. RESULTS: HPESU leaders act as IEs in three ways. First, HPESU leaders construct arguments and position statements about how the HPESU resolves an institution's problem(s). This theorization discourse justifies the existence and support of the HPESU. Second, the leaders strategically cultivate relationships with the leader of the institution within which the HPESU sits, the leaders of large academic groups with which the HPESU partners, and the clinician educators who want careers in health professions education. Third, the leaders work to increase the local visibility of the HPESU. CONCLUSIONS: Practical insights into how institutional leaders interested in launching an HPESU can harness these findings are discussed.

Assessment of biological responses of EpiAirway 3-D cell constructs versus A549 cells for determining toxicity of ambient air pollution.

CONTEXT: EpiAirway™ 3-D constructs are human-derived cell cultures of differentiated airway epithelial cells that may represent a more biologically relevant model of the human lung. However, limited information is available on their utility for exposures to air pollutants at the air-liquid interface (ALI). OBJECTIVE: To assess the biological responses of EpiAirway™ cells in comparison to the responses of A549 human alveolar epithelial cells after exposure to air pollutants at ALI. METHODS: Cells were exposed to filtered air, 400 ppb of ozone (O3) or a photochemically aged Synthetic Urban Mixture (SynUrb54) consisting of hydrocarbons, nitrogen oxides, O3 and other secondary oxidation products for 4 h. Basolateral supernatants and apical washes were collected at 9 and 24 h post-exposure. We assessed cytotoxicity by measuring lactate dehydrogenase (LDH) release into the culture medium and apical surface. Interleukin 6 (IL-6) and interleukin 8 (IL-8) proteins were measured in the culture medium and in the apical washes to determine the inflammatory response after exposure. RESULTS: Both O3 and SynUrb54 significantly increased basolateral levels of LDH and IL-8 in A549 cells. No significant changes in LDH and IL-8 levels were observed in the EpiAirway™ cells, however, IL-6 in the apical surface was significantly elevated at 24 h after O3 exposure. CONCLUSION: LDH and IL-8 are robust endpoints for assessing toxicity in A549 cells. The EpiAirway™ cells show minimal adverse effects after exposure suggesting that they are more toxicologically resistant compared to A549 cells. Higher concentrations or longer exposure times are needed to induce effects on EpiAirway™ cells.

Understanding self-assessment as an informed process: residents' use of external information for self-assessment of performance in simulated resuscitations.

Self-directed learning requires self-assessment of learning needs and performance, a complex process that requires collecting and interpreting data from various sources. Learners' approaches to self-assessment likely vary depending on the learner and the context. The aim of this study was to gain insight into how learners process external information and apply their interpretation of this information to their self-assessment and learning during a structured educational activity. The study combined quantitative performance data with qualitative interview data. Pediatric residents led video-recorded simulated resuscitations and rated their crisis resource management skills on a validated 6-item instrument. Three independent observers rated the videos using the same instrument. During semi-structured interviews, each resident reviewed the video, rerated performance, discussed the self-assessment process, and interpreted feedback and observer scores. Transcripts were analyzed for themes. Sixteen residents participated. Residents' self-assessed scores ranged widely but usually fell within two points of the observers. They almost universally lowered their scores when self-assessing after the video review. Five major themes emerged from qualitative analysis of their interviews: (1) residents found self-assessment important and useful in certain contexts and conditions; (2) residents varied in their self-directed learning behaviors after the simulated resuscitation; (3) quantitative observer assessment had limited usefulness; (4) video review was difficult but useful; and (5) residents focused on their weaknesses and felt a need for constructive feedback to enhance learning. The residents in our study almost uniformly embraced the importance of self-assessment for all medical professionals. Even though video review had a negative impact on their self-assessment scores and was perceived as painful, residents saw this as the most useful aspect of the study exercises residents. They were less accepting of the quantitative assessment by observers. Residents explained their tendency to focus on weaknesses as a way to create an incentive for learning, demonstrating that self-assessment is closely linked to self-directed learning. How learners can use video review and external assessment most effectively to guide their self-directed learning deserves further study.

Integration of 3D imaging data in the assessment of aortic stenosis: impact on classification of disease severity.

BACKGROUND: In patients with aortic stenosis (AS), precise assessment of severity is critical for treatment decisions. Estimation of aortic valve area (AVA) with transthoracic echocardiographic (TTE)-continuity equation (CE) assumes a circular left ventricular outflow tract (LVOT). We evaluated incremental utility of 3D multidetector computed tomography (MDCT) over TTE assessment of AS severity. METHODS AND RESULTS: We included 51 patients (age, 81±8 years; 61% men; mean gradient, 42 ± 12 mm Hg) with calcific AS who underwent evaluation for treatment options. TTE parameters included systolic LVOT diameter (D) and continuous and pulsed wave (CW and PW) velocity-time integrals (VTI) through the LVOT and mean transaortic gradient. MDCT parameters included systolic LVOT area, ratio of maximal to minimal LVOT diameter (eccentricity index), and aortic planimetry (AVA(p)). TTE-CE AVA [(D(2)×0.786×VTIpw)/VTIcw] and dimensionless index (DI) [VTIpw/VTIcw] were calculated. Corrected AVA was calculated by substituting MDCT LVOT area into CE. The majority (96%) of patients had eccentric LVOT. LVOT area, measured on MDCT, was higher than on TTE (3.84 ± 0.8 cm(2) versus 3.03 ± 0.5 cm(2), P<0.01). TTE-AVA was smaller than AVA(p) and corrected AVA (0.67 ± 0.1cm(2), 0.82 ± 0.3 cm(2), and 0.86 ± 0.3 cm(2), P<0.01). Using TTE measurements alone, 73% of patients had congruence for severe AS (DI ≤0.25 and CE AVA <0.8 cm(2)), which increased to 92% using corrected CE. CONCLUSIONS: In patients with suspected severe AS, incorporation of MDCT-LVOT area into CE improves congruence for AS severity.

The assessment and management of cancer treatment-related diarrhea.

Cancer treatment-induced diarrhea affects a high percentage of patients with cancer that receive chemotherapy or radiation treatment. Widely used criteria for measuring treatment-induced diarrhea, such as the National Cancer Institute Common Toxicity Criteria, do not account for important characteristics of treatment-induced diarrhea. These characteristics include the assessment of the duration of the diarrhea, coexisting symptoms, abdominal cramping, or the presence of nocturnal diarrhea. Until recently, there were no universally accepted guidelines for the management of diarrhea. An expert panel developed guidelines with recommendations regarding assessment of the patient and treatment. These guidelines stress the importance of a thorough assessment of the patient, and treatment based upon severity of symptoms. By employing these guidelines, the aggressive management of diarrhea may impact the overall morbidity of this symptom. Education regarding the importance of diarrhea is essential. Patients who are informed will better understand their role in managing this side effect and when to contact their health care provider with emergent symptoms. Early recognition and management of diarrhea will be essential to improve control of diarrhea, and in turn will positively impact patients' quality of life.