Making Use of Natural Language Processing to Better Understand Medical Students' Self-Assessment of Clinical Skills.

PROBLEM: Reflective practice is necessary for self-regulated learning. Helping medical students develop these skills can be challenging since they are difficult to observe. One common solution is to assign students' reflective self-assessments, which produce large quantities of narrative assessment data. Reflective self-assessments also provide feedback to faculty regarding students' understanding of content, reflective abilities, and areas for course improvement. To maximize student learning and feedback to faculty, reflective self-assessments must be reviewed and analyzed, activities that are often difficult for faculty due to the time-intensive and cumbersome nature of processing large quantities of narrative assessment data. APPROACH: The authors collected narrative assessment data (2,224 students' reflective self-assessments) from 344 medical students' reflective self-assessments. In academic years 2019-2020 and 2021-2022, students at the University of Cincinnati College of Medicine responded to 2 prompts (aspects that surprised students, areas for student improvement) after reviewing their standardized patient encounters. These free-text entries were analyzed using TopEx, an open-source natural language processing (NLP) tool, to identify common topics and themes, which faculty then reviewed. OUTCOMES: TopEx expedited theme identification in students' reflective self-assessments, unveiling 10 themes for prompt 1 such as question organization and history analysis, and 8 for prompt 2, including sensitive histories and exam efficiency. Using TopEx offered a user-friendly, time-saving analysis method without requiring complex NLP implementations. The authors discerned 4 education enhancement implications: aggregating themes for future student reflection, revising self-assessments for common improvement areas, adjusting curriculum to guide students better, and aiding faculty in providing targeted upcoming feedback. NEXT STEPS: The University of Cincinnati College of Medicine aims to refine and expand the utilization of TopEx for deeper narrative assessment analysis, while other institutions may model or extend this approach to uncover broader educational insights and drive curricular advancements.

AI-Enabled Medical Education: Threads of Change, Promising Futures, and Risky Realities Across Four Potential Future Worlds.

BACKGROUND: The rapid trajectory of artificial intelligence (AI) development and advancement is quickly outpacing society's ability to determine its future role. As AI continues to transform various aspects of our lives, one critical question arises for medical education: what will be the nature of education, teaching, and learning in a future world where the acquisition, retention, and application of knowledge in the traditional sense are fundamentally altered by AI? OBJECTIVE: The purpose of this perspective is to plan for the intersection of health care and medical education in the future. METHODS: We used GPT-4 and scenario-based strategic planning techniques to craft 4 hypothetical future worlds influenced by AI's integration into health care and medical education. This method, used by organizations such as Shell and the Accreditation Council for Graduate Medical Education, assesses readiness for alternative futures and effectively manages uncertainty, risk, and opportunity. The detailed scenarios provide insights into potential environments the medical profession may face and lay the foundation for hypothesis generation and idea-building regarding responsible AI implementation. RESULTS: The following 4 worlds were created using OpenAI's GPT model: AI Harmony, AI conflict, The world of Ecological Balance, and Existential Risk. Risks include disinformation and misinformation, loss of privacy, widening inequity, erosion of human autonomy, and ethical dilemmas. Benefits involve improved efficiency, personalized interventions, enhanced collaboration, early detection, and accelerated research. CONCLUSIONS: To ensure responsible AI use, the authors suggest focusing on 3 key areas: developing a robust ethical framework, fostering interdisciplinary collaboration, and investing in education and training. A strong ethical framework emphasizes patient safety, privacy, and autonomy while promoting equity and inclusivity. Interdisciplinary collaboration encourages cooperation among various experts in developing and implementing AI technologies, ensuring that they address the complex needs and challenges in health care and medical education. Investing in education and training prepares professionals and trainees with necessary skills and knowledge to effectively use and critically evaluate AI technologies. The integration of AI in health care and medical education presents a critical juncture between transformative advancements and significant risks. By working together to address both immediate and long-term risks and consequences, we can ensure that AI integration leads to a more equitable, sustainable, and prosperous future for both health care and medical education. As we engage with AI technologies, our collective actions will ultimately determine the state of the future of health care and medical education to harness AI's power while ensuring the safety and well-being of humanity.

'Fragmented care': Asylum seekers' experience of accessing health care in NSW.

Medicare is central to accessing health care in Australia, yet many asylum seekers are ineligible for this scheme. In NSW, Medicare ineligible asylum seekers have some access to public health care under the Medicare Ineligible Asylum Seekers - Provision of Specified Public Health Services policy. This policy was updated in November 2020 to clarify services where a fee waiver applies for asylum seekers without Medicare. We examined the experiences of Medicare ineligible asylum seekers in accessing health care in New South Wales (NSW) in light of the revised policy. Employing qualitative methods, we conducted semi-structured interviews with asylum seekers (n = 7) and service providers (n = 6) in South Western Sydney. The interviews were analysed using thematic analysis and the socio-ecological model was used to interpret the data. Participants identified that chronic and mental health conditions are the main health issues for asylum seekers. Factors across the four levels of the socio-ecological model were identified as influencing the health care of asylum seekers, including a lack of awareness about health care rights at the individual level, support from relatives and friends at the interpersonal level, providers' lack of awareness of fee waivers at the organizational level and limited access to primary health care at the policy level. The results imply that Medicare ineligible asylum seekers in NSW do not have optimal access to health care which may worsen existing health disparities. Educational initiatives that improve service providers' and asylum seekers' awareness of the revised policy are needed to improve asylum seeker health equity in NSW.

Reconstructing Neurath's Ship: A Case Study in Reevaluating Equity in a Program of Assessment.

Inequity in assessment has been described as a "wicked problem"-an issue with complex roots, inherent tensions, and unclear solutions. To address inequity, health professions educators must critically examine their implicit understandings of truth and knowledge (i.e., their epistemologies) with regard to educational assessment before jumping to solutions. The authors use the analogy of a ship (program of assessment) sailing on different seas (epistemologies) to describe their journey in seeking to improve equity in assessment. Should the education community repair the ship of assessment while sailing or should the ship be scrapped and built anew? The authors share a case study of a well-developed internal medicine residency program of assessment and describe efforts to evaluate and enable equity using various epistemological lenses. They first used a postpositivist lens to evaluate if the systems and strategies aligned with best practices, but found they did not capture important nuances of what equitable assessment entails. Next, they used a constructivist approach to improve stakeholder engagement, but found they still failed to question the inequitable assumptions inherent to their systems and strategies. Finally, they describe a shift to critical epistemologies, seeking to understand who experiences inequity and harm to dismantle inequitable systems and create better ones. The authors describe how each unique sea promoted different adaptations to their ship, and challenge programs to sail through new epistemological waters as a starting point for making their own ships more equitable.

Interpersonal stress and nonsuicidal self-injury disorder in veterans: An ecological momentary assessment study.

INTRO: Nonsuicidal self-injury (NSSI) is associated with marked functional impairment and is a robust predictor of suicide attempts. Prevalence rates of NSSI, and self-directed violence more broadly, are elevated among military veterans. Despite the inclusion of interpersonal difficulty in the diagnostic criteria for NSSI disorder, the relationship between interpersonal risk factors and NSSI is not well-characterized, especially among veterans. This ecological momentary assessment (EMA) study investigated the hypothesis that interpersonal stressors and associated distress would precede and predict NSSI urge and engagement-but not vice versa-via cross-lagged multilevel modeling. METHOD: Forty veterans with NSSI disorder completed a 28-day EMA protocol with three daily prompts assessing NSSI urges, NSSI engagement, the occurrence of interpersonal stressors, and associated subjective interpersonal distress. RESULTS: Interpersonal stressors preceded and predicted subsequent NSSI urges, but not NSSI engagement, whereas subjective interpersonal distress preceded and predicted both NSSI urges and NSSI engagement. CONCLUSION: Results identified interpersonal stressors as a risk factor for NSSI urges, and interpersonal distress as a risk factor for both NSSI urges and NSSI engagement. Findings highlight the importance of temporally assessing interpersonal factors related to NSSI and suggest that interpersonal distress may be a modifiable risk factor for NSSI.

Warnings in early narrative assessment that might predict performance in residency: signal from an internal medicine residency program.

INTRODUCTION: Narrative assessment data are valuable in understanding struggles in resident performance. However, it remains unknown which themes in narrative data that occur early in training may indicate a higher likelihood of struggles later in training, allowing programs to intervene sooner. METHODS: Using learning analytics, we identified 26 internal medicine residents in three cohorts that were below expected entrustment during training. We compiled all narrative data in the first 6 months of training for these residents as well as 13 typically performing residents for comparison. Narrative data were blinded for all 39 residents during initial phases of an inductive thematic analysis for initial coding. RESULTS: Many similarities were identified between the two cohorts. Codes that differed between typical and lower entrusted residents were grouped into two types of themes: three explicit/manifest and three implicit/latent with six total themes. The explicit/manifest themes focused on specific aspects of resident performance with assessors describing 1) Gaps in attention to detail, 2) Communication deficits with patients, and 3) Difficulty recognizing the "big picture" in patient care. Three implicit/latent themes, focused on how narrative data were written, were also identified: 1) Feedback described as a deficiency rather than an opportunity to improve, 2) Normative comparisons to identify a resident as being behind their peers, and 3) Warning of possible risk to patient care. DISCUSSION: Clinical competency committees (CCCs) usually rely on accumulated data and trends. Using the themes in this paper while reviewing narrative comments may help CCCs with earlier recognition and better allocation of resources to support residents' development.

Supported playgroups as a setting for promoting physical activity of young children: findings from a feasibility study in south-west Sydney, Australia.

AIM: This study examined the feasibility of the supported playgroups (SPGs) setting to promote physical activity among young children and increase staff and parental knowledge of the physical activity recommendations for young children. METHODS: A 10-week Active Play@Playgroup intervention was conducted with 28 co-ordinators and 174 parents attending SPGs located in south-west Sydney. The intervention involved training and providing support to SPG co-ordinators to help promote active play and reduce screen time for young children. The effectiveness of the intervention was evaluated using pre- and post-intervention surveys. The main outcomes included changes in parents' knowledge of physical activity recommendations for 0- to 5-year-olds, and children's physical activity and sedentary behaviours outside of playgroup. RESULTS: The mean time children spent playing actively outdoors during the week increased (pre: 1.48 h/day vs. post: 1.95 h/day, P = 0.02), and the time parents spent playing actively with their children increased both during weekdays (pre: 1.57 h/day vs. post: 2.42 h/day, P = 0.01) and on the weekend (pre: 2.95 h/day vs. post: 3.83 h/day, P = 0.02). There was also a significant increase in the proportion of parents with correct knowledge of recommended screen time for 0- to <2-year-olds (pre: 7.9% vs. post: 23.4%, P = 0.02). CONCLUSIONS: It is feasible and potentially effective to use the SPG setting to promote active play among young children and increase parents' knowledge of physical activity recommendations for young children.

Understanding statements now a virtual cinch.

With a click of the mouse, some patients are accessing and paying their hospital bills online. Novant Health revamped its patient billing process so it's easier to understand and use. Developing a clear, concise billing statement and then implementing an online bill presentment and payment system resulted in improved customer relations, fewer payment processing errors, and faster receipt of payment.