Assessing the Application of Large Language Models in Generating Dermatologic Patient Education Materials According to Reading Level: Qualitative Study.

BACKGROUND: Dermatologic patient education materials (PEMs) are often written above the national average seventh- to eighth-grade reading level. ChatGPT-3.5, GPT-4, DermGPT, and DocsGPT are large language models (LLMs) that are responsive to user prompts. Our project assesses their use in generating dermatologic PEMs at specified reading levels. OBJECTIVE: This study aims to assess the ability of select LLMs to generate PEMs for common and rare dermatologic conditions at unspecified and specified reading levels. Further, the study aims to assess the preservation of meaning across such LLM-generated PEMs, as assessed by dermatology resident trainees. METHODS: The Flesch-Kincaid reading level (FKRL) of current American Academy of Dermatology PEMs was evaluated for 4 common (atopic dermatitis, acne vulgaris, psoriasis, and herpes zoster) and 4 rare (epidermolysis bullosa, bullous pemphigoid, lamellar ichthyosis, and lichen planus) dermatologic conditions. We prompted ChatGPT-3.5, GPT-4, DermGPT, and DocsGPT to "Create a patient education handout about [condition] at a [FKRL]" to iteratively generate 10 PEMs per condition at unspecified fifth- and seventh-grade FKRLs, evaluated with Microsoft Word readability statistics. The preservation of meaning across LLMs was assessed by 2 dermatology resident trainees. RESULTS: The current American Academy of Dermatology PEMs had an average (SD) FKRL of 9.35 (1.26) and 9.50 (2.3) for common and rare diseases, respectively. For common diseases, the FKRLs of LLM-produced PEMs ranged between 9.8 and 11.21 (unspecified prompt), between 4.22 and 7.43 (fifth-grade prompt), and between 5.98 and 7.28 (seventh-grade prompt). For rare diseases, the FKRLs of LLM-produced PEMs ranged between 9.85 and 11.45 (unspecified prompt), between 4.22 and 7.43 (fifth-grade prompt), and between 5.98 and 7.28 (seventh-grade prompt). At the fifth-grade reading level, GPT-4 was better at producing PEMs for both common and rare conditions than ChatGPT-3.5 (P=.001 and P=.01, respectively), DermGPT (P<.001 and P=.03, respectively), and DocsGPT (P<.001 and P=.02, respectively). At the seventh-grade reading level, no significant difference was found between ChatGPT-3.5, GPT-4, DocsGPT, or DermGPT in producing PEMs for common conditions (all P>.05); however, for rare conditions, ChatGPT-3.5 and DocsGPT outperformed GPT-4 (P=.003 and P<.001, respectively). The preservation of meaning analysis revealed that for common conditions, DermGPT ranked the highest for overall ease of reading, patient understandability, and accuracy (14.75/15, 98%); for rare conditions, handouts generated by GPT-4 ranked the highest (14.5/15, 97%). CONCLUSIONS: GPT-4 appeared to outperform ChatGPT-3.5, DocsGPT, and DermGPT at the fifth-grade FKRL for both common and rare conditions, although both ChatGPT-3.5 and DocsGPT performed better than GPT-4 at the seventh-grade FKRL for rare conditions. LLM-produced PEMs may reliably meet seventh-grade FKRLs for select common and rare dermatologic conditions and are easy to read, understandable for patients, and mostly accurate. LLMs may play a role in enhancing health literacy and disseminating accessible, understandable PEMs in dermatology.

Evaluating Live Virtual Chalk Talks as a Teaching Tool for Medical Students on a Dermatology Clerkship.

The COVID-19 pandemic has highlighted a continued need for innovative virtual teaching methods. Chalk talks, or brief illustrated interactive talks, are easily made virtual with an online whiteboard. We evaluated the efficacy of a live virtual chalk talk curriculum for medical students on their dermatology clerkship. A curriculum of one to three 1-h chalk talks was designed on the following topics: papulosquamous diseases, erythroderma, and immunobullous diseases. Talks were delivered monthly via Zoom to dermatology clerkship students. Pre- and post-talk surveys were administered to assess knowledge, confidence, and satisfaction. Compared with pre-talk, students (n = 18) achieved a greater percentage of possible points on knowledge assessment questions after the talks (41.0 ± 27.7% versus 90.4 ± 18.4%, p < 0.001). As assessed on a Likert scale (1 = not at all confident, 5 = extremely confident), students became more confident in differentiating conditions within each disease group and working up the conditions (2.02 ± 0.53 versus 3.53 ± 0.55, p < 0.001, and 2.09 ± 0.44 versus 3.76 ± 0.89, p < 0.001, respectively). Qualitative responses showed that students appreciated the student-teacher interactions. In conclusion, we found that live chalk talks are an effective and engaging way to teach dermatology to medical students in the virtual setting. Supplementary Information: The online version contains supplementary material available at 10.1007/s40670-023-01781-4.

Mood changes and clinical decision making in adolescent patients on isotretinoin therapy for acne vulgaris.

Although numerous studies have demonstrated no causal relationship between isotretinoin and depression or suicide, subtle mood changes and idiosyncratic mood symptoms have been reported in patients on isotretinoin treatment for acne vulgaris, and few studies have described the full range of mood symptoms and clinical course after a mood change arises. We reviewed 247 patients, ages 10-25 years, with acne vulgaris on isotretinoin and found that 26/247 (10.5%) patients experienced mood changes, the most common being depressive symptoms, anxiety, aggression, and emotional lability. Regardless of treatment management, 22/25 (88%) patients experienced improvement of mood symptoms to baseline, and 22/25 (88%) were able to complete their isotretinoin course without symptom recurrence. Our findings highlight the importance of monitoring for a broad range of mood changes in patients on isotretinoin, especially those related to a pre-existing mood disorder and including those which do not meet formal criteria for a psychiatric disorder.

Preventing Readmissions Through Effective Partnerships-Communication and Palliative Care (PREP-CPC): A Multisite Intervention for Encouraging Goals of Care Conversations for Hospitalized Patients Facing Serious Illness.

BACKGROUND: Despite evidence showing that goals of care (GOC) conversations increase the likelihood that patients facing a serious illness receive care that is concordant with their wishes, only a minority of at-risk patients receive the opportunity to engage in such conversations. OBJECTIVE: The Preventing Readmissions through Effective Partnerships-Communication and Palliative Care (PREP-CPC) intervention was designed to increase the frequency of GOC conversations for hospitalized patients facing serious illness. METHODS: The PREP-CPC employed a sequential, multicohort design using a yearlong mentored implementation approach to support nonpalliative care health-care professionals at participating hospitals to implement quality improvement projects focused on GOC conversations. RESULTS: Over the 3-year study period, 134 clinicians from 29 hospital teams were trained to facilitate GOC conversations. After the kickoff conference, participants reported improvements in their confidence in facilitating GOC conversations. The hospital teams then instituted site-specific pilot interventions to promote GOC conversations, identifying essential elements required for ongoing improvement. Since projects varied by hospital, results did as well, but reported positive outcomes included increased GOC conversations, increased Practitioner Orders for Life-Sustaining Treatment form completion rates, new screening and documentation methods, and increased support from leadership. CONCLUSIONS: The PREP-CPC pilot successfully engaged a diverse set of hospitals to participate in quality improvement collaborative promoting primary palliative care and more frequent GOC conversations. This initiative revealed several lessons that should guide future interventions.