Inductive thematic analysis of healthcare qualitative interviews using open-source large language models: How does it compare to traditional methods?

BACKGROUND: Large language models (LLMs) are generative artificial intelligence that have ignited much interest and discussion about their utility in clinical and research settings. Despite this interest there is sparse analysis of their use in qualitative thematic analysis comparing their current ability to that of human coding and analysis. In addition, there has been no published analysis of their use in real-world, protected health information. OBJECTIVE: Here we fill that gap in the literature by comparing an LLM to standard human thematic analysis in real-world, semi-structured interviews of both patients and clinicians within a psychiatric setting. METHODS: Using a 70 billion parameter open-source LLM running on local hardware and advanced prompt engineering techniques, we produced themes that summarized a full corpus of interviews in minutes. Subsequently we used three different evaluation methods for quantifying similarity between themes produced by the LLM and those produced by humans. RESULTS: These revealed similarities ranging from moderate to substantial (Jaccard similarity coefficients 0.44-0.69), which are promising preliminary results. CONCLUSION: Our study demonstrates that open-source LLMs can effectively generate robust themes from qualitative data, achieving substantial similarity to human-generated themes. The validation of LLMs in thematic analysis, coupled with evaluation methodologies, highlights their potential to enhance and democratize qualitative research across diverse fields.

Digital Dialogue-How Youth Are Interacting With Chatbots.

This Viewpoint describes the use of large language model chatbots in social, educational, and therapeutic settings and the need to assess when children are developmentally ready to engage with them.

All a-BORG! The Dangerous Drinking Trend Steamrolling Across Social Media and College Campuses.

ABSTRACT: Black out rage gallons (BORGs) are a troublesome drinking pattern emerging on social media platforms. The prevalence of BORGs has been increasing on college campuses and is demonstrating significant consequences. There is no known research on BORGs in addiction treatment settings. We suggest that future research will be necessary to understand their implication. Troublesome binge drinking is not a new problem among colleges. However, social media has seemed to ignite such trends.