The TRIPOD-LLM Statement: A Targeted Guideline For Reporting Large Language Models Use.

Large Language Models (LLMs) are rapidly being adopted in healthcare, necessitating standardized reporting guidelines. We present TRIPOD-LLM, an extension of the TRIPOD+AI statement, addressing the unique challenges of LLMs in biomedical applications. TRIPOD-LLM provides a comprehensive checklist of 19 main items and 50 subitems, covering key aspects from title to discussion. The guidelines introduce a modular format accommodating various LLM research designs and tasks, with 14 main items and 32 subitems applicable across all categories. Developed through an expedited Delphi process and expert consensus, TRIPOD-LLM emphasizes transparency, human oversight, and task-specific performance reporting. We also introduce an interactive website ( https://tripod-llm.vercel.app/ ) facilitating easy guideline completion and PDF generation for submission. As a living document, TRIPOD-LLM will evolve with the field, aiming to enhance the quality, reproducibility, and clinical applicability of LLM research in healthcare through comprehensive reporting. COI: DSB: Editorial, unrelated to this work: Associate Editor of Radiation Oncology, HemOnc.org (no financial compensation); Research funding, unrelated to this work: American Association for Cancer Research; Advisory and consulting, unrelated to this work: MercurialAI. DDF: Editorial, unrelated to this work: Associate Editor of JAMIA, Editorial Board of Scientific Data, Nature; Funding, unrelated to this work: the intramural research program at the U.S. National Library of Medicine, National Institutes of Health. JWG: Editorial, unrelated to this work: Editorial Board of Radiology: Artificial Intelligence, British Journal of Radiology AI journal and NEJM AI. All other authors declare no conflicts of interest.

Mistargeted retinal axons form synaptically segregated subcircuits in the visual thalamus of albino mice.

A Hebbian model of circuit remodeling predicts that two sets of inputs with sufficiently distinct activity patterns will synaptically capture separate sets of target cells. Mice in which a subset of retinal ganglion cells (RGCs) target the wrong region of the dorsal lateral geniculate nucleus (dLGN) provide the conditions for testing this prediction. In albino mice, mistargeted RGC axons form an island of terminals that is distinct from the surrounding neuropil. Blocking retinal activity during development prevents the formation of this island. However, the synaptic connectivity of the island was unknown. Here, we combine light and electron microscopy to determine if this activity-dependent island of axon terminals represent a synaptically segregated subcircuit. We reconstructed the microcircuitry of the boundary between the island and non-island RGCs and found a remarkably strong segregation within retinogeniculate connectivity. We conclude that, when sets of retinal input are established in the wrong part of the dLGN, the developing circuitry responds by forming a synaptically isolated subcircuit from the otherwise fully connected network. The fact that there is a developmental starting condition that can induce a synaptically segregated microcircuit has important implications for our understanding of the organization of visual circuits and for our understanding of the implementation of activity dependent development.

Does diversity beget diversity? A scientometric analysis of over 150,000 studies and 49,000 authors published in high-impact medical journals between 2007 and 2022.

Background: Health research that significantly impacts global clinical practice and policy is often published in high-impact factor (IF) medical journals. These outlets play a pivotal role in the worldwide dissemination of novel medical knowledge. However, researchers identifying as women and those affiliated with institutions in low- and middle-income countries (LMIC) have been largely underrepresented in high-IF journals across multiple fields of medicine. To evaluate disparities in gender and geographical representation among authors who have published in any of five top general medical journals, we conducted scientometric analyses using a large-scale dataset extracted from the New England Journal of Medicine (NEJM), Journal of the American Medical Association (JAMA), The British Medical Journal (BMJ), The Lancet, and Nature Medicine. Methods: Author metadata from all articles published in the selected journals between 2007 and 2022 were collected using the DimensionsAI platform. The Genderize.io API was then utilized to infer each author's likely gender based on their extracted first name. The World Bank country classification was used to map countries associated with researcher affiliations to the LMIC or the high-income country (HIC) category. We characterized the overall gender and country income category representation across the medical journals. In addition, we computed article-level diversity metrics and contrasted their distributions across the journals. Findings: We studied 151,536 authors across 49,764 articles published in five top medical journals, over a long period spanning 15 years. On average, approximately one-third (33.1%) of the authors of a given paper were inferred to be women; this result was consistent across the journals we studied. Further, 86.6% of the teams were exclusively composed of HIC authors; in contrast, only 3.9% were exclusively composed of LMIC authors. The probability of serving as the first or last author was significantly higher if the author was inferred to be a man (18.1% vs 16.8%, P < .01) or was affiliated with an institution in a HIC (16.9% vs 15.5%, P < .01). Our primary finding reveals that having a diverse team promotes further diversity, within the same dimension (i.e., gender or geography) and across dimensions. Notably, papers with at least one woman among the authors were more likely to also involve at least two LMIC authors (11.7% versus 10.4% in baseline, P < .001; based on inferred gender); conversely, papers with at least one LMIC author were more likely to also involve at least two women (49.4% versus 37.6%, P < .001; based on inferred gender). Conclusion: We provide a scientometric framework to assess authorship diversity. Our research suggests that the inclusiveness of high-impact medical journals is limited in terms of both gender and geography. We advocate for medical journals to adopt policies and practices that promote greater diversity and collaborative research. In addition, our findings offer a first step towards understanding the composition of teams conducting medical research globally and an opportunity for individual authors to reflect on their own collaborative research practices and possibilities to cultivate more diverse partnerships in their work.

Machine learning for healthcare that matters: Reorienting from technical novelty to equitable impact.

Despite significant technical advances in machine learning (ML) over the past several years, the tangible impact of this technology in healthcare has been limited. This is due not only to the particular complexities of healthcare, but also due to structural issues in the machine learning for healthcare (MLHC) community which broadly reward technical novelty over tangible, equitable impact. We structure our work as a healthcare-focused echo of the 2012 paper "Machine Learning that Matters", which highlighted such structural issues in the ML community at large, and offered a series of clearly defined "Impact Challenges" to which the field should orient itself. Drawing on the expertise of a diverse and international group of authors, we engage in a narrative review and examine issues in the research background environment, training processes, evaluation metrics, and deployment protocols which act to limit the real-world applicability of MLHC. Broadly, we seek to distinguish between machine learning ON healthcare data and machine learning FOR healthcare-the former of which sees healthcare as merely a source of interesting technical challenges, and the latter of which regards ML as a tool in service of meeting tangible clinical needs. We offer specific recommendations for a series of stakeholders in the field, from ML researchers and clinicians, to the institutions in which they work, and the governments which regulate their data access.

Subcellular pathways through VGluT3-expressing mouse amacrine cells provide locally tuned object-motion-selective signals in the retina.

VGluT3-expressing mouse retinal amacrine cells (VG3s) respond to small-object motion and connect to multiple types of bipolar cells (inputs) and retinal ganglion cells (RGCs, outputs). Because these input and output connections are intermixed on the same dendrites, making sense of VG3 circuitry requires comparing the distribution of synapses across their arbors to the subcellular flow of signals. Here, we combine subcellular calcium imaging and electron microscopic connectomic reconstruction to analyze how VG3s integrate and transmit visual information. VG3s receive inputs from all nearby bipolar cell types but exhibit a strong preference for the fast type 3a bipolar cells. By comparing input distributions to VG3 dendrite responses, we show that VG3 dendrites have a short functional length constant that likely depends on inhibitory shunting. This model predicts that RGCs that extend dendrites into the middle layers of the inner plexiform encounter VG3 dendrites whose responses vary according to the local bipolar cell response type.

Peer review of GPT-4 technical report and systems card.

The study provides a comprehensive review of OpenAI's Generative Pre-trained Transformer 4 (GPT-4) technical report, with an emphasis on applications in high-risk settings like healthcare. A diverse team, including experts in artificial intelligence (AI), natural language processing, public health, law, policy, social science, healthcare research, and bioethics, analyzed the report against established peer review guidelines. The GPT-4 report shows a significant commitment to transparent AI research, particularly in creating a systems card for risk assessment and mitigation. However, it reveals limitations such as restricted access to training data, inadequate confidence and uncertainty estimations, and concerns over privacy and intellectual property rights. Key strengths identified include the considerable time and economic investment in transparent AI research and the creation of a comprehensive systems card. On the other hand, the lack of clarity in training processes and data raises concerns about encoded biases and interests in GPT-4. The report also lacks confidence and uncertainty estimations, crucial in high-risk areas like healthcare, and fails to address potential privacy and intellectual property issues. Furthermore, this study emphasizes the need for diverse, global involvement in developing and evaluating large language models (LLMs) to ensure broad societal benefits and mitigate risks. The paper presents recommendations such as improving data transparency, developing accountability frameworks, establishing confidence standards for LLM outputs in high-risk settings, and enhancing industry research review processes. It concludes that while GPT-4's report is a step towards open discussions on LLMs, more extensive interdisciplinary reviews are essential for addressing bias, harm, and risk concerns, especially in high-risk domains. The review aims to expand the understanding of LLMs in general and highlights the need for new reflection forms on how LLMs are reviewed, the data required for effective evaluation, and addressing critical issues like bias and risk.

Steering clear of Akrasia: An integrative review of self-binding Ulysses Contracts in clinical practice.

In many jurisdictions, legal frameworks afford patients the opportunity to make prospective medical decisions or to create directives that contain a special provision forfeiting their own ability to object to those decisions at a future time point, should they lose decision-making capacity. These agreements have been described with widely varying nomenclatures, including Ulysses Contracts, Odysseus Transfers, Psychiatric Advance Directives with Ulysses Clauses, and Powers of Attorney with Special Provisions. As a consequence of this terminological heterogeneity, it is challenging for healthcare providers to understand the terms and uses of these agreements and for ethicists to engage with the nuances of clinical decision-making with such unique provisions surrounding patient autonomy. In theory, prospective self-binding agreements may safeguard patient's "authentic" wishes from future "inauthentic" changes of mind. In practice, it is unclear what may be comprised within these agreements or how-and to what effect-they are used. The primary focus of this integrative review is to curate the existing literature describing Ulysses Contracts (and analogous decisions) used in the clinical arena, in order to empirically synthesize their shared essence and provide insights into the traditional components of these agreements when used in practice, the requirements of their consent processes, and the outcomes of their utilization.

Developing and Validating a Prediction Model For Death or Critical Illness in Hospitalized Adults, an Opportunity for Human-Computer Collaboration.

Hospital early warning systems that use machine learning (ML) to predict clinical deterioration are increasingly being used to aid clinical decision-making. However, it is not known how ML predictions complement physician and nurse judgment. Our objective was to train and validate a ML model to predict patient deterioration and compare model predictions with real-world physician and nurse predictions. DESIGN: Retrospective and prospective cohort study. SETTING: Academic tertiary care hospital. PATIENTS: Adult general internal medicine hospitalizations. MEASUREMENTS AND MAIN RESULTS: We developed and validated a neural network model to predict in-hospital death and ICU admission in 23,528 hospitalizations between April 2011 and April 2019. We then compared model predictions with 3,374 prospectively collected predictions from nurses, residents, and attending physicians about their own patients in 960 hospitalizations between April 30, and August 28, 2019. ML model predictions achieved clinician-level accuracy for predicting ICU admission or death (ML median F1 score 0.32 [interquartile range (IQR) 0.30-0.34], AUC 0.77 [IQ 0.76-0.78]; clinicians median F1-score 0.33 [IQR 0.30-0.35], AUC 0.64 [IQR 0.63-0.66]). ML predictions were more accurate than clinicians for ICU admission. Of all ICU admissions and deaths, 36% occurred in hospitalizations where the model and clinicians disagreed. Combining human and model predictions detected 49% of clinical deterioration events, improving sensitivity by 16% compared with clinicians alone and 24% compared with the model alone while maintaining a positive predictive value of 33%, thus keeping false alarms at a clinically acceptable level. CONCLUSIONS: ML models can complement clinician judgment to predict clinical deterioration in hospital. These findings demonstrate important opportunities for human-computer collaboration to improve prognostication and personalized medicine in hospital.

Underrecognition of Dysmenorrhea Is an Iatrogenic Harm.

Many patients face years of recurrent and debilitating menstrual pain that affects their ability to work and study. Patients often normalize their severe pain as an expected part of menses. Both underrecognition and lack of awareness of available therapies for this remediable condition serve as a quintessential example of hermeneutic injustice. Hermeneutic injustice describes a structural lack of access to epistemic resources, such as shared concepts and knowledge. Pervasive menstrual stigma further discourages people with dysmenorrhea from discussing their symptoms and seeking health care. A lack of respect for women's experiences of pain in clinical encounters acts to worsen these issues and should be considered a source of iatrogenic harm. Health care workers can promote hermeneutic justice by preemptively destigmatizing discussions about menstruation and validating patients' concerns. On a systemic level, there should be greater awareness of dysmenorrhea and the various treatments availabe for it.

Large scale genotype- and phenotype-driven machine learning in Von Hippel-Lindau disease.

Von Hippel-Lindau (VHL) disease is a hereditary cancer syndrome where individuals are predisposed to tumor development in the brain, adrenal gland, kidney, and other organs. It is caused by pathogenic variants in the VHL tumor suppressor gene. Standardized disease information has been difficult to collect due to the rarity and diversity of VHL patients. Over 4100 unique articles published until October 2019 were screened for germline genotype-phenotype data. Patient data were translated into standardized descriptions using Human Genome Variation Society gene variant nomenclature and Human Phenotype Ontology terms and has been manually curated into an open-access knowledgebase called Clinical Interpretation of Variants in Cancer. In total, 634 unique VHL variants, 2882 patients, and 1991 families from 427 papers were captured. We identified relationship trends between phenotype and genotype data using classic statistical methods and spectral clustering unsupervised learning. Our analyses reveal earlier onset of pheochromocytoma/paraganglioma and retinal angiomas, phenotype co-occurrences and genotype-phenotype correlations including hotspots. It confirms existing VHL associations and can be used to identify new patterns and associations in VHL disease. Our database serves as an aggregate knowledge translation tool to facilitate sharing information about the pathogenicity of VHL variants.

An ethical analysis of clinical triage protocols and decision-making frameworks: what do the principles of justice, freedom, and a disability rights approach demand of us?

BACKGROUND: The expectation of pandemic-induced severe resource shortages has prompted authorities to draft and update frameworks to guide clinical decision-making and patient triage. While these documents differ in scope, they share a utilitarian focus on the maximization of benefit. This utilitarian view necessarily marginalizes certain groups, in particular individuals with increased medical needs. MAIN BODY: Here, we posit that engagement with the disability critique demands that we broaden our understandings of justice and fairness in clinical decision-making and patient triage. We propose the capabilities theory, which recognizes that justice requires a range of positive capabilities/freedoms conducive to the achievement of meaningful life goals, as a means to do so. Informed by a disability rights critique of the clinical response to the pandemic, we offer direction for the construction of future clinical triage protocols which will avoid ableist biases by incorporating a broader apprehension of what it means to be human. CONCLUSION: The clinical pandemic response, codified across triage protocols, should embrace a form of justice which incorporates a vision of pluralistic human capabilities and a valuing of positive freedoms.

Believing in black boxes: machine learning for healthcare does not need explainability to be evidence-based.

OBJECTIVE: To examine the role of explainability in machine learning for healthcare (MLHC), and its necessity and significance with respect to effective and ethical MLHC application. STUDY DESIGN AND SETTING: This commentary engages with the growing and dynamic corpus of literature on the use of MLHC and artificial intelligence (AI) in medicine, which provide the context for a focused narrative review of arguments presented in favour of and opposition to explainability in MLHC. RESULTS: We find that concerns regarding explainability are not limited to MLHC, but rather extend to numerous well-validated treatment interventions as well as to human clinical judgment itself. We examine the role of evidence-based medicine in evaluating inexplicable treatments and technologies, and highlight the analogy between the concept of explainability in MLHC and the related concept of mechanistic reasoning in evidence-based medicine. CONCLUSION: Ultimately, we conclude that the value of explainability in MLHC is not intrinsic, but is instead instrumental to achieving greater imperatives such as performance and trust. We caution against the uncompromising pursuit of explainability, and advocate instead for the development of robust empirical methods to successfully evaluate increasingly inexplicable algorithmic systems.