Investigating Cross-Domain Binary Relation Classification in Biomedical Natural Language Processing.

This paper addresses the challenge of binary relation classification in biomedical Natural Language Processing (NLP), focusing on diverse domains including gene-disease associations, compound protein interactions, and social determinants of health (SDOH). We evaluate different approaches, including fine-tuning Bidirectional Encoder Representations from Transformers (BERT) models and generative Large Language Models (LLMs), and examine their performance in zero and few-shot settings. We also introduce a novel dataset of biomedical text annotated with social and clinical entities to facilitate research into relation classification. Our results underscore the continued complexity of this task for both humans and models. BERT-based models trained on domain-specific data excelled in certain domains and achieved comparable performance and generalization power to generative LLMs in others. Despite these encouraging results, these models are still far from achieving human-level performance. We also highlight the significance of high-quality training data and domain-specific fine-tuning on the performance of all the considered models.

Subtyping Gastrointestinal Surgical Outcomes from Real World Data: A Comprehensive Analysis of UK Biobank.

Chronic gastrointestinal (GI) conditions, such as inflammatory bowel diseases (IBD), offer a promising opportunity to create classification systems that can enhance the accuracy of predicting the most effective therapies and prognosis for each patient. Here, we present a novel methodology to explore disease subtypes using our open-sourced BiomedSciAI toolkit. Applying methods available in this toolkit on the UK Biobank, including subpopulation-based feature selection and multi-dimensional subset scanning, we aimed to discover unique subgroups from GI surgery cohorts. Of a 12,073-patient cohort, a subgroup of 440 IBD patients was discovered with an increased risk of a subsequent GI surgery (OR: 2.21, 95% CI [1.81-2.69]). We iteratively demonstrate the discovery process using an additional cohort (with a narrower definition of GI surgery). Our results show that the iterative process can refine the subgroup discovery process and generate novel hypotheses to investigate determinants of treatment response.

Defining AMIA's artificial intelligence principles.

Recent advances in the science and technology of artificial intelligence (AI) and growing numbers of deployed AI systems in healthcare and other services have called attention to the need for ethical principles and governance. We define and provide a rationale for principles that should guide the commission, creation, implementation, maintenance, and retirement of AI systems as a foundation for governance throughout the lifecycle. Some principles are derived from the familiar requirements of practice and research in medicine and healthcare: beneficence, nonmaleficence, autonomy, and justice come first. A set of principles follow from the creation and engineering of AI systems: explainability of the technology in plain terms; interpretability, that is, plausible reasoning for decisions; fairness and absence of bias; dependability, including "safe failure"; provision of an audit trail for decisions; and active management of the knowledge base to remain up to date and sensitive to any changes in the environment. In organizational terms, the principles require benevolence-aiming to do good through the use of AI; transparency, ensuring that all assumptions and potential conflicts of interest are declared; and accountability, including active oversight of AI systems and management of any risks that may arise. Particular attention is drawn to the case of vulnerable populations, where extreme care must be exercised. Finally, the principles emphasize the need for user education at all levels of engagement with AI and for continuing research into AI and its biomedical and healthcare applications.

Childhood Height Growth Rate Association With the Risk of Islet Autoimmunity and Development of Type 1 Diabetes.

CONTEXT: Rapid growth has been suggested to promote islet autoimmunity and progression to type 1 diabetes (T1D). Childhood growth has not been analyzed separately from the infant growth period in most previous studies, but it may have distinct features due to differences between the stages of development. OBJECTIVE: We aimed to analyze the association of childhood growth with development of islet autoimmunity and progression to T1D diagnosis in children 1 to 8 years of age. METHODS: Longitudinal data of childhood growth and development of islet autoimmunity and T1D were analyzed in a prospective cohort study including 10 145 children from Finland, Germany, Sweden, and the United States, 1-8 years of age with at least 3 height and weight measurements and at least 1 measurement of islet autoantibodies. The primary outcome was the appearance of islet autoimmunity and progression from islet autoimmunity to T1D. RESULTS: Rapid increase in height (cm/year) was associated with increased risk of seroconversion to glutamic acid decarboxylase autoantibody, insulin autoantibody, or insulinoma-like antigen-2 autoantibody (hazard ratio [HR] = 1.26 [95% CI = 1.05, 1.51] for 1-3 years of age and HR = 1.48 [95% CI = 1.28, 1.73] for >3 years of age). Furthermore, height rate was positively associated with development of T1D (HR = 1.80 [95% CI = 1.15, 2.81]) in the analyses from seroconversion with insulin autoantibody to diabetes. CONCLUSION: Rapid height growth rate in childhood is associated with increased risk of islet autoimmunity and progression to T1D. Further work is needed to investigate the biological mechanism that may explain this association.

Progression of type 1 diabetes from latency to symptomatic disease is predicted by distinct autoimmune trajectories.

Development of islet autoimmunity precedes the onset of type 1 diabetes in children, however, the presence of autoantibodies does not necessarily lead to manifest disease and the onset of clinical symptoms is hard to predict. Here we show, by longitudinal sampling of islet autoantibodies (IAb) to insulin, glutamic acid decarboxylase and islet antigen-2 that disease progression follows distinct trajectories. Of the combined Type 1 Data Intelligence cohort of 24662 participants, 2172 individuals fulfill the criteria of two or more follow-up visits and IAb positivity at least once, with 652 progressing to type 1 diabetes during the 15 years course of the study. Our Continuous-Time Hidden Markov Models, that are developed to discover and visualize latent states based on the collected data and clinical characteristics of the patients, show that the health state of participants progresses from 11 distinct latent states as per three trajectories (TR1, TR2 and TR3), with associated 5-year cumulative diabetes-free survival of 40% (95% confidence interval [CI], 35% to 47%), 62% (95% CI, 57% to 67%), and 88% (95% CI, 85% to 91%), respectively (p < 0.0001). Age, sex, and HLA-DR status further refine the progression rates within trajectories, enabling clinically useful prediction of disease onset.

Islet Autoantibody Levels Differentiate Progression Trajectories in Individuals With Presymptomatic Type 1 Diabetes.

In our previous data-driven analysis of evolving patterns of islet autoantibodies (IAb) against insulin (IAA), GAD (GADA), and islet antigen 2 (IA-2A), we discovered three trajectories, characterized according to multiple IAb (TR1), IAA (TR2), or GADA (TR3) as the first appearing autoantibodies. Here we examined the evolution of IAb levels within these trajectories in 2,145 IAb-positive participants followed from early life and compared those who progressed to type 1 diabetes (n = 643) with those remaining undiagnosed (n = 1,502). With use of thresholds determined by 5-year diabetes risk, four levels were defined for each IAb and overlaid onto each visit. In diagnosed participants, high IAA levels were seen in TR1 and TR2 at ages <3 years, whereas IAA remained at lower levels in the undiagnosed. Proportions of dwell times (total duration of follow-up at a given level) at the four IAb levels differed between the diagnosed and undiagnosed for GADA and IA-2A in all three trajectories (P < 0.001), but for IAA dwell times differed only within TR2 (P < 0.05). Overall, undiagnosed participants more frequently had low IAb levels and later appearance of IAb than diagnosed participants. In conclusion, while it has long been appreciated that the number of autoantibodies is an important predictor of type 1 diabetes, consideration of autoantibody levels within the three autoimmune trajectories improved differentiation of IAb-positive children who progressed to type 1 diabetes from those who did not.

AI in Healthcare.

The potential value of AI to healthcare, and nursing in particular, ranges from improving quality and efficiency of care to delivering on the promise of personalized and precision medicine. AI systems may become virtually indispensable as ever more data is amassed about every aspect of health. AI can help reduce variability in care, while improving precision, accelerating discovery and reducing disparities. AI can empower patients and potentially allow healthcare professionals to relate to their patients as healers supported by the combined wisdom of the best medical research and analytic technology. There are, however, many challenges to understanding the optimal uses of AI; addressing the technological, systemic, regulatory and attitudinal roadblocks to successful implementation; and integrating AI into the fabric of health care. This paper provides a grounding in the origins and fundamental building blocks of AI, applications in healthcare and for nursing, and the critical challenges facing implementation in healthcare.

Simulating Screening for Risk of Childhood Diabetes: The Collaborative Open Outcomes tooL (COOL).

The Collaborative Open Outcomes tooL (COOL) is a novel, highly configurable application to simulate, evaluate and compare potential population-level screening schedules. Its first application is type 1 diabetes (T1D) screening, where known biomarkers for risk exist but clinical application lags behind. COOL was developed with the T1DI Study Group, in order to assess screening schedules for islet autoimmunity development based on existing datasets. This work shows clinical research utility, but the tool can be applied in other contexts. COOL helps the user define and evaluate a domain knowledge-driven screening schedule, which can be further refined with data-driven insights. COOL can also compare performance of alternative schedules using adjusted sensitivity, specificity, PPV and NPV metrics. Insights from COOL may support a variety of needs in disease screening and surveillance.

Comparison of Methods to Reduce Bias From Clinical Prediction Models of Postpartum Depression.

Importance: The lack of standards in methods to reduce bias for clinical algorithms presents various challenges in providing reliable predictions and in addressing health disparities. Objective: To evaluate approaches for reducing bias in machine learning models using a real-world clinical scenario. Design, Setting, and Participants: Health data for this cohort study were obtained from the IBM MarketScan Medicaid Database. Eligibility criteria were as follows: (1) Female individuals aged 12 to 55 years with a live birth record identified by delivery-related codes from January 1, 2014, through December 31, 2018; (2) greater than 80% enrollment through pregnancy to 60 days post partum; and (3) evidence of coverage for depression screening and mental health services. Statistical analysis was performed in 2020. Exposures: Binarized race (Black individuals and White individuals). Main Outcomes and Measures: Machine learning models (logistic regression [LR], random forest, and extreme gradient boosting) were trained for 2 binary outcomes: postpartum depression (PPD) and postpartum mental health service utilization. Risk-adjusted generalized linear models were used for each outcome to assess potential disparity in the cohort associated with binarized race (Black or White). Methods for reducing bias, including reweighing, Prejudice Remover, and removing race from the models, were examined by analyzing changes in fairness metrics compared with the base models. Baseline characteristics of female individuals at the top-predicted risk decile were compared for systematic differences. Fairness metrics of disparate impact (DI, 1 indicates fairness) and equal opportunity difference (EOD, 0 indicates fairness). Results: Among 573 634 female individuals initially examined for this study, 314 903 were White (54.9%), 217 899 were Black (38.0%), and the mean (SD) age was 26.1 (5.5) years. The risk-adjusted odds ratio comparing White participants with Black participants was 2.06 (95% CI, 2.02-2.10) for clinically recognized PPD and 1.37 (95% CI, 1.33-1.40) for postpartum mental health service utilization. Taking the LR model for PPD prediction as an example, reweighing reduced bias as measured by improved DI and EOD metrics from 0.31 and -0.19 to 0.79 and 0.02, respectively. Removing race from the models had inferior performance for reducing bias compared with the other methods (PPD: DI = 0.61; EOD = -0.05; mental health service utilization: DI = 0.63; EOD = -0.04). Conclusions and Relevance: Clinical prediction models trained on potentially biased data may produce unfair outcomes on the basis of the chosen metrics. This study's results suggest that the performance varied depending on the model, outcome label, and method for reducing bias. This approach toward evaluating algorithmic bias can be used as an example for the growing number of researchers who wish to examine and address bias in their data and models.

Islet Autoimmunity and HLA Markers of Presymptomatic and Clinical Type 1 Diabetes: Joint Analyses of Prospective Cohort Studies in Finland, Germany, Sweden, and the U.S.

OBJECTIVE: To combine prospective cohort studies, by including HLA harmonization, and estimate risk of islet autoimmunity and progression to clinical diabetes. RESEARCH DESIGN AND METHODS: For prospective cohorts in Finland, Germany, Sweden, and the U.S., 24,662 children at increased genetic risk for development of islet autoantibodies and type 1 diabetes have been followed. Following harmonization, the outcomes were analyzed in 16,709 infants-toddlers enrolled by age 2.5 years. RESULTS: In the infant-toddler cohort, 1,413 (8.5%) developed at least one autoantibody confirmed at two or more consecutive visits (seroconversion), 865 (5%) developed multiple autoantibodies, and 655 (4%) progressed to diabetes. The 15-year cumulative incidence of diabetes varied in children with one, two, or three autoantibodies at seroconversion: 45% (95% CI 40-52), 85% (78-90), and 92% (85-97), respectively. Among those with a single autoantibody, status 2 years after seroconversion predicted diabetes risk: 12% (10-25) if reverting to autoantibody negative, 30% (20-40) if retaining a single autoantibody, and 82% (80-95) if developing multiple autoantibodies. HLA-DR-DQ affected the risk of confirmed seroconversion and progression to diabetes in children with stable single-autoantibody status. Their 15-year diabetes incidence for higher- versus lower-risk genotypes was 40% (28-50) vs. 12% (5-38). The rate of progression to diabetes was inversely related to age at development of multiple autoantibodies, ranging from 20% per year to 6% per year in children developing multipositivity in ≤2 years or >7.4 years, respectively. CONCLUSIONS: The number of islet autoantibodies at seroconversion reliably predicts 15-year type 1 diabetes risk. In children retaining a single autoantibody, HLA-DR-DQ genotypes can further refine risk of progression.

Correlating eligibility criteria generalizability and adverse events using Big Data for patients and clinical trials.

Randomized controlled trials can benefit from proactive assessment of how well their participant selection strategies during the design of eligibility criteria can influence the study generalizability. In this paper, we present a quantitative metric called generalizability index for study traits 2.0 (GIST 2.0) to assess the a priori generalizability (based on population representativeness) of a clinical trial by accounting for the dependencies among multiple eligibility criteria. The metric was evaluated on 16 sepsis trials identified from ClinicalTrials.gov, with their adverse event reports extracted from the trial results sections. The correlation between GIST scores and adverse events was analyzed. We found that the GIST 2.0 score was significantly correlated with total adverse events and serious adverse events (weighted correlation coefficients of 0.825 and 0.709, respectively, with P < 0.01). This study exemplifies the promising use of Big Data in electronic health records and ClinicalTrials.gov for optimizing eligibility criteria design for clinical studies.

Implementing syndromic surveillance: a practical guide informed by the early experience.

Syndromic surveillance refers to methods relying on detection of individual and population health indicators that are discernible before confirmed diagnoses are made. In particular, prior to the laboratory confirmation of an infectious disease, ill persons may exhibit behavioral patterns, symptoms, signs, or laboratory findings that can be tracked through a variety of data sources. Syndromic surveillance systems are being developed locally, regionally, and nationally. The efforts have been largely directed at facilitating the early detection of a covert bioterrorist attack, but the technology may also be useful for general public health, clinical medicine, quality improvement, patient safety, and research. This report, authored by developers and methodologists involved in the design and deployment of the first wave of syndromic surveillance systems, is intended to serve as a guide for informaticians, public health managers, and practitioners who are currently planning deployment of such systems in their regions.

The use of data mining to investigate a possible quality problem with ultrasensitive HIV viral load data at a large reference laboratory.

Suppression of HIV viral load to <50 copies/mL, the lower limit of detection for the ultrasensitive assays, has been shown to correlate with favorable clinical outcome. Patients periodically exhibited transient or sustained low-level viremia based on this test. In order to investigate a possible quality concern, we used our corporate data warehouse to examine the patterns in our data over time as well as across geographic regions.