Biomedical knowledge graph-optimized prompt generation for large language models.

MOTIVATION: Large language models (LLMs) are being adopted at an unprecedented rate, yet still face challenges in knowledge-intensive domains such as biomedicine. Solutions such as pretraining and domain-specific fine-tuning add substantial computational overhead, requiring further domain-expertise. Here, we introduce a token-optimized and robust Knowledge Graph-based Retrieval Augmented Generation (KG-RAG) framework by leveraging a massive biomedical KG (SPOKE) with LLMs such as Llama-2-13b, GPT-3.5-Turbo, and GPT-4, to generate meaningful biomedical text rooted in established knowledge. RESULTS: Compared to the existing RAG technique for Knowledge Graphs, the proposed method utilizes minimal graph schema for context extraction and uses embedding methods for context pruning. This optimization in context extraction results in more than 50% reduction in token consumption without compromising the accuracy, making a cost-effective and robust RAG implementation on proprietary LLMs. KG-RAG consistently enhanced the performance of LLMs across diverse biomedical prompts by generating responses rooted in established knowledge, accompanied by accurate provenance and statistical evidence (if available) to substantiate the claims. Further benchmarking on human curated datasets, such as biomedical true/false and multiple-choice questions (MCQ), showed a remarkable 71% boost in the performance of the Llama-2 model on the challenging MCQ dataset, demonstrating the framework's capacity to empower open-source models with fewer parameters for domain-specific questions. Furthermore, KG-RAG enhanced the performance of proprietary GPT models, such as GPT-3.5 and GPT-4. In summary, the proposed framework combines explicit and implicit knowledge of KG and LLM in a token optimized fashion, thus enhancing the adaptability of general-purpose LLMs to tackle domain-specific questions in a cost-effective fashion. AVAILABILITY AND IMPLEMENTATION: SPOKE KG can be accessed at https://spoke.rbvi.ucsf.edu/neighborhood.html. It can also be accessed using REST-API (https://spoke.rbvi.ucsf.edu/swagger/). KG-RAG code is made available at https://github.com/BaranziniLab/KG_RAG. Biomedical benchmark datasets used in this study are made available to the research community in the same GitHub repository.

The scalable precision medicine open knowledge engine (SPOKE): a massive knowledge graph of biomedical information.

MOTIVATION: Knowledge graphs (KGs) are being adopted in industry, commerce and academia. Biomedical KG presents a challenge due to the complexity, size and heterogeneity of the underlying information. RESULTS: In this work, we present the Scalable Precision Medicine Open Knowledge Engine (SPOKE), a biomedical KG connecting millions of concepts via semantically meaningful relationships. SPOKE contains 27 million nodes of 21 different types and 53 million edges of 55 types downloaded from 41 databases. The graph is built on the framework of 11 ontologies that maintain its structure, enable mappings and facilitate navigation. SPOKE is built weekly by python scripts which download each resource, check for integrity and completeness, and then create a 'parent table' of nodes and edges. Graph queries are translated by a REST API and users can submit searches directly via an API or a graphical user interface. Conclusions/Significance: SPOKE enables the integration of seemingly disparate information to support precision medicine efforts. AVAILABILITY AND IMPLEMENTATION: The SPOKE neighborhood explorer is available at https://spoke.rbvi.ucsf.edu. SUPPLEMENTARY INFORMATION: Supplementary data are available at Bioinformatics online.

A biomedical open knowledge network harnesses the power of AI to understand deep human biology.

Knowledge representation and reasoning (KR&R) has been successfully implemented in many fields to enable computers to solve complex problems with AI methods. However, its application to biomedicine has been lagging in part due to the daunting complexity of molecular and cellular pathways that govern human physiology and pathology. In this article we describe concrete uses of SPOKE, an open knowledge network that connects curated information from 37 specialized and human-curated databases into a single property graph, with 3 million nodes and 15 million edges to date. Applications discussed in this article include drug discovery, COVID-19 research and chronic disease diagnosis and management.

Reduced FDG-PET brain metabolism and executive function predict clinical progression in elderly healthy subjects.

Brain changes reminiscent of Alzheimer disease (AD) have been previously reported in a substantial portion of elderly cognitive healthy (HC) subjects. The major aim was to evaluate the accuracy of MRI assessed regional gray matter (GM) volume, 18F-fluorodeoxyglucose positron emission tomography (FDG-PET), and neuropsychological test scores to identify those HC subjects who subsequently convert to mild cognitive impairment (MCI) or AD dementia. We obtained in 54 healthy control (HC) subjects a priori defined region of interest (ROI) values of medial temporal and parietal FDG-PET and medial temporal GM volume. In logistic regression analyses, these ROI values were tested together with neuropsychological test scores (free recall, trail making test B (TMT-B)) as predictors of HC conversion during a clinical follow-up between 3 and 4 years. In voxel-based analyses, FDG-PET and MRI GM maps were compared between HC converters and HC non-converters. Out of the 54 HC subjects, 11 subjects converted to MCI or AD dementia. Lower FDG-PET ROI values were associated with higher likelihood of conversion (p = 0.004), with the area under the curve (AUC) yielding 82.0% (95% CI = (95.5%, 68.5%)). The GM volume ROI was not a significant predictor (p = 0.07). TMT-B but not the free recall tests were a significant predictor (AUC = 71% (95% CI = 50.4%, 91.7%)). For the combination of FDG-PET and TMT-B, the AUC was 93.4% (sensitivity = 82%, specificity = 93%). Voxel-based group comparison showed reduced FDG-PET metabolism within the temporo-parietal and prefrontal cortex in HC converters. In conclusion, medial temporal and-parietal FDG-PET and executive function show a clinically acceptable accuracy for predicting clinical progression in elderly HC subjects.

Early indications of future cognitive decline: stable versus declining controls.

This study aimed to identify baseline features of normal subjects that are associated with subsequent cognitive decline. Publicly available data from the Alzheimer's Disease Neuroimaging Initiative was used to find differences in baseline clinical assessments (ADAScog, AVLT, FAQ) between cognitively healthy individuals who will suffer cognitive decline within 48 months and those who will remain stable for that period. Linear regression models indicated an individual's conversion status was significantly associated with certain baseline neuroimaging measures, including posterior cingulate glucose metabolism. Linear Discriminant Analysis models built with baseline features derived from MRI and FDG-PET measures were capable of successfully predicting whether an individual will convert to MCI within 48 months or remain cognitively stable. The findings from this study support the idea that there exist informative differences between normal people who will later develop cognitive impairments and those who will remain cognitively stable for up to four years. Further, the feasibility of developing predictive models that can detect early states of cognitive decline in seemingly normal individuals was demonstrated.

Evaluating imaging biomarkers for neurodegeneration in pre-symptomatic Huntington's disease using machine learning techniques.

The development of MRI measures as biomarkers for neurodegenerative disease could prove extremely valuable for the assessment of neuroprotective therapies. Much current research is aimed at developing such biomarkers for use in people who are gene-positive for Huntington's disease yet exhibit few or no clinical symptoms of the disease (pre-HD). We acquired structural (T1), diffusion weighted and functional MRI (fMRI) data from 39 pre-HD volunteers and 25 age-matched controls. To determine whether it was possible to decode information about disease state from neuroimaging data, we applied multivariate pattern analysis techniques to several derived voxel-based and segmented region-based datasets. We found that different measures of structural, diffusion weighted, and functional MRI could successfully classify pre-HD and controls using support vector machines (SVM) and linear discriminant analysis (LDA) with up to 76% accuracy. The model producing the highest classification accuracy used LDA with a set of six volume measures from the basal ganglia. Furthermore, using support vector regression (SVR) and linear regression models, we were able to generate quantitative measures of disease progression that were significantly correlated with established measures of disease progression (estimated years to clinical onset, derived from age and genetic information) from several different neuroimaging measures. The best performing regression models used SVR with neuroimaging data from regions within the grey matter (caudate), white matter (corticospinal tract), and fMRI (insular cortex). These results highlight the utility of machine learning analyses in addition to conventional ones. We have shown that several neuroimaging measures contain multivariate patterns of information that are useful for the development of disease-state biomarkers for HD.

Tfm-AR modulates the effects of ApoE4 on cognition.

Female mice are more susceptible to apolipoprotein E (apoE4)-induced cognitive deficits than male mice. These deficits can be antagonized by stimulating androgen receptors (ARs). To determine the role of AR in the cognitive effects of apoE4, we backcrossed mutant mice with a naturally occurring defect in the AR [testicular feminization mutant (tfm)] onto the Apoe-/- background to eliminate mouse apoE gene resulting in non-functional AR, and crossed the tfm/Apoe-/- female mice with apoE4 transgenic male mice. We behaviorally compared Apoe-/-, apoE4, tfm, and tfm/apoE4 male mice. Apoe-/-, apoE4, and tfm mice showed hippocampus-dependent novel location recognition but tfm/apoE4 mice did not. In contrast, all groups showed hippocampus-independent novel object recognition. Hippocampus-dependent learning and memory were also assessed in the water maze. In the water maze probe trial following the second day of hidden platform training, Apoe-/- and apoE4 mice showed spatial memory retention, but tfm and tfm/ApoE4 mice did not. In the water maze, probe trial following the third day of hidden platform training, Apoe-/-, apoE4, and tfm/Apoe-/- mice showed spatial memory retention, but tfm mice did not. These data support an important role for AR in protecting against the detrimental effects of apoE4 on hippocampus-dependent learning and memory.

Effects of sex on object recognition and spatial navigation in humans.

Human tests designed to mirror rodent tests of object recognition and spatial navigation were administered to adult cognitively healthy humans. Facial recognition was also assessed. There was no sex difference in facial recognition, consistent with earlier studies. In the object recognition test, the test-retest NINL total scores during the same visit were highly correlated, comparable to the test-retest correlations obtained in the established facial recognition test. There were no effects of sex on object recognition. However, in the spatial navigation test, there were effects of sex on spatial learning and memory during the session with the hidden, but not visible, target. These tests might be useful to compare assessments of object recognition and spatial learning and memory in humans and animal models.

Age-dependent measures of anxiety and cognition in male histidine decarboxylase knockout (Hdc-/-) mice.

Histidine decarboxylase deficient (Hdc(-/-)) and wild-type male mice on the C57Bl6/J background were used to determine the role of histamine in brain function. 3-5 (Y) and 12-14 (MA) month-old Hdc(-/-) mice showed hypoactivity and increased measures of anxiety in the open field, light-dark, elevated plus-maze, and elevated zero maze tests. Y Hdc(-/-) mice showed superior performance in the hidden sessions of the water maze and passive avoidance memory retention. In contrast, Y Hdc(-/-) mice were impaired in novel location recognition, spent less time searching in the target quadrant and more time searching in the outer zone of the water maze during the probe trials. These behaviors are likely due to increased measures of anxiety and are not found in MA Hdc(-/-) mice. These data support a role for histamine in anxiety and cognition and underline the importance of considering age and potential effects on measures of anxiety in the interpretation of the role of histaminergic neurotransmission in cognitive function.