Differences in information accessed in a pharmacologic knowledge base using a conversational agent vs traditional search methods.

INTRODUCTION: Clinicians rely on pharmacologic knowledge bases to answer medication questions and avoid potential adverse drug events. In late 2018, an artificial intelligence-based conversational agent, Watson Assistant (WA), was made available to online subscribers to the pharmacologic knowledge base, Micromedex®. WA allows users to ask medication-related questions in natural language. This study evaluated search method-dependent differences in the frequency of information accessed by traditional methods (keyword search and heading navigation) vs conversational agent search. MATERIALS AND METHODS: We compared the proportion of information types accessed through the conversational agent to the proportion of analogous information types accessed by traditional methods during the first 6 months of 2020. RESULTS: Addition of the conversational agent allowed early adopters to access 22 different information types contained in the 'quick answers' portion of the knowledge base. These information types were accessed 117,550 times with WA during the study period, compared to 33,649,651 times using traditional search methods. The distribution across information types differed by method employed (c2 test, P < .0001). Single drug/dosing, FDA/non-FDA uses, adverse effects, and drug administration emerged as 4 of the top 5 information types accessed by either method. Intravenous compatibility was accessed more frequently using the conversational agent (7.7% vs. 0.6% for traditional methods), whereas dose adjustments were accessed more frequently via traditional methods (4.8% vs. 1.4% for WA). CONCLUSION: In a widely used pharmacologic knowledge base, information accessed through conversational agents versus traditional methods differed. User-centered studies are needed to understand these differences.

Health Care Equity in the Use of Advanced Analytics and Artificial Intelligence Technologies in Primary Care.

The integration of advanced analytics and artificial intelligence (AI) technologies into the practice of medicine holds much promise. Yet, the opportunity to leverage these tools carries with it an equal responsibility to ensure that principles of equity are incorporated into their implementation and use. Without such efforts, tools will potentially reflect the myriad of ways in which data, algorithmic, and analytic biases can be produced, with the potential to widen inequities by race, ethnicity, gender, and other sociodemographic factors implicated in disparate health outcomes. We propose a set of strategic assertions to examine before, during, and after adoption of these technologies in order to facilitate healthcare equity across all patient population groups. The purpose is to enable generalists to promote engagement with technology companies and co-create, promote, or support innovation and insights that can potentially inform decision-making and health care equity.

Accuracy of an Artificial Intelligence System for Cancer Clinical Trial Eligibility Screening: Retrospective Pilot Study.

BACKGROUND: Screening patients for eligibility for clinical trials is labor intensive. It requires abstraction of data elements from multiple components of the longitudinal health record and matching them to inclusion and exclusion criteria for each trial. Artificial intelligence (AI) systems have been developed to improve the efficiency and accuracy of this process. OBJECTIVE: This study aims to evaluate the ability of an AI clinical decision support system (CDSS) to identify eligible patients for a set of clinical trials. METHODS: This study included the deidentified data from a cohort of patients with breast cancer seen at the medical oncology clinic of an academic medical center between May and July 2017 and assessed patient eligibility for 4 breast cancer clinical trials. CDSS eligibility screening performance was validated against manual screening. Accuracy, sensitivity, specificity, positive predictive value, and negative predictive value for eligibility determinations were calculated. Disagreements between manual screeners and the CDSS were examined to identify sources of discrepancies. Interrater reliability between manual reviewers was analyzed using Cohen (pairwise) and Fleiss (three-way) κ, and the significance of differences was determined by Wilcoxon signed-rank test. RESULTS: In total, 318 patients with breast cancer were included. Interrater reliability for manual screening ranged from 0.60-0.77, indicating substantial agreement. The overall accuracy of breast cancer trial eligibility determinations by the CDSS was 87.6%. CDSS sensitivity was 81.1% and specificity was 89%. CONCLUSIONS: The AI CDSS in this study demonstrated accuracy, sensitivity, and specificity of greater than 80% in determining the eligibility of patients for breast cancer clinical trials. CDSSs can accurately exclude ineligible patients for clinical trials and offer the potential to increase screening efficiency and accuracy. Additional research is needed to explore whether increased efficiency in screening and trial matching translates to improvements in trial enrollment, accruals, feasibility assessments, and cost.

Artificial Intelligence Clinical Evidence Engine for Automatic Identification, Prioritization, and Extraction of Relevant Clinical Oncology Research.

PURPOSE: We developed a system to automate analysis of the clinical oncology scientific literature from bibliographic databases and match articles to specific patient cohorts to answer specific questions regarding the efficacy of a treatment. The approach attempts to replicate a clinician's mental processes when reviewing published literature in the context of a patient case. We describe the system and evaluate its performance. METHODS: We developed separate ground truth data sets for each of the tasks described in the paper. The first ground truth was used to measure the natural language processing (NLP) accuracy from approximately 1,300 papers covering approximately 3,100 statements and approximately 25 concepts; performance was evaluated using a standard F1 score. The ground truth for the expert classifier model was generated by dividing papers cited in clinical guidelines into a training set and a test set in an 80:20 ratio, and performance was evaluated for accuracy, sensitivity, and specificity. RESULTS: The NLP models were able to identify individual attributes with a 0.7-0.9 F1 score, depending on the attribute of interest. The expert classifier machine learning model was able to classify the individual records with a 0.93 accuracy (95% CI, 0.9 to 0.96, P < .0001), and sensitivity and specificity of 0.95 and 0.91, respectively. Using a decision boundary of 0.5 for the positive (expert) label, the classifier demonstrated an F1 score of 0.92. CONCLUSION: The system identified and extracted evidence from the oncology literature with a high degree of accuracy, sensitivity, and specificity. This tool enables timely access to the most relevant biomedical literature, providing critical support to evidence-based practice in areas of rapidly evolving science.

Comparison of an oncology clinical decision-support system's recommendations with actual treatment decisions.

OBJECTIVE: IBM(R) Watson for Oncology (WfO) is a clinical decision-support system (CDSS) that provides evidence-informed therapeutic options to cancer-treating clinicians. A panel of experienced oncologists compared CDSS treatment options to treatment decisions made by clinicians to characterize the quality of CDSS therapeutic options and decisions made in practice. METHODS: This study included patients treated between 1/2017 and 7/2018 for breast, colon, lung, and rectal cancers at Bumrungrad International Hospital (BIH), Thailand. Treatments selected by clinicians were paired with therapeutic options presented by the CDSS and coded to mask the origin of options presented. The panel rated the acceptability of each treatment in the pair by consensus, with acceptability defined as compliant with BIH's institutional practices. Descriptive statistics characterized the study population and treatment-decision evaluations by cancer type and stage. RESULTS: Nearly 60% (187) of 313 treatment pairs for breast, lung, colon, and rectal cancers were identical or equally acceptable, with 70% (219) of WfO therapeutic options identical to, or acceptable alternatives to, BIH therapy. In 30% of cases (94), 1 or both treatment options were rated as unacceptable. Of 32 cases where both WfO and BIH options were acceptable, WfO was preferred in 18 cases and BIH in 14 cases. Colorectal cancers exhibited the highest proportion of identical or equally acceptable treatments; stage IV cancers demonstrated the lowest. CONCLUSION: This study demonstrates that a system designed in the US to support, rather than replace, cancer-treating clinicians provides therapeutic options which are generally consistent with recommendations from oncologists outside the US.

Effect of an Artificial Intelligence Clinical Decision Support System on Treatment Decisions for Complex Breast Cancer.

PURPOSE: To examine the impact of a clinical decision support system (CDSS) on breast cancer treatment decisions and adherence to National Comprehensive Cancer Center (NCCN) guidelines. PATIENTS AND METHODS: A cross-sectional observational study was conducted involving 1,977 patients at high risk for recurrent or metastatic breast cancer from the Chinese Society of Clinical Oncology. Ten oncologists provided blinded treatment recommendations for an average of 198 patients before and after viewing therapeutic options offered by the CDSS. Univariable and bivariable analyses of treatment changes were performed, and multivariable logistic regressions were estimated to examine the effects of physician experience (years), patient age, and receptor subtype/TNM stage. RESULTS: Treatment decisions changed in 105 (5%) of 1,977 patients and were concentrated in those with hormone receptor (HR)-positive disease or stage IV disease in the first-line therapy setting (73% and 58%, respectively). Logistic regressions showed that decision changes were more likely in those with HR-positive cancer (odds ratio [OR], 1.58; P < .05) and less likely in those with stage IIA (OR, 0.29; P < .05) or IIIA cancer (OR, 0.08; P < .01). Reasons cited for changes included consideration of the CDSS therapeutic options (63% of patients), patient factors highlighted by the tool (23%), and the decision logic of the tool (13%). Patient age and oncologist experience were not associated with decision changes. Adherence to NCCN treatment guidelines increased slightly after using the CDSS (0.5%; P = .003). CONCLUSION: Use of an artificial intelligence-based CDSS had a significant impact on treatment decisions and NCCN guideline adherence in HR-positive breast cancers. Although cases of stage IV disease in the first-line therapy setting were also more likely to be changed, the effect was not statistically significant (P = .22). Additional research on decision impact, patient-physician communication, learning, and clinical outcomes is needed to establish the overall value of the technology.

Evaluation of an artificial intelligence clinical trial matching system in Australian lung cancer patients.

OBJECTIVE: The objective of this technical study was to evaluate the performance of an artificial intelligence (AI)-based system for clinical trials matching for a cohort of lung cancer patients in an Australian cancer hospital. METHODS: A lung cancer cohort was derived from clinical data from patients attending an Australian cancer hospital. Ten phases I-III clinical trials registered on clinicaltrials.gov and open to lung cancer patients at this institution were utilized for assessments. The trial matching system performance was compared to a gold standard established by clinician consensus for trial eligibility. RESULTS: The study included 102 lung cancer patients. The trial matching system evaluated 7252 patient attributes (per patient median 74, range 53-100) against 11 467 individual trial eligibility criteria (per trial median 597, range 243-4132). Median time for the system to run a query and return results was 15.5 s (range 7.2-37.8). In establishing the gold standard, clinician interrater agreement was high (Cohen's kappa 0.70-1.00). On a per-patient basis, the performance of the trial matching system for eligibility was as follows: accuracy, 91.6%; recall (sensitivity), 83.3%; precision (positive predictive value), 76.5%; negative predictive value, 95.7%; and specificity, 93.8%. DISCUSSION AND CONCLUSION: The AI-based clinical trial matching system allows efficient and reliable screening of cancer patients for clinical trials with 95.7% accuracy for exclusion and 91.6% accuracy for overall eligibility assessment; however, clinician input and oversight are still required. The automated system demonstrates promise as a clinical decision support tool to prescreen a large patient cohort to identify subjects suitable for further assessment.

Artificial intelligence-based conversational agent to support medication prescribing.

OBJECTIVE: This article describes the system architecture, training, initial use, and performance of Watson Assistant (WA), an artificial intelligence-based conversational agent, accessible within Micromedex®. MATERIALS AND METHODS: The number and frequency of intents (target of a user's query) triggered in WA during its initial use were examined; intents triggered over 9 months were compared to the frequency of topics accessed via keyword search of Micromedex. Accuracy of WA intents assigned to 400 queries was compared to assignments by 2 independent subject matter experts (SMEs), with inter-rater reliability measured by Cohen's kappa. RESULTS: In over 126 000 conversations with WA, intents most frequently triggered involved dosing (N = 30 239, 23.9%) and administration (N = 14 520, 11.5%). SMEs with substantial inter-rater agreement (kappa = 0.71) agreed with intent mapping in 247 of 400 queries (62%), including 16 queries related to content that WA and SMEs agreed was unavailable in WA. SMEs found 57 (14%) of 400 queries incorrectly mapped by WA; 112 (28%) queries unanswerable by WA included queries that were either ambiguous, contained unrecognized typographical errors, or addressed topics unavailable to WA. Of the queries answerable by WA (288), SMEs determined 231 (80%) were correctly linked to an intent. DISCUSSION: A conversational agent successfully linked most queries to intents in Micromedex. Ongoing system training seeks to widen the scope of WA and improve matching capabilities. CONCLUSION: WA enabled Micromedex users to obtain answers to many medication-related questions using natural language, with the conversational agent facilitating mapping to a broader distribution of topics than standard keyword searches.

Artificial Intelligence Tool for Optimizing Eligibility Screening for Clinical Trials in a Large Community Cancer Center.

PURPOSE: Less than 5% of patients with cancer enroll in clinical trials, and 1 in 5 trials are stopped for poor accrual. We evaluated an automated clinical trial matching system that uses natural language processing to extract patient and trial characteristics from unstructured sources and machine learning to match patients to clinical trials. PATIENTS AND METHODS: Medical records from 997 patients with breast cancer were assessed for trial eligibility at Highlands Oncology Group between May and August 2016. System and manual attribute extraction and eligibility determinations were compared using the percentage of agreement for 239 patients and 4 trials. Sensitivity and specificity of system-generated eligibility determinations were measured, and the time required for manual review and system-assisted eligibility determinations were compared. RESULTS: Agreement between system and manual attribute extraction ranged from 64.3% to 94.0%. Agreement between system and manual eligibility determinations was 81%-96%. System eligibility determinations demonstrated specificities between 76% and 99%, with sensitivities between 91% and 95% for 3 trials and 46.7% for the 4th. Manual eligibility screening of 90 patients for 3 trials took 110 minutes; system-assisted eligibility determinations of the same patients for the same trials required 24 minutes. CONCLUSION: In this study, the clinical trial matching system displayed a promising performance in screening patients with breast cancer for trial eligibility. System-assisted trial eligibility determinations were substantially faster than manual review, and the system reliably excluded ineligible patients for all trials and identified eligible patients for most trials.

Gßγ directly modulates vesicle fusion by competing with synaptotagmin for binding to neuronal SNARE proteins embedded in membranes.

Gi/o-coupled G protein-coupled receptors can inhibit neurotransmitter release at synapses via multiple mechanisms. In addition to Gßγ-mediated modulation of voltage-gated calcium channels (VGCC), inhibition can also be mediated through the direct interaction of Gßγ subunits with the soluble N-ethylmaleimide attachment protein receptor (SNARE) complex of the vesicle fusion apparatus. Binding studies with soluble SNARE complexes have shown that Gßγ binds to both ternary SNARE complexes, t-SNARE heterodimers, and monomeric SNAREs, competing with synaptotagmin 1(syt1) for binding sites on t-SNARE. However, in secretory cells, Gßγ, SNAREs, and synaptotagmin interact in the lipid environment of a vesicle at the plasma membrane. To approximate this environment, we show that fluorescently labeled Gßγ interacts specifically with lipid-embedded t-SNAREs consisting of full-length syntaxin 1 and SNAP-25B at the membrane, as measured by fluorescence polarization. Fluorescently labeled syt1 undergoes competition with Gßγ for SNARE-binding sites in lipid environments. Mutant Gßγ subunits that were previously shown to be more efficacious at inhibiting Ca2+-triggered exocytotic release than wild-type Gßγ were also shown to bind SNAREs at a higher affinity than wild type in a lipid environment. These mutant Gßγ subunits were unable to inhibit VGCC currents. Specific peptides corresponding to regions on Gß and Gγ shown to be important for the interaction disrupt the interaction in a concentration-dependent manner. In in vitro fusion assays using full-length t- and v-SNAREs embedded in liposomes, Gßγ inhibited Ca2+/synaptotagmin-dependent fusion. Together, these studies demonstrate the importance of these regions for the Gßγ-SNARE interaction and show that the target of Gßγ, downstream of VGCC, is the membrane-embedded SNARE complex.

A transient interaction between the phosphate binding loop and switch I contributes to the allosteric network between receptor and nucleotide in Gαi1.

Receptor-mediated activation of the Gα subunit of heterotrimeric G proteins requires allosteric communication between the receptor binding site and the guanine nucleotide binding site, which are separated by >30 Å. Structural changes in the allosteric network connecting these sites are predicted to be transient in the wild-type Gα subunit, making studies of these connections challenging. In the current work, site-directed mutants that alter the energy barriers between the activation states are used as tools to better understand the transient features of allosteric signaling in the Gα subunit. The observed differences in relative receptor affinity for intact Gαi1 subunits versus C-terminal Gαi1 peptides harboring the K345L mutation are consistent with this mutation modulating the allosteric network in the protein subunit. Measurement of nucleotide exchange rates, affinity for metarhodopsin II, and thermostability suggest that the K345L Gαi1 variant has reduced stability in both the GDP-bound and nucleotide-free states as compared with wild type but similar stability in the GTPγS-bound state. High resolution x-ray crystal structures reveal conformational changes accompanying the destabilization of the GDP-bound state. Of these, the conformation for Switch I was stabilized by an ionic interaction with the phosphate binding loop. Further site-directed mutagenesis suggests that this interaction between Switch I and the phosphate binding loop is important for receptor-mediated nucleotide exchange in the wild-type Gαi1 subunit.

Energetic analysis of the rhodopsin-G-protein complex links the α5 helix to GDP release.

We present a model of interaction of Gi protein with the activated receptor (R*) rhodopsin, which pinpoints energetic contributions to activation and reconciles the ß2 adrenergic receptor-Gs crystal structure with new and previously published experimental data. In silico analysis demonstrated energetic changes when the Gα C-terminal helix (α5) interacts with the R* cytoplasmic pocket, thus leading to displacement of the helical domain and GDP release. The model features a less dramatic domain opening compared with the crystal structure. The α5 helix undergoes a 63° rotation, accompanied by a 5.7-Å translation, that reorganizes interfaces between α5 and α1 helices and between α5 and ß6-α5. Changes in the ß6-α5 loop displace αG. All of these movements lead to opening of the GDP-binding pocket. The model creates a roadmap for experimental studies of receptor-mediated G-protein activation.

Conformational flexibility and structural dynamics in GPCR-mediated G protein activation: a perspective.

Structure and dynamics of G proteins and their cognate receptors, both alone and in complex, are becoming increasingly accessible to experimental techniques. Understanding the conformational changes and timelines that govern these changes can lead to new insights into the processes of ligand binding and associated G protein activation. Experimental systems may involve the use of, or otherwise stabilize, non-native environments. This can complicate our understanding of structural and dynamic features of processes such as the ionic lock, tryptophan toggle, and G protein flexibility. While elements in the receptor's transmembrane helices and the C-terminal α5 helix of Gα undergo well-defined structural changes, regions subject to conformational flexibility may be important in fine-tuning the interactions between activated receptors and G proteins. The pairing of computational and experimental approaches will continue to provide powerful tools to probe the conformation and dynamics of receptor-mediated G protein activation.

Linking receptor activation to changes in Sw I and II of Gα proteins.

G-protein coupled receptors catalyze nucleotide exchange on G proteins, which results in subunit dissociation and effector activation. In the recent ß2AR-Gs structure, portions of Switch I and II of Gα are not fully elucidated. We paired fluorescence studies of receptor-Gαi interactions with the ß2AR-Gs and other Gi structures to investigate changes in Switch I and II during receptor activation and GTP binding. The ß2/ß3 loop containing Leu194 of Gαi is located between Switches I and II, in close proximity to IC2 of the receptor and the C-terminus of Gα, thus providing an allosteric connection between these Switches and receptor activation. We compared the environment of residues in myristoylated Gαi proteins in the heterotrimer to that upon receptor activation and subsequent GTP binding. Upon receptor activation, residues in both Switch regions are less solvent-exposed, as compared to the heterotrimer. Upon GTPγS binding, the environment of several residues in Switch I resemble the receptor-bound state, while Switch II residues display effects on their environment which are consistent with their role in GTP binding and Gßγ dissociation. The ability to merge available crystal structures with solution studies is a powerful tool to gain insight into conformational changes associated with receptor-mediated Gi protein activation.

Crystal structures of acetate kinases from the eukaryotic pathogens Entamoeba histolytica and Cryptococcus neoformans.

Acetate kinases (ACKs) are members of the acetate and sugar kinase/hsp70/actin (ASKHA) superfamily and catalyze the reversible phosphorylation of acetate, with ADP/ATP the most common phosphoryl acceptor/donor. While prokaryotic ACKs have been the subject of extensive biochemical and structural characterization, there is a comparative paucity of information on eukaryotic ACKs, and prior to this report, no structure of an ACK of eukaryotic origin was available. We determined the structures of ACKs from the eukaryotic pathogens Entamoeba histolytica and Cryptococcus neoformans. Each active site is located at an interdomain interface, and the acetate and phosphate binding pockets display sequence and structural conservation with their prokaryotic counterparts. Interestingly, the E. histolytica ACK has previously been shown to be pyrophosphate (PP(i))-dependent, and is the first ACK demonstrated to have this property. Examination of its structure demonstrates how subtle amino acid substitutions within the active site have converted cosubstrate specificity from ATP to PP(i) while retaining a similar backbone conformation. Differences in the angle between domains surrounding the active site suggest that interdomain movement may accompany catalysis. Taken together, these structures are consistent with the eukaryotic ACKs following a similar reaction mechanism as is proposed for the prokaryotic homologs.

Myristoylation exerts direct and allosteric effects on Gα conformation and dynamics in solution.

Coupling of heterotrimeric G proteins to activated G protein-coupled receptors results in nucleotide exchange on the Gα subunit, which in turn decreases its affinity for both Gßγ and activated receptors. N-Terminal myristoylation of Gα subunits aids in membrane localization of inactive G proteins. Despite the presence of the covalently attached myristoyl group, Gα proteins are highly soluble after GTP binding. This study investigated factors facilitating the solubility of the activated, myristoylated protein. In doing so, we also identified myristoylation-dependent differences in regions of Gα known to play important roles in interactions with receptors, effectors, and nucleotide binding. Amide hydrogen-deuterium exchange and site-directed fluorescence of activated proteins revealed a solvent-protected amino terminus that was enhanced by myristoylation. Furthermore, fluorescence quenching confirmed that the myristoylated amino terminus is in proximity to the Switch II region in the activated protein. Myristoylation also stabilized the interaction between the guanine ring and the base of the α5 helix that contacts the bound nucleotide. The allosteric effects of myristoylation on protein structure, function, and localization indicate that the myristoylated amino terminus of Gα(i) functions as a myristoyl switch, with implications for myristoylation in the stabilization of nucleotide binding and in the spatial regulation of G protein signaling.

Allosteric mechanisms of G protein-Coupled Receptor signaling: a structural perspective.

G protein-Coupled Receptors (GPCRs) use a complex series of intramolecular conformational changes to couple agonist binding to the binding and activation of cognate heterotrimeric G protein (Gαßγ). The mechanisms underlying this long-range activation have been identified using a variety of biochemical and structural approaches and have primarily used visual signal transduction via the GPCR rhodopsin and cognate heterotrimeric G protein transducin (G(t)) as a model system. In this chapter, we review the methods that have revealed allosteric signaling through rhodopsin and transducin. These methods can be applied to a variety of GPCR-mediated signaling pathways.

Interaction of a G protein with an activated receptor opens the interdomain interface in the alpha subunit.

In G-protein signaling, an activated receptor catalyzes GDP/GTP exchange on the G(α) subunit of a heterotrimeric G protein. In an initial step, receptor interaction with G(α) acts to allosterically trigger GDP release from a binding site located between the nucleotide binding domain and a helical domain, but the molecular mechanism is unknown. In this study, site-directed spin labeling and double electron-electron resonance spectroscopy are employed to reveal a large-scale separation of the domains that provides a direct pathway for nucleotide escape. Cross-linking studies show that the domain separation is required for receptor enhancement of nucleotide exchange rates. The interdomain opening is coupled to receptor binding via the C-terminal helix of G(α), the extension of which is a high-affinity receptor binding element.

Coupling efficiency of rhodopsin and transducin in bicelles.

G protein coupled receptors (GPCRs) can be activated by various extracellular stimuli, including hormones, peptides, odorants, neurotransmitters, nucleotides, or light. After activation, receptors interact with heterotrimeric G proteins and catalyze GDP release from the Gα subunit, the rate limiting step in G protein activation, to form a high affinity nucleotide-free GPCR-G protein complex. In vivo, subsequent GTP binding reduces affinity of the Gα protein for the activated receptor. In this study, we investigated the biochemical and structural characteristics of the prototypical GPCR, rhodopsin, and its signaling partner, transducin (G(t)), in bicelles to better understand the effects of membrane composition on high affinity complex formation, stability, and receptor mediated nucleotide release. Our results demonstrate that the high-affinity complex (rhodopsin-G(t)(empty)) forms more readily and has dramatically increased stability when rhodopsin is integrated into bicelles of a defined composition. We increased the half-life of functional complex to 1 week in the presence of negatively charged phospholipids. These data suggest that a membrane-like structure is an important contributor to the formation and stability of functional receptor-G protein complexes and can extend the range of studies that investigate properties of these complexes.