Deep Learning Proteins using a Triplet-BERT network.

Modern sequencing technology has produced a vast quantity of proteomic data, which has been key to the development of various deep learning models within the field. However, there are still challenges to overcome with regards to modelling the properties of a protein, especially when labelled resources are scarce. Developing interpretable deep learning models is an essential criterion, as proteomics research requires methods to understand the functional properties of proteins. The ability to derive quality information from both the model and the data will play a vital role in the advancement of proteomics research. In this paper, we seek to leverage a BERT model that has been pre-trained on a vast quantity of proteomic data, to model a collection of regression tasks using only a minimal amount of data. We adopt a triplet network structure to fine-tune the BERT model for each dataset and evaluate its performance on a set of downstream task predictions: plasma membrane localisation, thermostability, peak absorption wavelength, and enantioselectivity. Our results significantly improve upon the original BERT baseline as well as the previous state-of-the-art models for each task, demonstrating the benefits of using a triplet network for refining such a large pre-trained model on a limited dataset. As a form of white-box deep learning, we also visualise how the model attends to specific parts of the protein and how the model detects critical modifications that change its overall function.

Modelling Drug-Target Binding Affinity using a BERT based Graph Neural network.

Understanding the interactions between novel drugs and target proteins is fundamentally important in disease research as discovering drug-protein interactions can be an exceptionally time-consuming and expensive process. Alternatively, this process can be simulated using modern deep learning methods that have the potential of utilising vast quantities of data to reduce the cost and time required to provide accurate predictions. We seek to leverage a set of BERT-style models that have been pre-trained on vast quantities of both protein and drug data. The encodings produced by each model are then utilised as node representations for a graph convolutional neural network, which in turn are used to model the interactions without the need to simultaneously fine-tune both protein and drug BERT models to the task. We evaluate the performance of our approach on two drug-target interaction datasets that were previously used as benchmarks in recent work.Our results significantly improve upon a vanilla BERT baseline approach as well as the former state-of-the-art methods for each task dataset. Our approach builds upon past work in two key areas; firstly, we take full advantage of two large pre-trained BERT models that provide improved representations of task-relevant properties of both drugs and proteins. Secondly, inspired by work in natural language processing that investigates how linguistic structure is represented in such models, we perform interpretability analyses that allow us to locate functionally-relevant areas of interest within each drug and protein. By modelling the drug-target interactions as a graph as opposed to a set of isolated interactions, we demonstrate the benefits of combining large pre-trained models and a graph neural network to make state-of-the-art predictions on drug-target binding affinity.

Mechanisms of Resistance to Prostate-Specific Membrane Antigen-Targeted Radioligand Therapy in a Mouse Model of Prostate Cancer.

Prostate-specific membrane antigen (PSMA)-targeted radioligand therapy (RLT) is effective against prostate cancer (PCa), but all patients relapse eventually. Poor understanding of the underlying resistance mechanisms represents a key barrier to development of more effective RLT. We investigate the proteome and phosphoproteome in a mouse model of PCa to identify signaling adaptations triggered by PSMA RLT. Methods: Therapeutic efficacy of PSMA RLT was assessed by tumor volume measurements, time to progression, and survival in C4-2 or C4-2 TP53-/- tumor-bearing nonobese diabetic scid γ-mice. Two days after RLT, the proteome and phosphoproteome were analyzed by mass spectrometry. Results: PSMA RLT significantly improved disease control in a dose-dependent manner. Proteome and phosphoproteome datasets revealed activation of genotoxic stress response pathways, including deregulation of DNA damage/replication stress response, TP53, androgen receptor, phosphatidylinositol-3-kinase/AKT, and MYC signaling. C4-2 TP53-/- tumors were less sensitive to PSMA RLT than were parental counterparts, supporting a role for TP53 in mediating RLT responsiveness. Conclusion: We identified signaling alterations that may mediate resistance to PSMA RLT in a PCa mouse model. Our data enable the development of rational synergistic RLT-combination therapies to improve outcomes for PCa patients.

Expanding the Vocabulary of a Protein: Application of Subword Algorithms to Protein Sequence Modelling.

Deep learning has proven to be a useful tool for modelling protein properties. However, given the variability in the length of proteins, it can be difficult to summarise the sequence of amino acids effectively. In many cases, as a result of using fixed-length representations, information about long proteins can be lost through truncation, or model training can be slow due to the use of excessive padding. In this work, we aim to overcome these problems by expanding upon the original vocabulary used to represent the protein sequence. To this end, we utilise two prominent subword algorithms that have been previously used to reach state-of-the-art results in various Natural Language Processing tasks. The algorithms are used to encode the original protein sequence into a set of subsequences before they are analysed by a Doc2Vec model. The pre-trained encodings produced by each algorithm are tested on a variety of downstream tasks: four protein property prediction tasks (plasma membrane localization, thermostability, peak absorption wavelength, enantioselectivity) as well as drug-target affinity prediction tasks over two datasets. Our results significantly improve on the state-of-the-art for these tasks, demonstrating the benefits of using subword compression algorithms for modelling proteins.

Magnetic resonance evaluation of renal artery stenosis in a swine model: performance of low-dose gadobutrol versus gadoterate meglumine in comparison with digital subtraction intra-arterial catheter angiography.

PURPOSE: The aim of this study was to compare low-dose imaging with gadobutrol and gadoterate meglumine (Gd-DOTA) for evaluation of renal artery stenosis with 3-T magnetic resonance angiography (MRA) in a swine model. METHOD AND MATERIALS: A total of 12 experimental animals were evaluated using equivalently dosed gadobutrol and Gd-DOTA for time-resolved and static imaging. For time-resolved imaging, the time-resolved imaging with stochastic trajectories (TWIST) technique (temporal footprint, 4.4 seconds) was used; a dose of 1 mL of gadobutrol was injected at 2 mL/s and a dose of 2 mL of Gd-DOTA was injected at both 2 and 4 mL/s. For a separate static acquisition, doses were doubled. The static scans were used for stenosis gradation and the time-resolved scans for comparison of enhancement dynamics, signal-to-noise ratio (SNR), and qualitative assessments. RESULTS: The average magnitude of difference in the stenosis measurements with static gadobutrol scans relative to digital subtraction intra-arterial catheter angiography (mean [SD], 7.4% [5.6%]) was less than with both the 2 mL/s (10.6% [6.2%]) and 4 mL/s (11.5% [7.8%]) Gd-DOTA MRA protocols. On time-resolved scans, peak signal-to-noise ratio was greatest with the gadobutrol protocol (P < 0.05), and the gadobutrol TWIST scan was preferred to the TWIST Gd-DOTA scan in terms of image quality and stenosis visualization in every case for every reader. CONCLUSION: Low-dose gadobutrol (~0.05 mmoL/kg) contrast-enhanced MRA results in improved accuracy of renal artery stenosis assessments relative to equivalently dosed Gd-DOTA at 3 T.