Supervised fine-tuning of pre-trained antibody language models improves antigen specificity prediction.

Antibodies play a crucial role in adaptive immune responses by determining B cell specificity to antigens and focusing immune function on target pathogens. Accurate prediction of antibody-antigen specificity directly from antibody sequencing data would be a great aid in understanding immune responses, guiding vaccine design, and developing antibody-based therapeutics. In this study, we present a method of supervised fine-tuning for antibody language models, which improves on previous results in binding specificity prediction to SARS-CoV-2 spike protein and influenza hemagglutinin. We perform supervised fine-tuning on four pre-trained antibody language models to predict specificity to these antigens and demonstrate that fine-tuned language model classifiers exhibit enhanced predictive accuracy compared to classifiers trained on pre-trained model embeddings. The change of model attention activations after supervised fine-tuning suggested that this performance was driven by an increased model focus on the complementarity determining regions (CDRs). Application of the supervised fine-tuned models to BCR repertoire data demonstrated that these models could recognize the specific responses elicited by influenza and SARS-CoV-2 vaccination. Overall, our study highlights the benefits of supervised fine-tuning on pre-trained antibody language models as a mechanism to improve antigen specificity prediction.

Language model-based B cell receptor sequence embeddings can effectively encode receptor specificity.

High throughput sequencing of B cell receptors (BCRs) is increasingly applied to study the immense diversity of antibodies. Learning biologically meaningful embeddings of BCR sequences is beneficial for predictive modeling. Several embedding methods have been developed for BCRs, but no direct performance benchmarking exists. Moreover, the impact of the input sequence length and paired-chain information on the prediction remains to be explored. We evaluated the performance of multiple embedding models to predict BCR sequence properties and receptor specificity. Despite the differences in model architectures, most embeddings effectively capture BCR sequence properties and specificity. BCR-specific embeddings slightly outperform general protein language models in predicting specificity. In addition, incorporating full-length heavy chains and paired light chain sequences improves the prediction performance of all embeddings. This study provides insights into the properties of BCR embeddings to improve downstream prediction applications for antibody analysis and discovery.

Histopathologic and Machine Deep Learning Criteria to Predict Lymphoma Transformation in Bone Marrow Biopsies.

CONTEXT.­: Large cell transformation (LCT) of indolent B-cell lymphomas, such as follicular lymphoma (FL) and chronic lymphocytic leukemia (CLL), signals a worse prognosis, at which point aggressive chemotherapy is initiated. Although LCT is relatively straightforward to diagnose in lymph nodes, a marrow biopsy is often obtained first given its ease of procedure, low cost, and low morbidity. However, consensus criteria for LCT in bone marrow have not been established. OBJECTIVE.­: To study the accuracy and reproducibility of a trained convolutional neural network in identifying LCT, in light of promising machine learning tools that may introduce greater objectivity to morphologic analysis. DESIGN.­: We retrospectively identified patients who had a diagnosis of FL or CLL who had undergone bone marrow biopsy for the clinical question of LCT. We scored morphologic criteria and correlated results with clinical disease progression. In addition, whole slide scans were annotated into patches to train convolutional neural networks to discriminate between small and large tumor cells and to predict the patient's probability of transformation. RESULTS.­: Using morphologic examination, the proportion of large lymphoma cells (≥10% in FL and ≥30% in CLL), chromatin pattern, distinct nucleoli, and proliferation index were significantly correlated with LCT in FL and CLL. Compared to pathologist-derived estimates, machine-generated quantification demonstrated better reproducibility and stronger correlation with final outcome data. CONCLUSIONS.­: These histologic findings may serve as indications of LCT in bone marrow biopsies. The pathologist-augmented with machine system appeared to be the most predictive, arguing for greater efforts to validate and implement these tools to further enhance physician practice.

The Spectral Underpinning of word2vec.

Word2vec introduced by Mikolov et al. is a word embedding method that is widely used in natural language processing. Despite its success and frequent use, a strong theoretical justification is still lacking. The main contribution of our paper is to propose a rigorous analysis of the highly nonlinear functional of word2vec. Our results suggest that word2vec may be primarily driven by an underlying spectral method. This insight may open the door to obtaining provable guarantees for word2vec. We support these findings by numerical simulations. One fascinating open question is whether the nonlinear properties of word2vec that are not captured by the spectral method are beneficial and, if so, by what mechanism.

Multiplexed (18-Plex) Measurement of Signaling Targets and Cytotoxic T Cells in Trastuzumab-Treated Patients using Imaging Mass Cytometry.

PURPOSE: Imaging mass cytometry (IMC) uses metal-conjugated antibodies to provide multidimensional, objective measurement of protein targets. We used this high-throughput platform to perform an 18-plex assessment of HER2 ICD/ECD, cytotoxic T-cell infiltration and other structural and signaling proteins in a cohort of patients treated with trastuzumab to discover associations with trastuzumab benefit. EXPERIMENTAL DESIGN: An antibody panel for detection of 18 targets (pan-cytokeratin, HER2 ICD, HER2 ECD, CD8, vimentin, cytokeratin 7, ß-catenin, HER3, MET, EGFR, ERK 1-2, MEK 1-2, PTEN, PI3K p110 α, Akt, mTOR, Ki67, and Histone H3) was used with a selection of trastuzumab-treated patients from the Hellenic Cooperative Oncology Group 10/05 trial (n = 180), and identified a case-control series. RESULTS: Patients that recurred after adjuvant treatment with trastuzumab trended toward a decreased fraction of HER2 ECD pixels over threshold compared with cases without recurrence (P = 0.057). After exclusion of the lowest HER2 expressers, 5-year recurrence events were associated with reduced total extracellular domain (ECD)/intracellular domain (ICD) ratio intensity in tumor (P = 0.044). These observations are consistent with our previous work using quantitative immunofluorescence, but represent the proof on identical cell content. We also describe the association of the ECD of HER2 with CD8 T-cell infiltration on the same slide. CONCLUSIONS: The proximity of CD8 cells as a function of the expression of the ECD of HER2 provides further evidence for the role of the immune system in the mechanism of action of trastuzumab.