Shared computational principles for language processing in humans and deep language models.

Departing from traditional linguistic models, advances in deep learning have resulted in a new type of predictive (autoregressive) deep language models (DLMs). Using a self-supervised next-word prediction task, these models generate appropriate linguistic responses in a given context. In the current study, nine participants listened to a 30-min podcast while their brain responses were recorded using electrocorticography (ECoG). We provide empirical evidence that the human brain and autoregressive DLMs share three fundamental computational principles as they process the same natural narrative: (1) both are engaged in continuous next-word prediction before word onset; (2) both match their pre-onset predictions to the incoming word to calculate post-onset surprise; (3) both rely on contextual embeddings to represent words in natural contexts. Together, our findings suggest that autoregressive DLMs provide a new and biologically feasible computational framework for studying the neural basis of language.

The "Narratives" fMRI dataset for evaluating models of naturalistic language comprehension.

The "Narratives" collection aggregates a variety of functional MRI datasets collected while human subjects listened to naturalistic spoken stories. The current release includes 345 subjects, 891 functional scans, and 27 diverse stories of varying duration totaling ~4.6 hours of unique stimuli (~43,000 words). This data collection is well-suited for naturalistic neuroimaging analysis, and is intended to serve as a benchmark for models of language and narrative comprehension. We provide standardized MRI data accompanied by rich metadata, preprocessed versions of the data ready for immediate use, and the spoken story stimuli with time-stamped phoneme- and word-level transcripts. All code and data are publicly available with full provenance in keeping with current best practices in transparent and reproducible neuroimaging.

Clinical Response of Live-Attenuated, Listeria monocytogenes Expressing Mesothelin (CRS-207) with Chemotherapy in Patients with Malignant Pleural Mesothelioma.

PURPOSE: Malignant pleural mesothelioma (MPM) is an aggressive cancer associated with poor prognosis. CRS-207 is a live-attenuated Listeria monocytogenes engineered to express mesothelin, a tumor-associated antigen highly expressed in MPM. CRS-207 induces antitumor immune responses and increases susceptibility of neoplastic cells to immune-mediated killing. PATIENTS AND METHODS: Patients with unresectable MPM, ECOG 0 or 1, and adequate organ and pulmonary function were enrolled in this multicenter, open-label phase Ib study. They received two priming infusions of 1 × 109 CFU CRS-207, followed by pemetrexed/cisplatin chemotherapy, and CRS-207 booster infusions. Primary objectives were safety and induction of immune response. Secondary/exploratory objectives included tumor response, progression-free survival (PFS), overall survival (OS), immune subset analysis, and gene-expression profiling of tumor. RESULTS: Of 35 evaluable patients, 89% (31/35) had disease control with one complete response (3%), 19 partial responses (54%), and 10 stable disease (29%). The estimated median duration of response was 5.0 months (95% CI, 3.9-11.5). The median PFS and OS were 7.5 (95% CI, 7.0-9.9) and 14.7 (95% CI, 11.2-21.9) months, respectively. Tumor size reduction was observed post-CRS-207 infusion prior to chemotherapy in 11 of 35 (31%) patients. No unexpected treatment-related serious adverse events or deaths were observed. IHC analysis of pre- and post-CRS-207 treatment tumor biopsies revealed possible reinvigoration and proliferation of T cells, increased infiltration of dendritic and natural killer cells, increased CD8:Treg ratio, and a shift from immunosuppressive M2-like to proinflammatory M1-like macrophages following CRS-207 administration. CONCLUSIONS: Combination of CRS-207 and chemotherapy induced significant changes in the local tumor microenvironment and objective tumor responses in a majority of treated patients.

Insulin Regulates Glycogen Synthesis in Human Endometrial Glands Through Increased GYS2.

Context: Glycogen synthesis is a critical metabolic function of the endometrium to prepare for successful implantation and sustain embryo development. Yet, regulation of endometrial carbohydrate metabolism is poorly characterized. Whereas glycogen synthesis is attributed to progesterone, we previously found that the metabolic B isoform of the insulin receptor is maximally expressed in secretory-phase endometrium, indicating a potential role of insulin in glucose metabolism. Objective: We sought to determine whether insulin or progesterone regulates glycogen synthesis in human endometrium. Design, Participants, Outcome Measurements: Endometrial epithelial cells were isolated from 28 healthy women and treated with insulin, medroxyprogesterone (MPA), or vehicle. Intracellular glycogen and the activation of key enzymes were quantified. Results: In epithelia, insulin induced a 4.4-fold increase in glycogen, whereas MPA did not alter glycogen content. Insulin inactivated glycogen synthase (GS) kinase 3α/ß (GSK3α/ß), relieving inhibition of GS. In a regulatory mechanism, distinct from liver and muscle, insulin also increased GS by 3.7-fold through increased GS 2 (GYS2) gene expression. Conclusions: We demonstrate that insulin, not progesterone, directly regulates glycogen synthesis through canonical acute inactivation of GSK3α/ß and noncanonical stimulation of GYS2 transcription. Persistently elevated GS enables endometrium to synthesize glycogen constitutively, independent of short-term nutrient flux, during implantation and early pregnancy. This suggests that insulin plays a key, physiological role in endometrial glucose metabolism and underlines the need to delineate the effect of maternal obesity and hyperinsulinemia on fertility and fetal development.

Quantitative Multiplex Immunohistochemistry Reveals Myeloid-Inflamed Tumor-Immune Complexity Associated with Poor Prognosis.

Here, we describe a multiplexed immunohistochemical platform with computational image processing workflows, including image cytometry, enabling simultaneous evaluation of 12 biomarkers in one formalin-fixed paraffin-embedded tissue section. To validate this platform, we used tissue microarrays containing 38 archival head and neck squamous cell carcinomas and revealed differential immune profiles based on lymphoid and myeloid cell densities, correlating with human papilloma virus status and prognosis. Based on these results, we investigated 24 pancreatic ductal adenocarcinomas from patients who received neoadjuvant GVAX vaccination and revealed that response to therapy correlated with degree of mono-myelocytic cell density and percentages of CD8+ T cells expressing T cell exhaustion markers. These data highlight the utility of in situ immune monitoring for patient stratification and provide digital image processing pipelines to the community for examining immune complexity in precious tissue sections, where phenotype and tissue architecture are preserved to improve biomarker discovery and assessment.

Endometrial Cancer-Associated FGF18 Expression Is Reduced by Bazedoxifene in Human Endometrial Stromal Cells In Vitro and in Murine Endometrium.

Endometrial cancer develops during exposure to estrogen unopposed by progesterone. Traditional formulations for menopausal hormone therapy include a progestin in women with a uterus. However, progestin exposure increases breast cancer risk in postmenopausal women. Alternatives to progestin include bazedoxifene (BZA), a selective estrogen receptor modulator, which prevents estrogen induced endometrial hyperplasia in clinical trials. Molecular mechanisms responsible for BZA's antiproliferative effect are not fully elucidated. We profiled endometrial adenocarcinoma, hyperplasia, and normal proliferative endometrium for differential expression in genes known to be regulated by estrogens or progesterone. Fibroblast growth factor (FGF)18, a paracrine growth factor promoting epithelial proliferation, was significantly increased in adenocarcinoma. Progesterone represses FGF18 by inducing heart and neural crest derivatives expressed transcript 2 (HAND2) in stromal cells. Notably, we confirmed lower HAND2 mRNA in adenocarcinoma, along with higher FGF tyrosine kinase receptor 2 and E74-like factor 5, collectively promoting FGF18 activity. We hypothesized BZA reduces epithelial proliferation by inhibiting FGF18 synthesis in stromal cells. To determine whether BZA regulates FGF18, we treated primary stromal cells with BZA or vehicle. In vitro, BZA reduced FGF18, but did not affect, HAND2. CD1 female mice received either BZA, conjugated estrogen (CE), or combined BZA/CE for 8 weeks. CE-treated mice had nearly 3-fold higher FGF18 expression. In contrast, BZA-treated mice, alone or with CE, had similar FGF18 as controls. Unexpectedly, BZA, alone or with CE, reduced HAND2 more than 80%, differing from progesterone regulation. Reduction of FGF18 is a potential mechanism by which BZA reduces endometrial proliferation and hyperplasia induced by estrogens. However, BZA works independently of HAND2, revealing a novel mechanism for progestin-free hormone therapy in postmenopausal women.

Differential Expression of IR-A, IR-B and IGF-1R in Endometrial Physiology and Distinct Signature in Adenocarcinoma.

CONTEXT: Type 2 diabetes and obesity are risk factors for endometrial hyperplasia and cancer, suggesting that hyperinsulinemia contributes to pathogenesis. Insulin action through insulin receptor (IR) splice variants IR-A and IR-B regulates cellular mitogenesis and metabolism, respectively. OBJECTIVE: We hypothesized that IR-A and IR-B are differentially regulated in normal endometrium, according to mitogenic and metabolic requirements through the menstrual cycle, as well as in endometrial hyperplasia and cancer. DESIGN: IR-A, IR-B, and IGF-1 receptor (IGF-1R) mRNA was quantified in endometrium, endometrial epithelial and stromal cells, and in vitro after hormone stimulation. SETTING: Academic center. PATIENTS: Endometrium was collected from women with regular cycles (n = 71), complex hyperplasia (n = 5), or endometrioid adenocarcinoma (n = 11). INTERVENTION(S): In vitro sex-steroid treatment. MAIN OUTCOME MEASURE(S): IR-A and IR-B expression Results: IR-A increased dramatically during the early proliferative phase, 20-fold more than IR-B. In early secretory phase, IR-B and IGF-1R expression increased, reaching maximal expression, whereas IR-A decreased. In adenocarcinoma, IR-B and IGF-1R expression was 5- to 6-fold higher than normal endometrium, whereas IR-A expression was similar to IR-B. Receptor expression was unrelated to body mass index. CONCLUSION: IR-A was elevated during the normal proliferative phase, and in endometrial hyperplasia and adenocarcinoma. The dramatic early rise of IR-A in normal endometrium indicates IR-A is the predominant isoform responsible for initial estrogen-independent endometrial proliferation as well as that of cancer. IR-B is elevated during the normal secretory phase when glucose uptake and glycogen synthesis support embryo development. Differing from other cancers, IR-B expression equals mitogenic IR-A in endometrial adenocarcinoma. Differential IR isoform expression suggests a distinct role for each in endometrial physiology and cancer.

Development of a Quantitative PCR Assay for Detection of Human Insulin-Like Growth Factor Receptor and Insulin Receptor Isoforms.

The biological activity of insulin and the insulin-like growth factor (IGF) ligands, IGF-I and IGF-II, is based in part on the relative abundance and distribution of their target receptors: the insulin receptor (IR) splice variants A (IR-A) and B (IR-B) and IGF 1 receptor (IGF-1R). However, the relative quantity of all three receptors in human tissues has never been measured together on the same scale. Due to the high homology between insulin receptor (IR)-A and IR-B proteins and lack of antibodies that discern the two IR splice variants, their mRNA sequence is the most reliable means of distinguishing between the receptors. Hence, highly specific primers for IR-A, IR-B, and IGF-1R mRNA were designed to accurately detect all three receptors by quantitative RT-PCR and enable direct quantification of relative receptor expression levels. A standard concentration curve of cDNA from each receptor was performed. Assay specificity was tested using competition assays and postamplification analysis by gel electrophoresis and cloning. Forward and reverse primer concentrations were optimized to ensure equal efficiencies across primer pairs. This assay enables a specific molecular signature of IGF/insulin signaling receptors to be assayed in different tissues, cell types, or cancers.

mRNA-binding protein TIA-1 reduces cytokine expression in human endometrial stromal cells and is down-regulated in ectopic endometrium.

BACKGROUND: Cytokines and growth factors play important roles in endometrial function and the pathogenesis of endometriosis. mRNAs encoding cytokines and growth factors undergo rapid turnover; primarily mediated by adenosine- and uridine-rich elements (AREs) located in their 3'-untranslated regions. T-cell intracellular antigen (TIA-1), an mRNA-binding protein, binds to AREs in target transcripts, leading to decreased gene expression. OBJECTIVE: The purpose of this article was to determine whether TIA-1 plays a role in the regulation of endometrial cytokine and growth factor expression during the normal menstrual cycle and whether TIA-1 expression is altered in women with endometriosis. METHODS: Eutopic endometrial tissue obtained from women without endometriosis (n = 30) and eutopic and ectopic endometrial tissues from women with endometriosis (n = 17) were immunostained for TIA-1. Staining intensities were evaluated by histological scores (HSCOREs). The regulation of endometrial TIA-1 expression by immune factors and steroid hormones was studied by treating primary cultured human endometrial stromal cells (HESCs) with vehicle, lipopolysaccharide, TNF-α, IL-6, estradiol, or progesterone, followed by protein blot analyses. HESCs were engineered to over- or underexpress TIA-1 to test whether TIA-1 regulates IL-6 or TNF-α expression in these cells. RESULTS: We found that TIA-1 is expressed in endometrial stromal and glandular cells throughout the menstrual cycle and that this expression is significantly higher in the perimenstrual phase. In women with endometriosis, TIA-1 expression in eutopic and ectopic endometrium was reduced compared with TIA-1 expression in eutopic endometrium of unaffected control women. Lipopolysaccharide and TNF-α increased TIA-1 expression in HESCs in vitro, whereas IL-6 or steroid hormones had no effect. In HESCs, down-regulation of TIA-1 resulted in elevated IL-6 and TNF-α expression, whereas TIA-1 overexpression resulted in decreased IL-6 and TNF-α expression. CONCLUSIONS: Endometrial TIA-1 is regulated throughout the menstrual cycle, TIA-1 modulates the expression of immune factors in endometrial cells, and downregulation of TIA-1 may contribute to the pathogenesis of endometriosis.

Derivation of HLA types from shotgun sequence datasets.

The human leukocyte antigen (HLA) is key to many aspects of human physiology and medicine. All current sequence-based HLA typing methodologies are targeted approaches requiring the amplification of specific HLA gene segments. Whole genome, exome and transcriptome shotgun sequencing can generate prodigious data but due to the complexity of HLA loci these data have not been immediately informative regarding HLA genotype. We describe HLAminer, a computational method for identifying HLA alleles directly from shotgun sequence datasets (http://www.bcgsc.ca/platform/bioinfo/software/hlaminer). This approach circumvents the additional time and cost of generating HLA-specific data and capitalizes on the increasing accessibility and affordability of massively parallel sequencing.

IL-18 induces emphysema and airway and vascular remodeling via IFN-γ, IL-17A, and IL-13.

RATIONALE: Chronic obstructive pulmonary disease (COPD) is characterized by chronic inflammation, alveolar destruction, and airway and vascular remodeling. However, the mechanisms that lead to these diverse alterations have not been defined. OBJECTIVES: We hypothesized that IL-18 plays a central role in the pathogenesis of these lesions. METHODS: We generated and characterized lung-specific, inducible IL-18 transgenic mice. MEASUREMENTS AND MAIN RESULTS: Here we demonstrate that the expression of IL-18 in the mature murine lung induces inflammation that is associated with the accumulation of CD4(+), CD8(+), CD19(+), and NK1.1(+) cells; emphysema; mucus metaplasia; airway fibrosis; vascular remodeling; and right ventricle cardiac hypertrophy. We also demonstrate that IL-18 induces type 1, type 2, and type 17 cytokines with IFN-γ-inhibiting macrophage, lymphocyte, and eosinophil accumulation while stimulating alveolar destruction and genes associated with cell cytotoxicity and IL-13 and IL-17A inducing mucus metaplasia, airway fibrosis, and vascular remodeling. We also highlight interactions between these responses with IL-18 inducing IL-13 via an IL-17A-dependent mechanism and the type 1 and type17/type 2 responses counterregulating each another. CONCLUSIONS: These studies define the spectrum of inflammatory, parenchymal, airway, and vascular alterations that are induced by pulmonary IL-18; highlight the similarities between these responses and the lesions in COPD; and define the selective roles that type 1, type 2, and type 17 responses play in the generation of IL-18-induced pathologies.

Exhaustive T-cell repertoire sequencing of human peripheral blood samples reveals signatures of antigen selection and a directly measured repertoire size of at least 1 million clonotypes.

Massively parallel sequencing is a useful approach for characterizing T-cell receptor diversity. However, immune receptors are extraordinarily difficult sequencing targets because any given receptor variant may be present in very low abundance and may differ legitimately by only a single nucleotide. We show that the sensitivity of sequence-based repertoire profiling is limited by both sequencing depth and sequencing accuracy. At two timepoints, 1 wk apart, we isolated bulk PBMC plus naïve (CD45RA+/CD45RO-) and memory (CD45RA-/CD45RO+) T-cell subsets from a healthy donor. From T-cell receptor beta chain (TCRB) mRNA we constructed and sequenced multiple libraries to obtain a total of 1.7 billion paired sequence reads. The sequencing error rate was determined empirically and used to inform a high stringency data filtering procedure. The error filtered data yielded 1,061,522 distinct TCRB nucleotide sequences from this subject which establishes a new, directly measured, lower limit on individual T-cell repertoire size and provides a useful reference set of sequences for repertoire analysis. TCRB nucleotide sequences obtained from two additional donors were compared to those from the first donor and revealed limited sharing (up to 1.1%) of nucleotide sequences among donors, but substantially higher sharing (up to 14.2%) of inferred amino acid sequences. For each donor, shared amino acid sequences were encoded by a much larger diversity of nucleotide sequences than were unshared amino acid sequences. We also observed a highly statistically significant association between numbers of shared sequences and shared HLA class I alleles.