A "data sharing trust" model for rapid, collaborative science.

Complex datasets provide opportunities for discoveries beyond their initial scope. Effective and rapid data sharing and management practices are crucial to realize this potential; however, they are harder to implement than post-publication access. Here, we introduce the concept of a "data sharing trust" to maximize the value of large datasets.

Systemic Immunity Is Required for Effective Cancer Immunotherapy.

Immune responses involve coordination across cell types and tissues. However, studies in cancer immunotherapy have focused heavily on local immune responses in the tumor microenvironment. To investigate immune activity more broadly, we performed an organism-wide study in genetically engineered cancer models using mass cytometry. We analyzed immune responses in several tissues after immunotherapy by developing intuitive models for visualizing single-cell data with statistical inference. Immune activation was evident in the tumor and systemically shortly after effective therapy was administered. However, during tumor rejection, only peripheral immune cells sustained their proliferation. This systemic response was coordinated across tissues and required for tumor eradication in several immunotherapy models. An emergent population of peripheral CD4 T cells conferred protection against new tumors and was significantly expanded in patients responding to immunotherapy. These studies demonstrate the critical impact of systemic immune responses that drive tumor rejection.

Epigenomic-Guided Mass Cytometry Profiling Reveals Disease-Specific Features of Exhausted CD8 T Cells.

Exhausted CD8 T (Tex) cells are immunotherapy targets in chronic infection and cancer, but a comprehensive assessment of Tex cell diversity in human disease is lacking. Here, we developed a transcriptomic- and epigenetic-guided mass cytometry approach to define core exhaustion-specific genes and disease-induced changes in Tex cells in HIV and human cancer. Single-cell proteomic profiling identified 9 distinct Tex cell clusters using phenotypic, functional, transcription factor, and inhibitory receptor co-expression patterns. An exhaustion severity metric was developed and integrated with high-dimensional phenotypes to define Tex cell clusters that were present in healthy subjects, common across chronic infection and cancer or enriched in either disease, linked to disease severity, and changed with HIV therapy. Combinatorial patterns of immunotherapy targets on different Tex cell clusters were also defined. This approach and associated datasets present a resource for investigating human Tex cell biology, with implications for immune monitoring and immunomodulation in chronic infections, autoimmunity, and cancer.

High response rate to PD-1 blockade in desmoplastic melanomas.

Desmoplastic melanoma is a rare subtype of melanoma characterized by dense fibrous stroma, resistance to chemotherapy and a lack of actionable driver mutations, and is highly associated with ultraviolet light-induced DNA damage. We analysed sixty patients with advanced desmoplastic melanoma who had been treated with antibodies to block programmed cell death 1 (PD-1) or PD-1 ligand (PD-L1). Objective tumour responses were observed in forty-two of the sixty patients (70%; 95% confidence interval 57-81%), including nineteen patients (32%) with a complete response. Whole-exome sequencing revealed a high mutational load and frequent NF1 mutations (fourteen out of seventeen cases) in these tumours. Immunohistochemistry analysis from nineteen desmoplastic melanomas and thirteen non-desmoplastic melanomas revealed a higher percentage of PD-L1-positive cells in the tumour parenchyma in desmoplastic melanomas (P = 0.04); these cells were highly associated with increased CD8 density and PD-L1 expression in the tumour invasive margin. Therefore, patients with advanced desmoplastic melanoma derive substantial clinical benefit from PD-1 or PD-L1 immune checkpoint blockade therapy, even though desmoplastic melanoma is defined by its dense desmoplastic fibrous stroma. The benefit is likely to result from the high mutational burden and a frequent pre-existing adaptive immune response limited by PD-L1 expression.

A Meta-Analysis Approach to Gene Regulatory Network Inference Identifies Key Regulators of Cardiovascular Diseases.

Cardiovascular diseases (CVDs) represent a major concern for global health, whose mechanistic understanding is complicated by a complex interplay between genetic predisposition and environmental factors. Specifically, heart failure (HF), encompassing dilated cardiomyopathy (DC), ischemic cardiomyopathy (ICM), and hypertrophic cardiomyopathy (HCM), is a topic of substantial interest in basic and clinical research. Here, we used a Partial Correlation Coefficient-based algorithm (PCC) within the context of a meta-analysis framework to construct a Gene Regulatory Network (GRN) that identifies key regulators whose activity is perturbed in Heart Failure. By integrating data from multiple independent studies, our approach unveiled crucial regulatory associations between transcription factors (TFs) and structural genes, emphasizing their pivotal roles in regulating metabolic pathways, such as fatty acid metabolism, oxidative stress response, epithelial-to-mesenchymal transition, and coagulation. In addition to known associations, our analysis also identified novel regulators, including the identification of TFs FPM315 and OVOL2, which are implicated in dilated cardiomyopathies, and TEAD1 and TEAD2 in both dilated and ischemic cardiomyopathies. Moreover, we uncovered alterations in adipogenesis and oxidative phosphorylation pathways in hypertrophic cardiomyopathy and discovered a role for IL2 STAT5 signaling in heart failure. Our findings underscore the importance of TF activity in the initiation and progression of cardiac disease, highlighting their potential as pharmacological targets.

CD40 agonistic monoclonal antibody APX005M (sotigalimab) and chemotherapy, with or without nivolumab, for the treatment of metastatic pancreatic adenocarcinoma: an open-label, multicentre, phase 1b study.

BACKGROUND: Standard chemotherapy remains inadequate in metastatic pancreatic adenocarcinoma. Combining an agonistic CD40 monoclonal antibody with chemotherapy induces T-cell-dependent tumour regression in mice and improves survival. In this study, we aimed to evaluate the safety of combining APX005M (sotigalimab) with gemcitabine plus nab-paclitaxel, with and without nivolumab, in patients with pancreatic adenocarcinoma to establish the recommended phase 2 dose. METHODS: This non-randomised, open-label, multicentre, four-cohort, phase 1b study was done at seven academic hospitals in the USA. Eligible patients were adults aged 18 years and older with untreated metastatic pancreatic adenocarcinoma, Eastern Cooperative Oncology Group performance status score of 0-1, and measurable disease by Response Evaluation Criteria in Solid Tumors version 1.1. All patients were treated with 1000 mg/m2 intravenous gemcitabine and 125 mg/m2 intravenous nab-paclitaxel. Patients received 0·1 mg/kg intravenous APX005M in cohorts B1 and C1 and 0·3 mg/kg in cohorts B2 and C2. In cohorts C1 and C2, patients also received 240 mg intravenous nivolumab. Primary endpoints comprised incidence of adverse events in all patients who received at least one dose of any study drug, incidence of dose-limiting toxicities (DLTs) in all patients who had a DLT or received at least two doses of gemcitabine plus nab-paclitaxel and one dose of APX005M during cycle 1, and establishing the recommended phase 2 dose of intravenous APX005M. Objective response rate in the DLT-evaluable population was a key secondary endpoint. This trial (PRINCE, PICI0002) is registered with ClinicalTrials.gov, NCT03214250 and is ongoing. FINDINGS: Between Aug 22, 2017, and July 10, 2018, of 42 patients screened, 30 patients were enrolled and received at least one dose of any study drug; 24 were DLT-evaluable with median follow-up 17·8 months (IQR 16·0-19·4; cohort B1 22·0 months [21·4-22·7], cohort B2 18·2 months [17·0-18·9], cohort C1 17·9 months [14·3-19·7], cohort C2 15·9 months [12·7-16·1]). Two DLTs, both febrile neutropenia, were observed, occurring in one patient each for cohorts B2 (grade 3) and C1 (grade 4). The most common grade 3-4 treatment-related adverse events were lymphocyte count decreased (20 [67%]; five in B1, seven in B2, four in C1, four in C2), anaemia (11 [37%]; two in B1, four in B2, four in C1, one in C2), and neutrophil count decreased (nine [30%]; three in B1, three in B2, one in C1, two in C2). 14 (47%) of 30 patients (four each in B1, B2, C1; two in C2) had a treatment-related serious adverse event. The most common serious adverse event was pyrexia (six [20%] of 30; one in B2, three in C1, two in C2). There were two chemotherapy-related deaths due to adverse events: one sepsis in B1 and one septic shock in C1. The recommended phase 2 dose of APX005M was 0·3 mg/kg. Responses were observed in 14 (58%) of 24 DLT-evaluable patients (four each in B1, C1, C2; two in B2). INTERPRETATION: APX005M and gemcitabine plus nab-paclitaxel, with or without nivolumab, is tolerable in metastatic pancreatic adenocarcinoma and shows clinical activity. If confirmed in later phase trials, this treatment regimen could replace chemotherapy-only standard of care in this population. FUNDING: Parker Institute for Cancer Immunotherapy, Cancer Research Institute, and Bristol Myers Squibb.

Highly multiplexed simultaneous detection of RNAs and proteins in single cells.

To enable the detection of expression signatures specific to individual cells, we developed PLAYR (proximity ligation assay for RNA), a method for highly multiplexed transcript quantification by flow and mass cytometry that is compatible with standard antibody staining. When used with mass cytometry, PLAYR allowed for the simultaneous quantification of more than 40 different mRNAs and proteins. In primary cells, we quantified multiple transcripts, with the identity and functional state of each analyzed cell defined on the basis of the expression of a separate set of transcripts or proteins. By expanding high-throughput deep phenotyping of cells beyond protein epitopes to include RNA expression, PLAYR opens a new avenue for the characterization of cellular metabolism.

GateFinder: projection-based gating strategy optimization for flow and mass cytometry.

Motivation: High-parameter single-cell technologies can reveal novel cell populations of interest, but studying or validating these populations using lower-parameter methods remains challenging. Results: Here, we present GateFinder, an algorithm that enriches high-dimensional cell types with simple, stepwise polygon gates requiring only two markers at a time. A series of case studies of complex cell types illustrates how simplified enrichment strategies can enable more efficient assays, reveal novel biomarkers and clarify underlying biology. Availability and implementation: The GateFinder algorithm is implemented as a free and open-source package for BioConductor: https://nalab.stanford.edu/gatefinder. Supplementary information: Supplementary data are available at Bioinformatics online.

Mapping the Fetomaternal Peripheral Immune System at Term Pregnancy.

Preterm labor and infections are the leading causes of neonatal deaths worldwide. During pregnancy, immunological cross talk between the mother and her fetus is critical for the maintenance of pregnancy and the delivery of an immunocompetent neonate. A precise understanding of healthy fetomaternal immunity is the important first step to identifying dysregulated immune mechanisms driving adverse maternal or neonatal outcomes. This study combined single-cell mass cytometry of paired peripheral and umbilical cord blood samples from mothers and their neonates with a graphical approach developed for the visualization of high-dimensional data to provide a high-resolution reference map of the cellular composition and functional organization of the healthy fetal and maternal immune systems at birth. The approach enabled mapping of known phenotypical and functional characteristics of fetal immunity (including the functional hyperresponsiveness of CD4+ and CD8+ T cells and the global blunting of innate immune responses). It also allowed discovery of new properties that distinguish the fetal and maternal immune systems. For example, examination of paired samples revealed differences in endogenous signaling tone that are unique to a mother and her offspring, including increased ERK1/2, MAPK-activated protein kinase 2, rpS6, and CREB phosphorylation in fetal Tbet+CD4+ T cells, CD8+ T cells, B cells, and CD56loCD16+ NK cells and decreased ERK1/2, MAPK-activated protein kinase 2, and STAT1 phosphorylation in fetal intermediate and nonclassical monocytes. This highly interactive functional map of healthy fetomaternal immunity builds the core reference for a growing data repository that will allow inferring deviations from normal associated with adverse maternal and neonatal outcomes.

Exploring the diversity of SPRY/B30.2-mediated interactions.

The SPla/Ryanodine receptor (SPRY)/B30.2 domain is one of the most common folds in higher eukaryotes. The human genome encodes 103 SPRY/B30.2 domains, several of which are involved in the immune response. Approximately 45% of human SPRY/B30.2-containing proteins are E3 ligases. The role and function of the majority of SPRY/B30.2 domains are still poorly understood, however, in several cases mutations in this domain have been linked to congenital disorders. The recent characterization of SPRY/B30.2-mediated protein interactions has provided evidence for a role of this domain as an adaptor module to assemble macromolecular complexes, analogous to Src homology (SH)2, SH3, and WW domains. However, functional and structural evidence suggests that SPRY/B30.2 is a more versatile fold, allowing a wide range of binding modes.

A Proteome-wide Domain-centric Perspective on Protein Phosphorylation.

Phosphorylation is a widespread post-translational modification that modulates the function of a large number of proteins. Here we show that a significant proportion of all the domains in the human proteome is significantly enriched or depleted in phosphorylation events. A substantial improvement in phosphosites prediction is achieved by leveraging this observation, which has not been tapped by existing methods. Phosphorylation sites are often not shared between multiple occurrences of the same domain in the proteome, even when the phosphoacceptor residue is conserved. This is partly because of different functional constraints acting on the same domain in different protein contexts. Moreover, by augmenting domain alignments with structural information, we were able to provide direct evidence that phosphosites in protein-protein interfaces need not be positionally conserved, likely because they can modulate interactions simply by sitting in the same general surface area.

Regulation dynamics of Leishmania differentiation: deconvoluting signals and identifying phosphorylation trends.

Leishmania are obligatory intracellular parasitic protozoa that cause a wide range of diseases in humans, cycling between extracellular promastigotes in the mid-gut of sand flies and intracellular amastigotes in the phagolysosomes of mammalian macrophages. Although many of the molecular mechanisms of development inside macrophages remain a mystery, the development of a host-free system that simulates phagolysosome conditions (37 °C and pH 5.5) has provided new insights into these processes. The time course of promastigote-to-amastigote differentiation can be divided into four morphologically distinct phases: I, signal perception (0-5 h after exposure); II, movement cessation and aggregation (5-10 h); III, amastigote morphogenesis (10-24 h); and IV, maturation (24-120 h). Transcriptomic and proteomic analyses have indicated that differentiation is a coordinated process that results in adaptation to life inside phagolysosomes. Recent phosphoproteomic analysis revealed extensive differences in phosphorylation between promastigotes and amastigotes and identified stage-specific phosphorylation motifs. We hypothesized that the differentiation signal activates a phosphorylation pathway that initiates Leishmania transformation, and here we used isobaric tags for relative and absolute quantitation to interrogate the dynamics of changes in the phosphorylation profile during Leishmania donovani promastigote-to-amastigote differentiation. Analysis of 163 phosphopeptides (from 106 proteins) revealed six distinct kinetic profiles; with increases in phosphorylation predominated during phases I and III, whereas phases II and IV were characterized by greater dephosphorylation. Several proteins (including a protein kinase) were phosphorylated in phase I after exposure to the complete differentiation signal (i.e. signal-specific; 37 °C and pH 5.5), but not after either of the physical parameters separately. Several other protein kinases (including regulatory subunits) and phosphatases also showed changes in phosphorylation during differentiation. This work constitutes the first genome-scale interrogation of phosphorylation dynamics in a parasitic protozoa, revealing the outline of a signaling pathway during Leishmania differentiation. The mass spectrometry proteomics data have been deposited to the ProteomeXchange Consortium (identifier PXD000671). Data can be viewed using ProteinPilot™ software.

Single-cell systems-level analysis of human Toll-like receptor activation defines a chemokine signature in patients with systemic lupus erythematosus.

BACKGROUND: Activation of Toll-like receptors (TLRs) induces inflammatory responses involved in immunity to pathogens and autoimmune pathogenesis, such as in patients with systemic lupus erythematosus (SLE). Although TLRs are differentially expressed across the immune system, a comprehensive analysis of how multiple immune cell subsets respond in a system-wide manner has not been described. OBJECTIVE: We sought to characterize TLR activation across multiple immune cell subsets and subjects, with the goal of establishing a reference framework against which to compare pathologic processes. METHODS: Peripheral whole-blood samples were stimulated with TLR ligands and analyzed by means of mass cytometry simultaneously for surface marker expression, activation states of intracellular signaling proteins, and cytokine production. We developed a novel data visualization tool to provide an integrated view of TLR signaling networks with single-cell resolution. We studied 17 healthy volunteer donors and 8 patients with newly diagnosed and untreated SLE. RESULTS: Our data revealed the diversity of TLR-induced responses within cell types, with TLR ligand specificity. Subsets of natural killer cells and T cells selectively induced nuclear factor κ light chain enhancer of activated B cells in response to TLR2 ligands. CD14(hi) monocytes exhibited the most polyfunctional cytokine expression patterns, with more than 80 distinct cytokine combinations. Monocytic TLR-induced cytokine patterns were shared among a group of healthy donors, with minimal intraindividual and interindividual variability. Furthermore, autoimmune disease altered baseline cytokine production; newly diagnosed untreated SLE patients shared a distinct monocytic chemokine signature, despite clinical heterogeneity. CONCLUSION: Mass cytometry defined a systems-level reference framework for human TLR activation, which can be applied to study perturbations in patients with inflammatory diseases, such as SLE.

Phosphoproteomic analysis of differentiating Leishmania parasites reveals a unique stage-specific phosphorylation motif.

Protists of the genus Leishmania are obligatory intracellular parasites that cause a wide range of cutaneous, mucocutaneous, and visceral diseases in humans. They cycle between phagolysosomes of mammalian macrophages and the sand fly midgut, proliferating as intracellular amastigotes and extracellular promastigotes, respectively. Exposure to a lysosomal environment, i.e. acidic pH and body temperature, signals promastigotes to differentiate into amastigotes. Time course analyses indicated that Leishmania differentiation is a highly regulated and coordinated process. However, the role of posttranslational events such as protein phosphorylation in this process is still unknown. Herein, we analyzed and compared the phosphoproteomes of L. donovani amastigotes and promastigotes using an axenic host-free system that simulates parasite differentiation. Shotgun phosphopeptide analysis revealed 1614 phosphorylation residues (p-sites) corresponding to 627 proteins. The analysis indicated that the majority of the p-sites are stage-specific. Serine phosphorylation in a previously identified trypanosomatid-specific "SF" motif was significantly enriched in amastigotes. We identified a few phosophotyrosines (pY), mostly in proteins known to participate in signal transduction pathways. The analysis indicated that Leishmania contains proteins with multiple p-sites that are phosphorylated at distinct stages of the life cycle. For over half of the phosphorylation events, changes in phosphoprotein abundance did not positively correlate with changes in protein abundance, suggesting functional regulation. This study compares, for the first time, the phosphoproteins of L. donovani axenic promastigotes and amastigotes and provides the largest data set of the Leishmania phosphoproteome to date.

Alternative splicing tends to avoid partial removals of protein-protein interaction sites.

BACKGROUND: Anecdotal evidence of the involvement of alternative splicing (AS) in the regulation of protein-protein interactions has been reported by several studies. AS events have been shown to significantly occur in regions where a protein interaction domain or a short linear motif is present. Several AS variants show partial or complete loss of interface residues, suggesting that AS can play a major role in the interaction regulation by selectively targeting the protein binding sites. In the present study we performed a statistical analysis of the alternative splicing of a non-redundant dataset of human protein-protein interfaces known at molecular level to determine the importance of this way of modulation of protein-protein interactions through AS. RESULTS: Using a Cochran-Mantel-Haenszel chi-square test we demonstrated that the alternative splicing-mediated partial removal of both heterodimeric and homodimeric binding sites occurs at lower frequencies than expected, and this holds true even if we consider only those isoforms whose sequence is less different from that of the canonical protein and which therefore allow to selectively regulate functional regions of the protein. On the other hand, large removals of the binding site are not significantly prevented, possibly because they are associated to drastic structural changes of the protein. The observed protection of the binding sites from AS is not preferentially directed towards putative hot spot interface residues, and is widespread to all protein functional classes. CONCLUSIONS: Our findings indicate that protein-protein binding sites are generally protected from alternative splicing-mediated partial removals. However, some cases in which the binding site is selectively removed exist, and here we discuss one of them.

Mapping the human phosphatome on growth pathways.

Large-scale siRNA screenings allow linking the function of poorly characterized genes to phenotypic readouts. According to this strategy, genes are associated with a function of interest if the alteration of their expression perturbs the phenotypic readouts. However, given the intricacy of the cell regulatory network, the mapping procedure is low resolution and the resulting models provide little mechanistic insights. We have developed a new strategy that combines multiparametric analysis of cell perturbation with logic modeling to achieve a more detailed functional mapping of human genes onto complex pathways. A literature-derived optimized model is used to infer the cell activation state following upregulation or downregulation of the model entities. By matching this signature with the experimental profile obtained in the high-throughput siRNA screening it is possible to infer the target of each protein, thus defining its 'entry point' in the network. By this novel approach, 41 phosphatases that affect key growth pathways were identified and mapped onto a human epithelial cell-specific growth model, thus providing insights into the mechanisms underlying their function.

Phosphate binding sites identification in protein structures.

Nearly half of known protein structures interact with phosphate-containing ligands, such as nucleotides and other cofactors. Many methods have been developed for the identification of metal ions-binding sites and some for bigger ligands such as carbohydrates, but none is yet available for the prediction of phosphate-binding sites. Here we describe Pfinder, a method that predicts binding sites for phosphate groups, both in the form of ions or as parts of other non-peptide ligands, in proteins of known structure. Pfinder uses the Query3D local structural comparison algorithm to scan a protein structure for the presence of a number of structural motifs identified for their ability to bind the phosphate chemical group. Pfinder has been tested on a data set of 52 proteins for which both the apo and holo forms were available. We obtained at least one correct prediction in 63% of the holo structures and in 62% of the apo. The ability of Pfinder to recognize a phosphate-binding site in unbound protein structures makes it an ideal tool for functional annotation and for complementing docking and drug design methods. The Pfinder program is available at http://pdbfun.uniroma2.it/pfinder.

Phosfinder: a web server for the identification of phosphate-binding sites on protein structures.

Phosfinder is a web server for the identification of phosphate binding sites in protein structures. Phosfinder uses a structural comparison algorithm to scan a query structure against a set of known 3D phosphate binding motifs. Whenever a structural similarity between the query protein and a phosphate binding motif is detected, the phosphate bound by the known motif is added to the protein structure thus representing a putative phosphate binding site. Predicted binding sites are then evaluated according to (i) their position with respect to the query protein solvent-excluded surface and (ii) the conservation of the binding residues in the protein family. The server accepts as input either the PDB code of the protein to be analyzed or a user-submitted structure in PDB format. All the search parameters are user modifiable. Phosfinder outputs a list of predicted binding sites with detailed information about their structural similarity with known phosphate binding motifs, and the conservation of the residues involved. A graphical applet allows the user to visualize the predicted binding sites on the query protein structure. The results on a set of 52 apo/holo structure pairs show that the performance of our method is largely unaffected by ligand-induced conformational changes. Phosfinder is available at http://phosfinder.bio.uniroma2.it.

Phospho3D 2.0: an enhanced database of three-dimensional structures of phosphorylation sites.

Phospho3D is a database of three-dimensional (3D) structures of phosphorylation sites (P-sites) derived from the Phospho.ELM database, which also collects information on the residues surrounding the P-site in space (3D zones). The database also provides the results of a large-scale structural comparison of the 3D zones versus a representative dataset of structures, thus associating to each P-site a number of structurally similar sites. The new version of Phospho3D presents an 11-fold increase in the number of 3D sites and incorporates several additional features, including new structural descriptors, the possibility of selecting non-redundant sets of 3D structures and the availability for download of non-redundant sets of structurally annotated P-sites. Moreover, it features P3Dscan, a new functionality that allows the user to submit a protein structure and scan it against the 3D zones collected in the Phospho3D database. Phospho3D version 2.0 is available at: http://www.phospho3d.org/.

Identification of binding pockets in protein structures using a knowledge-based potential derived from local structural similarities.

BACKGROUND: The identification of ligand binding sites is a key task in the annotation of proteins with known structure but uncharacterized function. Here we describe a knowledge-based method exploiting the observation that unrelated binding sites share small structural motifs that bind the same chemical fragments irrespective of the nature of the ligand as a whole. RESULTS: PDBinder compares a query protein against a library of binding and non-binding protein surface regions derived from the PDB. The results of the comparison are used to derive a propensity value for each residue which is correlated with the likelihood that the residue is part of a ligand binding site. The method was applied to two different problems: i) the prediction of ligand binding residues and ii) the identification of which surface cleft harbours the binding site. In both cases PDBinder performed consistently better than existing methods. PDBinder has been trained on a non-redundant set of 1356 high-quality protein-ligand complexes and tested on a set of 239 holo and apo complex pairs. We obtained an MCC of 0.313 on the holo set with a PPV of 0.413 while on the apo set we achieved an MCC of 0.271 and a PPV of 0.372. CONCLUSIONS: We show that PDBinder performs better than existing methods. The good performance on the unbound proteins is extremely important for real-world applications where the location of the binding site is unknown. Moreover, since our approach is orthogonal to those used in other programs, the PDBinder propensity value can be integrated in other algorithms further increasing the final performance.