Design, validation and implementation of an automated e-alert for acute kidney injury: 6-month pilot study shows increased awareness.

BACKGROUND: Acute kidney injury (AKI) is defined as a sudden episode of kidney failure but is known to be under-recognized by healthcare professionals. The Kidney Disease Improving Global Outcome (KDIGO) guidelines have formulated criteria to facilitate AKI diagnosis by comparing changes in plasma creatinine measurements (PCr). To improve AKI awareness, we implemented these criteria as an electronic alert (e-alert), in our electronic health record (EHR) system. METHODS: For every new PCr measurement measured in the University Medical Center Utrecht that triggered the e-alert, we provided the physician with actionable insights in the form of a memo, to improve or stabilize kidney function. Since e-alerts qualify for software as a medical device (SaMD), we designed, implemented and validated the e-alert according to the European Union In Vitro Diagnostic Regulation (IVDR). RESULTS: We evaluated the impact of the e-alert using pilot data six months before and after implementation. 2,053 e-alerts of 866 patients were triggered in the before implementation, and 1,970 e-alerts of 853 patients were triggered after implementation. We found improvements in AKI awareness as measured by (1) 2 days PCr follow up (56.6-65.8%, p-value: 0.003), and (2) stop of nephrotoxic medication within 7 days of the e-alert (59.2-63.2%, p-value: 0.002). CONCLUSION: Here, we describe the design and implementation of the e-alert in line with the IVDR, leveraging a multi-disciplinary team consisting of physicians, clinical chemists, data managers and data scientists, and share our firsts results that indicate an improved awareness among treating physicians.

Prognostic gene expression analysis in a retrospective, multinational cohort of 155 multiple myeloma patients treated outside clinical trials.

OBJECTIVES: Typically, prognostic capability of gene expression profiling (GEP) is studied in the context of clinical trials, for which 50%-80% of patients are not eligible, possibly limiting the generalizability of findings to routine practice. Here, we evaluate GEP analysis outside clinical trials, aiming to improve clinical risk assessment of multiple myeloma (MM) patients. METHODS: A total of 155 bone marrow samples from MM patients were collected from which RNA was analyzed by microarray. Sixteen previously developed GEP-based markers were evaluated, combined with survival data, and studied using Cox proportional hazard regression. RESULTS: Gene expression profiling-based markers SKY92 and the PR-cluster were shown to be independent prognostic factors for survival, with hazard ratios and 95% confidence interval of 3.6 [2.0-6.8] (P < .001) and 5.8 [2.7-12.7] (P < .01) for overall survival (OS). A multivariate model proved only SKY92 and the PR-cluster to be independent prognostic factors compared to cytogenetic high-risk patients, the International Staging System (ISS), and revised ISS. A substantial number of high-risk individuals could be further identified when SKY92 was added to the cytogenetic, ISS, or R-ISS. In the cytogenetic standard-risk group, ISS I/II, and R-ISS I/II, 13%, 23%, and 23% of patients with adverse survivals were identified. CONCLUSIONS: For the first time, this study confirmed the prognostic value of GEP markers outside clinical trials. Conventional prognostic models to define high-risk MM are improved by the incorporation of GEP markers.

Gene expression profiling impacts treatment decision making in newly diagnosed multiple myeloma patients in the prospective PROMMIS trial.

Multiple myeloma (MM) is a heterogeneous hematologic malignancy associated with several risk factors including genetic aberrations which impact disease response and survival. Thorough risk classification is essential to select the best clinical strategy to optimize outcomes. The SKY92 molecular signature classifies patients as standard- or high-risk for progression. The PRospective Observational Multiple Myeloma Impact Study (PROMMIS; NCT02911571) measures impact of SKY92 on risk classification and treatment plan. Newly diagnosed MM patients had bone marrow aspirates analyzed for SKY92. Physicians completed a questionnaire for each patient capturing risk classification, hypothetical treatment plan, and physician confidence in the treatment plan, before and after unblinding SKY92. One hundred forty seven MM patients were enrolled. Before unblinding SKY92, physicians regarded 74 (50%) patients as clinical standard-risk. After unblinding SKY92, 16 patients were re-assigned as high-risk by the physician, and for 15 of them treatment strategy was impacted, resulting in an escalated treatment plan. For the 73 (50%) clinical high-risk patients, SKY92 indicated 46 patients to be standard-risk; for 31 of these patients the treatment strategy was impacted consistent with a de-escalation of risk. Overall, SKY92 impacted treatment decisions in 37% of patients (p < 0.001). For clinical decision-making, physicians incorporated SKY92, and the final assigned clinical risk was in line with SKY92 for 89% of patients. Furthermore, SKY92 significantly increased the confidence of the physicians' treatment decisions (p < 0.001). This study shows potential added value of SKY92 in MM for treatment decision making.

Transportability and Implementation Challenges of Early Warning Scores for Septic Shock in the ICU: A Perspective on the TREWScore.

The increased use of electronic health records (EHRs) has improved the availability of routine care data for medical research. Combined with machine learning techniques this has spurred the development of early warning scores (EWSs) in hospitals worldwide. EWSs are commonly used in the hospital where they have been developed, yet few have been transported to external settings and/or internationally. In this perspective, we describe our experiences in implementing the TREWScore, a septic shock EWS, and the transportability challenges regarding domain, predictors, and clinical outcome we faced. We used data of 53,330 ICU stays from Medical Information Mart for Intensive Care-III (MIMIC-III) and 18,013 ICU stays from the University Medical Center (UMC) Utrecht, including 17,023 (31.9%) and 2,557 (14.2%) cases of sepsis, respectively. The MIMIC-III and UMC populations differed significantly regarding the length of stay (6.9 vs. 9.0 days) and hospital mortality (11.6% vs. 13.6%). We mapped all 54 TREWScore predictors to the UMC database: 31 were readily available, seven required unit conversion, 14 had to be engineered, one predictor required text mining, and one predictor could not be mapped. Lastly, we classified sepsis cases for septic shock using the sepsis-2 criteria. Septic shock populations (UMC 31.3% and MIMIC-III 23.3%) and time to shock events showed significant differences between the two cohorts. In conclusion, we identified challenges to transportability and implementation regarding domain, predictors, and clinical outcome when transporting EWS between hospitals across two continents. These challenges need to be systematically addressed to improve model transportability between centers and unlock the potential clinical utility of EWS.

Analytical Validation of SKY92 for the Identification of High-Risk Multiple Myeloma.

Multiple myeloma (MM) is an incurable plasma cell cancer with a large variability in survival. Patients with MM classified as high risk by the SKY92 gene expression classifier are at high risk of relapse and short survival. Analytical validation of the SKY92 assay was performed with primary bone marrow specimens from 12 patients with MM and 7 reference cell line specimens. The SKY92 results were 100% concordant with the reference and/or their expected result for sensitivity, specificity, microarray stability, and RLT buffer stability. The SKY92 results were 90% concordant for primary specimen stability, 96.4% concordant for intermediate precision, and 80% to 100% concordant for RNA stability. For the cell-line reproducibility, the concordance was at least 92.9%, except for one near-cut point specimen. For the clinical specimen reproducibility, the concordance was 100%, except for two near-cut point specimens. Three independent laboratories were concordant in ≥77.8% and ≥92.9% of experiments for patient specimens and cell lines, respectively. Statistical acceptance thresholds were developed as Δ ≤1.48 (change in SKY92 score) and SD ≤0.45 (SD across SKY92 scores). Using the Clinical and Laboratory Standards Institute method of choice (EP05-A2/A3), restricted maximum likelihood, the observed Δ values (0 to 1.14) and SDs (0.22 to 0.31) passed acceptance criteria. Thus, we successfully present analytical validation for the SKY92 assay as a prognostic molecular test for individual patients with MM.

Prognostic and predictive performance of R-ISS with SKY92 in older patients with multiple myeloma: the HOVON-87/NMSG-18 trial.

The standard prognostic marker for multiple myeloma (MM) patients is the revised International Staging System (R-ISS). However, there is room for improvement in guiding treatment. This applies particularly to older patients, in whom the benefit/risk ratio is reduced because of comorbidities and subsequent side effects. We hypothesized that adding gene-expression data to R-ISS would generate a stronger marker. This was tested by combining R-ISS with the SKY92 classifier (SKY-RISS). The HOVON-87/NMSG-18 trial (EudraCT: 2007-004007-34) compared melphalan-prednisone-thalidomide followed by thalidomide maintenance (MPT-T) with melphalan-prednisone-lenalidomide followed by lenalidomide maintenance (MPR-R). From this trial, 168 patients with available R-ISS status and gene-expression profiles were analyzed. R-ISS stages I, II, and III were assigned to 8%, 75%, and 7% of patients, respectively (3-year overall survival [OS] rates: 80%, 65%, 33%, P = 8 × 10-3). Using the SKY92 classifier, 13% of patients were high risk (HR) (3-year OS rates: standard risk [SR], 70%; HR, 28%; P < .001). Combining SKY92 with R-ISS resulted in 3 risk groups: SKY-RISS I (SKY-SR + R-ISS-I; 15%), SKY-RISS III (SKY-HR + R-ISS-II/III; 11%), and SKY-RISS II (all other patients; 74%). The 3-year OS rates for SKY-RISS I, II, and III are 88%, 66%, and 26%, respectively (P = 6 × 10-7). The SKY-RISS model was validated in older patients from the CoMMpass dataset. Moreover, SKY-RISS demonstrated predictive potential: HR patients appeared to benefit from MPR-R over MPT-T (median OS, 55 and 14 months, respectively). Combined, SKY92 and R-ISS classify patients more accurately. Additionally, benefit was observed for MPR-R over MPT-T in SKY92-RISS HR patients only.

Identification of stage I/IIA melanoma patients at high risk for disease relapse using a clinicopathologic and gene expression model.

PURPOSE: Patients with stage I/IIA cutaneous melanoma (CM) are currently not eligible for adjuvant therapies despite uncertainty in relapse risk. Here, we studied the ability of a recently developed model which combines clinicopathologic and gene expression variables (CP-GEP) to identify stage I/IIA melanoma patients who have a high risk for disease relapse. PATIENTS AND METHODS: Archival specimens from a cohort of 837 consecutive primary CMs were used for assessing the prognostic performance of CP-GEP. The CP-GEP model combines Breslow thickness and patient age, with the expression of eight genes in the primary tumour. Our specific patient group, represented by 580 stage I/IIA patients, was stratified based on their risk of relapse: CP-GEP High Risk and CP-GEP Low Risk. The main clinical end-point of this study was five-year relapse-free survival (RFS). RESULTS: Within the stage I/IIA melanoma group, CP-GEP identified a high-risk patient group (47% of total stage I/IIA patients) which had a considerably worse five-year RFS than the low-risk patient group; 74% (95% confidence interval [CI]: 67%-80%) versus 89% (95% CI: 84%-93%); hazard ratio [HR] = 2.98 (95% CI: 1.78-4.98); P < 0.0001. Of patients in the high-risk group, those who relapsed were most likely to do so within the first 3 years. CONCLUSION: The CP-GEP model can be used to identify stage I/IIA patients who have a high risk for disease relapse. These patients may benefit from adjuvant therapy.

Gene Networks Constructed Through Simulated Treatment Learning can Predict Proteasome Inhibitor Benefit in Multiple Myeloma.

PURPOSE: Proteasome inhibitors are widely used in treating multiple myeloma, but can cause serious side effects and response varies among patients. It is, therefore, important to gain more insight into which patients will benefit from proteasome inhibitors. EXPERIMENTAL DESIGN: We introduce simulated treatment learned signatures (STLsig), a machine learning method to identify predictive gene expression signatures. STLsig uses genetically similar patients who have received an alternative treatment to model which patients will benefit more from proteasome inhibitors than from an alternative treatment. STLsig constructs gene networks by linking genes that are synergistic in their ability to predict benefit. RESULTS: In a dataset of 910 patients with multiple myeloma, STLsig identified two gene networks that together can predict benefit to the proteasome inhibitor, bortezomib. In class "benefit," we found an HR of 0.47 (P = 0.04) in favor of bortezomib, while in class "no benefit," the HR was 0.91 (P = 0.68). Importantly, we observed a similar performance (HR class benefit, 0.46; P = 0.04) in an independent patient cohort. Moreover, this signature also predicts benefit for the proteasome inhibitor, carfilzomib, indicating it is not specific to bortezomib. No equivalent signature can be found when the genes in the signature are excluded from the analysis, indicating that they are essential. Multiple genes in the signature are linked to working mechanisms of proteasome inhibitors or multiple myeloma disease progression. CONCLUSIONS: STLsig can identify gene signatures that could aid in treatment decisions for patients with multiple myeloma and provide insight into the biological mechanism behind treatment benefit.

Model Combining Tumor Molecular and Clinicopathologic Risk Factors Predicts Sentinel Lymph Node Metastasis in Primary Cutaneous Melanoma.

PURPOSE: More than 80% of patients who undergo sentinel lymph node (SLN) biopsy have no nodal metastasis. Here we describe a model that combines clinicopathologic and molecular variables to identify patients with thin and intermediate thickness melanomas who may forgo the SLN biopsy procedure due to their low risk of nodal metastasis. PATIENTS AND METHODS: Genes with functional roles in melanoma metastasis were discovered by analysis of next generation sequencing data and case control studies. We then used PCR to quantify gene expression in diagnostic biopsy tissue across a prospectively designed archival cohort of 754 consecutive thin and intermediate thickness primary cutaneous melanomas. Outcome of interest was SLN biopsy metastasis within 90 days of melanoma diagnosis. A penalized maximum likelihood estimation algorithm was used to train logistic regression models in a repeated cross validation scheme to predict the presence of SLN metastasis from molecular, clinical and histologic variables. RESULTS: Expression of genes with roles in epithelial-to-mesenchymal transition (glia derived nexin, growth differentiation factor 15, integrin ß3, interleukin 8, lysyl oxidase homolog 4, TGFß receptor type 1 and tissue-type plasminogen activator) and melanosome function (melanoma antigen recognized by T cells 1) were associated with SLN metastasis. The predictive ability of a model that only considered clinicopathologic or gene expression variables was outperformed by a model which included molecular variables in combination with the clinicopathologic predictors Breslow thickness and patient age; AUC, 0.82; 95% CI, 0.78-0.86; SLN biopsy reduction rate of 42% at a negative predictive value of 96%. CONCLUSION: A combined model including clinicopathologic and gene expression variables improved the identification of melanoma patients who may forgo the SLN biopsy procedure due to their low risk of nodal metastasis.

Predicting treatment benefit in multiple myeloma through simulation of alternative treatment effects.

Many cancer treatments are associated with serious side effects, while they often only benefit a subset of the patients. Therefore, there is an urgent clinical need for tools that can aid in selecting the right treatment at diagnosis. Here we introduce simulated treatment learning (STL), which enables prediction of a patient's treatment benefit. STL uses the idea that patients who received different treatments, but have similar genetic tumor profiles, can be used to model their response to the alternative treatment. We apply STL to two multiple myeloma gene expression datasets, containing different treatments (bortezomib and lenalidomide). We find that STL can predict treatment benefit for both; a twofold progression free survival (PFS) benefit is observed for bortezomib for 19.8% and a threefold PFS benefit for lenalidomide for 31.1% of the patients. This demonstrates that STL can derive clinically actionable gene expression signatures that enable a more personalized approach to treatment.

Potential therapeutic and economic value of risk-stratified treatment as initial treatment of multiple myeloma in Europe.

Biomarkers associated with prognosis in multiple myeloma (MM) can be used to stratify patients into risk categories. An attractive alternative to uniform treatment (UT), risk-stratified treatment (RST) is proposed where high-risk patients receive bortezomib-based regimens while standard-risk patients receive alternative less costly regimens. An early Markov-type decision analytic model evaluated the potential therapeutic and economic value of different RST strategies compared with UT in MM patients in key European countries. Results suggest RST strategies were both cheaper and more effective than UT across all countries, with the molecular marker-only strategy RST-SKY92 producing maximum health gains (0.031-0.039 QALYs). The conclusions remained consistent in the univariate sensitivity analyses. These findings should encourage stakeholders to support the adoption of RST approaches in MM.

Prognostic Validation of SKY92 and Its Combination With ISS in an Independent Cohort of Patients With Multiple Myeloma.

BACKGROUND: High risk and low risk multiple myeloma patients follow a very different clinical course as reflected in their PFS and OS. To be clinically useful, methodologies used to identify high and low risk disease must be validated in representative independent clinical data and available so that patients can be managed appropriately. A recent analysis has indicated that SKY92 combined with the International Staging System (ISS) identifies patients with different risk disease with high sensitivity. PATIENTS AND METHODS: Here we computed the performance of eight gene expression based classifiers SKY92, UAMS70, UAMS80, IFM15, Proliferation Index, Centrosome Index, Cancer Testis Antigen and HM19 as well as the combination of SKY92/ISS in an independent cohort of 91 newly diagnosed MM patients. RESULTS: The classifiers identified between 9%-21% of patients as high risk, with hazard ratios (HRs) between 1.9 and 8.2. CONCLUSION: Among the eight signatures, SKY92 identified the largest proportion of patients (21%) also with the highest HR (8.2). Our analysis also validated the combination SKY92/ISS for identification of three classes; low risk (42%), intermediate risk (37%) and high risk (21%). Between low risk and high risk classes the HR is >10.

Prediction of high- and low-risk multiple myeloma based on gene expression and the International Staging System.

Patients with multiple myeloma have variable survival and require reliable prognostic and predictive scoring systems. Currently, clinical and biological risk markers are used independently. Here, International Staging System (ISS), fluorescence in situ hybridization (FISH) markers, and gene expression (GEP) classifiers were combined to identify novel risk classifications in a discovery/validation setting. We used the datasets of the Dutch-Belgium Hemato-Oncology Group and German-speaking Myeloma Multicenter Group (HO65/GMMG-HD4), University of Arkansas for Medical Sciences-TT2 (UAMS-TT2), UAMS-TT3, Medical Research Council-IX, Assessment of Proteasome Inhibition for Extending Remissions, and Intergroupe Francophone du Myelome (IFM-G) (total number of patients: 4750). Twenty risk markers were evaluated, including t(4;14) and deletion of 17p (FISH), EMC92, and UAMS70 (GEP classifiers), and ISS. The novel risk classifications demonstrated that ISS is a valuable partner to GEP classifiers and FISH. Ranking all novel and existing risk classifications showed that the EMC92-ISS combination is the strongest predictor for overall survival, resulting in a 4-group risk classification. The median survival was 24 months for the highest risk group, 47 and 61 months for the intermediate risk groups, and the median was not reached after 96 months for the lowest risk group. The EMC92-ISS classification is a novel prognostic tool, based on biological and clinical parameters, which is superior to current markers and offers a robust, clinically relevant 4-group model.

Detection of CEBPA double mutants in acute myeloid leukemia using a custom gene expression array.

Double (bi-allelic) mutations in the gene encoding the CCAAT/enhancer-binding protein-alpha (CEBPA) transcription factor have a favorable prognostic impact in acute myeloid leukemia (AML). Double mutations in CEBPA can be detected using various techniques, but it is a notoriously difficult gene to sequence due to its high GC-content. Here we developed a two-step gene expression classifier for accurate and standardized detection of CEBPA double mutations. The key feature of the two-step classifier is that it explicitly removes cases with low CEBPA expression, thereby excluding CEBPA hypermethylated cases that have similar gene expression profiles as a CEBPA double mutant, which would result in false-positive predictions. In the second step, we have developed a 55 gene signature to identity the true CEBPA double-mutation cases. This two-step classifier was tested on a cohort of 505 unselected AML cases, including 26 CEBPA double mutants, 12 CEBPA single mutants, and seven CEBPA promoter hypermethylated cases, on which its performance was estimated by a double-loop cross-validation protocol. The two-step classifier achieves a sensitivity of 96.2% (95% confidence interval [CI] 81.1 to 99.3) and specificity of 100.0% (95% CI 99.2 to 100.0). There are no false-positive detections. This two-step CEBPA double-mutation classifier has been incorporated on a microarray platform that can simultaneously detect other relevant molecular biomarkers, which allows for a standardized comprehensive diagnostic assay. In conclusion, gene expression profiling provides a reliable method for CEBPA double-mutation detection in patients with AML for clinical use.

Detection of mutant NPM1 mRNA in acute myeloid leukemia using custom gene expression arrays.

Mutations in the gene encoding nucleophosmin (NPM1) carry a prognostic value for patients with acute myeloid leukemia (AML). Various techniques are currently being used to detect these mutations in routine molecular diagnostics. Incorporation of accurate NPM1 mutation detection on a gene expression platform would enable simultaneous detection with various other expression biomarkers. Here we present an array-based mutation detection using custom probes for NPM1 WT mRNA and NPM1 type A, B, and D mutant mRNA. This method was 100% accurate on a training cohort of 505 newly diagnosed unselected AML cases. Validation on an independent cohort of 143 normal-karyotype AML cases revealed no false-negative results, and one false positive (sensitivity 100.0% and specificity 98.7%). Based on this, we conclude that this method provides a reliable method for NPM1 mutation detection. The method can be applied to other genes/mutations as long as the mutant alleles are sufficiently highly expressed.

A standardized microarray assay for the independent gene expression markers in AML: EVI1 and BAALC.

High levels of BAALC, ERG, EVI1 and MN1 expression have been associated with shorter overall survival in AML but standardized and clinically validated assays are lacking. We have therefore developed and optimized an assay for standardized detection of these prognostic genes for patients with intermediate cytogenetic risk AML. In a training set of 147 intermediate cytogenetic risk cases we performed cross validations at 5 percentile steps of expression level and observed a bimodal significance profile for BAALC expression level and unimodal significance profiles for ERG and MN1 levels with no statistically significant cutoff points near the median expression level of BAALC, ERG or MN1. Of the possible cutoff points for expression levels of BAALC, ERG and MN1, just the 30th and 75th percentile of BAALC expression level and the 30th percentile of MN1 expression level cutoff points showed clinical significance. Of these only the 30th percentile of BAALC expression level reproduced in an independent verification (extended training) data set of 242 cytogenetically normal AML cases and successfully validated in an external cohort of 215 intermediate cytogenetic risk AML cases. Finally, we show independent prognostic value for high EVI1 and low BAALC in multivariate analysis with other clinically relevant molecular AML markers. We have developed a highly standardized molecular assay for the independent gene expression markers EVI1 and BAALC.

Integration of clinical and gene expression data has a synergetic effect on predicting breast cancer outcome.

Breast cancer outcome can be predicted using models derived from gene expression data or clinical data. Only a few studies have created a single prediction model using both gene expression and clinical data. These studies often remain inconclusive regarding an obtained improvement in prediction performance. We rigorously compare three different integration strategies (early, intermediate, and late integration) as well as classifiers employing no integration (only one data type) using five classifiers of varying complexity. We perform our analysis on a set of 295 breast cancer samples, for which gene expression data and an extensive set of clinical parameters are available as well as four breast cancer datasets containing 521 samples that we used as independent validation.mOn the 295 samples, a nearest mean classifier employing a logical OR operation (late integration) on clinical and expression classifiers significantly outperforms all other classifiers. Moreover, regardless of the integration strategy, the nearest mean classifier achieves the best performance. All five classifiers achieve their best performance when integrating clinical and expression data. Repeating the experiments using the 521 samples from the four independent validation datasets also indicated a significant performance improvement when integrating clinical and gene expression data. Whether integration also improves performances on other datasets (e.g. other tumor types) has not been investigated, but seems worthwhile pursuing. Our work suggests that future models for predicting breast cancer outcome should exploit both data types by employing a late OR or intermediate integration strategy based on nearest mean classifiers.

Knowledge driven decomposition of tumor expression profiles.

BACKGROUND: Tumors have been hypothesized to be the result of a mixture of oncogenic events, some of which will be reflected in the gene expression of the tumor. Based on this hypothesis a variety of data-driven methods have been employed to decompose tumor expression profiles into component profiles, hypothetically linked to these events. Interpretation of the resulting data-driven components is often done by post-hoc comparison to, for instance, functional groupings of genes into gene sets. None of the data-driven methods allow the incorporation of that type of knowledge directly into the decomposition. RESULTS: We present a linear model which uses knowledge driven, pre-defined components to perform the decomposition. We solve this decomposition model in a constrained linear least squares fashion. From a variety of options, a lasso-based solution to the model performs best in linking single gene perturbation data to mouse data. Moreover, we show the decomposition of expression profiles from human breast cancer samples into single gene perturbation profiles and gene sets that are linked to the hallmarks of cancer. For these breast cancer samples we were able to discern several links between clinical parameters, and the decomposition weights, providing new insights into the biology of these tumors. Lastly, we show that the order in which the Lasso regularization shrinks the weights, unveils consensus patterns within clinical subgroups of the breast cancer samples. CONCLUSION: The proposed lasso-based constrained least squares decomposition provides a stable and relevant relation between samples and knowledge-based components, and is thus a viable alternative to data-driven methods. In addition, the consensus order of component importance within clinical subgroups provides a better molecular characterization of the subtypes.

A comprehensive analysis of prognostic signatures reveals the high predictive capacity of the proliferation, immune response and RNA splicing modules in breast cancer.

INTRODUCTION: Several gene expression signatures have been proposed and demonstrated to be predictive of outcome in breast cancer. In the present article we address the following issues: Do these signatures perform similarly? Are there (common) molecular processes reported by these signatures? Can better prognostic predictors be constructed based on these identified molecular processes? METHODS: We performed a comprehensive analysis of the performance of nine gene expression signatures on seven different breast cancer datasets. To better characterize the functional processes associated with these signatures, we enlarged each signature by including all probes with a significant correlation to at least one of the genes in the original signature. The enrichment of functional groups was assessed using four ontology databases. RESULTS: The classification performance of the nine gene expression signatures is very similar in terms of assigning a sample to either a poor outcome group or a good outcome group. Nevertheless the concordance in classification at the sample level is low, with only 50% of the breast cancer samples classified in the same outcome group by all classifiers. The predictive accuracy decreases with the number of poor outcome assignments given to a sample. The best classification performance was obtained for the group of patients with only good outcome assignments. Enrichment analysis of the enlarged signatures revealed 11 functional modules with prognostic ability. The combination of the RNA-splicing and immune modules resulted in a classifier with high prognostic performance on an independent validation set. CONCLUSIONS: The study revealed that the nine signatures perform similarly but exhibit a large degree of discordance in prognostic group assignment. Functional analyses indicate that proliferation is a common cellular process, but that other functional categories are also enriched and show independent prognostic ability. We provide new evidence of the potentially promising prognostic impact of immunity and RNA-splicing processes in breast cancer.

Pooling breast cancer datasets has a synergetic effect on classification performance and improves signature stability.

BACKGROUND: Michiels et al. (Lancet 2005; 365: 488-92) employed a resampling strategy to show that the genes identified as predictors of prognosis from resamplings of a single gene expression dataset are highly variable. The genes most frequently identified in the separate resamplings were put forward as a 'gold standard'. On a higher level, breast cancer datasets collected by different institutions can be considered as resamplings from the underlying breast cancer population. The limited overlap between published prognostic signatures confirms the trend of signature instability identified by the resampling strategy. Six breast cancer datasets, totaling 947 samples, all measured on the Affymetrix platform, are currently available. This provides a unique opportunity to employ a substantial dataset to investigate the effects of pooling datasets on classifier accuracy, signature stability and enrichment of functional categories. RESULTS: We show that the resampling strategy produces a suboptimal ranking of genes, which can not be considered to be a 'gold standard'. When pooling breast cancer datasets, we observed a synergetic effect on the classification performance in 73% of the cases. We also observe a significant positive correlation between the number of datasets that is pooled, the validation performance, the number of genes selected, and the enrichment of specific functional categories. In addition, we have evaluated the support for five explanations that have been postulated for the limited overlap of signatures. CONCLUSION: The limited overlap of current signature genes can be attributed to small sample size. Pooling datasets results in more accurate classification and a convergence of signature genes. We therefore advocate the analysis of new data within the context of a compendium, rather than analysis in isolation.