Correction to: Blood transcriptomic discrimination of bacterial and viral infections in the emergency department: a multi-cohort observational validation study.

An amendment to this paper has been published and can be accessed via the original article.

Blood transcriptomic discrimination of bacterial and viral infections in the emergency department: a multi-cohort observational validation study.

BACKGROUND: There is an urgent need to develop biomarkers that stratify risk of bacterial infection in order to support antimicrobial stewardship in emergency hospital admissions. METHODS: We used computational machine learning to derive a rule-out blood transcriptomic signature of bacterial infection (SeptiCyte™ TRIAGE) from eight published case-control studies. We then validated this signature by itself in independent case-control data from more than 1500 samples in total, and in combination with our previously published signature for viral infections (SeptiCyte™ VIRUS) using pooled data from a further 1088 samples. Finally, we tested the performance of these signatures in a prospective observational cohort of emergency department (ED) patients with fever, and we used the combined SeptiCyte™ signature in a mixture modelling approach to estimate the prevalence of bacterial and viral infections in febrile ED patients without microbiological diagnoses. RESULTS: The combination of SeptiCyte™ TRIAGE with our published signature for viral infections (SeptiCyte™ VIRUS) discriminated bacterial and viral infections in febrile ED patients, with a receiver operating characteristic area under the curve of 0.95 (95% confidence interval 0.90-1), compared to 0.79 (0.68-0.91) for WCC and 0.73 (0.61-0.86) for CRP. At pre-test probabilities 0.35 and 0.72, the combined SeptiCyte™ score achieved a negative predictive value for bacterial infection of 0.97 (0.90-0.99) and 0.86 (0.64-0.96), compared to 0.90 (0.80-0.94) and 0.66 (0.48-0.79) for WCC and 0.88 (0.69-0.95) and 0.60 (0.31-0.72) for CRP. In a mixture modelling approach, the combined SeptiCyte™ score estimated that 24% of febrile ED cases receiving antibacterials without a microbiological diagnosis were due to viral infections. Our analysis also suggested that a proportion of patients with bacterial infection recovered without antibacterials. CONCLUSIONS: Blood transcriptional biomarkers offer exciting opportunities to support precision antibacterial prescribing in ED and improve diagnostic classification of patients without microbiologically confirmed infections.

Correction: The effect of uncertainty in patient classification on diagnostic performance estimations.

[This corrects the article DOI: 10.1371/journal.pone.0217146.].

The effect of uncertainty in patient classification on diagnostic performance estimations.

BACKGROUND: The performance of a new diagnostic test is typically evaluated against a comparator which is assumed to correspond closely to some true state of interest. Judgments about the new test's performance are based on the differences between the outputs of the test and comparator. It is commonly assumed that a small amount of uncertainty in the comparator's classifications will negligibly affect the measured performance of a diagnostic test. METHODS: Simulated datasets were generated to represent typical diagnostic scenarios. Comparator noise was introduced in the form of random misclassifications, and the effect on the apparent performance of the diagnostic test was determined. An actual dataset from a clinical trial on a new diagnostic test for sepsis was also analyzed. RESULTS: We demonstrate that as little as 5% misclassification of patients by the comparator can be enough to statistically invalidate performance estimates such as sensitivity, specificity and area under the receiver operating characteristic curve, if this uncertainty is not measured and taken into account. This distortion effect is found to increase non-linearly with comparator uncertainty, under some common diagnostic scenarios. For clinical populations exhibiting high degrees of classification uncertainty, failure to measure and account for this effect will introduce significant risks of drawing false conclusions. The effect of classification uncertainty is magnified further for high performing tests that would otherwise reach near-perfection in diagnostic evaluation trials. A requirement of very high diagnostic performance for clinical adoption, such as a 99% sensitivity, can be rendered nearly unachievable even for a perfect test, if the comparator diagnosis contains even small amounts of uncertainty. This paper and an accompanying online simulation tool demonstrate the effect of classification uncertainty on the apparent performance of tests across a range of typical diagnostic scenarios. Both simulated and real datasets are used to show the degradation of apparent test performance as comparator uncertainty increases. CONCLUSIONS: Overall, a 5% or greater misclassification rate by the comparator can lead to significant underestimation of true test performance. An online simulation tool allows researchers to explore this effect using their own trial parameters (https://imperfect-gold-standard.shinyapps.io/classification-noise/) and the source code is freely available (https://github.com/ksny/Imperfect-Gold-Standard).

Physician agreement on the diagnosis of sepsis in the intensive care unit: estimation of concordance and analysis of underlying factors in a multicenter cohort.

BACKGROUND: Differentiating sepsis from the systemic inflammatory response syndrome (SIRS) in critical care patients is challenging, especially before serious organ damage is evident, and with variable clinical presentations of patients and variable training and experience of attending physicians. Our objective was to describe and quantify physician agreement in diagnosing SIRS or sepsis in critical care patients as a function of available clinical information, infection site, and hospital setting. METHODS: We conducted a post hoc analysis of previously collected data from a prospective, observational trial (N = 249 subjects) in intensive care units at seven US hospitals, in which physicians at different stages of patient care were asked to make diagnostic calls of either SIRS, sepsis, or indeterminate, based on varying amounts of available clinical information (clinicaltrials.gov identifier: NCT02127502). The overall percent agreement and the free-marginal, inter-observer agreement statistic kappa (κ free) were used to quantify agreement between evaluators (attending physicians, site investigators, external expert panelists). Logistic regression and machine learning techniques were used to search for significant variables that could explain heterogeneity within the indeterminate and SIRS patient subgroups. RESULTS: Free-marginal kappa decreased between the initial impression of the attending physician and (1) the initial impression of the site investigator (κ free 0.68), (2) the consensus discharge diagnosis of the site investigators (κ free 0.62), and (3) the consensus diagnosis of the external expert panel (κ free 0.58). In contrast, agreement was greatest between the consensus discharge impression of site investigators and the consensus diagnosis of the external expert panel (κ free 0.79). When stratified by infection site, κ free for agreement between initial and later diagnoses had a mean value + 0.24 (range - 0.29 to + 0.39) for respiratory infections, compared to + 0.70 (range + 0.42 to + 0.88) for abdominal + urinary + other infections. Bioinformatics analysis failed to clearly resolve the indeterminate diagnoses and also failed to explain why 60% of SIRS patients were treated with antibiotics. CONCLUSIONS: Considerable uncertainty surrounds the differential clinical diagnosis of sepsis vs. SIRS, especially before organ damage has become highly evident, and for patients presenting with respiratory clinical signs. Our findings underscore the need to provide physicians with accurate, timely diagnostic information in evaluating possible sepsis.

Validation of a Host Response Assay, SeptiCyte LAB, for Discriminating Sepsis from Systemic Inflammatory Response Syndrome in the ICU.

RATIONALE: A molecular test to distinguish between sepsis and systemic inflammation of noninfectious etiology could potentially have clinical utility. OBJECTIVES: This study evaluated the diagnostic performance of a molecular host response assay (SeptiCyte LAB) designed to distinguish between sepsis and noninfectious systemic inflammation in critically ill adults. METHODS: The study employed a prospective, observational, noninterventional design and recruited a heterogeneous cohort of adult critical care patients from seven sites in the United States (n = 249). An additional group of 198 patients, recruited in the large MARS (Molecular Diagnosis and Risk Stratification of Sepsis) consortium trial in the Netherlands ( www.clinicaltrials.gov identifier NCT01905033), was also tested and analyzed, making a grand total of 447 patients in our study. The performance of SeptiCyte LAB was compared with retrospective physician diagnosis by a panel of three experts. MEASUREMENTS AND MAIN RESULTS: In receiver operating characteristic curve analysis, SeptiCyte LAB had an estimated area under the curve of 0.82-0.89 for discriminating sepsis from noninfectious systemic inflammation. The relative likelihood of sepsis versus noninfectious systemic inflammation was found to increase with increasing test score (range, 0-10). In a forward logistic regression analysis, the diagnostic performance of the assay was improved only marginally when used in combination with other clinical and laboratory variables, including procalcitonin. The performance of the assay was not significantly affected by demographic variables, including age, sex, or race/ethnicity. CONCLUSIONS: SeptiCyte LAB appears to be a promising diagnostic tool to complement physician assessment of infection likelihood in critically ill adult patients with systemic inflammation. Clinical trial registered with www.clinicaltrials.gov (NCT01905033 and NCT02127502).

Diagnostic Accuracy of a Host Gene Expression Signature That Discriminates Clinical Severe Sepsis Syndrome and Infection-Negative Systemic Inflammation Among Critically Ill Children.

OBJECTIVES: SeptiCyte Lab (Immunexpress, Seattle, WA), a molecular signature measuring the relative expression levels of four host messenger RNAs, was developed to discriminate critically ill adults with infection-positive versus infection-negative systemic inflammation. The objective was to assess the performance of Septicyte Lab in critically ill pediatric patients. DESIGN: Prospective observational study. SETTING: Pediatric and Cardiac ICUs, Seattle Children's Hospital, Seattle, WA. PATIENTS: A cohort of 40 children with clinically overt severe sepsis syndrome and 30 children immediately postcardiopulmonary bypass surgery was recruited. The clinically overt severe sepsis syndrome children had confirmed or highly suspected infection (microbial culture orders, antimicrobial prescription), two or more systemic inflammatory response syndrome criteria (including temperature and leukocyte criteria), and at least cardiovascular ± pulmonary organ dysfunction. INTERVENTIONS: None (observational study only). MEASUREMENTS AND MAIN RESULTS: Next-generation RNA sequencing was conducted on PAXgene blood RNA samples, successfully for 35 of 40 (87.5%) of the clinically overt severe sepsis syndrome patients and 29 of 30 (96.7%) of the postcardiopulmonary bypass patients. Forty patient samples (~ 60% of cohort) were reanalyzed by reverse transcription-quantitative polymerase chain reaction, to check for concordance with next-generation sequencing results. Postcardiopulmonary bypass versus clinically overt severe sepsis syndrome descriptors included the following: age, 7.3 ± 5.5 versus 9.0 ± 6.6 years; gender, 41% versus 49% male; Pediatric Risk of Mortality, version III, 7.0 ± 4.6 versus 8.7 ± 6.4; Pediatric Logistic Organ Dysfunction, version II, 5.1 ± 2.2 versus 4.8 ± 2.8. SeptiCyte Lab strongly differentiated postcardiopulmonary bypass and clinically overt severe sepsis syndrome patients by receiver operating characteristic curve analysis, with an area-under-curve value of 0.99 (95% CI, 0.96-1.00). Equivalent performance was found using reverse transcription-quantitative polymerase chain reaction. There was no significant correlation between the score produced by the SeptiCyte Lab test and measures of illness severity, immune compromise, or microbial culture status. CONCLUSIONS: SeptiCyte Lab is able to discriminate clearly between clinically well-defined and homogeneous postcardiopulmonary bypass and clinically overt severe sepsis syndrome groups in children. A broader investigation among children with more heterogeneous inflammation-associated diagnoses and care settings is warranted.

A Molecular Host Response Assay to Discriminate Between Sepsis and Infection-Negative Systemic Inflammation in Critically Ill Patients: Discovery and Validation in Independent Cohorts.

BACKGROUND: Systemic inflammation is a whole body reaction having an infection-positive (i.e., sepsis) or infection-negative origin. It is important to distinguish between these two etiologies early and accurately because this has significant therapeutic implications for critically ill patients. We hypothesized that a molecular classifier based on peripheral blood RNAs could be discovered that would (1) determine which patients with systemic inflammation had sepsis, (2) be robust across independent patient cohorts, (3) be insensitive to disease severity, and (4) provide diagnostic utility. The goal of this study was to identify and validate such a molecular classifier. METHODS AND FINDINGS: We conducted an observational, non-interventional study of adult patients recruited from tertiary intensive care units (ICUs). Biomarker discovery utilized an Australian cohort (n = 105) consisting of 74 cases (sepsis patients) and 31 controls (post-surgical patients with infection-negative systemic inflammation) recruited at five tertiary care settings in Brisbane, Australia, from June 3, 2008, to December 22, 2011. A four-gene classifier combining CEACAM4, LAMP1, PLA2G7, and PLAC8 RNA biomarkers was identified. This classifier, designated SeptiCyte Lab, was validated using reverse transcription quantitative PCR and receiver operating characteristic (ROC) curve analysis in five cohorts (n = 345) from the Netherlands. Patients for validation were selected from the Molecular Diagnosis and Risk Stratification of Sepsis study (ClinicalTrials.gov, NCT01905033), which recruited ICU patients from the Academic Medical Center in Amsterdam and the University Medical Center Utrecht. Patients recruited from November 30, 2012, to August 5, 2013, were eligible for inclusion in the present study. Validation cohort 1 (n = 59) consisted entirely of unambiguous cases and controls; SeptiCyte Lab gave an area under curve (AUC) of 0.95 (95% CI 0.91-1.00) in this cohort. ROC curve analysis of an independent, more heterogeneous group of patients (validation cohorts 2-5; 249 patients after excluding 37 patients with an infection likelihood of "possible") gave an AUC of 0.89 (95% CI 0.85-0.93). Disease severity, as measured by Sequential Organ Failure Assessment (SOFA) score or Acute Physiology and Chronic Health Evaluation (APACHE) IV score, was not a significant confounding variable. The diagnostic utility of SeptiCyte Lab was evaluated by comparison to various clinical and laboratory parameters available to a clinician within 24 h of ICU admission. SeptiCyte Lab was significantly better at differentiating cases from controls than all tested parameters, both singly and in various logistic combinations, and more than halved the diagnostic error rate compared to procalcitonin in all tested cohorts and cohort combinations. Limitations of this study relate to (1) cohort compositions that do not perfectly reflect the composition of the intended use population, (2) potential biases that could be introduced as a result of the current lack of a gold standard for diagnosing sepsis, and (3) lack of a complete, unbiased comparison to C-reactive protein. CONCLUSIONS: SeptiCyte Lab is a rapid molecular assay that may be clinically useful in managing ICU patients with systemic inflammation. Further study in population-based cohorts is needed to validate this assay for clinical use.

YuGene: a simple approach to scale gene expression data derived from different platforms for integrated analyses.

Gene expression databases contain invaluable information about a range of cell states, but the question "Where is my gene of interest expressed?" remains one of the most difficult to systematically assess when relevant data is derived on different platforms. Barriers to integrating this data include disparities in data formats and scale, a lack of common identifiers, and the disproportionate contribution of a platform to the 'batch effect'. There are few purpose-built cross-platform normalization strategies, and most of these fit data to an idealized data structure, which in turn may compromise gene expression comparisons between different platforms. YuGene addresses this gap by providing a simple transform that assigns a modified cumulative proportion value to each measurement, without losing essential underlying information on data distributions or experimental correlates. The Yugene transform is applied to individual samples and is suitable to apply to data with different distributions. Yugene is robust to combining datasets of different sizes, does not require global renormalization as new data is added, and does not require a common identifier. YuGene was benchmarked against commonly used normalization approaches, performing favorably in comparison to quantile (RMA), Z-score or rank methods. Implementation in the www.stemformatics.org resource provides users with expression queries across stem cell related datasets. Probe performance statistics including poorly performing (never expressed) probes, and examples of probes/genes expressed in a sample-restricted manner are provided. The YuGene software is implemented as an R package available from CRAN.

Human papillomavirus (HPV) genotypes in an Australian sample of anal cancers.

Human papillomavirus (HPV) causes most cases of anal cancers. In this study, we analyzed biopsy material from 112 patients with anal cancers in Australia for the presence of HPV DNA by the INNO LiPA HPV genotyping assay. There were 82% (92) males and 18% (20) females. The mean age at diagnosis was significantly (p = 0.006) younger for males (52.5 years) than females (66 years). HIV-infected males were diagnosed at a much earlier mean age (48.2 years) than HIV negative (56.3 years) males (p = 0.05). HPV DNA was detected in 96.4% (108) of cases. HPV type 16 was the commonest, at 75% (81) of samples and being the sole genotype detected in 61% (66). Overall, 79% (85) of cases had at least one genotype targeted by the bivalent HPV (bHPV) vaccine, 90% (97) by the quadrivalent HPV (qHPV) vaccine and 96% (104) by the nonavalent HPV (nHPV) vaccine. The qHPV vaccine, which is now offered to all secondary school students in Australia, may prevent anal cancers in Australia. However, given the mean age of onset of this condition, the vaccine is unlikely to have a significant impact for several decades. Further research is necessary to prove additional protective effects of the nHPV vaccine.

Progression to and spontaneous regression of high-grade anal squamous intraepithelial lesions in HIV-infected and uninfected men.

OBJECTIVE: To quantify incidence of, and risk factors for, progression to and spontaneous regression of high-grade anal squamous intraepithelial lesions (ASILs). DESIGN: Retrospective review of patients at St Vincent's Hospital Anal Cancer Screening Clinic during a period when high-grade ASILs were not routinely treated (2004-2011). METHODS: All patients who had an anal Papanicolaou smear or high-resolution anoscopy were included, except for patients with previous anal cancer. High-grade anal intraepithelial neoplasia (HGAIN) was defined as a composite of histologically confirmed grade 2 or 3 anal intraepithelial neoplasia (AIN2/3) and/or high-grade squamous intraepithelial lesion on anal cytology. Analyses were repeated restricting to histologically confirmed AIN3. RESULTS: There were 574 patients: median age 45 years (interquartile range, IQR 36-51), 99.3% male and 73.0% HIV-infected [median HIV duration was 13.8 years (IQR 6.4-19.8), median CD4+ T-lymphocyte count was 500 cells/µl (IQR 357-662), 83.5% had undetectable plasma HIV viral load]. Median follow-up was 1.1 years (IQR 0.26-2.76). Progression rate to HGAIN was 7.4/100 person-years (95% confidence interval, CI 4.73-11.63). No risk factor for progression to HGAIN was identified; progression to AIN3 was more likely with increasing age (Ptrend = 0.004) and in those who were HIV-infected [hazard ratio 2.8 (95% CI 1.18-6.68) versus HIV-uninfected; P = 0.019], particularly in those whose nadir CD4+ T-lymphocyte count was less than 200 cells/µl (Ptrend = 0.003). In 101 patients with HGAIN, 24 (23.8%) patients had spontaneous regression [rate 23.5/100 person-years (95% CI 15.73-35.02)], mostly to AIN1. Regression was less likely in older patients (Ptrend = 0.048). Two patients with HGAIN developed anal cancer. CONCLUSION: High-grade ASILs frequently spontaneously regress. Longer-term, prospective studies are required to determine whether these regressions are sustained.

Anal cytological abnormalities are poor predictors of high-grade intraepithelial neoplasia amongst HIV-positive men who have sex with men.

BACKGROUND: Although anal squamous cell carcinomas (ASCC) are rare in the general community, rates of ASCC among HIV-positive men who have sex with men (MSM) approach those of major cancers in the general community, such as colorectal and lung cancers. Anal cytology and high-resolution anoscopy (HRA) have been proposed as methods for the diagnosis of high-grade anal intraepithelial neoplasia (HGAIN), the precursor of ASCC. To determine the prevalence of anal disease among HIV-positive MSM, we investigated anal cytological and histological findings in men from a large HIV clinic in Sydney, Australia. METHODS: This was a single-centre study conducted between October 2008 and January 2010. Participants self-collected cytology specimens, and those yielding abnormal cytology results of atypical cells of undetermined significance, atypical cells of undetermined significance - possibly high-grade (ASC-H) and high-grade squamous intraepithelial lesions (HSIL) were offered HRA. In addition, of those yielding low-grade squamous intraepithelial lesions results, a systematically selected group (25%) were offered HRA. RESULTS: Of the 1339 HIV-positive MSM who attended the clinic during the study period, 291 (31.8%) were finally included in the study, 262 yielded technically satisfactory cytological results and 101 (36.7%) participants underwent HRA. HGAIN was identified in 55 (54.5%) of the 101 men undergoing HRA. HGAIN was diagnosed in 28 (52.7%) without cytological ASC-H or HSIL results. CONCLUSIONS: Despite the poor correlation between anal cytological and histological findings, high levels of HGAIN were identified in HIV-positive MSM attending this clinical service.

Prevalence and predictors of high-grade anal intraepithelial neoplasia in a community-based sample of homosexual men.

BACKGROUND: We report the prevalence and predictors for high-grade anal intraepithelial neoplasia (HGAIN) in community-based cohorts of HIV-negative and HIV-positive homosexual men in Sydney, Australia. METHODS: A cross-sectional study of consecutive participants in both cohorts was performed in 2005 (204 HIV-negative and 128 HIV-positive men). Anal swabs collected by a research nurse underwent cytological analysis, using the ThinPrep procedure, and human papillomavirus (HPV) testing. Participants who had cytological abnormalities other than low-grade squamous epithelial lesions (SIL) were referred for high resolution anoscopy (HRA). RESULTS: A total of 114 men had cytological abnormalities (24.3% of HIV-negative and 57.5% of HIV-positive men, odds ratio (OR)=4.21, 95% confidence interval (CI) 2.57-6.90). However, only three (2.3%) HIV-positive men and no HIV-negative men had high-grade SIL on anal cytology. Seventy-seven men were referred for HRA, of whom 63 (81.8%) attended. Histologically confirmed HGAIN was detected in 21 (33.3%). The prevalence of HGAIN was higher in HIV-positive men (10.8%) than in HIV-negative men (5.0%, OR=2.29, 95% CI 0.93-5.63, P=0.071). HGAIN was not related to age but was strongly associated with the detection of high-risk types of anal HPV (OR=10.1, 95% CI 1.33-76.2) rather than low-risk types (OR=1.97, 95% CI 0.74-5.25). CONCLUSION: HGAIN was prevalent in homosexual men across all age groups and was more than twice as common in HIV-positive men compared with HIV-negative men. The presence of high-risk anal HPV was highly predictive of HGAIN.

Harvest: an open-source tool for the validation and improvement of peptide identification metrics and fragmentation exploration.

BACKGROUND: Protein identification using mass spectrometry is an important tool in many areas of the life sciences, and in proteomics research in particular. Increasing the number of proteins correctly identified is dependent on the ability to include new knowledge about the mass spectrometry fragmentation process, into computational algorithms designed to separate true matches of peptides to unidentified mass spectra from spurious matches. This discrimination is achieved by computing a function of the various features of the potential match between the observed and theoretical spectra to give a numerical approximation of their similarity. It is these underlying "metrics" that determine the ability of a protein identification package to maximise correct identifications while limiting false discovery rates. There is currently no software available specifically for the simple implementation and analysis of arbitrary novel metrics for peptide matching and for the exploration of fragmentation patterns for a given dataset. RESULTS: We present Harvest: an open source software tool for analysing fragmentation patterns and assessing the power of a new piece of information about the MS/MS fragmentation process to more clearly differentiate between correct and random peptide assignments. We demonstrate this functionality using data metrics derived from the properties of individual datasets in a peptide identification context. Using Harvest, we demonstrate how the development of such metrics may improve correct peptide assignment confidence in the context of a high-throughput proteomics experiment and characterise properties of peptide fragmentation. CONCLUSIONS: Harvest provides a simple framework in C++ for analysing and prototyping metrics for peptide matching, the core of the protein identification problem. It is not a protein identification package and answers a different research question to packages such as Sequest, Mascot, X!Tandem, and other protein identification packages. It does not aim to maximise the number of assigned peptides from a set of unknown spectra, but instead provides a method by which researchers can explore fragmentation properties and assess the power of novel metrics for peptide matching in the context of a given experiment. Metrics developed using Harvest may then become candidates for later integration into protein identification packages.

Computational methods for protein identification from mass spectrometry data.

Protein identification using mass spectrometry is an indispensable computational tool in the life sciences. A dramatic increase in the use of proteomic strategies to understand the biology of living systems generates an ongoing need for more effective, efficient, and accurate computational methods for protein identification. A wide range of computational methods, each with various implementations, are available to complement different proteomic approaches. A solid knowledge of the range of algorithms available and, more critically, the accuracy and effectiveness of these techniques is essential to ensure as many of the proteins as possible, within any particular experiment, are correctly identified. Here, we undertake a systematic review of the currently available methods and algorithms for interpreting, managing, and analyzing biological data associated with protein identification. We summarize the advances in computational solutions as they have responded to corresponding advances in mass spectrometry hardware. The evolution of scoring algorithms and metrics for automated protein identification are also discussed with a focus on the relative performance of different techniques. We also consider the relative advantages and limitations of different techniques in particular biological contexts. Finally, we present our perspective on future developments in the area of computational protein identification by considering the most recent literature on new and promising approaches to the problem as well as identifying areas yet to be explored and the potential application of methods from other areas of computational biology.