Validation of a Host Response Assay, SeptiCyte LAB, for Discriminating Sepsis from Systemic Inflammatory Response Syndrome in the ICU.

RATIONALE: A molecular test to distinguish between sepsis and systemic inflammation of noninfectious etiology could potentially have clinical utility. OBJECTIVES: This study evaluated the diagnostic performance of a molecular host response assay (SeptiCyte LAB) designed to distinguish between sepsis and noninfectious systemic inflammation in critically ill adults. METHODS: The study employed a prospective, observational, noninterventional design and recruited a heterogeneous cohort of adult critical care patients from seven sites in the United States (n = 249). An additional group of 198 patients, recruited in the large MARS (Molecular Diagnosis and Risk Stratification of Sepsis) consortium trial in the Netherlands ( www.clinicaltrials.gov identifier NCT01905033), was also tested and analyzed, making a grand total of 447 patients in our study. The performance of SeptiCyte LAB was compared with retrospective physician diagnosis by a panel of three experts. MEASUREMENTS AND MAIN RESULTS: In receiver operating characteristic curve analysis, SeptiCyte LAB had an estimated area under the curve of 0.82-0.89 for discriminating sepsis from noninfectious systemic inflammation. The relative likelihood of sepsis versus noninfectious systemic inflammation was found to increase with increasing test score (range, 0-10). In a forward logistic regression analysis, the diagnostic performance of the assay was improved only marginally when used in combination with other clinical and laboratory variables, including procalcitonin. The performance of the assay was not significantly affected by demographic variables, including age, sex, or race/ethnicity. CONCLUSIONS: SeptiCyte LAB appears to be a promising diagnostic tool to complement physician assessment of infection likelihood in critically ill adult patients with systemic inflammation. Clinical trial registered with www.clinicaltrials.gov (NCT01905033 and NCT02127502).

Harvest: an open-source tool for the validation and improvement of peptide identification metrics and fragmentation exploration.

BACKGROUND: Protein identification using mass spectrometry is an important tool in many areas of the life sciences, and in proteomics research in particular. Increasing the number of proteins correctly identified is dependent on the ability to include new knowledge about the mass spectrometry fragmentation process, into computational algorithms designed to separate true matches of peptides to unidentified mass spectra from spurious matches. This discrimination is achieved by computing a function of the various features of the potential match between the observed and theoretical spectra to give a numerical approximation of their similarity. It is these underlying "metrics" that determine the ability of a protein identification package to maximise correct identifications while limiting false discovery rates. There is currently no software available specifically for the simple implementation and analysis of arbitrary novel metrics for peptide matching and for the exploration of fragmentation patterns for a given dataset. RESULTS: We present Harvest: an open source software tool for analysing fragmentation patterns and assessing the power of a new piece of information about the MS/MS fragmentation process to more clearly differentiate between correct and random peptide assignments. We demonstrate this functionality using data metrics derived from the properties of individual datasets in a peptide identification context. Using Harvest, we demonstrate how the development of such metrics may improve correct peptide assignment confidence in the context of a high-throughput proteomics experiment and characterise properties of peptide fragmentation. CONCLUSIONS: Harvest provides a simple framework in C++ for analysing and prototyping metrics for peptide matching, the core of the protein identification problem. It is not a protein identification package and answers a different research question to packages such as Sequest, Mascot, X!Tandem, and other protein identification packages. It does not aim to maximise the number of assigned peptides from a set of unknown spectra, but instead provides a method by which researchers can explore fragmentation properties and assess the power of novel metrics for peptide matching in the context of a given experiment. Metrics developed using Harvest may then become candidates for later integration into protein identification packages.

The effect of uncertainty in patient classification on diagnostic performance estimations.

BACKGROUND: The performance of a new diagnostic test is typically evaluated against a comparator which is assumed to correspond closely to some true state of interest. Judgments about the new test's performance are based on the differences between the outputs of the test and comparator. It is commonly assumed that a small amount of uncertainty in the comparator's classifications will negligibly affect the measured performance of a diagnostic test. METHODS: Simulated datasets were generated to represent typical diagnostic scenarios. Comparator noise was introduced in the form of random misclassifications, and the effect on the apparent performance of the diagnostic test was determined. An actual dataset from a clinical trial on a new diagnostic test for sepsis was also analyzed. RESULTS: We demonstrate that as little as 5% misclassification of patients by the comparator can be enough to statistically invalidate performance estimates such as sensitivity, specificity and area under the receiver operating characteristic curve, if this uncertainty is not measured and taken into account. This distortion effect is found to increase non-linearly with comparator uncertainty, under some common diagnostic scenarios. For clinical populations exhibiting high degrees of classification uncertainty, failure to measure and account for this effect will introduce significant risks of drawing false conclusions. The effect of classification uncertainty is magnified further for high performing tests that would otherwise reach near-perfection in diagnostic evaluation trials. A requirement of very high diagnostic performance for clinical adoption, such as a 99% sensitivity, can be rendered nearly unachievable even for a perfect test, if the comparator diagnosis contains even small amounts of uncertainty. This paper and an accompanying online simulation tool demonstrate the effect of classification uncertainty on the apparent performance of tests across a range of typical diagnostic scenarios. Both simulated and real datasets are used to show the degradation of apparent test performance as comparator uncertainty increases. CONCLUSIONS: Overall, a 5% or greater misclassification rate by the comparator can lead to significant underestimation of true test performance. An online simulation tool allows researchers to explore this effect using their own trial parameters (https://imperfect-gold-standard.shinyapps.io/classification-noise/) and the source code is freely available (https://github.com/ksny/Imperfect-Gold-Standard).

Correction: The effect of uncertainty in patient classification on diagnostic performance estimations.

[This corrects the article DOI: 10.1371/journal.pone.0217146.].

Progression to and spontaneous regression of high-grade anal squamous intraepithelial lesions in HIV-infected and uninfected men.

OBJECTIVE: To quantify incidence of, and risk factors for, progression to and spontaneous regression of high-grade anal squamous intraepithelial lesions (ASILs). DESIGN: Retrospective review of patients at St Vincent's Hospital Anal Cancer Screening Clinic during a period when high-grade ASILs were not routinely treated (2004-2011). METHODS: All patients who had an anal Papanicolaou smear or high-resolution anoscopy were included, except for patients with previous anal cancer. High-grade anal intraepithelial neoplasia (HGAIN) was defined as a composite of histologically confirmed grade 2 or 3 anal intraepithelial neoplasia (AIN2/3) and/or high-grade squamous intraepithelial lesion on anal cytology. Analyses were repeated restricting to histologically confirmed AIN3. RESULTS: There were 574 patients: median age 45 years (interquartile range, IQR 36-51), 99.3% male and 73.0% HIV-infected [median HIV duration was 13.8 years (IQR 6.4-19.8), median CD4+ T-lymphocyte count was 500 cells/µl (IQR 357-662), 83.5% had undetectable plasma HIV viral load]. Median follow-up was 1.1 years (IQR 0.26-2.76). Progression rate to HGAIN was 7.4/100 person-years (95% confidence interval, CI 4.73-11.63). No risk factor for progression to HGAIN was identified; progression to AIN3 was more likely with increasing age (Ptrend = 0.004) and in those who were HIV-infected [hazard ratio 2.8 (95% CI 1.18-6.68) versus HIV-uninfected; P = 0.019], particularly in those whose nadir CD4+ T-lymphocyte count was less than 200 cells/µl (Ptrend = 0.003). In 101 patients with HGAIN, 24 (23.8%) patients had spontaneous regression [rate 23.5/100 person-years (95% CI 15.73-35.02)], mostly to AIN1. Regression was less likely in older patients (Ptrend = 0.048). Two patients with HGAIN developed anal cancer. CONCLUSION: High-grade ASILs frequently spontaneously regress. Longer-term, prospective studies are required to determine whether these regressions are sustained.