Net Reclassification Index and Integrated Discrimination Index Are Not Appropriate for Testing Whether a Biomarker Improves Predictive Performance.

One of the goals of the Critical Path Institute's Predictive Safety Testing Consortium (PSTC) is to promote best practices for evaluating novel markers of drug induced injury. This includes the use of sound statistical methods. For rat studies, these practices have centered around comparing the area under the receiver-operator characteristic curve for each novel injury biomarker to those for the standard markers. In addition, the PSTC has previously used the net reclassification index (NRI) and integrated discrimination index (IDI) to assess the increased certainty provided by each novel injury biomarker when added to the information already provided by the standard markers. Due to their relatively simple interpretations, NRI and IDI have generally been popular measures of predictive performance. However recent literature suggests that significance tests for NRI and IDI can have inflated false positive rates and thus, tests based on these metrics should not be relied upon. Instead, when parametric models are employed to assess the added predictive value of a new marker, following (Pepe, M. S., Kerr, K. F., Longton, G., and Wang, Z. (2013). Testing for improvement in prediction model performance. Stat. Med. 32, 1467-1482), the PSTC recommends that likelihood based methods be used for significance testing.

Kidney injury molecule-1 outperforms traditional biomarkers of kidney injury in preclinical biomarker qualification studies.

Kidney toxicity accounts both for the failure of many drug candidates as well as considerable patient morbidity. Whereas histopathology remains the gold standard for nephrotoxicity in animal systems, serum creatinine (SCr) and blood urea nitrogen (BUN) are the primary options for monitoring kidney dysfunction in humans. The transmembrane tubular protein kidney injury molecule-1 (Kim-1) was previously reported to be markedly induced in response to renal injury. Owing to the poor sensitivity and specificity of SCr and BUN, we used rat toxicology studies to compare the diagnostic performance of urinary Kim-1 to BUN, SCr and urinary N-acetyl-beta-D-glucosaminidase (NAG) as predictors of kidney tubular damage scored by histopathology. Kim-1 outperforms SCr, BUN and urinary NAG in multiple rat models of kidney injury. Urinary Kim-1 measurements may facilitate sensitive, specific and accurate prediction of human nephrotoxicity in preclinical drug screens. This should enable early identification and elimination of compounds that are potentially nephrotoxic.