Reporting trends of p values in the neurosurgical literature.

ABSTRACT

OBJECTIVE: The objective of this study was to evaluate the trends in reporting of p values in the neurosurgical literature from 1990 through 2017. METHODS: All abstracts from the Journal of Neurology, Neurosurgery, and Psychiatry (JNNP), Journal of Neurosurgery (JNS) collection (including Journal of Neurosurgery Spine and Journal of Neurosurgery Pediatrics), Neurosurgery (NS), and Journal of Neurotrauma (JNT) available on PubMed from 1990 through 2017 were retrieved. Automated text mining was performed to extract p values from relevant abstracts. Extracted p values were analyzed for temporal trends and characteristics. RESULTS: The search yielded 47,889 relevant abstracts. A total of 34,324 p values were detected in 11,171 abstracts. Since 1990 there has been a steady, proportionate increase in the number of abstracts containing p values. There were average absolute year-on-year increases of 1.2% (95% CI 1.1%-1.3%; p < 0.001), 0.93% (95% CI 0.75%-1.1%; p < 0.001), 0.70% (95% CI 0.57%-0.83%; p < 0.001), and 0.35% (95% CI 0.095%-0.60%; p = 0.0091) of abstracts reporting p values in JNNP, JNS, NS, and JNT, respectively. There have also been average year-on-year increases of 0.045 (95% CI 0.031-0.059; p < 0.001), 0.052 (95% CI 0.037-0.066; p < 0.001), 0.042 (95% CI 0.030-0.054; p < 0.001), and 0.041 (95% CI 0.026-0.056; p < 0.001) p values reported per abstract for these respective journals. The distribution of p values showed a positive skew and strong clustering of values at rounded decimals (i.e., 0.01, 0.02, etc.). Between 83.2% and 89.8% of all reported p values were at or below the "significance" threshold of 0.05 (i.e., p ≤ 0.05). CONCLUSIONS: Trends in reporting of p values and the distribution of p values suggest publication bias remains in the neurosurgical literature.