When possible, report a Fisher-exact P value and display its underlying null randomization distribution.
Proc Natl Acad Sci U S A
; 117(32): 19151-19158, 2020 08 11.
Article
in En
| MEDLINE
| ID: mdl-32703808
ABSTRACT
In randomized experiments, Fisher-exact P values are available and should be used to help evaluate results rather than the more commonly reported asymptotic P values. One reason is that using the latter can effectively alter the question being addressed by including irrelevant distributional assumptions. The Fisherian statistical framework, proposed in 1925, calculates a P value in a randomized experiment by using the actual randomization procedure that led to the observed data. Here, we illustrate this Fisherian framework in a crossover randomized experiment. First, we consider the first period of the experiment and analyze its data as a completely randomized experiment, ignoring the second period; then, we consider both periods. For each analysis, we focus on 10 outcomes that illustrate important differences between the asymptotic and Fisher tests for the null hypothesis of no ozone effect. For some outcomes, the traditional P value based on the approximating asymptotic Student's t distribution substantially subceeded the minimum attainable Fisher-exact P value. For the other outcomes, the Fisher-exact null randomization distribution substantially differed from the bell-shaped one assumed by the asymptotic t test. Our conclusions:
When researchers choose to report P values in randomized experiments, 1) Fisher-exact P values should be used, especially in studies with small sample sizes, and 2) the shape of the actual null randomization distribution should be examined for the recondite scientific insights it may reveal.Key words
Full text:
1
Collection:
01-internacional
Database:
MEDLINE
Main subject:
Randomized Controlled Trials as Topic
Type of study:
Clinical_trials
/
Prognostic_studies
/
Risk_factors_studies
Limits:
Humans
Language:
En
Journal:
Proc Natl Acad Sci U S A
Year:
2020
Type:
Article