Original post http://blogs.warwick.ac.uk/simongates/entry/even_heroes_get/ 18 September 2015

I recently read David Sackett’s 2004 paper from Evidence-based Medicine “Superiority trials, non-inferiority trials, and prisoners of the 2-sided null hypothesis “ (Evid Based Med 2004;9:38-39 doi:10.1136/ebm.9.2.38). [links don’t seem to be working, will edit later if I can].

In it I found this:

“As it happened, our 1-sided analysis revealed that the probability that our nurse practitioners’ patients were worse off (by ⩾5%) than our general practitioners’ patients was as small as 0.008.”

I’m pretty sure that 0.008 probability isn’t from a Bayesian analysis and is a misinterpretation of a p-value. It isn’t the probability of the null hypothesis being false! It really isn’t! Obviously that got past the reviewers of this manuscript without comment.

Edit: I’ve got the paper now. It’s a result from a one-tailed test for non-inferiority. The null hypothesis is that the intervention group was worse by 5% or more on their measure of function, p=0.008 so they reject the hypothesis of inferiority. But, as usual, that’s the probability of getting the data (or more extreme data) if the null hypothesis is true – not the probability of the null hypothesis.