New England Journal of Medicine for the last time

Sorry for going on yet again about the New England Journal of Medicine, but I recently had the experience of publishing a trial there (this one) and interacting with their statistical guidelines, which I have discussed before (here and here).

The issue about p-values for baseline characteristics came up, of course.  Here’s what came back from the Editor:

It is a general policy of the Journal to request that the table of baseline characteristics (here Table 1) should include a footnote indicating which (if any) of the listed characteristics differ at P<0.05. The editors appreciate that, in a randomized trial, all differences between groups occur at random, and therefore the P value does not have its conventional meaning here. However, it is the view of our statistical advisers that such P values can be useful in calling attention to notable imbalances (if any) between the groups.

The table duly appeared with the footnote as required:

table 1 footnote

This strikes me as possibly the worst of the available options.  If we gave the exact p-value for each of the variables, that would give us a measure of how unusual the data were compared to the expectation of zero difference, which makes some sense.  Just saying there were no significant differences is misleading, because the whole concept of significance tests does not apply here.  We’re not performing tests with the intention of rejecting the null hypothesis if we find p < 0.05, because we already know that it is true. It’s randomised, so we already know that differences are only due to chance.

It’s good that the editor recognised that the use of the p-value for baseline characteristics is different from the usual use, but not so good that they ask for those with p < 0.05 to be indicated.  That will inevitably get interpreted by almost everyone as indicating “significance,” “rejection of the null hypothesis” and all of that stuff.  I think it would be much better to call attention to large imbalances using something other than a p-value, and definitely something other than just indicating where p < 0.05.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s