A study proposal came across my desk recently about evaluation of a new test for infection in a certain group of patients. The potential benefit was that the test uses a chemical marker of infection that is thought to increase rapidly early in the infection process, so it would potentially allow earlier diagnosis and treatment of the infection.
However, the analysis proposed to look for differences in the levels of the marker between patients with confirmed infection and those without. This is asking the wrong question: the issue is not whether the levels of marker differ between infected and non-infected patients. If this is being proposed as a test that will identify infected patients, presumably there is already a pretty good idea that levels of the marker differ. The important issue here is whether the marker is good at identifying those patients that have real infections i.e. it is a diagnostic question of sensitivity and specificity. The most important number is probably the positive predictive value: if a positive test result misses a lot of infected patients, it isn’t going to be much use in clinical practice.
A similar situation arose in a systematic review we did a few years ago of risk factors for chronic disability after whiplash injury. In this, a number of studies had recorded risk factors of whiplash-injured patients (such as injury severity, pre-existing pain, and so on) and whether they developed long-term problems, then analysed whether the risk factors differed between the patients who had recovered and those who had ongoing problems. Again, this is not addressing the right question. What we want to know is how good are risk factors that a clinician can assess early on at predicting long-term whiplash-associated problems.
Originally posted at http://blogs.warwick.ac.uk/simongates?num=10&start=30 on 26 September 2013.