Few recent debates in the US have been more poisonous than discussions about healthcare. They have been accompanied by all sorts of dodgy claims and counterclaims, often with statistical evidence to back them up.
Take the statement by former New York City mayor Rudy Giuliani in his campaign to win the 2008 Republican presidential nomination. He quoted – a disease he had himself suffered – in the US as 82 per cent, and compared this with the chance of survival under the UK’s taxpayer-funded National Health Service of just 44 per cent.
Survival rates a factor of 2 apart in two comparably developed countries? If right, surely that would be a damning indictment of the deadly inadequacy of socialised medicine. And there’s no doubting Giuliani’s figures were right.
Advertisement
Right – but also misleading. “Giuliani’s numbers are meaningless for making comparisons across groups that differ dramatically in how the diagnosis is made,” observed Gigerenzer and colleagues in a 2008 paper on risk communication ().
That is because Giuliani was quoting five-year survival rates – the number of people diagnosed with a disease in a given year who are still alive five years later. But while prostate cancer in the US is generally diagnosed through screening, in the UK it is diagnosed on the basis of symptoms. Screening tends to pick up the disease earlier, leading to one source of bias in the comparison.
Suppose that of a group of men with prostate cancer all die at the age of 70. If the men do not develop symptoms until they are 67 or later, the five-year survival rate based on a symptoms approach is 0 per cent. Suppose, on the other hand, that screening had picked up the cancer in all of these men at age 64. The five-year survival rate in this case is 100 per cent, despite the fact that mortality is the same. Better survival rates don’t necessarily indicate a better outcome.
That is obviously an oversimplification, as earlier diagnosis through screening presumably increases the chance that corrective measures can be taken. But screening is not 100 per cent accurate. First there are false positives, in which the test incorrectly flags a healthy person as having cancer. Prostate screening also picks up non-progressive cancers, which will never lead to symptoms, let alone death. The exact extent of this overdiagnosis is unclear, but a rough estimate is that 48 per cent of men diagnosed in this way don’t have a progressive form of the cancer ().
Tricky comparison
False diagnosis and overdiagnosis both result in unnecessary treatment, and, potentially, significant harm – in the case of prostate cancer, men left impotent and incontinent. But overdiagnosis also inflates the five-year survival rate by including men who would not have died of prostate cancer anyway. “In the context of screening, survival is a biased metric,” says Gigerenzer. “The bottom line is that to learn which country is doing better, you need to compare mortality rates.”
The annual mortality from a disease is the proportion of people in the whole population who die from it in a given year. So which comes out better, the US or the UK? Figures from the period 2003 to 2007 published by the US National Cancer Institute indicate an age-adjusted mortality from prostate cancer of . Similar figures from Cancer Research UK for 2008 point to a mortality of . In statistical terms, that is a dead heat. Higher survival does not necessarily mean fewer deaths.
This kind of bias makes it tricky to compare survival rates in different countries, a difficulty often explicitly acknowledged by the authors of academic studies that use the metric (). Equally often, that subtlety is overlooked by politicians and journalists in search of a shocking sound bite or headline. So next time you are told that one country outperforms or underperforms another on some vital metric of health, take a close look at whether it is survival or mortality that is being quoted. If it’s the former, take the figure with a pinch of salt. Be aware, though, that this may increase your risk of heart disease.
Read more: “Spin doctors: The truth behind health scare headlines“
