A needle in the eye for science

On my commute home the other day I was listening to the Radio 4 PM show, which had an article about acupuncture. It was inspired by a news item about a recent trial (also reported by BBC news online) and was framed as an interview with two experts. One expert (I couldn't find out his name) was pro-acupuncture, the other (Ben Goldacre) was quite sceptical.

Now I have no axe to grind about acupuncture - it looks kinda sharp, but people seem to like it and as far as I can tell no-one's bled to death yet, but I found the whole article deeply depressing. Why? Because the scientific results were being horrifically misrepresented, and even the sceptic seemed oblivious to the perversion.

So here's the experiment and results in a nutshell:

Over 1100 patients take part in the study. One group is given conventional therapy, one is given acupuncture and the third is given a sham acupuncture treatment (that looks like acupuncture, but is essentially just randomly pricking people with needles).

The results:

1. 27% of the conventional group show an improvement
2. 47% of the acupuncture group show an improvement
3. 44% of the sham group show an improvement

So the question, science fans, is what does this tell us about acupuncture?

The pro-acupuncture guy stated (with no sense of shame as far as I could tell) that this proves that acupuncture works. Remember I was driving? I almost crashed.

Never have I seen such an incredible twisting of a set of results (actually that's not quite true, a few years back I saw an article in the Bristol Evening Post that said a car was stolen every 30 minutes in Bristol, and then a councillor was quoted as saying that this was ok, as a car was stolen every 3 min in the UK, and so Bristol was plainly better than most :-/ Maths, people. Maths! )

To me these results show two things (within the scope of the study):

1. Acupuncture is not significantly better than a random sham activity
2. Both Acupuncture and the sham are better than conventional treatment

So the most probable conclusion is:

1. Acupuncture benefits significantly from a Placebo effect, that is so strong that it even beats conventional medicine

This is not the same as "Acupuncture works", in fact its a damn site closer to "Acupuncture is all a load of bollards".

I'm sure that the German scientists behind the work came to this conclusion (the more balanced wording that is :-) but I was dismayed at the complete lack of ability of the general public, even a Radio 4 audience, to understand the results in an unbiased way.

The sceptic was trying to be nice, and was arguing that although the improvements were impressive, we should be spending our cash on less expensive Placebo effects. But I wanted him to just pick up the massive logical club lying conveniently next to him, and wallop that damn needle guy right in the family arguments.

Don't get me wrong, I haven't written acupuncture off completely. This is just one study, and it does clearly indicate that the Placebo effect is worth investing (and acupuncture is one way). But please don't get overexcited about your meridians.

And don't even get me started on homeopathy (where's my axe!). Dawkins says it best: