Imagine a therapy that supposedly cures most ailments and for which almost 100 per cent of all the published studies conclude is effective — in other words, a panacea which has been tested but never faulted by science. This can only mean that the treatment in question is a miracle cure which is useful for every single condition and in every single setting. Would we not all love to know such a therapy?
Simple common sense tells us, however, that such miracle cures cannot exist — unless, of course, we consider the wide range of treatments that fall under the umbrella of Traditional Chinese Medicine.
Take acupuncture, for instance. Most traditional acupuncturists will try to convince you that acupuncture is a veritable panacea, a treatment that works for anything and everything ranging from acne to zoster. In case you find this hard to believe, go on the internet and try to find a single condition for which acupuncture is not claimed to be effective.
What is more, acupuncture trials hardly ever generate negative findings — at least those that originate from China. We and others have shown that Chinese trials of acupuncture as good as never suggest that acupuncture does not work. This has led to the bizarre situation where one does no longer need to read the paper reporting a new Chinese study because one already knows what it shows, namely that acupuncture is effective. If that is so, one does not even need to conduct the study, since one already knows the outcome before the research has started.
Perhaps you think the ‘Western’ scientists who disclosed this baffling phenomenon are chauvinists who, for one reason or another, want to discredit Chinese science. In this case, you would probably want to wait for a team of Chinese researchers repeating our investigations.
Wait no more.
Chinese researchers identified all randomised controlled trials (RCTs) of acupuncture published in Chinese journals. A total of 840 RCTs were included in their assessment, and 838 of them (99.8 per cent) reported positive results. Only two trials (0.2 per cent) reported negative results. The authors concluded: ‘Publication bias might be a major issue in RCTs on acupuncture published in Chinese journals … which is related to high risk of bias. We suggest that all trials should be prospectively registered in international trial registry in future.’
For many years I, too, had been inclined to give my Chinese colleagues the benefit of the doubt and assumed that publication bias — the phenomenon where negative results tend to remain unpublished — might be the explanation. If so, trial registration would indeed be the answer. But think of it: publication bias might provide a reason for a preponderance of positive findings but it cannot truly explain that close to zero per cent of negative results see the light of day. There must be other factors involved.
One obvious explanation could be that many or most of the Chinese studies are — dare I say it? — dodgy to the point of being fraudulent. This allegation seems so outlandish that I would never have voiced it, unless there is some pretty solid evidence to back it up.
A recent survey of Chinese clinical trials has revealed fraudulent practice on a massive scale. China’s food and drug regulator carried out a year-long review and concluded that more than 80 per cent of clinical studies are ‘fabricated’. The investigators evaluated data from 1,622 clinical trial programmes of drugs awaiting approval by the regulator. Much of the data were found to be incomplete, failed to meet analysis requirements or were untraceable. Some institutions were suspected of deliberately hiding or deleting records of adverse effects, and tampering with evidence that did not meet expectations. ‘Clinical data fabrication was an open secret even before the inspection,’ an unnamed Chinese hospital director was quoted as saying. Contract research organisations seem to have become ‘accomplices in data fabrication due to cutthroat competition and economic motivation’.
The human rights activist Mai Ke went one step further, claiming that there is an ‘all-pervasive culture of fakery’ across all products made in the country. ‘It’s not just the medicines,’ he told Radio Free Asia. ‘In China, everything is fake, and if there’s a profit in pharmaceuticals, then someone’s going to fake them too.’
Crucially, he stressed that the problem also extends to Traditional Chinese Medicines: ‘It’s just harder to regulate the fakes with traditional medicines than it is with Western pharmaceuticals, which have strict manufacturing guidelines.’
Academic ethics is an underdeveloped field in China; this leads to a culture that is accepting of academics manipulating data. ‘I don’t think that the 80 per cent figure is overstated,’ another Chinese insider commented.
Considering data fabrication on such an epidemic scale, it seems much easier to understand the above-mentioned phenomenon, where nearly 100 per cent of Chinese acupuncture studies generate positive findings. Such trials heavily pollute the worldwide evidence, particularly because the Chinese trials constitute a major chunk of the current evidence base in this area.
If we agree that data fabrication has seriously detrimental effects, we must ask what we can do about it. I feel we have little choice but to distrust the evidence that originates from China. At the very minimum, we must scrutinise it thoroughly and sceptically. Whenever it looks too good to be true, we ought to discard it as unreliable.
Edzard Ernst, emeritus professor at the University of Exeter, is the author of Homeopathy: The Undiluted Facts and the awardee of the John Maddox Prize 2015 for standing up for science. He blogs at edzardernst.com.