Fakery on a massive scale means we can’t trust studies from China

Imagine a therapy that supposedly cures most ailments and for which almost 100 per cent of all the published studies conclude is effective — in other words, a panacea which has been tested but never faulted by science. This can only mean that the treatment in question is a miracle cure which is useful for every single condition and in every single setting. Would we not all love to know such a therapy?

Simple common sense tells us, however, that such miracle cures cannot exist — unless, of course, we consider the wide range of treatments that fall under the umbrella of Traditional Chinese Medicine.

Take acupuncture, for instance. Most traditional acupuncturists will try to convince you that acupuncture is a veritable panacea, a treatment that works for anything and everything ranging from acne to zoster. In case you find this hard to believe, go on the internet and try to find a single condition for which acupuncture is not claimed to be effective.

What is more, acupuncture trials hardly ever generate negative findings — at least those that originate from China. We and others have shown that Chinese trials of acupuncture as good as never suggest that acupuncture does not work. This has led to the bizarre situation where one does no longer need to read the paper reporting a new Chinese study because one already knows what it shows, namely that acupuncture is effective. If that is so, one does not even need to conduct the study, since one already knows the outcome before the research has started.

Perhaps you think the ‘Western’ scientists who disclosed this baffling phenomenon are chauvinists who, for one reason or another, want to discredit Chinese science. In this case, you would probably want to wait for a team of Chinese researchers repeating our investigations.

Wait no more.

Chinese researchers identified all randomised controlled trials (RCTs) of acupuncture published in Chinese journals. A total of 840 RCTs were included in their assessment, and 838 of them (99.8 per cent) reported positive results. Only two trials (0.2 per cent) reported negative results. The authors concluded: ‘Publication bias might be a major issue in RCTs on acupuncture published in Chinese journals … which is related to high risk of bias. We suggest that all trials should be prospectively registered in international trial registry in future.’

For many years I, too, had been inclined to give my Chinese colleagues the benefit of the doubt and assumed that publication bias — the phenomenon where negative results tend to remain unpublished — might be the explanation. If so, trial registration would indeed be the answer. But think of it: publication bias might provide a reason for a preponderance of positive findings but it cannot truly explain that close to zero per cent of negative results see the light of day. There must be other factors involved.

One obvious explanation could be that many or most of the Chinese studies are — dare I say it? — dodgy to the point of being fraudulent. This allegation seems so outlandish that I would never have voiced it, unless there is some pretty solid evidence to back it up.

A recent survey of Chinese clinical trials has revealed fraudulent practice on a massive scale. China’s food and drug regulator carried out a year-long review and concluded that more than 80 per cent of clinical studies are ‘fabricated’. The investigators evaluated data from 1,622 clinical trial programmes of drugs awaiting approval by the regulator. Much of the data were found to be incomplete, failed to meet analysis requirements or were untraceable. Some institutions were suspected of deliberately hiding or deleting records of adverse effects, and tampering with evidence that did not meet expectations. ‘Clinical data fabrication was an open secret even before the inspection,’ an unnamed Chinese hospital director was quoted as saying. Contract research organisations seem to have become ‘accomplices in data fabrication due to cutthroat competition and economic motivation’.

The human rights activist Mai Ke went one step further, claiming that there is an ‘all-pervasive culture of fakery’ across all products made in the country. ‘It’s not just the medicines,’ he told Radio Free Asia. ‘In China, everything is fake, and if there’s a profit in pharmaceuticals, then someone’s going to fake them too.’

Crucially, he stressed that the problem also extends to Traditional Chinese Medicines: ‘It’s just harder to regulate the fakes with traditional medicines than it is with Western pharmaceuticals, which have strict manufacturing guidelines.’

Academic ethics is an underdeveloped field in China; this leads to a culture that is accepting of academics manipulating data. ‘I don’t think that the 80 per cent figure is overstated,’ another Chinese insider commented.

Considering data fabrication on such an epidemic scale, it seems much easier to understand the above-mentioned phenomenon, where nearly 100 per cent of Chinese acupuncture studies generate positive findings. Such trials heavily pollute the worldwide evidence, particularly because the Chinese trials constitute a major chunk of the current evidence base in this area.

If we agree that data fabrication has seriously detrimental effects, we must ask what we can do about it. I feel we have little choice but to distrust the evidence that originates from China. At the very minimum, we must scrutinise it thoroughly and sceptically. Whenever it looks too good to be true, we ought to discard it as unreliable.

Edzard Ernst, emeritus professor at the University of Exeter, is the author of Homeopathy: The Undiluted Facts and the awardee of the John Maddox Prize 2015 for standing up for science. He blogs at edzardernst.com.

  • Central power

    We can trust China full stop. They are such nice chaps these Politburo members.

  • Grumbledog

    Acupuncture’s alright but Homeopathy turns you gay.

  • MichaelWoodhead2

    That RFA story is a distortion of the China FDA report. It did not find that 80% of trials had fabricated data, only that they did not meet new regulatory standards and needed to be resubmitted due to insufficient documentation – not quite the same thing.

    • Do you have a link to the CFDA report?

      • Malcolm Dison

        I believe the CFDA report was in Chinese. I could only find dead links but you may be able to find it on the CFDA website. Here’s a link to their page rebutting sensationalist claims made in the media http://eng.sfda.gov.cn/WS03/CL0757/165480.html

  • charles buck (UK)

    for actual fakery on a massive scale we need look no further than mainstream medicine. Ernst presupposes the moral higher ground when those at the very top of medicine are dismayed at how bent it has become. In his campaigning Ernst is either ignorant of this crisis in science or chooses to ignore it. Here are 3 quotes from some very prominent figures in medicine:
    “…drug companies don’t sell drugs, they sell lies about drugs…. Virtually everything we know about drugs is what the drug companies have chosen to tell us and our doctors… if you don’t think the system is out of control then please email me and explain why drugs are the third leading cause of death”

    Peter Gotzsche – Co-founder of the Cochrane Collaboration, the world’s leading evidence based medicine organisation. Author of Deadly Medicines and Organised Crime – how big pharma has corrupted healthcare

    “the benefits of drugs are exaggerated, often because of serious distortions of the evidence behind the drugs, a crime that can be attributed confidently to the industry… which has systematically corrupted science to play up the benefits and downplay the harms…”
    Richard Smith – former chief editor British Medical Journal

    “we landed in our present mess because of [the] … failure of clinical scientists, their institutions and the editors of the journals… to understand how thoroughly they were caught up by the marketers who paid them. I believe it will take a revolution to sweep away the decades of self-dealing by the industry”
    Drummond Rennie deputy editor JAMA

    • Do you think we should trust the conclusions of studies on TCM conducted in China?

    • Malcolm Dison

      As I understand it, the CFDA report is largely about mainstream medicine which is why it is rather disappointing, from the point of view of rationality and intellectual integrity, that Prof Ernst appears to twist the data to support his view of acupuncture and TCM whilst not acknowledging the implications of the report for science based medicine. There’s nothing to suggest the CFDA report even considered acupuncture.

    • edzard ernst

      a bit like the man who commented to a report of deaths on the railways: BUT THERE ARE FAR MORE FATALITIES ON THE ROADS!!!

      • John S.

        Your statement reveals your lack of common sense and your intellectual dishonesty (ie, your immorality).

        Anyone with some knowledge or real intelligence would see that your example is an exercise in misguiding the reader because the analogy you resorted to to counter Charles Buck’s excellent comment isn’t a suitable comparison (comparing railways deaths to road deaths).

        Much more people use roads than railways in industrialized nations. Therefore, it’s rather evident that more fatalities occur on the roads. Hence your glib argument is bogus.

        A much better comparison is between conventional medicine and alternative medicine in general because a similar percentage of people are utilizing their therapies. Or a specifically good comparison would be between the pool of users of medical drugs and the pool of users of dietary supplements in the US which is about equal. That comparison shows that practically no one dies from supplement use throughout a particular year yet over 100,000 people perish annually from the correct consumption of pharma drugs as a published author of the Orthomolecular Medicine News organization pointed out and then asks, “What Does A Health-Oriented, Reasonable, Humane Culture Focus On Eliminating? – A Serial Killer Who Murders A Handful Of People A Year, Or A Serial Killer Who Kills Over A Hundred Thousand People A Year?” (search online for the article “Health Risks of Dietary Supplements – The Proper Perspective” by Rolf Hefti).

        Anyone who looks at your “scientific” work carefully over the last few decades can see it’s saturated with fact distortions, misleading arguments, and unsubstantiated allegations (=”fakery on a massive scale”) against many therapies of alternative medicine, as one can expect from a well paid dogmatic member and mouthpiece for and of the commercialized allopathic medical establishment. Just read the 3 quotes Charles Buck had posted!

        Your disinformation serves to distract the mostly unwitting public from recognizing what the biggest threat to their health is: the huge conventional medical business. Not acupuncture, supplements or alternative medicine in general.

        • edzard ernst

          “Anyone who looks at your “scientific” work carefully over the last few decades can see it’s saturated with fact distortions, misleading arguments, and unsubstantiated allegations (=”fakery on a massive scale”) against many therapies of alternative medicine, as one can expect from a well paid dogmatic member and mouthpiece for and of the commercialized allopathic medical establishment.”
          MAMY THANKS!
          anyone stating such unsubstantiated ad hominem attacks and other fallacies, particularly without having the guts to disclose his full name, merely shows how wrong his position really is.

        • Anyone with some knowledge or real intelligence would see that your example is an exercise in misguiding[sic] the reader because the analogy you resorted to to counter Charles Buck’s excellent comment isn’t a suitable comparison (comparing railways deaths to road deaths).

          Ah… That wasn’t presented to counter @charlesbuckuk:disqus’s comment. It was simplifying the comment. It was Charles that was making that comparison and misleading the reader.

          You qualified your statement to a group of people “with some knowledge or real intelligence” which, in my mind, suggests you think of yourself as being among them. Classic Dunning-Kruger effect.

          There’s no point in addressing the rest of your comment because you’ve started from a false premise.

        • Vic Nurcombe

          This is a ridiculous pivot away from the question at hand, and irrelevant. The Chinese admit they fabricate data. None of the studies in the East can be replicated in the west.
          Pass a double-blind, placebo-controlled test, preferably many times in different places….then you have an actual medicine. Otherwise all you have is either ignorance, or deliberate falsification. If Chinese medicine worked as advertised, then tell me why Chinese Health statistics are so catastrophically bad???? A medical tradition that had 3,000 years head-start on the west, and they didn’t even understand that microbes can cause diseases. But that near-extinction sumatran tiger bones are good to get erections. It’s no wonder the Chinese spend their time laughing at the gullibility of certain westerners.

  • Poot

    Exams are cheated on an industrial scale there.

  • Malcolm Dison

    I’m a little shocked at the inaccurate, sensationalist, biased reporting that Professor Ernst presents here. I am only an undergraduate student but even I know that a scientist checks his sources and checks his conclusions for confirmation and other bias. There are certainly serious issues, but the figures, if you care to look at them do not support the assertion of “fakery on a massive scale”. I also think it is rather dishonest to use the CFDA study to take a swipe at acupuncture. Make no mistake, acupuncture is, in my opinion, 100% quackery, but the CFDA study doesn’t say it is and I think your extrapolation is misleading.

    • I think you’ve misunderstood Prof Ernst’s argument.

      • Malcolm Dison

        That’s entirely possible and I will withdraw my criticism if I am shown that is the case. My understanding of Prof Ernst’s argument is as follows :

        Data issued on July 22, 2015 showed that 1622 new drug applications to the CFDA were required to perform self examination on authenticity, integrity and compliance of clinical date. After deducting 193 applications exempted from clinical trial, 83% of the remaining 1429 were voluntarily withdrawn. Some were withdrawn for non-conformance to good clinical practice; some were for data incompleteness, which could not be traced back, and were insufficient to demonstrate the safety and effectiveness for the drug in application; some were for untruthful data, part of which cannot be excluded of the possibility of deliberate data fraud. As of September 2016 30 applications have been rejected due to “defects of authenticity”. This represents 2% of the original 1622 or 35% of the total number of applications examined. Therefore 80% of clinical data originating in China have been fabricated and therefore acupuncture and TCM are bunk.

        Prof Ernst has taken a second hand and inaccurate statistic to generate a sensationalist headline to support his particular point of view when it objectively does no such thing, but he also utterly ignores the fact that the data have grave implications for science based medicine. The Professor is, in fact, peddling pseudoscience.

        • edzard ernst

          my argument is that several groups have independently shown that nearly all Chinese acupuncture trials yield positive findings. this is unlikely explicable by just publication bias. therefore a recent report about ‘epic’ proportions of fakery [not my terminology] in Chinese research might better explain the phenomenon. [the headlines are never mine but from the Spectator]

          • Interesting. @edzardernst:disqus , does Spectator consult or take suggestions from the article author for the headlines?

          • edzard ernst

            no consultation has ever taken place; each time the headline comes as a surprise to me – but I find this quite interesting; I am learning how headlines are being done.

          • *Many* media sources could improve with just a little consultation or input from their authors.

  • M P Jones

    Chinese scientific rigour aside, acupuncture is a particularly difficult subject because the art, correctly executed, has largely been forgotten. This is even true in China where proper acupuncture and Chinese medicine traditions have to a large degree been replaced by Western medicine and poorly trained acupuncturists.

    The situation is even worse in the West. In the UK I suppose we may still have perhaps 10 properly trained and competent acupuncturists who know where the points are and who know how to read pulses, most of them trained at a school which existed in East Grinstead. I suppose these people will be in their fifties and sixties by now. What most call acupuncture nowadays is but a parody. It takes at least ten years of study and practice to become a competent acupuncturist and I suppose that if you want to find properly trained acupuncturists in any number you will need to visit Tibet.

    Hence, even using proper scientific methods you will most likely find that acupuncture no longer provides the cures it once did – though even incompetently used it can still be used for simple tasks like pain relief. In the hands of a competent practitioner it is extremely useful and versatile, particularly if correctly combined with proper old Chinese herbal medicine.

    • …acupuncture is a particularly difficult subject because the art, correctly executed, has largely been forgotten

      If this is the case, how do you know it? And if you know it how to you account for the wider profession not knowing it? Especially if it works.

      Your argument is flawed from the outset.

      • M P Jones

        I have, through direct contacts, been following the decline of Chinese, and the transition to Western, medicine for the past 25 years. My remarks are based partly on my own observations, partly on conversations with practitioners working in China and in the West. It is a long story, too long to report here. The current situation has many causes, one being the structure of the health system in China, another that few can be bothered to study and practise for ten years in order to make a rather meager living. As a result there are very few good teachers left anywhere.

        • Vic Nurcombe

          I spent some time in China, and met the Dean of Shanghai’s best medical school, at Fudan University. I was part of testing acupuncture in Australia decades ago……and remembered how dismally it failed even basic tests. When I talked to the Dean about this, all he could do was laugh…”only you westerners still believe……”. He recounted how Chairman Mao “reinvented” much of TCM in the 1950s, because he had no way of delivering modern medicine to 600 million poor peasants. People love *magic*, and TCM filled a void. Mao himself despised TCM, on the simple basis that it just doesn’t work. I don’t think rhino horn will give you a better erection…..I don’t believe that driving a tube into the guts of a living bear for its “bile” has any efficacy, and I don’t believe that eating a still-beating cobra heart will “freshen my blood”. How stupid would one have to be to buy this nonsense? Human life-span is the infallible measure of whether one’s medical tradition is working. The west has almost tripled life expectancy since 1900. Chinese health statistics have been a catastrophe for more than 3,000 years. Sounds like a failure to me, strictly on the numbers provided by the Chinese themselves.

          • M P Jones

            I am not an apologist for acupuncture or anything else. I am not particularly interested in this area, nor qualified (I use Western medicine in case I have a medical need :-)). My only errand was to point out a potential fallacy when testing ‘acupuncture’ due to the lack of qualified practitioners.

          • I am not an apologist for acupuncture…

            You should be aware that you are coming across as one.

            …or anything else. I am not particularly interested in this area,

            Engaging with the comments here suggests otherwise.

            nor qualified (I use Western medicine in case I have a medical need :-)).

            It’s not “Western” medicine. It’s just medicine. It’s used everywhere. If you want to prefix it with something try “modern”, “actual” or “science-based”. It’s the product of a process, not a geographical region.

            My only errand was to point out a potential fallacy when testing ‘acupuncture’ due to the lack of qualified practitioners.

            You concede it’s a potential fallacy but when it’s pointed out why your position may not be correct you double down. This is what makes you appear to be an apologist for acupuncture.

        • I have, through direct contacts…[snip]…there are very few good teachers left anywhere.

          So that explains how you think you know it. The important part that you didn’t address was how do you account for the wider profession not knowing it? Especially if it works.

          The claim that “few can be bothered to study and practise for ten years in order to make a rather meager living” is incorrect on both counts. If this was the case I wouldn’t be walking past 3 acupuncture clinics on the way to work. There is good money in quackery.

          Your argument is still flawed from the outset.

  • Thomas McCay

    Accupuncturists do not even use the same points for the same condition and there is no way to accurately tell where the points are. Like chiropractors, each one has a different story about the same patient. Five did different chiropractors will find five different “sublaxations” on the same spinal X-ray.

    The kind of medical fakery being talked about hear is at the heart of Chinese traditional medicine. Your rhino horn aphrodisiacs are completely fake as are so many of the “traditional” treatments.

    Fakery and cheating of every sort is “traditional” in China. I have not touched a food product from China for many years. No one who takes a serious look at China’s food industry would ever consume anything made there because you have no way of knowing what is in it. There are virtually no controls and regulations. In my city, the city food safety inspectors inspectors have given up because the majority of Chinese owned restaurants in town REFUSE to follow health and safety regulations.

    Note, China is a country, not a race. Chinese practices have nothing to do with genetics and what I’ve said about China is not a comment on Asia, Asian countries, or Asian peoples. It is about the corruption that is hip deep in Chinese society. It starts at the top and goes all the way to the very bottom. If you look at how the FDA is ran in America, you see the same things happening in America. When every one lies and cheats all the time, you are in real trouble. Society begins to lose cohesion. Sadly, dishonesty is part of human nature.

    People blindly defending these practices need to have a long talk with themselves and ask themselves, why????

  • Robert Davidson

    Every type of fraud related to the conduction of a clinical trial is just inadmissible. And, the information, which you have provided here that over 80 per cent of clinical trial studies are fabricated, is more than disturbing. The complete data management and the proper conduction of a trial is of great importance to patients’ health in general. But, unfortunately, some institutions alter data on account of personal and financial interests. Good clinical practices enforce tight guidelines on ethical aspects of a clinical study, but unfortunately, as we see, not all organizations comply with them. That’s why, for example, good clinical practice certification is really necessary, as it aims to prove that the staff of the organization that obtains it, has become familiar with the latest guidelines. Therefore, they can ensure that the data and results they provide are trustworthy.