Debunking the Myth that "Only 10-20%
of Medical Procedures Are Proved"

Stephen Barrett, M.D.

Critics of standard medicine sometimes quote a 1978 OTA report which stated that "only 10 to 20 percent of procedures currently used in medical practice have been shown to be efficacious by controlled trial." [1] Some promoters of unproven "alternative" practices cite this statistic to justify whatever they do.

The OTA report did not explain how the "10 to 20%" figure had been determined, but in a 1995 letter to The Lancet, statistician Kerr L. White, a former OTA advisor, revealed that he was its source [2]. White's based his estimate on a 1963 survey of the prescribing practices of 19 family doctors in a northern British town for two weeks—one in December 1960, and another in March 1961. [3]

The survey wasn't trying to evaluate the science of medical practice. Its purpose was to look toward controlling prescribing costs in terms of "standard" (i.e. "generic") versus "proprietary" drugs. The "intent" of each prescription was analyzed according to how specific it was for the condition. These were divided into categories of "standard" or "proprietary" drugs. Intent was "specific" for the condition for which it was prescribed only about 10% of the time; "probable" in about 22%; "possible" in 26%; "hopeful" in 28%; "placebo" in 10%; and, "not stated" in 3.6%. From these data, White estimated that 10-20% of the medicine practiced was evidence-based; the combined placebo and Hawthorne effects accounted for another 20-40%; and the rest (which he referred to as a "mystery") accounted for 40-70%.

It's obvious that a small survey of this type provides no basis for evaluating the degree of evidence-based medicine overall. Moreover, the survey's setting had little relevance standard American medicine in 1978 and much less today. But White explained:

Some 20 years ago, as a member of the original Health Advisory Panel to the US Congressional Office of Technology Assessment, I ventured the 10-20% figure again and invited anyone to provide more timely data. No one could. The figure was immortalized in OTA circles and publications for almost a decade. In countless addresses and conferences I often challenged others to provide better evidence but none was forthcoming. So the northern industrial town "armchair" assessment persisted.

White created a myth that was damaging to standard medicine and useful to the promoters of quackery. The intent of the 1978 OTA report was not to advocate an open marketplace, or that there be less science in health care. Its purpose was to find ways to improve the process through which the safety and efficacy of health care technologies are established. White merely was emphasizing deficiencies in the then-current system. Nevertheless, he can be criticized for implying in a report to Congress that the survey results reflected standard medical practice in the USA. It's safe to assume that nobody responded to his "challenge" because definitive studies had not been done.

Most clinical studies are focused on whether or not methods work rather than how diligently they are used. But some studies have looked at the extent to which medical decisions in various settings were evidence-based. The figures they report depend in part on what practices they study and what criteria they use to judge the evidence. A 2001 report that summarized 15 such studies found that scientifically supportable interventions ranged from 45% to 97%, with a median of 77% [4]. I haven't found a more recent compilation, but, as Professor Edzard Ernst pointed out in 2017 in his blog and a subsequent comment:

References

  1. US Congress, Office of Technology Assessment. Assessing the Efficacy and Safety of Medical Technologies. Washington D.C., Sept 1978.
  2. White KR. Evidence-based medicine (letter). The Lancet 346:837-838, 1995.
  3. Forsyth G. An enquiry into the drug bill. Medical Care 1:10-16, 1963
  4. Imrie RH, Ramey DW. The evidence for evidence-based medicine . The Scientific Review of Alternative Medicine 5(2):104-107, 2001.
  5. Ernst E. Is conventional medicine based on good evidence? Edzard Ernst Blog, May 16, 2017.

Thus article was posted on August 12, 2018.

Links to Recommended Vendors

  • PharmacyChecker.com: Compare drug prices and save money at verified online pharmacies.
  • ConsumerLab.com: Evaluates the quality of dietary supplement and herbal products.
  • Amazon.com: Discount prices, huge inventory, and superb customer service
  • OnlyMyEmail: Award-winning anti-spam services.
  • 10 Types: Website design, development, and hosting with superb technical support.