Analysis of the April 2003 Draft Report of the New Zealand Ministerial Advisory Committee on Complementary and Alternative Health (MACCAH)

June 26, 2003

Section 1: Review of Complementary and
Alternative Medicine (CAM) in New Zealand and Abroad
1.3 Research, Evidence and Efficacy
[Comments in red by Stephen Barrett, M.D.]
The evidence base for complementary and alternative medicine is generally perceived to be poor. [That’s because it is poor. The vast majority of so-called “CAM” methods and products have never been studied and never will be. Of those that have, most have not been substantiated. That isn’t surprising because most of them make no sense.] Although more than 4000 trials have been carried out (Peters and Gillam 2001), there is still a shortage of strong evidence on the safety and efficacy of many CAM treatments. The reasons for this lack of high-quality evidence include the difficulties of applying standard medical research methods to some forms of CAM, and the lack of funding available for CAM research. [Both of these are lame excuses. Most CAM practitioners don’t do even the simplest of studies, which would be to follow what happens to their patients and report what they observe. This requires no outside funding.] However, the quality and clinical relevance of CAM research is said to be improving (Vickers 2000). [Some is better, some not. My impression is that as the quality of the studies improve, the less support they provide for the claims. I don’t doubt that a few herbs and dietary supplements will be found useful, and I suspect that spinal manipulation for back pain will find more support. But most other studies will be negative.]

There is currently a move towards evidence-based medicine in health care worldwide. Proponents of evidence-based medicine believe that decisions on which treatment to use should be based on sound evidence produced by well-conducted research studies. Quantitative methods such as randomised controlled trials (RCTs) and systematic reviews are regarded as the ‘gold standard’ in evidence-based medicine. The quality of evidence for particular treatments is sometimes graded according to the type of research studies from which it is derived.

Table 2: The hierarchy of evidence model (Source: Clinical Evidence Online 2002)
I.
II.
III.
IV.
V..
VI.
VII.
Systematic reviews and meta-analyses
Randomised controlled trials with definitive results
Randomised controlled trials with non-definitive results
Cohort studies
Case-control studies
Cross-sectional surveys
Case reports

There has been considerable debate over whether methodologies such as the RCT, with its emphasis on readily quantifiable outcomes and standardised treatment for all participants, can be effectively applied to CAM. [The claim that “CAM” methods can’t be tested with established scientific methods is a ploy.] For example, critics argue that while RCTs aim to determine whether treatments have an effect over and above the ‘placebo effect’, interaction between patient and practitioner is an essential component of many CAM modalities. [This is a bogus argument because it is possible to design controlled studies that take such factors into account.] ‘The healing process is traditionally in three parts: the self-healing properties of the body; the changes induced by non-specific effects of the therapist and the setting in which the therapy takes place; and specific effects of physical and pharmacological interventions’ (Kleijnen et al 1994). The second part is often referred to as the placebo effect.]

However, others have argued that it is both possible and appropriate to apply the key principles of evidence-based medicine ­ including the RCT methodology ­ to CAM (Vickers 1999).

NEW ZEALAND

Very little research has been published in New Zealand on the safety or efficacy of CAM. No quantitative research has been published in New Zealand, nor does it appear that any large-scale studies have been carried out.

There is, however, a great deal of anecdotal evidence regarding the safety and efficacy of some complementary and alternative therapies. There are also many unpublished case studies collected by complementary and alternative practitioners supporting the efficacy of various therapies. [“Evidence” of this type might be reason to consider doing appropriate studies of the method has a scientifically plausible rationale.]

There is currently no specific policy on CAM research. However, the Ministry of Health is establishing a database of known evidence for the safety and efficacy of CAM treatments. This will take the form of a searchable website presenting summaries of existing international CAM research and providing links to other evidence-based CAM websites. The database is expected to go live in the latter half of 2003. It has funding of $600,000 over four years.

UNITED KINGDOM

In 2000 the UK House of Lords Select Committee on Science and Technology reported that very little high-quality research on CAM has been carried out. There are, however, a variety of environments within which CAM research takes place in the UK, including the following:

  • The Department of Complementary Medicine at the University of Exeter is based within a school of postgraduate medicine and supports a Chair of Complementary Medicine. The Department carries out systematic reviews of research, as well as conducting its own clinical trials and surveys. [Most of the Department’s publications have been very well reasoned. Nearly all have concluded that what they have reviewed has been unsubstantiated.]
  • There is a Complementary Medicine Research Unit of the School of Medicine at the University of Southampton.
  • The Marylebone Health Centre, a GP practice, offers CAM therapies alongside conventional care and supports practice-based research.

The House of Lords Select Committee recommended that a small number of centres of excellence be established. However, in its response the UK Department of Health stated that such a move would be premature given the limited research capacity in CAM (Department of Health [London] 2001).

A strategy to build research capacity in CAM has since been developed by the Department of Health’s Research and Development Workforce Capacity Implementation Group in conjunction with the Foundation for Integrated Medicine and the Research Council for Complementary Medicine (Grey and Bailey 2001). [Footnote: The foundation was up in 1983 to carry out and promote rigorous research into complementary medicine.] Funded research posts will be created at a small number of selected higher education institutions. It is hoped that this initiative will enable the development of several centres of excellence in CAM research in the United Kingdom (A. Walker, Department of Health [London], personal communication 2002). The United Kingdom CAM research strategy was formally launched in April 2002.

As stated in their response to the Select Committee, the Department of Health is also planning to issue a call for research proposals on the use of CAM in the care of patients with cancer (Department of Health [London] 2001).

The Medical Research Council plans to encourage collaboration among researchers in the biomedical and CAM fields. At least two groups have expressed interest in applying for Council funding to pursue CAM research. These are based at academic institutes in Southampton and Bournemouth (B. Smith, Department of Health [London], personal communication 2002).

UNITED STATES

In 1998 Congress established the National Center for Complementary and Alternative Medicine (NCCAM) at the National Institutes of Health (NIH). NCCAM has an annual budget of around $US 68 million to develop and promote CAM research. [The budget is now over $100 million.] It conducts its own studies and also supports and funds the work of other organisations. In addition, NCCAM is involved in training CAM researchers and providing information on CAM. NCCAM is currently focusing on:

  • evaluating the safety and efficacy of widely used natural products such as herbal remedies and nutritional and food supplements (eg, mega-doses of vitamins)
  • supporting pharmacological studies to determine the potential interactions of CAM products with standard treatment medications.
  • evaluating CAM modalities such as acupuncture and chiropractic. NCCAM replaces a previous organisation, the Office of Alternative Medicine (OAM). Whereas OAM was primarily concerned with co-ordinating research and disseminating information, NCCAM has a much greater capacity to initiate and fund research.

[So far, despite spending more than $100 million, NCCAM has not concluded that any of the studied methods are either effective or ineffective. Critics believe that NCCAM should be abolished.]

There are also several CAM research centres based in academic institutions. These include the Center for Integrative Medicine and the Cochrane Collaboration Complementary Medicine Field at the University of Maryland.

The White House Commission on Complementary and Alternative Medicine Policy (WHCCAMP) has made several recommendations for improving the quality and co-ordination of CAM research, including increased funding for Federal agencies that carry out research on CAM (WHCCAMP 2002). [The WHCCAMP report and its recommendations have been thoroughly discredited. For one thing, in calling for more research, the Commission deliberately avoided mentioning that research might provide a basis for eliminating unsafe and ineffective methods and that vigorous action should be taken to ban methods that are already known to be unsafe or ineffective.]

CANADA

Apart from the evaluation of CAM products carried out by the Natural Health Products Directorate, there is as yet no national strategy on CAM research in Canada. The Natural Health Products Directorate has a comparatively small research budget of $Can 3 million over the next three years.

There are, however, many researchers working in the field in universities, and also two specialist research centres that focus specifically on CAM: the Research Centre for Alternative Medicine in Calgary, and the Tzu Chi Institute for Complementary and Alternative Medicine in Vancouver. [The Tzu Chi Institute Web site states that was schduled to close in March 2003.]

Research capacity and opportunities for collaboration in CAM research are growing. In 1999 the Integrative Medicine and Health Network was established. This group includes both CAM and non-CAM practitioners, researchers and scientists. It has proposed that a Canadian Office for Complementary and Alternative Health Care be established to guide the development of a coherent CAM research policy. The proposed Office would be part of the Canadian Institutes of Health Research, which is currently under development (de Bruyn 2001).

AUSTRALIA

Apart from the evaluation of complementary medicines and their ingredients carried out under the auspices of the Therapeutic Goods Administration, there is no national strategy on CAM research. In its 2002­03 Federal Budget Submission, the Australian Medical Association called on the federal government to provide $A 1 million to fund research on the potential benefits and adverse effects of complementary medicines and therapies (Australian Medical Association 2002), but this bid was not successful.

Current Australian research initiatives that deal with CAM include:

SINGAPORE

Traditional Chinese medicine (TCM) is covered by the Singapore government’s life sciences research initiative. Some TCM research has been carried out by the National University of Singapore, the Health Sciences Authority, the National Cancer Centre and other institutions. There have been no structured studies of the improvement of outcomes by other CAM modalities.

CHINA

There are 72 traditional Chinese medicine (TCM) research institutes throughout China (Wong Kum Leng, Singapore Ministry of Health, Presentation 2001). TCM research is funded and co-ordinated by the State Administration of Traditional Chinese Medicine. There is a large amount of published and ongoing research on drugs and treatment techniques. Research is also under way to identify the mechanisms by which acupuncture works. Acupuncture has traditionally been explained as healing through manipulation of chi (vital energy), but Chinese as well as Western researchers now wish to understand it in modern scientific terms (Moyers 1993).