Unconventional Cancer Treatments

Appendix A: Methods of the Study

The Assessment Process

John Dingell, Chairman of the U.S. House of Representatives Committee on Energy and Commerce wrote to OTA in August 1986 asking that a study be done of treatments for cancer that are "out of the mainstream." The request stated:

Many of these treatments may be without benefit, some may actually be harmful, and some, probably a small number, may have value. However, there is a general lack of objective information about them, thus making rational decisions about such alternative therapies extremely difficult.

The letter asked OTA to describe the treatments and look into policy issues surrounding their availability and evaluation. Congressman Dingell's letter also recognized the letters OTA had received from then-Congressman Molinari and 42 other Members of the House and Senate requesting that OTA review the existing data on the efficacy of a particular treatment, Immuno-Augmentative Therapy (IAT), and design a formal evaluation plan for that treatment. Congressman Dingell suggested that OTA consider the IAT work as a case study within the larger study. By statute, OTA may undertake assessments at the request of the Chairman of any full committee of the Congress. The Chairman may request the work personally, on behalf of a ranking minority member, or on behalf of a majority of the committee members. OTA's Board may also request work, as may the Director of OTA, but individual Members of Congress, such as then-Congressman Molinari do not have authority to request assessments.

In response to Congressman Dingell and the requests about IAT, OTA proposed a study titled "Nontraditional Methods of Cancer Management: Science and Policy Issues," which was approved by the Technology Assessment Board (TAB; OTA's governing body) in September 1986. (The title was changed twice, based on advice of the Advisory Panel and others, ending with the published title of Unconventional Cancer Treatments.) The study was to begin in January 1987, with a final report to be delivered to TAB in June 1988 (with publication some months later), and preceding that, the case study on IAT to be delivered in December 1987. Because of the difficulty of gathering information for the study and the extensive interactions with the public and Congress concerning it, the TAB delivery date was extended four times, and the report was finally delivered to TAB in July 1990.

Project Advisory Panel and IAT Working Group

One of the first tasks was the appointment of an Advisory Panel, a feature of every major OTA project. Advisory Panels include individuals from outside the Federal Government with expertise in the various areas covered in the assessment, and representing the important points of view on the issue at hand. Advisory Panels do not write, nor do they take responsibility for, the content of OTA reports, but their participation is considered essential to producing fair and authoritative reports.

Choosing an Advisory Panel for this study required OTA to go beyond the mainstream medical sphere in which it usually operates. Many contacts with unconventional representatives were made through an initial contact with Michael Lerner, President of Commonweal, who was asked to be a special consultant to the study. In addition, a long list of individuals recommended to be on the Advisory Panel was received unsolicited from a group called the "Coalition for the Evaluation of Alternative Therapy," a coalition of preexisting groups that appeared to have formed in response to the OTA study. (The Coalition no longer exists.) The Advisory Panel was chosen with consultation from Dr. Lerner. The chairperson, chosen by OTA, is Rosemary Stevens, a medical historian who had not worked specifically in the area of unconventional medicine. The Advisory Panel contains individuals generally supportive of unconventional treatments (8 members), individuals who were openly opposed (2), and individuals with technical expertise clearly allied to mainstream medicine and research, but who had not taken a position against unconventional treatments (8). Dr. Lerner also functioned very much like an Advisory Panel member in his capacity as special consultant.

In addition to the Advisory Panel, the project staff appointed a second group, the "IAT Working Group," to assist with designing a clinical trial protocol for IAT. This group consisted of individuals with technical expertise in clinical trial design, plus an appointed representative each from the National Cancer Institute (NCI) and the Food and Drug Administration (FDA). Lawrence Burton, developer of IAT, was asked to participate as well; he appointed a patient, the founder of the IAT Patients Association, to represent him and also asked that a statistician (the husband of one of Burton's patients) who was interested in IAT, be included. This was done. Dr. Lerner was associated with this group as well.

Meetings of the Advisory Panel

First Meeting

The Advisory Panel first met in July 1987. A preliminary outline of the report was presented by the project staff. Areas for contract work had been identified, as had some potential contractors. At that time, however, only one contract had been let, to Michael Lerner, to produce a "conceptual framework" for analyzing the various treatments to be covered, and to provide background information on a wide range of treatments. Advisory Panel guidance was solicited for prospective contractors for the other areas. The meeting was notable for bringing the unconventional treatment supporters together with the mainstream in a neutral forum. Discussion was generally non-confrontational and informative. However, undoubtedly because of the difficulty of the topic and lack of precedence for a study of this type, no clear direction for the report as a whole emerged.

Second Meeting

The second Advisory Panel meeting was held in late July 1988. A partial draft of the report was sent to the Advisory Panel for discussion at this meeting. OTA had asked the panel not to circulate this draft to others because of its preliminary nature, but, as it turned out, it was widely copied and circulated, and a large number of observers at the panel meeting had copies. One, Robert Houston, had prepared a critique, "Objections to a Cover-Up: The OTA Report on Alternative Therapies," which he distributed at the meeting. Other groups, e.g., Project Cure and the IAT Patients Association, also passed out literature. Observer comments were allowed by the chairperson as appropriate. The tense atmosphere and combative nature of many of the observers and panel members strained the discussion. There was a great deal of criticism of the draft, largely from the panel members on the unconventional side. Their main concerns were, first, that there had not been enough time for them to review the draft, that the draft was incomplete, and particularly, that policy issues were presented orally at the meeting, but had not yet been written. There was also criticism that too much emphasis had been placed on adverse effects, that the "scientific development" of the treatments was not discussed sufficiently, and that traditional practitioners and New Age approaches were given too prominent a place.

Third Meeting

The latter half of 1988 and all of 1989 was spent rewriting the report almost in its entirety, relying less on contract papers and more on OTA staff research, which proved necessary for a thorough treatment of the subject. A complete draft, with policy options, was sent to the Advisory Panel about one month before a meeting in early March 1990. Copies of the draft were also sent to more than 200 other individuals and groups for review before the meeting. OTA invited requests from outside reviewers to address the meeting if they had serious criticisms of the report. Fifteen responded, and their statements took up the morning of the meeting. These were:

The presentations ranged from reasoned critique to presentation of additional information to shouted personal attacks on the integrity of the project staff.

In the afternoon, the panel discussed the draft. There seemed to be two major themes: first that, throughout the draft, OTA had failed to highlight the "middle ground," except in the chapter on psychological and behavioral approaches. Second, that what was needed for fair treatment of unconventional cancer treatments was a "level playing field." There was also considerable discussion about the tone of the report, which was perceived as unduly critical of unconventional treatments. To the extent possible, given the hostile atmosphere, policy options were discussed, as well as other parts of the report. As at the second meeting, many spectators, in addition to those scheduled, were allowed opportunities to speak.

The OTA Director, along with the Assistant Director for Health and Life Sciences, the Health Program Manager, the project staff, and other OTA officials, were present for the entire meeting. A number of TAB staff and other Congressional staff members also attended.

IAT Case Study

The conduct of the IAT case study is discussed in the latter part of chapter 6. The IAT Working Group met twice during the course of the study, in March 1987 and May 1988. OTA staff (accompanied by an FDA official on the second trip) met with Burton and his representatives in the Bahamas twice. These meetings are all discussed in some detail in chapter 6.

Workshop on Evaluation Methods

Early in the project, in October 1987, OTA held a 2-day workshop at Commonweal, in Bolinas, California, hosted by Michael Lerner, to explore issues related to evaluating unconventional cancer treatments. The idea was to bring together experts in evaluation methodology with individuals knowledgeable about the details of unconventional cancer treatments. Some members of the Advisory Panel, members of the IAT Working Group, one evaluation expert from the National Cancer Institute, and several others attended. Some of the ideas that arose from the workshop are discussed in chapter 12.

The Review Process

About 250 copies of the February 1990 draft were sent out for review. Comments were requested by the end of March, but the deadline was extended for anyone asking for more time. Comments were received through the end of May from a total of approximately 75 individuals and organizations. Many comments consisted mainly of attacks on the integrity of the project staff and other OTA officials. Others were of a more substantive nature. Eight members of the Advisory Panel generally supportive of unconventional treatments wrote a set of joint comments, including discussion of the "middle ground" and "level playing field" issues of the third panel meeting. Robert Houston again wrote a long critique, which was published in March 1990 by "People Against Cancer," entitled "Misinformation from OTA on Unconventional Cancer Treatments."

Revisions to the report included attempting to obtain and incorporate, to the extent possible, new material suggested by reviewers, and some restructuring in response to comments (e.g., elimination of the chapter on spiritual approaches). The final report is significantly more complete as a result of the review. In addition, the IAT case study, whose planning with Burton had recently come to an unsuccessful end, was folded into a separate chapter about IAT. In addition to the usual editing done at OTA, the Advisory Panel chairperson offered to edit the summary and options chapter (chapter 1), as the last step before the final draft was sent to TAB, to assist with what were referred to as "tone problems" by Advisory Panel members. In their joint letter to OTA, a group of panel members referred to the "distinguished Advisory Board chairman, Rosemary Stevens, Ph.D.," stating that they would be "very happy with a tone that reflected her judicious historian's balance." All of Dr. Stevens editing suggestions were incorporated into the final version.

Mail-in Campaigns Relating to the Project

OTA and Members of Congress, particularly the membership of TAB, have been the object of mail-in campaigns by several unconventional treatment advocacy groups during the course of the project. Thousands of pre-printed postcards and letters (e.g., from the Coalition for Nutrition and Health, Project Cure, the Foundation for the Advancement of Innovative Medicine), and tearsheets from an alternative magazine (Health Freedom News, the magazine of the National Health Federation) have been received. The content of these has varied, but they have generally been highly critical of OTA practices, the project staff and other OTA officials, and the draft report. OTA did not, in general, respond individually to these form letters.

TAAC Meeting

The February 28, 1989 meeting of OTA's Technology Assessment Advisory Committee (TAAC) was devoted to this project. (TAAC is a Congressionally mandated group of 10 eminent individuals appointed by the Technology Assessment Board to advise OTA. In addition, the Comptroller General of the United States and the Director of the Congressional Research Service serve as statutory members.) Rosemary Stevens, the Chairperson of the Advisory Panel, Michael Lerner, special consultant to the project, and Richard Riegelman, a member of the Advisory Panel, also participated in the meeting. In a memorandum to TAAC members, the Director of OTA gave this purpose to the discussion:

The sharpness of the controversy about the substance and approach to this study has greatly exceeded the normal clash of opinions accompanying OTA's studies. For this reason, we are asking the TAAC and three guests to consider the fairness and thoroughness of the study approach and results.

Briefing materials were sent to TAAC members before the meeting, acquainting them both with the assessment itself and the controversies that had arisen around it. At the meeting, the history of the project was reviewed, in both content and process. Plans for finishing the project and for ensuring objectivity to the end of the process were discussed in detail. After finishing at OTA, TAAC met with TAB and discussed its review of this study.

Communication With Congressional Staff

Over the course of this project, about half of all the Congressional offices contacted OTA by letter or telephone for information. These requests were usually in followup to contacts by constituents, who either wrote individual letters or participated in one of several mail-in campaigns organized by advocacy groups. Project staff discussed the project by telephone and provided current "one-pagers" on the main project and on IAT in response to these requests.

Communication With the Public

OTA received hundreds of telephone calls and letters (both individual and mass-produced) about this study. Most phone callers were looking for information about particular treatments, usually on behalf of a friend or relative with cancer. Most had found out about the study through articles in alternative magazines or papers or by word of mouth. To the extent possible, project staff provided general information or directed them to other sources of information. The one-page study descriptions were also sent to the public. Particularly during the period of the draft review, many people called and wrote to register disapproval of the report. In general, these were not people who had seen copies, but were repeating views publicized by advocacy organizations.

Other Inputs to Report

Contractor Papers

Michael S. Evers, J.D.: "Legal Constraints on the Availability of Unorthordox Cancer Treatments: Freedom of Choice Viewpoint"

The purpose of this contract was to describe the laws, regulations, and other legal constraints on unconventional cancer treatments, specifically giving the legal basis for the "freedom of choice" point of view. Evers heads one of the major unconventional treatment advocacy groups. It was used in writing chapters 10 and 11. A similar contract was awarded to Ronald D. Schwartz and Rebecca L. Burke, to represent the "consumer protection" point of view.

Vicki S. Freimuth, Ph.D.: "The Public Search for Information on Unorthodox Cancer Treatments: The CIS Experience"

The purpose of this contract was to describe the way in which the National Cancer Institute's Cancer Information Service handles requests for information about unconventional cancer treatments. The contract included an analysis of all calls recorded by the CIS over a 4-year period in which unconventional treatments were discussed. Information from this contract appears in chapters 7 and 8.

Janice Guthrie: "Sources of Information on Unorthodox Cancer Therapies" and "Personal Narrative"

Under this contract, Ms. Guthrie provided OTA with a comprehensive list of sources of information on unconventional cancer treatments and she obtained for OTA brochures, audio tapes, and other sources of information from specific clinics and practitioners. Her narrative, referred to in chapter 7, describes her personal experience with unconventional cancer treatments. The material provided under this contract was used in many places in the report.

Sharon Hammond: "An Examination of the Public Education Efforts of Three Mainstream Cancer Organizations"

The purpose of this contract was to describe the educational activities related to unconventional cancer treatment of the American Cancer Society, the National Cancer Institute, and the American Society for Clinical Oncology. Some of this information appears in chapter 8.

David J. Hufford, Ph.D.: "Cultural and Social Perspectives on Unorthodox Cancer Treatment"

This report provided general background and context for unconventional cancer treatments. It is referred to in several places in the report.

David J. Hufford, Ph.D.: "Selected, Unorthodox Cancer Practitioners"

This report describes "New Age" and traditional healers, faith healers, Christian Science healers, and others. Hufford's report was instrumental in helping to understand these healing systems and in deciding not to cover them in detail in the report.

David J. Hufford, Ph.D.: "Health Food Store Survey on Alternative Cancer Treatment Information"

Under this contract, Hufford coordinated a survey by graduate students of information about unconventional cancer treatments available in health food stores three cities. The results are reported in chapter 7.

Michael Lerner, Ph.D.: "Toward a Framework for the Analysis of Unconventional Cancer Therapies"

This contract report served to help categorize treatments generally by content, and described positive aspects of a number of specific treatments in each category. It also provided general background material. Material from this report is referred to in a number of places in the report.

Daniel J. Morris, M.D.: "Feasibility of Identifying and Gaining Access to Medical Records of IAT Patients Who Have Also Been Seen in Florida Medical Facilities Since January 1986"

The purpose of this contract was to determine whether any useful information about IAT could be gathered from other medical institutions where IAT patients had been treated. Dr. Morris discussed this approach at an IAT Working Group meeting. It is discussed briefly in chapter 6.

Anne Paxton: "Practitioners of Unorthodox Cancer Treatments"

The purpose of this contract was to describe various types of unconventional practitioner (e.g., holistic physicians, naturopaths, homeopaths). Little of the information from this contract was used in the final report.

Terence M. Phillips, Ph.D., D.Sc.: "Critical Review of Published Pre-Clinical Studies by Lawrence Burton, Ph.D."

The purpose of this contract was to review Burton's published work of the 1950s and 1960s, on fruitflies and mice, mainly, which Burton says is the basis of Immuno- Augmentative Therapy (IAT). Phillips is a clinical immunologist and protein chemist. Material from his report appears in chapter 6.

Terence M. Phillips, Ph.D., D.Sc.: "Review and Analysis of Lawrence Burton's Patented Processes and Products"

OTA was urged by Burton's supporters to review his patents. Phillips was asked to analyze the patents, critique the procedures, and determine, if possible, what materials would be produced by them. He was also asked to determine any relationship to Burton's published pre-clinical work. Material from this patent review appears in chapter 6.

Ronald D. Schwartz, J.D., Rebecca L. Burke, J.D.: "Legal Constraints on the Availability of Unorthodox Cancer Treatments: Consumer Protection Point of View"

This purpose of this contract was to describe the laws, regulations, and other legal constraints on unconventional cancer treatments, specifically giving the legal basis for the "consumer protection" point of view. It was used in writing chapters 10 and 11. (See above, report by Michael S. Evers.)

Patricia Spain Ward, Ph.D.: "History of Hoxsey Treatment," "History of Gerson Therapy," and "History of BCG"

The purpose of this contract was to describe the historical antecedents and development of three popular unconventional cancer treatments. (A fourth, macrobiotics, was included in the statement of work but was dropped by mutual consent of the contractor and OTA.) Material from these reports appears mainly in chapters 3 and 4.

Robert Watson: "Quality of Life Assessment Instruments: A Review of 32 Current Measures and One Classic"

The purpose of this contract was to provide an annotated bibliography of methods used to assess quality of life. It was decided later in the project not to cover this in detail.

Jack Z. Yetiv, M.D., Ph.D.: "Adverse Medical Consequences of Unorthodox Cancer Treatments"

This report provided information on reported and suspected adverse effects of unconventional treatments, to complement the selectively positive information in the Lerner contract. The contract was let after receiving Lerner's draft, in which he stated that his emphasis was on the positive aspects, and he had not covered the "casualties of unconventional cancer therapies" thoroughly.

Other Sources

A paper prepared by Keith I. Block, M.D., an Advisory Panel member, and Charlotte Gyllenhall, Ph.D., ("Nutrition: An Essential Tool in Cancer Therapy") was used as a primary source in chapter 3. Extensive conversations with Richard Jaffe, an attorney associated with several unconventional practitioners, provided much of the basis for the "freedom of choice" discussion in chapter 10 and for some of the ideas presented in chapter 9 concerning insurance coverage for unconventional cancer treatments.

A review of 50 case histories of patients in the Kelley program, as described in an unpublished manuscript by Nicholas Gonzalez, M.D., was carried out by members of the Advisory Panel at the request of OTA. The results are reported in chapter 3. Some Advisory Panel members also reviewed case histories of patients treated with a macrobiotic regimen, as reported by Vivien Newbold, M.D. This also is reported in chapter 3. To obtain information about laboratory testing of the herbs contained in Hoxsey's formulas and Essiac, OTA had searches of the published literature carried out by NAPRALERT, which maintains a data base on natural products.


OTA Report Contents ||||| OTA Report Index ||||| Quackwatch Home Page

This page was posted on August 3, 1998.