Health Systems

Determining whether a patient should be treated using a new technology--be it a promising screening test, a new surgical device, or an organ transplant--can be hard work. In an era when clinicians increasingly feel daunted by expectations for accessibility and service, clinicians and members alike are further challenged--and vexed--by the need to interpret and understand new technology and its applicability to individual members' needs. This endeavor requires understanding that the technology's effectiveness must be integrated with four other factors:

• The clinician's desire to help the patient,
• The patient's desire to be helped,
• Mutual intent of clinician and patient to find the right treatment, and
• Efforts of vendors to position and establish their products.

Choosing to use effective new technology further requires clinicians and patients to understand and participate in a fully informed decision with which they may be inexperienced or uncomfortable. Moreover, this challenge is occurring in a context of discontent and suspicion about the delivery and cost of health care. In this context, economic considerations and resultant stewardship tradeoffs can affect decisions of individual clinicians and health plan members whose preferences are more compatible with a nonexistent health care system: health care without financial constraint.

Choosing new technology for use by clinicians, health plans, and health plan members can thus be summarized as a threefold challenge:

• Ascertaining medical appropriateness;
• Creating a shared decision between clinician, health plan member, and the health plan; and
• Working within constraints of the health plan member's insurance coverage.

Part 1 of this article addresses the first of these challenges: technology assessment and its roots in evidence-based medicine. Many clinicians and health plan members may perceive decisions about technology appropriateness and benefit coverage to be an intrusive reality of medical practice; nonetheless, my intent here is to examine technology assessment as an increasingly valuable opportunity for improving members' health. This commentary is intended to represent this Permanente physician's personal opinions and perspective and is not a policy statement of the Interregional New Technologies Committee, the Care Management Institute, the Permanente Federation, or any other body within Kaiser Permanente (KP) or of the Technology Evaluation Center of the Blue Cross and Blue Shield (BC-BS) Association.

Determining Medical Appropriateness
For most--if not all--clinical conditions, clinicians and health plan members find themselves increasingly buffeted by vast amounts of information from varied sources about what treatments potentially "work" and what treatments appear to add value. This trend is especially true for new (and often, promising) technology.

The relative roles of clinician, health plan member, and health plan in supplying and interpreting clinically relevant information are in flux. Information technology allows health plan members to access knowledge which previously was only rarely available to nonclinicians. Clinicians commonly encounter patients whose didactic knowledge of certain relevant clinical material rivals or exceeds that of the clinician.

Previously serving as suppliers of clinical content, clinicians are becoming decision-making partners with patients by helping them to refine clinical context from an ocean of content. The clinician's memory increasingly must access, store, recall, and integrate an ever-growing, ever-evolving knowledge base. Information technology, such as the Internet and the electronic medical record, are beginning to address this challenge but as yet do not meet the full range of needs for timely and relevant information at times when decisions are made and care is delivered. In addition, increasingly broad access to information and to methods of data storage is creating an escalating demand for a credible, durable way to assign relevance and importance to competing knowledge resources.

The clinician's role is thus migrating from the benevolent, wise, paternal "Marcus Welby" model to a more demanding and complex three-part role: 1) dissect and solve clinical problems, 2) explicitly identify and characterize options, and 3) participate in achieving a shared, well-informed decision with an empowered health plan member or patient. (Of course, the fictional Dr Welby practiced in a simpler time of less medical knowledge and fewer treatment options. He also treated fewer patients in a year and was more likely to have sufficient time to be "all things" to those in his care. More patients with more complex medical histories and dilemmas transit through one episode of "ER"--the current television hospital drama--than "Marcus Welby, MD" saw in his entire fictional medical career.) The requirements of clinical judgment and experience are enduring; however, the currently evolving clinical paradigm also requires greater competence in knowledge management as well as the ability to explicitly define clinical context.

Evidence vs Eminence
Clinicians have always based their decisions on the evidence available and known to them. This evidence was acquired through personal experience in clinical practice, by reading the medical literature, and in formal discussions and informal interactions with peers. Historically, this evidence guided clinical practice; and in educating practitioners, this evidence was given further context by the "eminence" of its source. From chief resident to department chairperson to national expert, personal eminence conveyed credibility, was appropriate, was earned, and got us to where we are now. The dilemma is whether this experiential and implicit approach to knowledge acquisition and analysis is adequate for the task ahead.

Can an unaided individual or group consistently and comprehensively analyze the growing body of medical knowledge in an experience-based, implicit manner? Will this analysis be reproducible across settings and across time? In part, the current variability (and resultant expense) of medical care delivery has been attributed to this historical reliance on implicit medical decisions. The theory now being tested is that sharing knowledge explicitly, agreeing to a definition of evidence, and then basing practice decisions on that definition will result in delivery of higher quality, less variable medical care.

Several examples of new technology being implemented before adequate research had clearly shown effectiveness have supported this concern about using implicit evidence for analyzing the suitability of new technology. A prominent example is the ongoing controversy over the benefit of high-dosage chemotherapy with autologous bone marrow infusion for treating metastatic breast cancer. A decade of advocacy for this seemingly intuitive and logical clinical intervention included medical-legal pressure and legislative mandates for insurance coverage. The appropriateness of the intervention was ultimately called into question by results of randomized controlled trials that failed to confirm a consistent benefit justifying the considerable risk.^1,2 This question has important implications for overall health care costs and for future health plan decisions. Of even greater concern to clinicians who care for these patients is the failure of members to make a fully informed decision: Because the projected benefits of the intervention were substantially overstated--they were based on implicit observation of initial trials only--some patients referred for the intervention were not adequately informed about the balance of its risks and benefits. This and other examples have stimulated the quest for shared standards (and processes) of evidence analysis to better substantiate expected benefit more explicitly and reproducibly. Stated otherwise, the desire is to better communicate what is known and what is as yet unconfirmed--and thus what is potentially both promising and harmful.

The KP Interregional New Technologies Committee (described on page 46 in this issue of The Permanente Journal) is charged with helping clinicians and health plan members to make decisions about the general medical appropriateness of new medical technology on the basis of what is known about this technology.

An enduring key principle of Permanente Medicine that should be emphasized is that the ultimate decision about the medical appropriateness of using a particular technology for any given KP Health Plan member is made by the clinicians responsible for the care of that member. In certain complex situations, such as organ transplantation, KP clinicians seek a decision on medical appropriateness from committees of informed and involved clinicians.

The KP Interregional New Technologies Committee uses an explicit process to analyze and summarize the evidence supporting use of new technology. This approach is modeled after criteria articulated by an ongoing collaborative venture between KP and the BC-BS Technology and Evaluation Center:

1. Is the technology subject to licensing by an oversight body such as the US Food and Drug Administration (FDA)? If so, has the technology been approved by that body?

Of importance is that the "evidence standard" and threshold used by the FDA for licensing a technology as "safe and effective" is often insufficient to fully support the medical appropriateness of applying the technology to a specific clinical situation. FDA approval is thus necessary--but is itself insufficient--to justify use of new medical technology.

2. Does adequate evidence support the appropriateness of using the technology?

Meeting this criterion enables determination of whether the data support conclusions "beyond any reasonable bias," and this evidence generally requires gathering well-conceived, well-conducted clinical trials; for most technology, controlled, peer-reviewed, randomized trials are the necessary standard of evidence.

3. If the second criterion is met, is it effective?

Does the technology improve the relevant health outcome?

4. Is the effect of the technology at least as great as other interventions for the relevant medical condition?

5. Can the observed benefit be achieved outside the investigational setting?

Technology that meets the above criteria will generally be medically appropriate for use in applicable clinical settings.

The converse, however--ie, when technology fails to meet one or more of these criteria--presents a more complicated situation. Within KP and the INTC, the overall evidence is weighed before clinicians are given a recommendation about the general appropriateness of using the technology. Technology that fails to meet the BC-BS Technology Evaluation Center criteria may be characterized as not medically appropriate for general use by KP members. When technology produces conclusive evidence of no effect (ie, no benefit) or evidence of net harm, the technology is characterized as generally inappropriate for use by KP Health Plan members. This circumstance may also be reflected in health plan coverage as communicated to members and purchasers: ineffective therapies may be excluded from benefit coverage.

However, "insufficient evidence" (eg, failure to meet the second criterion) does not mean that intervention using the technology is never medically appropriate for any KP member; instead, each clinician and member must reevaluate the balance of risks and benefits for the member and reconsider the member's clinical condition in an explicit and shared manner.

Not all interventions will meet the above "standard of evidence." In applying evidence to clinical decision making, David Eddy, MD, PhD³ has characterized a pragmatic clinical approach which can be summarized as follows:

• If evidence of benefit exists for an intervention, support use of the intervention and, for each patient, balance the intervention with other, "competing" interventions of comparable effectiveness;
• If the intervention produces no effect or harms a patient, do not use it for that patient;
• If the evidence supporting use of the intervention is inconclusive, use a conservative approach:
  • For new technology, examine it in a research setting (otherwise, evidence of effectiveness will remain inconclusive);
  • For "old" technology, do not promote its use beyond its known benefits.

In the subsequent part of this discussion, to be published in the next issue of The Permanente Journal, the challenge of integrating technology assessment into the pursuit of an informed and ideally shared clinical decision between clinician and member will be considered.

References
1. Lippman ME. High-dose chemotherapy plus autologous bone marrow transplantation for metastatic breast cancer. N Engl J Med 2000 Apr 13;342(15):1119-20.
2. Stadmauer EA, O'Neill A, Goldstein LJ, et al. Conventional-dose chemotherapy compared with high-dose chemotherapy plus autologous hematopoietic stem-cell transplantation for metastatic breast cancer. N Engl J Med 2000 Apr 13;342(15):1069-76.
3. Expert says medical practice should be confined to areas with solid scientific basis. Gene therapy weekly 1999 Nov 8;16-7.

Back to Health Systems Index | To Next Health Systems article