Original Research

A Clinical Information System Research Landscape
By Dean F Sittig, PhD; Brian L Hazlehurst, PhD; Ted Palen, MD, PhD; John Hsu, MD; Holly Jimison, PhD; Mark C Hornbrook, PhD

Background

Information technology could drive progress in health care in the 21st century.¹ Although people have studied how to use technology to improve health care for over 50 years,² there remains much to learn if we are to take full advantage of this potential.³

A recent Institute of Medicine report, Crossing the Quality Chasm: A New Health System for the 21st Century, identified the development and application of more sophisticated clinical information systems (CIS) as essential for health care.⁴ Vigorous research is needed on all aspects of CIS for health care for full leverage of state-of-the-art technology to deliver the highest-quality, lowest-cost patient care.

A clinical information system is a collection of various information technology applications that provides a centralized repository of information related to patient care across distributed locations. This repository represents the patient's history of illnesses and interactions with providers by encoding knowledge capable of helping clinicians decide about the patient's condition, treatment options, and wellness activities. The repository also encodes the status of decisions, actions underway for those decisions, and relevant information that can help in performing those actions. The database could also hold other information about the patient, including genetic, environmental, and social contexts.

We will define such a CIS as a computer-based system that is dedicated to the collection, storage, manipulation,^a and presentation of all the clinical information important to delivery of patient care.⁵ We must examine the organizational and social issues surrounding these systems to truly understand their potential use, benefit, and impact on health care delivery overall.⁶

To capture key aspects of the field and to establish priorities for developing leading-edge clinical applications, we have defined a comprehensive research landscape. This landscape encompasses a broad range of research questions to help us better understand the design, development, implementation, and evaluation of CIS as well as how such systems affect health care delivery. Without a map of the landscape, CIS research and development tends to focus on short-term projects designed to meet the immediate goals of the individuals or groups involved in a specific project. Although these goals are clearly correct at a micro level, such decisions may be both shortsighted and counter to long-term goals of the CIS in particular and the organization in general. At the request of KP's National Research Council, the authors of this article have developed--and describe here--this CIS research landscape, which includes current CIS research activities and highlights potential new areas for research. (The succeeding article in this series describes specific research questions and potential projects that would fit within the CIS research agenda.)

Aspects of CIS Research

Evaluating Functional Aspects of a CIS
Friedman and Wyatt⁷ identified five main aspects of interest in the study of CIS. They noted that, to be comprehensive, each system would be evaluated on each of the following aspects:

• Need for the system (ie, nature of the problems to be addressed and how frequently these problems occur);
• Development process (ie, the development team and its methodology);
• Intrinsic structure (ie, parts and functions of the system that can be observed or inspected without actually running the system, such as flow charts or mockups of the graphical user interface);
• Functionality (ie, system response time, accuracy, reliability, or ease of use);
• Impact (ie, how the system affects the health care providers, patients, processes, and organizations who use the system).

Results of Using a CIS: Identifying Progress
In the last 50 years, numerous research efforts have been designed to help investigators learn more about CIS and their effect on the health care delivery process. The following sections highlight several of the most important dependent variables that have been examined.⁸ Within each study area, research can span all functional aspects of a CIS. For example, researchers might be evaluating a new CIS feature to detect adverse drug events⁹ and thus improve quality of care in an intensive care unit, but they may illustrate in the process the need for a system to help clinicians order the appropriate antibiotic.¹⁰

Clinical information systems are often touted for their potential ability to improve quality of health care. One way they improve care is by supporting clinicians in the decision-making process.¹¹ The most widely used CIS function for this support is presenting patient-specific information in a legible, organized, and timely manner.¹² Other CIS interventions that have been examined include allowing clinicians to:

• access the medical literature,¹³
• ask clinical¹⁴ or administrative questions of aggregates of patient data,
• receive automatic warnings or suggestions when the patient's data satisfy certain logical rules,¹⁵
• receive critiques when proposing therapies¹⁶ or ordering diagnostic tests,
• access guidelines for standards of care,¹⁷
• analyze tradeoffs and the likelihood of alternative outcomes (decision analysis),¹⁸ and
• receive lists of differential diagnoses.¹⁹

Related to quality of health care, safety of the systems and their underlying software has recently become an important topic.^20,21 Studies²² have examined patient safety, especially as it relates to errors of omission and commission made by clinicians or the entire health care system. In a seminal study, McDonald²³ found that physicians prompted by computer were more likely to respond to various clinical events for common conditions routinely managed or caused by medications. He concluded that these oversights were due more to "man's limitations as a data processor rather than to correctable human deficiencies."^23:abstract

In cost-related evaluations, many investigators have examined whether a CIS can affect health care resource utilization. For example, Tierney et al²⁴ looked at how a physician's direct entering of patient orders affects the charges assessed the patient during an inpatient stay. Evans et al¹⁰ showed that a complex clinical decision support system integrated with a comprehensive electronic patient record could both improve quality of care and reduce its costs. Time efficiency of clinical computers has been investigated from the standpoint of the effect of a CIS on the time required to perform certain clinical tasks.²⁵ Investigators have also examined whether a CIS can improve various patient-focused, time-related measures, including: outpatient clinic waiting time, time to receive appropriate treatment,¹⁵ and hospital length of stay.

Research has also examined various issues surrounding clinicians and providers. Satisfaction of patients with their clinicians and the care they receive as well as satisfaction of clinicians with their work environment is a major concern.²⁶ Adoption of CIS as a routine component of the process of delivering patient care has received considerable attention. The historical patterns of technology implemented in health care have been investigated to try to understand the technologic and sociologic factors that create barriers or facilitate the process.^27,28 By providing direct access to relevant clinical information at the time and place it is needed,²⁹ a CIS can have a positive effect on both patient and provider education.³⁰ Various electronic-information-technology-based communication systems have been examined for their ability to improve continuity of care for patients by improving access and coordinating the activities of clinicians.^31,32

The article³³ most often credited with starting the entire field of computer applications in medicine focused on a statistical method of reasoning about medical conditions. Investigators then began experimenting with different ways to implement algorithmic reasoning techniques that exploit the variety of relationships that exist in different medical domains. These include associations, probabilities, causality, functional relationships, temporal relations, locality, similarity, and routine clinical practice.³⁴ The process of representing, maintaining, querying, and reasoning about time-oriented clinical data is another major theoretical and practical research area in CIS.³⁵

The CIS Research Process: Asking Specific Questions
Given this CIS research background, a CIS clearly is much more than simple installation of a computer system within a health care institution. CIS represents a major change in the way health care is delivered. Although the list of potential research questions related to CIS design, development, and implementation is lengthy, reviewing a small group of examples is useful. The next 13 generic research questions are followed by a specific sample question that could be asked and answered with appropriate CIS functionality by a group of experienced CIS researchers.³⁶

1. Does the system work as designed?

Are the alerts and reminders generated for a specific patient correct and "of use" to the clinician?

2. What is the impact of various system enhancements or modifications?

Does the new patient summary display screen help clinicians quickly understand the patient's past medical conditions and treatment and allow more meaningful discussion of the current reason for the visit?

3. Is the system used as anticipated?

After implementation of a physician order entry (POE) system, the percentage of orders entered by physicians could be examined.

4. Does the system produce the desired results?

Does the new POE system actually reduce occurrence of adverse drug events?

5. Does the system work better than the procedures it replaced?

Does the new clinical laboratory alerting system reduce the time patients spend in a critically abnormal physiologic state compared with the previous telephone notification system?

6. Is the system cost-effective?

Does the increased time clinicians spend entering data during the patient visit lead to efficiency or improvement in quality within the overall health care system and thus justify its continued use of this practice?

7. How well have individuals been trained to use the system?

What percentage of clinicians can successfully perform a series of tasks required to manage a simulated patient encounter?

8. What is the anticipated long-term impact on how departments interact?

When routine drug/drug-interaction checking is moved out of the pharmacy and onto the clinician's desktop machine, what is the impact on the pharmacy department's morale and productivity?

9. What are the long-term effects on the delivery of medical care?

Does provision of regular health maintenance reminders to clinicians at the point of care improve the long-term health outcomes of chronically ill patients?

10. Will the system have an impact on management of the organization?

Does incorporation of real-time reminders about the current drug formulary decrease variability in prescribing behavior and thus reduce costs of managing the organization?

11. To what extent does impact depend on the practice setting?

Can the same data entry and review screens be used by both primary care and specialty care clinicians?

12. Can we establish a performance baseline against which future CIS enhancement can be compared? Can such regularly collected clinical or administrative data be used to measure impact of future CIS enhancement? And are there additional data items that, if recorded, would present a different picture?

13. Does the increasing complexity of modern medicine and the CIS required to implement it help or hinder clinicians and their patients? And how does use of the CIS affect the patient-provider relationship?

The CIS Landscape

Our goal was to map the CIS research landscape so that organizational and science goals could be realized through the funding and conduct of actual CIS research. This landscape would, in theory, encompass all relevant projects. To visualize this landscape, we needed to describe the relevant research by classifying projects along important dimensions of the landscape.

A CIS potentially touches and affects all aspects of the health care delivery system. Therefore, a research agenda should address the key components of any potential system and the way that these components interact in the course of care delivery. We thus began by considering an abstract model of health care delivery that made explicit these key components and the interaction among them.

Key Participants in the Health Care Delivery System

The most important participants in the health care delivery system are the patients and their families who receive the care and the clinicians (physicians, nurses, and members of the allied health professions) who provide the care. In addition, we identified the interaction between these two sets of participants, which we termed the "processes" of delivering the care. These processes describe or control the way care is delivered. We also recognized that the organizations (hospitals, integrated delivery networks, health maintenance organizations, insurance companies, and government agencies) supporting the providers are key participants in the overall system. Considering entire populations of patients as a single participant is also often valuable. Finally, we identified the data, information, and knowledge that the CIS must record, store, and display for all of the other participants.

Key Need for Improvement in Health Care

Once we had identified the key participants involved in the health care delivery system, we began to identify key aspects of the care delivery system that require improvement. We decided to use the six areas targeted for improvement from the Institute of Medicine's recent report on improving quality of the health care delivery system, ie, that it be safe, effective, patient-centered, timely, efficient, and equitable. Table 1 combines these six areas for improvement with the key participants involved in the health care delivery system to create the CIS research landscape. Sample research questions that could be pursued within each of the intersecting boxes are shown. (The succeeding article in this series will explore each of these research areas in detail.)

Good research operates on the basis of carefully building knowledge across time and eventually enters the everyday operations of a health care system. Therefore, we wanted to portray the actual research process--an iterative cycle of knowledge acquisition and application building (Figure 1)--and its relation to operational aspects of health care delivery. In most instances, this relation is slow and iterative. Moreover, good research can take place anywhere in the research cycle, and a specific researcher may choose to enter the CIS research process at any point in the cycle; thus, the focus of particular research questions in a specific research area can change depending on the research stage within the research process. Good research can take the form of basic research--which focuses on how processes work--or applied research--which focuses on using these processes to implement change (Figure 1).

Summary

The scientific fields associated with design, development, implementation, and evaluation of CIS have made tremendous progress to date. Although various aspects of these systems have been successfully incorporated into the routine health care delivery processes, much remains to be discovered, implemented, and tested. The CIS research landscape we have described should help researchers and research funders alike focus their time, effort, and money on important research questions. The answers to these questions should, in turn, significantly improve the overall health care delivery process. In the succeeding article, we will describe how we used this research landscape in conjunction with the known operational, financial, technical, governmental, and social constraints present within KP to develop a specific CIS research agenda.

^a Manipulation implies existence of information management tools to provide clinical reminders and alerts, linkages with knowledge sources for health-care decision support, and analysis of aggregate data.

References

1. President's Information Technology Advisory Committee. Report to the President: information technology research: investing in our future. [Arlington (VA): National Coordination Office for Computing, Information, and Communications]; 1999. Available on the World Wide Web (accessed March 26, 2002): www.ccic.gov/ac/report/.
2. Collen MF. A history of medical informatics in the United States, 1950 to 1990. [Indianapolis (IN)]: American Medical Informatics Association; 1995.
3. President's Information Technology Advisory Committee. Panel on Transforming Health Care. Final report: President's Information Technology Advisory Committee meeting. [Washington (DC)]: President's Information Technology Advisory Committee, Panel on Transforming Health Care]; 2001. Available on the World Wide Web (accessed March 26, 2002): www.hpcc.gov/ac/pitac-7feb01/healthcare.pdf.
   
4. Institute of Medicine, Committee on Quality of Health Care in America. Crossing the quality chasm: a new health system for the 21st century. Washington (DC): National Academy Press; 2001.
5. Institute of Medicine, Committee on Improving the Patient Record. Dick RS, Steen EB, editors. The computer-based patient record: an essential technology for health care. Washington (DC): National Academy Press; 1991. p 12.
6. Kaplan B, Brennan PF, Dowling AF, Friedman CP, Peel V. Toward an informatics research agenda: key people and organizational issues. J Am Med Inform Assoc 2001 May-Jun;8(3):235-41.
7. Friedman CP, Wyatt JC. Evaluation methods in medical informatics. New York: Springer-Verlag; 1997. p 42.
8. Rogers EM. Diffusion of innovations. New York: Free Press; 1995. p 88.
9. Evans RS, Pestotnik SL, Classen DC,
   et al. Development of a computerized adverse drug event monitor. Proc Annu Symp Comput Appl Med Care 1991. p 23-7.
10. Evans RS, Pestotnik SL, Classen DC, et al. A computer-assisted management program for antibiotics and other antiinfective agents. N Engl J Med 1998 Jan 22;338(4):232-8.
11. Hunt DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical decision support systems on physician performance and patient outcomes: a systematic review. JAMA 1998 Oct 21;280(15):1339-46.
12. Clayton PD, Hripcsak G. Decision support in healthcare. Int J Biomed Comput 1995 Apr;39(1):59-66.
13. Hersh WR, Hickam DH. How well do physicians use electronic information retrieval systems? A framework for investigation and systematic review. JAMA 1998 Oct 21;280(15):1347-52.
14. DeLong ER, Nelson CL, Wong JB, et al. Using observational data to estimate prognosis: an example using a coronary artery disease registry. Stat Med 2001 Aug 30;20(16):2505-32.
15. Kuperman GJ, Teich JM, Tanasijevic MJ, et al. Improving response to critical laboratory results with automation: results of a randomized controlled trial. J Am Med Inform Assoc 1999 Nov-Dec;6(6):512-22.
16. Miller PL, Black HR. Medical plan-analysis by computer: critiquing the pharmacologic management of essential hypertension. Comput Biomed Res 1984 Feb;17(1):38-54.
17. Cabana MD, Rand CS, Powe NR, et al. Why don't physicians follow clinical practice guidelines? A framework for improvement. JAMA 1999 Oct 20;282(15):1458-65.
18. Edwards A, Elwyn G. The potential benefits of decision aids in clinical medicine. JAMA 1999 Aug 25;282(8):779-80.
19. Miller RA, Pople HE Jr, Myers JD. Internist-1, an experimental computer-based diagnostic consultant for general internal medicine. N Engl J Med 1982 Aug 19;307(8):468-76.
20. Miller RA, Gardner RM. Recommendations for responsible monitoring and regulation of clinical software systems. American Medical Informatics Association, Computer-based Patient Record Institute, Medical Library Association, Association of Academic Health Science Libraries, American Health Information Management Association, American Nurses Association. J Am Med Inform Assoc 1997 Nov-Dec;4(6):442-57.
21. Rigby M, Forsstrom J, Roberts R, Wyatt J. Verifying quality and safety in health informatics services. BMJ 2001 Sep 8;323(7312):552-6.
22. Kohn LT, Corrigan JM, Donaldson MS, editors. To err is human: building a safer healthcare system. Washington (DC): National Academy Press; 2000.
23. McDonald CJ. Protocol-based computer reminders, the quality of care and the non-perfectability of man. N Engl J Med 1976 Dec 9;295(24):1351-5.
24. Tierney WM, Miller ME, Overhage JM, McDonald CJ. Physician inpatient order writing on microcomputer workstations. Effects on resource utilization. JAMA 1993 Jan 20;269(3):379-83.
25. Overhage JM, Perkins S, Tierney WM, McDonald CJ. Controlled trial of direct physician order entry: effects on physicians' time utilization in ambulatory primary care internal medicine practices. J Am Med Inform Assoc 2001 Jul-Aug;8(4):361-71.
26. Mitchell E, Sullivan F. A descriptive feast but an evaluative famine: systematic review of published articles on primary care computing during 1980-97. BMJ 2001 Feb 3;322(7281):279-82.
27. Wong HJ, Legnini MW, Whitmore HH. The diffusion of decision support systems in health care: are we there yet? J Healthc Manag 2000 Jul-Aug;45(4):240-9; discussion 249-53.
28. Ash J. Organizational factors that influence information technology diffusion in academic health sciences centers. J Am Med Inform Assoc 1997 Mar-Apr;4(2):102-11.
29. Chueh H, Barnett GO. "Just-in-time" clinical information. Acad Med 1997 Jun;72(6):512-7.
30. Balas EA, Austin SM, Mitchell JA, Ewigman BG, Bopp KD, Brown GD. The clinical value of computerized information services. A review of 98 randomized clinical trials. Arch Fam Med 1996 May;5(5):271-8.
31. Balas EA, Jaffrey F, Kuperman GJ, et al. Electronic communication with patients. Evaluation of distance medicine technology. JAMA 1997 Jul 9;278(2):152-9.
32. Baur C. Limiting factors on the transformative powers of e-mail in patient-physician relationships: a critical analysis. Health Commun 2000;12(3);239-59.
33. Ledley RS, Lusted LB. Reasoning foundations of medical diagnosis. Science 1959 July 3;130(3366):9-21.
34. Long WJ. Medical informatics: reasoning methods. Artif Intell Med 2001 Aug;23(1):71-87.
35. Shahar Y, Combi C. Timing is everything. Time-oriented clinical information systems. West J Med 1998 Feb;168(2):105-13.
36. Anderson JG, Aydin CE. Overview: theoretical perspectives and methodologies for the evaluation of health care information systems. In: Anderson JG, Aydin CE, Jay SJ, editors. Evaluating health care information systems: methods and applications. Thousand Oaks (CA): Sage Publications; 1994. p 5-29.