Clinical Information System Research Landscape
F Sittig, PhD; Brian L Hazlehurst, PhD; Ted Palen, MD, PhD; John Hsu,
MD; Holly Jimison, PhD; Mark C Hornbrook, PhD
information systems (CIS) could drive progress in health care in
the 21st century. We must examine the organizational and social
issues surrounding these systems to truly understand their potential
use, benefit, and impact on health care delivery overall. After
extensive review of the literature on CIS research, we produced
a "CIS research landscape." This landscape enabled us
to examine and potentially improve delivery of health care services
from the perspective of its major constituents (ie, patients and
their families, clinicians, processes for delivering care, organizations,
patient populations) by using the information captured in CIS. We
then identified aspects of the care delivery system which need to
be addressed to improve the quality of care delivered: the care
must be safe, effective, patient-centered, timely, efficient, and
equitable. In addition to the static picture of this research landscape,
we needed to portray the research process and how it relates to
operational aspects of health care delivery. The CIS research landscape
that we describe should help researchers and research funders alike
focus their time, effort, and money on important questions. The
answers to these questions should in turn greatly improve the overall
health care delivery process. In a subsequent article, we will describe
how we used this research landscape in conjunction with the known
operational, financial, technical, governmental, and social constraints
of Kaiser Permanente (KP) to develop a specific CIS research agenda.
technology could drive progress in health care in the 21st century.1
Although people have studied how to use technology to improve health care
for over 50 years,2 there remains much to learn if we are to
take full advantage of this potential.3
Institute of Medicine report, Crossing the Quality Chasm: A
New Health System for the 21st Century, identified the development
and application of more sophisticated clinical information systems (CIS)
as essential for health care.4 Vigorous research is needed
on all aspects of CIS for health care for full leverage of state-of-the-art
technology to deliver the highest-quality, lowest-cost patient care.
information system is a collection of various information technology applications
that provides a centralized repository of information related to patient
care across distributed locations. This repository represents the patient's
history of illnesses and interactions with providers by encoding knowledge
capable of helping clinicians decide about the patient's condition, treatment
options, and wellness activities. The repository also encodes the status
of decisions, actions underway for those decisions, and relevant information
that can help in performing those actions. The database could also hold
other information about the patient, including genetic, environmental,
and social contexts.
define such a CIS as a computer-based system that is dedicated to the
collection, storage, manipulation,a and presentation of all
the clinical information important to delivery of patient care.5
We must examine the organizational and social issues surrounding these
systems to truly understand their potential use, benefit, and impact on
health care delivery overall.6
key aspects of the field and to establish priorities for developing leading-edge
clinical applications, we have defined a comprehensive research landscape.
This landscape encompasses a broad range of research questions to help
us better understand the design, development, implementation, and evaluation
of CIS as well as how such systems affect health care delivery. Without
a map of the landscape, CIS research and development tends to focus on
short-term projects designed to meet the immediate goals of the individuals
or groups involved in a specific project. Although these goals are clearly
correct at a micro level, such decisions may be both shortsighted and
counter to long-term goals of the CIS in particular and the organization
in general. At the request of KP's National Research Council, the authors
of this article have developed--and describe here--this CIS research landscape,
which includes current CIS research activities and highlights potential
new areas for research. (The succeeding article in this series describes
specific research questions and potential projects that would fit within
the CIS research agenda.)
Aspects of CIS
Functional Aspects of a CIS
Friedman and Wyatt7 identified five main aspects of interest
in the study of CIS. They noted that, to be comprehensive, each system
would be evaluated on each of the following aspects:
for the system (ie, nature of the problems to be addressed and how frequently
these problems occur);
process (ie, the development team and its methodology);
structure (ie, parts and functions of the system that can be observed
or inspected without actually running the system, such as flow charts
or mockups of the graphical user interface);
(ie, system response time, accuracy, reliability, or ease of use);
(ie, how the system affects the health care providers, patients, processes,
and organizations who use the system).
of Using a CIS: Identifying Progress
In the last 50 years, numerous research efforts have been designed
to help investigators learn more about CIS and their effect on the health
care delivery process. The following sections highlight several of the
most important dependent variables that have been examined.8
Within each study area, research can span all functional aspects of a
CIS. For example, researchers might be evaluating a new CIS feature to
detect adverse drug events9 and thus improve quality of care
in an intensive care unit, but they may illustrate in the process the
need for a system to help clinicians order the appropriate antibiotic.10
information systems are often touted for their potential ability to improve
quality of health care. One way they improve care is by supporting clinicians
in the decision-making process.11 The most widely used CIS
function for this support is presenting patient-specific information in
a legible, organized, and timely manner.12 Other CIS interventions
that have been examined include allowing clinicians to:
the medical literature,13
clinical14 or administrative questions of aggregates of patient
automatic warnings or suggestions when the patient's data satisfy certain
critiques when proposing therapies16 or ordering diagnostic
guidelines for standards of care,17
tradeoffs and the likelihood of alternative outcomes (decision analysis),18
lists of differential diagnoses.19
to quality of health care, safety of the systems and their underlying
software has recently become an important topic.20,21 Studies22
have examined patient safety, especially as it relates to errors of omission
and commission made by clinicians or the entire health care system. In
a seminal study, McDonald23 found that physicians prompted
by computer were more likely to respond to various clinical events for
common conditions routinely managed or caused by medications. He concluded
that these oversights were due more to "man's limitations as a data
processor rather than to correctable human deficiencies."23:abstract
evaluations, many investigators have examined whether a CIS can affect
health care resource utilization. For example, Tierney et al24
looked at how a physician's direct entering of patient orders affects
the charges assessed the patient during an inpatient stay. Evans et al10
showed that a complex clinical decision support system integrated with
a comprehensive electronic patient record could both improve quality of
care and reduce its costs. Time efficiency of clinical computers has been
investigated from the standpoint of the effect of a CIS on the time required
to perform certain clinical tasks.25 Investigators have also
examined whether a CIS can improve various patient-focused, time-related
measures, including: outpatient clinic waiting time, time to receive appropriate
treatment,15 and hospital length of stay.
has also examined various issues surrounding clinicians and providers.
Satisfaction of patients with their clinicians and the care they receive
as well as satisfaction of clinicians with their work environment is a
major concern.26 Adoption of CIS as a routine component of
the process of delivering patient care has received considerable attention.
The historical patterns of technology implemented in health care have
been investigated to try to understand the technologic and sociologic
factors that create barriers or facilitate the process.27,28
By providing direct access to relevant clinical information at the time
and place it is needed,29 a CIS can have a positive effect
on both patient and provider education.30 Various electronic-information-technology-based
communication systems have been examined for their ability to improve
continuity of care for patients by improving access and coordinating the
activities of clinicians.31,32
most often credited with starting the entire field of computer applications
in medicine focused on a statistical method of reasoning about medical
conditions. Investigators then began experimenting with different ways
to implement algorithmic reasoning techniques that exploit the variety
of relationships that exist in different medical domains. These include
associations, probabilities, causality, functional relationships, temporal
relations, locality, similarity, and routine clinical practice.34
The process of representing, maintaining, querying, and reasoning about
time-oriented clinical data is another major theoretical and practical
research area in CIS.35
Research Process: Asking Specific Questions
Given this CIS research background, a CIS clearly is much more than
simple installation of a computer system within a health care institution.
CIS represents a major change in the way health care is delivered. Although
the list of potential research questions related to CIS design, development,
and implementation is lengthy, reviewing a small group of examples is
useful. The next 13 generic research questions are followed by a specific
sample question that could be asked and answered with appropriate CIS
functionality by a group of experienced CIS researchers.36
the system work as designed?
alerts and reminders generated for a specific patient correct and "of
use" to the clinician?
is the impact of various system enhancements or modifications?
new patient summary display screen help clinicians quickly understand
the patient's past medical conditions and treatment and allow more meaningful
discussion of the current reason for the visit?
the system used as anticipated?
implementation of a physician order entry (POE) system, the percentage
of orders entered by physicians could be examined.
the system produce the desired results?
new POE system actually reduce occurrence of adverse drug events?
the system work better than the procedures it replaced?
new clinical laboratory alerting system reduce the time patients spend
in a critically abnormal physiologic state compared with the previous
telephone notification system?
the system cost-effective?
increased time clinicians spend entering data during the patient visit
lead to efficiency or improvement in quality within the overall health
care system and thus justify its continued use of this practice?
well have individuals been trained to use the system?
of clinicians can successfully perform a series of tasks required to
manage a simulated patient encounter?
is the anticipated long-term impact on how departments interact?
drug/drug-interaction checking is moved out of the pharmacy and onto
the clinician's desktop machine, what is the impact on the pharmacy
department's morale and productivity?
are the long-term effects on the delivery of medical care?
of regular health maintenance reminders to clinicians at the point of
care improve the long-term health outcomes of chronically ill patients?
the system have an impact on management of the organization?
of real-time reminders about the current drug formulary decrease variability
in prescribing behavior and thus reduce costs of managing the organization?
what extent does impact depend on the practice setting?
same data entry and review screens be used by both primary care and
specialty care clinicians?
we establish a performance baseline against which future CIS enhancement
can be compared? Can such regularly collected clinical or administrative
data be used to measure impact of future CIS enhancement? And are there
additional data items that, if recorded, would present a different picture?
the increasing complexity of modern medicine and the CIS required to
implement it help or hinder clinicians and their patients? And how does
use of the CIS affect the patient-provider relationship?
The CIS Landscape
was to map the CIS research landscape so that organizational and science
goals could be realized through the funding and conduct of actual CIS
research. This landscape would, in theory, encompass all relevant projects.
To visualize this landscape, we needed to describe the relevant research
by classifying projects along important dimensions of the landscape.
A CIS potentially
touches and affects all aspects of the health care delivery system. Therefore,
a research agenda should address the key components of any potential system
and the way that these components interact in the course of care delivery.
We thus began by considering an abstract model of health care delivery
that made explicit these key components and the interaction among them.
in the Health Care Delivery System
important participants in the health care delivery system are the patients
and their families who receive the care and the clinicians (physicians,
nurses, and members of the allied health professions) who provide the
care. In addition, we identified the interaction between these two sets
of participants, which we termed the "processes" of delivering
the care. These processes describe or control the way care is delivered.
We also recognized that the organizations (hospitals, integrated delivery
networks, health maintenance organizations, insurance companies, and government
agencies) supporting the providers are key participants in the overall
system. Considering entire populations of patients as a single participant
is also often valuable. Finally, we identified the data, information,
and knowledge that the CIS must record, store, and display for all of
the other participants.
Key Need for Improvement
in Health Care
had identified the key participants involved in the health care delivery
system, we began to identify key aspects of the care delivery system that
require improvement. We decided to use the six areas targeted for improvement
from the Institute of Medicine's recent report on improving quality
of the health care delivery system, ie, that it be safe, effective, patient-centered,
timely, efficient, and equitable. Table
1 combines these six areas for improvement with the key participants
involved in the health care delivery system to create the CIS research
landscape. Sample research questions that could be pursued within each
of the intersecting boxes are shown. (The succeeding article in this series
will explore each of these research areas in detail.)
operates on the basis of carefully building knowledge across time and
eventually enters the everyday operations of a health care system. Therefore,
we wanted to portray the actual research process--an iterative cycle of
knowledge acquisition and application building (Figure
1)--and its relation to operational aspects of health care delivery.
In most instances, this relation is slow and iterative. Moreover, good
research can take place anywhere in the research cycle, and a specific
researcher may choose to enter the CIS research process at any point in
the cycle; thus, the focus of particular research questions in a specific
research area can change depending on the research stage within the research
process. Good research can take the form of basic research--which focuses
on how processes work--or applied research--which focuses on using these
processes to implement change (Figure
fields associated with design, development, implementation, and evaluation
of CIS have made tremendous progress to date. Although various aspects
of these systems have been successfully incorporated into the routine
health care delivery processes, much remains to be discovered, implemented,
and tested. The CIS research landscape we have described should help researchers
and research funders alike focus their time, effort, and money on important
research questions. The answers to these questions should, in turn, significantly
improve the overall health care delivery process. In the succeeding article,
we will describe how we used this research landscape in conjunction with
the known operational, financial, technical, governmental, and social
constraints present within KP to develop a specific CIS research agenda.
Manipulation implies existence of information management tools to provide
clinical reminders and alerts, linkages with knowledge sources for health-care
decision support, and analysis of aggregate data.
Information Technology Advisory Committee. Report to the President:
information technology research: investing in our future. [Arlington
(VA): National Coordination Office for Computing, Information, and Communications];
1999. Available on the World Wide Web (accessed March 26, 2002): www.ccic.gov/ac/report/.
MF. A history of medical informatics in the United States, 1950 to 1990.
[Indianapolis (IN)]: American Medical Informatics Association; 1995.
Information Technology Advisory Committee. Panel on Transforming Health
Care. Final report: President's Information Technology Advisory Committee
meeting. [Washington (DC)]: President's Information Technology Advisory
Committee, Panel on Transforming Health Care]; 2001. Available on the
World Wide Web (accessed March 26, 2002): www.hpcc.gov/ac/pitac-7feb01/healthcare.pdf.
of Medicine, Committee on Quality of Health Care in America. Crossing
the quality chasm: a new health system for the 21st century. Washington
(DC): National Academy Press; 2001.
of Medicine, Committee on Improving the Patient Record. Dick RS, Steen
EB, editors. The computer-based patient record: an essential technology
for health care. Washington (DC): National Academy Press; 1991. p 12.
B, Brennan PF, Dowling AF, Friedman CP, Peel V. Toward an informatics
research agenda: key people and organizational issues. J Am Med Inform
Assoc 2001 May-Jun;8(3):235-41.
CP, Wyatt JC. Evaluation methods in medical informatics. New York: Springer-Verlag;
1997. p 42.
EM. Diffusion of innovations. New York: Free Press; 1995. p 88.
RS, Pestotnik SL, Classen DC,
et al. Development of a computerized adverse drug event monitor. Proc
Annu Symp Comput Appl Med Care 1991. p 23-7.
RS, Pestotnik SL, Classen DC, et al. A computer-assisted management
program for antibiotics and other antiinfective agents. N Engl J Med
1998 Jan 22;338(4):232-8.
DL, Haynes RB, Hanna SE, Smith K. Effects of computer-based clinical
decision support systems on physician performance and patient outcomes:
a systematic review. JAMA 1998 Oct 21;280(15):1339-46.
PD, Hripcsak G. Decision support in healthcare. Int J Biomed Comput
WR, Hickam DH. How well do physicians use electronic information retrieval
systems? A framework for investigation and systematic review. JAMA 1998
ER, Nelson CL, Wong JB, et al. Using observational data to estimate
prognosis: an example using a coronary artery disease registry. Stat
Med 2001 Aug 30;20(16):2505-32.
GJ, Teich JM, Tanasijevic MJ, et al. Improving response to critical
laboratory results with automation: results of a randomized controlled
trial. J Am Med Inform Assoc 1999 Nov-Dec;6(6):512-22.
PL, Black HR. Medical plan-analysis by computer: critiquing the pharmacologic
management of essential hypertension. Comput Biomed Res 1984 Feb;17(1):38-54.
MD, Rand CS, Powe NR, et al. Why don't physicians follow clinical practice
guidelines? A framework for improvement. JAMA 1999 Oct 20;282(15):1458-65.
A, Elwyn G. The potential benefits of decision aids in clinical medicine.
JAMA 1999 Aug 25;282(8):779-80.
RA, Pople HE Jr, Myers JD. Internist-1, an experimental computer-based
diagnostic consultant for general internal medicine. N Engl J Med 1982
RA, Gardner RM. Recommendations for responsible monitoring and regulation
of clinical software systems. American Medical Informatics Association,
Computer-based Patient Record Institute, Medical Library Association,
Association of Academic Health Science Libraries, American Health Information
Management Association, American Nurses Association. J Am Med Inform
Assoc 1997 Nov-Dec;4(6):442-57.
M, Forsstrom J, Roberts R, Wyatt J. Verifying quality and safety in
health informatics services. BMJ 2001 Sep 8;323(7312):552-6.
LT, Corrigan JM, Donaldson MS, editors. To err is human: building a
safer healthcare system. Washington (DC): National Academy Press; 2000.
CJ. Protocol-based computer reminders, the quality of care and the non-perfectability
of man. N Engl J Med 1976 Dec 9;295(24):1351-5.
WM, Miller ME, Overhage JM, McDonald CJ. Physician inpatient order writing
on microcomputer workstations. Effects on resource utilization. JAMA
1993 Jan 20;269(3):379-83.
JM, Perkins S, Tierney WM, McDonald CJ. Controlled trial of direct physician
order entry: effects on physicians' time utilization in ambulatory primary
care internal medicine practices. J Am Med Inform Assoc 2001 Jul-Aug;8(4):361-71.
E, Sullivan F. A descriptive feast but an evaluative famine: systematic
review of published articles on primary care computing during 1980-97.
BMJ 2001 Feb 3;322(7281):279-82.
HJ, Legnini MW, Whitmore HH. The diffusion of decision support systems
in health care: are we there yet? J Healthc Manag 2000 Jul-Aug;45(4):240-9;
J. Organizational factors that influence information technology diffusion
in academic health sciences centers. J Am Med Inform Assoc 1997 Mar-Apr;4(2):102-11.
H, Barnett GO. "Just-in-time" clinical information. Acad Med
EA, Austin SM, Mitchell JA, Ewigman BG, Bopp KD, Brown GD. The clinical
value of computerized information services. A review of 98 randomized
clinical trials. Arch Fam Med 1996 May;5(5):271-8.
EA, Jaffrey F, Kuperman GJ, et al. Electronic communication with patients.
Evaluation of distance medicine technology. JAMA 1997 Jul 9;278(2):152-9.
C. Limiting factors on the transformative powers of e-mail in patient-physician
relationships: a critical analysis. Health Commun 2000;12(3);239-59.
RS, Lusted LB. Reasoning foundations of medical diagnosis. Science 1959
WJ. Medical informatics: reasoning methods. Artif Intell Med 2001 Aug;23(1):71-87.
Y, Combi C. Timing is everything. Time-oriented clinical information
systems. West J Med 1998 Feb;168(2):105-13.
JG, Aydin CE. Overview: theoretical perspectives and methodologies for
the evaluation of health care information systems. In: Anderson JG,
Aydin CE, Jay SJ, editors. Evaluating health care information systems:
methods and applications. Thousand Oaks (CA): Sage Publications; 1994.
full contents list >>