Computerized Expert Health Assessment With Automated Health Education | to pdf >>

by David M. Paperny, MD, FAAP

Needs and Background
The sequelae of adolescent sexual activity, pregnancy, childbearing, suicide, and substance abuse are very costly social, economic, and health problems, and there is now a real impetus in Kaiser Permanente to address these problems in a cost- and time-effective manner. Many programs have tried to identify high-risk adolescents, to provide health education to prevent behaviors with poor outcomes, and to promote professional intervention, but most have had only limited success.¹ Studies suggest that most adolescents rarely seek care from their usual physician in areas of sexuality, substance abuse, and emotional upset.² Therefore, teenagers may fail to realize they need health education and services.³ The sensitive nature of the issues creates discomfort, avoidance, and confidentiality problems and presents numerous barriers to effective preventive health measures now required by HEDIS and various accreditation agencies. This requires time-consuming clinical encounters, and many providers experience discomfort when engaging these problem areas.^4,5,6 This discomfort is partly due to the nature of the issues, lack of prearranged referrals and resources, and the unpredictable time required, which affects the routine clinic schedule. A comprehensive health history requires a complete behavioral history.^3,7 A face-to-face interview may yield biased responses and may involve interpersonal barriers such as guilt, mistrust, embarrassment, and confidentiality, which can prevent the clinician from delivering important health risk reduction messages. The use of a pre-interview written questionnaire obviates some barriers and expedites the face-to-face clinical interview by obtaining a more accurate behavioral health history.^8,9

Description of Clinical Tool
The individualized attention of a personal interview and counseling can be simulated through the use of an expert, interactive multimedia computer; it controls feedback so that branching and decision-making depend on the patient's responses. It also eliminates other interpersonal barriers such as avoidance, denial, discomfort, and confidentiality issues. Because most youths are familiar and comfortable with computers, the computer's ability to respond with selective, personalized feedback creates intense attraction.^10,11 Computers have been used very effectively to take medical and behavioral histories.^12,13,14,15 Patients have indicated that they prefer interactive computer programs to human interviewing or to human advice on sensitive topics.^16,17

Adolescents readily reveal sensitive information to a computer.¹⁸ My previously reported study¹⁹ compared 265 anonymous computer users and a matched group of 294 users who were pre-directed to share a printout of sensitive questions with a clinician. Both groups showed comparable sensitivity that was greater than that of a matched written questionnaire group of 240, suggesting the superiority of the computer over a written questionnaire for detecting sensitive issues. I found that the computer is perceived by teenagers as anonymous and nonjudgmental, and adolescents are more likely to share personal information with a computer if they know they will get immediate, individualized feedback from the computer.

The "Youth Health Provider" multimedia program was developed and evaluated as an expert tool to assist in solving limitations in biosocial screening and adolescent evaluation and to provide multiple levels of interactive health education. An interactive multimedia presentation followed by printed information together provide specific health education, directed medical advice, and referrals. These referrals to health resources empower Kaiser Permanente patients and facilitate professional interventions. Additionally, the computer conserves professional time when the clinician is given a problem-list printout from the teenager's assessment.

Design of Interactive Multimedia Computer Program
A practical health screening and educational tool requires an interactive multimedia computer program with internal consistency, built-in clinical logic validations, and reliable educational feedback formulations acceptable to patients. The program is designed to privately take a comprehensive medical and behavioral health history covering all the preventive health issues required by the AMA Guidelines for Adolescent Preventive Services.²⁰ The software was developed and revised after numerous pilot tests and checks for operational validity and reliability. The program accomplishes a health interview and evaluation in the history-taking format a clinician would follow, and it uses complex logic checking to assess health behaviors and provides feedback to the adolescent. The entire program is self-service, requiring no supervision. The specific computerized procedures:

Complete privacy while using the computer is strictly enforced. The youth types his or her first name, and the computer addresses him or her personally by name. For cultural appropriateness and to maximize rapport, all video presentations show peers of the same race, culture, and gender as the patient. The program opens with an introduction by a youth "peer counselor" explaining the purpose of the program. It emphasizes giving honest answers and repeatedly suggests that the youth share printout information with the clinician.

Questions are spoken through headphones and are printed onscreen, and each answer requires pressing only one button or touchscreen. This interactive branching program takes a directed history based on specific screening questions and on previous answers, logically proceeding as would the physician. The interview usually takes about 15 minutes to complete. Some teenagers may be asked only 50 minimum screening questions, whereas others can branch to 350 possible questions based on responses requiring more in-depth exploration. The program internally validates certain responses for consistency and reconfirms crucial branch-point questions, maximizing specificity of interview. The program database will permanently keep an encrypted record of the user's responses, which is retrieved at later interviews by identifier password or Kaiser Permanente medical record number. Thus, subsequent questioning will not ask the forever-positive questions again (i.e.: ever had sex, high blood pressure, etc.), and this way an intelligent follow-up of previously identified issues can be accomplished. The software was designed to operate on externally fed questions from a text database for easy modification and customization.

Educational Feedback
After interview assessment, the computer gives the adolescent different forms of personalized information as a multimedia presentation followed by a printout. The presentation (consisting of up to 60 adolescent health education topics) may deliver specific subsegments of a topic only on need. Identified health problems and needs are prioritized according to risk determined and are presented with highest priority first. In order to easily modify and customize the presentations on short notice or respond to rising health concerns, the presentation driver was designed to operate on external audio and video files which can be amended, deleted, or inserted as needed. To make the program compatible with any Kaiser Permanente clinic patient-flow pattern, the total time spent on the interactive visual portion may be determined by the teenager or clinician, and the patient may return later to continue the presentation where he or she stopped. All interview questions, anticipatory guidance, and health education are presented in a way that is specific for the gender, race, and culture of the user.

Printed medical advice includes specific health observations and recommendations. The printed handouts are written versions of the scripts of the multimedia presentations. To save printer paper and time in busy clinic settings, the patient may be directed by the computer to take preprinted handouts by number from a rack next to the computer. Medical referrals to resources for specific services are made through selectively printed local telephone numbers or through Kaiser Permanente resource numbers as well as national toll-free 800-numbers. There are also specific referrals to call and hear prerecorded telephone health messages such as Kaiser Permanente HealthPhone (1-800-33-ASK-ME) on relevant health topics. Referrals are also made to appropriate health education computer games and to computer-assisted instruction programs. For example, the "Babygame" addresses parenting desires and needs, and the "Romance" game covers sexual survival skills, information on abstinence, responsible sexual decision-
making, and contraception.²¹

The computer completes the interview, prints feedback, dispenses specific handouts, then administers relevant audio-visual selections from its library of high-impact health education multimedia presentations. Assessment data for the clinician are provided as a printed problem list or can be uploaded to an electronic medical record. The patient may offer this tear-off data voluntarily, which facilitates accuracy of the history and expedites the clinical evaluation.

Results
Subjects
I compared 3,327 adolescents at the Kaiser Permanente Honolulu clinic with 288 adolescents in detention, in runaway shelters, and in a youth correction facility. The age range was 13 to 19 years (mean age of 15.5 years), and half were female. Anonymous response data were saved on disk. There were only about 3% refusals due either to time constraints or to "computer shyness."

Methods
Four different evaluation approaches were used to study the following topics: a) educational evaluation to compare computerized multimedia vs. printed material, b) user responses to the computerized interview, c) impact of the program and printout on individual adolescents, and d) risk profiles compared for the two adolescent populations: clinic and detention. This information is used to determine how the computer can provide more comprehensive risk profiles than conventional medical evaluations, to externally validate computer-collected data with other surveys, and to look at high-risk health needs of both groups.

Findings
Computerized multimedia vs. printed material
I compared two different educational media by measuring improvements in users' knowledge about smoking and sex. Test instruments based on the content of each presentation script were developed to assess impact of two different automated interventions. For a random sample of 595 anonymous, matched clinic subjects, one third had audio-visual presentations administered by computer; the second group watched no presentations but were given handouts identical to the scripts; and the last (control) group was made up of computer users without handouts or presentations. Measurements of six knowledge items about smoking and sex were separately made for the control group and two experimental groups. Media users increased their knowledge of oral contraceptives, HIV testing, Depo-Provera, cigarette costs, smoker health care costs, and nonsmoker longevity. The 215 computer/presentation users had 57% more knowledge improvements than the 194 computer/handout users; both experimental groups showed significantly greater knowledge gains (p<0.05) than the control group of 186.

Reactions to computer interview and program feedback
The interview asked for a self-report of their opinion about their computer interaction. The computer asked the interview assessment questions at the end of each health screening interview, before any feedback. Therefore, these responses reflect only the youths being questioned by the computer, rather than any reaction to the presentation or printout. The computer asked: "How honest and accurate have you been with me on these questions?" 84.9% of the teenagers responded that they were totally honest and accurate, 8.7% responded that they were not completely honest, 5.0% said that they couldn't understand some of it, and 1.4% indicated that they were "pretty inaccurate." Only 0.9% more teenagers in detention said they could not understand some of the program, but none said they were "pretty inaccurate." When all teenagers were asked how they would prefer to be interviewed, the computer was preferred by 88.6%, 5.8% preferred a face-to-face personal interview, and 5.6% preferred a questionnaire interview. When asked if it is easier to talk honestly about these kinds of questions with a person or a computer, 84.3% picked the computer.

Impact of the program and printout
A separate subgroup of 200 subjects who had previously used the computer and had received feedback was given surveys by nurses and asked to complete them anonymously; 97.5% responded that they had told the computer their "real and true" information, and 96.2% felt that the computer "asked good questions." There were 95% who reported that they did read the printout, 94% said that feedback advice made good sense to them, and 97% felt the content applied to them. Only 44% said they spontaneously shared the printout with a doctor, nurse, or adult. When asked if they would like to use the computer again sometime, 87% responded affirmatively.

Computer-generated risk profiles developed from user responses
I evaluated how the computer can capture sensitive interview data not normally shared with health providers and how the interaction between provider and patient might improve because of disclosed information which would otherwise be unknown to the provider. Table 1 is a summary.

Twenty-two percent of all teenagers used marijuana at least monthly; 15% of males and 10% of females used it weekly or more frequently. No alcohol use was found in 56% of teenagers; however, 28% admitted to drinking up to twice a month, and 17% drank at least every weekend.

The computer interview found that 14% of all teenagers had previously attempted suicide (9.8% of males, 17.8% of females). There were positive statistical associations (all p < 0.001) between drug use and other personal issues: school problems; arguments, fights, or misunderstandings with friends, parents, or others; worries, pressure, or stress; problems at home with parents or with other family members; previous suicide attempts; and sexual abuse.

Forty-three percent of teens were sexually experienced. The age of initiation of sexual intercourse is shown in Table 2. Sexually experienced teens were asked frequency of any kind of contraceptive use: 31% always, 23% sometimes, and 46% never. I found 34% of sexually experienced males and 29% of such females felt that birth control pills were unsafe. There were 11.6% of sexually experienced girls who wondered if they had something wrong with them so they could not get pregnant, and 41% of these girls were having intercourse more than once every two weeks. As many as 4.7% of males and 10.5% of females said they had a recent genital discharge or dysuria. Only 24% of males wanted information or pamphlets on birth control, whereas 40% of females wanted this information.

For female teenagers, 16.1% were determined by the computer to be at risk for pregnancy at the time of interview. They were then assessed for other behaviors: Alcohol use on a regular basis was associated with twice the chance of pregnancy (p < 0.001). A girl using any marijuana had three times more chance of pregnancy than nonusers (p < 0.001). Recreational drug use was associated with more than twice the chance of pregnancy (p < 0.001). Of sexually experienced females, 39% were possibly pregnant. Only 57% of sexually experienced females had ever had a pelvic exam. Reported pregnancy (or its possibility) was combined with other responses such as drug and alcohol use and other high-risk behaviors to determine some of the educational feedback appropriate for a given user.

The computer interview found that 14.5% of girls and 4.8% of boys had been sexually abused. The mean age at first occurrence was 10 years old. Sexually abused teenagers (n=407) had significantly more alcohol and marijuana abuse (26% vs. 13%; p<.001), and other substance abuse (78% vs. 10%; p<.001). They were four times more likely to have attempted suicide (41%) than those who were not abused (10%). In one clinic subgroup, the computer recorded sexual abuse for 47 males and 170 females, and 53% of these males and 63% of these females shared their printout with a clinician. All these 132 teenagers whose abuse surfaced as a result of sharing the printout with a clinician had a positive outcome. Counseling was provided, and 1 out of 8 required active intervention such as reporting.

Detainees
The 288 teens in detention and runaway shelters were compared with those in clinic. These highest-risk teenagers (often with low reading ability) were quite capable of engaging the interview program reliably and with valid, consistent results, yet it took them 50% longer to complete when the computer did not read the questions aloud. There was no difference in reported honesty, and detainees preferred a computer over a personal interview just as nondetainees did but were even less willing to fill out written questionnaires (3.1%) than nondetainees (5.9%). Comparing the responses of detained teenagers to the others, substance abuse was 8 times more common, and recreational drug use was found in 40% of the detainees (vs. 6.6% for nondetainees). Detainees had four times the risk of suicide. Twice as many detainees (82% vs. 43%) were sexually experienced, and twice as many female detainees (35% vs. 16%) were then at risk for being pregnant when compared with nondetainees. The computer also detected that 16% of males and 41% of females were sexually abused. These data are consistent with known statistics for detainees.^22,23

Observations and Kaiser Permanente Clinical Experience
Each installation required dedicated space, integration into the clinical setting, staff training on protocol, and designated staff to replenish handouts and to turn on computers each morning. Smooth implementation occurred when noninterference with patient flow was assured and when preestablished resources were in place for referrals.

A champion at each site was important initially to promote the program, but within six months after implementation, pediatricians and nurse practitioners were routinely referring their teen patients with suspected psychosocial or sexual problems to the computer for help with assessment. With the computer, the Kaiser Permanente Adolescent Consultation Clinic was able to greatly expedite assessments of complex patients who were referred. This made it possible to increase by approximately 25-35% the number of patients that could be comprehensively evaluated.

A useful clinic protocol was initiated where the intake nurse routinely asked every teenager to hand her the response printout so she could give it to the physician before seeing the teenaged patient; this was rarely a disclosure issue, with only about 1 in 40 adolescents declining to share their printout. When the physician referred to printed sensitive responses during interview, the teenagers' usual reaction was acceptance of the need to open a discussion. Only 3 out of 3,327 teenagers asked if their answers were being recorded in the computer, and the explanation of anonymous recording satisfied them.

Discussion
Automated health assessment combined with directed multimedia education can promote optimal decision-making and quality health care to more patients, can expedite accurate clinical assessment, and can provide health education for good health choices. Such technology seems almost critical for Kaiser Permanente as a time-efficient approach with a preventive health emphasis. Most of the cost of implementing the program is incurred at startup, but in the long term, automated health assessment may be less expensive than retraining or hiring staff. The computer can facilitate a realistic understanding of the consequences and outcomes of health behaviors. It provides the opportunity to connect patients to their caregivers and can measurably improve health education.

The global responses to the issues, when compared to known norms, suggest accurate input to the computer. Because nearly all patients read the printout, the necessary information is reaching each teenager (96% said the printout was applicable to them). Only 44% spontaneously shared their printout with a doctor, a nurse, or an adult, but nearly all would do so when asked, which suggests that this automated method provides a solution for addressing sensitive issues and breaks down barriers to delivery of and receptivity to health messages. Patients clearly preferred the computer over a questionnaire or personal interview.

The computer also enabled collection of statistical health data for research or survey purposes. Medical record data links to and from clinical office information systems allow transfer of patient data, making the computer fully interfaceable with an electronic medical record. With this reference database in place, intelligent interval assessment and follow-up of prior evaluations allows patients to monitor their health behaviors and consequences.

Multimedia health education offers visual presentation with audio explanation, making information quickly and easily understandable. Immediate feedback provides personal reinforcement of material and is highly motivating. In the clinical setting, computerized interview appeared to decrease anxiety associated with sensitive issues for both doctor and patient. Using it to address these issues can demonstrate that the physician is concerned about these aspects of the patient's life and can facilitate more open communication. Physicians seemed to have found computer-assisted video a reliable, time-saving health education tool, as the face-to-face interview was comfortably primed with behavioral health information. Estimates of physician interview time which can be saved suggest that the computer is fast, economical, and cost-effective.²⁴ Enthusiasm for use of technology was not limited to patients and practitioners, but administrators saw it as a marketing tool and as a progressive solution to educational needs. Automated interview and health education almost eliminates avoidance, mistrust, and discomfort in sharing sensitive problems. It enhances clinicians' ability to promote quality health care to more patients.

Interactive computer-assisted identification of high-risk behaviors and health needs is thorough, accurate, painless, easy, and saves professional time. Such an evaluation combined with interactive educational multimedia is credible and provides better retention. Computerized health assessment with educational multimedia may be one of the most promising interventions for health promotion and disease management at Kaiser Permanente.

The author wishes to acknowledge support of this program in part by an educational grant from March of Dimes Chapter of the Pacific.

---

References:
1. McAnarney ER. Discontinuity: a dilemma for adolescents. Pediatrics 1987;80:95456.
2. Marks A, Malizio J, Hoch J, Brody R, Fisher M. Assessment of health needs and willingness to utilize health care resources of adolescents in a suburban population. J Pediat 1983;102:456-60.
3. Marks A, Fisher M. Health assessment and screening during adolescence. Pediatrics 1987;80(1 Pt 2):131-58.
4. Elster AB. Confronting the crisis in adolescent health: visions for change. J Adolesc Health 1993;14:5058.
5. Fleming M, ed. Healthy youth 2000: a middecade review. Chicago, IL: American Medical Association, 1996.
6. Gans JE, Alexander B, Chu RC, Elster AB. The cost of comprehensive preventive medical services for adolescents. Arch Pediatr Adolesc Med 1995;149:122634.
7. Marks A. Aspects of biosocial screening and health maintenance in adolescents. Pediatr Clin North Am 1980;27:15361.
8.Cavanaugh RM Jr. Obtaining a personal and confidential history from adolescents: an opportunity for prevention. J Adolesc Health Care 1986;7:118-22.
9. Deisher RW, Paperny DM. Variations in sexual behavior of adolescents. In: Kelley VC, editor. Brenemann's Practice of Pediatrics. New York: Harper & Row 1982:ch 25.
10. Chen M. How effective are microcomputerbased programs for health education: a prospective view. Health Educ 1983;14:889.
11. Bosworth K, Gustafson D, Hawkins R, et al: Adolescents, health education, and computers: the body awareness resource network (BARN). Health Educ Microcomputers 1983 Oct:4860.
12. Space L: The Computer as psychometrician. Behavioral Research Method Instruction 1981;13:595.
13. Greist JH, Gustafson DH, Stauss FF, Rowse GL, Laughren TP, Chiles JA. A computer interview for suicide-risk prediction. Am J Psychiatry 1973;130:1327-32.
14. Grossman JH. Barnet GO, McGuire MT, Swedlow DB. Evaluation of computer-acquired patient histories. JAMA 1971;215:1286-91.
15. Stout R. New approaches to the design of computerized interviewing and testing situations. Behavioral Research Methods Instruction 1981;13:436.
16. Fisher LA, Johnson TS, Porter D, Bleich HL, Slack WV. Collection of a clean voided urine specimen: a comparison among spoken, written, and computerbased instructions. Am J Public Health 1977;67:6404.
17. Slack W. Computerbased interviewing system dealing with nonverbal behavior as well as keyboard responses. Science 1971;171:8487.
18. Millstein SG, Irwin CE Jr. Acceptability of computeracquired sexual histories in adolescent girls. J Pediatr 1983;103:8159.
19. Paperny DM, Aono JY, Lehman RM, Hammar SL, Risser J. Computerassisted detection and intervention in adolescent highrisk health behaviors. J Pediatr 1990;116:45662.
20. Elster AB, Kuznets NJ, editors. AMA guidelines for adolescent preventive services (GAPS): recommendations and rationale. Baltimore: Williams & Wilkins; 1994.
21. Paperny DM, Starn JR. Adolescent pregnancy prevention by health education computer games: computerassisted instruction of knowledge and attitudes. Pediatrics 1989;83:74252.
22. Jones RJ, Gruber KJ, Timbers CD. Incidence and situational factors surrounding sexual assault against delinquent youths. Child Abuse Negl 1981;5:43140.
23. McCormack A, Janus MD, Burgess AW. Runaway youths and sexual victimization: gender differences in an adolescent runaway population. Child Abuse Negl 1986;10:38795.
24. Personal communication: Dr Homer Chin, Vice-President, Kaiser Permanente Northwest Region.