Generally speaking, employee opinions were strongly and positively related to customer (member) opinions. More specifically:

Unfortunately, clear links were not found between either employee or customer opinions and business performance (eg, financial measures). We continue to examine these relationships, and others, in an attempt to further our understanding of these complex issues. We hope to report these results in future issues of The Permanente Journal.

Note: These early findings represent a first step toward understanding the relationships between what employees think and how Kaiser Permanente performs. Future submissions to The Permanente Journal will report additional findings, including specific examination of the relation between employee opinions and the STAR Care Index as well as a companion article that provides the practical implications of this work, written by senior Human Resources executives within Kaiser Permanente.

Background
Project Origin
This project has grown out of more general work conducted by Kaiser Permanente's Employee Survey Resource Network. The ESRN's mission is to provide oversight for the Physician and Employee Survey Program, as well as to provide consultative support for the Divisions as they administer surveys. The project described in this document, chartered by Jim Williams and defined by Fran Sincere (the ESRN sponsor), has a goal of defining the role of employee surveys and customer and other business measures.

The ESRN consists of Kaiser Permanente staff, either from or representing all Divisions, as well as external representation from a survey research consulting firm. The members include Lee Jacobs, Sherilyn Kam, Deborah Kesselring, Deborah Konitsney, Julie Kwan, Debra Lowry, Fran Sincere, Melanie Young, and Scott Brooks.

Purpose of Investigation
Kaiser Permanente recently established the administration and use of employee surveys within Divisions as a part of Business Fundamentals. This positioning underscores the value of listening and learning from the employee population. Along with this value is the commonly held belief, often explicitly stated, that listening and responding to employees will impact the treatment of customers, and bottom-line performance.

Some research within Kaiser Permanente has suggested a link, although not a strong one, between customer satisfaction and enrollment terminations. The current research is designed to bring the employee perspective into consideration, to evaluate the relationships between employee, customer, and business performance measures. This type of examination is called linkage research.

Clearly, Kaiser Permanente is facing new definitions of competitiveness and service delivery. These issues have created a demand within the Human Resources staff and line management of the organization for understanding what linkage research is and what it can offer. The charter for this linkage project is as a pilot to prompt the additional data collection efforts required for a program-wide study. The results presented here are based on data from four Regions.

Other Research
The number of studies addressing linkage research is growing. Since Schneider's original research in a bank setting,¹ authors have increasingly supported the general notion that employee opinions are empirically related to customer, financial, or other operational measures. Wiley² summarized the most common findings in a Linkage Research Model (Figure 1). Employee perceptions of certain leadership practices (customer orientation, quality focus, training, and involvement), along with adequate information/job knowledge, teamwork, satisfaction, and retention, are most often related to customer and business performance measures.

Linkage research is often exploratory. That is, there are general expectations and common findings, yet these studies often include a complicated array of employee opinion and other variables. Explicit hypotheses, based in psychological dynamics and not just past findings, are less common. Studies that have simplified the variables involved in order to form clear hypotheses have generally focused on the broad, summary attitudes of job satisfaction or general morale (eg, Ryan et al.³).

The goals of this study were most generally to explore the relationships of employee opinions, customer opinions, and business performance that are specific to Kaiser Permanente. In general, we expected the results to focus on which employee opinions were most related to customer opinions--and on the "Leadership Practices" in the Linkage Research Model.

Methods
Sample
Four Kaiser Permanente Regions were selected for this study depending on the availability of data on employee attitudes, customer satisfaction, financial performance, and productivity. In the years representing the data collection period (1995 and 1996), these four Regions (Colorado, Hawaii, North Carolina, and Southern California) represented over three million members (approximately 41% of the Kaiser Permanente Program membership) in 12 medical centers and 133 medical offices. For this study, employee opinion ratings from nonpatient care locations (eg, Regional offices, administrative offices) were excluded from the analyses. This was done because there were no (external) customer satisfaction ratings associated with these locations that could be linked with employee opinions.

The initial plan was to measure everything (customer satisfaction, productivity, financial performance, employee attitudes) at the facility (eg, hospital or clinic) level. However, some data were not available at that level. As a result, the unit of analysis varies among the Regions. In three Regions, the unit of analysis is the medical center or medical office (a relatively finer level). In one Region, however, the unit of analysis is an "MSA"a cluster of medical centers and related medical offices (a relatively broader level). Though not directly comparable, we decided to take advantage of all the information available, resulting in 42 data points (36 medical centers/offices + 6 MSAs).

Measures
The data for this study came from a variety of different sources. We used two subjective measures (customer satisfaction and employee opinions) and several objective measures (eg, financial performance). Each is described briefly below.

Customer Satisfaction Survey
A random sample of currently enrolled members (customers) is selected to participate in a telephone survey interview each month. The survey, Satisfaction Tracking and Reporting (STAR), consists of several questions that are asked in every Region of the Program and that assess customer satisfaction with the Health Plan (eg, dues, coverage), care providers (eg, interest and attention paid by the doctor, knowledge and ability of the doctor), auxiliary and ancillary services (eg, x-ray), and system issues (eg, time spent waiting in the examination room, ability to get an appointment when needed). In addition, each Region has the option of adding custom questions designed to collect information on issues of specific interest to that Region (eg, the introduction of a health handbook, use of the Internet). Each Region determines the number of interviews to be conducted for the Region for each quarter. An external vendor conducts the interviews in order to protect respondent anonymity and to standardize the survey administration.

For the time period covered in this study, Region A conducted 2,088 interviews, Region B conducted 1,238 interviews, Region C conducted 3,097 interviews, and Region D conducted 13,545 interviews.

Of the customer survey questions, we chose to look initially at only seven, focusing on the items most appropriate from a rational perspective. In particular, we focused on the items that asked respondents to rate their satisfaction with the following aspects of their Kaiser Permanente experience:

These items relate to the respondents' quality of care and quality of service, which are directly under the control of the organization's employees on a day-to-day basis. We did not initially focus on items that asked respondents to rate their satisfaction with coverage, benefits, premiums, and like items, as these were considered to be larger system issues that are not under the direct control of the employees most likely to have direct contact with our customers. For example, a nurse in a clinic has a more direct effect on a member's perception of the care received than on the member's perception that medical coverage is adequate. All items in the current analyses were rated on a scale from 1 to 10, where 1 represented "extremely dissatisfied" and 10 represented "extremely satisfied."

Employee Opinion Survey
Each of the four Regions in this study conducted an Employee Opinion Survey (EOS) consisting of 52 "core" items. These items, which were asked in all participating Regions, covered the following 12 categories:

Each Region could choose to add optional items or to include Region-specific items on their surveys. However, for this study, only the core items were included in the analyses. EOS items were rated on a variety of five-point scales (eg, "Agree strongly" to Disagree strongly" and "A great deal" to "none"). All of the EOS scales included a middle, or neutral point. The response rates of the Regions in the study were 41%, 63%, 75%, and 90%.

Financial Performance
Financial performance was defined by three measures. The first measure was net fixed assets per year-end member. These assets include building, land, and equipment. The second measure was percentage increase in total expense PMPM (per member per month). The third measure was PMPM cost. These measures were chosen because they are representative and recognizable within the Kaiser Permanente Program and because they are important from a business perspective.

Productivity
Productivity was measured by the number of doctor office visits (DOVs) per 1000 members. Defining productivity in the health care industry is difficult at best because it is fundamentally different than in a manufacturing setting, where you can objectively count the number of widgets (or computers or hair dryers) produced. The DOV per 1000 members measure was chosen on the basis of discussions with organization staff well versed in operational issues. We acknowledge that this measure is not the best measure of productivity. However, more traditional measures of productivity, such as revenue per employee or DOVs per employee are not standard measures within the Program and were not available at the time of this research.

Other Indicators of Organizational Performance
In addition, we included two measures of organizational performance that are not strictly financial or productivity measures. The first measure was number of year-end members, a simple count of the number of members associated with a given patient care location (ie, the number of members who go to a specific location for most of their care). The second measure was the percentage of members at the facility who are tagged to a primary care provider (PCP). When a health plan member has one provider who provides most of the care for that member, the member is considered "tagged" or "assigned" to that provider. This is in contrast to an untagged member, who sees different doctors depending on who is available. Tagging members to a PCP is generally considered desirable because of greater continuity of care and the establishment of a personal relationship with a specific health care provider.

Taken as a group, we viewed these measures as a good starting point for the examination of potential relationships between employee opinions and the organizational success.

Analyses
The initial analyses are based in correlations between the employee, customer, and business performance data. As a general guideline, correlations of about .10 may be considered "small," correlations of .24 may be considered "medium," and correlations of .40 or more may be considered "large." (These standards may differ from standards defined by a clinical perspective.)

Previous research has shown that the size of the unit (eg, bank branch or medical facility) and its geographic location moderate the relation between employee opinions and organizational performance indices.⁴ That is, the relation between employee opinions and organizational performance differs depending on size of the bank branch on the basis of its geographic location, or both.

Based upon these findings and the accepted methods, our primary focus was on the partial correlations controlling for unit size and location (ie, the correlations taking into account size and region of the unit). As of the initial findings, we do not have an exact count for unit size. Instead, we used a surrogate measure--the number of employee survey respondents for that unit. To control for location, we used dummy-coded variables to represent the Regions.

In future analyses, we expect to incorporate other variables into our analyses (eg, type of facility, type of customers) and to determine whether they affect the observed relation.

Employee-Customer Results
The partial correlations between the employee opinion scales and customer opinions are presented in Table 1. The scales are in bold face type. Items within the scales are listed only if they are significantly correlated with at least one customer item.

Observations/Interpretations
Overall
1. Employee opinions correlated strongly and positively with customer opinions. This finding is important because these relationships were documented despite the inherent messiness of comparing different Regions with different survey practices. This finding further suggests that the underlying relationships may be strong.

2. A number of key relations are indicated that suggest a rich opportunity for exploration. The following points focus on only the more general relations.

3. In general, employee perceptions of the leadership practices indicated by the Linkage Research Model (customer emphasis, quality focus, involvement, and training) all showed some relations to customer opinions. This finding adds a degree of reassurance that the findings here are consistent with past research. Of course, Kaiser Permanente-specific issues are highlighted as well.

General Patterns
4. Overall customer satisfaction is related to employee opinions of doctors (respect/support from, quality), organizational flexibility, and training for customer service.

5. Of all the employee survey topics, employee perceptions of doctors (respect/support from, quality) are the most strongly related to customer perceptions. This finding suggests that employees and the members have the same opinions of the doctors.

6. The customer opinion most related to employee workplace perceptions is the customer's ability to see a doctor when needed. Customers' ability to see their provider may be their window into the operational effectiveness of that location. If a unit is generally running smoothly according to the employees, perhaps customers are more able to see their doctor in a timely manner. The specific employee perceptions related to this ability to see a doctor are:

This customer opinion may be generally related to workplace organization/management/communication. We are planning to examine this dynamic in more depth.

7. Service training is also strongly related to customer service perceptions in general, even the perceptions specifically about the physicians. This finding is intriguing: Future investigation may show that a service orientation by nonmedical staff (developed through training) may enhance the image of the physicians in the minds of customers.

8. The commitment item did not correlate with any customer or business performance variable, even at p </= .10.

Employee Opinions--Health Plan Membership Results
When we looked at the relation of employee opinions and health plan membership (ie, number of members affiliated with a specific location), we initially found generally negative relations. Although none of these relations were statistically significant, it was distressing nonetheless that more positive employee opinions were associated with fewer members. (Although one can imagine the larger the membership, the larger the workload!) Also distressing was that there was no relation between these variables, even after we controlled for facility size and geographic location.

So we looked instead at the percentage change in membership from the previous year, and the picture changed dramatically. Controlling for facility size and geographic location, we found that larger increases in membership were more likely to be found in facilities in which employees knew more about the mission/vision of Kaiser Permanente (r = .37, p < .05) than in facilities in which employees knew less about the mission/vision. In addition, larger membership increases were more likely to be found in facilities in which employees agreed more strongly that:

Taken as a whole, these results suggest that management communication, openness, and fairness go hand-in-hand with membership growth.

However, the picture was less clear when we looked at the correlation between membership changes and teamwork, especially with respect to teamwork with physicians. Two employee opinion survey items were on the topic of teamwork with physicians, and both of them were negatively correlated with membership changes. Larger membership increases tended to be found in facilities where employees tended to disagree that they are treated with respect by the physicians in their work and that physicians in their work unit support them in providing quality service to their customers. On the surface, this finding implies that poorer relationships between employees and physicians would be associated with larger membership increases.

This relation seems counterintuitive and is therefore particularly intriguing. Perhaps the larger the membership increase, the higher the physicians' workload, resulting in poorer relationships with coworkers. As a cautionary note, a speculation such as this one needs to be investigated further before being given much weight. We present it here simply to illustrate that the counterintuitive relation may have a simple and useful explanation.

Employee Opinions-Business Performance (Financial, Productivity) Results
Financial Performance
In general, financial performance was not related to employee opinions, with three exceptions. First, better financial performance was related to stronger employee agreement that the organization does a good job of letting them know what is going on (r = .33, p < .05). Second, better financial performance was related to stronger employee agreement that they trust the information they get from organizational leaders (r = .43, p < .01). Both of these correlations suggest that improved communication between the organization and its employees is associated with better financial performance. The final item that was significantly correlated with financial performance was "I personally agree with most of this organization's goals" (r = .33, p < .05).

Taken as a whole, these results suggest that open communication and identification with company goals are related to better financial performance. As a cautionary note, it may also be the case that business success leads to feeling better about leadership credibility, organizational communication, and agreement with organizational goals as opposed to the opposite. There have been cases in other organizations where employees read about their company's great financial performance in the newspaper and consequently reevaluate their own organizational views. This finding needs to be explored.

Productivity
None of the employee survey items correlated significantly with productivity measures. In a way, this finding was not surprising. We mentioned earlier that the notion of productivity in a service industry is difficult to grasp. But we went ahead with a "nothing-ventured, nothing-gained" mindset. It is not difficult to imagine that more patients can be seen if more staff, equipment, and supplies exist.

We believe that additional data may be needed to clarify the nature of the relation (ie, if there really is one) between specific employee opinions and productivity. Patient acuity may be a factor that moderates this relation. To wit, it may be that sicker patients require more time be spent with them, and fewer discrete office visits can be logged. Patient acuity is a factor we have not yet considered statistically, partly because (to our knowledge) the organization does not have a standard way of measuring it. But it remains a potential avenue to explore and is definitely something to keep in mind.

Customer Opinions-Business Performance Results
We did not find any significant relation between customer satisfaction and business performance (financial, productivity, membership). Previous research has shown or posited a link between customer satisfaction and financial success³ (eg, Wiley, 1996). Yet we did not find this to be the case in our study. The lack of findings, although disappointing, is still worthwhile to present for three reasons. First, people are interested in the bottom line and how to improve it. Second, the study illustrated how linkages with the bottom line should not be automatically assumed. Should such relations truly exist, they are likely to be complicated. Third, the study provided information upon which future research can build. For example, such future research may need to be more sensitive to changes in business performance and other measures over time (Rogelberg & Creamer, 1994).

A Note on the Business Performance Results
We are undertaking additional analyses to further understand the linkages between employee and customer opinions and business results. To uncover strong relations right from the start would have been preferable. However, some study limitations may explain this lack of initial findings.

The 42 facilities included in this study were not directly comparable. Some indication exists that this lack of comparability would affect the business results more than the others. Employee experiences are fairly comparable, even across different industries (eg, everyone can address whether their supervisor is fair). Customer issues may be less universal, but courtesy, attentiveness, treatment, and overall satisfaction may be common issues within health care, regardless of the specific type of facility. Costs, however, may be different.

Some of the facilities included here provide only outpatient care, and some provide a combination of inpatient and outpatient care. Some facilities provide mostly primary care (eg, Pediatrics, Internal Medicine, General Practice, Family Practice), and some provide mostly specialty care (eg, Cardiology, Allergy, Gastroenterology). The costs associated with these different facility characteristics may vary greatly. Within a given Region, a facility that provides both inpatient and outpatient care may have a PMPM cost of more than $50 whereas a clinic that provides only outpatient care may have a PMPM cost of less than $1. In addition, the composition of facilities differs across Regions. Some of the Regions in our study had no inpatient facilities, ie, inpatient care was obtained through contract hospitals. Others had one or more inpatient facilities. The impact of these differences may also be seen in areas other than cost.

We also discovered nontrivial differences in how Regions define seemingly standard and objective measures. What we believed were standard operating measures of financial performance were often computed differently, whether due to misunderstanding of the criteria or to lack of experience with computing it at such a fine level of detail. At the time of this research, most of the "standard" financial measures were reported in the Program at the Regional level. But, for this study, we asked for data at the facility (eg, clinic or hospital) level. We were sometimes successful at getting data at that level and sometimes not. In one Region, we received data that looked so different from data from other Regions that, upon questioning, we discovered that the data represented not single facilities but groupings of facilities.

Thus, the most comparable measures across facilities (ie, employee and customer opinions) may be the ones where we found the strongest relationships.

Overall
We are encouraged and excited by these results. Despite some of the limitations described, we found strong relations between employee and customer opinions. We clearly need further investigation to understand some of the more perplexing findings. We also are currently limited by the correlational analyses, which inform us that relationships exist but not what causes what. Even if we don't know what causes what, the employee issues significantly related to the important aspects of customer satisfaction should be pursued. The chance of being able to use employee opinions to drive customer satisfaction even higher is worth a lot of attention. The possibility that directly improving customer opinions will enhance an employee's quality of worklife (eg, pride in the company) is also worth pursuing.

This type of research, even if tentative, gives us better information than we previously had. The implications are many and complex. For example, because employees and members think alike when it comes to opinions about doctors, efforts to improve service should potentially focus on the employee-physician relationships. In addition, because perception of service training is related to a wide variety of customer outcomes (including opinions about the doctors), efforts to improve employee skills and comfort in service interactions may have a marked impact on the entire customer experience.

Before we go too far, we need to hold some conversations to calibrate what information is useful and where we should spend our efforts. It is our hope that this article prompts discussion among organization members at all levels. We look forward to presenting more results, and the aforementioned companion piece, in a future issue of the Journal.

Acknowledgments: We would like to acknowledge the following individuals for their hard work and diligence in obtaining the necessary data for the analyses, and for enduring our many questions with patience and grace: Jim Hart, Tim McGinley, Kate Shigetani, and Karen Stevens. We would also like to acknowledge the following individuals for their support and assistance in helping to make this study a reality: Lee Jacobs, MD, Fran Sincere, MSIR, Jim Williams, Employee Survey Resource Network. Finally, we would like to thank both Bob Jako, PhD, and an anonymous reviewer for comments on a previous version of this paper.

References

Based upon research conducted in the Colorado, Hawaii, North Carolina, and Southern California Regions (data collected before the conversion to Divisions).