Using Implementation and Dissemination Concepts to Spread 21st-Century Well-Child Care at a Health Maintenance Organization | pdf>>
Arne Beck, PhD; David A Bergman, MD; Alanna K Rahm, MS; James W Dearing, PhD; Russell E Glasglow, PhD
Abstract
We describe here the use of a conceptual framework for implementing and disseminating in a Health Maintenance Organization an evidence-based model of well-child care (WCC) that includes developmental and preventive services recommended by the American Academy of Pediatrics. Twenty-first Century WCC is a parent-centered, team-based, primary care model that combines online previsit assessments--completed by parents and caregivers regarding clinic-based weight, growth, and development assessments--with vaccinations and anticipatory guidance. Nurses, nurse practitioners, developmental specialists, and pediatricians all play roles in the WCC model. Patient and clinician interaction, health records, and resources are all facilitated through a Web-based diagnostic, management, tracking, and resource information tool. Implementation and dissemination concepts and their attendant practices and tools can reliably be used to augment strategic decisions about how to best disseminate and implement innovations in health care delivery. Unlike innovations that are embedded only in technical systems, validated models of team-based health care have multiple components that must be made compatible with complex sociotechnical systems. Interpersonal communication, work, coordination, and judgment are key processes that affect implementation quality. Implementation can involve tailoring to a particular site and customizing either the model or the organizational context to accommodate it.
Introduction
Well-child care (WCC) has been a cornerstone of pediatric practice in the US for many decades, accounting for almost 25% of pediatric visits and more than 50% of all visits in the first year of life. Well-child office visits are designed to identify key health concerns and to provide anticipatory guidance early so that concerns can be effectively addressed before school entry. Unfortunately, WCC interventions of proven benefit that address developmental and behavioral issues often are not provided during well-child visits, developmental assessment is infrequently completed1,2 and developmental and behavioral problems that could have been detected and treated with evidence-based practices are too often neither detected nor treated during WCC.3–6 The gap between evidence-based WCC and WCC as delivered persists despite pediatricians’ desires to do a better job at providing these services. Limitations on time, resources, and training contribute to the gap, as do other systems-level barriers.7–11
The increasing application of empiric lessons from implementation and dissemination research to health care delivery holds promise for addressing the failure to bring proven interventions to scale.12-21 The purpose of this article is to describe the early-stage application of a model for achieving high-quality implementation of 21st-century WCC at Kaiser Permanente Colorado (KPCO). After this effort and working in collaboration with interested Kaiser Permanente (KP) pediatric leaders programwide, our subsequent goal is dissemination of 21st-century WCC to pediatric practices within other KP regions. We also seek to disseminate this work to other non-KP health care settings such as safety-net clinics.
We present preliminary results from our work, focusing on findings from social network maps that depict who seeks advice from whom about pediatric-care improvements and from focus groups that identify barriers to and facilitators of implementation and help in the design of the 21st-century WCC model.
Setting
KPCO is a nonprofit group-model Health Maintenance Organization (HMO) serving approximately 485,000 members, including 100,000 children in the Denver, Boulder, and Colorado Springs metropolitan areas. KPCO has 18 outpatient facilities, most of which include pediatric and family practices where WCC is delivered. KPCO also has in place an electronic health record containing an electronic personal health record through which members can communicate with their clinicians by e-mail. Current WCC at KPCO includes visits for children at ages 2 weeks; 2, 4, 6, 12, and 15 months; and yearly thereafter for the child’s first five years.
Planning for Well-Child Care
An initial year-long planning grant enabled us to plan and conduct pilot tests for a phased implementation of WCC. We met with teams of clinicians and parents to develop and to test responses to three interventions:
1. The tailored WCC encounter using a Web-based previsit assessment to determine child and family needs, the Child Health and Development Interactive System (CHADIS), developed by the Center for Promotion of Child Development through Primary Care.
2. The use of different visit types such as brief physician visits and e-visits to tailor the visit to the biopsychosocial needs of the child and to the parents’ and physicians’ preferences.
3. An extended visit for children with special health care needs.
Plan-do-study-act (PDSA) rapid-improvement cycles (developed by the Institute for Healthcare Improvement) were conducted in two large pediatric practices within KPCO to assess the feasibility of the three visit types. Seven clinicians and 70 parents participated in the cycles. After the WCC visit, clinicians and families were surveyed regarding their experience, and the length of the visit and resources required were documented by clinician self-report. Results indicated that 75% of parents felt that the online previsit assessment improved or very much improved the WCC visit, although 20% of parents found the online assessment somewhat or very difficult to use. All of the parents found the e-visit, or the e-visit with brief clinician visit, to be an acceptable or very acceptable alternative to the standard WCC visit. The majority of clinicians and families participating in the extended visit for children with special health care needs were satisfied with this type of visit. Clinicians felt that the extended visit enabled them to better identify problems, enhanced the efficiency of the visit, and increase care coordination. Families felt that the extended visit afforded them more opportunity to communicate with their pediatrician and helped them to bring together the various aspects of their child’s care. All seven clinicians felt that the use of the new model helped to focus the visit and would continue or would definitely continue to use the system. Clinicians reported that the time required to review the previsit summary page ranged from 30 seconds to 2 minutes. More detailed results of the PDSA cycles are described elsewhere.22
An analysis modeling the cost of these visit types suggested that the WCC model is cost neutral compared with the current system of WCC at KPCO (Debra Ritzwoller, PhD, personal communication, 2009 Mar 15).a The cost models were based on the assumption that the combination of previsit Web-based assessment and increased use of more time-efficient visits, such as e-visits and brief physician visits for the larger population of low-risk children, would offset the increased costs of targeting more intensive WCC services for the smaller population of children with special health care needs.
A subsequent grant (Commonwealth Fund grant 20080150) is providing support to conduct phased implementation of WCC that is based on our planning results.
Method
Framework for Implementation of Well-Child Care
After a review of the literature on implementation and dissemination frameworks,12,17,23-26 we chose PRISM (Practical, Robust Implementation and Sustainability Model) as the primary conceptual framework guiding our implementation efforts.26 It is represented in Figure 1. The model incorporates concepts from literature about the diffusion of innovations, the Chronic Care Model, model for improvement, and RE-AIM (reach, effectiveness, adoption, implementation, and maintenance).27-29 PRISM takes into account the characteristics of the intervention, the external environment, the implementation and sustainability infrastructure, the recipients of the intervention, and how these factors influence the adoption, implementation, and maintenance of the intervention. We focused on specific elements of PRISM especially germane to our implementation. This parsimonious approach enabled us to apply a comprehensive and complex conceptual framework for implementation to a real-world clinical setting.
We also augmented components of the PRISM model related to organizational characteristics and organizational perspective with social network mapping and clinician and staff focus-group methods, respectively. These measurement methods provide critical information on the attitudes of clinicians and staff toward the innovation and their pattern of communication--specifically, advice seeking regarding the innovation that impedes or facilitates its adoption.
Selection of the Initial Two Implementation Sites
Selection of the initial implementation sites required a consideration of such organizational characteristics of the recipients as their culture, management support and communication, shared goals and cooperation, leadership, systems and training, data and decision support, staffing and incentives, and expectations of sustainability.26
We obtained sponsorship from the regional department chief for pediatrics and pediatric department chiefs at each site, all of whom expressed interest in the project and helped the project team understand site-specific and planwide barriers to WCC implementation. The initial two sites were also chosen because each had clinicians who participated in the planning process and were interested in implementing the model in their clinics. Moreover, the regional department chief for pediatrics recommended these sites. The sites are of different sizes: One is a large practice at a large medical office consisting of ten pediatricians split into two clinician teams; the other is a smaller practice with three pediatricians, located at a smaller medical office. These individuals identified members of the steering committees described in the next section.
Formation of Site-Specific Steering Committees
(Implementation and Sustainability Infrastructure)
Site-specific steering committees were assembled as a way of planning for implementation and sustainability of WCC. The committees provided a dedicated team, addressed training and support, adapted procedures to their site, and shared best practices regarding WCC. Staff and clinician roles and responsibilities for sustaining WCC were discussed so that once research support for the implementation effort was withdrawn, WCC would continue.
The pediatric chiefs at each site provided the research team with nominees for site-specific steering committees that included broad representation of clinicians and staff (pediatricians, nurses, nurse practitioners, medical assistants or licensed practical nurses). Steering committees also included a parent representative. Committee charters were then developed and approved for each site. Committees continue to meet monthly and are responsible for implementation planning for WCC within their respective practices. Meeting agenda items for the steering committees include the impact of WCC on clinic work flow, roles of clinic staff in implementing WCC, interfacing Web-based CHADIS assessments with the electronic health record, and integrating pediatric chronic care coordinators or the developmental specialist into WCC visits for children with special health care needs.
Using Data to Drive Implementation: Mapping the Advice Networks in Each Practice
Another component of organizational characteristics of recipients that we thought important to add to the PRISM model was social network analysis to identify advice-seeking patterns regarding innovations in pediatric care. Social network analysis provides statistical and graphic output and has a long history in public health research.30 Its unique perspective is a focus on the relational characteristics between and among individuals, rather than on the characteristics of the individuals themselves. Sociometric data can identify opinion leaders who can influence the responsiveness and decisions of other caregivers to adopt an innovation like WCC.31 Intervening with sociometrically identified opinion leaders in health care settings has been shown to dramatically improve and sustain patient outcomes.32
Social network statistics provide numeric properties of networks, groups, and individual units. These measures include
• Betweenness: The number of links within the network that include a given individual, indicating how much an individual has the potential to control information flows in the network.
• Power: A measure of informal power that uses the measures of betweenness and closeness (an indicator of access) to identify individuals who have quick access themselves but stand in the way of access for other individuals within the network. Individuals may use this power for positive or negative purposes.
• Network reach: The number of individuals who can be reached in a certain number of steps without being redundant. Individuals with high reach can facilitate dissemination efforts.
We administered a brief sociometric (“who to whom”) questionnaire to all physicians, nurses, and other caregivers in each of the two practices to identify which individuals function as advice-sources (opinion leaders) for innovations in WCC. The questionnaire asked clinicians and staff to identify individuals from whom they sought advice regarding new ideas in pediatric care, and how often they communicated about any topic with that person.
Results
1. Social Network Maps
Data from sociometric questionnaires were analyzed using the social network analysis program InFlow 3.0 (www.orgnet.com). Figures 2, 3, and 3a display the social network maps for each site. Site A is the smaller clinic and site B is the larger clinic. Fourteen clinicians and staff participated from site A and 30 clinicians and staff from site B. We limited the maps so that they show only the most frequent ties (communicate daily or more, communicate weekly), and we examined network-centrality metrics of betweenness, power, and reach to identify opinion leaders at each clinic.
Site A
The network map for site A was revised to account for two opinion leaders who left the clinic after the surveys were completed; the department chief who moved; and a pediatrician who retired. The revised map contains only one group consisting of 15 individuals. On the basis of the strength of ties leading into individuals, we identified individual 007 as the strongest advice leader as nominated by her colleagues. This individual is also a formal leader within the department, the nurse manager. Analysis of betweenness identified individual 013, a department physician, as the top opinion leader. Individual 012, another physician, was identified as the individual with the next highest amount of control, and individuals 007 and 004 were identified equally as the third most influential opinion leaders. However, individual 012 is somewhat peripheral to the network, serving as the node linking three other members to the rest of the network; advice runs through 012, not around this individual. Therefore, this score reflects the dependence of others on individual 012. Regardless of the statistical rank, 012 was not determined to be an opinion leader, because 026 and 027 (the “individuals” 012 goes to for advice) are not individuals at all but external resources.
Site B
The map for site B revealed three distinct network groups. Group A, the main group, consists of 36 individuals. Group B consists of only three members and is an artifact of how the respondent chose to answer the question, listing external sources, such as the Internet, that are not individuals, for frequency of advice; therefore, group B was excluded from the analysis. Group C consists of six members: three members (027, 028, and 056) are individuals within the network; the other three members are collectives and the Internet. Because the individuals in this group are in fact part of the network but also operate as a distinct group of professionals outside of the clinic (the pediatric chronic care coordinators), group C was included and analyzed as a distinct group within the larger clinic network.
A total of five opinion leaders were identified from site B. Four opinion leaders were identified from group A on the basis of the metrics of betweenness and power: 021 (developmental specialist), 019 and 008 (both physicians), and 025 (medical assistant). One additional opinion leader was identified from group C, 028, a registered nurse serving as a pediatric chronic care coordinator.
2. Clinician and Staff Focus Groups
Focus groups have a long history in qualitative research and provide valuable information from the participant perspective. We conducted focus groups with pediatricians, registered nurses, and licensed practical nurses to gauge implementation readiness, understand potential barriers and facilitators to WCC implementation at the practice sites, and to address perceived attributes of the innovation (WCC), including burden (complexity and cost), usability and adaptability, trialability, and ability to observe results. Focus-group topics included core components of WCC, which clinicians should deliver it, types of WCC visits (brief visits, e-visits, extended visits for children with special health care needs), and the use of CHADIS.
A total of 15 clinicians and staff participated in three separate focus groups. A summary of the findings follows.
An ideal WCC system was articulated in which screening forms would be completed by parents before the visit, WCC visits would not be scheduled at the last minute, necessary resource material would be ready for parents at the visit, a wellness specialist would be on hand for addressing childhood obesity, and a pediatric chronic care coordinator would be available for children with special health care needs. WCC content would be parent-centered, addressing weight, growth, development, vaccinations, chronic disease prevention, and safety and providing anticipatory guidance.
Nurses, nurse practitioners, and the developmental specialist were all seen as able to deliver WCC, including anticipatory guidance and review of screening questionnaires. However, maintaining the relationship between parent and pediatrician was raised as a concern regarding visits with other clinician types.
Different WCC visit types were viewed as feasible, depending on the circumstances. For example, brief physician visits were seen as viable if the clinician did a brief once-over with the child during the visit, with the visit managed by a nurse, nurse practitioner, or developmental specialist. E-visits were thought to be attractive for parents concerned about convenience, but they were not seen as a substitute for WCC visits; rather, e-visits were considered useful for follow-up about various health conditions (asthma, autism, Attention Deficit Hyperactivity Disorder, enuresis), medication management, and weight management. Extended visits for children with special health needs were uniformly embraced by clinicians and nursing staff. The extra time for the visit, combined with the availability of a pediatric chronic care coordinator, was seen as a more suitable visit for this population with complex multiple health conditions. The extended visit would allow for a coordinated “one-stop shopping” approach whereby all of the health issues and parent concerns could be addressed during one visit instead of being fragmented among several visits.
The use of standardized, Web-based e-visit assessment (CHADIS) was viewed very positively by clinicians and staff because it would give clinicians advance notice of parent concerns and allow them to tailor the visit more appropriately and efficiently. However, logistical concerns were raised, including lack of Internet savvy and/or access among some parents, coordinating with the call center where appointments are made, and increased workload for staff who must contact parents before the visit to acquaint them with CHADIS. Alternatives to using a Web-based version of CHADIS were also discussed, including using handheld data-collection devices or kiosks in the waiting room.
Data from the social network maps and focus groups will help us better understand attributes of WCC from which we will tailor the model and create effective workflows, as well as tailor targeted clinician and family messages (social marketing) to promote adoption of WCC. In this way, the implementation process is planned as an effort driven primarily by the clinicians and staff at the two sites rather than an intervention imposed on them by the project team. Of course, logistic adaptations to the implementation site must be balanced with efforts to retain core WCC components to preserve fidelity to this model. The constant tension between external and internal validity is common and one that must be thoughtfully addressed to ensure successful implementation efforts.
Using the RE-AIM Model
Once implementation is under way, we will use measures of spread to assess the impact of our implementation efforts at the two sites. Specifically, the RE-AIM framework will be used to plan for and measure spread of new programs.29 RE-AIM uses validated measures of reach, effectiveness, adoption, implementation, and maintenance to assess the spread of the intervention across different systems of care. For this implementation effort, we operationalize each of the RE-AIM components as follows:
• Reach: Percentage and representativeness to KPCO membership of parents invited who participate in WCC
• Effectiveness: Percentage of WCC visits in which standardized developmental and behavioral screening are conducted, referral and referral completion, and any unanticipated consequences of increased screening and assessment (eg, use of more services or offsets of other services)
• Adoption: Proportion and representativeness of practitioners and practices that participate in WCC
• Implementation: Consistency of delivery of WCC procedures across staff and over time, and the time and costs associated with these activities
• Maintenance: Percentage of practitioners and practices that replicate and continue the WCC program and whether and how they change or adapt the program.
Because measures of impact from the RE-AIM model are primarily summative and have data that have not yet been collected, we will present the data in a future article.
Discussion
Our work was guided by selected components of the PRISM framework for implementation that focuses on organizational characteristics of the recipients, organizational perspective, implementation and sustainability infrastructure, and how these factors are known to influence the adoption, implementation, and maintenance of the intervention.
We specifically reported findings from social network maps and clinician and staff focus groups. Social network analysis of advice seeking identified key personnel at each implementation site on the basis of the network centrality metrics of betweenness, power, and network reach. We will recruit these opinion leaders to talk positively about WCC33 and provide them with communication materials and resources to reinforce the positive characteristics of WCC, address barriers to implementation, and encourage decisions to adopt it in their clinics.34
During our experience with site A, we also learned that the results of social network analysis can be outdated quickly when employee turnover occurs. However, it was of interest to observe how the social network changed as a result of this turnover and subsequent introduction of a new formal leader into the site.
Focus groups and discussions with key opinion leaders helped identify barriers and facilitators to implementation. These more operational data address what PRISM labels as the implementation and sustainability infrastructure and the recipients of the intervention. Focus-group results pointed to the need for WCC that is member-centric, is logistically feasible (eg, having multiple modes of completing previsit assessment; being compatible with current appointment-scheduling procedures, including the challenge of advanced access for WCC appointments that compress the previsit assessment period), is flexible (offering choice of clinician and visit type), offers visits tailored to the needs of children and their parents (eg, longer visits for children with special health care needs), and accounts for clinicians’ time constraints both at the visit and in pre- and postvisit documentation.
Once we have completed implementation at the two initial sites, we will spread the model to three additional pediatric practices by applying some of the same implementation and dissemination concepts, including social network maps, focus groups, discussion with key opinion leaders, and communication of social-marketing messages. In addition, we will invite clinicians and staff from practices that have not yet implemented WCC to observe how the model works at the initial implementation sites.
Ultimately we plan to implement WCC across all pediatric practices at KPCO and then collaborate with interested KP leaders programwide to make the model available to other KP regions for implementation while adapting it to their local practices. We also plan to promote its adoption in similar integrated health care systems. In addition, we are currently collaborating with a large local community health system with safety-net clinics that serve a population primarily of Medicaid recipients and Latinos. We are adapting WCC to this population on the basis of user and stakeholder feedback (eg, bilingual materials, potential alternate forms of previsit assessment than the Internet, and use of a promotora [health educator] to guide parents and children through the WCC visits) so that if the adapted program is successful, it may also be spread to other community health centers and safety-net clinics.
Conclusion
We have described here the application of implementation and dissemination conceptual frameworks from the literature to facilitate the spread of WCC. Although our results are preliminary, we believe the use of these frameworks has thus far significantly aided our efforts in the implementation of WCC at KPCO.
We believe that the use of an implementation and dissemination conceptual framework augmented with focus group and social network mapping methods provides valuable data that have helped us develop a detailed and tailored implementation plan for 21st-century WCC, facilitating the implementation process and improving our chances of success in spreading and sustaining WCC in our pediatric practices. We reiterate that because models for implementation and dissemination are context sensitive and complex, selecting, modifying, and adding elements from these frameworks for our specific application to WCC proved a fruitful approach.
a Health Economist, Institute for Health Research, Kaiser Permanente CO.
Disclosure Statement
This research is funded in part by The Commonwealth Fund, grant #20080150.
Acknowledgments
We would like to acknowledge the exceptional support and collaboration from the Pediatrics Departments at the Westminster, Southwest, and Arapahoe Medical Offices, in CO and especially Bruce Doenecke, MD; Sharla Fellers; and Adrienne Silver, MD. We would also like to thank Susan Pharo, MD, Regional Department Chief for Pediatrics, for her strong support of the WCC project. Finally, we thank Ms Dilsha Happel for her valuable participation as a parent member of the advisory committee; Jennifer Dickman and Rasheed Lawal for their insightful assistance with data collection and analysis; and Sharma Cummings for her excellent administrative support.
Katharine O’Moore-Klopf, ELS, of KOK Edit provided editorial assistance.
References
1. Schuster MA, Duan N, Regalado M, Klein DJ. Anticipatory guidance: what information do parents receive? What information do they want? Arch Pediatr Adolesc Med 2000 Dec;154(12):1191–8.
2. Triggs EG, Perrin EC. Listening carefully. Improving communication about behavior and development. Recognizing parental concerns. Clin Pediatr (Phila) 1989 Apr;28(4):185–92.
3. Costello EJ, Edelbrock C, Costello AJ, Dulcan MK, Burns BJ, Brent D. Psychopathology in pediatric primary care: the new hidden morbidity. Pediatrics 1988 Sep;82(3 Pt 2):415–24.
4. Lavigne JV, Binns HJ, Christoffel KK, et al. Behavioral and emotional problems among preschool children in pediatric primary care: prevalence and pediatricians’ recognition. Pediatric Practice Research Group. Pediatrics 1993 Mar;91(3):649–55.
5. Wertheimer R, Croan T, Moore KA, Hair EC. Attending kindergarten and already behind: a statistical portrait of vulnerable young children [monograph on the Internet]. Washington, DC: Child Trends; 2003. research brief no. 2003-20 [cited 2009 Jun 21]. Available from: www.childtrends.org/Files/AttendingKindergartenRB.pdf.
6. Regalado M, Halfon N. Primary care services promoting optimal child development from birth to age 3 years: review of the literature. Arch Pediatr Adolesc Med 2001 Dec;155(12):1311–22.
7. Hochstein M, Sareen H, Olson L, et al. A comparison of barriers to the provision of developmental assessments and psychosocial screenings during pediatric health supervision. Periodic survey of fellows #46 [Abstract on the Internet]. Presented at the Pediatric Academic Societies annual meeting in Baltimore, MD, 2001, Apr 28–May 1 [cited 2009 Jun 21]. Abstract available from: www.aap.org/research/periodicsurvey/ps46pas2.htm.
8. American Academy of Pediatrics. Periodic Survey of Fellows 21. Elk Grove Village, IL: American Academy of Pediatrics; 1993.
9. American Academy of Pediatrics. Periodic Survey of Fellows 43. Elk Grove Village, IL: American Academy of Pediatrics; 2000.
10. American Academy of Pediatrics. Periodic Survey of Fellows 60. Elk Grove Village, IL: American Academy of Pediatrics; 2004.
11. Chung PJ, Lee TC, Morrison JL, Schuster MA. Preventive care for children in the United States: quality and barriers. Annu Rev Public Health 2006;27:491–515.
12. Kerner J, Rimer B, Emmons K. Introduction to the special section on dissemination: dissemination research and research dissemination: how can we close the gap? Health Psychol 2005 Sep;24(5):443–6.
13. Bero LA, Grilli R, Grimshaw JM, Harvey E, Oxman AD, Thomson MA. Closing the gap between research and practice: an overview of systematic reviews of interventions to promote the implementation of research findings. The Cochrane Effective Practice and Organization of Care Review Group. BMJ 1998 Aug 15;317(7156):465–8.
14. Rubenstein LV, Pugh J. Strategies for promoting organizational and practice change by advancing implementation research. J Gen Intern Med 2006 Feb;21 Suppl 2:S58–S64.
15. Glasgow RE, Emmons KM. How can we increase translation of research into practice? Types of evidence needed. Annu Rev Public Health 2007;28:413–33.
16. Feifer C, Fifield J, Ornstein S, et al. From research to daily clinical practice: what are the challenges in “translation”? Jt Comm J Qual Saf 2004 May;30(5):235–45. Erratum in: Jt Comm J Qual Saf 2004 Aug;30(8):413.
17. Berwick DM. Disseminating innovations in health care. JAMA 2003 Apr 16;289(15):1969–75.
18. Bradley EH, Webster TR, Baker D, et al. Translating research into practice: speeding the adoption of innovative health care programs. Issue Brief (Commonw Fund) 2004 Jul;(724):1–12.
19. Stetler CB, Legro MW, Wallace CM, et al. The role of formative evaluation in implementation research and the QUERI experience. J Gen Intern Med 2006 Feb;21 Suppl 2:S1–8.
20. Hagedorn H, Hogan M, Smith JL, et al. Lessons learned about implementing research evidence into clinical practice. Experiences from VA QUERI. J Gen Intern Med 2006 Feb;21 Suppl 2:S21–4.
21. Lavis JN, Robertson D, Woodside JM, McLeod CB, Abelson J; Knowledge Transfer Study Group. How can research organizations more effectively transfer research knowledge to decision makers? Milbank Q 2003;81(2):221–48,171-2.
22. Bergman DA, Beck A, Rahm AK. The use of Internet-based technology to tailor well-child care encounters. Pediatrics 2009 Jul;124(1):e37-e43.
23. Fixsen DL, Naoom SF, Blase KA, Friedman RM, Wallace F. Implementation research: a synthesis of literature [monograph on the Internet]. Tampa (FL): University of South Florida, Louis de la Parte Florida Mental Health Institute, the National Implementation Research Network (FMHI Publication #231); 2005 [cited 2009 Jun 21]. Available from: http://cfs.fmhi.usf.edu/resources/publications/NIRN_Monograph_Full.pdf.
24. Solberg LI. Improving medical practice: a conceptual framework. Ann Fam Med 2007 May–Jun;5(3):251–6.
25. Sterling S, Weisner C. Translating research findings into practice: example of treatment services for adolescents in managed care. Alcohol Res Health 2006;29(1):11–8.
26. Feldstein AC, Glasgow RE. A practical, robust implementation and sustainability model (PRISM) for integrating research findings into practice. Jt Comm J Qual Patient Saf 2008 Apr;34(4):228–43.
27. Solberg LI. Guideline implementation: what the literature doesn’t tell us. Jt Comm J Qual Improv 2000 Sep;26(9):525–37.
28. Solberg LI, Brekke ML, Fazio CJ, et al. Lessons from experienced guideline implementers: attend to many factors and use multiple strategies. Jt Comm J Qual Improv 2000 Apr;26(4):171–88.
29. Glasgow RE. Evaluation of theory-based interventions: the RE-AIM model. In: Glanz K, Rimer BK, Lewis FM, editors. Health behavior and health education: theory, research, and practice. San Francisco: Jossey-Bass; 2002. p 119–27.
30. Luke DA, Harris JK. Network analysis in public health: history, methods, and applications. Annu Rev Public Health 2007;28:69–93.
31. Valente TW, Davis RL. Accelerating the diffusion of innovations using opinion leaders. Ann Am Acad Pol Soc Sci 1999 Jan 1;566(1):55–67.
32. Althabe F, Buekens P, Bergel E, et al; Guidelines Trial Group. A behavioral intervention to improve obstetrical care. N Engl J Med 2008 May 1;358(18):1929–40.
33. Rogers EM. Diffusion of innovations. 5th ed. New York: Free Press; 2003.
34. Dearing JW, Maibach EW, Buller DB. A convergent diffusion and social marketing approach for disseminating proven approaches to physical activity promotion. Am J Prev Med 2006 Oct;31(4 Suppl):S11–23.
