Assessing Health Department Readiness for Public Health

Loading...

East Tennessee State University

Digital Commons @ East Tennessee State University Electronic Theses and Dissertations

8-2014

Assessing Health Department Readiness for Public Health Accreditation through Quality Improvement Christian L. Williams East Tennessee State University

Follow this and additional works at: http://dc.etsu.edu/etd Part of the Public Health Commons Recommended Citation Williams, Christian L., "Assessing Health Department Readiness for Public Health Accreditation through Quality Improvement" (2014). Electronic Theses and Dissertations. Paper 2417. http://dc.etsu.edu/etd/2417

This Dissertation - Open Access is brought to you for free and open access by Digital Commons @ East Tennessee State University. It has been accepted for inclusion in Electronic Theses and Dissertations by an authorized administrator of Digital Commons @ East Tennessee State University. For more information, please contact [email protected]

Assessing Health Department Readiness for Public Health Accreditation through Quality Improvement

_____________________

A dissertation presented to the faculty of the College of Public Health East Tennessee State University

In partial fulfillment of the requirements for the degree Doctor of Public Health in Community Health

_____________________

by Christian L. Williams August 2014

_____________________

Dr. Robert Pack, Chair Dr. Deborah Slawson Dr. Robin Pendley

Keywords: Quality Improvement, Public Health Accreditation, Health Departments

ABSTRACT

Assessing Health Department Readiness for Public Health Accreditation through Quality Improvement by Christian L. Williams Engaging in quality improvement (QI) activities can help local and state health departments improve current processes, develop more effective new processes, increase leadership capacity, and prepare for public health accreditation. Public health organizations that have implemented QI processes have seen improvements in health outcome indicators, delivery of the 10 essential services, patient satisfaction, and performance management. Quality improvement is the foundation of the Public Health Accreditation Board‟s (PHAB) program and further pushes health departments, at both the local and state level, to adopt QI activities within their organizations. There are numerous potential benefits associated with accreditation in public health, one of the most important being that accreditation sets a benchmark for public health agencies. It also helps create a platform of continuous quality improvement that should increase efficiency, decrease waste, and improve health outcomes.

The purpose of this study was to determine the current status of QI processes in a sample of regional and metro health departments across the state of Tennessee and to assess whether those health departments with a formal QI process demonstrate an increased readiness for public health accreditation compared to those without a formal QI process in place. A survey tool aimed at assessing QI processes and efforts within health departments including the organization‟s: 1) QI culture, 2) QI capacity and competency, 3) QI alignment and spread, and 4) readiness for public health accreditation was used. In addition to the survey tool, respondents were also asked about types of QI processes used within their health department and their associated outcomes.

Initial results revealed that the majority of respondents reported high levels of QI maturity in their respective health department sites. However, further analysis of qualitative data indicated

2

that most sites were engaged in quality assurance (QA) practices rather than true QI processes and activities.

Overall, study results indicate that further training in QI practices is needed in order to enhance performance and align with PHAB standards. The results from this study could be used to help gauge QI processes and accreditation readiness at appropriate intervals following training and education.

3

DEDICATION I would like to thank God first and foremost for always guiding my path. To my mother for encouraging me from a very young age to always pursue my dreams.

4

ACKNOWLEDGEMENTS I would like to thank my dissertation committee (Drs. Robert Pack, Deborah Slawson, Robin Pendley, and Brian Martin) for their invaluable support and guidance throughout this process. They have all played important roles in my academic and professional development. I would also like to acknowledge Mrs. Paula Masters for helping me develop the concept for this study, and providing mentorship and friendship over the last 7 years. Thank you to Dr. Michelle Reece, for refreshing me on the nuances of working in SPSS. Finally, I would like to acknowledge Tennessee Department of Health for approving this study and providing access to its employees.

5

TABLE OF CONTENTS

Page ABSTRACT ..........................................................................................................................

2

LIST OF TABLES ................................................................................................................

9

LIST OF FIGURES ..............................................................................................................

10

Chapter 1. INTRODUCTION .........................................................................................................

12

Quality Improvement in Public Health ......................................................................

12

Public Health Accreditation .......................................................................................

13

Summary of the Issue.................................................................................................

14

Significance................................................................................................................

15

Research Purpose .......................................................................................................

16

Specific Aims .............................................................................................................

16

Hypotheses .................................................................................................................

17

2. LITERATURE REVIEW .............................................................................................

18

Quality Improvement .................................................................................................

18

History of Quality Improvement and Total Quality Management in Other Industries .............................................................................................................

18

Selected Quality Improvement Models ....................................................................

23

Plan Do Check Act ..............................................................................................

23

Six Sigma ............................................................................................................

25

Lean .....................................................................................................................

27

Baldrige Criteria for Performance Excellence ....................................................

28

History of Quality Improvement in Public Health ....................................................

31

Continuous Quality Improvement (CQI) ............................................................

32

6

Chapter

Page

Total Quality Management (TQM) ...........................................................................

34

Quality Improvement in Public Health: Current State ..............................................

35

QI Effectiveness in Public Health Agencies .......................................................

35

QI Activities in State and Local Health Departments ...................................

41

Public Health Accreditation ......................................................................................

43

Public Health Accreditation Board (PHAB) .......................................................

46

Domains and Standards .......................................................................................

46

Accreditation in Public Health Departments: Current Status .............................

47

Tennessee Department of Health ..............................................................................

47

History of QI within TDH ...................................................................................

49

Current Status ................................................................................................

50

3. METHODS ...................................................................................................................

51

Study Design .............................................................................................................

51

Study Sample ............................................................................................................

51

Measures ...................................................................................................................

53

Survey Administration ..............................................................................................

54

Data Analysis ............................................................................................................

55

Aim 1 ..................................................................................................................

55

Aim 2 ..................................................................................................................

56

Aim 3 ..................................................................................................................

57

Hypotheses ..........................................................................................................

57

4. RESULTS .....................................................................................................................

58

Descriptive Statistics .................................................................................................

58

Descriptive Statistics for Survey Sample ............................................................

61

Aim 1: Classify Health Department Sites based on their level of QI Engagement ......................................................................................................... 7

65

Chapter

Page

Aim 2: Current Status of QI Processes in Regional and Metro Health Departments ..................................................................................................

69

Aim 3: Readiness for Public Health Accreditation ...................................................

81

Hypotheses ................................................................................................................

80

Hypothesis 1 ........................................................................................................

82

Hypothesis 2 ........................................................................................................

82

5. DISCUSSION ...............................................................................................................

84

Introduction ...............................................................................................................

84

Summary of Findings ................................................................................................

84

Aim 1 ..................................................................................................................

84

Aim 2 ..................................................................................................................

86

Aim 3 ..................................................................................................................

87

Study Limitations ......................................................................................................

88

Sample Size .........................................................................................................

88

Social Desirability ...............................................................................................

89

Research Engagement .........................................................................................

89

Recommendations and Future Research ...................................................................

89

Contribution to Public Health ...................................................................................

90

Conclusion ................................................................................................................

90

REFERENCES ......................................................................................................................

92

APPENDICES .......................................................................................................................

98

Appendix A: PHAB Domains and Standards ...........................................................

98

Appendix B: Measures ...............................................................................................

101

Appendix C: Cross walk of QI Maturity Tool to PHAB Measures ..........................

106

VITA .....................................................................................................................................

109

8

LIST OF TABLES

Table

Page

1. Deming's 14-Point Program ............................................................................................

21

2. Regional Health Offices and Their Corresponding Counties ..........................................

52

3. QI Maturity Classification by QI Maturity Score Range .................................................

56

4. Population Size by Region or Metro ................................................................................

59

5. Agency Expenditures by Region or Metro ......................................................................

60

6. Agency Size (Full Time Equivalents) by Region or Metro .............................................

61

7. Number of Respondents by Site ......................................................................................

62

8. Number of Respondents by Position................................................................................

63

9. Descriptive Information on QI Items ...............................................................................

64

10. Comparison of QI Classification and Responses to, What Informal QI Processes/Activities Does Your Health Department Use? .........................................

78

11. Comparison of QI Classification and Responses to, Please Describe One Process or Project in Which QI Concepts Were Employed. ....................................

79

12. Accreditation Readiness Score by Site ............................................................................

81

9

LIST OF FIGURES Figure

Page

1. Shewart‟s PDCA Cycle ...................................................................................................

19

2. Crosby‟s Absolute Requirements of Quality ...................................................................

23

3. PDCA Cycle.....................................................................................................................

25

4. Six Sigma DMAIC Process .............................................................................................

27

5. The 5 S‟ in Lean ...............................................................................................................

28

6. Baldrige Criteria for Performance Excellence Framework .............................................

29

7. Baldrige Health Care Criteria for Performance Excellence .............................................

30

8. Quality Improvement Conceptual Framework ................................................................

41

9. Linking Public Health Accreditation and Outcomes .......................................................

45

10. Map of Tennessee Regions and Metropolitan Counties ..................................................

49

11. QI Classification in Tennessee Regional Health Departments ........................................

66

12. QI Classification in Tennessee Metro Health Departments .............................................

67

13. QI Classification by Position or Title ..............................................................................

67

14. Mean Quality Improvement Score by Region .................................................................

68

15. Mean Quality Improvement Score by Metro ...................................................................

69

16. Domain 1 Scores by Site ..................................................................................................

70

17. Domain 2 Scores by Site ..................................................................................................

71

18. Domain 3 Scores by Site ..................................................................................................

71

19. Types of Formal QI by Site .............................................................................................

72

20. Responses to the Question, What Informal QI Processes Does Your Health Department Use? .........................................................................................

73

21. Responses to the Question, What is Your Motivation for Using Formal QI Processes? ...................................................................................................................

74

22. Responses to the Question, What Outcomes Do You Expect to See in Your Health Department from Using Formal QI Processes? ............................................................... 10

74

Figure

Page

23. Responses to, Please Describe One Positive Outcome that You Have Seen in Your Health Department from the Use of QI............................................................................

75

24. Responses to, Please Describe One Negative Outcome that You Have Seen in Your Health Department from the Use of QI............................................................................

76

25. Responses to, Please Describe One Process or Project in Which QI Concepts Were Employed. ..............................................................................................................

77

26. Percent That Plan to Apply for Accreditation, TNCPE, and Currently Have or Have Worked with an AHD ........................................................................................

80

27. Correlation Between Domain 3 and the Assigned Readiness Score................................

82

28. Correlation Between QI Maturity Score and the Assigned Readiness Score ..................

83

11

CHAPTER 1 INTRODUCTION Quality Improvement in Public Health The mission of public health is to promote physical and mental health and prevent disease, injury, and disability. The public health system operates under three core functions and 10 essential services. The three core functions are assessment, policy development, and assurance. The 10 essential services are those activities that all public health agencies should undertake and are 1) monitor health status; 2) diagnose and investigate health problems; 3) inform, educate, and empower; 4) mobilize community partnerships; 5) develop policies; 6) enforce laws and regulations; 7) link people to health services; 8) assure competent workforce; 9) evaluate health services; and 10) engage in research to solve health problems (Centers for Disease Control and Prevention [CDC], 2013). Historically, many industries have recognized the value of implementing and using quality improvement (QI) methods to improve service delivery and process performance (Riley et al., 2010). QI methods have been implemented in healthcare, engineering, service industries, and emergency response organizations, but there have been few attempts to implement QI methods to achieve similar performance outcomes in the public health setting (Madamala, Sellers, Pearsol, Dickey, & Jarris, 2010; Riley et al., 2010). An ongoing commitment to quality has been the foundation for continued success in public health practice (Derose, Schuster, Fielding, & Asch, 2002). The principle behind performance improvement in a public health department is to produce healthier people and communities. Implementing QI processes in public health agencies (including local and state health departments) can lead to an improved public health system 12

(Madamala et al., 2010). Quality improvement is a distinct management process focused on activities that are responsive to community needs and improving population health. According to the formal definition, QI is “a continuous ongoing effort to achieve measurable improvements in the efficiency, effectiveness, performance, accountability, outcomes, and other indicators of quality in services or processes which achieve equity and improve the health of the community” (Corso, Lenaway, Beitsch, Landrum, & Deutsch, 2010, p.20). QI can encompass activities at the program level, department level, or may be organization wide. Quality improvement activities at the program level are often referred to as “small qi,” while quality improvement activities that are organization wide are referred to as “Big QI” (Riley et al., 2010). Lessons from other industries suggest that the implementation of QI processes can improve overall organization performance. Public Health Accreditation In the last several years, focus has turned to establishing a national voluntary accreditation program for public health agencies. Accreditation is a well-established process for improving performance within an organization (Riley, Bender, & Lownik, 2012). Previous state accreditation programs in Michigan, Missouri, and North Carolina have demonstrated that QI and accreditation are sound strategies for strengthening health department performance (Madamala, Sellers, Beitsch, Pearsol, & Jarris, 2012). Public health accreditation is the measurement of health department performance against a set of nationally recognized, practicefocused, and evidence-based standards. The Public Health Accreditation Board (PHAB) is the national organization charged with administering the public health accreditation program. The goal of accreditation is to improve and protect the health of the public by advancing the quality and performance of tribal, state, 13

local, and territorial public health departments (PHAB, 2011; Riley et al., 2012). The accreditation domains and standards set forth by PHAB are intended to document the capacity of public health departments to address and carry out their three core functions and 10 essential health services (PHAB, 2011; Riley et al., 2012). Continuous quality improvement (CQI) is essential for high level performance. As such, PHAB incorporated the concept of CQI into the accreditation process to promote continuous gains in performance (Riley et al., 2012). Madamala et al. (2012) found accreditation may be the pivotal factor in strengthening QI within public health agencies. Summary of the Issue Engaging in quality improvement activities can help local and state health departments improve current processes, develop more effective new processes, increase leadership capacity, and prepare for public health accreditation (Baker, Beitsch, Landrum, & Head, 2007; Gorenflo, 2010). The 2010 Association of State and Territorial Health Officials (ASTHO) Profile of State Public Health found that while 72% of state health agencies (SHAs) plan to seek public health accreditation, the number of SHAs with a formal QI process in place decreased from 27% in 2007 to 22% in 2010 (ASTHO, 2011). Furthermore, a study that analyzed results from the National Public Health Performance Standards Program‟s (NPHPSP) Version 2 instruments and 2005 evaluation results from ASTHO and the National Association of County and City Health Officials (NACCHO) found that QI was one of the poorest performing areas in both SHAs and local health departments (LHDs) (Corso et al., 2010). Results from the ASTHO profile also suggest that most SHAs practice QI on a project-by-project approach, do not engage in organizational wide QI, and fail to involve all staff in the process (Madamala et al., 2010).

14

The 2013 National Profile of Local Health Departments study by the National Association of County and City Health Officials (NACCHO) found that 40% of LHDs are undecided about applying for public health accreditation, and only 23% have formal agencywide QI programs (NACCHO, 2013b). In 2013, 13% of LHDs reported not being engaged in any type of QI activities, a decrease from 16% in 2010 (NACCHO, 2011, 2013b). Significance Healthcare organizations that implement quality improvement processes often experience a range of benefits including improved patient health outcomes, improved efficiency within the organization, reduced waste and costs, and improved communication that could result in additional funding from external resources (US Department of Health and Human Services [HHS], Health Resources and Services Administration [HRSA], 2011). Furthermore, those organizations that engage in QI often see improved efficiency and effectiveness in their core programs; ideally leading to improved community health status (Baker et al., 2007; Riley et al., 2010). The importance of implementing QI within the public health system is further highlighted by the emergence of public health accreditation. The prerequisites required by PHAB to complete an application for accreditation include the completion of a health assessment, a health improvement plan, and a strategic plan within the last 5 years (Madamala et al., 2012). Quality improvement is such an integral part of accreditation that one of the domains addressed by PHAB in the accreditation process deals exclusively with QI and CQI. Engaging in QI processes and accreditation aligns with current Healthy People 2020 objectives that deal with public health infrastructure (PHI). Objective PHI-16 is to increase the proportion of tribal, state, and local public health agencies that have implemented an agency15

wide quality improvement process (HHS, Healthy People 2020, 2011). Objective PHI-17 is to increase the proportion of tribal, state, and local public health agencies that are accredited (HHS, Healthy People 2020, 2011). Both of these objectives are considered under development as currently no baseline measures are available. Quality improvement and accreditation are at the forefront of the public health system today. Looking at other industries as an example, the implementation of QI as a management approach could help push public health agencies into becoming higher performing organizations (Riley et al., 2010). Research Purpose The purpose of this study is to determine the current status of QI processes in a sample of regional and metro health departments across the state of Tennessee and to examine whether those health departments with a formal QI process demonstrate an increased readiness for public health accreditation compared to those without a formal QI process in place. A survey tool aimed at assessing QI processes and efforts within health departments including the organization‟s: 1) QI culture, 2) QI capacity and competency, 3) QI alignment and spread, and 4) readiness for public health accreditation was used. Specific Aims Aim 1: Classify health department sites along a continuum based on their current level of QI engagement. Aim 2: Determine the current status of QI processes in a sample of regional and metro health departments in the state of Tennessee. Aim 3: Identify those health departments within the sample that demonstrate an increased readiness for public health accreditation.

16

Hypotheses Hypothesis 1: Health departments that engage in organization wide formal QI show an increased readiness for public health accreditation. Hypothesis 2: Health departments that demonstrate a higher QI maturity level as evidenced by their QI activities demonstrate an increased capacity for accreditation.

17

CHAPTER 2 LITERATURE REVIEW Quality Improvement According to W. Edwards Deming, one of the preeminent leaders on quality improvement, “If you can‟t describe what you are doing as a process, then you don‟t know what you‟re doing” (as cited in Gorenflo, 2010, pp. 83-84). In many sectors the terms total quality management (TQM), continuous quality improvement (CQI), and quality improvement (QI) are used interchangeably. Regardless of the term used, each describes a structured organizational process for improving quality and efficiency (McLaughlin & Kaluzny, 1999). History of Quality Improvement and Total Quality Management in Other Industries The basis for QI can be linked to the work of several U.S. contributors: Walter Shewart, W. Edwards Deming, Joseph M. Juran, Armand V. Feigenbaum, and Philip B. Crosby (McLaughlin & Kaluzny, 1999). The core of QI is based on Scientific Management, a management theory that emerged during the turn of the century and was focused on the physical efficiency of an individual worker (McLaughlin & Kaluzny, 1999). Walter Shewart is considered the grandfather of quality improvement and was one of the first to be published in the field. When Shewart worked at Hawthorne Plant in Cicero, Illinois he met W. Edwards Deming and Joseph M. Juran, both of whom went on to champion his methods in other fields (Best & Neuhauser, 2006). Shewart, while working for Bell Laboratories, promoted the idea that price was not an indication of value. He is most recognized for the creation of statistical process control and the Plan-Do-Check-Act (PDCA) cycle, both of which are still used today. The Shewart Cycle, or PDCA cycle as it is more commonly known, combines management philosophies with statistical analysis (Best & Neuhauser, 2006). The 18

constant evaluation of management processes and policies, as seen in the PDCA cycle, leads to continuous improvement. An example of the PDCA cycle is depicted in Figure 1 below.

Plan: What changes are desirable? Are new observations needed? If yes, plan a change or test. Decide how the observations will be used.

Do: Search for available data or implement the change/test decided upon.

Act: Standardize the process

Check: Observe the effects of the change or test.

Figure 1. Shewart‟s PDCA Cycle. Adapted from “Defining Quality Improvement: Past, Present, and Future.” By C. P. McLaughlin and A. D. Kaluzny, 1999.

W. Edwards Deming, an American mathematical physicist, worked in Japan during the 1950s to help rebuild its economy after the end of World War II. Although the Japanese implemented his processes in the 1950s, U.S. industries did not start using his business practices until the 1980s (Saunders & Saunders, 1994). Deming also worked as a consultant to many companies including Ford Motor Company, Xerox, and Florida Power and Light. He was a proponent of statistical process control (SPC) based on the work by Shewart. The purpose of SPC is to distinguish processes reflecting normal variance from those with irregular variance and

19

to then monitor the influence of system change on variance patterns (Saunders & Saunders, 1994). According to Deming‟s method of SPC, achieving quality means refining all processes so that all observed variance remains within the parameters of quality set by product specifications. Deming is also credited with developing a theory of management that assists in developing management strategies and techniques based on SPC data, called Theory D (Saunders & Saunders, 1994). According to Theory D quality is largely defined by 1) what customers want and are willing to pay for and 2) building quality into a process is less expensive than attempting to eliminate defects after the fact (Saunders & Saunders, 1994). Deming believed that the majority of quality issues are management controlled instead of worker controlled; therefore, quality management should be a top-down organization-wide commitment (McLaughlin & Kaluzny, 1999). In addition to Theory D, Deming created a 14-point program to help management improve quality. Deming‟s 14-point program is shown in Table 1 below.

20

Table 1. Deming’s 14-Point Program 1. Create constancy of purpose toward improvement of product and service, with the aim to become competitive and to stay in business, and to provide jobs.

8. Drive out fear. Create trust. Create a climate for innovation. 9. Break down barriers between departments.

2. Learn the new philosophy, top management and everyone.

10. Eliminate slogans, exhortations, and targets for the work force asking for zero defects and new levels of productivity.

3. Cease dependence on inspection to achieve quality. Eliminate the need for inspection on a mass basis by building quality into the product in the first place.

11. Eliminate numerical quotas for production. Substitute leadership. Eliminate management by objectives. 12. Remove barriers that rob people of pride of workmanship.

4. End the practice of awarding business on the basis of price tag alone.

13. Encourage education and selfimprovement for everyone.

5. Improve constantly and forever the system of production and service.

14. Take action to accomplish the transformation.

6. Institute training on the job. 7. Institute and teach leadership.

Table created by author based on “W. Edwards Deming, Quality Analysis, and Total Behavior Management.” By R. R. Saunders and J. L. Saunders, 1994.

Joseph Moses Juran is known for emphasizing the management aspect of quality control. He published the Quality Control Handbook in 1951 introducing the concept that quality control should be conducted as a central part of management function (Nofal, Omaim, & Zairi, 2005). Juran felt that when quality control issues were delegated to subordinate staff members and removed from the management hierarchy that it led to negative effects on quality overall. Namely, that no one in the organization felt responsible for quality or quality improvement (Nofal et al., 2005). There are four main principles that guide Juran‟s approach to quality control: 1) it is the responsibility of management, 2) a quality policy should be established, 3) quality goals should be established, and 4) once a goal has been established, management should provide the resources needed to accomplish said goal (Nofal et al., 2005). Those in healthcare often

21

follow Juran‟s “Quality Trilogy,” which describes the steps in the following quality processes: quality planning, quality control, and quality improvement (McLaughlin & Kaluzny, 1999). Building on Deming‟s SPC approach, Armand V. Feigenbaum provided the theoretical constructs for TQM and is responsible for coining the term itself (McLaughlin & Kaluzny, 1999). Feigenbaum was the head of Quality Control Services at General Electric Company during the 1960s and 1970s. It is at GE where he developed and implemented total quality controls and quality management. He published Total Quality Control in 1961, which introduced the concept that all departments are responsible for achieving quality. His contributions to the field can be summarized by the following two principles: 1) quality is the responsibility of everyone, from the unskilled worker to upper level management; and 2) costs must be minimized by a quality improvement program (Nofal et al., 2005). Philip B. Crosby saw quality improvement from a slightly different perspective. Rather than focusing on statistical process controls like his predecessors, he instead focused on the concept of “zero defects” (McLaughlin & Kaluzny, 1999). Crosby believed, “Quality is free. It is not a gift, but it is free. What costs money were the unquality things – all the actions that involve not doing jobs right the first time” (Nofal et al., 2005, p. 8). This concept led to his book Quality is Free, published in 1979. Crosby felt that quality could be described and achieved by adhering to what he described as the four absolute requirements of quality. These absolutes are depicted in Figure 2.

22

Do it right the first time (DRIFT)

Defect prevention is the only acceptable approach to quality control

Having "zero defects" is the only acceptable performance standard

Cost of quality is the only true measure of quality Figure 2. Crosby‟s absolute requirements of quality. Figure created by author based on “Defining Quality Improvement: Past, Present, and Future.” By C. P. McLaughlin and A. D. Kaluzny, 1999.

Selected Quality Improvement Models The following section describes quality improvement models that are commonly used in public health. Each of these models provides a framework to help public health agencies implement their QI processes or interventions. Plan Do Check Act The Plan Do Check Act (PDCA), sometimes referred to as the Plan Do Study Act, is one of the most commonly used QI tools in public health. As noted earlier, PDCA is based on work by Walter Shewart and made popular by W. Edwards Deming. The PDCA cycle has been embraced by public health departments because of both its simplicity and power (Gorenflo & Moran, 2010). It is often used to improve a service or program such as to increase immunization rates, improve front office processes, and decrease wait times (Tews, Sherry, Butler, & Martin, 2008).The cycle itself offers users a systematic, flexible, and straightforward approach to QI.

23

The cycle has its roots in the scientific method, as it involves developing, testing, and analyzing hypotheses (Gorenflo & Moran, 2010). The phases of the cycle have one underlying assumption, that the cause or problem will be addressed by testing one intervention. Because many times public health agencies may want to address more than one problem or test more than one intervention, they need to take into account that they must measure the effect of each intervention on the problem it was originally intended to address (Gorenflo & Moran, 2010). It is suggested that the PDCA cycle should be used in the following instances: 1) as a model for CQI, 2) when developing a new or improved process or service, 3) when planning data collection and analysis to verify and prioritize problems, and 4) when implementing any change in an organization (Tews, 2008). Step one of the cycle is „Plan,‟ the focus of this step is to identify an opportunity for improvement and then develop a plan to accomplish said improvement. During this stage, any issues or problem areas need to be identified, prioritized, and then selected (Tews et al., 2008). It is also during this stage that the current process or problem is described, data are collected to further describe the issue, and all possible causes of the problem should be identified. The next step is to identify ways to improve upon the problem and develop an improvement theory and action plan (Gorenflo & Moran, 2010). Step two of the cycle is „Do,‟ where the previously identified action plan is implemented. Additional steps include collecting data, documenting observations, and addressing any issues in the process. Step three is „Check‟ or „Study.‟ This step involves analyzing the effect of the intervention or test implemented in step two. The primary objective is to determine if the intervention or test was successful. The final step in the cycle, „Act,‟ is where it is decided whether to: 1) standardize the new improvement that was implemented, 2) adapt the test or 24

intervention and retest it, or 3) abandon the project all together. These decisions will be made based on data collected in steps one and two (Gorenflo & Moran, 2010; Tews et al., 2008). Figure 3 depicts the PDCA cycle with a brief description of each step.

Act •Adopt and standardize the change

Plan •Identify a problem or plan a change

Check •Analyze data •Determine if the plan or test was successful

Do •Implement the plan •Document all observations and any issues

Figure 3. PDCA Cycle. Figure created by author based on “The ABCs of PDCA.” by G. Gorenflo and J. W. Moran, 2010 and “Embracing Quality in Local Public Health: Michigan‟s Quality Improvement Guidebook.” by D. Tews, M. Sherry, J. A. Butler, and A. Martin, 2008.

Six Sigma Six Sigma was developed at Motorola in the mid-1980s as a way to improve working systems. Six Sigma measures quality in terms of defect rates and sets a target error rate of no more than 3.4 defects per million opportunities, or 6 standard deviations from the mean, thus deriving the name „Six Sigma‟ (DelliFraine, Langabeer, & Nembhard, 2010). Customer satisfaction is the primary focus of Six Sigma. It operates under the premise that the customer‟s expectations are what define quality (DelliFraine et al., 2010; Thomsett, 2005). In a healthcare setting Six Sigma is often used to address customer satisfaction, improve workforce efficiency, 25

and reduce costs (DelliFraine et al., 2010). Six Sigma uses the scientific method in its approach to improve quality. It applies the scientific method in four steps: 1) observe and define the problem, 2) develop a hypothesis, 3) determine the changes expected, and 4) test the new process to determine if it solved the problem (Thomsett, 2005). The Six Sigma process begins when an organization‟s management identifies top priorities it wishes to impact. The purpose of Six Sigma is to implement QI while also changing the organizational culture (Thomsett, 2005). Six Sigma uses a strategic application tool called DMAIC to help guide the process. DMAIC stands for define, measure, analyze, improve, and control. The „Define‟ phase consists of putting together a team to implement the process, documenting all stakeholders who will be affected by changes made, developing a project statement, and development of a process map. Data collection and evaluation take place during the „Measure‟ phase to collect baseline data and evaluate how a process or service is working. During the „Analyze‟ phase the team will analyze the data collected and determine the root cause of the top issues identified by management. The „Improve‟ phase involves finding solutions to the problem or root cause. Finally, the „Control‟ phase helps to ensure that the implemented changes continue to work through quality control and standardization of the new process (Thomsett, 2005). Figure 4 depicts the DMAIC process as it applies to Six Sigma.

26

• Find solution to root cause • Implement new process or change • Monitor and standardize

• Determine the issue • Form a QI team • Build the project

Improve/ Control

Define

Analyze

Measure

• Perform analysis to determine root cause

• Collect Baseline Data • Evaluate

Figure 4. Six Sigma DMAIC process. Figure created by author based on “Getting Started in Six Sigma.” by M. C. Thomsett, 2005.

Lean Originally started at Toyota, Lean relies on creating standardized processes to provide the best quality services as efficiently as possible. As such, Lean requires a cultural change in order to achieve performance improvement (DelliFraine et al., 2010). At the core of Lean is waste elimination. According to the Lean process, waste comes from the following areas or activities: 1) overproduction, 2) inventory, 3) transportation, 4) motion, 5) over-processing, 6) defects, 7) waiting, and 8) underusing staff (Teich & Faddoul, 2013). Lean should be viewed as a cultural transformation within the organization because it requires new habits, skills, and sometimes a new attitude (Toussaint & Berry, 2013). Lean when used in a health care setting can be defined as, “an organization‟s cultural commitment to applying the scientific method to designing, performing, and continuously improving the work delivered by teams of people, leading to measurably better value for patients 27

and other stakeholders” (Toussaint & Berry, 2013 p. 75). Lean is about performing smaller tasks better in order to create value through the collective effect of small improvements. Figure 5 outlines the steps in Lean and gives a description of each.

Sort •Identify unnecessary items and remove them

Set in Order •Arrange items close to where they are needed •Set limits to the amount stored

Shine •Cleanliness is a form of inspection •Eliminate dirt, dust, and scrap.

Standardize •Make the first three steps strong in habit. Share information and follow the standards to identify problems and eliminate them

Sustain •Work to the standards

Figure 5. The 5 S‟ in Lean. Figure created by author based on “Lean Handout.” by Eastman Chemical Company, 2013.

Baldrige Criteria for Performance Excellence On January 6, 1987, Congress passed the „Malcolm Baldrige National Quality Improvement Act of 1987.‟ The act had two main objectives: 1) to establish an award program, known as the Malcolm Baldrige National Quality Award and 2) to disseminate QI improvement strategies, techniques, and lessons learned (Best & Neuhauser, 2011). The Baldrige award is given annually to organizations that have demonstrated performance excellence. The award criteria are an excellent example of the practical application of QI. According to the criteria, “customer-driven quality is a key strategic business issue which needs to be an integral part of

28

overall business planning” (Dean & Bowen, 1994 p. 403). Figure 6 provides an overview of the Baldrige criteria for performance excellence framework.

Figure 6. Baldrige Criteria for Performance Excellence Framework. Reprinted from “Baldrige, PHAB, or PPHR: What are these and Where do We Start?” by H. Mullins, 2013. Reprinted with permission.

As referenced in Figure 6, there are seven categories for performance excellence that the award addresses. Initially, the awards program was aimed at businesses; however, health care based awards emerged in 2002 and awards for nonprofit organizations began in 2007 (Best & Neuhauser, 2011). The Health Care Criteria focus on health care and processes, customers, finance and markets, workforce, and leadership and governance (National Institute of Standards and Technology [NIST], 2012). The Health Care Criteria are adaptable to meet the needs of the organization using them and let the organization choose the most suitable tool or approach for facilitating QI (NIST, 2012). There are several advantages to implementing the Baldrige model such as1) the systematic approach of the process, 2) the adaptability of the framework and criteria, and 3) the use of organizational goals and objectives as the basis for measurement (Best & Neuhauser, 2011; Dean & Bowen). Figure 7 provides an example of the types of questions 29

addressed in each of these categories as they appear in the Health Care Criteria for Performance Excellence. Leadership

Strategic Planning

Customer Focus

Measurement, Analysis, and Knowledge Management

• Comprises 12% of total score • How do your senior leaders lead? • How do you govern and fulfill your societal responsibilities? • 8.5% of total score • How do you develop your strategy? • How do you deploy your strategy? • 8.5% of total score • How do you engage patients and stakeholders to serve their needs and build relationships? • How do you obtain and use information from your patients and stakeholders? • 9% of total score • How do you measure, analyze, and then improve organizational performance? • How do you manage your information, organizational knowledge, and information technology?

Workforce Focus

• 8.5% of total score • How do you engage your workforce to achieve organizational and personal success? • How do you build an effective and supportive workforce environment?

Operations Focus

• 8.5% of total score • How do you design your work systems? • How do you design, manage, and improve your key organizational work processes?

Results

• 45% of total score • What are your health care results? • What are your patient-and stakeholder-focused performance results? • What are your financial and marketplace performance results? • What are your workforce-focused performance results? • What are your process effectiveness results? • What are your leadership results?

Figure 7. Baldrige Health Care Criteria for Performance Excellence. Figure created by author based on “Did a Cowboy Rodeo Champion Create the Best Theory of Quality Improvement? Malcolm Baldrige and His Award.” by M. Best and D. Neuhauser, 2011.

In the state of Tennessee organizations interested in seeking performance excellence can apply for the Tennessee Center for Performance Excellence (TNCPE) Award. TNCPE is a nonprofit organization that strives to, “drive organizational excellence in Tennessee” (Tennessee Center for Performance Excellence [TNCPE], 2008, p.1). It began in 1993 and has since provided assessments and feedback to over 1,200 organizations in all industry sectors, including

30

healthcare. The TNCPE Award program uses the Baldrige Performance Excellence framework and criteria. TNCPE examiners help guide organizations through the award process and provide ongoing feedback (TNCPE, 2008). The models presented are just a small example of the various QI tools and frameworks available. Public health agencies often use the models presented for implementing their QI efforts due to their validity and straightforward approach (NACCHO, 2011). History of Quality Improvement in Public Health In order to implement quality improvement processes, a public health organization or agency must have a clear definition of its core activities. In 1988 the Institute of Medicine (IOM) created a framework that identifies and defines the three core functions of public health. These functions are assessment, policy development, and assurance (Mays, Hatzell, Kaluzny, & Halverson, 1999; Turnock & Handler, 1997). Public health organizations and agencies are also responsible for the delivery of the 10 essential public health services that align with the three core functions. Together, these two frameworks help form the basis for quality improvement and measurement in public health (Mays et al., 1999). In the late 1980s and early 1990s a variety of performance assessment and quality improvement activities were implemented within the field. Nationally, the 1990 Health Objectives, the predecessor for Healthy People 2000, Healthy People 2010, and currently Healthy People 2020 were developed by the U.S. Public Health Service to measure national health objectives (Mays et al., 1999). The Planned Approach to Community Health (PATCH) health planning tool developed by the CDC in 1985 outlined a protocol for public health agencies to identify and address health issues within their community. Likewise, the Assessment Protocol for Excellence in Public Health (APEX-PH), developed by NACCHO in 1991, was a 31

self-assessment workbook for public health agencies to assess internal capacity and their delivery of the 10 essential services (Mays et al., 1999; Turnock & Handler, 1997). Practice guidelines are another QI tool developed to assist public health agencies in the implementation of evidence based programs and interventions. Beginning in the mid-1990s community health report cards were being used to monitor and improve performance in public health service areas (Mays et al., 1999). Report cards offered a way for organizations to encourage continuous quality improvement, motivate performance improvement, set benchmarks for performance improvement, and create a framework for identifying best practices (Mays et al., 1999). In more recent years movement toward continuous quality improvement and total quality management has helped change the health care environment. CQI and TQM are based on five interrelated principles: 1) a focus on organizational processes as causes of failure instead of individuals, 2) the use of structured problem-solving and analytical approaches, 3) use of interdisciplinary teams, 4) employee empowerment to identify issues and opportunities for improved performance, and 5) a focus on both internal and external customers (Barton, 2010). Continuous Quality Improvement (CQI) In the health care sector interest in continuous quality improvement began to take form in the early 1990s. Uptake of CQI methods in health care was driven by the continuous need to improve medical quality and management. As mentioned earlier, these methods had already been in use in other industries beginning with Shewart‟s work in the 1920s with Western Electric Company (Kritchevsky & Simmons, 1991). Early efforts at implementing CQI practices within the health care sector occurred in the hospital and inpatient setting. In these settings CQI was used to help monitor procedures and to lower medical errors resulting in malpractice or increased mortality rates (Kritchevsky & Simmons, 1991). In truth, physicians had been implementing QI 32

practices for decades; a prime example being the case of Ignaz Semmelweis and puerperal fever. Semmelweis was a Hungarian physician who discovered the incidence of puerperal fever could be drastically reduced by mandating that every staff member wash their hands before assisting with labor and delivery in an obstetrics clinic. However, true implementation of these processes system wide and from a managerial perspective did not take place until the early 1990s (Kritchevsky & Simmons, 1991). Initially the push to implement CQI efforts in public health was to aid in the assessment of health outcomes and monitor improvement in clinical services (Dever, 1997). With the arrival of the National Public Health Performance Standards Program (NPHPSP) initiated in 1998 and community health report cards, further focus on CQI began to take shape (Institute of Medicine [IOM], 2003). The NPHPSP was designed to measure public health practices at both the state and local levels. The mission of NPHPSP is to “improve quality and performance, increase accountability, and increase the science base for public health practice” (IOM, 2003, p. 156). The standards are based on the 10 essential services and allow public health agencies to measure their performance against what is considered to be “optimum” standards (IOM, 2003). Since the development of NPHPSP many public health organizations and agencies have implemented continuous quality improvement processes to help with the delivery of services to the community and patients. In order to implement CQI processes, a paradigm shift must take place. CQI, unlike traditional management processes, is proactive, integrated across the entire agency, and employs a bottom up approach (Dever, 1997). CQI is built on the tenet that agencies and organizations should focus on improving all processes on every level, not just those processes with problem areas or issues. Furthermore, CQI focuses on the overall performance of everyone within an organization, not just those deemed unacceptable (Dever, 1997).

33

Total Quality Management (TQM) Total Quality Management (TQM) has its roots in the ideas and principles set forth by W. Edwards Deming, Dr. Joseph M. Juran, and Armand V. Feigenbaum. TQM is a “participative, systematic approach to planning and implementing a continuous organizational improvement process” (Kaluzny, McLaughlin, & Simpson, 1992, p. 257). TQM is a management approach that evolved from a focus on statistical process control to limit errors or defects and improve organizational performance (Dean & Bowen, 1994). It is characterized by its commitment to customer focus, continuous improvement, and teamwork. Another key characteristic of TQM is the involvement of all parts of the organization in implementing quality plans (Dean & Bowen, 1994). The concept of Total Quality Management was not implemented in U.S. industries until the 1980s. Organizations in the health care sector began taking notice about a decade later and most notably used TQM in clinical health settings to improve medical, administrative, and clinical care processes (Dean & Bowen, 1994). Changes in the public health arena saw statelevel public health agencies beginning to experiment with TQM in an effort to better serve internal and external customers (Berman, Milakovich, & West, 1996). At that time it was felt that public health agencies would benefit from TQM and managerial improvements due to their reactive nature. Public health agencies have historically based their planning around program and budget crises instead of using systematic planning approaches (Berman et al., 1996). The emergence of QI assessment tools and public health standards helped encourage the application of both CQI and TQM principles in public health. Kaluzny et al., (1992) found that integrating TQM into public health functions complemented and enhanced current assessment tools that were being used such as APEX-PH and helped to establish a more strategic direction

34

for the agency overall (Kaluzny et al., 1992). Berman et al. (1996) found that in a survey distributed to State Health Agencies (SHAs) in 1993, only about 29 states were currently using TQM in at least one program area. Furthermore, 44% of these states used TQM in fewer than five health service functions (Berman et al., 1996). TQM provides public health agencies an opportunity to continually improve services. Today, one of the best known TQM frameworks is the Malcolm Baldrige National Quality Award. Quality Improvement in Public Health: Current State Standards based programs such as the NPHPSP and the recent focus on public health accreditation have set the stage for the widespread adoption of QI processes in public health agencies (American Public Health Association [APHA], 2012; Dilley, Bekemeier, & Harris, 2012). The second version of NPHPSP released in 2007 reinforced the need for implementing QI processes by suggesting public health agencies adopt QI techniques such as the PDCA cycle (APHA, 2012). Quality improvement is the foundation of PHAB‟s accreditation program and further pushes health departments, at both the local and state level, to adopt QI activities within their organizations (APHA, 2012; Baker et al., 2007). Limited funding is also responsible for helping to drive QI in public health because QI activities can help improve the efficiency or effectiveness of a program, process, or organization (Dilley et al., 2012). With a continued focus on performance improvement and accreditation, it is expected that public health departments will increasingly implement QI activities. QI Effectiveness in Public Health Agencies Dilley et al. (2012) conducted a systematic review to identify public health systems that implemented QI interventions between 1990 and 2010. The 18 studies reviewed fell into three categories: 1) organization-wide QI interventions, 2) program or service-related QI interventions,

35

and 3) administrative or management practice QI interventions (Dilley et al., 2012). Those organizations that implemented organization-wide QI activities saw improvements in health outcome indicators, delivery of the 10 essential services, and outcomes in performance standards. Those that implemented QI at the program or service level saw improvements in patient wait times, improved patient satisfaction, and improved delivery of clinical services (Dilley et al., 2012). Finally, organizations that implemented QI processes at the administrative level found that they were able to decrease staffing costs, improve training, and improve workforce management (Dilley et al., 2012). There have been several local and state health departments that have implemented QI activities both at department level and organizational wide that have achieved great success. Many public health agencies find that implementing QI activities incrementally is an effective strategy for developing an organization-wide QI culture (APHA, 2012). This was the approach taken by Oklahoma State Department of Health (OSDH) when they introduced QI activities into their childhood obesity program, CATCH Kids Club. The program began with a QI framework that was so successful that the OSDH now uses QI initiatives in other programs across the state (APHA, 2012). OSDH was committed to implementing QI organizational wide and were awarded a grant for the Multi-State Learning Collaborative and included in the PHAB beta test. When Genesee County Health Department (GCHD) in Michigan was inundated with H1N1 related calls, they used the PDCA cycle to create a triage process for handling phone calls. Because they implemented this QI tool, they were able to handle the high volume of calls with no additional resources (APHA, 2012). Faced with extensive funding cuts, the South Carolina Department of Health and Environment Control (SCDHEC) decided to use a QI approach to increase available slots in its STD clinics. By implementing “fast track” appointments for

36

asymptomatic patients, they were able to increase the total number of STD services, reduce wait time, and open 101 appointment slots (APHA, 2012). They used the PDCA cycle in order to use the “fast track” approach, where patients received lab work, minimal education, and a screening questionnaire. Many public health agencies find that once they‟ve implemented QI activities in one area or process, they begin to use QI in other areas promoting a culture of QI (APHA, 2012). Another success story about effectively using QI approaches in public health comes from the Buncombe County Department of Health (BCDH) in North Carolina who used a QI approach on an H1N1 public health preparedness communication project. They used the PDCA cycle along with the model for improvement (MFI) to determine where senior citizens in Buncombe County receive their preparedness information and in what format the message is best received (Harrison et al., 2012). The MFI establishes the aim, measures, and ideas for a QI project and then tests those ideas using the PDCA cycle. Through using these QI activities, staff was able to create and test a message about public health preparedness aimed at senior citizens. The use of this approach was so successful that staff decided to use it in other areas of the health department (Harrison et al., 2012). To date, one of the most effective approaches to QI in public health has come out of The Multi-State Learning Collaborative (MLC), which laid the groundwork for QI and accreditation in public health. The MLC initiative began in 2005, funded by the Robert Wood Johnson Foundation (RWJF) and managed by the National Network of Public Health Institutes (NNPHI) (Gillen, McKeever, Edwards, & Thielen, 2010). The initiative included three phases and ran until April 2011. Five states were selected to participate in phase 1: Illinois, Michigan, Missourt, North Carolina, and Washington. The original five states plus Florida, Kansas, Minnesota, New Hampshire, and Ohio participated in phase 2 (Robert Wood Johnson Foundation [RWJF], 2010). 37

Sixteen states participated in phase 3, all of the phase 2 states apart from Ohio, plus seven new states: Indiana, Iowa, Montana, New Jersey, Oklahoma, South Carolina, and Wisconsin (RWJF, 2010). The first phase of the initiative helped to inform the Exploring Accreditation project, which ultimately concluded that a voluntary national accreditation program was both needed and feasible (RWJF, 2010). Phase 2 was designed to integrate QI into existing capacity and performance assessment or accreditation efforts, while phase 3, known as MLC: Lead States in Public Health Quality Improvement, continued the emphasis on QI and accreditation (Gillen et al., 2010). During the second phase the grantee states recruited LHDs to participate in collaboratives that focused on learning QI skills and implementing QI projects. The participating HDs could choose from 10 target areas to implement QI activities; five of the areas were related to health outcomes and five were capacity related (Gillen et al., 2010). Through their work with the MLC, Michigan, Missouri, and North Carolina were able to enhance their existing accreditation programs. The remaining states were able to enhance their performance measurement programs and begin moving toward accreditation (RWJF, 2010). All of the states and LHDs participating in the MLC were given a QI Maturity Tool to assess QI domains as they relate to 1) organizational culture, 2) capacity and competency, 3) QI practice, and 4) alignment and spread (Joly, Booth, Shaler, & Mittal, 2012a). Results from the assessment indicated an increase in the number of LHDs that implemented QI activities as a result of the program, additionally LHDs reported improvement in QI capacity and competency as they relate to skills, methods, and investment (Joly et al., 2012a). MLC participants were also able to better integrate QI into existing processes and services and work toward an organizational culture committed to QI. 38

Results indicate that the implementation of QI efforts was effective in improving health outcomes as they related to HD services and helped increase capacity in areas such as customer service, health improvement planning, and workforce competence (Joly et al., 2012a). Overall, participation in the MLC helped to improve the understanding and application of QI as well as accelerate the pace of accreditation. As one program manager noted, “Quality improvement and accreditation are mutually supportive. We have a greater understanding now of the relationship between quality improvement and accreditation” (RWJF, 2010, p.23). It has become increasingly apparent that QI, along with accreditation, is imperative in improving the performance of both state and local public health agencies. For this reason it is necessary to recognize those factors that help sustain QI processes such as successful uptake and training of staff (Davis, 2010). From evaluating several statewide performance management and assessment systems, Davis (2010) has identified the following factors that appear to facilitate the uptake of QI in public health. These factors include: 1) encouraging public health leaders who facilitate, support, and provide necessary resources for implementing QI in their agencies, 2) instructing, training, and providing application opportunities to employees, 3) Forming national networks that support QI, and 4) providing the financial means necessary to encourage QI implementation (Davis, 2010). An equally important aspect of cultivating a culture of QI is effectively training staff in using QI processes and tools. Most of the training approaches documented have combined several techniques into one inclusive program. NACCHO used three types of training for LHDs participating in QI projects: 1) webcasts, 2) face-to-face workshops, and 3) applied training in the form of demonstration site projects (Davis et al., 2012). Both the Minnesota Department of Health and local health departments in North Carolina used distance learning as a training 39

strategy (Cornett et al., 2012; Riley et al., 2009). In addition to distance learning, North Carolina also used face-to-face workshops and a QI learning collaborative that included a 9 to 18 month training session to learn QI methods and improve performance (Cornett et al., 2012). Evaluation from this program found that staff felt the training program helped increase buy-in and excitement for QI. Surveys administered to program participants showed that QI training for public health employees should include didactic training on QI content, such as tools and approaches, and opportunities for application of said approaches (Davis et al., 2012; Riley et al., 2009). Figure 8 depicts a conceptual framework of the quality improvement process in public health to help better visualize the process from a management standpoint. It is important to note that appropriate training should take place before executing any QI process. The first step to implementing a QI initiative is to determine the aim behind the QI project (Harrison et al., 2012). Once the project has been selected a QI team that is responsible for carrying out the QI activities should be formed. The team should then choose an appropriate QI tool or approach that will assist them in accomplishing their aim (Harrison et al., 2012). If implemented properly, QI processes should lead to improved performance within the public health agency that in turn leads to improved health outcomes (Dilley et al., 2012).

40

Select a QI Project

Form a QI Team

Choose a QI Tool or Approach (e.g. PDCA, Lean, MFI)

Implement QI intervention/ process

Improved Performance

Improved Health Outcomes

Figure 8. Quality Improvement Conceptual Framework. Figure created by author based on “Quality Improvement Interventions in Public Health Systems: A Systematic Review.” by J. Dilley, B. Bekemeier, and J. Harris, 2012 and “Applying the Model for Improvement in a Local Health Department: Quality Improvement as an Effective Approach in Navigating the Changing Landscape of Public Health Practice in Buncombe, North Carolina.” By L. Harrison, E. Shook, G. Harris, C. S. Lea, A. Cornett, and G. Randolph, 2012.

QI Activities in State and Local Health Departments. As a result of the emphasis on voluntary accreditation in public health, many SHAs and LHDs have already begun using QI within their respective organizations. Surveys indicate that approximately 76.5% of SHAs are currently performing QI in some manner. However, only 7.8% of those SHAs have implemented QI agency wide (ASTHO, 2011a; Yeager et al., 2013). Approximately 65% of LHDs have used QI in some way (NACCHO, 2013b; Yeager et al., 2013). Additionally, studies suggest that LHDs located in a larger jurisdiction and that operate under a centralized governance system (meaning they are units of the SHA) are more likely to report engagement in formal QI efforts, have managers trained in QI, and provide QI training to their employees (Beitsch, Leep, Shah, Brooks, & Pestronk, 2010; Leep, Beitsch, Gorenflo, Solomon, & Brooks, 2009; Yeager et al.,

41

2013). It was also found that LHDs that operate under centralized governance are more likely to receive QI support from their SHA compared to those LHDs that are units of local government (Beitsch et al., 2010; Leep et al., 2009). A convenience sample of 30 LHD interviews conducted by Leep et al. (2009) found that most LHD leaders do not have a shared understanding of what constitutes as formal QI. Furthermore, the concept of agency-wide implementation varies greatly among LHDs (Leep et al., 2009). A study conducted by Beitsch, Rider, Joly, Leep, and Polyak (2013) used the MLC QI Maturity Tool and the NACCHO 2010 Profile of LHDs to classify LHDs based on their use and understanding of QI (Beitsch et al., 2013). They found that over one third of LHDs were classified as being in the beginning stages of QI. This stage is classified by the lack of QI practice, culture, and capacity within the organization. Approximately 15% were classified as emerging, indicating they had demonstrated some informal QI efforts over a short timeframe (Beitsch et al., 2013). One third were classified as progressing in QI, these LHDs had some experience in implementing and applying QI. Finally, 18% were classified as achieving and only 1% were classified as excelling in QI (Beitsch et al., 2013). Those LHDs that engage in QI reported using Baldrige Performance Management, Lean, or Six Sigma for their QI framework. The PDCA cycle was the most commonly cited QI tool, followed by process mapping, fishbone diagrams, and control charts (Beitsch et al., 2010; NACCHO 2011). Unfortunately, approximately 61% do not use any specific framework or QI tool in their LHD (NACCHO, 2011). Results were similar in SHAs that reported engaging in QI efforts, the PDCA cycle was the most popular approach to QI followed by Lean, scorecards, Baldrige, and Six Sigma (ASTHO, 2011).

42

Public Health Accreditation Public health department accreditation can be defined as “the development of a set of standards, a process to measure health department performance against those standards, and reward or recognition for those health departments who meet the standards” (PHAB, 2013 p. 1). The drive for public health accreditation began with the publication of the IOM‟s The Future of Public Health in 1988 that helped establish the three core functions of public health, the 10 essential services, and the National Public Health Performance Standards Program (IOM, 2003; Riley et al., 2012). The Future of the Public’s Health in the 21st Century, published in 2003, further explored the concept of accreditation and called for a committee to consider if accreditation in public health was needed and feasible (IOM, 2003; Russo, 2007). As a result, in 2004 the Robert Wood Johnson Foundation along with the CDC began the Exploring Accreditation Project that eventually led to the initiation of the Multi-State Learning Collaborative (MLC) in 2005. The MLC helped to identify effective practices and confirm that accreditation would be a useful and feasible endeavor for public health (Russo, 2007). Public health agencies have struggled with the concept of measuring their performance in terms of health outcomes for quite some time. It is hoped that one benefit of accreditation will be to finally document that public health agencies are meeting specified levels of performance and provide more accountability and credibility to their services (IOM, 2003; Russo, 2007). Accreditation programs have helped strengthen the health care delivery system, medical programs, and educational programs. Accreditation also helps inform the public about the quality of the services they receive (IOM, 2003). There are numerous potential benefits associated with accreditation in public health, one of the most important being that accreditation sets a benchmark for public health agencies. It also 43

helps create a platform of continuous quality improvement that should increase efficiency, decrease waste, and improve health outcomes (Riley et al., 2012; Russo, 2007). Furthermore, accreditation provides increased visibility and awareness of governmental public health, which could lead to greater public trust and increased support (Riley et al., 2012). Figure 9 depicts a logic model that illustrates the short-term, intermediate, and long-term outcomes that can be associated with accreditation of public health departments.

44

Figure 9. Linking Public Health Accreditation and Outcomes. Reprinted from “Linking Accreditation and Public Health Outcomes: A Logic Model,” by B.M. Joly, G. Polyak, M.V. Davis, J. Brewster, B. Tremain, C. Raevsky, and L.M. Beitsch, 2007. Reprinted with permission 45

Public Health Accreditation Board (PHAB) The Public Health Accreditation Board (PHAB) was established in 2007 by APHA, ASTHO, NACCHO, and the National Association of Local Boards of Health, as a voluntary nonprofit organization to serve as the national public health accrediting body (Riley et al., 2012). PHAB‟s scope extends only to governmental public health departments operated by tribes, states, local jurisdictions, and territories (PHAB 2011; Riley et al., 2012). In 2011 public health accreditation was executed on a national level. There are three prerequisites that health departments must complete and submit with their application in order to be accredited: 1) a community health assessment, 2) a community health improvement plan, and 3) a health department strategic plan. These items must have been completed within the previous 5 years in order to seek accreditation (PHAB, 2011). Domains and Standards PHAB has established domains, standards, and measures as part of the assessment process for public health department accreditation. Domains refer to a group of standards that relate to a wide-ranging group of public health services. There are 12 domains that PHAB assesses. The first 10 domains address the 10 essential services, Domain 11 addresses management and administration, and Domain 12 addresses governance. It should be noted that Domain 9 is dedicated to continuous quality improvement and its implementation within the public health department. The standards are the required level of achievement that the HD is expected to meet. All of the standards and measures have been developed with a strong emphasis on CQI. The measures evaluate if the standard was met (PHAB 2013; Riley et al., 2012). An outline of the 12 domains and their associated standards can be found in Appendix A.

46

Accreditation in Public Health Departments: Current Status According to the 2010 ASTHO Profile of State Public Health survey, 72% of state health departments plan to seek accreditation, and of those 47% plan to seek accreditation within the first 2 years of the program (ASTHO, 2011). As mentioned, in order to complete the application for accreditation, health departments must have completed the following within the past 5 years: a health assessment, a health improvement plan, and an agency-wide strategic plan. The ASTHO profile found that as of 2010 over two thirds of SHAs had ever completed a health assessment, and approximately 48% had completed a health assessment within the last 3 years (ASTHO, 2011). Almost 85% of the SHAs surveyed had a strategic plan in place, but only about half had completed a health improvement plan within the last 3 years (ASTHO, 2011). According to the NACCHO 2013 National Profile of Local Health Departments, 27% of LHDs plan to seek accreditation, and of those, 39% have not decided on a target year for applying (NACCHO, 2013b). According to the 2010 Profile, 60% of LHDs had completed a health assessment and 51% had participated in community health improvement planning. However, only 31% had developed an agency-wide strategic plan within the past 5 years (NACCHO, 2011). As of 2013, 70% had completed a health assessment and 56% had developed a strategic plan within the past 5 years (NACCHO, 2013b) Tennessee Department of Health The governance structure of state and local health departments varies across the U.S. Approximately 60% of all SHAs are governed by a board of health (Hyde & Shortell, 2012). In more than 80% of states legislatures are responsible for approving the SHA‟s budget, determining service fees, and establishing taxes to support public health. The majority of LHDs are also governed by boards of health (Hyde & Shortell, 2012). State and local health

47

departments can be classified into different categories based on their governance structure: 1) centralized or largely centralized, 2) decentralized or largely decentralized, 3) mixed, and 4) shared (ASTHO, 2012). Under centralized governance, local health departments are led by employees of the state and the state retains authority over fiscal decisions. In those states that are decentralized, the LHD is led by employees of local government and the local government retains authority over fiscal decisions. The majority of states are considered decentralized. There are six states that are considered mixed, including Tennessee. Those states operating under mixed governance have some LHDs that are led by employees of the state (centralized) and some that are led by employees of local government (decentralized). No one arrangement is predominant (ASTHO, 2012). Finally, some states have shared governance. In these cases the LHD may be led by either employees of the state or the local government. When they are led by state employees, the local government retains authority over fiscal decisions (ASTHO, 2012). In Tennessee the 89 LHDs are led by the state and report to their respective regional offices, while the six metropolitan health departments are led by county employees. The Director of the Tennessee Department of Health is the Commissioner, who reports directly to the governor of the state, and there is no board of health. The Tennessee Department of Health (TDH) was created in February 1923. Its mission is “to protect, promote, and improve the health and prosperity of people in Tennessee” (Tennessee Department of Health [TDH], 2013 p.1). There are 95 counties in the state of Tennessee, 89 of those counties contain an LHD that is led by one of the seven Regional Health Offices in the state. The other six counties have a metro health department serving their populations. The state is divided into seven regions, with a regional health office located in each one. Figure 10 is a

48

map of the state depicting the seven different regions and six counties that have a metro health department (depicted by contrasting colors in each region). Sullivan MidCumberland West

Upper Cumberland

East

Northeast

Davidson Knox Madison

Southeast

South Central

Hamilton Shelby Figure 10. Map of Tennessee Regions and Metropolitan Counties. Figure created by author based on “Local Health Department Map.” By Tennessee Department of Health (2014).

Each year approximately 1.4 million people are directly served by TDH through one of its 89 rural and six metro county health departments. Others are indirectly impacted by additional services such as inspections of restaurants and healthcare facilities, licensing of health professionals, and laboratory testing (TDH, 2013). The department is focused on protecting people‟s health through prevention. As such, the department provides the following services: immunizations, screenings, dental services, community health education, primary care, maternal and child health (including prenatal care, WIC, home visitations, etc.), vital records, and a safety net of care for underserved populations (TDH, 2013). History of QI within TDH TDH has been involved in some form of quality improvement since 1985. In 1985 focus shifted from corrective action reports and review audits to developing standards in an effort to integrate quality assurance into program activities (K. Shearon, personal communication, August 9, 2013). Quality improvement reviews were developed in 1986 and a manual that provided the

49

various evaluation forms for each service area was given to each health department site (K. Shearon, personal communication, August 9, 2013). By 1992 each region had a QI Administrator, QI Director, QI Nurse, and QI Support Person. In 1995 all employees received mandatory CQI training in the PDCA cycle and other QI tools. CQI training did not take place again until 2007. It was also in 1995 when the department began looking at specific health outcomes as they relate to clinical services provided. In 2008, there was a statewide concentration to improve standardization of the reviews and they were finally placed online. In July 2012 a yearlong project was undertaken to revise the fiscal review and provide newer technology for billing processes. From 2007 to 2013 there was excessive turnover among QI Directors, leaving some lapses in training. Therefore, the state QI Director travelled to each region and provided new regional QI Directors with one-on-one training (K. Shearon, personal communication, August 9, 2013). The state continues to use the reviews to evaluate staff in each program area. Current Status. According to the 2010 ASTHO Profile of State Public Health survey, TDH identified the following as their top five priorities: 1) funding; 2) workforce development, succession planning, and staff training; 3) informational technology upgrade; 4) personnel and employee classifications and hiring practices; and 5) preparedness (ASTHO, 2011). As of 2010 TDH had completed the three prerequisites for accreditation within the past 5 years.

50

CHAPTER 3 METHODS Study Design The Institutional Review Boards (IRB) for East Tennessee State University and Tennessee Department of Health, respectively, reviewed this study proposal and granted an exempt approval in March 2014. This chapter outlines the study sample, methodology, and data analysis plan. This study employed a mixed methods approach to assess the current status of QI processes in regional and metro health departments across the state of Tennessee and examined whether those health departments with a formal QI process demonstrated an increased readiness for public health accreditation. Study Sample The study sample included 13 health department sites in Tennessee, consisting of the seven regional and six metro health departments comprising representation from the entire state. In addition, the study was extended to include selected positions from the 89 LHDs located in the rural counties led by the regional offices. Each region and its corresponding counties are listed in Table 2.

51

Table 2. Regional Health Offices and Their Corresponding Counties Northeast

East

Carter Greene Hancock Hawkins Johnson Unicoi Washington

Anderson Blount Campbell Claiborne Cocke Grainger Hamblen Jefferson Loudon Monroe Morgan Roane Scott Sevier Union

Upper Cumberland Cannon Clay Cumberland DeKalb Fentress Jackson Macon Overton Pickett Putnam Smith Van Buren Warren White

Southeast

South Central Bedford Coffee Giles Hickman Lawrence Lewis Lincoln Marshall Maury Moore Perry Wayne

Bledsoe Bradley Franklin Grundy Marion McMinn Meigs Polk Rhea Sequatchie

Mid Cumberland Cheatham Dickson Houston Humphreys Montgomery Robertson Rutherford Stewart Sumner Trousdale Williamson Wilson

West Benton Carroll Chester Crockett Decatur Dyer Fayette Gibson Hardeman Hardin Haywood Henderson Henry Lake Lauderdale McNairy Obion Tipton Weakley

Twelve positions from each regional and metro health department (including the 89 LHDs within their respective regions) were chosen to be a part of the study due to their responsibilities and likelihood that they were involved in or familiar with QI processes. The 12 positions selected are: Regional Director Assistant Regional Director County Director (Rural LHDs Only) Medical Director Nursing Director Clinical Director Public Information Officer

Primary Care Director Program Director QI Director Accreditation Coordinator Personnel Officer

The 12 positions identified fell under Tier 2 and Tier 3 designations. Tier 2 employees are public health professionals with program management and/or supervisory responsibilities. Other responsibilities include program development, program implementation, program evaluation, establishing and maintaining community relations, managing timelines and work plans, and 52

presenting arguments and recommendations on policy issues (LIFEPATH, 2012). Tier 3 employees are public health professionals at a senior management level or leaders of public health organizations. In general these individuals are responsible for the major programs or functions of an organization, setting a strategy and vision for the organization, and/or building the organization‟s culture. Tier 3 public health professionals typically have staff who report to them (LIFEPATH, 2012). Measures A copy of the survey tool can be found in Appendix B. These questions were derived from the QI Maturity Tool that was developed to assess states involved in the Multi-State Learning Collaborative, funded by the Robert Wood Johnson Foundation (Joly, Booth, Mittal, & Shaler, 2012b; Joly, Booth, Mittal, & Zhang, 2013). The 29-item survey was based on three quality improvement domains: 1) organizational culture, 2) capacity and competency, and 3) alignment and spread (Joly et al., 2012b). Organizational culture describes a public health agency‟s values and norms that determine how it interacts with its staff and stakeholders (Joly et al., 2012b). The capacity and competency domain measures the functions, skills, and approaches used within an agency to assess and improve quality. The alignment and spread domain focuses on the support for QI within the agency as well as its diffusion (Joly et al., 2012b). Each item was rated on a 5-point Likert scale ranging from 1= “Strongly Disagree” to 5= “Strongly Agree.” This survey tool was used several times to evaluate the states that were involved with the MLC and demonstrated acceptable reliability in previous studies with Cronbach‟s alpha estimates ranging from 0.75 to 0.87 (Joly et al., 2013). In addition to the QI Maturity Tool, 10 additional questions were included that specifically addressed the types of formal QI processes used in the health department and their

53

overall intent and associated outcomes. For the purpose of this study, having a formal QI process in place was described as an organization that has 1) integrated QI into the agency strategic and operational plans, 2) formed a QI council that oversees the implementation of a detailed plan to ensure QI throughout the LHD, and 3) commonly uses data for problem-solving and decisionmaking. Informal or Ad hoc QI can be described as practicing discrete QI efforts in isolated instances throughout the LHD, without consistent use of data or alignment with the steps in a formal QI process (NACCHO, 2013a). In an effort to simplify the process and shorten respondent time, demographic questions about the health department such as funding structure, number of employees, and population served were obtained from their responses to NACCHO‟s 2013 National Profile of Local Health Departments Study (NACCHO, 2013b). Use of this data was approved by NACCHO through a data request agreement and was considered IRB exempt. Ninety-two of Tennessee‟s 95 counties completed the NACCHO Profile for a response rate of 97%. Survey Administration The survey was created in SurveyMonkey and consisted of 29 Likert scale items and 10 open-ended questions. The state‟s Performance Improvement Manager was identified by TDH‟s IRB to work with the investigator and identify potential respondents at each health department site based on the requested positions and titles. An email with a link to the survey was sent to regional and metro health department employees fitting the 12 selected positions. The initial email was sent by the state‟s Performance Improvement Manager so that respondents would know that the study was approved and coming from a reputable source. The investigator was not provided with contact information for potential respondents and all contact was made through the Performance Improvement Manager on the investigator‟s behalf. The survey was open from 54

April 1, 2014 through May 1, 2014. Respondents were able to complete the survey any time during this 30-day period from a venue of their choice, with a reminder email being sent to all respondents 2 weeks prior to the closing date. Data Analysis All data were analyzed using SPSS 21.0. Descriptive statistics were used to assess individual responses to all items, including the mean, standard deviation, and range of scores. Bivariate analyses were conducting for each aim and hypothesis where appropriate. NACCHO Profile Data was analyzed to depict each site‟s jurisdiction, expenditures, and size (number of FTE employees) by site. Because NACCHO Profile Data are reported by LHD, each LHD was recoded as their respective regional office or metro health department where appropriate, providing a response for each region and metro. Aim 1 To produce a QI maturity score by site (regional office or metro health department), a value was assigned to each Likert scale survey item (strongly agree=5, agree=4, neutral =3, disagree=2, strongly disagree=1, I don‟t know=0). Based on their overall score, each health department site was categorized into one of five QI classifications: 1) beginning, 2) emerging, 3) progressing, 4) achieving, or 5) excelling (Joly et al., 2013). Table 3 outlines how the QI maturity score was assigned into the appropriate classification. Because several employees from each site completed the survey, the mean score for each domain was calculated based on all responses. Data from a site were not included if fewer than three respondents completed the survey. The final mean score for each site was then used to determine QI maturity. The same process was then used to determine the QI maturity score for each staff position.

55

Table 3. QI Maturity Classification by QI Maturity Score Range QI Maturity Classification

Score Range

Beginning

≤ 99

Emerging

100-106

Progressing

107-120

Achieving

121-139

Excelling

≥140

Differences in QI maturity score by site designation were determined using a one-sided independent t-test. Differences in QI maturity score by position were determined using a oneway analysis of variance. Aim 2 To determine the current status of QI processes, mean scores for each domain were reported to depict each health department site‟s QI organizational culture, QI capacity, and QI alignment. Respondents were asked 10 open-ended questions in addition to the 29-item QI Maturity Tool. These questions asked respondents to further describe the types of QI processes or activities that were used in their LHD as well as their outcomes. For each of the qualitative questions, content analysis was conducted to identify emerging themes. Each question was then coded based on identified categories. The responses to these questions were then cross referenced to each site‟s overall QI maturity score to further assess the current status of QI processes in each site.

56

Aim 3 Questions from the QI Maturity Tool were cross-walked to specific PHAB standards and measures in domain nine. This process was used to gauge health department readiness for public health accreditation by identifying those standards that have been met with respect to quality improvement and performance management. To determine a readiness score the researcher matched survey items to appropriate PHAB measures within domain nine. Mean scores were reported for each of the Likert scale items on the QI Maturity Tool. Sites were assigned 1 point for each item if the mean score fell between 4.00 and 5.00 indicating that most respondents „agreed‟ or „strongly agreed‟ with the item. Some of the items matched more than one PHAB measure and in those cases were worth 2 points. A site could receive up to 29 “readiness” points. A higher score indicates an increased readiness for accreditation based on QI processes and activities. Hypotheses A Pearson‟s product-moment correlation tested the hypothesis that health departments that engage in organization wide formal QI show an increased readiness for accreditation, by examining the relationship between QI alignment (Domain 3 from the QI Maturity Tool) and accreditation readiness. To test the hypothesis that a higher QI maturity level (as assessed by the QI Maturity score) indicates an increased capacity for accreditation, a Pearson‟s product moment correlation examined the relationship between QI maturity score and accreditation readiness.

57

CHAPTER 4 RESULTS The results are organized into three sections. The first section consists of the descriptive statistics including demographics of each site as reported to NACCHO. The second and third sections present the results of the study by each research aim and corresponding hypotheses. The researcher assessed each site and position by their corresponding QI Maturity scores as well as their domain scores. Qualitative data were analyzed to assess the current status of QI processes across the state, including the type and extent of QI used. Descriptive Statistics Descriptive information about each health department site is provided in the following three tables (Tables 4 to 6). These data were provided by the Regional or County Director for each health department to NACCHO and included in the 2013 NACCHO Profile. Although data were reported by each LHD, for the purpose of this study only regional and metro health department information is reported. Table 4 depicts population size by health department site. Eighty-nine of Tennessee‟s 95 counties are considered rural, which can be seen by examining population size. Sixty-five LHDs reported serving a population of 50,000 or less, and 25 LHDs reported serving a population between 50,000 and 250,000. As expected, metro health departments tended to report a much larger population size than their rural counterparts.

58

Table 4. Population Size by Region or Metro Region or Metro

Population Size

# of counties (%)

Northeast

Less than 50,000 50,000-250,000

3 (42.9) 4 (57.1)

East

Less than 50,000 50,000-250,000

9 (60.0) 6 (40.0)

Upper Cumberland

Less than 50,000 50,000-250,000

12 (85.7) 2 (14.3)

Southeast

Less than 50,000 50,000-250,000

8 (80.0) 2 (20.0)

Mid-Cumberland

Less than 50,000 50,000-250,000 250,000-500,000

5 (41.7) 6 (50.0) 1 (8.3)

South Central

Less than 50,000 50,000-250,000

10 (83.3) 2 (16.7)

West

Less than 50,000 50,000-250,000

18 (94.7) 1 (5.3)

Sullivan

50,000-250,000

1(100.0)

Knox

250,000-500,000

1(100.0)

Hamilton

250,000-500,000

1(100.0)

Davidson

More than 500,000

1(100.0)

Madison

50,000-250,000

1(100.0)

Shelby

More than 500,000

1(100.0)

Annual expenditures by site are reported in Table 5, only eight counties reported expenditures of more than $3,000,000 with three of those counties being designated as metro health departments. Three counties in the Mid-Cumberland region also reported expenditures of over $3,000,000.

59

Table 5. Agency Expenditures by Region or Metro Region or Metro Northeast

Expenditures # of counties (%) $500,000-$3,000,000 6 (85.7) More than $3,000,000 1 (14.3)

East

Less than $500,000 1 (6.7) $500,000-$3,000,000 14 (93.3)

Upper Cumberland Less than $500,000 $500,000-$3,000,000 More than $3,000,000 Missing

1 (7.1) 6 (42.9) 1 (7.1) 6 (42.9)

Southeast

Less than $500,000 $500,000-$3,000,000 Missing

2 (20.0) 4 (40.0) 4 (40.0)

Mid-Cumberland

Less than $500,000 $500,000-$3,000,000 More than $3,000,000 Missing

3 (25.0) 5 (41.7) 3 (25.0) 1 (8.3)

South Central

Less than $500,000 $500,000-$3,000,000 Missing

1 (8.3) 2 (16.7) 9 (75.0)

West

Less than $500,000 $500,000-$3,000,000

4 (21.1) 15 (78.9)

Knox

More than $3,000,000 1 (100.0)

Davidson

More than $3,000,000 1 (100.0)

Shelby

More than $3,000,000 1 (100.0)

Note. Sullivan, Hamilton, and Madison counties did not report

The number of full time equivalents (FTE), employees who work more than 30 hours per week, by site is depicted in Table 6. Only eight counties reported having more than 50 FTEs; of those three are classified as metro health departments.

60

Table 6. Agency Size (Full Time Equivalents) by Region or Metro

Region or Metro

Agency Size (FTE)

# of counties (%)

Northeast

Less than 10 10-49 More than 50

1 (14.3) 5 (71.4) 1 (14.3)

East

Less than 10 10-49

1 (6.7) 14 (93.3)

Upper Cumberland

Less than 10 10-49 More than 50

5 (35.7) 8 (57.1) 1 (7.1)

Southeast

Less than 10 10-49 More than 50

6 (60.0) 3 (30.0) 1 (10.0)

Mid-Cumberland

Less than 10 10-49 More than 50

3 (25.0) 7 (58.3) 2 (16.7)

South Central

Less than 10 10-49

5 (41.7) 7 (58.3)

West

Less than 10 10-49 Missing

4 (21.1) 14 (73.7) 1 (5.3)

Knox

More than 50

1 (100.0)

Davidson

More than 50

1 (100.0)

Shelby

More than 50

1 (100.0)

Note. Sullivan, Hamilton, and Madison counties did not report

Descriptive Statistics for Survey Sample The survey was sent to 225 potential respondents. Out of those, 125 respondents completed the survey, for a response rate of 55.1%. Of that total 89% answered all of the Likert scale items and an average of 61% of respondents answered qualitative questions. The majority of respondents identified themselves as a Nursing Director or Supervisor, Program Director, or 61

County Director at 28.0%, 18.0%, and 16.1% respectively. Approximately, 34% of the overall respondents were in clinical positions. Responses were received from each regional health department and all of the metro health departments except Shelby County. Sixteen respondents did not indicate their health department site and are therefore not included in many of the analyses. A summary of the number of respondents from each site can be found in Table 7. Knox County had the largest number of respondents complete the survey (n=21), while Hamilton and Davidson counties had one respondent each. Due to their small sample size, Hamilton and Davidson counties are omitted from analysis depicting data by individual health department site. Table 7. Number of Respondents by Site Site Northeast East Upper Cumberland Southeast Mid-Cumberland South Central West Sullivan Knox Hamilton Davidson Madison Total Site Not Indicated Total

n 7 13 12 16 11 10 11 3 21 1 1 3 109 16 125

(%) 5.6 10.4 9.6 12.8 8.8 8.0 8.8 2.4 16.8 .8 .8 2.4 87.2 12.8 100.0

As previously noted, employees in 12 positions were selected to be a part of the study based on alignment of their job description with functions that are central to QI. Responses were 62

received from employees in nine of the selected positions with the majority of respondents identifying themselves as Nursing Directors/Supervisors, Program Directors, and County Directors respectively (Table 8). Fourteen respondents identified themselves as „other‟ and seven did not report their position. Table 8. Number of Respondents by Position Position or Title Regional Director Asst. Regional Director County Director Medical Director Nursing Director/Supervisor Clinical Director Program Director QI Director Public Information Officer Other Total Position Not Indicated Total

n

Percent (%)

8 3 19 3 33 4 22 8 4 14 118 7 125

6.4 2.4 15.2 2.4 26.4 3.2 17.6 6.4 3.2 11.2 94.4 5.6 100.0

Descriptive information for each survey item by domain is presented in Table 9. Respondents answered “agree” or “strongly agree” for most items on the QI Maturity Tool. Cronbach‟s alpha for this sample was 0.94 indicating consistent internal reliability. The highest scoring item (mean score=4.31) on the survey was item 26 located in Domain 3, “spending time and resources on QI is worth the effort.” The lowest scoring item was number 24; also in Domain 3, “staff have authority to make change” with a mean score of 2.99.

63

Table 9. Descriptive Information on QI items Responses in (%) SDa Item Description*

M (SD)

SA

1

2

3

4

5

DK

9

Domain #1: QI Organizational Culture 1. Leaders receptive to ideas for improving quality

4.25 (.87)

2.4

2.4

7.2

44.0

44.0

0.0

0.0

2. Impetus for improving quality is internal

4.18 (.93)

2.4

4.8

7.2

43.2

41.6

0.0

0.8

3. Leaders work together for common goals

4.23 (.97)

1.6

3.2

10.4

36.0

47.2

0.8

0.8

4. Staff help one another solve problems

4.25 (.79)

0.8

4.0

4.8

49.6

40.0

0.0

0.8

5. Staff routinely contribute to decisions

3.95 (1.1)

1.6

8.8

13.6

36.0

37.6

1.6

0.8

6. Leaders are trained in basic QI methods

3.52 (1.6)

0.8

5.6

8.0

41.6

28.8

13.6

1.6

7. Staff members are trained in basic QI methods

3.31 (1.5)

0.8

7.2

19.2

39.2

19.2

12.8

1.6

8. Staff have skills to assess quality of programs

4.00 (.96)

0.8

5.6

7.2

56.0

27.2

1.6

1.6

9. Agency has objective quality measures

3.94 (1.2)

0.8

4.0

10.4

44.0

34.4

4.8

1.6

10. Staff use methods to identify root causes

3.59 (1.2)

0.8

8.8

19.2

47.2

17.6

4.8

1.6

11. Staff use best or promising practices

3.91 (1.1)

0.8

4.8

8.8

51.2

28.0

4.0

2.4

12. Programs are continuously evaluated

3.93 (.99)

0.8

6.4

10.4

52.8

25.6

1.6

2.4

13. Agency routinely monitors programs/services

4.08 (1.1)

0.0

3.2

4.0

52.8

34.4

4.0

1.6

14. Agency has a QI officer

4.29 (1.1)

0.0

4.0

4.0

33.6

53.6

3.2

1.6

15. Agency has a QI council, committee, or team

3.84 (1.6)

0.0

4.0

5.6

33.6

43.2

11.2

2.4

16. Agency has a QI plan

4.15 (1.2)

0.0

0.8

4.0

44.8

43.2

5.6

1.6

17. Job descriptions include QI responsibilities

3.92 (1.3)

0.8

2.4

10.4

43.2

32.8

5.6

4.8

18. Staff are aware of external QI expertise

3.66 (1.4)

0.8

3.2

14.4

41.6

26.4

8.8

4.8

19. Staff at all levels participate in QI

4.01 (1.1)

0.8

4.8

8.8

42.4

34.4

3.2

5.6

20.Customer satisfaction information is routinely used

3.99 (1.2)

0.8

2.4

11.2

43.2

33.6

4.0

4.8

21. QI efforts are usually adopted by other programs

3.39 (1.5)

0.8

3.2

22.4

39.2

18.4

11.2

4.8

22. Accurate and timely data are available for QI

3.72 (1.2)

0.8

5.6

14.4

48.0

20.8

4.8

5.6

23.Improving quality is integrated into agency practice

4.00 (.99)

0.8

4.8

11.2

46.4

29.6

Domain #2: QI Capacity and Competency

Domain #3: QI Alignment and Spread

64

1.6

5.6

Table 9 (continued) Responses in (%) SDa Item Description*

24. Staff have authority to make change

M (SD)

1

SA 2

3

4

5

DK

9

2.99 (1.3)

3.2

26.4

23.2

27.2

9.6

4.8

5.6

3.07 (1.3)

5.6

19.2

26.4

28.0

10.4

4.0

6.4

effort

4.31 (.86)

1.6

0.0

7.2

40.8

44.8

0.8

4.8

27. Key decision makers believe QI is very important

4.25 (1.2)

1.6

1.6

6.4

27.2

53.6

4.0

5.6

28. Using QI will impact the health of the community

4.20 (.90)

0.0

1.6

9.6

43.2

38.4

1.6

5.6

4.07 (.98)

0.0

1.6

12.8

44.8

32.0

2.4

6.4

25. Staff have authority to work across program boundaries 26. Spending time and resources on QI is worth the

29. Staff and stakeholders will notice changes due to QI Note. *Full item wording in Appendix B a SD=Strongly Disagree SA=Strongly Agree 9=missing responses DK=I don‟t know

Descriptive results revealed that the majority of respondents identified themselves as a Nursing Director or Supervisor. Approximately 87 % of respondents endorsed item 14 that their health department site has a QI Officer, yet the response rate from that position was low at 6.4%. As noted in Table 9 there is little variation in responses, with respondents answering “agree” or “strongly agree” for most items on the QI Maturity Tool. The following sections further analyze the data by site and position. Aim 1: Classify Health Department Sites Based on Their Level of QI Engagement The first step in the analyses was to assign each site into the appropriate QI classification based on their responses to the 29-item QI Maturity Tool. The distribution of responses by QI classification revealed that the majority of respondents categorized their LHDs as either 65

“achieving” (n=40, 32.0%) or “progressing” (n=35, 28.0%). Similar responses are noted when looking at the distribution of scores based on site designation as either regional or metro (Figures 11 and 12). Based on their scores, 35% of rural health departments or regional offices were classified as “progressing,” indicating that these LHDs “have some QI experience and capacity but often lack commitment, have minimal opportunities for QI integration throughout the agency and are less sophisticated in their application and approach” (Joly et al., 2013). Approximately 38% were classified as “achieving,” which represents high levels of QI practice, a commitment to QI, and a willingness to engage in organizational change. Only 3% were classified as “excelling” indicating a high level of QI sophistication and a pervasive QI culture (Joly et al., 2013).

Figure 11. QI Classification in Tennessee Regional Health Departments In the metro health departments 22% were classified as “progressing” and 44% as “achieving.” None of the sites were classified as “emerging,” but approximately 15% identified as “excelling” in QI (Figure 12).

66

Figure 12. QI Classification in Tennessee Metro Health Departments As seen in Figure 13, QI classification by position reflects a similar distribution of responses. The highest QI scores were reported by Clinical Directors, QI Directors, Program Directors, Nursing Directors, and those respondents classified as „other‟.

Figure 13. QI Classification by Position or Title The mean QI scores by region and metro are depicted in Figures 14 and 15. Sites with scores in the 100 to 106 range classified as “Emerging.” Scores in the 107 to 120 range are 67

considered “progressing,” and scores in the 121-139 range are classified as “achieving.” A score of 140 or higher is considered “excelling.” Based on these scores, the East and West regions classified as “emerging,” Southeast, Mid-Cumberland, and South Central regions classified as “progressing,” and the Northeast and Upper Cumberland regions as “achieving.”

Figure 14. Mean Quality Improvement Score by Region In the metro health departments, three sites classified as “progressing,” Sullivan County, Knox County, and Madison County, respectively. Both Hamilton and Davidson were omitted from this analysis due to the low response rate for these two sites (n=1). No responses were received from Shelby County, the remaining metro health department site in the state.

68

Figure 15. Mean Quality Improvement Score by Metro No differences in QI score were found based on site designation as either region or metro, t (107) = -1.080, p=.283. This could be due to sample size for the metro sites n=29 compared to n=89 for regional sites. There was not a statistically significant difference in QI score by position as determined by one-way analysis of variance F (4,113) =2.064, p=.090 Aim 2: Current Status of QI Processes in Regional and Metro Health Departments To assess current QI processes in regional and metro health departments, domain scores for each site were calculated and cross referenced to qualitative responses from the survey. The QI Maturity Tool consists of three domains: organizational culture, capacity and competency, and alignment and spread, which are used to assess a public health agency‟s QI maturity (Joly et al., 2012b). Domain one, organizational culture, consists of the first five items on the QI Maturity Tool. The maximum score that one can receive for this domain is 25. Domain two, capacity and competency, encompasses items 6 through 16 with a total possible score of 55.The final domain, alignment and spread, includes items 17 through 29, with a total possible score of 65.

69

Domain one scores by site are depicted in Figure 16. Scores ranged from 18 to 23 with lower scores indicating that organizational values and norms are less centered on QI than at other sites. Madison County had the highest score at 23.3 while South Central had the lowest at 18.2 (Figure 16).

Figure 16. Domain 1 Scores by Site Domain two scores are depicted in Figure 17. Domain two scores ranged from 39 to 44, in this domain the highest attainable score is 55. This domain assesses QI skills, application, and function. A higher score indicates an organization with a high level of QI skill and application. The West region had the lowest domain two score at 39.7 while Sullivan and Knox counties both had domain two scores in the 47.3 range (Figure 17). Domain three assesses an organization‟s support and diffusion of QI. The highest attainable score in this domain is 65, with higher scores indicating a high level of internal support for QI processes and widespread diffusion throughout the organization. Scores in this domain ranged from 48 in the West region to 55.3 in the Northeast (Figure 18).

70

Figure 17. Domain 2 Scores by Site

Figure 18. Domain 3 Scores by Site Respondents were asked to identify the types of formal QI processes or activities used in their respective health department site. The types of formal QI processes used by site are depicted in Figure 19. The majority of sites reported using Lean most often as part of their formal QI efforts, followed by Baldrige, and then PDCA. It should be noted that all responses under the „other‟ category referred to chart audits and quality assurance (QA) activities used as part of the state mandated QI program. 71

Figure 19. Types of Formal QI by Site. Note. Multiple response category, percentages for each site will not equal 100 Informal QI refers to ad hoc processes or activities that are not organizational wide (NACCHO, 2013a). When asked “What informal QI processes does your health department use?” the majority of respondents (51.4%) reported using chart reviews or audits. Approximately 27% reported meetings as part of their informal QI process. It should be noted that approximately 16% of responses fell into the „other‟ category, as those responses did not align with other activities identified. For example, one respondent stated, “risk minimization,” as their LHD‟s informal QI process. Another respondent explained the process as “we consistently ask ourselves how we will measure our effectiveness and request that coworkers focus on their desired outcomes in implementation.” The distribution of responses is depicted in Figure 20.

72

Figure 20. Responses to the Question, What Informal QI Processes Does Your Health Department Use? Note: n=70, some responses fall into multiple thematic categories Respondents were asked to describe their motivation for using formal QI processes. Responses to this question are depicted in Figure 21. Most respondents cited their motivation as wanting to improve quality of care and services provided to their patients (33.7%) or improving health department efficiency and effectiveness (19.8%). Approximately 9% of responses fell under „other.‟ Examples of these responses include, “to be good stewards of tax payer money…,” and “to maintain a structured form of response.” Only a fraction of respondents (2.3%) mentioned public health accreditation as a motivating factor for using or implementing formal QI processes (Figure 21).

73

Figure 21. Responses to the Question, What is Your Motivation for Using Formal QI Processes? Note: n=86 The response to, “What outcomes do you expect to see in your health department from using formal QI processes?” saw similar response categories as the previous question, with improved quality of care and improved efficiency encompassing a large portion of answers at 25.5% and 24.4%, respectively (Figure 22). Approximately, 11% of respondents specifically mentioned wanting to improve audit outcomes, which is a quality assurance (QA) measure and not explicitly related to QI.

Figure 22. Responses to the Question, What Outcomes do You Expect to See in Your Health Department from Using Formal QI Processes? Note: n=86, some responses fall into multiple thematic categories 74

Respondents were asked to describe both positive and negative outcomes that they have seen in their respective health departments from the use of QI (Figures 23 and 24). Approximately 16.4% of respondents reported seeing an increase in staff morale and teamwork. Interestingly, 27.3% of the responses mentioned some quality assurance related outcomes, such as improved audit scores and improved coding or documentation. A large number of the responses (39.7%) fell into the „other‟ category. Those responses marked as „other‟ varied greatly and did not align with the other identified categories. For example, one respondent cited, “better understanding of purchasing procedures and policy,” as a positive outcome of using QI. While another respondent noted that, “the letter of commitment has increased accountability in our grant partnerships,” as one positive outcome they have seen in their LHD (Figure 23).

Figure 23. Responses to, Please Describe One Positive Outcome That You Have Seen in Your Health Department From the Use of QI. Note: n=73 Approximately 34% of respondents did not feel that there were any negative outcomes associated with QI implementation. Other respondents cited the use of QI as being time consuming (16.2%) and discouraging to staff (13.2%). Approximately 7% found the process to

75

be a waste of resources and 5.9% found QI implementation to be too redundant and repetitive (Figure 24). A large number of responses (17.6%) fell into the „other‟ category. Some examples of these responses include, “sometimes I think the way we conduct the QI audits and handle the results is not really improving the outcomes we want.” Another respondent cited finding that, “all of our patients were not satisfied with our services,” as a negative outcome of using QI (Figure 24).

Figure 24. Responses to, Please Describe One Negative Outcome That You Have Seen in Your Health Department From the Use of QI. Note: n=73 Finally, respondents were asked to describe one process or project in which QI concepts were used (Figure 25). Only 33.3% described a true QI process or activity as defined by NACCHO, while 24.2% specifically referred to chart audit and review processes. Approximately 30% of respondents described a process that could not be categorized as either QA or QI and were given the designation of „other.‟ For example, one respondent cited, “revamping segregation of duties in the region to eliminate redundancy and still being able to remain protected against fraudulent activities,” as an example of applying QI concepts in their LHD. While another respondent cited, “inventory management,” as the QI concept used in their site. In 76

total, over half of the respondents (67%) described a process not related to or considered to be QI (Figure 25).

Figure 25. Responses to, Please Describe One Process or Project in Which QI Concepts Were Employed. Note: n=66 To illustrate any differences between QI Maturity scores and responses to qualitative questions responses to two of the questions were cross referenced with each sites QI classification (Tables 10 and 11). Table 10 shows responses to the question, “What informal QI processes/activities does your HD use?” In this Table health department sites are arranged by their QI classifications from “emerging” to “achieving.” All respondents mentioned chart audits or reviews as one of their informal QI activities except Sullivan and Knox Counties. The majority of sites were classified as “progressing,” indicating that they have experience in QI but lack commitment to QI and have had minimal opportunities for integration. Although their classification indicates QI experience, South Central, West, Sullivan, and Madison did not specifically cite using any QI tools in their response to this question. The Upper-Cumberland region did not cite using any QI tools; however, its classification of “achieving,” would indicate a high level of QI practice and 77

commitment. The responses for the East region may reflect a more accurate response. Its QI classification was “Emerging,” and the majority of respondents cited using QA instead of QI. Table 10. Comparison of QI Classification and Response to What Informal QI Processes/Activities Does Your Health Department Use? Site-QI Classification (based on self-reported scores) East-Emerging

West-Emerging

Southeast-Progressing

Mid-CumberlandProgressing South Central-Progressing

Sullivan-Progressing Knox-Progressing

Madison-Progressing Northeast-Achieving

Upper CumberlandAchieving

Response to: “What informal QI processes/activities does your HD use?” 88.9% cited chart reviews/audits 22.2% cited informal meetings (e.g. feedback session, discussions) 11.1% cited a QI process/activity (e.g. Lean, PDCA, etc.) 57.1% cited chart reviews/ audits 14.3% cited formal meetings (e.g. management meetings, focus groups) 14.3% were not aware of what informal QI processes/activities were used 14.3% of responses fell into an „other‟ category 70.0% cited chart reviews/ audits 30.0% cited a QI process/activity (e.g. Lean, PDCA, etc.) 20.0% of responses fell into an „other‟ category 60.0% cited chart reviews/ audits 20.0% cited a QI process/activity (e.g. Lean, PDCA, etc.) 20.0% of responses fell into an „other‟ category 87.5% cited chart reviews/ audits 12.5% cited customer satisfaction surveys 12.5% of responses fell into an „other‟ category 50.0% cited informal meetings (e.g. feedback session, discussions) 50.0% cited formal meetings (e.g. management meetings, focus groups) 8.3% cited informal meetings (e.g. feedback session, discussions) 16.7% cited formal meetings (e.g. management meetings, focus groups) 33.3% cited using a QI process/activity (e.g. Lean, PDCA, etc.) 8.3% cited using customer satisfaction surveys 41.7% of responses fell into an „other‟ category 50.0% cited chart reviews/audits 50.0% cited customer satisfaction surveys 50.0% cited chart reviews/audits 25.0% cited informal meetings (e.g. feedback session, discussions) 25.0% cited formal meetings (e.g. management meetings, focus groups) 25.0% cited a QI process/activity (e.g. Lean, PDCA, etc.) 42.9% cited chart reviews/audits 28.6% cited informal meetings (e.g. feedback session, discussions) 28.6% cited formal meetings (e.g. management meetings, focus groups) 14.3% of responses fell into an „other‟ category

Note. n=70, some responses fell into multiple categories; therefore, totals will not equal 100%. Evaluation activities and use of data can be considered informal QI, the QI category depicts when respondents specifically listed a commonly used QI activity or process such as PDCA or Lean.

78

Table 11 depicts the responses to, “Please describe one process or project in which QI concepts were employed.” For this Table, health department sites are arranged by their QI classifications from “emerging” to “achieving.” As seen with the previous question, respondents in each site, except Knox County, reported using QA activities such as chart audits and reviews as part of their QI program. For this question responses marked as true QI indicates that they meet the formal definition of QI by NACCHO or used one of the common QI processes or tools (PDCA, Lean, etc.). A large percent of the response fell into the „other‟ category. In these instances, the response given did match QA or QI definitions, and the researcher was unable to appropriately determine what type of concept was used. Table 11. Comparison of QI Classification and Responses to, Please Describe One Process or Project in Which QI Concepts Were Employed Site-QI Classification (based on self-reported scores) East-Emerging

Response to: “Please describe one process or project in which QI concepts were employed.” 33.3% cited chart audits or reviews as their QI project 44.4% engaged in a true QI process or activity 22.2% of responses fell into an „other‟ category

West-Emerging

50.0% cited chart audits or reviews as their QI project 16.7% engaged in a true QI process or activity 16.7% were not aware of any QI concepts being used 16.7% of responses fell into an „other‟ category

Southeast-Progressing

South Central-Progressing

27.3% cited chart audits or reviews as their QI project 36.4% engaged in a true QI process or activity 18.2% were not aware of any QI concepts being used 18.2% of responses fell into an „other‟ category 40.0% engaged in a true QI process or activity 60.0% of responses fell into an „other‟ category 71.4% cited chart audits or reviews as their QI project 28.6% engaged in a true QI process or activity

Sullivan-Progressing

100.0% cited using customer satisfaction surveys as their QI project

Knox-Progressing

25.0% engaged in a true QI process or activity 25.0% cited using customer satisfaction surveys as their QI project 50.0% of responses fell into an „other‟ category

Madison-Progressing

100.0% engaged in a true QI process or activity

Mid-Cumberland-Progressing

79

Table 11 (continued) Site-QI Classification (based on self-reported scores) Northeast-Achieving

Upper Cumberland-Achieving

Response to: “Please describe one process or project in which QI concepts were employed.” 25.0% cited chart audits or reviews as their QI project 50.0% engaged in a true QI process or activity 25.0% of responses fell into an „other‟ category 14.3% cited chart audits or reviews as their QI project 28.6% engaged in a true QI process or activity 57.1% of responses fell into an „other‟ category

Note. n=66.

Respondents were asked if they plan to apply for public health accreditation or TNCPE in an effort to assess future plans and QI efforts. In addition, they were also asked if they have an Academic Health Department (AHD) or have collaborated with one on projects. Responses to these questions are outlined in Figure 26. Approximately, 46% plan on applying for accreditation, while 26.3% plan to apply for TNCPE. Additionally, approximately 34% reported having or working with an AHD. The „other‟ category encompasses such responses as “maybe” and “not sure, but may already participate.”

Figure 26. Percent that Plan to Apply for Accreditation, TNCPE, and Currently Have or Have Worked with an AHD 80

Aim 3: Readiness for Public Health Accreditation To assess readiness for public health accreditation, each site was assigned a “readiness” score based on its mean score for selected survey items cross-walked to appropriate measures in domain 9 of PHAB. A score of 21 or above indicates that the site has met at least three quarters of the measures outlined in domain 9. The detailed cross-walk of QI Maturity Tool items to PHAB Measures is presented in Appendix B. The readiness score for each site is depicted in Table 12. Knox County received the highest score at 24 points, indicating a high level of readiness for accreditation based on domain 9. The Southeast region received the lowest score (10 points), indicating that it has met approximately one third of the measures outlined in PHAB. Due to their low response rate (n=1), Hamilton and Davidson Counties were omitted from these data. Table 12. Accreditation Readiness Score by Site

Site Northeast East Upper Cumberland Southeast Mid-Cumberland South Central West Sullivan Knox Madison

n (number of respondents) 7 13 12 16 11 10 11 3 21 3

Mean Readiness Score (Maximum of 29 points) 23 17 22 10 14 15 11 22 24 19

81

Hypotheses Hypothesis 1 A Pearson product-moment correlation was computed to assess the relationship between Domain 3 and the assigned readiness score to determine if health departments that engage in organization wide formal QI show an increased readiness for public health accreditation. Results revealed a positive correlation between Domain 3 scores and readiness scores that were statistically significant (r=.297, n=109, p .005). Figure 27 summarizes the results in a scatterplot.

Figure 27. Correlation Between Domain 3 and the Assigned Readiness Score

Hypothesis 2 A Pearson product-moment correlation was computed to assess the relationship between QI Maturity scores and the assigned readiness score to determine if health departments that have a higher QI maturity level demonstrate an increased capacity for public health accreditation. Results revealed a positive correlation between QI Maturity scores and readiness

82

scores that were statistically significant (r=.331, n=109, p .005). Figure 28 summarizes the results in a scatterplot.

Figure 28. Correlation Between QI Maturity Score and Assigned Readiness Score

83

CHAPTER 5 DISCUSSION Introduction This study was an examination of the current status of QI in a sample of regional and metro health departments across the state of Tennessee to assess whether those health departments with a formal QI process demonstrate an increased readiness for public health accreditation. Previous studies suggest that public health agencies that implement organizationwide QI activities will see improvement in health outcomes, delivery of the 10 essential services, and an increase in capacity (Dilley et al., 2012; Joly et al., 2012a). The development of public health accreditation has seen an increased focus on QI processes and how it can benefit SHAs and LHDs (IOM, 2003; Russo, 2007). It is hoped that accreditation will help strengthen the public health system while also improving population-based outcomes. Thus, this study provides a way for health departments to measure their current QI Maturity and apply those findings to accreditation readiness. Summary of Findings Aim 1 The results of the QI Maturity Tool indicated that most of the health department sites are “progressing” or “achieving” in QI practice. This suggests that these LHDs have some QI knowledge and experience, have implemented QI processes throughout the organization, and are committed to organizational change (Joly et al., 2013). Some responses to the Likert scale items may reflect the unique structure of regional health departments compared to their metro counterparts, as indicated in question 24 of the

84

survey, “staff have the authority to make change.” This may be due to differences based on the level of governance (centralized versus decentralized) in regional and metro health departments as metro health department have a bit more autonomy than their regional counterparts. The investigator expected metro health departments to demonstrate a higher level of QI sophistication than the regional health departments, based on their level of governance. However, the three metro sites included in the analysis, Sullivan, Knox, and Madison counties, had comparable scores to their regional counterparts and were all classified as “progressing.” This observation contrasts to what previous studies have found. In 2010 Beitsch et al. found that LHDs that operate under centralized governance (in Tennessee, rural and regional health departments) are more likely to receive QI support from their SHA compared to LHDs that are units of local government (in Tennessee, metro health departments). However, the fact that QI scores for the regional and metro health departments were fairly consistent indicates that they are receiving the same type of training and support from TDH. QI scores by position were distributed similarly to the scores by site with most employees indicating that their LHD was “progressing” or “achieving” in QI. Those instances where the employees scored their LHD as “beginning” or “emerging” are likely affected by the sample size for that particular position. It is important to note that most of the respondents identified themselves as clinical, which may explain some of the responses to the open-ended questions discussed in Aim 2. These employees are generally more familiar with quality assurance measures (chart audits and reviews) and may be less likely to be trained in public health related QI processes. Overall, more variation in scores was expected; however, the majority of respondents answered “agree” or “strongly agree” to Likert scale items in this sample. When the same survey

85

tool was administered to LHDs in Minnesota, approximately 33% were classified as beginning, 15% as emerging, 33% as progressing, 18% as achieving, and only 1% as excelling (Beitsch et al., 2013). These variations in results may be attributed to the state definition of quality improvement in Tennessee. The current quality improvement program has been in existence since the mid-1980s. While this program is defined as QI, it specifically focuses on chart reviews and audits that the site‟s QI Director completes every 6 months. These activities are considered quality assurance activities in that the goal is to reduce errors in documentation and coding. While these audits are important to the LHDs processes and overall performance, they are not considered to be true QI as defined by NACCHO and PHAB. These contrasting definitions of QI may indicate that the QI Maturity classifications assigned to each site are not a true representation of their QI sophistication.

Aim 2 The purpose of Domain 1 is to assess an organization‟s QI culture, specifically focusing on its values and norms. Domain 2 focuses on QI capacity by assessing the function, skills, and application of QI processes. Finally, domain 3 is an assessment of the level of support for QI processes and the diffusion of QI throughout the organization. Scores for domain 1 were relatively high across all sites indicating that the organization values QI and it is a part of the regular culture. However, scores for domains 2 and 3 were slightly lower. Scores for domain 2 ranged from 39 to 47, with the highest attainable score being 55. This indicates that the health department sites may struggle with the application of QI processes and need more training to support that skill set. Scores for domain 3 ranged from 48 to 55, with the highest attainable score being 65. Interestingly, the highest scores for this domain were seen in the metro health

86

departments. This may be due to their governance structure and that they have a bit more autonomy. This may also indicate that regional sites do not feel they have full support to implement QI or that they have only implemented QI processes in certain areas or departments. Studies have found that in order to successfully implement QI processes organization-wide, an organization needs both internal and external support to take on this endeavor (Davis, 2010). While there may be a drive externally to move towards a performance improvement platform, expounding on the importance of QI to leadership within each site is equally important. Results from the qualitative questions suggest that the majority of respondents see QI as a way to improve patient services and programs within the health department. However, these questions revealed contrasting results when compared to the sites‟ QI Maturity score. Although QI Maturity scores indicated moderate to high levels of QI sophistication, when asked to describe specific QI processes in their health department, many respondents cited using quality assurance practices. This finding indicates a lack of knowledge or awareness as to what constitutes as formal QI. While quality improvement is a continuous process that focuses on improving processes and systems, quality assurance measures compliance against certain standards (NACCHO, 2013a). It is necessary to highlight the importance of both QA and QI in order to continually improve health department processes and performance. It is evident that each site is well-versed in QA practices, and to further enhance those skills additional QI training is warranted. Aim 3 Public health accreditation readiness scores were assigned to each site based on its responses to QI Maturity Tool items and how those items match PHAB measures within domain nine. A site could receive up to 29 “readiness” points, with a higher score indicating an increased

87

readiness for accreditation. A score of 21 indicates that the site has met at least 75% of the measures in domain nine. Readiness scores for this sample ranged from 10 to 24, with variability among each site. Due to the variability of responses and sample sizes for each site, the score assigned may not be a true indication of a site‟s readiness for accreditation. The readiness score is based on domain scores that specifically assess QI processes. However, qualitative data revealed that most sites were engaged in processes centered on QA, thus limiting the value and assessment capabilities of the readiness score. Study Limitations Sample Size Although this study had a fairly high response rate of 55%, the distribution of responses from across the state varied. There were no respondents from Shelby County and only one respondent each from Davidson and Hamilton counties. Bivariate analyses revealed no statistical differences between regions and metros in QI Maturity, but this may be due to lack of statistical power of the metro sample (n=29) as compared to regions (n=80). The investigator encountered issues with sample selection and follow-up during the survey period. The study design dictated that the investigator could not contact potential respondents directly and relied on a state appointed sponsor to develop a list of the requested positions and handle follow-up contact. This process may have inadvertently limited the sample size of the study specifically in regards to the sample size for each site because the investigator was limited on the number of reminder emails and follow-up attempts that could be made. Self-selection was also a limitation of this study. Although the study was made available to employees across the state, only 55% of potential respondents chose to complete the survey.

88

Previous studies have found that individuals with strong opinions or feelings on a subject matter are most likely to participate in a study, thus biasing results. Social Desirability Because the initial and follow-up emails were sent to potential respondents by a state employee, this could have led to social desirability responding on the survey items, explaining the lack of variation in responses. Although, respondents were told that all data would be reported in aggregate form, 16 respondents chose not to report their health department site. This may indicate a belief that respondents felt they were reporting directly to TDH instead of an outside observer. This is further evidenced by some of the responses to the qualitative questions. For instance, one respondent cited that motivation for implementing QI was, “to be good stewards of tax payer money.” Research Engagement This was the first time that the state engaged in a statewide research project led by a doctoral student. Therefore, in order to pursue this relationship a system structure had to be developed before the study was approved. Having an already established relationship with academic institutions and a system in place for receiving research requests will help limit delays and other issues for future researchers. Recommendations and Future Research This study serves as a pilot to assess QI status and accreditation readiness across the state. Future studies may examine facilitators and barriers to QI uptake with an increased focus on qualitative data in the form of interviews or focus groups, especially in those states emerging in QI.

89

The results of this study indicate that health department sites are highly engaged in QA activities across the state. In order to enhance current processes and further align with PHAB standards, future training should focus on education and hands-on application of QI processes. While this study could provide SHAs with a method for assessing QI across in LHDs, the QI Maturity Tool may not be as useful in states that have a burgeoning QI environment. Contribution to Public Health This study adds to the growing body of literature on quality improvement and public health accreditation. This tool can be used to assess the current status of QI in a variety of health department settings and easily disseminated in appropriate intervals to assess growth and change. This tool could also be useful in identifying deficiencies in training and resources among LHDs within a state. The method for cross walking the QI Maturity Tool to PHAB may be used for other survey tools and domains to assess overall readiness for public health accreditation, not just in regard to quality improvement. This study helped establish a research relationship with the state that may lead to potential collaborations in the form of practice-based research projects, student field placements, academic health departments, and the development of a research agenda. Conclusion Despite the lack of statistical significance on some outcomes, potentially due to low power, this study provides SHAs with an easy and appropriate way to assess current QI levels in their respective LHDs. This study highlights a learning and education opportunity for the state to further enhance performance outcomes. Results from this study can be used to identify areas of improvement in order to allocate appropriate resources to those sites demonstrating a lower level of QI sophistication as compared to their counterparts.

90

The method used for gauging accreditation readiness can be easily duplicated in other public health service areas, providing SHAs and LHDs with an easy method for determining if they have the capacity in place to move forward with accreditation. Overall, this study provides SHAs and LHDs with a baseline measure to appropriately address inconsistencies in QI engagement.

91

REFERENCES

American Public Health Association (APHA). (2012). Quality improvement in public health: It works! Washington, DC: American Public Health Association. Association of State and Territorial Health Officials (ASTHO). (2011). ASTHO profile of state public health: Volume two. Arlington, VA: ASTHO. Association of State and Territorial Health Officials (ASTHO). (2012). State public health agency classification: understanding the relationship between state and local public health. Arlington, VA: ASTHO Baker, S. L., Beitsch, L., Landrum, L. B., & Head, R. (2007). The role of performance management and quality improvement in a national voluntary public health accreditation system. Journal of Public Health Management and Practice : JPHMP, 13(4), 427-429. doi:10.1097/01.PHH.0000278039.46518.45 Barton, P. E. (2010). Issues in quality care. Understanding the U.S. health services system (4th ed., pp. 449). Chicago, IL: Health Administration Press and Association of University Programs in Health Administration. Beitsch, L. M., Leep, C., Shah, G., Brooks, R. G., & Pestronk, R. M. (2010). Quality improvement in local health departments: Results of the NACCHO 2008 survey. Journal of Public Health Management and Practice: JPHMP, 16(1), 49-54. doi:10.1097/PHH.0b013e3181bedd0c; 10.1097/PHH.0b013e3181bedd0c Beitsch, L. M., Rider, N. L., Joly, B. M., Leep, C., & Polyak, G. (2013). Driving a public health culture of quality: How far down the highway have local health departments traveled? Journal of Public Health Management and Practice: JPHMP, 19(6), 569-574. doi:10.1097/PHH.0b013e31828e25cf Berman, E. M., Milakovich, M. E., & West, J. P. (1996). Implementing TQM in state public health agencies. Journal of Health and Human Services Administration, 19(2), 182-205. Best, M., & Neuhauser, D. (2006). Walter A. Shewhart, 1924, and the Hawthorne factory. Quality & Safety in Health Care, 15(2), 142-143. doi:10.1136/qshc.2006.018093 Best, M., & Neuhauser, D. (2011). Did a cowboy rodeo champion create the best theory of quality improvement? Malcolm Baldrige and his award. BMJ Quality & Safety, 20(5), 465468. doi:10.1136/bmjqs.2010.050872; 10.1136/bmjqs.2010.050872 Centers for Disease Control and Prevention (CDC). (2013). The public health system and the 10 essential public health services. Retrieved October 17, 2013, from http://www.cdc.gov/nphpsp/essentialservices.html

92

Cornett, A., Thomas, M., Davis, M. V., Mahanna, E., Cordova, A., Herring, C., . . . Randolph, G. D. (2012). Early evaluation results from a statewide quality improvement training program for local public health departments in North Carolina. Journal of Public Health Management and Practice : JPHMP, 18(1), 43-51. doi:10.1097/PHH.0b013e31822d2e07; 10.1097/PHH.0b013e31822d2e07 Corso, L. C., Lenaway, D., Beitsch, L. M., Landrum, L. B., & Deutsch, H. (2010). The national public health performance standards: Driving quality improvement in public health systems. Journal of Public Health Management and Practice : JPHMP, 16(1), 19-23. doi:10.1097/PHH.0b013e3181c02800; 10.1097/PHH.0b013e3181c02800 Davis, M. V. (2010). Opportunities to advance quality improvement in public health. Journal of Public Health Management and Practice : JPHMP, 16(1), 8-10. doi:10.1097/PHH.0b013e3181c027e6; 10.1097/PHH.0b013e3181c027e6 Davis, M. V., Vincus, A., Eggers, M., Mahanna, E., Riley, W., Joly, B., . . . Bowling, M. J. (2012). Effectiveness of public health quality improvement training approaches: Application, application, application. Journal of Public Health Management and Practice : JPHMP, 18(1), E1-7. doi:10.1097/PHH.0b013e3182249505; 10.1097/PHH.0b013e3182249505 Dean, J. W., & Bowen, D. E. (1994). Management theory and total quality: Improving research and practice through theory development. Academy of Management Review, 19(3), 392. doi:10.5465/AMR.1994.9412271803 DelliFraine, J. L., Langabeer, J. R.,2nd, & Nembhard, I. M. (2010). Assessing the evidence of Six Sigma and Lean in the health care industry. Quality Management in Health Care, 19(3), 211-225. doi:10.1097/QMH.0b013e3181eb140e; 10.1097/QMH.0b013e3181eb140e Derose, S. F., Schuster, M. A., Fielding, J. E., & Asch, S. M. (2002). Public health quality measurement: Concepts and challenges. Annual Review of Public Health, 23, 1-21. doi:10.1146/annurev.publhealth.23.092601.095644 Dever, G. E. A. (1997). Improving outcomes in public health practice: Strategy and methods. Gaithersburg, MD: Aspen Dilley, J. A., Bekemeier, B., & Harris, J. R. (2012). Quality improvement interventions in public health systems: A systematic review. American Journal of Preventive Medicine, 42(5 Suppl 1), S58-71. doi:10.1016/j.amepre.2012.01.022; 10.1016/j.amepre.2012.01.022 Eastman Chemical Company. (2013). Lean. Unpublished material. Gillen, S. M., McKeever, J., Edwards, K. F., & Thielen, L. (2010). Promoting quality improvement and achieving measurable change: The lead states initiative. Journal of Public Health Management and Practice: JPHMP, 16(1), 55-60. doi:10.1097/PHH.0b013e3181bedb5d; 10.1097/PHH.0b013e3181bedb5d 93

Gorenflo, G., & Moran, J. W. (2010). The ABCs of PDCA. Unpublished manuscript. Harrison, L. M., Shook, E. D., Harris, G., Lea, C. S., Cornett, A., & Randolph, G. D. (2012). Applying the model for improvement in a local health department: Quality improvement as an effective approach in navigating the changing landscape of public health practice in buncombe county, north carolina. Journal of Public Health Management and Practice: JPHMP, 18(1), 19-26. doi:10.1097/PHH.0b013e31822de37c; 10.1097/PHH.0b013e31822de37c Hyde, J. K., & Shortell, S. M. (2012). The structure and organization of local and state public health agencies in the U.S.: A systematic review. American Journal of Preventive Medicine, 42(5 Suppl 1), S29-41. doi:10.1016/j.amepre.2012.01.021; 10.1016/j.amepre.2012.01.021 Institute of Medicine. (2003). The future of the public's health in the 21st century. No. RA445.F885 2003). Washington, DC: National Academies Press. Joly, B., Booth, M., Mittal, P., & Zhang, Y. (2013). Using the QI maturity tool to classify agencies along a continuum. Frontiers in Public Health Services and Systems Research, 2(3), 1. Joly, B. M., Booth, M., Mittal, P., & Shaler, G. (2012b). Measuring quality improvement in public health: The development and psychometric testing of a QI maturity tool. Evaluation & the Health Professions, 35(2), 119-147. doi:10.1177/0163278711433065; 10.1177/0163278711433065 Joly, B. M., Booth, M., Shaler, G., & Mittal, P. (2012a). Assessing quality improvement in local health departments: Results from the multi-state learning collaborative. Journal of Public Health Management and Practice: JPHMP, 18(1), 79-86. doi:10.1097/PHH.0b013e31823788da; 10.1097/PHH.0b013e31823788da Joly, B. M., Polyak, G., Davis, M. V., Brewster, J., Tremain, B., Raevsky, C., & Beitsch, L. M. (2007). Linking accreditation and public health outcomes: A logic model approach. Journal of Public Health Management and Practice: JPHMP, 13(4), 349-356. doi:10.1097/01.PHH.0000278027.56820.7e Kaluzny, A. D., McLaughlin, C. P., & Simpson, K. (1992). Applying total quality management concepts to public health organizations. Public Health Reports (Washington, D.C.: 1974), 107(3), 257-264. Kritchevsky, S. B., & Simmons, B. P. (1991). Continuous quality improvement. concepts and applications for physician care. JAMA: The Journal of the American Medical Association, 266(13), 1817-1823.

94

Leep, C., Beitsch, L. M., Gorenflo, G., Solomon, J., & Brooks, R. G. (2009). Quality improvement in local health departments: Progress, pitfalls, and potential. Journal of Public Health Management and Practice: JPHMP, 15(6), 494-502. doi:10.1097/PHH.0b013e3181aab5ca; 10.1097/PHH.0b013e3181aab5ca LIFEPATH. (2012). Workforce needs assessment. Johnson City, TN: College of Public Health, East Tennessee State University. Madamala, K., Sellers, K., Beitsch, L. M., Pearsol, J., & Jarris, P. (2012). Quality improvement and accreditation readiness in state public health agencies. Journal of Public Health Management and Practice: JPHMP, 18(1), 9-18. doi:10.1097/PHH.0b013e3182367d91; 10.1097/PHH.0b013e3182367d91 Madamala, K., Sellers, K., Pearsol, J., Dickey, M., & Jarris, P. E. (2010). State landscape in public health planning and quality improvement: Results of the ASTHO survey. Journal of Public Health Management and Practice: JPHMP, 16(1), 32-38. doi:10.1097/PHH.0b013e3181c029cc; 10.1097/PHH.0b013e3181c029cc Mays, G. P., Hatzell, T., Kaluzny, A. D., & Halverson, P. K. (1999). CQI in public health organizations. In C. P. McLaughlin, & A. D. Kaluzny (Eds.), Continuous quality improvement in health care: Theory, implementation, and applications (2nd ed., pp. 360). Gaithersburg, MD: Aspen McLaughlin, C. P., & Kaluzny, A. D. (1999). Defining qaulity improvement: Past, present, and future. In C. P. McLaughlin, & A. D. Kaluzny (Eds.), Continuous quality improvement in health care: Theory, implementation, and applications (2nd ed., pp. 3-33). Gaithersburg, MD: Aspen Mullins, H. (2013). Baldrige, PHAB, or PPHR: What are these and where do we start? Unpublished material. National Association of County and City Health Officials (NACCHO). (2011). 2010 national profile of local health departments. Washington, DC: NACCHO. National Association of County and City Health Officials (NACCHO). (2013a). Roadmap to a culture of quality improvement: A guide to leadership and success in local health departments. Retrieved October 25, 2013, from http://qiroadmap.org/assess/ National Association of County and City Health Officials (NACCHO). (2013b). 2013 national profile of local health departments. Washington, DC: NACCHO. National Institute of Standards and Technology (NIST). (2012). Baldrige by sector: Health care. Retrieved October 5, 2013, from http://www.nist.gov/baldrige/enter/health_care.cfm Nofal, A., Omaim, N., & Zairi, M. (2005). TQM: Theoretical insights, part 1. International Journal of Applied Quality Management, 2(2), 5. 95

Public Health Accreditation Board (PHAB). (2013). PHAB standards and measures: Version 1.5. Alexandria, VA: PHAB. Public Health Accreditation Board (PHAB). (2013). What is accreditation? Retrieved October 7, 2013, from http://www.phaboard.org/accreditation-overview/what-is-accreditation/ Riley, W., Parsons, H., McCoy, K., Burns, D., Anderson, D., Lee, S., & Sainfort, F. (2009). Introducing quality improvement methods into local public health departments: Structured evaluation of a statewide pilot project. Health Services Research, 44(5 Pt 2), 1863-1879. doi:10.1111/j.1475-6773.2009.01012.x; 10.1111/j.1475-6773.2009.01012.x Riley, W. J., Bender, K., & Lownik, E. (2012). Public health department accreditation implementation: Transforming public health department performance. American Journal of Public Health, 102(2), 237-242. doi:10.2105/AJPH.2011.300375; 10.2105/AJPH.2011.300375 Riley, W. J., Lownik, E. M., Scutchfield, F. D., Mays, G. P., Corso, L. C., & Beitsch, L. M. (2012). Public health department accreditation: Setting the research agenda. American Journal of Preventive Medicine, 42(3), 263-271. doi:10.1016/j.amepre.2011.10.021; 10.1016/j.amepre.2011.10.021 Riley, W. J., Moran, J. W., Corso, L. C., Beitsch, L. M., Bialek, R., & Cofsky, A. (2010). Defining quality improvement in public health. Journal of Public Health Management and Practice: JPHMP, 16(1), 5-7. doi:10.1097/PHH.0b013e3181bedb49; 10.1097/PHH.0b013e3181bedb49 Robert Wood Johnson Foundation (RWJF). (2010). Multistate learning collaborative: A midcourse report. Princeton, NJ: Robert Wood Johnson Foundation. Russo, P. (2007). Accreditation of public health agencies: A means, not an end. Journal of Public Health Management and Practice: JPHMP, 13(4), 329-331. doi:10.1097/01.PHH.0000278022.18702.22 Saunders, R. R., & Saunders, J. L. (1994). W. Edwards Deming, quality analysis, and total behavior management. The Behavior Analyst / MABA, 17(1), 115-125. Teich, S. T., & Faddoul, F. F. (2013). Lean management-the journey from Toyota to healthcare. Rambam Maimonides Medical Journal, 4(2), e0007. doi:10.5041/RMMJ.10107; 10.5041/RMMJ.10107 Tennessee Center for Performance Excellence (TNCPE). (2008). What we do. Retrieved October 5, 2013, from http://www.tncpe.org/what_we_do/index.php Tennessee Department of Health. (2013). What we do. Retrieved October 12, 2013, from http://health.state.tn.us/whatwedo.htm

96

Tews, D. S., Sherry, M. K., Butler, J. A., & Martin, A. (2008). Embracing quality in local public health: Michigan's quality improvement guidebook. Princeton, New Jersey: Robert Wood Johsnon Foundation. Thomsett, M. C. (2005). Getting started in six sigma. Hoboken, NJ: John Wiley & Sons Toussaint, J. S., & Berry, L. L. (2013). The promise of lean in health care. Mayo Clinic Proceedings.Mayo Clinic, 88(1), 74-82. doi:10.1016/j.mayocp.2012.07.025; 10.1016/j.mayocp.2012.07.025 Turnock, B. J., & Handler, A. S. (1997). From measuring to improving public health practice. Annual Review of Public Health, 18, 261-282. doi:10.1146/annurev.publhealth.18.1.261 U.S. Department of Health and Human Services (HHS), Health Resources and Services Administration (HRSA). (2011). Quality improvement. Washington, DC:U.S. Department of Health and Human Services. U.S. Department of Health and Human Services (HHS), Office of Disease Prevention and Health Promotion. (2013). Healthy people 2020. Retrieved 9/20, 2013, from http://healthypeople.gov/2020/topicsobjectives2020/objectiveslist.aspx?topicId=35 Yeager, V. A., Menachemi, N., Ginter, P. M., Sen, B. P., Savage, G. T., & Beitsch, L. M. (2013). Environmental factors and quality improvement in county and local health departments. Journal of Public Health Management and Practice: JPHMP, 19(3), 240-249. doi:10.1097/PHH.0b013e31825f74b2; 10.1097/PHH.0b013e31825f74b2

97

APPENDICES APPENDIX A: PHAB Domains and Standards

Domain 1: Conduct and disseminate assessments focused on population health status and public health issues facing the community

Standard 1.1: Participate in or Conduct a Collaborative Process Resulting in a Comprehensive Community Health Assessment Standard 1.2: Collect and Maintain Reliable, Comparable, and Valid Data That Provide Information on Conditions of Public Health Importance and On the Health Status of the Population Standard 1.3: Analyze Public Health Data to Identify Trends in Health Problems, Environmental Public Health Hazards, and Social and Economic Factors That Affect the Public‟s Health Standard 1.4: Provide and Use the Results of Health Data Analysis to Develop Recommendations Regarding Public Health Policy, Processes, Programs, or Interventions Standard 2.1: Conduct Timely Investigations of Health Problems and Environmental Public Health Hazards

Domain 2: Investigate health problems and environmental public health hazards to protect the community

Domain 3: Inform and educate about public health issues and functions Domain 4: Engage with

Standard 2.2: Contain/Mitigate Health Problems and Environmental Public Health Hazards Standard 2.3: Ensure Access to Laboratory and Epidemiologic/Environmental Public Health Expertise and Capacity to Investigate and Contain/Mitigate Public Health Problems and Environmental Public Health Hazards Standard 2.4: Maintain a Plan with Policies and Procedures for Urgent and Non-Urgent Communications Standard 3.1: Provide Health Education and Health Promotion Policies, Programs, Processes, and Interventions to Support Prevention and Wellness Standard 3.2: Provide Information on Public Health Issues and Public Health Functions Through Multiple Methods to a Variety of Audiences Standard 4.1: Engage with the Public Health System and the Community in Identifying and Addressing 98

the community to identify and address health problems

Domain 5: Develop public health policies and plans

Health Problems Through Collaborative Processes Standard 4.2: Promote the Community‟s Understanding of and Support for Policies and Strategies That will Improve the Public‟s Health Standard 5.1: Serve As a Primary and Expert Resource for Establishing and Maintaining Public Health Policies, Practices, and Capacity Standard 5.2: Conduct a Comprehensive Planning Process Resulting in a Tribal/State/Community Health Improvement Plan Standard 5.3: Develop and Implement a Health Department Organizational Strategic Plan Standard 5.4: Maintain an All Hazards Emergency Operations Plan Standard 6.1: Review Existing Laws and Work with Governing Entities and Elected/Appointed Officials to Update as Needed

Domain 6: Enforce public health laws

Standard 6.2: Educate Individuals and Organizations On the Meaning, Purpose, and Benefit of Public Health Laws and How to Comply Standard 6.3: Conduct and Monitor Public Health Enforcement Activities and Coordinate Notification of Violations among Appropriate Agencies

Domain 7: Promote strategies to improve access to health care services Domain 8: Maintain a competent public health workforce Domain 9: Evaluate and continuously

Standard 7.1: Assess Health Care Capacity and Access to Health Care Services Standard 7.2: Identify and Implement Strategies to Improve Access to Health Care Services Standard 8.1: Encourage the Development of a Sufficient Number of Qualified Public Health Workers Standard 8.2: Ensure a Competent Workforce through the Assessment of Staff Competencies, the Provision of Individual Training and Professional Development, and the Provision of a Supportive Work Environment. Standard 9.1: Use a Performance Management System to Monitor Achievement of Organizational Objectives 99

improve health department processes, programs, and interventions

Standard 9.2: Develop and Implement Quality Improvement Processes Integrated Into Organizational Practice, Programs, Processes, and Interventions

Standard 10.1: Identify and Use the Best Available Evidence for Making Informed Public Health Practice Decisions

Domain 10: Contribute to and apply the Standard 10.2: evidence base Promote Understanding and Use of Research Results, Evaluations, and Evidence-based of public health Practices With Appropriate Audiences Domain 11: Maintain administrative and management capacity

Domain 12: Maintain capacity to engage the public health governing entity

Standard 11.1: Develop and Maintain an Operational Infrastructure to Support the Performance of Public Health Functions Standard 11.2: Establish Effective Financial Management Systems Standard 12.1: Maintain Current Operational Definitions and Statements of the Public Health Roles, Responsibilities, and Authorities Standard 12.2: Provide Information to the Governing Entity Regarding Public Health and the Official Responsibilities of the Health Department and of the Governing Entity Standard 12.3: Encourage the Governing Entity‟s Engagement In the Public Health Department‟s Overall Obligations and Responsibilities

Created by author based on “PHAB Standards and Measures Version 1.5.” By Public Health Accreditation Board, 2013.

100

APPENDIX B: Measures The purpose of this study is to assess the current status of quality improvement (QI) processes in selected rural and metro health departments across the state of Tennessee. By completing this survey you are agreeing to be a voluntary participant in this research study. Please answer each question to the best of your ability; there are no right or wrong answers. You may choose to skip any questions that you are not comfortable answering. No identifying information will be used in summarizing the results from this study. If you have any questions regarding this study please feel free to contact the Principal Investigator, Ms. Christian L. Williams at 423-794-7193 or [email protected] You may also contact East Tennessee State University Institutional Review Board at 423-439-6002. Thank you for your participation! Instructions: For the purpose of this survey, having a formal QI process is described as an organization that has: 1) integrated QI into the agency strategic and operational plans, 2) formed a QI council that oversees the implementation of a detailed plan to ensure QI throughout the LHD, and 3) commonly uses data for problem-solving and decision-making. Informal or Ad hoc QI can be described as practicing discrete QI efforts in isolated instances throughout the LHD, without consistent use of data or alignment with the steps in a formal QI process. Key Terms: The term ‘leaders’ refers to leadership within your health department such as the Regional Director, County Director, or Medical Director. The term ‘staff’ refers to any personnel that reports directly to health department leadership.

101

Domain and Survey Items Domain #1: QI Organizational Culture 1. Leaders (e.g. board, senior management team) of my public health agency are receptive to new ideas for improving agency programs, services, and outcomes. 2. The impetus for improving quality in my public health agency is largely driven by an internal (comes from Director or other leadership member) desire to make our services and outcomes better. 3. The board and/or management team of my public health agency work together for common goals. 4. Staff consult with, and help, one another to solve problems. 5. Staff members are routinely asked to contribute to decisions at my public health agency. Domain #2: QI Capacity and Competency 6. The leaders of my public health agency are trained in basic methods for evaluating and improving quality, such as Plan-Do-Study-Act. 7. Staff at my public health agency who provide public health services are trained in basic methods for evaluating and improving quality, such as Plan-Do-Study-Act. 8. Many individuals responsible for programs and services in my public health agency have the skills needed to assess the quality of their program and services. 9. My public health agency has objective measures for determining the quality of many programs and services. 10. Many individuals responsible for programs and services at my public health agency routinely use systematic methods to understand the root causes of problems. 11. Many individuals responsible for

Strongly Disagree

Disagree

Neutral

Agree

Strongly Agree

I Don’t Know

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

102

programs and services at my public health agency routinely use best or promising practices when selecting interventions for improving quality. 12. Programs and services are continuously evaluated to see if they are working as intended and are effective. 13. The quality of many programs and services in my agency is routinely monitored. 14. My public health agency has a designated QI officer. 15. My public health agency has a QI council, committee, or team. 16. My public health agency has a QI plan. Domain #3: QI Alignment and Spread 17. Job descriptions for many individuals responsible for programs and services at my public health agency include specific responsibilities related to measuring and improving quality. 18. Agency staff is aware of external QI expertise to help measure and improve quality. 19. Staff members at all levels participate in QI efforts. 20. Customer satisfaction information is routinely used by many individuals responsible for programs and services in my public health agency. 21. Good ideas for measuring and improving quality in one program or service usually are adopted by other programs or services in my public health agency. 22. Accurate and timely data are available for program managers to evaluate the quality of their services on an ongoing basis. 23. Improving quality is well integrated into the way many individuals responsible for programs and services work in my public health agency. 24. Many individuals responsible for programs and services in my agency have the authority to change

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

103

25.

26. 27. 28. 29.

practices or influence policy to improve services within their areas of responsibility. When trying to facilitate change, staff has the authority to work within and across program boundaries. Spending time and resources on QI is worth the effort. The key decision makers in my agency believe QI is very important. Using QI approaches will impact the health of my community. Public health agency staff and stakeholders will notice changes in programs and services as a result of our QI efforts.

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

O

Demographic Questions: What health department site do you represent? (Site can be selected from a drop down menu) Please select your position: (Presented in a Drop Down Menu)            

Regional Director Assistant Regional Director County Director (Rural HDs Only) Medical Director Nursing Director Clinical Director Primary Care Director Program Director QI Director Accreditation Coordinator Personnel Officer Public Information Officer

Additional Questions regarding QI: 1. What formal quality improvement processes does your health department currently use? a. Lean b. PDCA Cycle c. Baldrige d. Six Sigma e. N/A f. Other, Please Describe:

104

2. What is your motivation for using formal QI processes in your health department? (Open Answer) 3. What outcomes do you expect to see in your health department from using formal QI processes? (Open Answer) 4. What informal QI processes/activities does your health department use? 5. Please describe one process/project in which QI concepts were employed. (Open Answer) 6. Please describe one positive outcome that you have seen in your health department from the use of QI. (Open Answer) 7. Please describe one negative outcome that you have seen in your health department from the use of QI. (Open Answer) 8. Is your health department planning to apply for Public Health Accreditation? 9. Is your health department planning to apply for a Tennessee Center for Performance Excellence (TNCPE) award? 10. Does your health department have an academic health department (AHD) or work with an AHD?

105

APPENDIX C: Cross walk of QI Maturity Tool to PHAB Measures Domain 1 - QI Organizational Culture Survey Item Description PHAB Measure Number 1 Leaders (e.g. board, senior management team) of my public health 9.1.1 A agency are receptive to new ideas for improving agency programs, services, and outcomes.

PHAB Measure Description and Purpose Staff at all organizational levels engaged in establishing and/or updating a performance management system. The purpose of this measure is to assess the health department’s engagement of leadership and staff in developing, establishing, using, and updating a performance management system for the organization. Performance management policy/system. The purpose of this measure is to assess the health department’s adoption of a department-wide performance management system.

2

The impetus for improving quality in my public health agency is largely driven by an internal (comes from Director or other leadership member) desire to make our services and outcomes better.

9.1.2 A

3

The board and/or management team of my public health agency work together for common goals.

9.1.1 A

Staff at all organizational levels engaged in establishing and/or updating a performance management system. The purpose of this measure is to assess the health department’s engagement of leadership and staff in developing, establishing, using, and updating a performance management system for the organization.

4

Staff consult with, and help, one another to solve problems.

9.2.2 A

Implemented quality improvement activities. The purpose of this measure is to assess the health department's use of quality improvement to improve processes, programs, and interventions.

5

Staff members are routinely asked to contribute to decisions at my 9.1.1 A public health agency.

Staff at all organizational levels engaged in establishing and/or updating a performance management system. The purpose of this measure is to assess the health department’s engagement of leadership and staff in developing, establishing, using, and updating a performance management system for the organization.

Domain 2- QI Capacity and Competency Survey Item Description PHAB Measure PHAB Measure Description and Purpose Number 6 The leaders of my public health agency are trained in basic methods 9.1.5 A Opportunities provided to staff for involvement in the for evaluating and improving quality, such as Plan-Do-Study-Act. department’s performance management. The purpose of this measure is to assess the health department’s support to expand and enhance performance management capacity in the department. 7

Staff at my public health agency who provide public health services 9.1.5 A are trained in basic methods for evaluating and improving quality, such as Plan-Do-Study-Act.

Opportunities provided to staff for involvement in the department’s performance management. The purpose of this measure is to assess the health department’s support to expand and enhance performance management capacity in the department.

8

Many individuals responsible for programs and services in my 9.1.5 A public health agency have the skills needed to assess the quality of their program and services.

Opportunities provided to staff for involvement in the department’s performance management. The purpose of this measure is to assess the health department’s support to expand and enhance performance management capacity in the department.

9

My public health agency has objective measures for determining the quality of many programs and services.

9.1.3 A: Implemented performance management system. The purpose of this measure is to assess the health department’s management practices for assessing performance and identifying and managing opportunities for improvement. 9.2.1 A: Established quality improvement program based on organizational policies and direction. The purpose of this measure is to assess the health department’s efforts to develop a quality improvement program that is integrated into all programmatic and operational aspects of the organization.

9.1.3 A 9.2.1 A

106

Domain 2- QI Capacity and Competency Survey Item Description Number 10 Many individuals responsible for programs and services at my public health agency routinely use systematic methods to understand the root causes of problems.

PHAB Measure PHAB Measure Description and Purpose 9.2.2 A

Implemented quality improvement activities. The purpose of this measure is to assess the health department's use of quality improvement to improve processes, programs, and interventions.

11

Many individuals responsible for programs and services at my 9.2.2 A public health agency routinely use best or promising practices when selecting interventions for improving quality.

Implemented quality improvement activities. The purpose of this measure is to assess the health department's use of quality improvement to improve processes, programs, and interventions.

12

Programs and services are continuously evaluated to see if they are 9.1.3 A working as intended and are effective.

Implemented performance management system. The purpose of this measure is to assess the health department’s management practices for assessing performance and identifying and managing opportunities for improvement.

13

The quality of many programs and services in my agency is routinely 9.1.3 A monitored.

Implemented performance management system. The purpose of this measure is to assess the health department’s management practices for assessing performance and identifying and managing opportunities for improvement.

14

My public health agency has a designated QI officer.

9.2.1 A

Established quality improvement program based on organizational policies and direction. The purpose of this measure is to assess the health department’s efforts to develop a quality improvement program that is integrated into all programmatic and operational aspects of the organization.

15

My public health agency has a QI council, committee, or team.

9.1.3 A 9.2.1 A

16

My public health agency has a QI plan.

9.2.1 A

9.1.3 A: Implemented performance management system. The purpose of this measure is to assess the health department’s management practices for assessing performance and identifying and managing opportunities for improvement. 9.2.1 A: Established quality improvement program based on organizational policies and direction. The purpose of this measure is to assess the health department’s efforts to develop a quality improvement program that is integrated into all programmatic and operational aspects of the organization. Established quality improvement program based on organizational policies and direction. The purpose of this measure is to assess the health department’s efforts to develop a quality improvement program that is integrated into all programmatic and operational aspects of the organization.

Domain 3- Alignment and Spread Survey Item Description PHAB Measure Number 17 Job descriptions for many individuals responsible for programs and 9.2.2 A services at my public health agency include specific responsibilities related to measuring and improving quality.

PHAB Measure Description and Purpose

18

NA

NA

9.1.1 A

Staff at all organizational levels engaged in establishing and/or updating a performance management system. The purpose of this measure is to assess the health department’s engagement of leadership and staff in developing, establishing, using, and updating a performance management system for the organization.

19

Agency staff is aware of external QI expertise to help measure and improve quality. Staff members at all levels participate in QI efforts.

107

Implemented quality improvement activities. The purpose of this measure is to assess the health department's use of quality improvement to improve processes, programs, and interventions.

Domain 3- Alignment and Spread Survey Item Description Number 20 Customer satisfaction information is routinely used by many individuals responsible for programs and services in my public health agency.

PHAB Measure PHAB Measure Description and Purpose

21

Good ideas for measuring and improving quality in one program or service usually are adopted by other programs or services in my public health agency.

9.1.2 A

22

Accurate and timely data are available for program managers to evaluate the quality of their services on an ongoing basis.

9.1.3 A

Implemented performance management system. The purpose of this measure is to assess the health department’s management practices for assessing performance and identifying and managing opportunities for improvement.

23

Improving quality is well integrated into the way many individuals responsible for programs and services work in my public health agency.

9.2.1 A

Performance management policy/system. The purpose of this measure is to assess the health department’s adoption of a department-wide performance management system.

24

Many individuals responsible for programs and services in my 9.1.1 A agency have the authority to change practices or influence policy to improve services within their areas of responsibility.

25

When trying to facilitate change, staff has the authority to work within and across program boundaries.

9.1.1 A

26

Spending time and resources on QI is worth the effort.

NA

27

The key decision makers in my agency believe QI is very important. 9.1.1 A

Staff at all organizational levels engaged in establishing and/or updating a performance management system. The purpose of this measure is to assess the health department’s engagement of leadership and staff in developing, establishing, using, and updating a performance management system for the organization.

28

Using QI approaches will impact the health of my community.

NA

NA

29

Public health agency staff and stakeholders will notice changes in programs and services as a result of our QI efforts.

9.1.4 A

Implemented systematic process for assessing customer satisfaction with health department services. The purpose of this measure is to assess the health department’s process for measuring the quality of customer relationships and service.

9.1.4 A 9.2.1 A

108

9.1.4 A:Implemented systematic process for assessing customer satisfaction with health department services. The purpose of this measure is to assess the health department’s process for measuring the quality of customer relationships and service. 9.2.1 A: Established quality improvement program based on organizational policies and direction. The purpose of this measure is to assess the health department’s efforts to develop a quality improvement program that is integrated into all programmatic and operational aspects of the organization. Performance management policy/system. The purpose of this measure is to assess the health department’s adoption of a department-wide performance management system.

Staff at all organizational levels engaged in establishing and/or updating a performance management system. The purpose of this measure is to assess the health department’s engagement of leadership and staff in developing, establishing, using, and updating a performance management system for the organization. Staff at all organizational levels engaged in establishing and/or updating a performance management system. The purpose of this measure is to assess the health department’s engagement of leadership and staff in developing, establishing, using, and updating a performance management system for the organization. NA

VITA CHRISTIAN L. WILLIAMS Education:

DrPH Community and Behavioral Health, East Tennessee State University College of Public Health, August 2014. M.P.H. Health Services Management and Policy, East Tennessee State University College of Public Health, August 2009. B.S.P.H. Community and Behavioral Health, East Tennessee State University College of Public Health, December 2007.

Experience:

Academic Health Department Coordinator, East Tennessee State University, College of Public Health, October 2012-present Graduate Assistant, East Tennessee State University, College of Public Health, August 2011-October 2012. Referral Specialist, Mountain States Medical Group; Johnson City, TN, July 2010-August 2011. Administrative Assistant, Mountain States Medical Group; Johnson City, TN, July 2009-July 2010. Graduate Assistant, East Tennessee State University, College of Public Health, January 2008-August 2009.

Publications:

Brooks, B, Blackley, D, Masters, P, May, S, Mayes, S, Williams, C, Pack, RP, (2014) Developing an Academic Health Department in Northeast Tennessee: A Sustainable Approach Through Student Leadership. Forthcoming in: Journal of Public Health Management and Practice. Mayes, G, Honeycutt, B, May, AS, Williams, CL, Mullins, H. (2013). 2012 Community Health Status Assessment: Sullivan County, Tennessee. Retrieved From: http://www.sullivanhealth.org/images/Health_Assessment/Final%20Version%20S ullivan%20County%20CHSA%20Nov%2022%202013.pdf

Posters:

Williams, C.L., Ward, R.K., McKeehan T.L., Southerland, J. (2013, November). Examining Healthy Weight Promotion and Support for Overweight/Obese Students in Appalachian Tennessee High Schools. Poster presented at: 2013 Annual Scientific Meeting of The Obesity Society, Atlanta, GA. Slawson, DL, Dalton, WT III, Wang, L, Littleton, MA, Lowe, E, Mozen, DM, Schetzina, KE, Stoots, JM, Williams, CL, Southerland, J, McKeehan, TL, Wu, T. 109

(2012, November). Design and Methods for an Intervention Utilizing Peer Facilitators to reduce Adolescent Obesity: Team Up for Healthy Living. Poster presented at: The Obesity Society Annual Meeting, San Antonio, TX. Williams, C.L., McKeehan, T.L. (2012, October). Utilizing Theory of Planned Behavior to Guide Interventions Aimed at Reducing Adolescent Obesity: Team Up for Healthy Living. PowerPoint presented at 6th Annual Southern Obesity Summit, Charlotte, NC Williams, C.L., Slawson, D.L., Dalton, W.T., Wang, L., Littleton, M.A., Lowe, E., Mozen, D.M., Schetzina, K.E., Stoots, J.M., Southerland, J., McKeehan, T.L., Wu, T. (2012, May). Design and Methods for An Intervention Utilizing Peer Facilitators to Reduce Adolescent Obesity: Team Up for Healthy Living. Poster presented at TPHA‟s East Tennessee Grand Division meeting, Knoxville, TN Williams, C.L., Slawson, D.L., Dalton, W.T., Wang, L., Littleton, M.A., Lowe, E., Mozen, D.M., Schetzina, K.E., Stoots, J.M., Southerland, J., McKeehan, T.L., Wu, T. (2012, April). Design and Methods for an Intervention Utilizing Peer Facilitators to Reduce Adolescent Obesity: Team Up for Healthy Living. Poster presented at the Appalachian Student Research Forum, Johnson City, TN

Organizations:

Chair, College of Public Health Career Counseling Advisory Group (January 2014-Present) Co-Chair/Coordinator, College of Public Health DrPH Workshops (September 2013-Present) College of Public Health Recruitment Taskforce (September 2013-Present) Sullivan County School Health Advisory Council (January 2013-Present) Regional Education and Action Coalition for Health (REACH!) (December 2012Present) Interim Coordinator, Sullivan County Health Council (October 2012-Present) College of Public Health Academic Excellence Workgroup (September 2012-May 2013) Secretary, College of Public Health Student Council (September 2012-December 2013) Graduate and Professional Student Association (September 2012-Present) American Cancer Society (May 2012-Present) Public Relations and Marketing Chair, Farmers Market at ETSU (November 2011-December 2012)

110

Loading...

Assessing Health Department Readiness for Public Health

East Tennessee State University Digital Commons @ East Tennessee State University Electronic Theses and Dissertations 8-2014 Assessing Health Depar...

2MB Sizes 2 Downloads 19 Views

Recommend Documents

Department of Public Health - Oral Health Program
Listings of My Health LA (MHLA) Dental. Note: The Los Angeles County Department of Public Health does not provide direct

Public Health Interventions - Minnesota Department of Health
This framework, known as the “intervention model,” defines the scope of public health nursing practice by type of in

Screening and Assessing - Minnesota Department of Health
Feb 2, 2012 - products-tables.pdf. • Trade Names (CDC). Table lists many vaccine products that are (or have been) used

Public Health Career Possibilities | Department of Public Health
While the following list of organizations is not comprehensive, it may be a helpful starting point for finding work in p

Wheel of Public Health Interventions - Minnesota Department of Health
some are sad. All 34 stories provide good opportunities to investigate how the. Intervention Wheel was applied. They are

AIR & HUMAN HEALTH Department of Public Health - ppt video
SO2, H2S, HF, HCl, CO2, NO, NO2, O3 Debu-debu logam : Seng, timah, Arsen, Silica Debu-debu kapas : debu asbes Tepunsari

Environmental Health Risk Assessment—Guidelines for assessing
Table A1: Calculation of pathogen reductions for a source water containing 1 organism per litre of Cryptosporidium, Rota

nurse protocols - Georgia Department of Public Health
Nov 14, 2011 - NURSE PROTOCOLS. FOR. REGISTERED PROFESSIONAL. NURSES. IN PUBLIC HEALTH. FOR 2012. Georgia Department of

Minnesota Public Health Information Network - Minnesota Department
Minnesota Public Health. Information Network. (MN-PHIN). Roadmap and Recommendations for Strategic Action. Report to the

Chapter 368a - Department Of Public Health
(12) re complainant's right to access information and records re complaints filed with department on or after October 1,