Evaluating the Quality of Health Care 1. Learning Objectives After reviewing this chapter readers should be able to: •
Define quality of health care;
Understand different approaches to assessing quality of health care;
Be aware of examples of structure, process, and outcome measures of care quality; and
Know approaches to developing or selecting measures of care quality for a research project.
2. Introduction Almost everyone would say that they want high quality health care and most people have an intuitive sense of what that means. When one wants to develop a research project related to quality of care, however, one quickly finds what is true in many areas of research; that it is much easier to have a sense of what quality is than it is to develop an operational definition and valid and reliable measures of quality.
Part of this complexity in defining quality of care is that different groups can have very different reasons for measuring quality and hence different measurement criteria and emphases.
For example: •
Clinicians or those who manage and provide clinical care might be interested in evaluating quality so that they can monitor and improve the services they are providing to individual patients.
Regulators may be interested in ensuring that care provided by a health care organization (e.g. health plan or hospital) meets a minimal standard and/or is making credible efforts to improve care quality.
Consumers and other purchasers may be most interested in information that they can use to select clinicians or health care organizations.
Although all of these parties might agree on a definition of high quality care they might select different measures and researchers studying these different areas might have similar variations in emphases.
2. Introduction There is still a tremendous need for more work in measuring and improving the quality of care in the United States. For example, Schuster et al., (1998) reviewed a large number of studies and found that only 50% of patients studied received recommended preventative care, only 70% received recommended acute care, 30% received contraindicated acute care, only 60% received recommended acute care, and 20% received contraindicated chronic care. In a subsequent study, McGlynn et al., (2003) found that participants only received about 55 percent of recommended care.
In spite of the pronouncement of many that “the United States has the best health care in the world” studies consistently find that care is far from optimal.
3. Defining Quality of Care There are many definitions of quality of care, but the Institute of Medicine (IOM) has proposed one that captures well the features of many other definitions and that has received wide acceptance (Institute of Medicine, 2001; Lohr & Committee to Design a Strategy for Quality Review and Assurance in Medicare, 1990): “The degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge.” As compelling as that definition is, it does not provide much guidance to a researcher interested in developing a measure or set of measures. A subsequent IOM report specified seven aims of a high quality medical care system that are more specific (Institute of Medicine, 2001): •
Safe – avoiding injuries to patients from the care that is supposed to help them.
Effective – providing services based on scientific knowledge to all who could benefit and refraining from providing services to those not likely to benefit (avoiding underuse and overuse).
Patient-centered – providing care that is respectful of and responsive to individual patient preferences, needs, and values and ensuring that patient values guide all clinical decisions.
Timely – reducing waits and sometimes harmful delays for both those who receive and those who give care.
Efficient – avoiding waste, in particular waste of equipment, supplies, ideas, and energy.
Equitable – providing care that does not vary in quality because of personal characteristics, such as gender, ethnicity, geographic location, and socioeconomic status.
3. Defining Quality of Care These aims describe two related, but distinct types of excellence; technical and interpersonal (Donabedian, 1965, 1988). Interpersonal excellence refers to care that meets the information, emotional, and physical needs of patients in a way that is consistent with their preferences and expectations. Another term for this type of care is "patient-centered care" (Cleary, P. D., Edgman-Levitan, et al., 1991). One important aspect of interpersonal care is patient involvement in decision making (Barry, Fowler, Mulley, Henderson, & Wennberg, 1995; Braddock, Edwards, Hasenberg, & et, 1999; President’s Commission for the Study of Ethical Problems in Medicine and Biomedical and Behavioral Research, 1982; Sepucha & Mulley Jr, 2009) http://www.salzburgglobal.org/current/news.cfm?IDMedia=59422 . It is important to distinguish between excellence of interpersonal care and patient satisfaction. Patient satisfaction is commonly measured and many consider it an indicator of medical care quality. However, patients may be satisfied with poor quality care (Cleary, P. D. & McNeil, 1988).
Thus, it is important to specify interpersonal aspects of high quality care and ask patients to report about those experiences. It may also be •
useful to rate the extent to which care met patient expectations, but it is important to recognize that high satisfaction does not necessarily imply high quality.
Exercise 1: For each of the following, decide which question is mainly a rating and which is mainly a report about care quality.
4. Types of Quality of Care Measures Although the aims above suggest how one might measure quality, there are multiple approaches to measuring different aspects of quality. One of the first comprehensive works that focused on quality of care was published in a series of three books by Avedis Donabedian (Donabedian, 1980, 1982, 1985). A subsequent article (Donabedian, 1965) summarized that work. Donabedian proposed that one could assess whether high quality care is provided by examining the structure of the setting in which care is provided, by measuring the actual process of care, and/or by assessing what the outcomes of care are. Structure refers to the characteristics of the setting in which care takes place. Measures of the setting used might include characteristics of: •
Physicians and hospitals (e.g., a physician's specialty or the ownership of a hospital);
Policies related to care delivery.
Increasingly, we view structure as not just the way clinics and hospitals are organized and operated, but by the policies they have in place that affect care quality. For example, processes for monitoring and promoting quality, incentives for high quality care, etc. can have an influence on how well care is delivered. A motivation for focusing on structure is the premise that the setting can be a strong determinant of care quality and given the proper system, good care will follow. For example, one would expect care to be of higher quality when all staff are clear about their roles and responsibilities, when there are strategies for monitoring adherence to recommended procedures, and there are systematic approaches to continuously improving care quality.
4. Types of Quality of Care Measures Process measures assess whether a patient received what is known to be good care. They can refer to anything that is done as part of the encounter between a physician or another health care professional and a patient, including interpersonal processes, such as providing information and emotional support, as well as involving patients in decisions in a way that is consistent with their preferences, etc. Outcomes refer to a patient's health status or change in health status (e.g., an improvement in symptoms or mobility) resulting from the medical care received. This includes intended outcomes, such as the relief of pain and unintended outcomes, such as complications. Although the term “outcomes” is sometimes used loosely to refer to results such as mammography rates, such measures are actually process measures in the Donabedian sense. There is also a category of measurement called intermediate outcomes. This includes measures like Hemoglobin A1c levels for people with diabetes and blood pressure measurements. These intermediate outcomes are often closely related to other health outcomes.
If quality-of-care criteria based on structural, process, or intermediate outcomes are to be credible, it must be demonstrated that variations in the attribute they measure lead to differences in health status outcomes.
For outcomes to be used as quality of care measures, they must reflect, or be responsive to, variations in the care being assessed (Deyo, Diehr, & Patrick, 1991; Terwee, Dekker, Wiersinga, Prummel, & Bossuyt, 2003). For example, we know that taking blood pressures is necessary for monitoring how well blood pressure is controlled and that controlling blood pressure reduces the probability of heart attacks, strokes and other bad outcomes. We also know that certain outcomes, such as death after being treated in a hospital for a heart attack is related to the quality of care provided.
Exercise 2: Drag and place each of the following examples into the appropriate category.
5. Outcome Measures There is a long history of using outcomes to assess care quality. The use of outcome data to evaluate health care dates back more than 150 years. In the 1830s, a physician named PierreCharles-Alexandre Louis started a group in Paris that discussed the use of statistics to examine patterns of medical care and outcomes. In 1838, a physician from that group named George Norris returned to the United States and examined the survival of patients who had an amputation. In subsequent work Norris compared surgery outcomes at the Pennsylvania Hospital with those of hospitals in other cities and counties (Cassedy, 1984). During the same period, Florence Nightingale developed innovative ways of presenting statistics to illustrate seasonal variations in patient mortality in the military field hospital she managed. She later used similar techniques to describe the conditions of medical care in the Crimean War (Bostridge, 2008). Later, Codman used information about outcomes from medical records at the Massachusetts General Hospital in Boston to assess the quality of care provided by different surgeons (Codman, 1914; Neuhauser, 1990). To use medical outcomes as a quality measure, one must usually calculate rates of certain outcomes for a group of patients since outcomes are determined by many factors and thus one usually assesses whether the probability of death, for example is higher or lower for one group compared to another. One could also develop explicit a priori criteria to determine whether the observed results of care are consistent with the outcome predicted by a model that has been validated on the basis of scientific evidence (Brook, McGlynn, & Cleary, 1996). For example, one might assess if the population of patients with diabetes and specific clinical characteristics are better or worse than expected.
5. Outcome Measures Outcomes now have been incorporated into a broad range of health care activities. Physicians providing clinical care routinely ask patients about outcomes to guide their therapy. In clinical research, patients' outcomes provide a measure of the effectiveness of different medical interventions. Outcome measures also have been used in health care organizations and systems to assess quality and guide efforts to improve it. Over the past decade there have been an increasing number of efforts to release information on patients' outcomes publicly. There also have been attempts to provide outcome data on health plans. For example, the National Committee for Quality Assurance (NCQA) reports rates of low birth weight and hospitalization rates for patients with asthma.
An outcome that has received increasing attention in recent years is preventable adverse events. An Institute of Medicine Report (Institute of Medicine, 2000) estimated that as many as 98,000 people die annually from medical errors made in hospitals. That report, which summarized years of careful research on this topic (Leape, 1994) is thought by many to have stimulated the current widespread interest in, and focus on, patient safety.
5. Outcome Measures Most of the efforts to monitor and/or report outcomes systematically have focused on mortality or other outcomes such as in-hospital complications, and physiologic function. However, such measures do not adequately reflect the full range of variations in health affected by care that are important to individuals. It is also important to measure the impact of medical and surgical care on symptoms, functioning, and emotional well-being. These types of outcomes often are referred to collectively as health-related quality of life (HRQL) (Wilson, I. B. & Cleary, 1995). The most commonly used measures of HRQL are short, multi-dimensional, generic measures of HRQL. A widely used measure of HRQL is the RAND 36-item health survey, also known as the SF-36 (Stewart, Hays, & Ware, 1988). The SF-36 is a generic measure of health that includes eight domains (physical functioning, general bodily pain, role limitations due to health problems, role limitations due to personal or emotional problems, emotional well-being, social functioning, and general health perceptions). Although this scale is comprehensive, reliable, and valid, outcomes in many of the domains are not closely linked to process of care for some patient groups. Thus, there also are many condition specific HRQL scales (McDowell & Newell, 1996).
5. Outcome Measures Research over the past decade has yielded HRQL scales that are much shorter, and almost as reliable and valid as the much longer measures that were used earlier (McDowell & Newell, 1996; Stewart, et al., 1988; Ware, Kosinski, & Keller, 1996). As a consequence, HRQL can be collected efficiently using standardized procedures. While such information is being collected, it also is possible to collect information about the salience, importance, and utility of different health states (Tsevat, et al., 1994). In 2004, an initiative was started to create a repository of patient reported outcome measures. The Patient Reported Outcomes Measurement Information System (PROMIS) is a system of measures of patient–reported health status for physical, mental, and social well–being. PROMIS survey measures assess what patients are able to do and how they feel. PROMIS’ measures can be used as primary or secondary endpoints in clinical studies of the effectiveness of treatment.
5. Outcome Measures If outcomes are the ultimate indicator of care quality, why did Donabedian propose other approaches to measuring quality of care and why have so many researchers used structure and process measures? One of the reasons is that many outcomes are often not closely linked to the quality of care, in part because outcomes are affected by many social and clinical factors that are not related to the treatment provided. This is particularly true for measures of HRQL (Wilson, I. B. & Cleary, 1995).
We often do not have the information necessary to identify the most powerful predictors of outcome that are not related to care. Even if we can identify the predictors we are often unsure about how best to specify them.
One of the major factors affecting post-treatment status is pre-treatment status (Cleary, P. D., Greenfield, et al., 1991). Pre-treatment health may be a predictor of post-treatment health and thus need to be a control variable, or the appropriateness of a procedure may be conditional on health status (e.g. the benefits of a certain type of survey are greater than the potential harm only for healthy patients) and thus need to be considered as a stratifying variable. In many situations, however, "pre-treatment" status is impossible to assess because patients are not identified at the beginning of their illness. Links between outcome and process are more likely when the patient group is well defined by medical condition and/or demographic characteristics, when there is a well accepted physiologic, biochemical, or psychological mechanism that links medical intervention with outcome, and when the outcomes are targeted for the medical condition.
5. Outcome Measures Another problem is that the outcome of interest may be too rare to give us a reliable picture of quality for the entity of interest. For example, counting breast cancer deaths might seem like a good way to assess the effectiveness of health plans’ screening and treatment programs for breast cancer, but only about one of every 1,000 women over the age of fifty die in any given year. Thus, screening (versus no screening) might change a group or clinic’s death rate by only three per 10,000 patients a year. It becomes even harder to detect differences when it is not an either/or situation. If rates of screening varied by ten percentage points between clinics or medical groups, the difference in death rates might be only three per 10,000 patients per year (Eddy, 1998). To detect such differences, one would need a very large sample, much larger than might be available for individual providers or even group practices or small health plans. If one wanted to assess the quality of diabetes care using available quality measures, some have estimated that a physician would need to have more than 100 patients with diabetes for the quality measures to have adequate reliability (Hofer, et al., 2000), again a limitation if one wanted to assess the quality of care provided by individual physicians.
Exercise 3: Match each of the following examples with the corresponding limitation of outcomes as quality of care measures.
5. Outcome Measures Often we lack information on the natural history of illnesses that are not treated. This would provide an important comparison against which outcomes for treated patients could be compared. In the case of procedures, measuring outcomes can have an unintended and perverse consequence: performing the procedure on healthier patients may give the appearance of better technical quality because the patients’ outcomes are better if the case-mix models do not adequately adjust for such differences. If a procedure was not indicated for the patient in the first place, the good outcome may not be attributable to the procedure and the patient may have being exposed to unnecessary risk since the procedure may not have been necessary (Dranove, Kessler, McClellan, & Satterthwaite, 2002). Other limitations of outcome measures are that in many situations the most relevant outcome takes a substantial amount of time to become manifest. For example, one of the outcomes of a hip replacement might be how long the replacement lasts and it may take many years to have a good sense of that. A limitation of outcome measures for quality improvement purposes is that even when outcomes are less than optimal, they may not provide insights into why there were poor outcomes and what needs to be changed to yield better outcomes. There are other practical difficulties in assessing outcomes. The data collection systems required to collect such data are not routinely available in clinical settings. When one does develop data collection systems there can be biases in the data. Specifically, for some measures, particularly patient reported ones, patients with worse outcomes may be less likely to respond to surveys. Thus, bias can result if response rates differ across settings.
6. Process Measures Process measures attempt to answer the question ”Did this patient receive the right care,” or “what percent of the time did patients of this type receive the right care?” Such measures are typically developed based on the known relationship between a process and outcomes. For example if one was examining the quality of care received by a patient with diabetes, one might assess whether the patient had undergone an annual funduscopic examination by an ophthalmologist or whether the patient's feet were professionally examined annually (Brook, et al., 1996). Such measures are used because research has demonstrated a link between those processes and important outcomes, such as retinopathy or foot amputations. A nurse or medical-record technician trained in quality assessment could compare what was done to what should have been done, and the result would be expressed as the proportion of times that the criteria were met. Such measures or criteria are typically developed by first identifying the condition of interest, and then synthesizing research evidence to create evidence-based guidelines for clinical care. Once one has identified the part of the medical care process that will be used, one defines patients who are eligible to receive care on the basis of guideline, create criterion to determine which patients received care in accordance with guideline, and divide number who received care in compliance with guideline by number of patients eligible to receive care.
6. Process Measures Researchers increasingly are recognizing that it is not adequate to simply assess individual processes of care, but rather groups or processes, or
An important limitation
“bundles” of activities that need to occur to lead to
on the usefulness of
a better outcome. For example, researchers
process measures is that
attempting to prevent catheter related bloodstream
much care is delivered in
infections learned from prior research that multiple
the absence of
activities, such as hand washing, full barrier
compelling evidence of
precautions, skin antisepsis with chlorhexidine,
avoiding the femoral site during catheter insertion,
there has been extensive
and removing unnecessary catheters are all
work on the
necessary to achieve the best outcomes (Lipitz-
development of evidence
Snyderman, et al., 2011). Similarly, interventions
to prevent ventilator associated pneumonia
(Institute of Medicine,
included a mechanical ventilator “bundle” consisting
2011), the evidence for
of use of semirecumbent positioning, daily
many of the specific
interruption of sedation infusions, and prophylaxis
things that clinicians do
for peptic ulcer disease and deep venous
thrombosis (Berenholtz, et al., 2011).
6. Process Measures Even when there are data supporting the appropriateness and effectiveness of a process or procedure, there often is more than one evidence-supported way to treat a condition. Frequently, for example, different approaches to treatment (e.g. radiation, versus surgery for prostate cancer) are developed and thought to be best by physicians in different specialties. This situation has given rise to a field of research referred to as comparative effectiveness research.
The goal of comparative effectiveness research is to evaluate which of two or more appropriate approaches to treating a condition is better (Institute of Medicine, 2009). That is, although there may be multiple approaches that are considered good quality care, this approach tries to assess which is the best.
It is also important to recognize that for many treatments that are “preference sensitive” whether or not a particular treatment or procedure is appropriate depends on patient preferences (Brook, et al., 1996; Sepucha & Mulley Jr, 2009) Sometimes, processes of care are too complicated for completely explicit criteria. For example, determining when a problem occurred or when an adverse event was preventable, may require some clinical judgment (Sullivan, et al., 2007). Such measures tend to be less reliable and usually provide less compelling evidence than measures with a strong research base, however. Even with the most careful protocols, there inevitably are many sources of variation in the way implicit criteria are implemented.
6. Process Measures Some of the advantages of process measures are that they are very specific, understandable to patients and providers, and in many cases can be easier than outcomes to measure. They also answer the intuitive question a provider might have: “am I doing the right thing for a patient? A related strength is that they are actionable because they indicate what should be changed. They also can be used to make inferences about individual providers. Disadvantages of process measures include the fact that we do not know how many processes of care are related to outcomes. Another shortcoming is related to the fact that sometimes hundreds, if not thousands of things are done in the course of caring for a patient with a complicated condition and it is difficult to develop and use enough measures to form a comprehensive assessment.
7. Structure Measures Probably the main advantage and attractiveness of structure measures is that they are concrete and usually easy to assess. For example, it is relatively easy to determine whether an intensive care unit has a specialty physician available 24 hours a day or if a health plan provides incentives to physicians who meet high standards of care, the training of physicians (Landon, et al., 2003; Landon, et al., 2002), whether a clinic specializes in particular types of care (Wilson, IB., et al., 2005), or the number of procedures performed per year (Hannan, et al., 1997). Structural characteristics that did not receive a great deal of attention when Donabedian did his seminal research include organizational culture, including to the priority that a clinic or hospital gives to quality as well as leadership, policies and procedures for maximizing the quality of care (Berwick, 1996; Institute of Medicine, 2001) The main disadvantage of such measures is that often the association between structure and process and/or structure and outcome are not well-established and developing evidence for such associations is difficult. One of the reasons for this is that the link between structure and process or outcome measures of quality are often very complex (Landon, Wilson, & Cleary, 1998) and consequently weak (Landon, Zaslavsky, Beaulieu, Shaul, & Cleary, 2001). Another weakness is that the most accessible structural variables often lack specificity. For example, one can relatively easily determine if a person is board certified in infectious diseases but it is much harder to develop a measure of the quality of that training or the extent to which the physician uses that knowledge or skills.
8. Comprehensiveness of Measures Available studies indicate that it is difficult to generalize from the quality of care for one set of symptoms or diseases to the quality of care for another set of symptoms or diseases. For example, a study of six medical and surgical interventions at six teaching hospitals showed that the rank ordering of the hospitals' performance in terms of both process and outcome measures differed depending on the intervention.(Cleary, P. D., Greenfield, et al., 1991). Even for a specific condition, the fact that one aspect or care is performed
Unfortunately, developing and
well may not predict the quality of a
using a comprehensive
different aspect of care for the same
assessment, such as the one used
condition (Wilson, .IB., et al., 2007).
by McGlynn et al. (McGlynn, et
Innovative organizations, such
al., 2003) is very complicated and
Intermountain Healthcare and the Boston
expensive. As providers of health
based Institute for Healthcare Improvement
care in the Unites States
are focusing on improving the systems that
increasingly adopt electronic
should support better care across a wide
medical records and continue
range of processes, but to date, it does not
standardizing the way information
appear that such comprehensive
is entered and shared, there
approaches to quality improvement are
undoubtedly will be increasing
widespread enough to allow one to
opportunities to develop more
generalize about quality on the basis of a
comprehensive process measures
few conditions, diagnoses, or symptoms.
of care based on information from
Such generalizations are especially
those systems.(Bates, Ebell,
problematic when different types of medical
Gotlieb, Zapp, & Mullins, 2003;
functions are evaluated, such as screening,
Poon, et al., 2010; Simon, S. R.,
prevention, diagnosis, and treatment. Thus,
R. Kaushal, P.D. Cleary, C.A.
a sound measure of the quality of care must
Jenter, L.A. Volk, E.J. Orav, E.
include separate measures for the care
Burdick, E.G. Poon, D.W. Bates,
provided by different clinicians or
2006; Simon, S. R., et al., 2009)
organizations or even different functions or procedures provided by the same clinician.
9. Selecting Measures Given the numerous considerations summarized above, selecting quality of care measures is a daunting task for someone who is not an expert in this type of assessment. However, by specifying different aspects of the measure requirements, one can narrow down the possibilities considerably.
In our opinion, for the reason summarized above, the assessment of quality should depend much more on process data than on outcome or structure data. Researchers should be aware however, that we recommend the use of measures with a strong evidence base and such evidence is lacking for many process of care that we would like to assess.
The first task is to specify the study design requirements. A major consideration, for example, is the unit of assessment. Although one usually is assessing the care of individuals, it is important to know if one wants to make decisions about the individual, a clinician, organization, or population (e.g. state). The requirements of such measurement tasks are very different. For example, if one is assessing the care of clinicians, there often is a “small numbers problem,” as described above for assessment of the impact of breast cancer screening and diabetes care. That is, a specific clinician may not see enough patients with diabetes during the study period to yield a reliable measure of his or her performance. If one is evaluating the care system, one must determine, for example, whether the focus is hospital care, ambulatory care, or the combined effect of care provides by multiple providers.
9. Selecting Measures There are, of course, exceptions to this, such as the method used to compare differences in outcome after coronary-artery bypass surgery. There has been extensive research on the best way to adjust statistically for case-mix differences when assessing the outcome of such surgery, there is strong evidence of the link between the quality of care and survival, death after coronaryartery bypass is common enough to be used as a measure of differences in the quality of care, and differences in mortality among institutions or among groups of patients receiving coronaryartery bypass surgery can be assessed relatively soon after the surgery. Furthermore, because numerous aspects of the care that are also difficult to measure influence postsurgical mortality (e.g., the physician's skill in the operating room), differences in survival may reflect differences in quality, or case selection, not revealed by a limited number of process assessments and case-mix adjustors. As another example, we might prefer to use adjusted outcomes to evaluate the potential effect of a policy change designed to reduce health care expenditures.
There recently has been a substantial increase in funding for comparative effectiveness research (Institute of Medicine, 2009). Many comparative effectiveness studies, which will focus on specific procedures in defined populations or systems changes, will appropriately use outcome measures.
9. Selecting Measures For any of these applications, measures based on explicit criteria derived from strong evidence are more reliable and valid. After the scientific literature has been reviewed, specific criteria of the quality of care are enumerated and categorized by the level of evidence (randomized controlled trials, observational studies, or expert opinion) supporting them. To be useful, these criteria must be as clinically detailed as possible and must cover diverse topics, such as what drugs were used in a patient with asthma, whether smoking cessation was recommended for smokers with stable angina, or whether the appropriate candidates were offered coronary-artery bypass surgery.
9. Selecting Measures After deciding what method of quality assessment should be used, the next step is to determine the appropriate sources of data. Data used in quality assessment are obtained from diverse sources, such as records maintained by insurance companies to reimburse physicians, clinical records maintained by health care professionals, pharmacy and laboratory data, and surveys of patients. Each source of data produces a different view of the quality of care (Wilson, A. & McDonald, 1994). For example, suppose we asked a patient who had been told she had breast cancer whether the doctor had discussed options for removing the cancer. Because of the emotional impact of the news, the patient might not remember whether the doctor had discussed therapeutic options. The doctor might not have recorded that discussion in the patient's medical record, but an audiotape would have captured the entire conversation.
The answer depends on the purpose of the assessment. If the purpose is to determine whether the patient comprehended the relevant information, then survey data are most appropriate (Cleary, P., 1999; Cleary, P. D., Edgman-Levitan, et al., 1991; Cleary, P. D. & McNeil, 1988).
Finally, if one wants to determine whether one doctor recorded enough information to allow another physician caring for the patient to know what had been done, then the medical record might be the best source of data. Thus, the appropriate source of data for quality assessment depends on the purpose for which the information will be used.
9. Selecting Measures Administrative data (e.g. billing data, patient characteristics) are maintained by health care organizations, private insurers, and public insurance programs, such as Medicare and Medicaid. Data often are coded using standardized codes, representing the patient’s diagnosis (DRGs; ICD) or the procedures conducted and/or billed for (CPT codes). Limitations of administrative include limited specificity, limited detail, and potential bias. Because reimbursement is often based on diagnosis or procedures performed, there can be incentives to use particular codes, often referred to as “up coding.” There are other types of data such as tumor registry data for all patients with cancer, pharmacy and laboratory data, and hospital discharge data. These types of data can provide information that is not available in administrative data (e.g. very specific details about the type of cancer a patient has) and often they can be linked to administrative data to provide more complete information about the patient and/or process of care.
9. Selecting Measures Medical Records maintained by health care providers usually contain detailed information about a patient’s clinical history and current status, and information on test results. Disadvantages are that it usually is very costly to abstract such data from medical records, unless they are in standardized electronic formats. Also documentation can be incomplete and unstandardized. For example, different providers may use different language and characteristics to describe the severity of a patient’s condition. As indicated previously, the expanding use of electronic medical records may transform the way we use medical record information for a variety of purposes, including the assessment of care quality. This is part of a broader social transformation in which enormous quantities of information is being produced in digital form and being made available to different groups and the public at large (King, 2011). Although one still needs to assess carefully the completeness and accuracy of such information, there is the possibility that the standardization of how we collect, store, and transmit such information will fundamentally change the value of such information by making it more accurate, useful, and accessible.
9. Selecting Measures Patient surveys can be used for many purposes in quality studies, but the main uses are to assess outcomes and processes from the patient’s perspective, or the extent to which care is “patient centered.” The outcomes of treatment are central to patients. As a result, many consider what is generally referred to as Health Related Quality of Life (HRQL), along with survival, key criteria of high quality medical care. In the ambulatory setting, these sorts of data are particularly important because ambulatory medical care is often directed at reducing morbidity more than decreasing mortality. An advantage of asking patients directly for information about outcomes is that this type of information is less susceptible to "gaming", or biased reporting to improve reimbursement or apparent performance, by providers. Patients certainly can be influenced by their providers and there are biases inherent in the way patients report information but the risk of "gaming" is lower than for information provided by physicians or nurses.
Exercise 4 The research purpose should drive the decision about what quality of care measures should be used. For each of the following research purposes, decide which measure would be the most appropriate.
10. Examples of Quality of Care Measures The Agency for Health Care Policy and Research has used both literature reviews and expert opinion to establish guidelines for care and quality-of-care criteria. The scientific literature has also been used to develop evidence-based practice guidelines and to evaluate both the appropriateness of use of procedures and the quality of inpatient care received by patients with multiple conditions (Brook, et al., 1996). The Cochrane Centre conducts similar reviews. The National Quality Forum (NQF) is a nonprofit organization with the mission of improving the quality of health care by: building a consensus on national priorities and goals for performance improvement and working in partnership to achieve them; endorsing national consensus standards for measuring and publicly reporting on performance; and promoting national goals through education and outreach programs.
10. Examples of Quality of Care Measures The National Committee for Quality Assurance is a private, not-for-profit organization dedicated to improving health care quality. Since its founding in 1990, NCQA has been a central organization in driving health care improvement. One of the ways in which NCQA has pursued its mission is by working with large employers, policymakers, doctors, patients and health plans to decide what aspect of care quality are most important and developing measures of those aspects of care. It has developed quality standards and performance measures for a broad range of health care entities. These measures and standards are the tools that organizations and individuals can use to identify opportunities for improvement. It has developed and implemented a variety of standards for improving health care quality that include structural characteristics of health care organizations, but one of its most widely known measurement sets is the Healthcare Effectiveness Data and Information Set (HEDIS), a set of measures that are used by more than 90 percent of U.S. health plans to measure performance on important dimensions of care and service.
12. Resources •
The Patient Reported Outcomes Measurement Information System (PROMIS)
The Agency for Health Care Policy and Research (AHRQ)
The Cochrane Centre
The National Quality Forum (NQF)
National Committee on Quality Assurance (NCQA)
13. References Barry, M. J., Fowler, F. J., Jr, Mulley, A. J., Henderson, J. V., & Wennberg, J. E. (1995). Patient reactions to a program designed to facilitate patient participation in treatment decisions for benign prostatic hyperplasia. Med Care. Bates, D. W., Ebell, M., Gotlieb, E., Zapp, J., & Mullins, H. C. (2003). A proposal for electronic medical records in U.S. primary care. J Am Med Informatics Assoc, 10, 1-10. Berenholtz, S., Pham, J. C., Thompson, D. A., Needham, D., Lubomski, L., Hyzy, R., et al. (2011). An intervention to reduce ventilator-associated pneumonia in the ICU. Infect Control Hosp Epidemiol, In Press. Berwick, D. M. (1996). A primer on leading the improvement of systems. BMJ, 312, 619-622. Bostridge, M. (2008). Florence Nightingale. The Making of an Icon. New York, NY: Farrar Straus and Giroux. Braddock, C. H., Edwards, K. A., Hasenberg, N. M., & et, a. (1999). Informed decision making in outpatient practice: time to get back to basics. 282, 2313-2320. Brook, R. H., McGlynn, E. A., & Cleary, P. D. (1996). Quality of health care. Part 2: measuring quality of care. N Engl J Med, 335(13), 966-970. Cassedy, J. H. (1984). American Medicine and Statistical Thinking, 1800-1860. Cambridge, MA: Harvard University Press. Cleary, P. (1999). The increasing importance of patient surveys. Br Med J, 319, 720-721. Cleary, P. D., Edgman-Levitan, S., Roberts, M., Moloney, T. W., McMullen, W., Walker, J. D., et al. (1991). Patients evaluate their hospital care: a national survey. Health Aff, 10(4), 254-267. Cleary, P. D., Greenfield, S., Mulley, A. G., Pauker, S. G., Schroeder, S. A., Wexler, L., et al. (1991). Variations in length of stay and outcomes for six medical and surgical conditions in Massachusetts and California. JAMA, 266(1), 73-79. Cleary, P. D., & McNeil, B. J. (1988). Patient satisfaction as an indicator of quality care. Inquiry, 25(1), 25-36.
Codman, E. A. (1914). The product of a hospital. Surg, Gynecol, Obstet, 18, 491-496. Deyo, R. A., Diehr, P., & Patrick, D. L. (1991). Reproducibility and responsiveness of health status measures statistics and strategies for evaluation. Controlled Clinical Trials, 12(4, Supplement 1), S142-S158. Donabedian, A. (1965). Evaluating the quality of medical care. Milbank Memorial Fund Quart, 44(3), 166-206. Donabedian, A. (1980). Explorations in quality assessment and monitoring. Vol. 1. The definition of quality and approaches to its assessment. Ann Arbor, Mich.: Health Administration Press. Donabedian, A. (1982). Explorations in quality assessment and monitoring. Vol. 2. The criteria and standards of quality. Ann Arbor, Mich.: Health Administration Press. Donabedian, A. (1985). Explorations in quality assessment and monitoring. Vol. 3. The methods and findings of quality assessment and monitoring: an illustrated analysis. Ann Arbor, Mich.: Health Administration Press. Donabedian, A. (1988). The quality of care. How can it be assessed? JAMA, 260(12), 17431748. Dranove, D., Kessler, D., McClellan, M., & Satterthwaite, M. (2002). Is More Information Better? The Effects of 'Report Cards' on Health Care Providers. Boston, MA: NBER. Eddy, D. M. (1998). Performance measurement: problems and solutions. Health Aff, 17(4), 725. Hannan, E. L., Racz, M., Ryan, T. J., McCallister, B. D., Johnson, L. W., Arani, D. T., et al. (1997). Coronary angioplasty volume-outcome relationships for hospitals and cardiologists. JAMA, 277(11), 892-898. Hofer, T. P., Hayward, R. A., Greenfield, S., Wagner, E. H., Kaplan, S. H., & Manning, W. G. (2000). The unreliability of individual physician "report cards" for assessing the costs and quality of care of a chronic disease. JAMA, 281(22), 2098-2105. Institute of Medicine. (2000). To Err is Human. Washington D.C.: National Academy Press.
Institute of Medicine. (2001). Crossing the Quality Chasm: A New Health System for the 21st Century. Washington D.C.: National Academy Press. Institute of Medicine. (2009). Initial National Priorities for Comparative Effectiveness Research. Washington, DC: The National Academies Press. Institute of Medicine. (2011). Clinical Practice Guidelines We Can Trust. Washington, DC: The National Academies Press. King, G. (2011). Ensuring the data-rich future of the social sciences. Science, 331(11 Feb.). Landon, B. E., Wilson, I. B., & Cleary, P. D. (1998). A conceptual model of the effects of health care organizations on the quality of medical care. JAMA, 279(17), 1377-1382. Landon, B. E., Wilson, I. B., Cohn, S. E., Fichtenbaum, C. J., Wong, M. D., Wenger, N. S., et al. (2003). Physician specialization and antiretroviral therapy for HIV. J Gen Intern Med, 18(4), 233-241. Landon, B. E., Wilson, I. B., Wenger, N. S., Cohn, S. E., Fichtenbaum, C. J., Bozzette, S., et al. (2002). Specialty training and specialization among physicians who treat HIV/ AIDS in the United States. J Gen Intern Med, 17, 12-22. Landon, B. E., Zaslavsky, A. M., Beaulieu, N. D., Shaul, J. A., & Cleary, P. D. (2001). Health plan characteristics and consumers' assessments of quality. Health Affairs, 20(2), 274-286. Leape, L. (1994). Error in medicine. JAMA, 272, 1851-1857. Lipitz-Snyderman, A., Steinwachs, D., Needham, D. M., Colantuoni, E., Morlock, L., & Pronovost, P. J. (2011). Impact of a statewide intensive care unit quality improvement initiative on hospital mortality and length of stay: retrospective comparative analysis. 342(d219). Lohr, K., & Committee to Design a Strategy for Quality Review and Assurance in Medicare (Eds.). (1990). Medicare: a strategy for quality assurance, Vol. 1. Washington, DC: IOM, National Academy Press. McDowell, I., & Newell, C. (1996). Measuring Health: A Guide to Rating Scales and Questionnaires. New York: Oxford University Press.
McGlynn, E. A., Asch, S. M., Adams, J., Keesey, J., Hicks, J., DeCristofaro, A., et al. (2003). The quality of health care delivered to adults in the United States. N Engl J Med, 348(26), 26352645. Neuhauser, D. (1990). Ernest Amory Codman, M.D., and end results of medical care. Int J Tech Assess Health Care, 6, 307-325. Poon, E. G., Wright, A., Simon, S. R., Jenter, C. A., Kaushal, R., Volk, L. A., et al. (2010). Relationship between use of electronic health record features and health care quality: Results of a Statewide Survey. Med Care, 48(3), 203-209. President’s Commission for the Study of Ethical Problems in Medicine and Biomedical and Behavioral Research. (1982). Making Health Care Decisions. Wash, DC: Gov’t Printing Office. Schuster, M. A., McGlynn, E. A., & Brook, R. H. (1998). How good is the quality of health care in the United States? Milbank Q, 76(4), 517-563. Sepucha, K., & Mulley Jr, A. G. (2009). A perspective on the patient's role in treatment decisions. Med Care Res Rev, 66, 53S-74S. Simon, S. R., R. Kaushal, P.D. Cleary, C.A. Jenter, L.A. Volk, E.J. Orav, E. Burdick, E.G. Poon, D.W. Bates. (2006). How and Why Are Physicians Using Electronic Health Records? A Statewide Survey. Arch Int Med. Simon, S. R., Soran, C. S., Kaushal, R., Jenter, C. A., Volk, L. A., Burdick, E., et al. (2009). Physicians’ usage of key functions in electronic health records from 2005 to 2007: A statewide survey. J Am Med Informatics Assoc, 16(4), 465-470. Stewart, A. L., Hays, R. D., & Ware, J. E. J. (1988). The MOS short-form General Health Survey (SF-36) 1. Conceptual framework and item selection. Med Care, 26, 724-735. Sullivan, A. F., Camargo, C. A., Cleary, P. D., Gordon, J. A., Guadagnoli, E., Kaushal, R., et al. (2007). The National Emergency Department Safety Study: Study rationale and design. Acad Emerg Med, 14(2), 1182-1189. Terwee, C. B., Dekker, F. W., Wiersinga, W. M., Prummel, M. F., & Bossuyt, P. M. M. (2003). On Assessing Responsiveness of Health-Related Quality of Life Instruments: Guidelines for Instrument Evaluation. Quality of Life Research, 12(4), 349-362.
Tsevat, J., Weeks, J. C., Guadagnoli, E., Tosteson, A. N., Mangione, C. M., Pliskin, J. S., et al. (1994). Using health-related quality-of-life information: clinical encounters, clinical trials, and health policy. J Gen Intern Med, 9(10), 576-582. Ware, J. J., Kosinski, M., & Keller, S. (1996). A 12-item short-form health survey. Construction of scales and preliminary tests of reliability and validity. Med Care, 34(3), 220-223. Wilson, A., & McDonald, P. (1994). Comparison of patient questionnaire, medical record, and audio tape in assessment of health promotion in general practice consultations. BMJ, 309, 14831485. Wilson, I., Landon, B., Ding, L., Zaslavsky, A. M., Shapiro, M. F., Bozzette, S. A., et al. (2005). A national study of the relationship of care site HIV specialization to early adoption of highly active antiretroviral therapy. Med Care, 43(1), 12-20. Wilson, I., Landon, B. E., Marsden, P. V., Hirschhorn, L. R., McInnes, K., Ding, L., et al. (2007). Correlations among quality measures in HIV Care in the United States: a cross-sectional study of care sites in 30 states. Br Med J, 335(7629), 1085-1091. Wilson, I. B., & Cleary, P. D. (1995). Linking clinical variables with health-related quality of life. A conceptual model of patient outcomes. JAMA, 273(1), 59-65.
14. Author Biographies Paul D. Cleary, Ph.D. is the Anna M.R. Lauder Professor of Public Health and Dean of the Yale School of Public Health. Dr. Cleary received his Ph.D. in sociology from the University of Wisconsin. He has been a member of the Institute of Medicine since 1994 and a member of the Connecticut Academy for Science and Engineering since 2007. In 1996, he was selected as a distinguished fellow of the Association for Health Services Research, and in 2002, received the Distinguished Investigator Award from the Academy for Health Services Research and Health Policy. Dr. Cleary’s research includes a study of how organizational characteristics affect the costs and quality of care for persons with AIDS; a national evaluation of a continuous quality improvement initiative in clinics providing care to HIV infected individuals; and a study of the long-term impact of patient-centered hospital care. He is Principal Investigator of one of the Consumer Assessment of Healthcare Providers and Systems (CAHPS) projects funded by the Agency for Healthcare Research and Quality (AHRQ) to develop surveys for consumers regarding their health plans and services. He also is Principal Investigator and Director of the Yale Center for Interdisciplinary Research on AIDS (CIRA). Margaret E. O’Kane has served as President of the National Committee for Quality Assurance (NCQA), an independent, nonprofit organization that improves health care quality through measurement, transparency and accountability, since 1990. Under O’Kane’s leadership, NCQA developed the first national system for measuring health plans’ quality performance, the first and most rigorous system for accrediting health plans, and the most widely adopted program for transforming medical practices into patient-centered medical homes. O’Kane has served as co-chair of the National Priorities Partnership and is a board member of the Foundation for Informed Decision Making and the American Board of Medical Specialties. She was elected a member of the Institute of Medicine in 1999 and received the 2009 Picker Institute Individual Award for Excellence in the Advancement of Patient-Centered Care. In 2011, Modern Healthcare named O’Kane one of the Top 25 Women in Healthcare.