Wachter, McClellan offer takes on measuring quality

Two of the nation’s leading experts on healthcare quality— Dr. Bob Wachter, professor and chief of the division of hospital medicine at UCSF Medical Center in San Francisco, and Dr. Mark McClellan, senior fellow and director of the Health Care Innovation and Value Initiative at the Brookings Institution and former administrator of the CMS— called for new ways of measuring clinical quality and outcomes during Modern Healthcare’s second annual Virtual Conference on Quality and Patient Safety on June 18. The following are edited excerpts from a plenary panel moderated by editorial programs manager Maureen McKinney.

“We are not very good yet at measuring the real patients who we actually see, those who have multiple comorbidities.”

DR. BOB WACHTER

Dr. Bob Wachter on “feeding the measurement beast”: I had the privilege to spend the day at Boeing and fly a 777 simulator. One of the things I learned is the degree to which they try to protect the pilots from too many measurement requirements, because they know they can get in the way of them focusing on their work. Bob Myers, Boeing’s chief flight deck engineer, a very interesting guy, said to me, “Airlines are always asking the pilots, ‘Can you just document what time you took off, how much gas you had when you started, and then, did you have any passenger complaints?’ So the pilots spend a fair amount of time on a computer or their iPad documenting things. But they don’t do that stuff below 10,000 feet.”

So I said, “Well, we’re constantly doing that stuff below 10,000 feet in terms of documentation in healthcare,” and Bob said, “That’s the difference. When you’re in the OR or with the patient, you’re below 10,000 feet, and you shouldn’t be doing that stuff.”

So the degree to which quality measurement has created an extraordinary burden for clinicians who have to enter all of that stuff in our records while we’re trying to take care of patients, I think it’s been vastly underestimated by the promulgators of quality measurements. It’s a very, very important issue.

Wachter on the need for measures that

reflect complexity: We are not very good yet at measuring the real patients who we actually see, those who have multiple comorbidities. We’re getting decent at measuring quality for the patient who happens to have just a myocardial infarction or just a stroke. But the real world problem is much more complex than that.

Wachter on diagnostic errors: Measures, of course, tend to focus us on the things that we can measure and therefore, by necessity, focus away from things that might be equally or more important but that we can’t measure very easily. We have no idea how to measure diagnostic errors. We’re decent at measuring medication errors, surgical errors and healthcare-associated infections, so of course these get a lot of attention in a world driven by measurement, and diagnostic errors get virtually no attention in the same world despite the fact that they are equally, if not more important as a safety hazard.

In today’s environment of transparency and pay-forperformance, a hospital can look great by giving patients with pneumonia the right antibiotics; giving heart failure patients ACE inhibitors and giving heart attack patients aspirin, even if they got every single diagnosis wrong.

Wachter on opportunities in quality measurement: One hope is that automated measures will flow directly from patient care. I think we’ve got a lot of work to do on this, but one can certainly see a day where this is possible.

It would also be healthy to move from process or structural measures to outcome measures, but not if we can’t risk-adjust those measures, and if we can’t tell whether patients really are older or sicker or more complex in a lot of different ways. I think our ability to do that will get better over time and it’s already getting better as

the science improves. Policymakers who manage this ecosystem need to get better at looking at life from the perspective of the measured, not just of the measurer, or of the beneficiaries of the measure. You can have 10 different quality programs all asking for different measures, and from where they sit, what they’re asking for is reasonable, but from the standpoint of the practicing physician taking care of a sick patient, it’s undoable. We have to be able to take on that perspective as we think through making the measurement process better. I think we’re going to get there, but we’ve had a somewhat rocky start.

Dr. Mark McClellan on promising signs in quality measurement: Measurement is far from perfect, but it is getting better. We’ve gone from measures of a limited number of never events and some process-of-care measures that can be calculated from claims or billing data, to increasingly measures that are based on clinical information, with an aim to do a much better job at tracking things like patient-reported functional status, combinations of risk factors for cardiovascular disease and condition-specific outcome measures. This is not easy.

Having better data available at the point of care for supporting patient decisions, making it more readily available for other uses, including quality reporting and payment, and generating measures as a byproduct of care delivery rather than having a separate process— all of that can make it easier to develop evidence on what works for particular kinds of patients and to support performance improvement.

I want to emphasize that while this may seem like a long way off, there are systems being developed now that are intended to do this. Most of the medical specialty societies and a range of private organizations are now developing clinical registry programs that are intended both to help support decision-making and to help provide better evidence on how different kinds of patients do with alternative treatments. Health information exchanges at the regional level are developing more capacity to identify gaps in care, and exchange information to support care delivery and report on performance measures. And the growing number of systems that are providing integrated care, either as ACOs or health plans, are also developing these kinds of capabilities.

“I do think there are some promising opportunities and some good examples of how we can actually get to quality-measurement implementation in a way that fully supports better care and reduces the burden on clinicians.”

DR. MARK MCCLELLAN

McClellan on novel approaches: The Food and Drug Administration has been piloting a system over the last few years called the Sentinel System. What it does is create a system of active and ongoing medical surveillance around safety issues for prescription drugs to develop a way of conducting queries on key issues—in this case questions of drug safety—pulling from data that are used throughout the healthcare system in a distributive fashion. So the FDA hasn’t set up its own data warehouse, but it has collaborated with different private health plans, an increasing number of electronic health record-based systems, as well as integrated systems of care, to come up with standard data models that can be derived, with no additional effort on the part of clinicians, from the data that they are using in actual practice related to their patients’ treatments and subsequent outcomes.

This FDA Sentinel distributed-analysis approach has focused not on solving all of the interoperability problems and all of the terminology problems with electronic data, but it focuses in on some practical issues, some of the most important events, like myocardial infarctions, or rare but serious side effects, and some robust ways of identifying from the diverse data systems out there the types of patients that might have these events based on the drugs they use and their other clinical and patient characteristics. Systems that are used for actual care delivery can produce reliable measures.

McClellan on the future of measurement: There are certainly a lot of problems with quality measurement today. While these challenges are real, the need for moving toward an increasing use of systems that rely on quality measures for payment and other purposes is not going to go away. But I do think there are some promising opportunities and some good examples of how we can actually get to quality-measurement implementation in a way that fully supports better care and reduces the burden on clinicians.

Wachter on the CMS’ new efficiency measure for value-based purchasing: I think there’s no consensus yet. I think there’s a general agreement that we have to shift our focus from just looking at quality and safety and the patient experience, but also driving efficiency at the same time. And yet, I think that effort is really in its infancy. We’re looking at adjusted cost-per-beneficiary. It’s a very early measure, and I suspect that there will be a lot of problems with it. Over time, it will need to get better and we’ll also need to begin looking at appropriateness measures: Did you do too much? Did the patient need that scan? Did the patient need that expensive drug when they could have gotten the cheaper drug? It’s still early in our ability to measure that and measure it fairly.

McClellan on the potential of PCORnet, the Patient-Centered Outcomes Research Institute’s new clinical data research infrastructure: The main focus of PCORI is comparative effectiveness, what treatments lead to better or worse outcomes for particular kinds of patients. PCORnet is confronting these issues of how you develop standard measures of meaningful outcomes from real-world systems of care delivery. This is hard to do for many types of measures, but following the principle of starting somewhere, I think the early PCORnet studies are going to focus on some important clinical outcomes that are relatively easy to measure. That’s going to be one more set of incentives, supports, momentum for trying to get to consistent, reliable ways of measuring meaningful outcomes and doing risk adjustment, and other things that need to go along with it from clinical practice. And because the same kinds of outcome measures that will be important in those comparative effectiveness studies will be important for quality measurement and improvement efforts as well, that’s potentially one added synergy for getting to better measures in practice.