Grading grievances
Leapfrog’s safety-score program still divisive
It’s not perfect but it’s as good as it can be, given available data, and it’s certain to improve over time. That’s how Dr. Robert Wachter describes the Hospital Safety Score program. Wachter is one of the experts the Washington-based Leapfrog Group tapped two years ago to help craft the controversial initiative, which relies on publicly available data to assign letter grades—A, B, C, D or F—to acute-care hospitals, based on their records of patient safety.
Leapfrog, an employer-backed not-forprofit group, launched the program a year ago, branding it both as a tool consumers can use when choosing sites of care and as a motivator to get hospitals to step up their improvement efforts.
The grades have drawn strong reactions from hospitals, with many officials calling them unfair characterizations of the safety of the care their organizations deliver.
Drawing from the CMS’ Hospital Compare, the Leapfrog Group’s annual survey and the American Hospital Association’s annual survey, the grading methodology was developed with the help of an eight-member expert panel that includes patient-safety heavy hitters such as Wachter;
Dr. Peter Pronovost, head of the Armstrong Institute for Patient Safety and Quality at Johns Hopkins Medicine, Baltimore; and Dr. Arnold Milstein, professor of medicine at Stanford University, Palo Alto, Calif.
“We thought there was enough data out there in the public domain to take our best shot, based on today’s understanding, with the recognition that it would be incomplete and the measures would get better,” Wachter, chief of the division of hospital medicine at UCSF Medical Center, San Francisco, said of the panel. “It is better to have some information out there rather than none.”
On May 8, Leapfrog released its third round of safety grades, showing relatively little change since the last update in November. Of the 2,514 hospitals covered in the latest update, 780 received an A, 638 received a B, 932 received a C, 148 received a D and 16 received an F. Nearly three quarters of hospitals received the same letter grade as last time, while only 1.9% saw a big change—up or down two or more grades—in their scores, a Leapfrog official said during a conference call.
Hospital officials’ response to the latest wave of grades was mixed.
Roxanne Moster, a spokeswoman for 466bed Ronald Reagan UCLA Medical Center, Los Angeles, said in a statement that the hospital was “disappointed but not surprised” by its most recent grade: a D, up from the F it received in November. She contended the low grade did “not reflect in any way the quality of care” the hospital provides, and said she expects to see higher marks in the future.
Michael Sanborn, president of 115-bed Baylor Medical Center at Carrollton (Texas), which saw its letter grade jump from a C to an A, credited the improvement to aggressive quality and patient-safety initiatives, efforts that he said have increased since the hospital was acquired by 11-hospital Baylor Health Care System, Dallas, in 2009.
Daviess Community Hospital, a 48-bed facility in Washington, Ind., went from an F to a C, a two-grade bump that David Bixler, the hospital’s CEO, said reflects close work with Leapfrog to improve adherence to best practices.
Officials at 146-bed Larkin Community Hospital, South Miami, Fla., attributed their D grade in November to issues related to their submission of data for Leapfrog’s annual survey. The hospital worked with Leapfrog to correct those problems, said CEO Sandy Sosa-Guerrero, and received an A this time around. “We always felt that we were an A hospital,” she said.
But while she called the grades valuable, she also acknowledged that they could be misinterpreted by patients. “It looks at a certain amount of very limited quality indicators,” Sosa-Guerrero said. “It’s a good start but it needs to be more inclusive.”
Jonathan Aquino, chief quality officer of 172bed Kern Medical Center, Bakersfield, Calif., whose grade slipped from a D to an F in the latest ratings, said Leapfrog’s score did not align with the hospital’s performance in statewide improvement initiatives. “We truly don’t believe we merit the grade that we got,” he said.
He cited methodological concerns that many other hospitals have pointed to, including Leapfrog’s reliance on older data. The most recent grades used data from 2010 and 2011.
Leah Binder, Leapfrog’s president and CEO, said the group uses the most up-to-date publicly available data, adding that “Leapfrog would be happy to work with hospitals to advocate for CMS to put data out faster.”
She also cited a recently released peerreviewed article in the Journal of Patient Safety, co-authored by many of the members of Leapfrog’s expert panel that reviews in detail the development of the group’s safety-score methodology.
The program bases grades on hospitals’ performance on 26 safety measures, including 15 process and structural measures, such as antibiotics within one hour of surgery, and 11 outcomes measures, such as air embolisms and late-stage pressure ulcers.
This latest round of scores reflects recent changes to the group’s grade-setting methodology, including an effort to better align reporting requirements among different sources. Leapfrog also put safeguards in place to ensure no single measure dominated hospitals’ safety scores, Missy Danforth, senior director of hospital ratings, said during the conference call.
Some of the measures, such as use of computerized physician-order entry, can be determined only if a hospital submits data to Leapfrog, leading some hospitals to argue that the grades are skewed in favor of those who complete the survey.
Wachter, however, said the expert panel worked hard to make the grading system “agnostic about whether hospitals submitted data to Leapfrog.” He also said the newly published study illustrates the lengths the panel went to in order to choose the right measures and give them appropriate weight.
“This is just one tool patients can use to get a picture of safety and quality, but it is a methodology that quite a few credible experts in the field said is as good as it can possibly be,” he said.