Specialty hospitals have an edge under CMS’ star ratings, analysis shows
Leaders at Northwestern Memorial Hospital pay close attention to the CMS’ hospital star ratings.
When the latest batch arrived in December, the quality team spent some time evaluating the underlying data, particularly how the academic medical center performed on the 57 quality measures considered for the rating compared with their peers, said Dr. Karl Bilimoria, vice president of quality at Northwestern Medicine, which includes the hospital.
“The star ratings come from the single-largest payer—the government—so they have a lot of credibility with the public,” he said.
Chicago-based Northwestern Memorial received four stars in December, unusually good compared with the 171 other major teaching hospitals included in CMS’ analysis.
Bilimoria admitted that even though he and his team care about the ratings, it doesn’t mean he believes they are particularly accurate or helpful to consumers. “The star ratings certainly cover important topics that we should focus on to improve quality,” he said, “but it’s how they put together the measures that might actually mislead patients and doctors if they use the data to identify where to go for care.”
The main problem is that under CMS’ methodology, Northwestern, a major teaching hospital with a Level 1 trauma center that’s located in a diverse city, is being compared with hospitals with such different patient populations and services that they can’t report all 57 measures, skewing the results for hospitals like Northwestern that can report all the measures.
Bilimoria’s concerns play out in a study of star-ratings data by consulting firm Sullivan, Cotter and Associates and Modern Healthcare.
The analysis found that most specialty hospitals received five-star ratings—the highest rating a facility can receive—while a small number of major teaching hospitals received the same recognition. Of the 74 specialty hospitals that received a star rating in December, 45—or 61%—received a five-star rating and 16—or 22%—received a four-star rating, according to the analysis. By comparison, of the 172 major teaching hospitals that received a star rating from the CMS, just 15—or 9%—got a five-star rating and 25—or 15%—received four stars.
The analysis also found that specialty hospitals only reported about half of the measures the CMS considers to assign ratings, and the measures reported were weighted higher under the methodology.
Experts and providers say the findings highlight consistent issues with the CMS’ star-ratings methodology. “There is a clear bias here toward the critical-access hospitals and the specialty hospitals and much of it is driven by the methodology in terms of how the measures are aggregated into a single rating,” said Andrew Wilson, research team leader at Altarum’s Center for Value in Health Care, of the results.
Even though specialty hospitals benefit under the methodology, it doesn’t mean they approve.
“We are a five-star hospital, and we don’t think it’s a very good rating system,” said Dr. Catherine MacLean, chief value medical officer of the Hospital for Special Surgery, which provides orthopedic and rheumatology services in New York. “You are comparing apples to oranges.”
Hospitals care deeply about the star ratings even though it’s unknown how often consumers use them. There
haven’t been any studies on how many Medicare beneficiaries use the Hospital Compare site, where the star ratings are featured, and the CMS can’t determine how many Hospital Compare visitors are beneficiaries.
The American Hospital Association, the Federation of American Hospitals and America’s Essential Hospitals all campaigned strongly against publishing the star ratings in 2016. The organizations’ main concern was the methodology, which they characterized as “flawed,” saying it “unfairly” penalizes teaching hospitals or those that serve large poor populations.
Even though hospitals opposed publishing the star ratings, high performers tout their success in public relations campaigns, saying the recognition is an indicator of quality care.
Although the CMS has tweaked the rating system over time, including trying to cluster similar hospitals, it hasn’t touched how it weighs the quality measures used to assess hospitals.
Hospitals can report up to 57 measures to obtain a star rating. These are divided and weighted into seven categories: mortality, safety of care, readmission, patient experience, effectiveness of care, timeliness of care and efficient use of medical imaging. The three outcome groups are each weighted the most at 22% each, as is patient experience. The other categories are each weighted at 4%. To qualify for a star rating, hospitals must report for at least three of the categories, one of which must be an outcome category. A total of 3,692 hospitals received a star rating last December.
“When you are looking at the star ratings, you really aren’t getting as clear of a picture on the performance of the hospitals as you would want to,” said Nancy Foster, vice president for quality and patient safety policy at the AHA. “You are getting this blend of measures that aren’t adequately adjusted … you end up with this strange distortion.”
The CMS disagrees with the critics, arguing that its star-ratings methodology “is not designed to advantage or disadvantage any type of hospital,” a spokesman said in an email, adding, “Our analyses show that hospitals’ star ratings are strongly correlated with the hospital performance on the individual quality measures.”
But the CMS data suggest that specialty hospitals benefited from higher ratings by reporting fewer measures— particularly outcome measures—than teaching hospitals.
Specialty hospitals that received a rating reported 27.2 of the 57 measures on average, according to Sullivan Cotter’s analysis. Comparatively, major teaching hospitals that received a rating reported almost all of the measures, 51.37 of the 57 measures on average.
Specialty hospitals were less likely to have the heavily weighted mortality measure considered in their star rating compared with major teaching hospitals. Data showed the CMS didn’t consider mortality for 71% of the five-star specialty hospitals.
Jeff Softcheck, principal at Sullivan Cotter, emphasized that specialty hospitals don’t report all the measures because their patients don’t present with those conditions. For instance, most specialty hospitals don’t provide care to patients with congestive heart failure or stroke. By comparison, the 172 major teaching hospitals reported all of the measures needed to fulfill the patient experience, readmissions, mortality and safety categories.
Because of how the methodology is set up, hospitals that perform below average on any of the four heavily weighted measures have to perform above average on the other three measures in order to get four stars or higher, Softcheck said.
“If you end up at a major teaching hospital you are really sick … you are seeing multiple specialists, your length of stay is long,” said Dr. Mark Rumans, chief medical officer at Sullivan Cotter. “Compare that to the experience at a specialty hospital, which has lengths of stay of one or two days.”
The data show that 52.5% of the 40 four- and five-star major teaching hospitals performed below average on readmissions. Most made up for their score by performing above average in the mortality, patient experience and safety categories.
Bilimoria at Northwestern suggests that the CMS raise the threshold for the number of measures a hospital must report in order to be included in the dataset. Others argue that the star ratings should display a hospital’s performance on measures by specific condition so it’s more helpful to patients.
Right now, the star ratings can’t tell a patient if the hospital is a good choice for a knee replacement because it combines all “these measures that aren’t correlated with each other—that is the fundamental problem,” said David Nerenz, director of the Center for Health Policy and Health Services Research at the Henry Ford Health System in Detroit.
David Levine, senior vice president of advanced analytics and informatics at Vizient, also criticized the CMS data’s timeliness, as it’s mostly from 2016.
The CMS said that it will continue to look for more opportunities to improve the star ratings and gather input from stakeholders. The AHA’s Foster confirmed that the organization continues to talk to the CMS about ways to improve the star ratings. The CMS is expected to release updated star ratings in July.
Editor’s Note: Sullivan Cotter used its own methodology to categorize hospitals by five types: specialty; critical-access; small or medium community; large community or minor teaching; and major teaching. The firm considers numerous sources to establish the groups, including the AHA Guide.