RAT­ING THE RATERS

EX­PERTS RAISE QUES­TIONS ABOUT HOSPI­TAL RATERS’ AS­SORTED METH­ODS AND LACK OF TRANS­PARENCY

Modern Healthcare - - NEWS - By Sabriya Rice

When ad­min­is­tra­tors and staff at 127-bed Bo­li­var Med­i­cal Cen­ter in Cleve­land, Miss., learned in March that their hospi­tal had scored 11 out of 100, plac­ing it at the bot­tom of a list of more than 2,500 U.S. hos­pi­tals in the 2014 Con­sumer Re­ports Hospi­tal Safety Re­port, they were sur­prised and dis­ap­pointed. That same month, the hospi­tal had earned a Gold Seal of Ap­proval from the Joint Com­mis­sion. And for three years in a row, Bo­li­var had won the Joint Com­mis­sion’s Top Per­former des­ig­na­tion for its use of ev­i­dence-based prac­tices in three care ar­eas. Plus, in spring 2013, it got an “A” hospi­tal safety grade from the Leapfrog Group, a not-for­profit rep­re­sent­ing large em­ploy­ers.

Since 2012, when Bo­li­var’s owner, LifePoint Hos­pi­tals, started par­tic­i­pat­ing in the federal Part­ner­ship for Pa­tients pro­gram, the hospi­tal had achieved sig­nif­i­cant qual­ity-of-care im­prove- ments, said Dr. Rusty Hol­man, LifePoint’s chief med­i­cal of­fi­cer. The hospi­tal saw a 45% drop in mor­tal­ity rates and a 9% re­duc­tion in 30-day read­mis­sions, he said.

Hol­man com­plains that Con­sumer Re­ports used older data that did not re­flect Bo­li­var’s im­proved qual­ity of care. “We ap­plaud the spirit in which rat­ings are in­tended,” he said. But over­all, he said, the pro­lif­er­a­tion of dif­fer­ent hospi­tal rat­ings is com­plex and con­fus­ing for hospi­tal lead­ers. “One can only imag­ine how dizzy­ing it is for con­sumers.”

Many or­ga­ni­za­tions have started pub­lish­ing

hospi­tal per­for­mance mea­sures and re­port cards in re­cent years, grow­ing out of the move­ment for im­proved qual­ity and pa­tient sat­is­fac­tion, lower costs, and greater ac­count­abil­ity and trans­parency. Among the or­ga­ni­za­tions pub­lish­ing these rat­ings and mea­sures are govern­ment agencies, news or­ga­ni­za­tions, health­care ac­cred­i­ta­tion and qual­ity groups, and com­pa­nies and not-for-prof­its fo­cused on trans­parency. The emer­gence of these re­views has put pres­sure on hospi­tal lead­ers to do what’s nec­es­sary to im­prove their scores.

But the var­i­ous re­ports use sig­nif­i­cantly dif­fer­ent method­olo­gies and have dif­fer­ent ar­eas of fo­cus, of­ten pro­duc­ing sharply dif­fer­ent rat­ings for the same hos­pi­tals dur­ing the same time pe­riod. Some hospi­tal lead­ers say this makes it more dif­fi­cult to know which ar­eas to pri­or­i­tize to im­prove their qual­ity of care and rank­ings.

Crit­i­cism erupts

Some rat­ings groups do not dis­close their method­ol­ogy on the grounds that it’s pro­pri­etary. That has prompted crit­i­cism from hos­pi­tals and in­de­pen­dent ex­perts. “It can take a con­sid­er­able amount of dig­ging on some sites to find the met­rics and how much they count to­ward the fi­nal rat­ing,” said Dr. Ashish Jha, a Har­vard Univer­sity pro­fes­sor of health pol­icy who serves on the ad­vi­sory com­mit­tee for the Leapfrog Group. “If a rat­ing pro­gram isn’t will­ing to make its method­ol­ogy com­pletely trans­par­ent, then no one should use it.”

Lately, some hospi­tal groups have started push­ing back by rat­ing the raters. Even so, some ex­perts say hos­pi­tals should pre­pare for even more rat­ings scru­tiny as con­sumers, fac­ing higher cost-shar­ing in their health plans, in­creas­ingly shop and com­pare health­care providers on qual­ity, ser­vice and price.

“The sci­ence of per­for­mance man­age­ment is still in the early stages and we have not all come to­gether and agreed on an eval­u­at­ing sys­tem. It’s a chaotic pic­ture,” said Dr. John Santa, med­i­cal di­rec­tor of Con­sumer Re­ports Health. “But if we want to get to knowl­edge, we have to go through that stage of con­fu­sion.”

Rea­son­able people dis­agree on what mea­sures are most im­por­tant to in­clude, which makes for sig­nif­i­cant dif­fer­ences in the var­i­ous rat­ings, Jha said. One prob­lem with that, though, is that hos­pi­tals can cherry-pick fa­vor­able rat­ings for mar­ket­ing pur­poses, whether or not those rat­ings have much va­lid­ity. “Any­one who wants to dodge ac­count­abil­ity can hang their hat on some ob­scure rat­ing that was good,” he said.

Ex­ec­u­tives am­biva­lent

A re­cent Mod­ern Health­care on­line sur­vey of read­ers in­di­cated that health­care ex­ec­u­tives are am­biva­lent about hospi­tal rat­ings ef­forts. Of more than 230 re­spon­dents, 53% said their fa­cil­ity had re­ceived a poor rat­ing from at least one rat­ings or­ga­ni­za­tion while re­ceiv­ing a high rat­ing on sim­i­lar mea­sures dur­ing the same time pe­riod from an­other group. More than 81% said there are too many groups pub­lish­ing rat­ings. Still, most re­spon­dents said the rat­ings were mod­er­ately valu­able.

“I would say that the mar­ket is crowded,” said Ali­cia Daugh­erty, prac­tice man­ager of re­search and in­sights at the Ad­vi­sory Board Co., which in Jan­uary 2013 pub­lished an anal­y­sis of the or­ga­ni­za­tions that pub­lish hospi­tal qual­ity data. “There are so many out there that pa­tients and or­ga­ni­za­tions have dif­fi­culty dis­tin­guish­ing.”

The Ad­vi­sory Board eval­u­ated 12 rat­ings groups, in­clud­ing in­de­pen­dent or­ga­ni­za­tions such as Leapfrog and To­tal Bench­mark So­lu­tions, news or­ga­ni­za­tions such as U.S. News & World Re­port, govern­ment mea­sures such as CMS’ Hospi­tal Com­pare and the Agency for Health­care Re­search and Qual­ity’s Pa­tient Safety In­di­ca­tors, and ac­cred­i­ta­tion groups such as the Joint Com­mis­sion.

Some groups use a star rat­ing sys­tem, some use a 1 to 100 per­cent­age scale, and oth­ers use an aca­demic-style A to F grad­ing range. The groups also vary on how fre­quently they pub­lish rat­ings, with some is­su­ing re­ports an­nu­ally and oth­ers of­fer­ing more fre­quent up­dates.

The raters rely on data sets from the govern­ment, such as the Medi­care Provider Anal­y­sis and Re­view and the Hospi­tal Con­sumer As­sess­ment of Health­care Providers and Sys­tems. Some cre­ate their own sur­veys and so­licit vol­un­tary re­sponses from the hos­pi­tals. Oth­ers use di­ag­nos­tic and pro­ce­dure cod­ing for spe­cific dis­eases, con­di­tions and ser-

vices. But not all groups dis­close how they weight the var­i­ous qual­ity mea­sures in pro­duc­ing their fi­nal scores. “They have to cre­ate a dis­tinct prod­uct,” Daugh­erty said.

Clear pur­pose state­ment

Now other or­ga­ni­za­tions are be­gin­ning to scru­ti­nize the rat­ings ser­vices to pro­vide hos­pi­tals guid­ance in de­cid­ing which ones they think re­quire at­ten­tion and ac­tion. In March, the As­so­ci­a­tion of Amer­i­can Med­i­cal Col­leges is­sued a set of guid­ing prin­ci­ples it hopes aca­demic med­i­cal cen­ters will use to eval­u­ate qual­ity re­ports.

They ad­vise hos­pi­tals to make sure a rat­ings group of­fers a clear and con­cise pur­pose state­ment, ex­plic­itly de­scribes the in­tended au­di­ence and of­fers trans­par­ent method­ol­ogy. The AAMC’s prin­ci­ples were en­dorsed by the Amer­i­can Hospi­tal As­so­ci­a­tion, Amer­ica’s Es­sen­tial Hos­pi­tals and the Fed­er­a­tion of Amer­i­can Hos­pi­tals, among oth­ers.

The In­formed Pa­tient In­sti­tute, a con­sumer-ori­ented not-for-profit based in Annapolis, Md., has rated the use­ful­ness of about 70 hospi­tal rat­ings, 70 on­line doc­tor rat­ing sites and 60 nurs­ing home re­port cards. The IPI gives each rat­ings group a grade rang­ing from A for “out­stand­ing” to F for “not worth your time,” based on 15 cri­te­ria in­clud­ing time­li­ness of the in­for­ma­tion, pre­sen­ta­tion and ease of use.

The IPI gave the CMS’ Hospi­tal Com­pare, U.S. News & World Re­port’s Amer­ica’s Best Hos­pi­tals, and Leapfrog’s Hospi­tal Safety Score a B, while giv­ing Health­grades and the Joint Com­mis­sion’s Qual­ity Check a C.

Last fall, the Health­care As­so­ci­a­tion of New York State

de­cided to ad­dress frus­tra­tions among its hospi­tal mem­bers that dif­fer­ent rat­ings groups had pub­lished sharply con­tra­dic­tory re­ports about the same hos­pi­tals. For ex­am­ple, one hospi­tal ranked in the top 20 by U.S. News and World Re­port’s Best Hos­pi­tals re­ceived a 49—a be­low-aver­age score—from Con­sumer Re­ports’ Hospi­tal Safety Rat­ing, and gar­nered a B from Leapfrog’s Hospi­tal Safety Score.

“Hos­pi­tals take (rat­ing sites) very se­ri­ously and use them to fig­ure out how to deliver bet­ter care,” said Kathleen Cic­cone, HANYS’ vice pres­i­dent of qual­ity and re­search ini­tia­tives. “But un­less there is some type of stan­dard­ized ap­proach with very trans­par­ent method­ol­ogy, it’s go­ing to be very dif­fi­cult for hos­pi­tals to re­ally ap­ply the rat­ings for the pur­poses of qual­ity im­prove­ment.”

So HANYS cre­ated its own eval­u­a­tion, called the “Re­port on Re­port Cards,” in which the as­so­ci­a­tion rated the raters on a scale of zero to three stars. HANYS gave the Joint Com­mis­sion’s Qual­ity Check and the CMS’ Hospi­tal Com­pare the high­est rat­ing of three stars. The Tru­ven Health An­a­lyt­ics 100 Top Hos­pi­tals, Health­grades’ Amer­ica’s Best Hos­pi­tals and Con­sumer Re­port’s Hospi­tal Safety Rat­ings re­ceived one star. U.S. News & World Re­port’s Best Hos­pi­tals re­ceived a half star.

The rat­ings groups ar­gue, how­ever, that HANYS has an

Con­sumer Re­ports’ Dr. John Santa said it’s hyp­o­crit­i­cal for hos­pi­tals to com­plain about rat­ings groups when they of­ten use fa­vor­able rank­ings in their mar­ket­ing and ad­ver­tis­ing or make mar­ket­ing claims about hav­ing the “best doc­tors” or “the most in­no­va­tive tech­nol­ogy” with­out good ev­i­dence.

ob­vi­ous con­flict of in­ter­est in rat­ing the raters and that it was not trans­par­ent about its own method­ol­ogy.

Rep­re­sen­ta­tives of Tru­ven and U.S. News dis­missed their low rat­ings from HANYS, say­ing the as­so­ci­a­tion’s re­port card did not take ac­count of their unique goals.

Tru­ven fo­cuses on help­ing hospi­tal lead­ers reach ac­tion­able bench­marks, said Jean Chenoweth, Tru­ven’s se­nior vice pres­i­dent of per­for­mance im­prove­ment.

U.S. News eval­u­ates hos­pi­tals on whether they ex­cel in treat­ing the most med­i­cally chal­leng­ing pa­tients, said Ben Harder, di­rec­tor of health­care anal­y­sis for U.S. News.

Health­grades did not re­spond to re­quests for com­ment.

Con­sumer Re­ports’ Santa said it’s hyp­o­crit­i­cal for hos­pi­tals to com­plain about rat­ings groups when they of­ten use fa­vor­able rank­ings in their mar­ket­ing and ad­ver­tis­ing or make claims about hav­ing the “best doc­tors” or “the most in­no­va­tive tech­nol­ogy” with­out good ev­i­dence.

“I chuckle when I get re­ports that hospi­tal CEOs are wor­ried or con­fused about rat­ings,” Santa said. “They’re not so con­fused that they are not us­ing com­par­isons in their own ad­ver­tis­ing.”

Hospi­tal lead­ers will have to ac­cept that con­sumers are start­ing to scru­ti­nize health­care the same way they look at other prod­ucts and ser­vices, said Leah Bin­der, pres­i­dent and CEO of the Leapfrog Group. “Con­sumers are ac­cus­tomed to re­view­ing a lot of re­view­ers and com­ing to their own con­clu­sions,” she said. “Hos­pi­tals shouldn’t be ex­empt.”

Eval­u­at­ing hos­pi­tals

In­deed, stud­ies show that con­sumers in­creas­ingly are con­sult­ing pub­lished rat­ings to choose providers. A re­port pub­lished Fe­bru­ary in JAMA found that 65% of sur­vey re­spon­dents were aware of on­line physi­cian rat­ings sites. Among those who used the sites, 35% re­ported se­lect­ing a doc­tor based on good re­views while 37% avoided a doc­tor based on bad re­views.

That is rais­ing the stakes for hos­pi­tals. In the re­cent Mod­ern Health­care on­line sur­vey, when asked how dam­ag­ing a poor rat­ing could be for a hospi­tal on an as­cend­ing 1 to 5 scale, 38% of re­spon­dents said 3.

While 79% said their hospi­tal had not been harmed by a neg­a­tive rat­ing, oth­ers said a poor rat­ing had hurt em­ployee morale, led to fewer re­fer­rals, and caused a re­duc­tion in pay­ments.

While hos­pi­tals would pre­fer a more uni­form ap­proach among the raters, that may not be prac­ti­cal, said Dr. Jeff Rice, CEO and founder of Health­care Blue­book, which helps pa­tients com­pare health­care prices. “I com­pletely ap­pre­ci­ate that hos­pi­tals would want to have one stan­dard­ized set of cri­te­ria,” Rice said. “But I don’t think there’s any way to keep people from in­no­vat­ing.”

Hos­pi­tals them­selves should take the lead in dis­clos­ing qual­ity and cost in­for­ma­tion in a clear and use­ful way, he ar­gued. “If they did that,” he said, “then oth­ers wouldn’t spend so much time try­ing to rein­vent the wheel.”

Newspapers in English

Newspapers from USA

© PressReader. All rights reserved.