Study takes poke at WHO’s medical advice
Quarter of suggestions based on poor evidence
TORONTO — In the confusing and fast-changing world of medicine, few voices carry as much weight as that of the World Health Organization.
But a Canadian-led analysis takes a poke at that reputation, concluding the UN agency often makes strong recommendations on how to treat or prevent illness based on weak evidence, potentially leading to patients getting less-than-optimum care.
The new study concluded that 73 of 289 strong recommendations the agency issued over a recent five-year stretch — on topics ranging from maternal health to tuberculosis — were based on low-quality evidence and warranted only conditional advice.
“Historically, WHO recommendations have been extremely untrustworthy and not evidencebased,” said Dr. Gordon Guyatt, the study’s lead author. “If the truth is that an intervention is ineffective or minimally effective, (patients and health workers) would want to know this.”
An official with the agency, however, says the problems are minor and that people have no reason to distrust its advice.
But Guyatt and other critics suggest the issue extends beyond the Geneva-based WHO. Other health organizations are likely still issuing advice based on flimsy science, the product of an unstructured process humorously dubbed GOBSAT — “good old boys sitting around the table,” he said.
Experts on guideline-writing panels are often influenced by conflicts of interest, whether a bias toward their own prior research or financial ties to pharmaceutical companies and, until recently at least, the WHO was no exception, said Guyatt, an epidemiology professor at McMaster University in Hamilton.
“They would sit around and do whatever they wanted to do and direct the world, and be very pleased they directed the world.”
The goal is to move such panels toward more “evidence-based medicine,” a phrase Guyatt coined and a ubiquitous buzzword in modern health care. It involves putting more weight on better-quality science.
Susan Norris, a WHO official who helps oversee the writing of expert guidelines and commissioned the Canadian-led research, said it has identified areas where the process can be improved, and changes are being implemented. But she objected to Guyatt’s description of some of the recommendations as “inappropriate,” saying the agency remains trustworthy.
“Incorrect advice is not being communicated,” Norris said. “WHO’s process is state of the art, in terms of how we develop guidelines. ... There’s always room for improvement, but the WHO has made tremendous improvements as the science has evolved.”
The agency’s guideline panels also have daunting tasks, she said, trying to make research findings relevant for front-line health workers and patients in a host of disparate countries around the world.
The WHO regularly convenes leading international experts to hash out advice, which is especially important in poor- and middle-income countries with less-developed medical systems, Guyatt said.
After concerns were raised seven years ago, the agency implemented a system, called GRADE, for rating — and publicly disclosing — the quality of evidence behind the advice it issued.
The new study found that of 289 strong recommendations in 33 guidelines issued from 2007 to 2012, 73 were not only based on weak evidence, but could not be justified under any of the exceptions the system allowed.
One such recommendation, for instance, strongly advised using “uterine massage” to treat postpartum hemorrhage — heavy bleeding after giving birth. Because the advice is backed up by “very low-quality” evidence and could delay more effective treatment, it should have been just a conditional recommendation, the study concluded.