USA TODAY International Edition
Reader beware
How to decode medical studies
Coffee is healthy. Coffee is not healthy. Maybe try a glass of red wine? Medical study findings have us constantly questioning what we should be drinking, eating and asking our doctor. (Just take a look at the recent news about the Mediterranean diet.) First, let’s get one thing straight. “Almost all of the studies about coffee and red wine are association studies,” said Howard Bauchner, editor in chief of the medical journal JAMA and The JAMA Network. For example, research might show people who drink coffee die later than those who don’t, but that doesn’t mean coffee is the fountain of youth – or even the cause of longer life. But some studies reveal more convincing evidence. For people struggling with serious diseases, such as cancer, treatments being tested in clinical trials could mean the difference between a longer or shorter life.
Here is how to identify when a study finding could change your life:
❚ Does the study involve humans or animals?
Many studies that begin using mice, rats, pigs or other animals fail when tested in humans.
Animals are helpful for studying the safety of potential new treatments, but beyond that, don’t count on them, Gregory Petsko, a neurological researcher at the Weill Cornell Medical School, told NPR.
Still, many studies are conducted on animals in addition to cells and humans.
“Research in humans is closest to application,” stressed Meagan Phelan, Science Press Package executive director at the American Association for the Advancement of Science.
❚ What kind of study is it?
Medical findings come in all shapes and sizes: reviews, meta analysis, case studies, surveys, opinion pieces, research reports and so on.
“The strongest, highest-quality evidence is derived from a randomized clinical trial,” Bauchner said.
Clinical trials are performed on a group of people to determine if a new treatment is safe and effective.
Be aware of the trial’s phase. Drugs tested in clinical trials typically go through a series of phases (one to four), with the first being a small group (20 to 80) of healthy people and the last being tested on large, diverse populations with FDA approval, the National Institute on Aging notes. Years could pass before an early trial makes it to a late trial.
❚ Is it peer-reviewed?
A peer-reviewed study has been assessed by an editorial board and published in a journal, prominent examples of which include the New England Journal of Medicine or JAMA. These studies have been through a high level of scrutiny, and the findings are often more respected in medical communities compared to non-peer-reviewed reports.
❚ What is the sample size?
Be wary of sample sizes that include fewer than 100 people.
If the research is focused on a rare issue, a small sample size might be fine. But if the research is focused on coffee drinkers, for example, there are enough people in the world that drink coffee that the sample size could easily be well over 1,000.
❚ Are the people involved like me?
Look at the age, race, ethnicity and medical history of the patients involved in the study. If you are a 65-year-old woman, and the research is based on a sample size of 25-year-old men, the treatment being discussed might not have the same outcome for you.
❚ Are risks discussed?
“A study that always talks about the benefits would give me concern,” Bauchner said.
Researchers showing a beneficial finding should also present risk or potential harm that might come along with the discussed treatment or lifestyle.
❚ What about limitations?
Researchers should be transparent about potential roadblocks they encountered in the study. For example, did researchers ask people about their diet or actually watch them eat? If patients were just questioned, they might not have remembered correctly.
Similarly, Phelan noted, see if authors compare their approach to existing alternatives to offer “a benchmark of sorts.”
❚ Who funded the study?
Company-funded research isn’t necessarily bad, but it can be a clear conflict of interest. Be skeptical of candy makers releasing research on the health benefits of dark chocolate or a drug company funding a study showing benefits of its products.
Phelan advised looking at a study’s “acknowledgements” for red flags.
❚ Be aware of bias.
Physicians, researchers and readers all come with their own set of biases that can affect how a study comes together and how it’s interpreted. It’s often not malicious, but it could skew study results.
A researcher who likes red wine might be more inclined to look for health benefits of the alcohol as opposed to harm. Similarly, a reader who enjoys a few glasses of wine a week might pay more attention to studies that tout its benefits compared to those who don’t.