Epidemiology fouls air pollution studies

Barely a week goes by without reading in a medical journal or newspaper about another epidemiology study claiming that present-day air pollution in North America causes a disease. According to epidemiology, air pollution still causes almost every disease you can think of… asthma, diabetes, heart disease, breast cancer, early childhood leukemia from maternal exposure, the list goes on. The World Health Organization even says that air pollution causes lung cancer.

Does it make sense that epidemiology can show air pollution causes early childhood leukemia from exposure in the womb? The probability of a result from an epidemiology study being true can be estimated using methods described by John Ioannidis of Stanford University in 2005. It depends on the disease prevalence rate (number of people in a population who have a disease at a given time divided by the total population). It also depends on the probability of a false positive, the probability of a false negative and an estimate of bias in a study, which is widespread in research.

If it is not very true, why should we be concerned?

I estimated the probability of results being true that suggest air pollution causes a disease using epidemiology. I used North American disease prevalence rates and a middle-of-the-road estimate of bias (35 per cent). The results are not very convincing: asthma — only 15 per cent true, diabetes — 13 per cent, heart disease — 10 per cent, breast cancer — four per cent, lung cancer — 0.5 per cent, and leukemia — practically zero (0.2 per cent).

In fact, any epidemiology study claiming air pollution causes a disease will have a less than 18 per cent probability being true at a prevalence rate of 0.1. This is notable since prevalence rates for basically all chronic diseases in western societies are less than 0.1. Debatable exceptions to this may be chronic obstructive pulmonary disease and dementia because of aging baby boomers. Prevalence rates of chronic diseases — particularly cancers — are too low in our population for air pollution-epidemiology studies to be meaningful other than for exploratory purposes.

Far too often epidemiology results are exaggerated and misused to inform the public about air pollution as a risk factor for chronic diseases. One is better off using a simple heads or tails coin toss to decide if air pollution causes a disease (50 per cent probability) rather than using epidemiology. On the other side of the coin and far more important for informing the public is the probability of an epidemiology result being true showing air pollution does not cause a disease. Here the probability is always close to or 100 per cent true. This shows how epidemiology evidence should be used and the truth about air pollution.

Another truth, mostly ignored and more disturbing, is that diseases and even mortalities supposedly caused by air pollution may only be statistical effects within epidemiology studies. They are unlikely real! As John Ioannidis explains, flexible methods in epidemiology lead to what is known as “vibration of effects” — showing an effect or no effect depending on how a study is done. There are various causes for this, including selective design, selective use of data, selective analysis and selective reporting of results.

The devil is truly in the details of how air pollutionepidemiology studies are done and what information epidemiologists choose to publish. Any study claiming air pollution causes a disease offers no credible evidence to the public if it suffers from vibration of effects. Such a study is biased and is no longer truly scientific. As Daniel Sarewitz of Arizona State University states, “a biased scientific result is no different from a useless one.”

When it comes to sharing results of these studies with the public, epidemiologists and too many reporters ignore another truth. They make bold claims that air pollution may cause a disease, but these claims are unlikely true. They seldom offer results about air pollution not causing a disease, even though these results are very likely true. This is a form of reporting bias (only offering selective results or suppressing results) that people should be aware of. Really, the true headline of air pollution-epidemiology studies is… Claims that present-day air pollution may cause are mostly false!