When news gathering goes down the crapper
As studies go, beauty.
Or a nightmare. Or a devious way to find something close to the truth.
Or just plain clickbait.
Stay with me now.
The problem was simple. Statistics Canada wanted to find out if people were telling the truth about their cannabis use, but the agency was afraid the facts of the matter were, well, hidden underground.
So, knowing a few variables about the size of various municipal populations, the rate at which cannabis metabolites are excreted by people who used marijuana, and sewage technol- it’s a headline ogy, Stats Can followed the lead of European science and went sample-gathering at 15 sewage treatment plants.
The idea was simple: sample the sewage, and backwards-calculate how much weed was being consumed. (The samples were taken between March and August of this year, but also will continue post-legalization on into March 2019.)
The result gave Halifax the “high” five; that city tested highest for cannabis metabolites in sewage ahead of Edmonton, Montreal, Toronto and Vancouver.
And that’s pretty much the way the study was cast in the media – complete with some outlets getting comments like, “should match up that study with unemployment stats – lazy stoners.”
But if you go back to the study itself, you’ll find it’s very clear on how it doesn’t prove anything of the kind, at least not at this point.
Among the caveats? “For ex- ample, the wastewater treatment plant in Halifax was situated in the core area of the city and as a result, the consumption of cannabis measured there may not represent the city’s entire metropolitan population.”
So, not apples to apples. OK – so that’s a simple one.
But there are more warnings: “Results for almost every site varied substantially from month to month. However, at this point, it is difficult to determine whether the variations were due to real changes in behaviour or to statistical phenomena related to sampling and flow rates.”
And there’s more: “In addition, differences in temperature, acidity and the presence of industrial chemicals or bacteria in the sewer system of each city could have an impact on the amount of cannabis detected in wastewater and affect the comparability of data from different cities.”
And more: “Furthermore, the month- to- month variation in loads is large. For example, because of the amount of monthly variation in Halifax, it can only be expected that the true annual usage is contained within the bounds of 603 mcg/person-week and 2,018 mcg/person-week twothirds of the time. Given this large range of uncertainty, it is all the more difficult to make comparisons between cities with very much confidence.”
Finally: “Statistics Canada will continue to collect more data from each participating wastewater treatment plant to assess whether the rates differ as a result of true differences in consumption rates, or as a result of methodology.”
Think about that sentence. Halifax’s weed win might be nothing more than a sampling glitch.
It’s a bit like sampling grocery store dumpsters in St. John’s, counting the spoiled heads of lettuce, and arguing ( without considering spoilage during long shipping, the amount of lettuce coming into the stores, and a host of other variables) that St. John’s residents eat fewer greens than people in other provinces.
The sewer/cannabis study is a piece of the puzzle, a tool in the toolbox for trying to put a real number on cannabis use.
It makes for a quick headline, but it’s far from complete science, and even Statistics Canada admits that.
It shows two things.
One is that consuming news is not as simple as reading one story.
The other thing?
A day will come when we will have no secrets. They’re sampling your sewage.