Where’s the data?
Holding science to prospectus standards would stop climate researchers from launching misrepresentations like the ‘ Hockey Stick’
In June, the energy and commerce committee of the U.S. House of Representatives opened an investigation of a prominent scientific study and the circumstances under which it became the centrepiece of a report by the United Nations Intergovernmental Panel on Climate Change.
The investigation has many observers, including climate scientists themselves, up in arms. The Washington Post called the committee action a “witch hunt,” while others have compared it to the Spanish Inquisition. The American Geophysical Union and the American Meteorological Society have written a joint letter of protest, accusing the House energy committee of undermining science and attempting to intimidate its authors. Editors of prominent journals like Nature and Science have weighed in on even stronger terms.
Although critics contend the issue is about scientific freedom, the questions actually pertain to disclosure, due diligence and the need for access to publicly funded scientific data when public policy is at stake. In reality, the investigation is not only entirely proper, but long overdue.
The saga begins in 1998, when Michael Mann and colleagues published a graph in Nature that they argued represents the air temperature history of the Northern Hemisphere for the last 1,000 years. Owing to its shape, the curve is called the “ Hockey Stick.” It shows a relatively constant air temperature ( with a slight decline) from A.D. 1000 until the late 1800s. But over the last century, the air temperature dramatically increases by about 0.6C, which, the authors and believers assert, proves that humans are indeed responsible for virtually all of the climate change of the past millennium. It was the Hockey Stick that originated the sound bite declaring 1998 to be the “ warmest year” of the millennium and the 1990s the “warmest decade” — a sound bite used by the Canadian government in making the case for adopting the Kyoto Protocol.
The Hockey Stick stands in stark contrast to a long-held view, amply supported by work of other researchers, that the last 1,000 years were characterized by a warm beginning (the Medieval Warm Period), a rapid cooling around A.D. 1500 (the Little Ice Age), and a latter- day recovery from this cooler period. The Hockey Stick became entwined with energy policy when the IPCC replaced this traditional view and featured the Hockey Stick prominently in its 2001 assessment of climate science — in a section written by Mann himself. It surprises many to learn that the IPCC assessment often is written by scientists who dominate the debate about specific issues.
Clearly such scientists have axes to grind and, in Mann’s case, he used the IPCC as a forum to promote his own research. Other IPCC authors admonished Mann to include other, less Hockey Stick-like representations in his assessment. They were ignored in the final report, however, and, owing to the influence that the IPCC reports carry, the Hockey Stick became a public icon, enthusiastically promoted by supporters of the hypothesis of greenhouse warming.
The statistical methods used by Mann and his colleagues have been the subject of much recent scrutiny. Based on our own research and a detailed comparison with the published evidence, Willie Soon and I raised the spectre of flawed statistics in the Hockey Stick when we testified with Mann at a U.S. Senate committee hearing in 2003. Subsequently, two Canadians with strong statistical training — energy analyst Stephen McIntyre and economist Ross McKitrick — attempted to replicate Mann’s results using the data he had supplied them. They found a number of errors, improper calculations, and misrepresentations of methodology. In the refereed literature, other researchers have expressed concerns about and demonstrated problems with the Hockey Stick. The McIntyre and McKitrick study led to a corrigendum in Nature, where Mann and his colleagues admitted to various inaccuracies in their original description of their data and analysis. Nature took the extremely unusual step of requiring Mann and co-authors to provide a new archive of data and a new verbal description of their methodology. But even with this revised release, key aspects of the Hockey Stick remain impossible to replicate — and replication is a hallmark of scientific inquiry. Mann continues to refuse requests for full disclosure, telling The Wall Street Journal
that to do so would amount to “ giving in to intimidation.”
Despite the importance of the Hockey Stick for climate policy and the repudiation of scientific ethics implicit in Mann’s statement, there was no reaction to The Wall Street Journal
article by the U.S. National Research Council or any learned societies and virtually no shock or surprise from climate scientists themselves. However, these extraordinary and injudicious remarks by Mann did attract the attention of the U.S. House energy and commerce committee, an important committee with broad investigatory powers, which carried out hearings on Enron, for example.
But the issue here goes beyond data and methodological documentation. The energy and commerce committee asked Mann and colleagues about the withholding (from their analysis) of vital statistical information that was highly adverse to their claims. This amounts to selectively choosing data to support their position and ignoring data that refutes it. But the academic community has misconstrued the intent of the committee by largely assuming it is attempting to decide nuances of statistical interpretation. In fact, the committee is on much more familiar turf than the learned societies have appreciated: Their request regards issues of disclosure, framed in the language of securities legislation — terminology with which the House committee is completely familiar. If science were subject to prospectus standards, withholding of such information would not have been permissible.
“Informational hoarding” is being challenged. Some academic journals now require publication of all data and computer code along with the article itself. The U. S. National Institutes of Health, which funds many billions of dollars worth of medical research, has mandated that large grants are conditional on data sharing. Other federal agencies are now beginning to consider NIH’s lead to provide verification of important findings.
Since the House energy committee is responsible for energy policy, it has every right to demand additional scrutiny for studies upon which energy policy is being made. Failing to disclose data or methods is not an acceptable option when energy policy is at stake. Moreover, since Mann was the author of the section of the IPCC that touted his own research before others had the opportunity to critically re- examine his work, serious questions must be raised about conflicts of interest within the IPCC and how it came to promote speculative findings that had not been independently evaluated and which since have been shown to be flawed.
The outrage expressed by the AGU, AMS and other scientific societies is hypocritical. Funding for climate science amounts to several billion dollars a year, but these groups strongly protest the accountability that goes with it. Both the AGU and AMS have adopted statements calling on the United States to change its energy policies in light of the climatechange issue. Yet while they insist that this research be the basis for policy decisions, they object to its scrutiny by policymakers.
In this instance, the House energy committee has uncovered a real problem in science — one that extends far beyond the climatechange issue. Scientists must demand that results and conclusions stand up to independent verification. Yet since the climatecommunity has failed to impose such standards on itself, it cannot be surprised if legislators have opted to do the job for them.