A much-cited study (paper here) concluded last month that the extent of warming in Australia in recent decades was so great compared to climate variations in the last millennium that it had to be mainly the result of warming from the human-driven buildup of greenhouse gases in the atmosphere. (Here’s a video interview from May with the lead author, Joëlle Gergis from the University of Melbourne.)
It’s the latest research in more than a decade of work producing a climate “hockey stick” — graphs of global or regional temperatures showing relatively little variation over a millennium or more and then a sharp uptick since the middle of the twentieth century (the blade at the end of the stick).
Now the paper, at the request of the authors, has been “put on hold” by the Journal of Climate after questions were raised publicly about one of the researchers’ methods, starting with a comment on Steve McIntyre’s Climate Audit blog. This field of study uses sophisticated statistical methods to derive meaning from scattered and variegated indirect indicators of past temperature — with tree rings being the most familiar example.
It is unclear whether the problem will affect the study’s conclusions. Depending on the result, readers of the initial burst of news could end up with a familiar sense of whiplash.
To see how quickly the research results made the rounds, check the headlines here. My unfavorite would be “IT’S OFFICIAL: Australia is warming and it is your fault,” in the Herald Sun. This is a classic case of what I’ve been calling “single-study syndrome,” the bias in the news process toward the “front-page thought” and tendency to forget that science is a herky-jerky process.
Over the weekend, I got in touch with David Karoly, one of the paper’s authors and a longtime contact on climate science, to confirm the accuracy of a post by McIntyre quoting him. He said all was accurate, adding this noted about the review of the work:
We cannot say yet whether the conclusions are changed or not until we have completed our review of the data and the results.
As I said in my e-mail to Stephen, “This is a normal part of science. The testing of scientific studies through independent analysis of data and methods strengthens the conclusions. In this study, an issue has been identified and the results are being re-checked.”
Indeed, this is an increasingly normal part of science these days. While the blogosphere comes with lots of noise, it also is providing a second level of review — after the initial round of closed peer review during the publication process — that in the end is making tough, emerging fields of science better than they would otherwise be.
The paper’s changed status was noted today by Ivan Oransky, one of the two science writers behind the invaluable Retraction Watch blog. I asked him by e-mail to characterize the value of this relatively new process of public review. He wrote:
I see this as a good example of how post-publication peer review can work. In general, blogs and other web critiques are already adding a great deal to the scientific process. Some researchers and journals welcome that, as seems to be true in this case. Others stubbornly refuse to engage with criticism from anywhere other than “official channels.” That’s very short-sighted, and it suggests that transparency and self-correction aren’t actually as high-priority as many scientists would like us to believe.
Adam [Marcus] and I expound on post-publication peer review here.
Perhaps inside-baseball, but just in case:
I think what happened here is akin to what you see in physics with arXiv [an archive for preprints of scientific papers], except that arXiv explicitly calls postings “pre-prints” instead of “advance online access” or something similar. We’ve been seeing more and more examples of papers that are published online, then withdrawn into a sort of purgatory. The reluctance to come out and say these papers are retracted also reflects that print vs. online bias. Sorry, folks, once something is published, you don’t get a pass on explaining what was wrong with it just because it hasn’t hit print yet.
Here’s an excerpt from Oransky’s post:
Post-publication peer review is indeed a “normal part of science,” as Karoly notes, and one that we’ve championed. While we’ll have to wait for the final outcome in this case, the authors should be commended for acting swiftly to try to correct the record. We’d only remind them, and the journal, that a publication is a publication, whether it’s online or in print, so if this paper turns out to be fatally flawed, it would need to be retracted and not simply disappear.
Despite the often contentious debates that erupt over climate change science, we’ve seen only one other retraction in the field since we launched in August 2010, when Edward Wegman was forced to retract a paper for plagiarism.