The Global Climate Change Consensus: My Experiment

The consensus was quantified in a Science study by Prof. Naomi Oreskes (Dec. 2004) in which she surveyed 928 scientific journal articles that matched the search [global climate change] at the ISI Web of Science. Of these, according to Oreskes, 75% agreed with the consensus view (either implicitly or explicitly), 25% took no stand one way or the other, and none rejected the consensus.

Benny Peiser attempted to replicate the study, and found 34 articles that "reject or doubt" the consensus view--that is, 3% rather than the 0% that Oreskes found in her sample. Note that Peiser has altered Oreskes' original category from "reject" to "reject or doubt" so it is logically possible that both are correct. Also, there were several other differences between the studies: Peiser included "all documents" in the database rather than just scientific articles, and he included Social Sciences and Arts & Humanities as well as Sciences. Peiser was kind enough to share the 34 articles that he says reject or doubt. A discussion of the 34 argues that probably two to five of them should count, and the two best examples are editorials, not scientific publications (which is probably why they were not included in Oreskes' study).

When faced with a controversy like this, the great thing is that you can do your own research. If you suspect Oreskes or Peiser (or both) might be biased, you can look at the data yourself.

So that's what I did. Of the 34 articles, I would say that #10 and #27 clearly reject the consensus, but they are editorials, not scientific papers (and #27 is from an oil industry trade association). #1 and #6 doubt, but again are not scientific papers. #7, #17, #31 and maybe #22 doubt, and #15 says that both greenhouse gases and solar activity are roughly equal contributers to warming; so I counted it as "doubt." So overall I would say that Oreskes is correct; that Peiser has not shown a peer-reviewed scientific paper that clearly rejects the consensus. I would also say that Peiser is correct in that he found at least 4 papers that place some doubt on some of the premises of the consensus, but he is widely wrong in claiming 34. Update (June 2007): Peiser has backed off his claims, and now says there is actually only one out of the 34 papers that rejects the consensus, and that one is an editorial, not a scientific paper (and therefore was not included in the Oreskes study).

Another thing that jumps out at me is that some of these abstracts are difficult to classify, but for others it is completely unfathomable how Peiser could consider them as rejections of the consensus. For example, in #18, Analysis of some direct and indirect methods for estimating root biomass and production of forests at an ecosystem level, the conclusion is that "one root method cannot be stated to be the best and the method of choice will be determined from researcher's personal preference, experiences, equipment, and/or finances". That certainly seems to me to be a highly technical article on how to measure roots, with nothing at all to say about the consensus on greenhouse gases. Or consider #24, Regional climate change: Trend analysis of temperature and precipitation series at selected Canadian sites, which explicitly talks about regional climate change and purposely avoids any discussion of global climate change. Because of these obvious errors, I support Science in rejecting Peiser's letter on the grounds that it is a poorly-executed experiment.

I could have stopped there, but I decided to try to do my own study. I didn't want to look solely at the sample of 34 articles that Peiser provided. I didn't have access to the ISI database, so I used a Google Scholar search for ["global climate change"] from 1993-2003, restricted to Biology, Life Sciences, and Environmental Science. I looked at the first 25 abstracts. Here are my results, compared to Oreskes and Peiser:

Exercise for the reader: one of the weaknesses of the Oreskes study (or at least the coverage of it) is that we don't see her numbers for explicit versus implicit endorsement; they are lumped together. What we'd really like to compare is explicit endorsements versus explicit rejections. It seems like one good way to get at this is with the query [anthropogenic "global climate change"]; examine the results and count the numbers on each side.

Another exercise for the reader: compare the results for the previous exercise, for 1993-2003, with the results for 2004-2006. Is the degree of support for the consensus increasing or decreasing?

Weakness of these studies: I've done serious experimental evaluation work in the past, and I know the importance of getting multiple people to make judgments on classifications. Repeat this experiment with at least three people classifying each article.

My final main reaction to reading these 59 abstracts is that an amazing amount of research went in to building up this consensus on global warming, but I hadn't heard much about the specifics. This is partly my fault, but is also another failing of the press. Reporters think (with some good reason) that the public is not interested in hearing about Analysis of some direct and indirect methods for estimating root biomass and production of forests at an ecosystem level and so they never cover such things. But by failing to talk about the years of research and the building on the works of others that go into producing a paper like that, reporters give all ideas equal footing: a half-baked whim with no evidence gets equal footing with a proven theory with hundreds of confirming studies, because it is too complicated to talk about the confirming studies.