How A Scientific Integrity Act Could Shift The Global Warming Debate

The Air Vent, 23 November 2010 Up until now, peer review has been held up as the gold standard in scientific discourse. Recent developments in the climate science arena, such as Climategate, have led many to conclude that peer review is not all that it is cracked up to be. Having said that, peer review may well be perfectly adequate as a scientific standard when the issues in debate are the mating habits of squirrels. However, if the issue in debate is whether or not trillions of dollars should be spent combating global warming, perhaps a new more rigorous standard should be applied. I propose that henceforth, five levels of scientific rigour be defined. In brief, they are Level zero which is grey literature from advocacy organisations such as the WWF. Level one, which is the current peer review process. Level two, which I will call replicatable, is the current peer review process but with mandatory archiving of data and software code within six months of publication. Level three, which I will call audited, is where an authoritative body of some sort holds a competition on the internet to “find something wrong” with the calculations in the paper with a prize for any independent researcher who can find incorrect calculations. Level four is what I will call Cross Examined and is where the paper in question is deemed so important that, a full scale “internet trial” is conducted. You can think of it as a Scopes Monkey Trial of the researchers and their paper by competent legal personnel advised by scientists. It would mainly consist of oral testimony but with anyone on the internet free to comment and interject in any forum they wish. Naturally these comments can inform the questions put to the researchers. The main effect of this idea would be to shift the public debate to a new level. Instead of the phrase “the science is peer reviewed” being used to silence sceptics, the phrase “the science is only at level 1” would be used to prompt debate. What is needed to give this idea teeth is a law. Let us call it the Scientific Integrity Act. Let me focus the discussion from here forward on the United States as most people from around the world will be familiar with how the process of government operates there. The key thing about the Scientific Integrity act is that it should place a limit of the amount of government spending and / or related industry and consumer costs, that can be legislated for based on the level at which the science is at. To take an example. Level 1 science, would not be allowed to justify any government spending or imposed costs. Level two science might perhaps justify $10 million dollars of government spend and $50 million of imposed costs and so on. Needless to say, any proposal with trillion dollar price tags would need level four science to justify it. Another facet of the law would be that researchers in receipt of a federal grant have to designate their research paper as being at a particular level. In order to encourage realistic assessments of papers by their authors, a penalty system would be included that would operate like this. If a researcher designates their paper as a level three, then should an independent researcher find something wrong with the maths, the funding agency who funded the study would pay a fee to the independent researcher. The fee would be the equivalent of the cost of producing the paper. Naturally a funding agency that got hit as a result of the sloppy mathematics of a researcher would be less likely to fund more bad science by the researcher. So the big question is of course how would level four scientific papers be selected. I see it working like this. Any government agencies, or legislators, that wish to propose laws based on science, have to nominate ten scientific papers to be assessed at level 4. One would hope they would pick the ten best papers, but just in case, there would be a provision for a minority of legislators in the relevant legislature to substitute five of them. IE if the majority in a 100 seat legislature propose some legislation and nominate 10 papers, then a grouping of 20 legislators would have the right to substitute five of them for other papers. I would not see anyone sitting in judgment on level four papers. Remember in order to get to Level four, they will already have been audited at Level three so the math will be correct or at least defensible. Rather I would see questioning both orally and via written submissions of researchers as to why they made particular decisions and interpretations. This information would then be available on the web to anyone and the preponderance of scientific opinion and comment would influence legislators to vote for or against the legislation. Let us consider a hypothetical example. Supposing a researcher decided to use a non-standard mathematical technique that had the effect of producing a particular shape for plotted data even if random number were fed into it. Then, if the researcher tried to defend this notion on the stand, one would expect that the legal teams questioning him would expose this and that the legislators would spot this for what it was. (Then again, one would hope that the paper in question would not have made it past level three in any case.) Some people, especially AGW believers, may view this proposal as being expensive, both in terms of cost and in that it would delay much needed action. I would maintain the opposite. Currently billions are being spent to persuade an increasingly sceptical public that the world is about to fry and despite the billions, legislative attempts to curb carbon emissions have stalled in most countries across the world. The cheapest and fastest way to get action would be to simply put the top ten research papers that prove the AGW hypothesis through a level four Scopes Monkey type trial. If the science emerges unscathed from this process, then I for one can see myself persuaded. I suspect that many other sceptics, as opposed to contrarians, would similarly be persuaded. So let us issue the challenge to the AGW community. Name your top ten papers and let’s put them on trial. With regard to how to progress this idea, I would suggest three things. Firstly, someone or somebody should do an audit of all the scientific papers referenced in the IPCC reports to see if they are at level one or two. Level two remember is where the data and code is released within six months of the publication of the paper. Perhaps a project similar to Surfacestations.org might be the way to go. Given what I read on the various sceptic climate blogs, I suspect that little enough of the science would make it to level two. As an aside, Donna LaFramboise already did a similar audit looking for grey literature references across one of the IPCC reports and found that 35% of the references were level zero. Imagine the change in discourse if one could confidently say that most of the science in IPCC 4 is level zero or one. Secondly shift the debate. When discussing the issue with friends or in the media, explain the levels and ask them would they think it reasonable that the science behind a measure that will cost billions should be subject to a certain amount of scrutiny. And finally, all those Republican controlled state legislatures across America should put forward a Scientific Integrity act in their state. Given the power balance federally there would be no point in introducing the act federally at this time. However, a couple of state legislatures enacting a Scientific Integrity bill would put huge pressure on the AGW community to nominate the top ten papers and put them on trial. The Air Vent, 23 November 2010