There’s been a bit of a debate over population predictions on last week’s review of Peoplequake. Among the bones of contention are the UN population estimates, and a 2005 UNEP report on environmental refugees, currently the subject of much skeptic tub-thumping.
Making predictions is a tricky business, but it’s also a natural and necessary process. Reading the future is not a mystical exercise for soothsayers. We do it all the time. If you want to plan ahead in anything, you have to make some kind of judgement call about the future. Should I take an umbrella with me on my walk to the newsagents? Do I need to buy a pint of milk while I’m there? The answers to those simple questions will be founded on current circumstances, previous experience, and educated guesswork.
This is true on the macro level as well as the micro. As individuals, families, towns, nations, and as a species, living well in the present depends on us remembering the past and anticipating the future.
The bigger the scale, the greater the consequences of getting it wrong. I don’t have any great problem with the local council’s estimate of the number of school places needed for 2015, but there is a closed-down school site near my house from a previous over-estimate. On my way to London, I regularly have cause to lament the underestimating of passenger numbers by the train company, as a four-car train pulls into an eight-car platform. A closed school and an overcrowded train are the result of complex predictions gone wrong, and they’re expensive and inconvenient mistakes.
The biggest scale of all is predictions that affect the future of the planet. They carry the most serious consequences, and they’re the most complex and the most controversial. There’s no single body responsible for species-wide planning, but a patchwork of international institutions, government departments, think tanks, NGOs and research institutes. There are patterns of behaviour affecting our future, and being able to read them and respond is vital. Feeding the world, stewarding resources, protecting species, keeping the climate in balance, protecting human life and avoiding conflict – all of these depend on long range forecasting, identifying trends, and drawing up policy accordingly.
We can’t afford not to read the future like this. Neither do we want to get it wrong, not least because a false prediction undermines future warnings. There are fresh predictions in the papers every week, on both dangers and good news, growth forecasts and profit expectations. So how do we tell good predictions from bad ones? How do we spot truth from guesswork? Danger from alarmism? How to judge undue pessimism or optimism?
Here are a few standards by which to judge predictions:
- What is the actual source?
There are at least three levels to a prediction. It probably originates in science, and a balanced, carefully worded extrapolation or observation. For example, an arctic scientist might observe that “If every summer ice melt was as severe as 2007, there would be no Arctic summer ice left by 2015” The university press office spots an opportunity to score some publicity for their institution: “Scientists warn of Arctic sea ice collapse by 2015”. Then the journalists add a third level of spin to the press release, and the headline comes out as “Ice-free Arctic by 2015”. Never take a newspaper story at face value and always take a press release with a pinch of salt. The truth of it, if it’s there to be found, will be in the original study or report. If it’s a sensational story, it’s likely that there’s a frustrated scientist somewhere at the end of the chain who actually said nothing of the sort. - Is the prediction conditional?
Honest predictions are always conditional. If we carry on, this will happen. The unspoken counterpoint is of course that if we do change, it won’t happen. It’s a matter of humility and a way of covering your back, but it’s good science too, as science is an evolving discipline. It recognises that we can’t see the future, and that our guesses about it are tied to current trends. Smart predictions should include variables or alternative scenarios as much as possible, and recognise that things can change. Most scientists are only prepared to give quite narrow predictions, and it’s the series of ‘ifs’ that precede a prediction that are the measure of its success or failure. - How much data underpins the trend?
Most predictions are extrapolations of a trend, so the key question is whether or not there is sufficient data to detect one. Until 2010, you could start your temperature chart at the record high of 1998 and display a cooling trend. Or you can start your chart with the dire ice melt of 2007 and claim that Arctic ice is making a recovery. At the level of climate or demographics, it takes decades to form and identify a trend. The longer the data record, the more reliable the prediction. - What is the agenda? This is a matter of reputation. Some scientists are always firing off dire prognostications, and you have to wonder why. A sensational claim is a great way to make a name for yourself. It doesn’t mean it’s false, of course. It means we need to look at the track record of the predictor, or the institution backing their work. And where was the claim made? Was it originally made in a peer-reviewed paper, or was it announced in a speech or a book launch?
Of course, it’s possible for something to be responsibly worded and based on sound data, and still be wrong. It’s a delicate art. Predictions alone aren’t enough to make our decisions. We need to balance it with risk analysis, the cost of action and the cost of inaction, but that’s another story. As a final precaution against making rash predictions, here’s a Google search engine view of the future, compiled by XKCD.
The Earthbound Report 
A well argued post, Jeremy. Congratulations. It deserves wider attention than I fear it’s likely to get here.
On the matter of bad prediction, this puts it well. It’s a message written by the sadly late Julian Simon on the 25th anniversary of the first “Earth Day”. It was written 16 years ago but, depressingly, is as apposite today as it was then. I urge you to read it.
You advise not “taking a newspaper story at face value”. I agree – so decided to do an experiment. This morning the BBC carries a scary sounding story about climate change (as it commonly does). This one (here) is headed “Climate change to reduce US West water supply – report” and its first paragraph reads, “Scarce water supplies in the western US will probably dwindle further as a result of climate change, causing problems for millions in the region, a government report has said.” Interior Secretary Ken Salazar is reported as saying,
This all sounds pretty grim.
But let’s look at the report itself – the Internet makes that easy. And here are some extracts from the Executive Summary (I went no further for today’s purposes – but I would expect the detail to be even more qualified):
So the whole thing is vague, confused, full of gobbledegook and uncertain. The BBC’s journalist could have discovered that in a few minutes – but didn’t bother.
An interesting case study, and I for one am very grateful to those who place their full reports on the internet. The more transparency the better, although I’m not a scientist and also appreciate good journalism to interpret the harder stuff and put things in context for me.
I note again your cynicism about models however – but the entire scientific method is predicated on modelling, surely? And you start your model off with every scrap of empirical data you can find and then project from there, so it’s hardly a choice between data or model.
I’m not so much cynical about computer models (they have important uses) as critical of those who try to use them to verify hypotheses. The Scientific Method is not based on modelling as you suggest – except sometimes as a part of the Method’s first step, i.e. the establishment of a hypothesis. The essence of the Scientific Method is the verification of that hypothesis by testing it against empirical (real-world) data. And computer models, although they may sometimes include empirical data, are by definition human artefacts relying on uncertain human assumption. (The above US water supply report demonstrates nicely just how much uncertainty there is, for example, in climate-related models.)
An example:
Several years ago, doctors began to suspect that smoking might be related to, or even a cause of, lung cancer. Researchers gathered initial data, ran various medical and laboratory models and concluded that the suspicions seemed likely to be justified. So the smoking/cancer hypothesis was established. Then researchers tested that hypothesis against real-world data – and did so on a massive scale, both geographically and demographically. The hypothesis survived intact. The results were published and independent researchers examined, replicated and accepted them. Thus the hypothesis was verified and the smoking/cancer link confirmed.
The process is elegantly summarised here by the great Richard Feynman.
As this thread is about prediction, I should perhaps have noted that a scientific hypothesis is usually a prediction. Thus the smoking/cancer researchers were, in effect, saying, “look at the world and you’ll find that, where people smoke a lot (whatever their social background, location etc.), there’s a high incidence of lung cancer”. The prediction was correct. Had it not been, the hypotheses would have been falsified and therefore disproved.
So, in verifying hypotheses, scientists make predictions – although, as you say, they usually ensure they are quite narrow.
Incidentally, no matter how many times a prediction matches observation, that is not proof of the hypothesis. In contrast, if just one result does not agree with the hypothesis, the hypothesis is disproven. Thus Einstein upset Newton’s theory – and, were new discoveries to be made that confounded Einstein’s predictions of relativity, his hypothesis too would be invalid.
I realise this is slightly O/T (so my apologies) – but, further to the above, anyone interested in the Scientific Method (especially re climate science) may like to read this article and subsequent posts.
(And, Jeremy, Judith Curry’s “Climate Etc.” is not one of those nasty “skeptic blogs”.)
Interesting comments from Curry, not a particularly good original article that she’s commenting on – full of the usual dismissals of climate science. (They don’t factor in the sun – yes they do. We can’t tell the human causes from the natural ones – yes we can)
Jeremy: a couple of days ago, you accused me of “looking for reasons to support the beliefs [I] already have” – the precise opposite of what I do (see e.g. this and subsequent comments). Yet here you are quite happily accusing someone of “dismissing climate science” when he has the temerity to raise uncomfortable (for you) views about current thinking and methodology. In fact, it’s the essence of science that it takes all views, especially uncomfortable ones, into account and in particular observes the truth (as Nichol says),
Whatever makes you think that “climate science” is any different? Do you really think, for example, that it’s correct to regard Svensmark’s theories about the importance of cosmic ray fluxes as a “dismissal” of climate science?
Robin, you’re linking to Anthony Watts again…
And no, I’m not dismissing the article you mention because it raises uncomfortable views, but because when someone writes an article talking about “the IPCC’s warming alarmism” and “warmists”, they’re working an agenda. I already know that they’ve chosen to believe. And then just in case we had any doubts, he goes and confirms it with a series of statements that proves he doesn’t know what he’s talking about.
“Do they carry out different experiments (i.e., collect new and different datasets) which might give more or better information?” Yes they do – climate science is constantly seeking new data sets. “Do they look at possible solar influences instead of carbon dioxide?” Yes they do – climate scientists aren’t idiots, and of course they’re watching solar activity.
Ha ha. Yes, I linked to Anthony Watts – because it demonstrated a current example (there’ve been others) of my disagreeing with him. Or do you think that, if on his blog I linked to Jeremy Williams, that would mean that I thought that to “make wealth history” was a wise objective? It’s almost as if you didn’t understand what it means to be a sceptic.
And, yes, Nicol is “working an agenda” (as you are) – and his is that climate science should not be exempt from the tenets of the Scientific Method. Surely you don’t have a problem with that?
No problem with having an agenda. All I ask is that people get their facts straight.
Well done for disagreeing with Watts. My question is – if you’re interested in finding out the truth, why do you look to a conspiracy theorist TV weatherman, rather than reading peer-reviewed science?
Hmm, Jeremy, you seem to be in accusatory mood today.
Have you any evidence that I seek “the truth” (about climate science) from blogs (be they supporters or critics of the dangerous AGW hypothesis) rather than from the peer-reviewed science?
Re Watts, have you any evidence that he is a “conspiracy theorist”?
Re Nicol “getting his facts straight”, have you any evidence that the IPCC has taken “variations in solar energy, exotic charged particles in the solar wind and cosmic ray fluxes” into account in explaining global temperature change over the last 150 years? Or that it has taken into account the work (including many peer-reviewed papers) of “Svensmark, Spencer, Lindzen, Soon, Shaviv, Scafetta and McLean”?
I’m not being accusatory, it’s a genuine question. If you seek the truth, which is what you claim you want to do, why go straight to blogs that are dedicated to presenting one side of the debate?
For the record, here’s the IPCC’s write-up on solar activity:
http://www.ipcc.ch/publications_and_data/ar4/wg1/en/tssts-2-4.html
And here’s the notes on indirect solar activity, including Svensmark:
http://www.ipcc.ch/publications_and_data/ar4/wg1/en/ch2s2-7-1-3.html
For any other names in your list, just use the IPCC’s search bar:
http://www.ipcc.ch/publications_and_data/ar4/wg1/en/contents.html
As for Watts, I give you his surfacestations.org project, which claims that the US temperature record is flawed because the weather stations are poorly sited. It’s a daft project – satellite temperature records back up the instrumental record. Are they sited in car parks too? Even if the ground stations were recording higher temperatures because of the heat island effect, they would still record a trend that was exactly the same – just a couple of degrees higher. And even if all the US data was wrong, it still wouldn’t disprove a global warming trend.
I don’t think even Watts believes his own theory any more, which is why his study was published by the Heartland Institute instead of a scientific journal. The Journal of Geophysical Research published a report into his data and you can read the results here:
ftp://ftp.ncdc.noaa.gov/pub/data/ushcn/v2/monthly/menne-etal2010.pdf
Jeremy:
Your question is genuine only in the sense that “Have you stopped beating your wife yet?” is genuine. It’s a question based on an assumption. Therefore, it need not be answered without evidence supporting the assumption – namely that I seek “the truth” (about climate science) from blogs rather than from peer-reviewed science. (A hint: that is not my practice. Your assumption is unjustified.)
Re Watts, my question was: have you any evidence that he is a “conspiracy theorist”? The validity or otherwise of his surface station work is not such evidence. Try again.
Re solar energy and the various other factors I mentioned, my question was not “Has the IPCC noted them?” but “Has the IPCC taken them into account over in explaining global temperature change over the last 150 years?” A very different question.
This last raises some huge, very interesting and controversial issues – about uncertainty and the current state of climate science. I’d be glad to expand on that here if you wish. But do you really want me to? After all, it’s completely O/T and, in any case, this is not another climate change blog. And all the better for it.
No, it’s a genuine question and I’m interested in the answer. Let me rephrase it – what is it that you see in Wattsupwiththat and websites like it?
Thank you – that’s an utterly different question. One I can answer. (And I’ll take the rephrasing as an apology for your earlier unwarranted accusation – thanks.)
What I see in Watts’s blog (and in a whole range of blogs including – you may be surprised to hear – blogs that support the dangerous AGW hypothesis) is a rich diversity of comment and opinion about climate change and related issues. Such opinion is particularly valuable when it links to real evidence, to learned articles and, in particular, to research (especially, but not exclusively, when peer-reviewed and published). And these things, in turn, are particularly valuable and interesting when they challenge such views as I may have formed. OK?
Really? Okay then. But I guess we have totally different ideas of which theories are the dangerous ones.
Yes, really.
PS: the two climate change blogs for which I have come to have most respect are Climate Etc. (here) and Climate Audit (here). The former because Judith Curry (who is not a sceptic) writes with clarity and intelligence and attracts some most interesting comment – welcoming views from all sides. The latter because Steve McIntyre keeps his views on the wider hypothesis to himself, focusing very effectively on his area of expertise: painstaking statistical analysis and matters arising from its application. He too welcomes all points of view and is impressively courteous to those who disagree with him.