By Alan Davies, September 17, 2013
Oxford’s Bent Flyvbjerg reckons underestimating the costs of transport infrastructure and overestimating demand is so chronic we need to consider sueing forcasters and maybe having some serve time
Yet these days early estimates of the cost of building new rail lines are routinely criticised as inflated or excessive, usually by those who feel strongly that a particular project, or class of projects, should proceed. Some believe governments strategically over-estimate costs (like the $114 billion cost of east-coast High Speed Rail) in order to deflate public pressure to build them.
The available evidence, however, overwhelmingly shows the opposite: that governments, authorities and technical experts consistently under-estimate the cost of projects and over-estimate demand. This is a subject I explored a couple of years ago (Why do the worst infrastructure projects get built?) and it’s worth revisiting.
I drew on the work of Professor Bent Flyvbjerg from Oxford University’s Said Business School, who argues that under-estimating the cost of major infrastructure projects and over-estimating the demand is so chronic that forecasters deserve some harsh medicine. He says:
Some forecasts are so grossly misrepresented that we need to consider not only firing the forecasters but suing them too – perhaps even having a few serve time.Australians have plenty of experience with underperforming infrastructure projects. For starters, just in transport alone, there’s Brisbane’s Clem 7 and Airport Link road tunnels, Sydney’s Lane Cove and Cross City tunnels, the Brisbane and Sydney airport trains, Melbourne’s Myki ticketing fiasco, and the 2,250 km Freightlink rail line connecting Adelaide and Darwin. And they’re just the ones we know about!
Professor Flyvbjerg says cost overruns in the order of 50% in real terms are common for major infrastructure projects and overruns above 100% are not uncommon. Writing in the Oxford Review of Economic Policy (Survival of the un-fittest: why the worst infrastructure gets built – and what we can do about it), he argues that demand and benefit forecasts that are wrong by 20–70% compared with the actual outcome are also common.
Transport projects are among the worst performers (see exhibit). Professor Flyvbjerg examined 258 transport projects in 20 nations over a 70 year time frame. He found the average cost overrun for rail projects is 44.7% measured in constant prices from the build decision. For bridges and tunnels, the equivalent figure is 33.8%, and for roads 20.4%. The difference in cost overrun between the three project types is statistically significant and the size of the standard deviations shown in the exhibit demonstrates the high degree of uncertainty and risk associated with these sorts of projects.
He also found that nine out of 10 projects have cost overruns; they happen in all nations; they’ve been a constant over the last 70 years; and cost estimates have not improved over time.
And it’s not just under-estimation of costs. Errors in forecasts of travel demand for rail and road infrastructure are also endemic. He found that actual passenger traffic for rail projects is on average 51.4% lower than forecast traffic. He says:
This is equivalent to an average overestimate in rail passenger forecasts of no less than 105.6 per cent. The result is large benefit shortfalls for rail. For roads, actual vehicle traffic is on average 9.5 per cent higher than forecasted traffic. We see that rail passenger forecasts are biased, whereas this is less the case for road traffic forecasts.He also found that nine out of ten rail projects over-estimate traffic; 84% are wrong by over ±20%; it occurs in all countries studied; and has not improved over time.
Thus the risk associated with rail projects in particular is extraordinary. They face both an average cost overrun of 44.7% and an average demand shortfall of 51.4%.
Professor Flyvbjerg identifies three possible causes for these errors. One is technical error – this relates to factors like inadequate data and the inherent difficulty of forecasting the future. It is the customary defence when projects fail. Another is psychological error, like optimism bias. Finally, there’re political and economic explanations, where promoters, investors and politicians deliberately under-estimate costs and over-estimate benefits.
Based on the various projects he examined, Professor Flyvbjerg rejects both technical and psychological theories as the primary cause of errors. He points the finger at political explanations – “strategic misrepresentation”, as he describes it. Promoters, advocates and forecasters have an incentive to intentionally under-estimate costs and over-estimate benefits to obtain approval and funding for their projects. This results in:
an inverted Darwinism i.e. survival of the unfittest. It is not the best projects that get implemented, but the projects that look best on paper…..Forecasting is mainly another kind of rent-seeking behaviour, resulting in a make-believe world of misrepresentation which makes it extremely difficult to decide which projects deserve undertaking and which do not.Apart from “having a few forecasters serve time”, the key way to address the problem, he recommends, is via use of ‘reference class forecasting’. In essence, it involves comparing the project against a large sample of similar past projects to determine the range of cost and benefit miscalculations.
Virginia Postrel interviewed Professor Flyvbjerg for Bloomberg.com. He told her when ‘reference class forecasting’ was introduced in the UK it stopped a number of projects dead in their tracks. “This has never happened before”, he told her.
It’s possible that the initial assessment of $114 billion to construct east-coast HSR might be too high. But if it were to go ahead, it’s far more likely it would ultimately be shown to be too low. For the most part, claims that early estimates are too high are another case of strategic misrepresentation.