Five years ago, not long after the 2011 riots, David Cameron – in a speech aimed directly at tabloid headlines – blamed 120,000 “families from hell”. And in 2012, the Department of Communities and Local Government (DCLG) launched the “troubled families” programme, with a budget of £448m. It claimed that these 120,000 families cost the taxpayer £9bn a year.
But the programme’s evaluation, published today, is the perfect case study of how the manipulation of statistics by politicians and civil servants led directly to bad policy and to the wasting of hundreds of millions of pounds of taxpayers’ money.
You can find the evaluation report, produced by a consortium led by research company Ecorys and including the National Institute of Economic and Social Research (NIESR) here, but for those who do not have time to trawl through all the findings the bottom line is quite simple.
As far as we can tell from extensive and voluminous analysis of tens of thousands of individual records, using data from local authorities, the Department for Work and Pensions, HMRC, the Department for Education and the Police National Computer, the troubled families programme had no discernible impact on the key outcomes it was supposed to improve at all. It didn’t make people any more (or less) likely to come off benefits, to get jobs, to commit fewer crimes and so on.
What matters here is not that the programme didn’t achieve what it set out to do. Successful policymaking requires experimentation and risk-taking – and by definition, sometimes that results in failure.
But it was government’s deliberate misrepresentation of the data and statistics that led to badly formulated targets, which in turn translated into a funding model that could have been designed to waste money. Bad stats meant bad policy.
It’s not as if they weren’t warned, time and again, by me and other researchers. As far back as February 2012, I explained the fundamental flaw in the analysis: that the government was taking a set of families who were undeniably poor and disadvantaged, and redefining them – without a shred of evidence – as dysfunctional and antisocial.
This flawed assumption made its way into the prime minister’s speech. And then this obviously flawed analysis was used to allocate local targets and funding.
In March 2015 ministers decided to pre-empt the result of the evaluation, claiming in a DCLG report: “More than 105,000 troubled families were turned around saving taxpayers an estimated £1.2bn.”
This was untrue. We – including the responsible civil servants – had in fact absolutely no idea whether the programme had saved taxpayers anything at all; and if it had, how much. As I wrote at the time, “the £1.2bn is pure, unadulterated fiction”.
But it was worse than that. Anyone who read this report in detail would have realised that the targeting and funding model was resulting in huge misallocations of money. Each of the councils involved were somewhat miraculously turning around the exact number of “troubled families’ they had been asked to target – 2,385 in Manchester, more than 2,000 in Leeds and so on.
In other words this was not – as the government had claimed – a “payments by results” programme. It was make up the results as you go along and cash the cheques. Numbers that had absolutely no basis in any objective reality had first become the basis for targets, then for claimed “success”, and then for money.
Given this sorry history, the results of the evaluation should hardly come as a surprise.
So who’s to blame? The senior civil servant who directed the troubled families programme, Louise Casey, must be averse to “evidence-based” policy. Why else would her instincts would have been to press ahead, and to ignore both the evidence and the warnings from me and from others?
And while most of the blame rightly rests at the top with ministers, including the former prime minister, and the responsible senior civil servants at the communities department – and they should be held accountable – it is also important to note that the normal checks and balances that should have picked up on all this simply failed. What on earth did the Treasury spending team think it was doing? Where was the National Audit Office? As far as I can tell, it produced one frankly mediocre report that fudged or buried the key points – which were all in the public domain.
Belatedly the public accounts committee, one of the two key parliamentary committees on this issue together with the DCLG – has awakened from its stupor and announced an inquiry into the programme.
But we don’t need to wait for their report to learn a few basic lessons. The most obvious one is that statistics and facts do actually matter, whatever Casey might say. They translate directly into policy, and hence into real outcomes for real people. But to avoid distortions by politicians along the way, the production of statistics and the analysis of evidence need to be genuinely independent.