Category Archives: Research

What works at a local level?

First produced as a briefing for Alliance for Useful Evidence/SOLACE Foundation policy seminar

As the axe swings on funding, we need the best available evidence to tell us how to get the most from our budgets. If the grave Audit Commission predictions are correct, local authorities will need to find 26% further cuts in 2014/15, on top of the ones that have already kicked-in. Robust research and analysis can help point to where we might do more with less.

But what evidence do we need?  Grand scale high-cost evaluations with armies of analysts are not be feasible in a time of austerity.  Similarly, evidence that is not robust will not help us find interventions that work. We are going to need to be realistic about what evidence we can marshal to help local services. The evidence has to be proportionate to the challenge.

Fortunately, some evidence is free and accessible. The Education Endowment Foundation, for instance, has a ‘pupil premium toolkit’[1] that is popular with schools and local decision-makers at it gives an easily-digested but robust snapshot  of the evidence of  class sizes, teaching assistants, homework and other features on educational attainment. But some councils have taken a system-wide approach to using evidence to improve local services.  Birmingham City Council has hardwired research into decision-making. At the highest level in the city – and across political parties – it has been recognised that current decision making, especially on resource allocation is partially ‘flying blind’, without the evidence needed to work out priorities between major spending programmes.[2] What Birmingham has done is to move towards ‘outcome focussed planning’. To put it simply, this means starting with desired outcomes, testing different approaches and then re-designing services, rather than the more traditional focus on inputs and processes. Evidence has to underpin all of this.

An example of this in practice in Birmingham can be seen in Shard End Community budget pilot that helps families with complex needs. Based in one of the most deprived areas in the east of city, Birmingham  has implemented a variety of family  interventions based on a depth of evidence, such as ‘Triple P’, ‘Incredible Years’, ‘PATHS’ and Family Nurse Partnerships. Hopefully it will be a win/win situation – not only will this result in better outcomes for families, it will create long term financial benefits for the programme.  They have mapped out a whole range of longer term fiscal savings through these early interventions across all the Birmingham wards.

Question 1:  what evidence do we need in a time of austerity to help local decision-making?

More local bang for the buck

The Birmingham approach is ambitious in its desire to transform thinking on social outcomes.  But there can also be smaller, targeted interventions based on technical analysis that can help save money.  The waste company WRAP have used data analysis to help West Oxfordshire District Council save around £500,000 a year on an integrated refuse, recycling and waste collection service. WRAP are a good example of an external body providing practical, research-based advice, creating smarter waste and recycling services, and also helping with procurement strategy and tender documentation.

We also need evidence to see if high profile cost-saving initiatives are actually doing their job – and saving us money.  Do we really know, for instance, if shared services make a difference?  A report from the Local Government Association says that five local shared services arrangements have saved a total of £30m through a range of measures, including integrating IT systems and better procurement[3]. The report was the first to provide a detailed insight into the scale of savings that have been achieved through sharing back office functions like IT and legal, and teaming up to deliver frontline services like waste disposal and road maintenance.

The significance of this LGA report is that it highlights how we must evaluate and test local initiatives – sometimes ‘in-flight’ rather than waiting until it’s too late at the end – to see if efficiency savings are really being made.  The evidence form this LGA report is useable across local government – alongside the report, they built a new, accessible, Excel-based prototype tool to understand the benefits of sharing front and back office services. It will help others develop baseline information to track the level of savings and performance benefits that can be achieved.

Local authorities have a good story to tell central government in raising productivity[4].  We also need to learn from the bad stories of where things aren’t working.  In these difficult financial times, we can’t afford the luxury of ploughing on with things that don’t work. It will, of course, take considerable political mettle and media management to shut down projects, programmes and policies that are not working.  Trying to explain that particular services will be cut because of a robust cost-benefit analysis will not always cut it with voters, journalists or the political opposition.  But without credible evidence it may be much harder to make the case for the cutting of particular services.

Tapping into new sources of income

There is another financial bonus to evidence. Councils are making evidence work for them by finding alternatives sources of income through social finance. In order to scale up community-based budgets like Shard End, Birmingham are working with the Cabinet Office to develop evidence to make the business case for social impact bonds. Birmingham’s ‘Brighter Futures’ project uses robust cost-benefit analysis to inform future investments. Based on methodology from the Washington State Institute for Public Policy, it will seek ways of reallocating resources away from expensive late interventions to greater investment in early intervention. It is hoped that the model will be used by other councils to predict savings, track financial efficiencies, and provide an evidence base to inform future commissioning.

There are, however, significant challenges with this social investment model.  As the chief executive of Birmingham Stephen Hughes pointed out in a recent speech[5], it is not always clear if the evidence exists to back up social investment bids. And if we do have good evidence to help better outcomes, is that always translated into predictable cost savings?

Family Nurse Partnerships is another example of using evidence-based approaches to access new national resources. Run by the NHS and local authorities, the Government has committed to doubling the evidence-based capacity to at least 13,000 places by 2015. This scheme is a highly effective approach for young first time mothers.  It was inspired by 30 years of evidence from the US on the benefits of intensive home visits by specially trained family nurses, from early pregnancy until the child is two. The outcomes are wide ranging, from improved early language development, to reductions in children’s injuries and abuse.  Although we don’t yet know what the savings might be for UK local authorities, cost savings in the US are around US$ 3-5 return for every dollar invested. A positive recent formative evaluation in England pointed to the potential for early impacts and for cost savings. We will also find more about its value to local authorities in 2013, when a large scale research trial will report early findings.  But the benefits look highly promising and could work across local areas.

Sources of information on ‘what works’

It can be a bemusing experience finding sources of information on ‘what works’. Take education as an example. There are a plethora of online resources, ranging from targeted easy-to-read online resources like the Pupil Premium Toolkit, to great big online libraries of meta-analysis, such as the EPPI Centre.  The Department for Education itself does not have a single bank of evidence available across children’s services, but there are various resources that local areas can access.  In particular, the Centre for Excellence and Outcomes[6] reviews, interprets and publishes the evidence base online. Evidence is also being driven through commissioning and productivity by the Children’s Improvement Board (that includes SOLACE, LGA, Department for Education and others).  The Board’s remit includes reviewing evidence on ‘what works’. But how many people are aware of these resources?

Some analysts in government departments also provide a ‘what works’ function.  The Department for Work and Pensions, for example, publishes a wealth of evidence. The 160-page DWP report into ‘What Works for Whom’  reviewed the stock of evaluation in order to identify which interventions have worked most effectively for key customer groups, such as young people and long-term unemployed[7].  An informal group of central government analysts working on localism and decentralisation has also been set up – the Localism Analysts’ Group. They may have a role in supporting evidence for local needs.

Question 2: would a single online evidence hub pointing to all available resources on ‘what works’ be useful for local decision-makers?.

One new source of evidence is on the horizon.  New ‘what works’ institutes in key areas of social policy are being developed by the Government.  Trailed in in the  Civil Service Reform Plan and  Open Public Services White Paper, the proposed new infrastructure is sometimes referred to as ‘NICE-type institutes’,  following the National Institute for Health and Clinical Excellence in the health service. The plans for this new infrastructure are still being worked out by the Cabinet Office and others – including the BIG Lottery, ESRC and Nesta.  Current thinking is that there will be sector-specific centres – perhaps with one central coordinating body – and that they will synthesise and communicate the evidence base to decision makers, commissioners and practitioners.  Although no institutes have been finalised, there is currently exploratory work underway on sector-specific ‘what works’ institutes focusing on ageing well and local economic growth. These institutes would seek to synthesise and disseminate evidence relevant both to national policymakers and local commissioners and providers.

Question 3: what would be the most useful function these national ‘what works’ institutes could provide to help local decision-makers?

Collaborations and scaling up

A common theme for many successful local evidence-based initiatives is the need to work with partners.  Success – and new sources of income – often comes through working alongside other national and local bodies. New Economy Manchester – a commission to advance economic growth and prosperity across Greater Manchester – have committed to using national partners to drive the adoption of new approaches, and have been working with Whitehall,other large cities and international partners, such as the Washington State Institute for Public Policy. They have been in the vanguard, along with peers such as Birmingham and Westminster, in developing an evidence-based approach to the design, implementation and funding of local services, using new methods of cost-benefit analysis, Randomized Controlled Trials and other techniques. Their use of evidence was a vital part of the debate around Manchester’s ‘city deal’. Manchester focused on how to extract local revenues, such as the proposals to ‘earnback’ taxes from central government and plough back into local economy. Evidence was a key part in their success in making the case for their ‘city deal’ and raised the quality of debate about how to boost growth, based on sound research.

One challenge is that lessons of ‘what works’ from local areas may not be applicable elsewhere.  Examples of social interventions in say, Fife, may not be relevant to Birmingham’s Shard End.  Smaller sample sizes mean lessons learnt will be specific to local context and not generalizable to other areas. There may also not be consistent ways of measuring and collecting data – leading to even less chances of comparing local areas to see if things are working. There are therefore potential economies of scale and benefits from ensuring, perhaps by some national body, that information and evidence is produced and disseminated effectively.

Collaboration between local projects and programmes in gathering evidence must be one way forward. The Mayor of London’s Project Oracle is one attempt to bolster collaboration between social intervention projects, government and academia. Not only have they brought together disparate groups to figure out ‘what works’, they have also set down a common standard of evidence for everybody to buy into.

Project Oracle arose from the Mayor’s ‘Time for Action’[8], a programme to tackle serious youth violence and to ‘understand and share what really works’. It was set up in response to a recognition that there was not a clear understanding of what programmes work to tackle youth violence.  They are building a sustainable body of evidence to help future policymaking, and have a consistent method and standard of evidence. Oracle employs an online ‘refereed’ resource to share examples of good practice across the capital. So single projects with names such as The Gang*Star Project  Growing Against Gangs and Violence, Safer Children Project and many more, are working to an agreed set of standards of evidence, so that they can collectively figure out ‘what works’, alongside the Met Police, Youth Justice Board,  Director of Children’s Services – and academia. It’s a remarkable collaboration and may offer a model for collaboration for other local social initiatives

Question 4: what collaborations are needed to help provide better evidence on ‘what works’?

 Question 5: what can central government do to support greater local evidence-informed decision-making and practice?


Despite the need for research, local analytical capacity has reduced over recent years as headcount falls.[9] Now, more than ever, analytical teams have to justify their existence to senior officers and councillors. At a recent conference of local analysts and researchers, the chief executive of Coventry Council and President-elect of SOLACE Martin Reeves said that researchers must “focus on outcomes for the people we serve – nothing else matters”. Analysts and researchers should view this as a “moment in time to transform the profession”, using their insights to provide evidence on effectiveness, not just efficiency, manage and reduce demand and design new services. There are many challenges in finding ‘what works’ in local areas. Decision-makers in local government, charities or business may not have the capacity or capability to access and interpret evidence on ‘what works’.  Central government could have a role here, supporting local areas by providing more access to data and knowledge.  For example, there might be some centralised standards for evaluation, kitemarks on effective interventions or user-friendly synthesis on the latest research. The forthcoming ‘what works’ institutes flagged up in the recent Civil Service Reform Plan may help provide this service to local authorities.

But there is no point getting the supply of evidence right if there is no demand for it in the first place. If we can get the incentives right on the demand-side, we may also find a body of useful evidence grows of its own accord. Payment by Results and social investment models may provide the motivation/incentives, rewarding local areas for achieving outcomes based on evidence-based approaches.

We have to make every pound work harder and research, evaluation and evidence are key to this. We need to make sure we gather intelligence that can really help to make a difference to people on the ground.   It needs to help make savings and efficiencies, striving for more for less.  We can’t afford the luxury of ivory towers and research for research’s sake. But intelligence and analysis can help focus our minds – and shrinking resources – on ‘what works’.

Jonathan Breckon, Manager, Alliance for Useful Evidence

This article was written to support to What Works at a Local Level? event held by the SOLACe Foundation and the Aliiance for Useful Evidence on 13th September 2012. The post raise questions around the use of evidence across local public services an issue that local government will return to at the SOLACE Summit in October.

[2] Bovaird, Tony and Kenny, Richard (2012), Modelling Birmingham: Using Strategy Maps to Compare Outcome Pathways, Paper presented to Performance Management Association Conference, University of Cambridge, 11-13 July 2012.

[4] This research follows the recent publication of the Commons Public Accounts Committee report into shared services which demonstrated that local authorities are significantly outperforming central government in this area.

[7] See for example, the 160 page review of evidence and meta-analysis for policy areas covered by DWP

[9] A report based on interviews with 25 Chief Executives concludes that “monitoring and evaluation roles appear to be one of the foremost casualties of back office efficiencies in many, if not all, authorities”  Serving deprived communities in a recession, Annette Hastings, Glen Bramley, Nick Bailey and David Watkins, January 2012, Joseph Rowntree Foundation


Leave a comment

Filed under Learning and development, Research, SOLACE Summit, Uncategorized

Usable data, useful evidence

The Alliance for Useful Evidence held its summer networking event on July 24th. I attended and was interested to see relatively low representation from local government compared to central government and the third sector. Yet local government has much to gain from thinking about using evidence more effectively, and much to contribute to the debate to ensure the specific issues it faces are taken into account.

At the event, David Willetts discussed the relationship between evidence and politics. On the one hand, he claimed, empirical evidence can work to bridge the gap between different ideological viewpoints – yet on the other, the politician’s role is to ‘respond to events by instinct in accordance with party politics’ rather than to make evidence-based decisions. Willetts argued that academic research sometimes fails to take politics into account and this may be reflected in a lack of sophistication in policy analysis relative to the treatment of data.

Willetts’s focus on the effects of politics is a useful reminder that “pure” empirical evidence can rarely be isolated from other influences. He presented payment-by-results programmes as an exemplar of evidence-based decision-making, yet the controversies over Atos’s delivery of the work capability assessment are currently showing how supposedly objective, empiricist, methodologies – such as eligibility criteria – can become intensely politicised.

Local authorities face additional issues in this regard. They must take account of the political dimension of decisions but, since it’s particularly important for them to be responsive to the communities they serve, they are tied still more directly to local viewpoints and preferences when making decisions. If local values conflict with the evidence, should they be considered less valid?

Secondly, this close relationship to the community means that the most rich and valuable information can be that which comes from frontline workers – at the same time, this can be the most difficult to capture and analyse systematically because it is so varied, specific, and bound up in the practicalities of doing the job rather than evidencing it.

Thirdly, evidence, to be effective for local government decision-making, needs a local focus, on whether an intervention is right for this particular place. National statistics and academic research can help paint a picture – and Rehema White presents ingenious options for commissioning locally focused research – but I’d suggest that a local focus means we also need to consider how we use and share data both internally and locally, and how this can be “converted” into evidence.

At the Alliance event, Geoff Mulgan pointed to the ‘“difficultness” of evidence’, and suggested that a ‘surge’ in available data is not the same thing as a surge in available evidence. I would argue that we understand that we have to “do stuff” to data to make it useful as evidence; analyse it or cross-reference it with other data sets: however, we tend to focus on doing this statically rather than dynamically.

Thus, resources are targeted towards bringing datasets together and analysing them at a particular point in time, and the output of this process forms the basis of strategy. Dynamic data is instead organised so that datasets interact and are reported on in real time. Analysis becomes continuous feedback, rather than a snapshot, and the relationship between strategy and data itself becomes dynamic.

Initiatives such as integrated Customer Relationship Management systems can create these links internally, but there are both extra benefits to be realised and challenges to be faced when linking data held by different organisations – not just access, confidentiality and consent, but the technicalities of making those links. Local authorities thus need to target their strategic thinking, and allocate resources, at the beginning of the data-collection process, asking not only how they currently want their data to interact, but how it could be organised to make it as open as possible to future coordination with other agencies.

Is this realistic for local authorities at present? I’m carrying out a research project for SOLACE looking at good practice in storing and sharing data, and would be very interested to hear of further examples, particularly of dynamic inter-agency data sharing, in comments or by email at

Elissa Rospigliosi is a National Management Trainee at Dacorum Borough Council and a participant on the SOLACE Foundation’s Future Leaders’ Scheme, SOLACE Springboard.


Leave a comment

Filed under Learning and development, Research, Uncategorized

From ‘ivory tower’ to engagement: : strengthening links between universities and local authorities on carbon emission reduction

Posted by Dr Rehema White, Academic in Sustainable Development, School of Geography and Geosciences, University of St Andrews

Reflections on an ESRC/SFC/LARCI funded project with a programme to strengthen links between universities and local authorities

Our project, Enhancing Local authorities Community Engagement: Co-designing& Prototyping Strategies for Carbon Emission Reduction, was partly supported by Fife Council. It was awarded a grant within the Engaging Scottish Local Authorities Programme, funded jointly by the UK’s Economic and Social Research Council (ESRC), Scottish Funding Council (SFC) and the then Local Authorities and Research Councils’ Initiative (LARCI). The goals of the programme were to establish better relationships between Universities and Local Authorities and to promote applied and topical research in Scotland.

In order to minimise the impacts of climate change, Scotland has ambitious targets to reduce carbon emissions, and, of course, local authorities have responsibilities to help meet these targets. The aim of this project was for Fife Council and academics from the Universities of St Andrews and Dundee to develop together a strategy for local authorities to engage local communities in reduction of their carbon footprints. Existing information was collated and synthesised. Seminars on Energy, Transport, Food and Community brought together local people, local authority staff, academics and NGO representatives to highlight successful community projects and explore how local authorities could more effectively help local people (for example, through strategic planning for community renewables, providing a single contact person, jointly identifying priorities). Workshops, meetings, interviews and attendance at other events further informed the research. We co-designed a Community Engagement for Carbon Emission Reduction (CECER) Strategy for Fife Council and disseminated findings across other local authorities, partly with the assistance of Sustainable Scotland Network. The six key areas were:

(1) Local authorities shifting more from service provision towards community support

(2) Building community resilience

(3) Creating infrastructure and processes to help community action

(4) Supporting poorer communities

(5) Strategically altering budget distributions because of financial constraints and investment potential in renewables

(6) Raising awareness through seeing the local authority itself as a community of interest

It was concluded that local authorities can strengthen networks and communication, especially with successful communities in their area. Despite the different cultures between local authorities and communities, in both, key individuals played important leadership roles. A shift from “service provision” to “community enabling” will help local authorities mobilise the potential of communities to respond to climate change and other sustainability challenges.

What was it like to ‘step down from the ivory tower?’ Well, many academics are now actively engaged in practical activities. I had sat on the Fife Environmental Partnership for 3 years prior to this project. Certainly pre-existing as well as carefully nurtured trust, relationships and understanding assisted us in working together. We also found that real partnership across HEIs and local authorities was facilitated by recognition of the excellence and values of each institution. We respected our differences! As academics, we were impressed by the management efficiencies of local government processes; our initial theoretical attempts at strategies required action plans and SMART (Strategic, Measurable, Achievable, Realistic and Time bounded) targets. We had to learn to change our language and not assume knowledge of literature or acronyms. We had to work to different timescales. Whilst we often wanted to promote longterm learning and work towards longterm goals, local authorities needed to engage in the short term, often in response to policy initiatives and procedural deadlines.  Our project benefited from a complex but inclusive and effective governance approach. Finally, we learnt that we would not get far without buy in from senior managers. It was challenging but interesting work, and I recommend that you try to involve academics from your neighbourhood university in what you do. Not only might you learn new ways of thinking, you might inspire locally relevant research to help you make more informed decisions.

Research and evidence-based policy making will be a key theme of the SOLACE Summit in Coventry on October 16-18th 2012. It will include a workstream that will cover why evidence is important to local authorities, applying evidence in the real world and using evidence to generate savings and better outcomes.  More information and details of how to book are available here.

Leave a comment

Filed under Carbon reduction, Research, Scotland