Evaluation of the Scottish Climate Challenge Fund

The Scottish Climate Challenge Fund (CCF) aims to help Scottish communities reduce their carbon emissions. Totalling £27.4m over three years (2008-11), the fund is designed to provide grants ranging from a few thousand Pounds up to £1m.

The fund is run by Keep Scotland Beautiful on behalf of the Scottish Government, working in partnership with a supporting alliance of organisations coordinated by the Sustainable Development Commission (SDC) Scotland. The alliance includes the Energy Saving Trust, Sustrans, and the Scottish Council for Voluntary Organisations (SCVO), amongst others, and these organisations work to ensure that communities are supported as they put their carbon emission reduction plans into action.

A total of 331 projects from 261 different communities received funding from the Scottish CCF. Broadly speaking, project activities relate to energy efficiency, energy generation, food, waste and transport. All projects have communities at their core, are located in Scotland, and aim to make a measurable and significant reduction in carbon emissions.

Objectives
The review had two main aims:

• To learn lessons about the critical success factors of particular community-led climate change projects. Building understanding of the factors that make community projects a success (and the potential pitfalls) will allow the Scottish Government to better support community initiatives in the future.
• To draw conclusions about projects’ specific impacts on local people, economy and landscapes. While the main focus of the fund is on carbon savings, this also entails exploring other impacts that might include local capacity building, building better social ties within communities, or bringing economic benefits to local areas

Brook Lyndhurst teamed up with Edinburgh-based carbon assessment firm Ecometrica who conducted carbon savings assessments for selected projects. Ecometrica also helped to identify ways in which carbon measurement could be streamlined and improved on future funds.

Methodology
Because of the size of the fund, and the breadth and inventiveness of funded projects, it was decided that a focused evaluation, drawing on specific case studies, would be the most productive methodological route. In consultation with Brook Lyndhurst, 20 projects were selected from a long list of 36 chosen by Keep Scotland Beautiful and the Scottish Government. The shortlisted projects were chosen on the basis that they were likely to yield useful learning for the fund as a whole when subjected to evaluative analysis.

The methodology employed sought to build up an insightful and balanced picture of the outputs and impacts of each project, alongside the barriers and challenges they faced and the factors which played a key role in the success of the project. At the same time, we sought to minimise the burden placed upon projects by the evaluation. It comprised:

• Initial site visits to the projects, including in-depth face-to-face interviews with each of the project managers;
• Interviews with project participants, including a proportion with staff, volunteers and key partners (both face to face and by telephone) numbering around 30 per project;
• Discussion groups with participants or members of the target audience;
• Attendance at project events and short surveys of attendees;
• Analysis of ‘success diaries’ kept by the projects to record key breakthroughs in their work during the evaluation period;
• Final telephone interviews with project managers at the conclusion of the fund; and
• Carbon assessments of eight of the projects, carried out by Ecometrica.

In order to ensure that the large volumes of qualitative data generated by this approach could be effectively analysed, the early stages of the project were spent drawing up an analytical framework which was closely linked to the Scottish Government’s evaluation objectives. This framework was used to inform the structure of topic guides and questionnaires, and fed into the spreadsheets used to record data. This is an approach used on several previous evaluations, including our review of NESTA’s Big Green Challenge and Big Green Challenge Plus.

View the final report here