How can evidence-based education programmes be effectively scaled up in England?

A case study

At a glance

Education programmes proven to improve academic performance need to be implemented across lots of schools to achieve the greatest positive impact. Many fail to scale up this way.

BIT partnered with the Education Endowment Foundation (EEF) and the Schools, Students, and Teachers Network (SSAT) to evaluate the process of scaling the Embedding Formative Assessment (EFA) programme. We also acted as a learning partner, offering regular feedback and capacity building to support the scaling effort. Over four years, we uncovered and addressed seven areas that were critical to scaling up effectively.

More than 140 schools were recruited to the programme. Examples of our evaluation findings were that government subsidies increased school participation and that good monitoring and evaluation by the delivery organisation is crucial. Our recommendations were put into action and now serve as a foundation for how to scale interventions across the English school system.

"This project highlighted just how hard it is to scale evidence-based programmes in schools - and any sector really. To do it well, delivery organisations, and capacity building organisations like ours, need a diverse toolkit that combines programme, strategy and organisational development with knowledge of the policy context and market in which you’re trying to scale. But this complexity and focus on implementation is why we love working on projects like this."

Patrick Taylor, Principal Research Advisor, BIT

The challenge

Many education programmes have been proven to improve student outcomes in large-scale randomised controlled trials, but have failed to scale up due to a host of factors.

This is a big problem. Programmes that are known to improve teacher practice and student learning are not reaching all the schools, educators and children that need them.

The EFA programme was an example of this.

Well-implemented formative assessment can transform teaching and learning. It covers a set of practices in which teachers gather evidence about students’ progress and understanding. They then use that evidence to inform what they do next, such as whether to continue with a topic or to move on to another.

The EFA programme was created by the SSAT. It’s a two-year professional development programme to help schools embed formative assessment practices. When the programme was tested in 140 English schools, an impact evaluation showed that students in intervention schools made two additional months’ progress in their academic performance versus students in comparison schools.

While there was strong evidence that the programme boosted attainment, implementing it well at scale was not guaranteed. Among many other things, successful scaling often requires programmes to be adapted and for significant organisational and management changes to happen, including expanding the workforce, automating processes and increasing marketing efforts.

Without rigorously evaluating and supporting the EFA programme’s scale-up, it ran the risk of falling by the wayside.

What we did

From 2019 to 2023, BIT partnered with the EEF and SSAT to evaluate the process of scaling the programme. We also served as a learning partner for SSAT, offering regular feedback and capacity building to support the scaling effort.

We applied our expertise in rigorous evaluation, scaling and deep knowledge of the English education system throughout this work. These skills enabled us to pinpoint what it takes to effectively scale up interventions in schools, while providing practical and timely insights to help implement the EFA programme widely.

The research covered seven topics that are critical to scaling, with detailed questions under each topic.

We used a mixed-methods approach to answer these questions. We gathered both quantitative and qualitative data from the central team at SSAT, the staff responsible for training and supporting new schools and the schools themselves.

We combined administrative data and surveys with case studies of 10 schools implementing the programme. The case studies wove together insights from observations of the programme in practice and interviews with teachers and school leaders.

Our team collected quantitative and qualitative data at the same time, analysed them separately and then integrated them during final interpretations. This allowed us to compare and contrast insights, triangulating them to help address the research topics more comprehensively.

The impact

Our evaluation generated extensive and detailed recommendations for how SSAT could increase the EFA programme’s scalability. As of September 2024, 350 schools are implementing EFA. A Department for Education subsidy to the fee paid by schools helped triple the sales conversion rate (of expressions of interest to sign-ups) from 4% to 12% in the final year of the evaluation. We recommended that similar subsidies be used to support scaling EFA and other evidence-based programmes where cost is a barrier to participation.

We also found evidence suggesting that the programme became embedded in schools. Close to 95% of teachers surveyed said that they had used EFA strategies, and more than half of them reported changing their general teaching practices as a result of the programme.

Overall, EFA was implemented closely to the core programme with few deviations. Most adaptations that were made were considered valuable and in-line with the programme's theory of change.

But monitoring fidelity was not as straightforward. For instance, 100% of SSAT’s frontline staff believed their schools were running EFA professional development sessions for the correct amount of time, but only 40% of teachers agreed. This inconsistency shows how important robust monitoring and evaluation is to scaling up effectively.

SSAT enacted many of the recommendations from BIT’s evaluation reports. For example, with our support, they improved their monitoring and evaluation systems and processes. We also helped them apply behavioural science to enhance their marketing materials.

What we learnt

More attention needs to be paid to scaling

All key stakeholders - schools, school networks, intervention developers, policy makers, and researchers - need to think about and plan for scale much more. This thinking needs to start early on in a programme’s life. This scaling framework, developed by CEI and BIT and funded by the EEF, is designed to support this thinking.

Provide continuous support

Acting as a ‘learning partner’ and providing regular feedback to the delivery organisation was crucial. This went beyond a standard evaluation and helped to build capacity and apply timely, practical insights to the scaling effort in real time.

Subsidies can drive participation

We found that a government subsidy to schools was a powerful tool for overcoming financial barriers to participation. The sales conversion rate of interested schools more than tripled after the subsidy was introduced, demonstrating that the cost of a programme can be a significant barrier to scaling up. However, subsidies need to be carefully delivered. Growth targets that accompany such subsidies can stymie scaling efforts if they draw focus away from the organisational and programme developments required to support scaling.

More attention needs to be paid to scaling

All key stakeholders - schools, school networks, intervention developers, policy makers, and researchers - need to think about and plan for scale much more. This thinking needs to start early on in a programme’s life. This scaling framework, developed by CEI and BIT and funded by the EEF, is designed to support this thinking.

Provide continuous support

Acting as a ‘learning partner’ and providing regular feedback to the delivery organisation was crucial. This went beyond a standard evaluation and helped to build capacity and apply timely, practical insights to the scaling effort in real time.

Subsidies can drive participation

We found that a government subsidy to schools was a powerful tool for overcoming financial barriers to participation. The sales conversion rate of interested schools more than tripled after the subsidy was introduced, demonstrating that the cost of a programme can be a significant barrier to scaling up. However, subsidies need to be carefully delivered. Growth targets that accompany such subsidies can stymie scaling efforts if they draw focus away from the organisational and programme developments required to support scaling.