How to raise attainment in mathematics
The results and takeaways from a large-scale RCT
Two weeks ago, I shared a paper about our work with Google’s LearnLM model, which demonstrated that AI-assisted human tutoring can be more effective than human tutoring alone.
This week, I want to share another paper we published, this time highlighting the results of a large-scale randomised controlled trial we carried out last year. You can read the full paper here.
The Paper at a Glance
Who was involved?
The trial involved 2,901 Year 7 students across 20 UK schools during the 2023-2024 academic year.
What was the setup?
This was an “active control” trial. In most studies, one group of students uses a new product, while the control group receives nothing. We wanted to improve this approach. Our intervention groups used Eedi for their homework, whereas the control groups continued with business as usual, which for 89% of them meant using platforms like Sparx or MathsWatch. This makes the results particularly robust, as Eedi had to demonstrate its value against tools that schools are already using.
What were the headline results?
The effect size across all students using Eedi was 0.17, which was statistically significant. According to the Education Endowment Foundation (EEF) conversion, this equates to 2 months of additional progress in a single year compared to the control group.
For students who used Eedi regularly each week for just 10 to 15 minutes, the impact doubled to an effect size of 0.34, or 4 months of additional progress compared to the control group.
These positive effects were seen across the board, including for Pupil Premium (PP) students who answered 120 questions across the year, and those with lower prior attainment.
What does this mean for classroom teachers?
The obvious takeaway is… use Eedi!
But I think the paper offers four takeaways that any teacher can apply, regardless of the homework or assessment platform you use.
1. Ensure homework feeds into lessons
For me, this is the big one, and it is something I discussed with Adam Boxer: supplementary data (whether generated digitally or through paper-based homework) is most valuable when it informs live teaching rather than being treated as a disconnected exercise.
Getting students to participate in any homework, especially online homework, is challenging. I often visit schools where fewer than 20% of students complete their online assignments. The teachers in our study who saw the greatest learning gains were those who used Eedi’s reporting insights in their lessons. They gave whole-class feedback on problematic questions or gathered specific students into small intervention groups based on shared misconceptions.
Once students realise that their homework and assessments influence what happens in lessons, they take both more seriously.
2. Use multiple-choice diagnostic questions
Long-time readers will know that I love a few things in life more than multiple-choice diagnostic questions. A good diagnostic question allows teachers to pinpoint precisely why students are struggling, not just that they are struggling.
Teachers in our study who actively used data about student misconceptions were able to make their lessons more targeted and responsive to student needs. For instance, 36% of teachers used the identified misconceptions to inform their future lesson planning. By knowing the specific nature of student misunderstanding—whether it’s an arithmetic error or a conceptual error—you can tailor your teaching to address the root issue.
You don’t have to use diagnostic questions for homework. You could use them as part of your Do Now, or to check prerequisite knowledge before teaching a new idea, or as part of an Exit Ticket to see what students have learned that lesson. The ease with which you can collect responses from all students and quickly interpret the results makes them a perfect tool for in-class formative assessment.
And of course, you can find 1000s of diagnostic questions on all subjects at diagnosticquestions.com, and tips on how to use them in this book.
3. Set shorter homework more frequently
The teachers in our study who saw the greatest learning gains were those who got their students using Eedi for regular, short sessions. 10-15 minutes a week was the sweet spot. This felt manageable to students – as opposed to some epic 1-hour homework – meaning more students engaged. And as the weeks went by, those 10 and 15 minutes added up to a lot of questions answered. Students who kept this up saw double the rate of progress compared to students who used Eedi less frequently.
The finding that shorter, more frequent homework leads to greater learning gains than longer, less frequent homeworks is well established (see this literature review for more). Some schools I have supported over the last few years have experimented by changing their homework schedules from, say, a one-hour homework a fortnight, or a 30-minute homework a week, to two 15-minute homework a week. In every case, the proportion of students completing the homework has risen.
4. Competition can boost motivation
Teachers in our study observed that leaderboards boosted student engagement. The key was to ensure that the same students were not always at the top, so focus was not only on achievement but also on effort and improvement. Who answered the most questions? Whose score improved the most since last week?
Public recognition can motivate students to keep putting in the effort and give a kick up the bum to those who need it most.
Summary
If this has made you want to check out Eedi, then you can find out more here.
But even if not, I hope this study gives you some practical ideas for boosting student participation in homework, and thus improves their learning.
What questions do you have?
What do you agree with, and what have I missed?
Let me know in the comments below!
Thanks so much for reading, and have a great week!
Craig
P.S. Check out my new 16-book series, The Tips for Teachers guides to…
