10 things we know about testing 📝
Insights from my new favourite research paper
This newsletter is made possible because of Eedi. Check out our brand-new set of diagnostic quizzes, videos, and practice questions for every single maths topic, ready to use in the classroom, and all for free, here.
Exciting news this week… brace yourselves… I have a new favourite research paper about retrieval. Woo hoo!
Yes, for many years, my go-to was the classic 2011 paper, Ten Benefits of Testing and Their Applications to Educational Practice, by Henry Roediger and colleagues. But that has been supplanted by a lovely paper entitled Testing (quizzing) boosts classroom learning: A systematic and meta-analytic review by Chunliang Yang and colleagues.
There are four main reasons why I like this paper:
It is recent, having been written in 2021
It is large scale, involving a review of 222 studies
It is both cross-subject and cross-phase
Most importantly of all, each of the studies included is classroom-based, whereas much of the research into retrieval is conducted in a lab setting.
Oh, and please don’t be put off by the word test in the title. Think instead of retrieval opportunities. In other words, the times we challenge students to think hard about concepts they have met in the past.
I would strongly advise you to read the paper for yourselves. However, I have picked out 10 questions about retrieval practice that this paper attempts to answer, along with my own reflections. We have incorporated these findings into our new Eedi Retrieval System that I will tell you about at the end of the post.
10 questions about testing… with answers!
Question 1: Is testing effective?
Answer: Yes!
Testing, by comparison with other strategies (e.g., concept mapping, restudying and note- taking) overall boosts student attainment (effect size = 0.499). This is known as the Testing Effect.
This is the finding that every reader will be aware of. If we want students to remember something, we need to provide opportunities for them to retrieve that knowledge at several points in the future. Sure, this feels much more challenging than simply re-reading notes or watching a video, but it is that very effort that cements the knowledge into long-term memory.
Question 2: Does the format of the test matter?
Answer: No!
Test-enhanced learning generalizes to a variety of test formats, including matching, Fill-in-the-blank, Short answer, Multiple choice, Cued recall, Free recall.
Two key takeaways for me here. First, the format of the retrieval opportunity does not need to match the format of the end exam. Second, this gives us an opportunity to vary the format of our retrieval opportunities. For example, I am a big fan of including multiple-choice diagnostic questions in homework or low-stakes quizzes.
Question 3: Do students need corrective feedback?
Answer: Yes!
Presenting corrective feedback following quizzing enhances the benefits of testing. Rowland (2014) found that corrective feedback doubles the benefits of testing.
This is obvious, but important. Students need to know whether they are right or wrong following the retrieval opportunity. This could be done in class, or at the point they complete the retrieval opportunity at home, via an online homework platform such as Eedi. This stops misconceptions from becoming embedded, and also provides the students with an opportunity to pause, reflect and self-explain.
Question 4: Do more tests lead to better results?
Answer: Yes!
The more occasions class content is tested, the larger the learning gains.
This does not mean a high-stakes test every lesson. But it does mean providing more retrieval opportunities than I certainly did early on in my career. These retrieval opportunities could be during the Do Now, when past content is woven into the current topic, for homework, or in a regular low-stakes quiz. The key point is ensuring students have lots of chances to think hard about concepts they have met in the past to ensure those concepts become embedded and accessible in their memory.
Question 5: Over what time period should we do retrieval practice?
Answer: Long and consistent!
Testing benefits systematically increased from single-class treatments, treatments lasting less than a semester, and treatments lasting a whole semester, to those lasting longer than a semester.
The key point here is that we need to move away from teaching and assessing concepts in a nice tidy box before moving on to the next idea. For retrieval to be effective, it needs to be for the long term. So, provide an opportunity for students to retrieve what they have just learned pretty quickly - perhaps in the Do Now of the next lesson - and then gradually space out those retrieval opportunities as time progresses, including them in homework, low-stakes quizzes, and move into new ideas.
Question 6: Is testing effective at all age groups?
Answer: Yes!
The testing effect occurs in primary (elementary), secondary (middle and high), and postsecondary (university/college) education.
This is good news, and one of the reasons I like this paper. Of course, the nature of the retrieval opportunity is likely to be different - a written, low-stakes quiz is unlikely to be suitable for very young students, for example. But the key principle of regularly testing students on concepts they have encountered in the past remains true.
Question 7: Is there a gender difference?
Answer: No!
Male and female students obtain comparable learning benefits from testing
Again, this is good news.
Question 8: Is testing effective in all subjects?
Answer: Yes!
The enhancing effect of testing applies across 18 academic subject categories.
The good news keeps coming! The nature of the retrieval opportunity is likely to differ across subjects, but remember to take into account the response to Question 2 and experiment with different formats for the retrieval opportunity.
Question 9: Is testing just good for facts?
Answer: No!
Testing not only enhances learning of facts but also facilitates knowledge comprehension and application
This is a common misconception that needs addressing. Retrieval opportunities are often made up of questions that test students’ ability to recall basic facts and procedures. But of course, they do not need to be. You can include any type of question in a retrieval opportunity. Indeed, in Tips for Teachers, I argue that you should include more challenging questions in retrieval opportunities, otherwise the only chance students have to think deep about a concept is when the concept is first being taught.
But even if retrieval opportunities are dominated by questions on facts and procedures, that is not a bad thing. One way to think about this is that without facts at their disposal, it is very difficult for students to engage in any higher-order thinking as they have nothing to build that thinking upon. Another important point is that facts and procedures are not learned in isolation, and an ability to recall them enables students to make connections between them.
Question 10: Do the stakes of the test matter?
Answer: No, but…
Low-stakes quizzes have the same effect on learning. Higher stakes quizzes may lead to more motivation… But Low-stakes quizzes may reduce test anxiety.
This is an interesting one. My teaching used to be dominated by end-of-unit high-stakes assessments, where scores would be recorded, grades assigned, and students could ultimately move sets as a result. Such infrequent, high-stakes tests are unlikely to provide enough retrieval opportunities to help students remember, and the stress they cause may inhibit enjoyment and ultimately learning in the long term.
Now I use lots of low-stakes quizzes. Students mark their own, and I never record their scores. I make sure my students realise that these quizzes are first and foremost a tool of learning, and when they see that these quizzes help them remember things, I find the motivation takes care of itself.
How retrieval practice works on Eedi
When we surveyed teachers to find out the feature they most wanted to see on Eedi, retrieval practice came out as number one. So, we have implemented it.
Once you have set three topic assignments for your students, you unlock the ability to set retrieval practice. Eedi learns exactly what each of your students needs, and responds accordingly. We set each student their very own, bespoke retrieval quiz, based on questions they have struggled with, but also topics they have mastered in the past to make sure they don’t forget things.
And just like all our quizzes on Eedi, students can resolve any difficulties by watching our high-quality worked example videos, they can secure their knowledge with our carefully designed fluency and intelligent practice questions, and finally master the concept with our stretch and challenge questions.
All of this is orchestrated for you and your students automatically once you assign retrieval practice.
Oh, and it is 100% free :-)
To get started, and help your students remember all the things they once knew, just head over to eedi.com
What retrieval opportunities do you provide your students with?
What challenges do you face?
Let me know in the comments below!
🏃🏻♂️ Before you go, have you…🏃🏻♂️
… checked out our incredible, brand-new, free resources from Eedi?
… read my latest Tips for Teachers newsletter about following tumbleweed with a Turn and Talk?
… listened to my most recent podcast about mastery?
… considered booking some CPD, coaching, or maths departmental support?
… read my Tips for Teachers book?
Thanks so much for reading and have a great week!
Craig
When I taught in the UK, I used to do an in-class test every second week, for all of my classes. Yes, I would record the scores for my own personal markbook, but the scores didn't count towards the school's assessment of the students. I found it worked really well... although I wish I knew more about the different ways of testing at the time!
When I moved to Australia, I found the culture very much against testing. One school (I no longer teach at) was infatuated with "pre-testing"... they gave students a test on a topic, which would usually waste an entire lesson, before the content was taught, expecting the results to be very bad. It did not particularly inform the "teaching" because even if every student in a class knows a topic, us teachers had to provide evidence that we had taught it anyway. But it allowed the school to show the post-topic test as evidence of how much the students had learned. Literally a scheme to protect teachers and nothing else. And it meant less time to actually teach.
Oh, and there was very little, if any, corrective feedback. That was a great skill I had learned from UK teaching, completely ignored at this Australian school.
Enough ranting... thanks for this article. It has reminded me of my need to keep pushing for better teaching methods, and I'm going to find more opportunities for "retrieval". (I love how the change of language makes people on board with an idea they've been trained to oppose!)