Can AI Chatbots anticipate student misconceptions?
The capabilities of AI in September 2024 – Part 1
Do you set exclusively paper-based homework at your school and are state-funded and UK based? We’re looking for new schools to take part in our Year 7 research project to test the effectiveness of Eedi whilst doubling the rate of maths learning for disadvantaged students. If your school sounds like the perfect candidate, you have one week to apply by booking a call with the team by clicking here. There are 4 spaces left and it starts this month.
***
This is Part 1 of a trilogy of posts looking at AI's capabilities of supporting teachers with their planning as of September 2024. The posts are:
1. Can AI chatbots anticipate student misconceptions? (This post)
2. Can AI chatbots write a good multiple-choice diagnostic question?
3. Can AI chatbots plan a maths lesson?
I'm excited to revisit this series of posts in 12 months to see what advances have been made.
***
Introduction
I will test AI’s current capabilities to do three things that will aid teacher planning. And by aid, I am not just talking about saving time (which is, of course, important) but doing so without sacrificing - or by hopefully improving - the quality of the materials produced.
I will test AI’s current capabilities to perform what I consider to be three tasks of increasing complexity, starting with anticipating student misconceptions.
Why is anticipating student misconceptions important and why is it hard?
If teachers cannot anticipate students’ misconceptions and mistakes ahead of time, they leave themselves vulnerable to surprises in the classroom. Teachers then have to try to interpret and deal with a student's surprising wrong answer amidst the noise of a busy lesson, which rarely leads to an optimal response. But flip this around – if teachers can anticipate where students might struggle, they can plan for that struggle before they step into the classroom. This might mean shaping their explanation or choice of examples to tackle a misconception head-on, or planning a response, such as a counter-example, to a certain mistake or misconception if it should occur.
Anticipating students’ misconceptions and mistakes is hard for everyone, but especially for novice teachers. They are likely to have spent their school lives in top sets surrounded by students who “got” maths, and thus, they may find it difficult to predict where and how others might struggle. It took me many iterations of teaching topics over several years to get anywhere decent at predicting where my students might come unstuck.
My attempt
I wanted to see how good a selection of AI chatbots are at anticipating student misconceptions regarding subtraction involving negative numbers. So I had something to compare their output against, I had a go myself. Here are some of the things I have seen students do over the years:
Missapply the rule “two minuses make a plus” to any two minus signs. So, -7 - 3 becomes 7 + 3.
Travel in the wrong direction along the numberline, so 3 - (-2) = 1
Skip 0 when counting, so 2 - 5 = -4
Always do the biggest number subtract the smallest number, so 3 - 5 becomes 5 - 3 = 2.
Are there any others you would add to your list?
The AI Chatbots
For this exploration, I used four of the most popular AI chatbots available at the moment:
All are free to use once registered.
The prompt
Those of you who have dabbled in the world of AI Chatbots will know all about prompts. A prompt is needed to get a response from the bot, and writing a good one is quite a challenge. You need enough specificity, without being too leading.
Here is the prompt I gave to all four chatbots:
I am a maths teacher about to teach my Year 7 students to subtract negative numbers. I would like you to give me a list of the most common mistakes my students will likely make and misconceptions they may hold in order of prevalence.
For each mistake or misconception, please provide:
1. a concrete example of the mistake or misconception so I can recognise it
2. an explanation of why students do this
3. an explanation I can use to help students understand better
4. a question I can use to check my students’ have understood
Let’s see how they did.
Chatbot #1: Gemini
Gemini offers four suggestions:
1. Misconception: Subtracting a negative is the same as adding a positive.
Example: 5 - (-3) = 5 - 3
Why students do this: This is a common extension of the rule "two negatives make a positive" when multiplying or dividing. Students may mistakenly apply this rule to addition and subtraction.
Explanation: While it's true that multiplying or dividing two negatives results in a positive, subtraction is a different operation. Subtracting a negative is like taking away a debt, which is essentially the same as adding money.
Check question: "If you have 5 apples and someone takes away -3 apples, do you have 2 apples or 8 apples?"
2. Mistake: Incorrectly applying the "Keep, Change, Flip" method.
Example: 5 - (-3) = 5 + (-3) = 2
Why students do this: The "Keep, Change, Flip" method is often used for dividing fractions, and students may incorrectly apply it to subtraction.
Explanation: While the "Keep, Change, Flip" method is helpful for division, it doesn't apply to subtraction. When subtracting a negative, we simply change the subtraction to addition and the negative to a positive.
Check question: "Can you explain why '5 - (-3)' is the same as '5 + 3'?"
3. Mistake: Forgetting to change the sign of the second number.
Example: 5 - (-3) = 5 - 3
Why students do this: Students may simply forget to change the sign of the second number when subtracting a negative.
Explanation: Subtracting a negative is like adding its positive counterpart. Therefore, we must change the subtraction sign to an addition sign and the negative sign to a positive sign.
Check question: "What is the first thing you do when you see a subtraction problem like '5 - (-3)'?"
4. Misconception: Subtracting a larger number from a smaller number always results in a negative answer.
Example: 3 - 5 = -2
Why students do this: This misconception may stem from previous experiences with subtraction, where the minuend is always greater than the subtrahend.
Explanation: Subtracting a larger number from a smaller number results in a negative difference. This is essentially the same as borrowing money. For example, if you have $3 and you spend $5, you now owe $2.
Check question: "Is it possible to get a positive answer when subtracting a larger number from a smaller number? Explain why or why not."
On the surface, these look great - If you specify a response structure to a Chatbot, they are very good at complying. But when you dig a bit deeper, problems emerge. Here are three:
"If you have 5 apples and someone takes away -3 apples, do you have 2 apples or 8 apples?" is probably the worst concrete example I have ever heard
I have never encountered a student who mistakenly did keep, flip, change when subtracting negatives. That doesn’t mean it is not a valid one, but it seems a stretch.
The example of the misconception given in number 4 is a correct answer
To try Gemini yourself, click here.
Chatbot #2: ChatGTP
ChatGTP offers 5 possibilities, along with a reassuringly cheery opening:
Certainly! Teaching Year 7 students to subtract negative numbers can be tricky, and it's great that you're preparing for common mistakes and misconceptions. Here’s a list of common issues, ordered by prevalence, with examples, explanations, and strategies to help your students understand better.
1. Misconception: Subtracting a negative number is the same as subtracting a positive number.
Example:
Students might incorrectly solve 7 − (−3) as 7 − 3 and get 4 instead of the correct answer, 10.
Explanation:
Students often confuse subtracting a negative number with simply subtracting a positive number because they may not fully grasp the concept of “subtracting a negative” as “adding the positive equivalent.”
Explanation to Help Understand:
Explain that subtracting a negative number is the same as adding the positive version of that number. You can use a real-world analogy: if you owe someone $3 and then your debt is reduced by $3, you’re actually gaining $3. So, − (−3) means you are adding 3.
Question to Check Understanding:
If you have 15 − (−4), how would you solve it, and what is the result?
2. Misconception: Adding a negative number always results in a smaller number.
Example:
A student might incorrectly solve 5 − (−2) as 5 + (−2) and get 3 instead of the correct answer, 7.
Explanation:
Students might confuse the operation of subtracting a negative with adding a negative number. They think subtracting a negative number will decrease the original number, similar to adding a negative number.
Explanation to Help Understand:
Clarify that subtracting a negative number actually increases the value because it’s like adding a positive number. Illustrate this with number lines or practical examples.
Question to Check Understanding:
If you subtract −6 from 2, do you get a number larger or smaller than 2? What is the result?
3. Mistake: Not applying the correct sign to the result.
Example:
Students might solve −3 − (−5) and get −8 instead of 2.
Explanation:
Students may make errors with signs, either by forgetting to change the signs when subtracting a negative number or by misunderstanding how signs interact in subtraction.
Explanation to Help Understand:
Use the rule: “Subtracting a negative number is the same as adding its positive counterpart.” Emphasize how the signs should change, and use visual aids like number lines to show how moving in the positive direction affects the result.
Question to Check Understanding:
What is −4 − (−7)? Show your steps and explain why the result is what it is.
4. Misconception: Treating subtraction of negatives as a separate rule from addition.
Example:
A student may handle −2 − (−3) differently from how they handle −2 + 3, leading to incorrect results if they don't see the relationship between these operations.
Explanation:
Students might not realize that subtracting a negative number is essentially related to addition and that the rules of arithmetic apply similarly.
Explanation to Help Understand:
Reinforce that subtracting a negative number always means adding the positive equivalent, linking it directly with addition. Create practice problems where they first add the positive number and then see the pattern.
Question to Check Understanding:
If you have −8 − (−4), how is this operation related to addition? Solve the problem and explain the relationship.
5. Mistake: Misinterpreting the negative signs in multi-step problems.
Example:
In a problem like −4 − (−3) − (−2), students might incorrectly handle the multiple negatives, resulting in an incorrect final answer.
Explanation:
Students may get confused by multiple negative signs and how to correctly apply the rules for each step.
Explanation to Help Understand:
Break the problem down into smaller steps, solving each operation one at a time. Use clear and structured steps to show how each negative subtraction is handled. Practice with simpler problems first before moving to more complex ones.
Question to Check Understanding:
How would you solve −5 − (−2) − (−3)? Solve it step by step and explain how each negative affects the result.
These strategies should help address common pitfalls and improve your students' understanding of subtracting negative numbers. Good luck with your teaching!
My first impression is that ChatGTP is good at this. Very good at this. Better than Gemini, and in fact, probably better than me. I like the way it frames the first three misconceptions, and I did not think of the final two misconceptions, but these seem worth considering. Most impressive of all is the explanation it offers to teachers to help their students understand - I found these genuinely useful. The subsequent check for understanding questions are also great - they don’t appear to be a copy-and-paste job, but look to be carefully through thorough. Yes, this is good stuff.
To try ChatGTP yourself, click here.
Chatbot #3: Co-pilot
Co-pilot starts bright and breezy too:
Certainly! Teaching students to subtract negative numbers can be challenging, but understanding common mistakes and misconceptions will help you address them effectively. Here are some prevalent issues your Year 7 students might encounter when subtracting negative numbers, along with explanations and strategies to overcome them:
Misinterpreting Arithmetic Signs:
Example: When faced with the expression -3 - 2, students might mistakenly interpret it as “negative three minus two,” resulting in an answer of -5.
Explanation: Students often confuse the subtraction sign with the addition sign. They forget that subtracting a negative is equivalent to adding a positive.
Helpful Explanation: Remind students that subtracting a negative number is the same as adding its positive counterpart. For instance, -3 - 2 is equivalent to -3 + (-2), which equals -5.
Check Understanding: Ask students to evaluate -3 - 2 and explain their reasoning.
Believing Negative Numbers Are Smaller Than Zero:
Example: Students may think that -3 is larger than 2 because it has a minus sign.
Explanation: In mathematics, “smaller” refers to the value, not the size. Negative numbers are less than zero, so -3 is actually smaller than 2.
Helpful Explanation: Emphasize that negative numbers represent values below zero. When comparing, focus on the numerical value, not the sign.
Check Understanding: Ask students to compare -3 and 2, explaining which is smaller.
Forgetting the Rules for Multiplication and Division:
Example: Students might incorrectly multiply or divide negative numbers.
Explanation: They forget that the product of two negatives is positive, and the quotient of two negatives is also positive.
Helpful Explanation:
Multiplication: -3 × -2 equals 6 because two negatives make a positive.
Division: (-6) ÷ (-3) equals 2 because dividing two negatives yields a positive result.
Check Understanding:
For multiplication: Ask students to find -4 × -5.
For division: Ask students to calculate (-12) ÷ (-4).
Remember to use visual aids like a number line to reinforce concepts. Regular practice and discussions will help students gain confidence in working with negative numbers. Good luck with your teaching! 📚🔢
Right, Co-pilot has a few issues here:
The example given for misconception #1 is a perfectly correct way of thinking about the problem: When faced with the expression -3 - 2, students might mistakenly interpret it as “negative three minus two,” resulting in an answer of -5.
Misconception #2 - Believing Negative Numbers Are Smaller Than Zero - is a weird one. At first I thought it was making a point about magnitude, but I don’t think it is, and in fact is just wrong.
Misconception #3 is not useful for the concept I asked it to think about - subtracting with negative numbers
To try Co-pilot yourself, click here.
Chatbot #4: Claude
Claude dives straight in:
Here's a list of common mistakes and misconceptions students often have when learning to subtract negative numbers, along with the information you requested:
1. Ignoring the subtraction sign
Example: 5 - (-3) = 2 (incorrect)
Why students do this: They focus on the negative sign of the second number and ignore the subtraction operation, treating it as addition instead.
Explanation for students: When we subtract a negative number, it's the same as adding its positive counterpart. Think of it as removing a debt - if you remove a $3 debt, you gain $3.
Check question: What is the result of 7 - (-4)? Explain your reasoning.
2. Incorrectly applying the "two negatives make a positive" rule
Example: -8 - (-5) = -13 (incorrect)
Why students do this: They've learned that multiplying two negatives gives a positive, and overgeneralize this rule to subtraction.
Explanation for students: In subtraction, we're not combining two negatives. We're taking away a negative number, which is like adding its opposite. Imagine walking backwards (-8 steps) and then deciding not to walk backwards 5 steps (-(-5)). You'd end up 5 steps closer to where you started.
Check question: Calculate -10 - (-6). Draw a number line to show your reasoning.
3. Confusion with the order of operations
Example: 4 - 7 - (-2) = -5 (incorrect, they calculated (4-7) first, then subtracted -2)
Why students do this: They may not understand that subtraction should be performed from left to right when no other operations take precedence.
Explanation for students: When we have multiple subtractions, we work from left to right. Let's break it down step by step: First, 4 - 7 = -3, then -3 - (-2) = -3 + 2 = -1.
Check question: Solve 8 - 3 - (-5) step by step, explaining each step.
4. Misunderstanding the concept of "less than"
Example: -7 is less than -3, so -7 - (-3) = -10 (incorrect)
Why students do this: They confuse the idea of a number being "less than" another with it being "further to the left" on a number line.
Explanation for students: When we subtract, we're finding the distance and direction between two numbers. On a number line, -7 is 4 units to the left of -3. So -7 - (-3) is the same as -7 + 3 = -4.
Check question: Which is greater: -5 - (-2) or -5 - (-8)? Explain using a number line.
5. Difficulty with mixed positive and negative numbers
Example: 6 - (-2) - 3 = 1 (incorrect, they calculated 6 - (-2) correctly but then subtracted 3 incorrectly)
Why students do this: They may correctly apply the rule for subtracting a negative but then struggle to continue the calculation with a positive number.
Explanation for students: Let's break this down step by step. First, 6 - (-2) = 6 + 2 = 8. Now we have 8 - 3, which is straightforward subtraction, giving us 5.
Check question: Calculate 9 - (-4) - 5. Show your work and explain each step.
By addressing these common misconceptions and mistakes, you can help your students develop a stronger understanding of subtracting negative numbers. Remember to use visual aids like number lines and real-world analogies to reinforce these concepts.
Claude does some good things here. It is the only Chatbot to pick up on the misapplication of the two minuses make a plus rule. Whilst I am not convinced the example given in #3 is an issue with the order of operations, I can definitely see one of the minus signs going missing when students work through problems like this, and it is not an example I had on my list. I also think its explanations for students are nice, using concrete examples when needed. I don't know where the answer of 1 comes from in the example in misconception #5, and misconception #4 as a whole feels a bit muddled, but apart from that it is good stuff.
To try Claude yourself, click here.
So, can Chatbots anticipate students’ misconceptions in a useful way for teachers right now?
Definitely. ChatGTP in particular. Imagine having an output like that for each concept in your scheme of work. Not just the standard list of misconceptions - which, in fact, is rare enough in the schemes of work I see - but something actually useful in addition to those: explanations of why students go wrong in these ways, things you can try to help them understand better, and a question to ask to check that things now make sense. If teachers engaged with this output, then you have something really useful.
Let’s not forget how long the output took to produce. Once that prompt was written, ChatGTP banged it out in less than 5 seconds. And of course, that prompt can be copied and pasted for any topic.
How long did my (inferior) list take me? Well, it took me 5 minutes to write but 20 years of experience to create. Teachers don’t have that time, and kids can’t afford to wait.
But, of course, a word of caution. These AI Chatbots are incredibly sure of themselves. The confidence in which Co-pilot proclaims that Believing Negative Numbers Are Smaller Than Zero is a common misconception students hold, or how Gemini states, without hesitation, that 3 - 5 = -2 is wrong, brings to mind a dodgy used car salesman declaring that the old banger he is trying to flog is an absolute steal. “Confident bullshitters” is an apt phrase for the current state of some of these AI Chatbots, and it is easy to get suckered in.
However, I still suggest you experiment with this. Try it for a topic. Use my prompt or tweak it accordingly. Sense-check it more than once. Then share the output with colleagues and discuss how it could impact your planning and teaching.
Want to know more?
Over the next two weeks, we will look at more complex tasks, so stay tuned for that. In addition, I heartily recommend signing up to Neil Almond’s free teacher-focussed AI newsletter. Each week, he compiles the latest AI developments and also shares a prompt you can use right away that might save you time, help you do your job better, or maybe both!
Have you used a Chatbot to aid your teaching?
What worked, and what didn’t?
Let me know in the comments below!
🏃🏻♂️ Before you go, have you…🏃🏻♂️
… checked out our incredible, brand-new, free resources from Eedi?
… read my latest Tips for Teachers newsletter about asking students to write their names on their mini-whiteboards?
… listened to my most recent podcast with Ollie Lovell about a recent lesson he taught?
… considered booking some CPD, coaching, or maths departmental support?
… read my Tips for Teachers book?
Thanks so much for reading and have a great week!
Craig