Can AI chatbots plan a maths lesson?

The capabilities of AI in September 2024 – Part 3

This newsletter is made possible by Eedi. As we kick off the new academic year, we're excited to introduce some fantastic new features on Eedi to make setting up your classes a breeze. We've added Google Classroom syncing (read more here) and improved Wonde syncing (read more here), making it easier than ever to get started or re-sync your existing classes. And remember, Eedi is completely FREE—switch to us and save money while simplifying your classroom setup!

***

This is Part 3 of a trilogy of posts looking at AI's capabilities of supporting teachers with their planning as of September 2024. The posts are:

1. Can AI chatbots anticipate student misconceptions?

2. Can AI chatbots write a good multiple-choice diagnostic question?

3. Can AI chatbots plan a maths lesson? (This post)

I'm excited to revisit this series of posts in 12 months to see what advances have been made.

***

Introduction

I will test AI’s current capabilities to do three things that will aid teacher planning. And by aid, I am not just talking about saving time (which is, of course, important) but doing so without sacrificing - or by hopefully improving - the quality of the materials produced.

I will test AI’s current capabilities to perform what I consider to be three tasks of increasing complexity. Two weeks ago the Chatbots did really well at anticiapting student misconceptions, but last week they struggled with writing a multiple-choice diagnostic question. So now we come to the ultimate challenge: planning an entire lesson.

Why is planning a lesson important, and why is it hard?

The old saying goes that proper preparation prevents poor performance. A well-planned lesson helps you assess where your students are at, fill in any gaps, and then build new knowledge upon these foundations. Moreover, a well-planned lesson allows you to respond to your students' changing needs, offering support and challenge when needed.

Lesson planning was the biggest consumer of my non-contact time for the first ten years of my teaching career. And the outputs of all that time spent were nothing to shout about. Looking back, I spent too much time creating or searching for bright and shiny things - animations, colours, cutting and sticking - and not enough time on what really matters when it comes to learning: clear and concise explanations, carefully planned checks for understanding, well-chosen and sequenced examples, and activities that help all students think hard about the core concept.

Lesson planning is a skill that I learned through experience, with many points of failure along the way. And while I struggled, my students’ learning suffered.

My attempt

I think a lot about lesson planning these days. Planning an upcoming lesson is an ever-present part of the rehearsal stage of coaching and often appears in the discussions with the heads of the departments I support. In a few weeks, I will write a post about a departmental lesson structure I have devised for a school I support. But here I will simply say that I think good maths lessons have the following components:

1. A retrieval Do Now

2. Prerequisite knowledge checks

3. An explanation / worked example

4. Well-chosen independent practice

With each phase typified by mass participation, well-chosen questions to check for understanding, followed by appropriate responses by the teacher.

The AI Chatbots

So, I tried to plan a lesson using our favourite foursome from the previous two weeks: Gemini, ChatGTP, Co-pilot and Claude. But the results were underwhelming. This is no surprise. The jump from anticipating misconceptions to writing diagnostic questions was too far for most of the Chatbots. You can expect a significant struggle when you consider what is needed to plan a complete lesson.

Fortunately, there is an alternative. Oak National Academy - the organisation that sprung up during the pandemic to offer free videos and lessons to the millions of students learning from home - has created Aila.

Aila is a free, AI-powered lesson assistant that promises to help you create a tailor-made lesson plan, complete with resources, in minutes.

Sounds too good to be true? Let’s see…

The prompt

I wanted a lesson for Year 9 students studying reverse percentage problems. So, questions like this:

Felicity buys a dress in a sale. It is reduced by 40% down to a price of £54. Work out the original price of the dress.

Aila has a nice interface for getting the ball rolling. You give the subject, year group, and lesson title:

Aila is immediately honest about its limitations:

The lesson subject you’ve chosen is in an area where AI tools like me don’t work at our best. I’m trying to get better on my accuracy here, but please double-check everything before using this plan in the classroom. Thank you!

Let’s see what it can come up with.

Lesson components - Part 1

We get two sections of our lesson plan by giving just that prompt: Learning outcome and Learning cycles. Here is what they look like:

What I like here is that we get the option - and this is true at each stage of the lesson creation process - to suggest modifications to the output:

Aila is not designed to do all the work for the teacher. It is a lesson planning assistant, and wants teacher input all along the way.

I had one request here:

For each of the learning cycles bullet points, please give an example of a question a student would have to answer

I have a big issue with vague learning objectives. I don’t think they are all that useful for teachers or students as they are crammed full of technical mumbo-jumbo and, as such, can be interpreted in many different ways. Fortunately, Aila was able to provide an example for each bullet point:

• Understand the concept of reverse percentages. Example Question: What is the original price of an item if its price after a 20% increase is £60?

• Apply reverse percentages to calculate original values. Example Question: If a computer's price after a 15% decrease is £425, what was the original price?

• Solve practical problems involving reverse percentages. Example Question: A shop offers a 10% discount on all items. If the final price after the discount is £36, what was the original price?

The variety of questions is not great, but I think this may be a symptom of the topic itself rather than Aila’s limitations.

Lesson components - Part 2

Next, Aila produced Prior knowledge, Key learning points, Misconceptions, and Keywords. Here is what each of those looks like:

This is impressive stuff. However, the danger with information like this - and I have seen this so many times on the schemes of work in the schools I support - is that it is just glanced at or not even looked at by teachers planning lessons. This is not as big an issue here as it has taken 30 seconds to produce - the thousands of hours teachers around the country have spent writing documents with prior knowledge lists, key learning points, misconceptions, and sets of keywords is frightening. But I wanted to see if Aila could produce something concrete and useful that I could use in a lesson.

So I gave it the following prompts:

Prior learning check

For each of the prior knowledge bullet points, please create a multiple-choice question that I can use to check students’ understanding, and a follow-up question that I can use to re-test that understanding if needed. Please order the questions so the follow-up question is next to the corresponding original question.

Here is the output:

1. What is 20% of 50?

• 10

• 15

• 20

2. What is 10% of 70?

• 14

• 17

• 7

3. Convert 0.75 to a percentage.

• 7.5%

• 75%

• 750%

4. Convert 0.25 to a percentage.

• 2.5%

• 25%

• 250%

5. If a shirt's price increases by 25% and the original price is £40, what is the new price?

• £45

• £50

• £55

6. If a jacket's price increases by 10% and the original price is £60, what is the new price?

• £65

• £66

• £70

7. How do you convert a percentage to a decimal?

• Add 100

• Divide by 100

• Multiply by 100

8. How do you convert a decimal to a percentage?

• Add 100

• Divide by 100

• Multiply by 100

9. What is 15% of 200?

• 25

• 30

• 35

10. What is 25% of 80?

• 15

• 20

• 30

11. If a calculator shows a result of 0.2, what percentage is this?

• 2%

• 20%

• 200%

12. If a calculator shows a result of 0.5, what percentage is this?

• 5%

• 50%

• 500%

This is impressive. I can use these questions right away in the classroom to check how secure my students’ understanding is. And if that understanding is not secure, I can explain and re-check for understanding with the corresponding follow-up question.

Key learning points

For each of the key learning bullet points, please give an example so I can understand exactly what you mean

Here is the output:

• Reverse percentages are used to find the original value before a percentage change. Example: If a product's price after a 20% increase is £60, the original price is calculated using reverse percentages.

• To find the original value, divide the final value by the decimal multiplier. Example: For a final value of £60 after a 20% increase, divide £60 by 1.20 to get the original value of £50.

• The decimal multiplier for a percentage increase is 1 plus the percentage as a decimal. Example: For a 20% increase, the decimal multiplier is 1 + 0.20 = 1.20.

• The decimal multiplier for a percentage decrease is 1 minus the percentage as a decimal. Example: For a 15% decrease, the decimal multiplier is 1 - 0.15 = 0.85.

Once again, a concrete example helps cut through the vagueness of a description, and Aila is pretty good at providing this.

Misconceptions

For each of the misconceptions, please could you also give an example of a child’s response who holds this belief?

Here is the output:

Reverse percentages involve subtracting the percentage from the final value

This is incorrect because reverse percentages require dividing the final value by the decimal multiplier, not subtracting the percentage. Example: A child might say, 'If an item's price after a 20% increase is £60, then the original price is £60 - 20% = £48.'

A percentage increase and decrease of the same value will return the original value

This is incorrect due to the compounding effect. The base changes after the first percentage operation, altering the outcome of the second. Example: A child might say, 'If an item increases by 20% and then decreases by 20%, it will go back to the original price.'

Multiplying by the percentage is the same as multiplying by the decimal multiplier

This is incorrect. Multiplying by the percentage itself does not convert it to a decimal form, which is necessary for accurate calculations. Example: A child might say, 'To find the original price after a 25% increase, you multiply by 25% instead of dividing by 1.25.'

This is as good - if not better - than the output of the AI Chatbots from episode 1 of this trilogy.

Keywords

Please can you give the etymology of each of the keywords

Aila didn’t bother responding to this. Maybe I was pushing my luck at this stage.

Lesson components - Part 3

With the background complete, Aila then moves into the meat of the lesson. Aila frames this around the concept of Learning Cycles.

Learning Cycle 1 is entitled Understanding Reverse Percentages, and its output looks like this:

Understanding Reverse Percentages

15 minutes

Teacher explanation

• Introduce the concept of reverse percentages and why they are useful.

• Explain that reverse percentages are used to find the original value before a percentage change.

• Use a worked example to demonstrate the process: 'If the final price after a 20% increase is £60, what was the original price?'

• Show the calculation step-by-step: Divide £60 by 1.20 to get £50.

• Highlight the importance of using the correct decimal multiplier for percentage increases and decreases.

Accompanying slide details

A step-by-step worked example showing how to calculate the original price from a final price after a percentage increase.

Image search suggestion

reverse percentage calculation example

Slide text

Reverse percentages are used to find the original value before a percentage change. Example: If a product's price after a 20% increase is £60, the original price is calculated using reverse percentages.

Check for Understanding

1. What is the original price of an item if its price after a 25% increase is £75?

• £60

• £65

• £70

2. If the final price after a 10% decrease is £45, what was the original price?

• £40

• £50

• £55

Practice

Calculate the original price for the following scenarios:

1. An item's price after a 15% increase is £69.

2. The final price after a 20% decrease is £80.

3. A product's price after a 30% increase is £104.

Feedback

Model answer: 1. £60, 2. £100, 3. £80. Divide the final price by the appropriate decimal multiplier to find the original price.

The remaining Learning Cycles are structured in the same way and cover:

• Applying Reverse Percentages

• Solving Practical Problems with Reverse Percentages

Again, this is good stuff, but I wanted to push Aila further. So, I gave the following prompts:

Please can you give a detailed script for each teacher explanation

Here is Aila’s output for Learning Cycle 1’s explanation

• Today, we are going to learn about reverse percentages. Reverse percentages are used to find the original value before a percentage change. This is useful in many real-life situations, such as calculating the original price of an item before a discount was applied.

• Let's start with an example: If the final price after a 20% increase is £60, what was the original price? To solve this, we need to divide the final price by the decimal multiplier for the percentage increase.

• The decimal multiplier for a 20% increase is 1.20 (1 + 0.20). So, we divide £60 by 1.20, which gives us £50. This means the original price was £50 before the 20% increase.

• It's important to use the correct decimal multiplier for percentage increases and decreases. For a percentage increase, the multiplier is 1 plus the percentage as a decimal. For a percentage decrease, the multiplier is 1 minus the percentage as a decimal.

• Let's go through another example together to make sure we understand.

Having been messing about with these Chatbots for some time now, it is all too easy to get a bit blase about what is happening here. At the click of a button, I have a clear and concise explanation for how to solve reverse percentage problems. Of course, it is not perfect. It is procedural and cries out for a bar model or equivalent visual representation. But it is a starting point. I can build from this, editing it rather than starting from scratch. For a less experienced teacher, that is gold.

Next, I wanted a bit more practice:

Please can you create 10 practice questions, with answers, for each practice section

This caused Aila no trouble at all.

1. An item's price after a 15% increase is £69. Answer: £60

2. The final price after a 20% decrease is £80. Answer: £100

3. A product's price after a 30% increase is £104. Answer: £80

4. An item's price after a 25% increase is £125. Answer: £100

5. The final price after a 10% decrease is £90. Answer: £100

6. A product's price after a 40% increase is £140. Answer: £100

7. An item's price after a 50% increase is £150. Answer: £100

8. The final price after a 5% decrease is £95. Answer: £100

9. A product's price after a 35% increase is £135. Answer: £100

10. An item's price after a 45% increase is £145. Answer: £100

But notice Aila’s obsession with answers of £100. This is not ideal, as the solution can be found by addition, subtraction or inspection. So, I wondered if Aila could change things:

Please can you change any practice questions that have a solution of £100 into questions where the answer is something different

Aila told me they had done this, but they were fibbing because none of the questions changed. I tried numerous different prompts, but couldn’t get a different result.

Finally, I wanted a bit more variety in the questions:

Please can you also create two challenge questions at the end of each practice section that will challenge my highest-acheiving students?

Here is the attempt:

11. An antique's price after a 60% increase is £320. What was the original price? Answer: £200

12. A car's price after a 75% increase is £5250. What was the original price? Answer: £3000

These were not quite the problem-solving questions I had in mind. Aila has obviously not been trained using Don Steward’s resources.

Lesson components - Part 4

Alia then offers to produce an Exit Quiz:

1. What is the original price of an item if its price after a 25% increase is £100?

• £125

• £75

• £80

2. If a jacket's price after a 30% decrease is £70, what was the original price?

• £100

• £120

• £90

3. A product is sold with a 15% discount, and the final price is £85. What was the original price?

• £100

• £80

• £95

4. If the final price after a 20% increase is £72, what was the original price?

• £60

• £70

• £75

5. A shop offers a 10% discount on an item, and the final price is £45. What was the original price?

• £40

• £50

• £55

6. If a computer's price after a 40% increase is £700, what was the original price?

• £500

• £600

• £650

There are still a few £100s knocking around in there, but at least we have some variety. And the multiple choice formats would allow me to assess the whole class's understanding if I didn’t have mini-whiteboards to hand.

Lesson resources

With the lesson planning process complete, I was able to download Word, pdf or Google Doc versions of the following:

1. Lesson plan

2. Starter quiz

3. Lesson slides

4. Worksheet

5. Exit quiz

So, can an AI chatbot plan a lesson?

Sort of.

I am impressed with some of the output. I particularly like the prior knowledge check, follow-up questions, and the concrete examples of misconceptions Alia produced. The sequences of the three learning cycles were logical and should lead to a coherent learning experience for students. This is definitely a time-saver. I have seen heads of department and senior teachers spend hours creating these tasks in July and through the summer holidays - I have been there myself.

I also like the script for the teacher explanation. Scripting takes time, so no-one does it. But I have listened to - and delivered - enough waffly, unfocussed explanations to realise just how important this part of the lesson planning process is. Aila provides a great starting point.

But the practice questions are concerning. The obsession with £100 is something a competent teacher putting this lesson would never do, and it could have significant implications for students’ understanding. Notice also the lack of decent problem-solving questions, despite my pushing Aila to add some challenges.

Hence, I must supplement the materials produced here with some tried and trusted sources. Probably Corbett Maths for better practice questions, and Median, Open Middle or my own Eedi for better problem-solving materials.

Also, whilst this was not a case of: “click one button, have a cup of tea, and voila, there is a perfect lesson” - Aila pushed me to check and make suggestions at each stage - I didn’t need to do this. I could have typed in my initial topic request, hit continue a few times, and 10 seconds later, my lesson would have spat out. So, this could have been the equivalent of grabbing the first thing thrown up by a Google search or a dip into a favourite website. This lesson planning tool does not solve the problem of lazy teachers.

I wasn’t happy with the output, so I gave prompts and made tweaks. Aila responded pretty well each time (apart from its obsession with £100), but I wonder how much experience and expertise a teacher needs to spot deficiencies and prompt Aila to make those changes.

Finally, Aila has some limitations at the moment. I asked it to produce a lesson on circle theorems, and whilst everything seemed great in the creation process, when I looked at the finished materials, all the diagrams were either grey boxes or blank spaces:

The creation of diagrams seems like an insurmountable challenge at the moment, which rules out some significant areas of mathematics. But I am sure it will be solved soon.

But let’s not lose sight of the positive. I went from having no lesson to having something I could work with and improve in 5 minutes. In the hands of an experienced teacher (or group of teachers), looking to make changes and improvements to an existing scheme of work - or even looking to start one from scratch - I have no doubts that a tool such as Aila will not only save countless hours, but also produce results not too far off the quality that those teachers could produce on their own.

Try Aila yourself. Pick an upcoming unit in your scheme of work, and see what Aila can do for you.

Want to know more?

In addition to the post in this series, I heartily recommend signing up to Neil Almond’s free teacher-focussed AI newsletter. Each week, he compiles the latest AI developments and also shares a prompt you can use right away that might save you time, help you do your job better, or maybe both!

I hope you have enjoyed this trilogy of posts. If you have a moment, please let me know in the comments.

Have you used a Chatbot to aid your teaching?

What worked, and what didn’t?

Let me know in the comments below!

🏃🏻‍♂️ Before you go, have you…🏃🏻‍♂️

1. … checked out our incredible, brand-new, free resources from Eedi?

2. … read my latest Tips for Teachers newsletter about writing a student’s answer on the board after a Cold Call?

3. … listened to my most recent podcast with Ollie Lovell about a recent lesson he taught?

4. … considered booking some CPD, coaching, or maths departmental support?

5. … read my Tips for Teachers book?

Thanks so much for reading and have a great week!

Craig