When Good Work Stops Meaning Good Thinking

Forty minutes into the exam, screen gone, phone gone, Harry (let’s call him Harry; you may not know this Harry but I’m sure you know a Harry or two) is stuck on the first question. He has to plan an essay with relevant key information, but the crutch he has used so effectively over the past few months to help him plan essays and prompt him to add key elements is not there today. The plan was always made with his input, though not always his thinking. Today there is only him, a pen, the paper and the clock.

Throughout the term, everything about Harry’s work had said he was doing well. That, it turns out, was part of the problem.

Let me rewind. Harry is in Year 11, in the middle of his GCSEs, the conscientious sort who worries about getting it right. He has a phone full of apps that will happily do his homework, a laptop with access to the internet, and nobody who has ever sat down and shown him how to use any of it well. He has been left to work that out for himself, which is exactly where most students his age are.

The numbers make him ordinary. Roughly two in three children aged thirteen to eighteen now use generative AI for their schoolwork (National Literacy Trust, 2025). In the schools I work with it runs higher. And the habitual users are not the strugglers or cheats we might picture. They are the most able (and often the most anxious), those in Year 11 and Year 13 with the most to lose and the most pressure to perform. In most cases they are the students we have traditionally worried about the least, at least in terms of progress and attainment.

Up until today, when Harry got stuck on an essay, he asked for help with the planning. Behind on some reading, he asked for a summary. A paragraph was challenging him, so he asked for it again in simpler words. Take each of those on its own and you would praise the instinct. He is using a resource to get unstuck, which is what we tell children to do. The trouble was never in any single step. It was in what a whole term of them added up to.

On the surface, it works.

Here is what makes this so hard to detect. Harry’s essays are better organised. His revision looks more structured. He puts his hand up more often, and sounds surer of himself when he does. Every box we tick to satisfy ourselves that learning is happening gets ticked, because the proxies we have always trusted as signs of learning are precisely the things the tool is really good at reproducing.

Ask the students themselves and they will not tell you the machine does their work. They will tell you it is a partner. A study buddy. A convenient tutor who is able to field questions at 11 pm. They are being honest. The help they experience is real, and to them it feels like learning.

Which leaves the rest of us with an awkward question. If the work looks good, and the child feels helped, and nothing on our radar is setting off alarms, on what grounds do we say anything is wrong?

Professor Becky Allen, co-founder of Teacher Tapp and advisor to Alpha School, has the image that helps encapsulate this issue. Many invisible things, she points out, can cast the same visible shadow. The essay is the shadow. The understanding is the thing casting it. And two very different children can throw an identical shadow on the page. One has thought the argument through. The other asked a machine to write it. On paper they are the same essay, and most of the time we cannot tell them apart.

For our entire careers, a good piece of work has been directly linked to a good piece of thinking and understanding. Generative AI has broken that link.

Professor Daniel Willingham gives us the rule of thumb. We remember what we think about. Now repeated to the point of cliché, memory, he says, is the residue of thought. So the question to ask of Harry’s essay is not whether he used AI. It is whose thinking happened here. If the tool planned it, found the quotations, lined up the facts and shaped the argument, then the residue formed in the tool. Harry has an essay. Whether he has an understanding of it is another matter.

If you want to know why the same tool can help one child and hollow out another, the answer is a word that is also often overworked in education: metacognition.

It means thinking about your own thinking, on purpose, and it comes down to three moves. Planning how you will approach something. Monitoring how it is going. Evaluating how it went. The Education Endowment Foundation rates teaching students these moves among the highest-impact, lowest-cost things a school can do, worth something like seven months of additional progress in a year.

Now look again at those three moves. Plan, monitor, evaluate. They are exactly the moves AI is happiest to take off our hands. The skill the evidence tells us to protect is the very one the tool is most eager to perform.

The same tool, sometimes the very same prompt, can sit on either side of the line. Ask it “here is what I know, what am I missing on this topic?” and the effort stays with you. Tell it to “plan my response” and you have handed the most of the effort over to the machine. “Quiz me and find the bits I cannot explain” sends a student back into the material; “tell me the answer to this question” bypasses the material altogether. Used one way, the tool pushes a student’s thinking. Used the other, it does the thinking for them.

Which is the genuinely challenging bit, and also the hopeful one. The helpful version and the harmful version look almost identical from the outside. Same student, same machine, sometimes the same words typed in. The technology does not decide which one is happening. The use decides. That is impossible to police from the front of a classroom, and it is also why none of this should be cause for doom and gloom. The tool that stopped Harry from learning is the same tool that can push him further.

So return to Harry, a few weeks on, same exam season, same LLM browser window on his laptop. One thing has changed, and it is not the technology. He stops asking it to answer questions and starts prompting it to ask him questions.

He writes his argument in the prompt box and tells it to explain his back to him, so he can hear where it wobbles. He asks it to find his gaps. He tells it to test him and to withhold the answers. He asks it to argue against the position he has taken, so he has to defend it. He is doing more work, not less. The tool has become a way of stress-testing his thinking instead of a substitute for it, and he leaves the session understanding more than when he sat down.

The that is apposite here is that of the personal trainer. A good one does not lift the weights for you. They make sure you lift them, and that you lift them properly. They spot you.

This changes the question we ask of students. Most policies ask, did you use AI, which invites a yes or a no and drives the whole thing underground. A better question is this: could you stand behind your work without it? Could you explain it, defend it, answer for it, if the tool were taken away? That moves us off policing and onto more honest territory, and most students will answer it straight, because it is a fair question to ask.

It also puts some of the responsibility back where it belongs, with us. If a task can be completed without thinking, some students will complete it without thinking. That is no character flaw. It is what most people do when a shortcut is on offer, especially if they are not aware of the cost involved. So it is worth looking hard at the tasks we set and asking where, exactly, the thinking is meant to happen, and whether a machine could do that part instead.

Daisy Christodoulou, Director of Educatoin at No More Marking, makes the point that writing is often how a thought forms in the first place. Putting something into sentences is frequently the act that clarifies it. This is what I am doing right this moment. When a student hands the writing to a machine, the part they smooth away is the part where the understanding was meant to leave a mark.

None of this is a verdict on AI, and I am wary of ending on alarm. Socrates worried that writing would destroy memory, that people would trust marks on a page instead of what they carried in their heads. In a narrow sense he was right. We just don’t learn thousands of lines of epic poetry like we once did. In every other sense, however, writing allowed knowledge to be passed on, not just from person to person, but across continents and centuries. Powerful tools tend to arrive with a real risk attached, and the answer is rarely to refuse them. The answer is to understand the opportunity as well as the cost.

Which brings us back to the exam hall, and to a different Harry walking into it. Same clock, same pen, same blank paper, same clock. Only this time the plan is one he knows how to make, because he has made it before, in his own head, with the tool pushing him rather than carrying him. He can talk you through every line.

Who is doing the thinking here?

This time, it’s all him.

This piece began as a talk, “Who is doing the thinking?”, at researchED by the Beach in Bournemouth. My thanks to the organising team for the invitation and to everyone who came along.

Featured image by researchEd by the Beach.