Skip to content
Tutorials Center
My learning

Topics

Lesson 40 of 46 ยท Learning Technology and Projects

AI and learning

How generative AI can help or harm learning: the Bastani field study, cognitive offloading, early tutoring trials, and prompts and habits that keep you thinking.

In a few years, generative AI has gone from a curiosity to a tool that millions of students use every day. It can explain a concept in five different ways, write practice questions, give feedback on an essay and solve most homework problems in seconds. For a learner, that last ability is both the opportunity and the danger. A tool that can do the task is tempting to use instead of learning to do it, and everything you have read in this course suggests that doing the task yourself is the point.

This is a fast-moving area. Most studies are recent, many have been reported first as preprints, and the tools change faster than studies can be run. This lesson therefore focuses on durable principles, the best early evidence and clear labels for what is not yet known.

Two cards: AI that gives answers, where practice scores rise but later scores without AI fall, and AI as a tutor, where practice scores rise and later scores hold steady
Schematic of the pattern reported in a high-school field study: unrestricted answers raised practice scores but lowered later unaided scores, while a tutor with guardrails largely avoided the harm.

What you will learn

  • Why practice performance with AI can mislead
  • What a large field study of AI in school mathematics found
  • What cognitive offloading is, and what research on search engines does and does not show
  • How to use AI for retrieval, hints and feedback without losing the effort
  • What the evidence cannot yet tell us

Key terms

Generative AI: systems, such as large language models, that produce text, code or images in response to prompts.

Cognitive offloading: using external aids to reduce mental effort.

Guardrails: instructions or design features that stop an AI tool from simply giving full answers.

Generation effect: material you generate yourself is remembered better than material you only read (Norman Slamecka and Peter Graf, 1978).

Hallucination: a fluent but false statement produced by a language model.

Performance is not learning

One idea from the previous chapters is worth repeating: performance during practice is a poor guide to learning. In the interleaving studies, blocked practice produced the best practice scores and the worst test scores. With AI the gap can be larger. A learner who asks an AI to solve every problem will have excellent homework, because the homework measures the AI. What the learner can do alone, with a blank page, may not change at all.

Retrieval, generation and effortful problem solving are the engines of durable learning. Any tool that removes them removes the learning. Any tool that asks you to produce, recall, explain or try first keeps them.

A field study in mathematics

The best-known controlled study so far was carried out by Hamsa Bastani and colleagues in a high school in Turkey, with about a thousand students, and reported first as a preprint in 2024 and later in a peer-reviewed journal. In a design with randomly assigned groups, students practiced mathematics in a session with one of three kinds of support: no AI (a control), a standard chat assistant ("GPT Base") or a version designed as a tutor ("GPT Tutor") that was given the teacher's solutions and instructed to give hints and not answers.

The reported pattern is the one in the picture. During practice, both AI groups did much better than the control group: roughly 48 percent better with the standard assistant and more than double with the tutor version. Then came an exam with no AI allowed. Students who had practiced with the standard assistant scored about 17 percent lower than the control group, while the tutor group's results were close to the control. Students in the standard-assistant group also tended to overestimate how much they had learned.

Two readings are fair. First, easy access to answers can lower learning even while it raises practice scores. Second, the harm was not inevitable, since a design that required effort avoided most of it. The study has limits: it was one school, one subject, a few practice sessions and a particular age group, and it does not show what happens when students use AI over months or in other subjects.

Cognitive offloading and the search-engine effect

Humans have always used tools to offload memory, from notebooks to calendars. Cognitive offloading frees mental capacity and is often sensible: you do not need to memorize every phone number. Evan Risko and Sam Gilbert (2016) reviewed this research and noted that offloading brings costs as well as benefits, because people may remember less of what they expect to look up.

An influential study by Betsy Sparrow, Jenny Liu and Daniel Wegner (2011) reported a "Google effect": people who expected to have access to information later remembered it less, but remembered where to find it. This study did not hold up well when it was repeated. A large replication project (Colin Camerer and colleagues, 2018) did not reproduce the main result. So the safest statement is that expecting access may reduce effort in encoding, but the size and reliability of the effect are uncertain. The theory is plausible and the evidence is mixed. That is typical of this area.

Tutoring with AI: early signs

Positive results are emerging for carefully designed tools. A small randomized crossover study in a university physics course at Harvard (Gregory Kestin and colleagues, 2025) reported that students using an AI tutor, built to guide them step by step with course-specific materials, learned more in less time than students in an active-learning class session. The design points in the same direction as the school study: the benefit came from a tool built on instructional principles such as scaffolding, feedback and active practice, and not from unrestricted chat. It was a single course at a highly selective university, so generalizing is risky.

What these early results have in common is that the tool's design decided whether it helped, which echoes Clark's truck.

How to study with AI without losing the effort

Use these rules, which follow from the earlier chapters.

  1. Try first. Attempt the problem or recall the idea before you ask. The attempt, even if wrong, makes the later explanation stick (productive failure, the generation effect).
  2. Ask for hints, not answers. Request the smallest hint, then the next. Tell the tool not to solve the problem.
  3. Have it quiz you. Ask for questions one at a time and wait until you have answered. This is retrieval practice with feedback.
  4. Explain back. Write your explanation and ask the AI to find what is missing. Do not ask it to write your explanation.
  5. Check the facts. Language models can state falsehoods fluently, so verify claims against your course materials or reliable sources. Be especially careful with references, quotations and numbers.
  6. Test yourself without it. Regularly do a closed-book, no-AI check. Predict your score first (calibration).
  7. Protect your data. Do not share private information with a tool whose data handling you do not know.

The builder below turns these rules into prompts you can paste into any assistant. It only assembles text and runs nothing.

Study-prompt builder

The second demonstration is a toy simulation with invented parameters. It assumes that skill grows only from effortful processing, and that an assistant that gives answers raises practice performance by supplying help without effort. It illustrates a mechanism, not a measured result.

Python Practice scores and real skill (toy simulation)

Press Run (or Ctrl + Enter) to see the result here.

Program input ยท what you'd type while the program runs

Type the answers before you press Run, one per line. Each time the program reads a line (with input()), it gets the next line from this box.

What we do not yet know

  • Long-term effects. Studies last days to weeks. What happens over years of daily use is unknown.
  • Which learners benefit. Strong students may use the tools to extend themselves, while weaker ones may lean on them. The evidence on who gains is thin.
  • Which tool designs work best. The tutor-mode results are encouraging, but each tool and each subject must be tested.
  • Effects on motivation and academic honesty. These are hard to measure and an active area of study.
  • Accuracy. Models have improved but still make errors, and learners may not notice them.

Common misconceptions

"If I understand the AI's answer, I have learned it." Understanding an explanation while reading it is not the same as being able to produce it later. That is the illusion of knowing again.

"AI makes learning effort obsolete." The cognitive processes that build memory and skill do not change because a tool exists.

"All AI tools are equally risky." Design matters. Hints and quizzes are different from answers.

Check your understanding

Quick check ยท 5 questions

of 5 answered You scored / 5 ยท perfect!

  1. 1. In the Bastani and colleagues field study, what happened to students who practiced with the standard (unrestricted) assistant when they later took an exam without AI?

    Practice scores rose with the assistant, but unaided exam scores fell, by roughly 17 percent in the reported results.
  2. 2. Which feature of the tutor version helped avoid the harm?

    Guardrails kept the learner doing the thinking.
  3. 3. What is the generation effect?

    Slamecka and Graf found better memory for self-generated material.
  4. 4. What is the best summary of the research on the Google effect on memory?

    The evidence is mixed, so claims should be cautious.
  5. 5. Which is the best way to use an AI assistant for studying?

    These habits preserve retrieval, generation and calibration.

Summary

  • Performance with AI help measures the help. What you can do alone measures your learning.
  • A large field study found that unrestricted AI raised practice scores and lowered unaided exam scores, while a tutor-style design with guardrails largely avoided the harm.
  • Cognitive offloading is natural, but its costs for memory are still debated, and the Google effect did not replicate well.
  • Early tutoring studies are encouraging for tools built on instructional principles.
  • Try first, ask for hints, get quizzed, verify facts and test yourself without the tool.

Next: Chapter 8 review