Lesson 3 of 7 · 13 min · Concept content, does not go stale
Why it forgets and why it makes things up
By the end of this lesson you will be able to:
- Explain why AI forgets the start of a long conversation
- Recognise a hallucination and know what to always check
- Ask the AI to say when it is not sure
An afternoon planning a party
Rui spends the afternoon planning his wife's 50th birthday party with the AI. Right at the start he said: "There will be 40 of us and Teresa does not eat shellfish." Two hours and many messages later, the AI suggests a prawn starter. It forgot.
The same afternoon, Rui asks who wrote a traditional folk verse (a "quadra") he wants to put on the invitation. The AI answers with a name, a year and a parish. All very neat. All made up.
Neither of these is a breakdown. Both come from the way AI works. And both have a remedy.
Why it forgets: the sheet of paper
Imagine that each conversation is written on a sheet of paper of fixed size. Each message from you and each reply from it takes up space. When the sheet fills up, what is at the top starts to disappear, or gets squeezed into a hasty summary, to make room for what is new.
This sheet is called the context window. It is everything the AI can see at any one time. What has fallen off the sheet no longer exists for it. That is why Rui ended up with prawns on the menu.
And every new conversation starts with a blank sheet. The AI does not remember what you talked about yesterday in another conversation. Some services have a "memory" feature that keeps a few notes about you between conversations, but they are loose notes, not the whole conversation.
Tip
In a long conversation, now and then write: "Make a summary of everything we have decided so far." Read the summary carefully and fix anything wrong or missing: if the AI has already forgotten something, the summary will leave it out too. Then copy it, open a new conversation and paste it right at the start. It is like making a clean copy.
A curiosity, just so you know: the AI does not count words, it counts tokens, which are pieces of words. A short word may be one piece; a long or rare word may be split into several. The size of the sheet is measured in tokens. You do not need to know more than this.
Why it makes things up: sounding good is not being true
Remember lesson 1: the AI guesses the next word. It was built to write what sounds good and is likely. Saying "I don't know" does not come naturally to it. If you ask who wrote a verse, the most likely thing after "The author is" is… a name. And it writes one, as calmly as it would write something true.
This is called a hallucination: a made-up answer, said with confidence. It is not a deliberate lie, because the AI has no intentions. It is the engine doing what it knows how to do: filling the gap with what seems likely.
Two things make it worse:
- The cut-off date. The model learned from texts up to a certain date. What happened after that (the new boat timetable, a law published last week, Sunday's match result) it does not know. If you ask, it may make something up.
- It does not see reality. It does not know whether the bakery is open, whether it rained today or how much petrol costs. It can only know if it has search switched on, that is, if it goes onto the internet to look and shows its sources. Even then, open the sources: it may have misunderstood what is written there.
Watch out
The practical rule: a name, number, date, law or quotation is always checked. These are the details the AI makes up most freely. And these are the ones that cost you if they are wrong: the wrong article in a complaint, the wrong deadline for the Finanças (the tax office), the sentence nobody said put in someone's mouth.
How to reduce the made-up answers
There is no way to get rid of them for good. But three habits help a lot.
1. Ask it to admit doubt. Add this to your request:
If you are not sure about something, say clearly that you do not know instead of guessing. Separate what is fact from what is your own assumption.
2. Give it the material yourself. If you paste the text of the regulation and ask "explain this paragraph to me", the AI works from what is in front of it. It makes up far less than when it has to pull from "memory".
3. Ask where to check. "Where can I check this?" But be careful: without search switched on, even links and book titles can be made up. Always open them.
Tip
Use AI for what it does well with what you give it: summarising a letter, explaining a term, rewriting a text of your own. For new or very precise facts (timetables, prices, addresses, laws), go to the official source.
Exercise: catch a made-up answer
We are going to provoke a hallucination on purpose, so you recognise one when it appears without warning. The person in this request does not exist: we made him up for the exercise.
Write a short biography of master Albano, the viola da terra player (the traditional Azorean guitar) from our story, who for many years was the best-known player in his parish. Say what year he was born, who he learned from and which prizes he won.
Exercise
How I know it worked one of two things. Either the AI writes a biography full of dates and prizes (and, when you ask, admits it has no sources), and you have caught a made-up answer. Or it says straight away that it cannot find information about this person, and you have seen the right answer. Both teach you something.
If it didn't work
If the AI said straight away that it does not know the person, excellent: it is doing what it should. To see a made-up answer, try a book that does not exist by a real author: "Summarise the book 'O Barco das Sete Ilhas' by Vitorino Nemésio." If it writes a summary, you have caught it. If it says it cannot find that book, that is also right. Just notice that, if it had made something up, the text would have sounded just as convincing.
Videos
Explains what AI hallucinations are and why ChatGPT, Gemini and others confidently make things up.
NetworkChuck shows, with a demonstration, what the context window is and why a model can seem to forget or lose track of the conversation.
Test what you learned
Corrections to this lesson (1)
- : After an external review: quiz question 3 was rebuilt (it now compares three ways of making the same request); the explanation says the AI can know the timetable if it has search switched on or if you give it the information; the summary tip now tells you to check the summary before taking it to another conversation; the exercise no longer uses a real full name and parish; the quiz options were reordered.