Why AI invents even a simple path through settings

An example: the counter that was not confirmed

The conversation began with the user asking, ‘Does talking to you use up a token limit?’ The assistant distinguished the message limit in ordinary chat from the Work/Codex allowance, then gave an instruction:

‘I can’t see your remaining limit; you can check it under Settings → Usage.’

After the question ‘Where can I see this limit and its usage?’, the answer was different:

‘For ordinary GPT-6 Pro chat, I have not found a confirmed counter showing “X of Y messages used”.’

The assistant acknowledged that it had given the earlier path too broadly. The subsequent questions therefore concerned the distinction between using an allowance and being able to see its current level. Not having established where a counter is displayed does not mean usage is not counted.

The invented part was the assurance that the indicated path solved the user’s problem. The ‘Usage’ tab itself may exist and concern other billing. A real screen name does not confirm every function attributed to it.

Documentation checked on 3 November 2026 describes the weekly GPT-6 Pro limit separately from Work and Codex limits. It points to ‘Usage’ settings for the credit balance and purchase of credits. That article does not give a location for a current GPT-6 Pro message counter. This is the limit of what that source establishes, not proof that such a counter exists nowhere. Source: About ChatGPT Pro tiers ↗.

The quotations come from a conversation record supplied by the user. It does not unambiguously identify the model that gave the mistaken instruction. GPT-6 Pro in the answer names the limit being discussed; it does not establish which model replied.

Why a plausible answer can be false

A language model learns relationships in texts and generates answers from learned patterns and available context. A well-phrased sentence is not automatically a verified fact. When information about a particular product function is missing, a model may match a familiar screen name to the question without grounds for that connection.

Research described by OpenAI also identifies a grading problem: if a correct answer earns points while a justified ‘I don’t know’ does not, guessing can improve a test score. This encourages apparently confident answers. It is one mechanism that can sustain hallucinations, not a complete explanation for every error. Source: Why language models hallucinate ↗.

In the counter example, one possible explanation is confusion between the message limit and the credit-billing screen. That is a hypothesis based on the answer’s content. We do not have access to the cause of this particular instruction being generated.

Interview with Professor Aleksander Mądry

In Łukasz Kijek’s interview with Professor Aleksander Mądry on ‘Biznes Klasa’, the indicated discussion of hallucinations begins at 1:11:52. The interviewee explains pressure to give an answer through analogies to an intern trying to perform well and a persuasive, highly rated forum post. He also notes that simply instructing a model ‘don’t make things up’ does not work perfectly.

These analogies explain model behaviour; they are not a measurement of every system’s performance or an attribution of human intentions. The discussion concerns approximately 1:11:52–1:15:39. Watch the interview from 1:11:52 ↗.

Apologies also require verification

After correcting an error, an assistant may generate a convincing explanation of why it was wrong. That explanation alone does not reveal the actual mechanism that led to the answer. Similarly, a claim that ‘I checked the documentation’ requires identifying exactly what was read and what conclusion follows.

In this example, confirming that a limit exists does not confirm where its counter is displayed. A good correction clearly separates those two facts instead of proposing another unverified path. OpenAI warns that ChatGPT may provide incorrect facts and invented sources while sounding confident. Source: Does ChatGPT tell the truth? ↗.

How to check a similar instruction

1. Establish exactly what you are looking for: a message limit, credit balance, reset date or usage history. Similar names may refer to different things.

2. Ask for a source confirming the specific screen function. A general subscription-plan article may confirm that a limit exists without locating the counter.

3. Compare the instruction with the current interface. If the stated function is not visible, check the source and access scope again instead of multiplying guesses.

4. Accept ‘not established’ as a valid result. Further help could consist of checking a limit message or asking support about a specific function, without promising that a counter definitely exists.

A reliable answer should distinguish fact, conjecture and missing information. Here it would have been enough to confirm the existence of a limit and state openly that the place to see its current measurement had not yet been established.