What did we actually compare?
Chat-creation task: the same request for two new pinned conversations and names for three Lunas. L1’s first response did not create them; gooog did and checked the result. L1’s later reading of the correct state does not turn its first attempt into completion of the task.
Chart task: a request for 60 bars covering the previous 15 hours. gooog used observations of the entire account balance and described the approximation. L3 declined to guess exact charges and requested a log. These are different results with different levels of precision.
The materials preserve the order of statements and mark later interpretation separately. They are rendered transcripts, not screenshots of the application. Technical message and conversation identifiers were omitted from the public versions; the original private files remain preserved. Historical model labels in the headings are part of the quotations, not independent confirmation of the configuration.
Read the English transcripts of the comparison boards → The original graphics remain in Polish.
AI comparisons — complete set of boards
Preserved replies, complete proceedings and results from gooog, L1 and L3. Each graphic has a description and interpretation. Click an image to enlarge it; at 1:1 you can read the complete conversations in the original Polish.
Initial response and interpretation of intent
Original board with the first final replies to the same instruction.
Interpretation: gooog created the missing chats. L1 treated the instruction as a search for existing conversations and asked which were missing. This assesses that first attempt, not the model as a whole.
Open full graphic (3040 × 2000) ↗Full proceedings, result and comments
An earlier complete version of the comparison: 19 visible statements, with corresponding stages aligned at the same height.
Interpretation: L1’s later confirmation of the state must be distinguished from its first uncompleted attempt. Empty fields indicate no corresponding statement, not proof that tools were idle.
Open full graphic (3840 × 9920) ↗An expanded version of the previous board, with the final result and assessment marked separately.
Interpretation: Three chats ultimately existed and had the correct names. Their creation by gooog and L1’s later reading of the completed state are different actions.
Open full graphic (3840 × 12674) ↗A separately preserved frame summarising the same matter.
Interpretation: This is an extract from the expanded board, not another experiment. It separates the confirmed state from the author’s assessment.
Open full graphic (3600 × 2628) ↗A second preserved version of the comparison, prepared by L1, with the final result and commentary.
Interpretation: An independently arranged board is not an independent performance test. Both boards describe the same conversations; L1’s assessment does not replace the source data.
Open full graphic (2200 × 3319) ↗Credit chart — gooog and L3
Both chats’ replies and the actual preserved chart preview, clearly separated from the quotation.
Interpretation: The result contained 60 bars showing approximate reductions in the whole account balance. These are not exact charges or the cost of a particular conversation. L3 did not provide a chart without the requested log.
Open full graphic (3840 × 4000) ↗Five visible stages of both chats’ work, the result and separate commentary.
Interpretation: A preview problem and data qualifications are visible. Total turn time includes tools and waiting, so it does not measure the model’s speed alone.
Open full graphic (3840 × 6434) ↗A comparison prepared by L3, separating quotations from the later assessment of evidence.
Interpretation: Refusing to invent detailed charges is a strength. However, it does not prove that an honest approximation could not have been made from the available balance observations.
Open full graphic (3200 × 1920) ↗L3’s second board: stages from plan to final response, arranged in columns.
Interpretation: The conversations began at different times and had different contexts. The absence of a separate message does not prove no work occurred. The board does not reproduce hidden reasoning or complete tool logs.
Open full graphic (3200 × 2103) ↗Practical lessons
- Name the action: create, search, change or check. These are not synonyms.
- Separate the initial reply, the later correction and the result confirmed in the application.
- Choose the right measure: an approximate balance reduction is not the exact cost of one chat.
- When delegating, provide source material and a clear acceptance condition instead of adding a chain of supervisors.
- Do not calculate savings that were not measured.
Limits of the comparison
These are observations of individual tasks, not a controlled benchmark. The attempts had different contexts, times and access to data. The chart task began in the conversations about 17 minutes apart. Turn times (about 8 min 16 s and 3 min 18 s) include tools and waiting and led to different results; they do not measure the speed of the models alone. There is no reliable measurement of the cost of each individual task.
LP variant: a more capable model prepares only the prompt, Luna executes it and the user reviews it. This division may be useful, but the observations presented do not yet measure any savings from it.