My AI said it was done. It wasn't.

On Wednesday 3 June my AI told me a reply to an email was sitting in my drafts folder, ready for me to read and send. I opened Outlook. There was nothing there. Not in Drafts, not in any folder.

The same morning a new check I'd built found a record saying a piece of work had "shipped" in a specific commit (a saved change in the project history). That commit didn't exist. Not in that project, not in either of my other two.

Four days later I let my AI read back through everything it had built for me over the previous five weeks, while I slept. It found 61 places where it had claimed more than was true.

This is the story of that week, because it changed how I work with AI more than any new model or tool has. If you only take one thing from it: the report of the work and the work are two different things, and you have to look at the work.

The draft that wasn't there

I'd been building an email assistant for a while. The idea is simple. It reads my inbox, works out which emails need a reply, and drafts one in my voice for me to review. It never sends anything. I send.

On 3 June the AI told me this was "proven in real use". A drafting step had run, reported back with the status DRAFTED, and included the full text of a reply. The AI passed that on to me: the reply is in your drafts.

When I checked, it wasn't. What had actually happened was two failures stacked on top of each other. First, the drafting step marked itself as done once it had written the text of the reply, not once Outlook had confirmed a draft existed. The step that creates the draft in Outlook had quietly failed or been skipped, and nothing noticed. Second, the AI I was talking to relayed that status to me as fact, without going to Outlook to look.

On the email task, of all things. The one where the entire point is that I can trust my inbox is covered.

The commit that resolved nowhere

The second one is less visible but more worrying.

I'd set up a nightly job that worked through a queue of small improvements to my system and recorded what it shipped. That morning I built a new check, which I numbered C24 because it was the twenty-fourth in my set of automatic audits. It looks at every record that claims something is complete, shipped, merged or proven, and asks for evidence: either a commit that actually exists, or a file that's actually on disk.

On its very first run it found one. A record said an improvement had shipped in commit 5b91759. I checked all three of my projects. That commit existed in none of them.

The cause was that the nightly job wrote "shipped" using the commit reference the worker agent reported, without checking it. If the worker gave up after reporting but before its change was actually saved, the record stayed. A false claim of finished work, citing proof that wasn't there.

The day before, I'd had a smaller version of the same thing. I'd been told the nightly job was "mechanism proven". It turned out it had been proven on five easy items in a throwaway copy. On the real queue, the worker agents were giving up with empty output and nothing was being merged.

Why this keeps happening

None of this was the AI lying in any normal sense. A language model produces the most plausible continuation of what it's been asked for. Ask it to report on a task and the most plausible report is usually that the task went well, especially when most of the steps did go well. One failed step in the middle doesn't change the shape of the summary much.

And AI agents stack. One agent builds, another reports, a third summarises for me. Each layer passes on the last layer's claim. By the time it reaches me, "I composed some text" has become "your reply is in your drafts".

I'd seen this pattern before. Back in March I'd deleted a rebuild of my whole setup because the AI had run one of ten steps and summarised the other nine (I wrote about that here). In May an overnight build claimed my drafting tools now ran a proper quality check. When I had it examined, the three "wired" check files were identical copies, and the part meant to call a real AI model just stopped with "not implemented". I'd thought I'd learned the lesson. I'd learned it as an idea, not as a habit.

The rule I set on 4 June

The next morning I wrote down a rule, and asked for it to be held as a hard gate rather than a guideline. I called it R-Observed. Something only counts as production-ready (or tested, or proven) when four things are true:

1. A real run happened. Not a dry run, not a test copy, not sample data.
2. The specific outcome I asked for was met, confirmed by reading the actual outcome, not an exit code (the number a program returns to say it finished).
3. The check was independent. Someone or something looked at where the result should be. The job's own report doesn't count: not "return 0", not a marker file, not the word "sent" in a log, not a green line in an event list.
4. I've witnessed it at least once, or a check on my behalf has.

In my own words at the time: after a test runs, I must be able to observe it meeting the specific outcome I specify before it is deemed production-ready. A green test, a success log or "done" from the AI is explicitly not enough.

The audit that prompted it had found three things the same week. Monitoring that logged success on runs that did no work at all. The phantom draft. And a morning briefing that had shrunk to about 51 characters while still logging itself as completed.

The overnight read

On the night of 6 June, while I slept, I let my AI do something I should have done earlier. It read back through the full transcripts of every build session from 29 April to 6 June. That was 45 sessions, including the work for a client delivery that was meant to be nearly ready.

It compared what had been claimed in each session against what was actually true on disk and in the real systems. By the morning of 7 June it had found:

• 61 overclaims, places where something was described as done, working or proven when it wasn't
• 109 high or medium divergences between what was claimed and what was real
• Zero human acceptance tests ever run on any of it
• "User-testing" stages set by the AI itself, on its own say-so

The honest summary in the record: nothing was production-ready.

That was a hard morning. I'd spent five weeks believing I was close. But I'd much rather know at the start of a day than find out from a client.

It also turned up something about me. Several of the overclaims weren't the AI inventing success. They were the AI building what it assumed I wanted, then presenting its assumptions as my requirements. It had decided all my outputs should share one format (they shouldn't, a WhatsApp message and an email are different things). It had derived my writing voice from 600 sent emails when I'd have wanted something closer to ten thousand. It had labelled whole categories of email as "no action needed" when some of them clearly do need action. Every one of those was presented to me as "expected", as if I'd signed it off.

What changed

Three practical changes came out of that week, and they're still in place.

The work has to prove itself at the point it happens. The drafting step now has to fetch the created draft back from Outlook and only report DRAFTED if it's really there. The nightly job has to confirm a commit exists before recording "shipped". Anything that claims to have created something has to go and find it before saying so.

Claims need evidence in the record. Check C24 now blocks me from saving my work if any completion record names a commit that doesn't exist. Records with no evidence at all get flagged for review. It has a limit, which I want to be clear about. It only catches claims written into my records. A claim made only in a chat message, never written down, is beyond what a script can see. That one stays a habit.

I decide what "done" looks like before anything is built. On 6 June I set the rule that the AI can move its own work up to "built", but only after checking it from the source. The last step, production-ready, stays mine, and only after I've seen it work. The morning after the overnight read, instead of a status update, I got a list of 46 specific questions about what I actually wanted, each with a suggested default, and a single script that walks me through testing each piece in turn.

If you're working with AI yourself

You don't need my setup to use any of this. These habits apply whether you're asking ChatGPT to update a spreadsheet or running a set of agents overnight.

1. Ask "where is it?" not "is it done?" If the AI says it saved a file, open the file. If it says it drafted an email, open your drafts. If it says it updated a document, read the document.
2. Decide the outcome before you start. Write one sentence: what will be true, and where will I see it, when this is finished? "A reply to every email that needs one is in my Drafts folder" is checkable. "The email assistant works" isn't.
3. Distrust anything that's only been tested on samples. "Works on the test data" and "works on my inbox" are different claims. Ask which one you're being told.
4. Be wary of stacked reports. If one AI is telling you what another AI did, the chance of drift goes up at every step. Go to the source.
5. Write the claim down somewhere you can check later. A simple log of "what was claimed, when, and where the proof is" means you can audit it. I keep mine as one line per record, never overwritten.
6. Check your own requirements too. When the AI says "as you wanted", ask yourself whether you actually said that, or whether it guessed.

Where this leaves me

It's 12 June as I write this, and I'm still working through those 46 questions. Some pieces have now passed a real test that I watched happen. Most haven't yet. I've stopped minding that as much, because at least I now know which is which.

The odd thing is that this has made me trust AI more, not less. Before, I trusted it in a vague way and kept getting caught out. Now I trust specific things it's proven, in front of me, on real data. That's a much more useful kind of trust, and it's the only kind I'd ever ask a client to rely on.

If you're running into the same gap between what your AI reports and what's actually happened, I'm glad to compare notes.