Research

Ten Thousand Prompts In

Working with AI is not prompting. It is building an operating system, and the receipts are in the logs.

By Merryl DMello, DMello Advisory · 4 min read

The claim: after more than ten thousand prompts of daily AI-assisted work, the measurable record shows that prompting skill is nearly irrelevant and system design is nearly everything. The evidence is not opinion. It is one user's complete behavioral log: 10,436 prompts, every session transcript, and the deltas as the system around the AI evolved. The single most telling number: per-action supervision ("yes", "proceed", "go ahead") peaked at 15.0% of all prompts early on and fell to 0.3% by the end. The human did not learn to trust faster. The system earned it.

What the log actually shows

The corpus, in raw form:

• 10,436 prompts, two work blocks a day, nearly every day
• median prompt length: 64 characters. Six prompts out of 8,945 exceeded 2,000 characters in the first part of the record
• 303 interactive main sessions, plus 1,192 subagent transcripts spawned by delegation
• 307 compaction events: single sessions grew past 40 MB and were compressed dozens of times each
• 72% of all sessions are not interactive at all: scheduled agents run standing jobs on their own, month after month

Read that again in aggregate: this is not a person conversing with a chatbot. This is a person operating a fleet, in short steering bursts, on a schedule they built.

Steering, not instructing

The median prompt is one line. The log's most important steering channel is not even a message: it is the interrupt. Counting every interruption of a running action across the corpus gives 131 interrupt events against roughly 51 typed corrections, a ratio of 2.6 to 1. Interrupts carry different information than corrections: they stop verbosity, question monologues, and cost-burning actions, things the operator never bothers to type. Corrections carry content accuracy.

Any team measuring "how often do we correct the AI" from chat text alone is measuring the minority channel and missing the dominant one. This is measurable, it is cheap to measure, and almost nobody does.

Trust as an engineered property

The supervision-tick collapse (15.0% of prompts in the first period measured, 3.3% later, 0.3% at the end, while median prompt length held steady) coincides with specific system changes, not with growing confidence: standing written directives instead of per-action approvals, automated permission checks, scheduled batch work that does not wait for confirmation, and a clear division of labour between stronger and cheaper models.

Around 205 automated executions in one month drew exactly one human intervention. Trust did not scale because the human got comfortable. It scaled because verification moved into the system: adversarial audit agents on every build, render gates that inspect actual video frames instead of trusting exit codes, budget caps enforced in code rather than requested in prompts.

Continuity as deliberate engineering

The marathon sessions are the counter-intuitive finding. Single delegated workstreams ran to 22 compactions and 136 MB of transcript. The expectation is degradation. The transcripts show none: after every compaction, the model resumed cleanly, recited exact pre-compaction figures, and re-read files defensively rather than trusting its own summary. Continuity was engineered with a two-tier memory: informal memory files for the AI, formal handover documents (what shipped, what broke, what was decided) for humans and for the next session. The failure modes that did appear were harness friction (stale scheduled wakeups, session-limit walls), not model decay.

The general principle: a session can be treated as a standing venture with institutional memory, not a conversation. Most users experience context loss as a wall. This corpus shows it is a problem you can build against.

The honest limits

Three things this log cannot support. First, one user: everything here is a case study, not a population claim, and should be argued as such. Second, the method has a vendor asymmetry: reasoning content is redacted in local Anthropic transcripts and stored in plaintext by other vendors, so private-reasoning analysis only works on part of the record, and any system built on transcripts must say so. Third, transcripts are a data-governance surface: the same corpus contained a live API key inside a model's plaintext reasoning. Local logs need secret scanning before they become an archive.

What would change my mind

• A high-volume user whose supervision rate stays flat while their output scales. Every operator I can find evidence of either automates or drowns.
• Evidence that long, carefully written prompts correlate with better outcomes than short steering bursts plus a strong system. The median-64-character prompt drove everything in this corpus.
• A transcript-preserving analysis that finds the trust migration happening without system changes, purely through familiarity. The timeline here tracks gates and delegation going live, not sentiment.

The method is the message

Nothing in this paper required special tooling. The analysis ran on local scripts over the operator's own logs: prompt history, session transcripts, interrupt and correction counting. Every organization running AI-assisted work already owns this data and almost none of it survives even 30 days under default retention settings. The first move is not adopting anything from this paper. It is keeping your logs.

The author runs dmello-advisory: working AI systems and measured workflows for small and mid-size companies, built by one person who has been shipping products for eighteen years at Intel, HP, Rackspace, BioNTech, and Phoenix Contact. dmello-advisory.com.

All research papers

Last updated: 16 October 2026