Faster agents for less: three days inside the MouRio harness

Everyone argues about which model to use. Very few teams look closely at everything around it: how the prompt is put together, which tools are offered, what the agent remembers and how each call gets billed. That's the harness, and it's where most of the cost and delay sits.

MouRio's harness was already good. But like a phone that gets a little faster every year, there's always more to find. So we spent three days on it and measured the results in production. Here's what came out of it:

• 93–99% of each follow-up prompt served from cache (live, ~120K-token prompts)
• 62% lower cost per conversation on replayed real questions
• 34% less time before the first model call in the slowest cases (P90, live)
• 0 answers worse in the replay, and 0 duplicate charges live

"Live" figures come from a production enterprise deployment, comparing equal time windows before and after each release. Client names and use cases are left out.

What a prompt cache is

An agent doesn't talk to a model just once. On every step it sends the whole job again: its instructions, the definitions of every tool it might use, the conversation so far and the newest message. The model reads all of it before it writes a word. A single question can take five to twenty steps, and each step re-sends everything.

Model providers keep a short-lived copy of prompts they've just processed. When the next request starts with exactly the same text, the provider skips re-reading that part and picks up where it left off. That's the prompt cache.

Think of a reader with a bookmark on page 200. Hand them the same book and they carry on from page 200. Change one word on page 3, and the bookmark is worthless: they start again from page 3.

Why it matters

• It's cheaper. Text read from the cache is billed at 10–25% of the normal input price, depending on the model.
• It's faster. In our production traffic, a model call that hit the cache came back about one second sooner than one that missed (for prompts under 60,000 tokens).
• It's unforgiving. Reuse stops at the first character that differs. A 120,000-token conversation whose first paragraph changed costs the same as one the model has never seen.

So the rule is simple to say and fiddly to keep: never change anything the model has already read, and put only what's new at the end.

What we found

We audited the harness end to end and found the start of the prompt changing in five places nobody meant it to. Each one quietly threw away the bookmark.

1. The clock

Before: An agent's instructions began with "Today is Wednesday, 22 October", so it could understand "this month". At midnight that line became "Thursday, 23 October", and every open conversation suddenly had a new first page. The next message in each one paid full price.

Now: The date travels with the newest message, right next to the question that needs it. The instructions never change.

2. The tool list

Before: A chat starts with the agent offering search and email. Three messages in, someone asks for a spreadsheet, so the agent adds a spreadsheet tool. The tool list sits at the very top of the prompt, so the whole conversation was billed again from scratch. On the most tool-hungry agents, that happened on up to 41% of calls.

Now: The tool list stays fixed for the whole conversation. New tools are reached through one permanent entry, and their details are added at the end, where nothing has been read yet.

3. The history

Before: After each turn, the harness rebuilt the conversation from its logs. It tidied things up along the way: merging two searches into one, removing a stray space. To the model it looked the same. To the provider it was a different conversation, so the bookmark only reached the previous turn.

Now: Each turn is stored exactly as the model saw it and replayed character for character.

4. The profile

Before: Agents with memory keep a short profile of the person near the top of the prompt. Midway through a chat, someone mentions they've moved from Mumbai to Pune. The profile changed, and the next message re-read the entire conversation at full price.

Now: The profile stays fixed for the conversation, and new facts ride along with the newest message. If a fact is corrected or removed, the profile is rebuilt once, so the old version is never shown again.

5. Attachments

Before: Someone attaches a ten-page scanned PDF, then asks eight follow-up questions. Every follow-up re-sent all ten page images at full price: eighty pages billed for one document.

Now: A file is shown to the model once. Later turns refer back to it, and the agent re-opens a page only when it actually needs to.

We also found setup work waiting in line when it didn't need to. Memory recall, tool discovery, safety checks and routing ran one after another before the model was even called. Now they run side by side.

What changed, in numbers

Cache: the model reads what it has already read

Time, notes and recalled memories now travel with the newest message. Tools stay fixed for the length of a conversation. Each turn is stored exactly as the model saw it and replayed byte for byte.

• In test, the share of each turn's first call served from cache went to 98% on turn 2 and 99% on turn 3.
• In production, a 60-tool enterprise assistant now reads 93–99% of its roughly 120,000-token prompt from cache on every follow-up call.
• Even brand-new conversations start warm. The first call of a fresh chat read 124,151 of 124,825 tokens from cache, because every chat with that assistant now opens with the same bytes.
• Summarising a long conversation now reuses the same cached prompt: 99.6% from cache, at $0.0545 instead of $0.1418.

Smaller prompts

We also trimmed what the model reads on every call. The heaviest tool descriptions shrank from up to 1,815 characters to about 500 without losing a rule, and large agents now pick from their 16 most relevant tools instead of all of them.

Speed: setup runs side by side

Memory recall, safety checks, routing and tool discovery now run at the same time instead of waiting in line, and shared work is done once per turn instead of up to three times.

• Median time before the first model call (live): 1.81 s → 1.29 s
• Slowest 10% (P90, live): 3.31 s → 2.20 s
• Median time before the first model call (test): 1.70 s → 0.65 s

What users and finance teams see

We replayed 12 real production questions, each with a follow-up, against the harness before and after the first round of changes:

Then the second round went live. Here's 21 hours of production traffic on each side of the release, agent by agent:

Across all traffic on our test platform, the average prompt per model call fell 44% (45,000 → 25,300 tokens) and cost per agent run fell 45% ($0.100 → $0.056).

What that means for a business: take the replay result of $2.48 → $0.93 per conversation. An assistant handling 100,000 conversations a month would go from about $248,000 to about $93,000 in model spend. That's roughly $155,000 saved every month, with the same model and answers just as good.

Built to be trusted

Speed and cost only count if the bill is right and nothing breaks. So every change came with checks that run against the real database, not mocks:

• Exact billing. Usage is now recorded off the critical path: one ledger row per model call, zero duplicate charges, and a write interrupted at shutdown is never charged twice.
• Budgets hold. Spending limits still stop a run at its limit, including budgets shared across workspaces.
• No surprises for existing deployments. There's nothing to configure and no migrations. Deployments set up the old way still send requests their provider accepts, and we checked that against the provider itself.
• Reviewed twice. An independent adversarial code review plus an automated security review. Every confirmed issue was fixed with a test that fails on the old code and passes on the new.
• No more slow leaks. Background workers no longer leave a cache connection open after each task.

The takeaway for anyone building agents: before you switch models to save money, audit your harness. Moving one timestamp can matter more than any model price cut.

What's the sneakiest cache breaker you've found in your own agent stack?

Methodology: "live" comparisons use equal windows of production traffic before and after each release. Costs and tokens come from the platform's own usage ledger, and timings come from its traces. Replays re-ran recorded real questions against each build. Cache figures are the provider's own reported cached-token counts. Results depend on the model, the agent and the workload.