Context compaction quietly deletes your agent's guardrails

An agent that follows your rules perfectly for forty turns can break them on turn forty-two — not because someone jailbroke it, but because the rules got summarised away when it compacted its context. Nothing throws. No log line records it. The constraint you wrote at the top of the system prompt is simply no longer in the conversation, and the model has no way to miss what it can't see. If you're building agents that run for hours rather than seconds, this is a failure mode you should assume you have.

Why agents compact at all

Long-horizon agents fill their context window. A coding agent exploring a repository, a support agent working through a queue, a research agent reading sources — each turn appends tool calls and results, and the transcript grows fast. The standard answer is compaction: summarise the older part of the conversation into something shorter and carry on.

This is not a hack; it's how long-running agents work. And it's necessary for a second reason beyond space, which is that models get worse as context grows regardless of the limit. Chroma's Context Rot report evaluated 18 models — including frontier models from every major lab — and found the effect universal:

Models do not use their context uniformly; instead, their performance grows increasingly unreliable as input length grows.

Their results are worth knowing in detail because they set up the problem. Performance dropped 20–50% between 10k and 100k+ tokens on simple retrieval tasks. Even a single distractor measurably hurt. And in the most counterintuitive finding, models did better on shuffled text than on a logically coherent document — structure hurt, consistently, across all 18.

So compaction is doing real work. Keeping everything is not a viable alternative. The problem is what compaction chooses to throw away.

The measurement

A June 2026 preprint, Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents, put a number on it. The author built a benchmark called ConstraintRot — long-horizon agent scenarios with deterministic tool-call evaluation, so a violation is an observed fact rather than a judgement call — and ran 7 model families across 1,323 episodes.

The framing sentence from the abstract is the whole problem in one line:

In-context governance constraints that agents reliably obey while visible can be silently removed by compaction, causing the same agent to perform prohibited tool actions later in the session.

The numbers:

• 0% → 30%. Violation rates were zero while the policy stayed visible, and rose to 30% after compaction — reaching 59% on the worst-performing model family.
• 0% vs 38%. Split by whether the constraint survived the summary: when it survived, violations stayed at 0%; when it was dropped, they hit 38%.
• An adversarial strategy designed to push constraints out of the summary defeated every model evaluated.
• A mitigation the author calls Constraint Pinning returned violations to 0% on the benchmark.

Read the second bullet again, because it isolates the cause cleanly. Same models, same tasks, same tools. The only variable that mattered was whether the rule was still in the window.

Why summarisers drop rules specifically

Here's the mechanism, and it's almost inevitable once you see it.

A summariser is asked to compress a transcript while preserving what matters. What it has to work with is a record of things that happened: searches run, files read, orders reviewed, decisions made. It keeps the narrative.

A standing prohibition is not a thing that happened. "Never issue a refund above ₹5,000 without human approval" generates no events — its entire observable footprint over forty turns is the absence of refunds above ₹5,000. To a summariser optimising for salience, a rule that has never been triggered looks exactly like a rule that isn't relevant. The better-behaved your agent has been, the less evidence there is that the constraint mattered.

Three things make it worse:

1. Rules live at the top. System policies are conventionally the first thing in the context, and compaction eats the oldest region first. The guardrail is structurally first in line.
2. Surviving isn't the same as being followed. Chroma's positional findings mean a constraint that gets folded into the middle of a long summary may still be poorly attended to. Present in the tokens ≠ present in the reasoning.
3. There's nothing to alert on. No exception, no failed assertion. The agent behaves consistently with the context it has. That's what "silently" means in the paper's title, and it's why this doesn't show up in testing that only runs short sessions.

What to actually do

1. Make the policy non-compactable. Whatever your framework calls it, the system rules should live outside the region that gets summarised and be re-injected verbatim every turn. That's the pinning result, and it's the cheapest fix on this list.
2. Enforce in the tool layer, not the prompt. A prompt that says "don't refund over ₹5,000" is a request. A refund tool that rejects amounts over ₹5,000 without an approval token is a rule. This is the same argument as sandboxing agent code: the model can't be the security boundary, and a compacted context is a vivid demonstration of why.
3. Re-check constraints at the moment of action. Agents plan early and act late. Validate against policy when the tool is called, not when the plan was drafted — the context may be materially different by then.
4. Run evals past the compaction boundary. If your test sessions are ten turns long, you cannot see this class of bug at all. Long-horizon behaviour needs long-horizon tests, which is another reason to grade the path rather than the answer.
5. Log compaction as a first-class event. Record when it fires and what survived. When an agent does something inexplicable, the first question should be whether it still had the rules.

Our opinion

The useful lesson here isn't about compaction — it's that anything you put in a prompt is advisory, and compaction just makes that unusually visible. Prompt-based guardrails have always been a request that the model cooperate. Usually it does, which is exactly what makes them dangerous: they work in every demo, every test, and most of production, right up until the context changes underneath them.

We'd put this in the same category as client-side validation. Nobody thinks disabling a submit button is a security control, because we all learned that lesson decades ago. Instructions in a system prompt currently enjoy the credibility that client-side validation had in 2004 — they look like enforcement, and they aren't. If a constraint matters enough to write down, it belongs somewhere a summariser can't reach: in the tool's own permission check, in an approval gate, in the API that the agent has to go through.

Two honest caveats. The paper is a single-author preprint from June 2026 and hasn't been peer-reviewed, so treat the specific percentages as directional rather than settled — though the underlying claim is easy to reproduce yourself, and the 0%-versus-38% split is hard to argue with as a mechanism. And compaction isn't the villain: it's necessary, it's well-motivated, and the fix isn't to compact less. It's to stop putting load-bearing rules in the compactable part of the system, which is good design regardless of what the benchmark says next year.

How Ashvara helps

We build agents where the constraints live in the plumbing — permission checks on the tools themselves, approval gates in front of anything irreversible, and evaluation that runs long enough to catch what a ten-turn test never will. The prompt is where we tell an agent what to do, not where we stop it doing the wrong thing.

That's the practical half of our AI solutions work, and it builds on the same discipline as context engineering generally. If you have an agent that behaves impeccably in testing and occasionally surprises you in a long session, tell us what it did — check the compaction log first, because it's often this.

Sources: Governance Decay: How Context Compaction Silently Erases Safety Constraints in Long-Horizon LLM Agents (Chen, June 2026 — preprint, not peer-reviewed); Chroma, Context Rot: How Increasing Input Tokens Impacts LLM Performance (July 2025, 18 models evaluated).