What actually happens to your prompts
Training opt-out and zero data retention are different controls - and ZDR isn't account-wide. It's per feature, and the ones that break it don't warn you.
8 min read
"We opted out of training" and "our data isn't retained" are two different guarantees, and most teams have the first while believing they have the second. Worse, zero data retention isn't a switch on your account — it's a property of each individual API feature, and adding the wrong one silently steps your data outside the arrangement without raising an error. If you can't name which features your app calls, you can't answer what happens to your users' data.
Three different things people mean by "private"
Almost every confused conversation about AI and data collapses three separate controls into one word.
- Not training on your data. On commercial API terms this is typically the default — Anthropic's API and data retention documentation states plainly that retained data "is never used for model training without your express permission." This is the guarantee most teams actually have.
- Not retaining your data. A separate question with a separate answer. A provider can decline to train on your prompts while still storing them for abuse monitoring, debugging, or a feature to function.
- Zero data retention. Prompts and responses are not stored at rest once the API response is returned. This is the one people mean when they say "it's private" — and it's the one you almost certainly don't have unless you negotiated it.
That third tier isn't self-serve. It requires a commercial agreement through a sales conversation, and — the detail that catches teams out — it's enabled per organisation, not per account. Spin up a second organisation and it starts without ZDR; enablement "does not automatically extend to other organizations under the same account."
The mechanism: ZDR fails open, HIPAA fails closed
Here's the part that should change how you build, and it's the sharpest thing in that documentation.
Both arrangements — ZDR and HIPAA readiness — cover only a subset of API features. But they behave in opposite ways when you use a feature outside the subset:
- HIPAA readiness fails closed. Send a request using a non-eligible feature and the API returns a
400naming the offending feature. You cannot accidentally process protected health information through an uncovered path; the platform stops you. - Zero data retention fails open. The documentation is explicit: "Nothing blocks the request." Using a non-eligible feature "is a choice to step outside your ZDR arrangement for that specific data," and that feature's own retention policy applies instead.
So a developer who adds file upload to a ZDR application gets no error, no warning, and no log line. The feature works. The files are now retained until explicitly deleted. Your privacy posture changed in a pull request that nobody flagged, because nothing in the system treats it as an exception.
A guarantee that degrades silently isn't a guarantee — it's a default you have to keep re-earning on every code review.
What quietly falls outside it
The pattern is consistent and worth internalising: stateful features cannot be zero-retention, because storing something is the point of them.
| Stays inside ZDR | Falls outside |
|---|---|
| Messages API, token counting | Batch API — 29-day retention |
| Thinking, effort, context editing | Files API — retained until you delete |
| Prompt caching, structured outputs (qualified) | Code execution — containers up to 30 days |
| Web search, web fetch | Managed Agents — transcripts persist until deleted |
| Client-side tools: bash, text editor, computer use, memory | MCP connector, agent skills |
Two nuances worth knowing. Some features are "Yes (qualified)" — your prompts aren't stored, but a bounded technical artifact is: prompt caching holds key-value representations in memory for the cache TTL, and structured outputs caches the JSON schema itself for up to 24 hours. Which produces a genuinely non-obvious rule: don't put sensitive data in schema definitions — property names, enum values, const values, or regex patterns — because the schema takes a different path from the message content.
And nothing is absolute. Even with an arrangement in place, content flagged by automated trust-and-safety systems may be retained for up to two years. Any promise you make to a customer should survive that sentence being true.
What to actually do
- Find out which tier you're on before you describe it to anyone. "We don't train on your data" is a claim you can probably make today. "Nothing is stored" usually isn't. Getting this wrong in a security questionnaire is worse than not having ZDR.
- Inventory the features, not just the endpoint. The right artifact is a list of every API capability your app calls, checked against the provider's eligibility table. Most teams have never written this down.
- Treat adding a stateful feature as a privacy decision. Files, batches, code execution, and hosted agents each change what's retained. Put that in the PR template, because the platform won't tell you.
- Check the platform, not just the vendor. Where a model runs through a cloud marketplace, the cloud provider is the data processor and its rules apply — the first-party arrangement doesn't follow the model across.
- Know what your capability choice costs. Some frontier models are designated Covered Models requiring 30-day retention and are unavailable under ZDR outright; a request from a zero-retention organisation returns a
400. That's a real trade-off between capability and retention, and it should be made deliberately rather than discovered. - Ask the better question: does this data need to leave the device at all? Every retention conversation assumes the data was sent. Often it didn't have to be.
Our opinion
The most reliable privacy guarantee is architectural, not contractual. A DPA is a promise about what someone else will do with your users' data. On-device processing is a statement that there is nothing to promise about. One survives a vendor changing its terms, a subprocessor being added, or a feature flag flipping in a sprint; the other depends on all three holding.
We build this way and we'll say plainly that it's a constraint, not a free win — on-device means smaller models, more engineering, and some things you simply can't do. Several of our apps process everything locally precisely because a journal or a food-allergen scan is data we'd rather never receive. When a product genuinely needs frontier capability, we send data and are honest that we've made a trade.
What we'd argue against is the middle position that most products occupy by accident: sending everything to an API, holding only a training opt-out, and describing it to users as private. That's not a lie anyone told deliberately. It's what happens when three different controls share one word.
How Ashvara helps
We design AI features with the data path decided up front — what runs on-device, what leaves, which API features are in play, and what that means for what's retained. Then we write it down, so the answer to a customer's security questionnaire is a document rather than an afternoon of archaeology. It's the same instinct behind why we build local-first and keeping health data on device.
That's part of our AI solutions work, and it's much cheaper to decide before launch than to retrofit when an enterprise buyer asks. If you're not sure which tier your product is actually on, tell us what you've built and we'll map the data path with you.
Sources: Anthropic, "API and data retention" — ZDR scope and per-organisation enablement, the feature eligibility table, fail-open versus fail-closed behaviour, qualified-retention features, Covered Model requirements, and retention of flagged content (platform.claude.com). Policies differ by provider and change over time; check your own provider's current documentation and contract.