The problem

Most “is it safe to use AI on our data” conversations stop at the legal question and never get to the competitive one. Anthropic’s ToS is genuinely clean: they don’t train on your prompts, you own your outputs, and that’s all true. But it misses the actual mechanism doing the damage. Every prompt from every paying customer gets anonymized and pooled — what a vendor calls a “data tumbler.” What comes out the other end isn’t your data anymore, legally, but it is a high-resolution map of which workflows are being automated at scale and where the money is moving. That map is the thing labs use to decide their next product line, and the evidence trail is not subtle: Cursor usage scaled and Claude Code shipped, Figma MCP usage spiked and Claude Design shipped, security-tool usage spiked and Claude Security shipped. Sending your business workflows through an AI lab isn’t data theft. It’s disclosing your competitive intelligence to the one party best positioned to act on it — and most companies have no framework for telling which of their AI usage falls into that bucket versus which is completely inconsequential.

The approach

I pulled this out of two long-form technical talks and turned the scattered claims into an applicable decision framework, because “be careful with AI and data” isn’t actionable and neither party in that conversation usually means the same thing by it.

The core of it is a single diagnostic test: “if a competitor could read this agent’s full trace, would it matter?” That question sorts work into two buckets fast. Commodity work — CRUD, DevOps glue, boilerplate — sits inside what every model has already seen ten thousand times; a competitor reading it learns nothing. IP-dense work — hand-built eval sets, proprietary workflows, user-data patterns specific to your business — is exactly what falls outside that distribution, and it’s the 20% of usage that actually needs protecting (the other 80% doesn’t, which matters because treating everything as sensitive is how security policies become theater nobody follows).

The second piece is the AI Sovereignty Ladder: five tiers running from the free consumer plan (you’re the product, no protection exists) through commercial API/subscription (aggregate sampling only), a model cloud like Bedrock or Vertex (the lab has no data rights at all inside your cloud contract), owning your own LLM gateway/routing layer (you own every trace, you can switch providers without leakage), up to renting GPUs and self-hosting open-weight models (you own the weights and every trace becomes a compounding asset you can fine-tune on later). The framework pairs the two: classify the work with the trace test, then match it to the minimum tier that’s actually proportionate — instead of either “ban AI everywhere” or “trust the ToS everywhere,” which are the two policies most orgs actually run today.

What I learned

The most useful move here was reasoning from incentives instead of from policy language. The real question isn’t “does this vendor care about my privacy” — it’s “what would a documented ToS violation cost them.” For a US enterprise-scale lab, the answer is close to existential: one confirmed violation triggers a mass enterprise exodus, so the incentive alignment is real and it’s what’s actually protecting you, not goodwill or an audit. That same lens is what makes overseas API providers a genuinely different risk category, independent of model quality — the enforcement mechanism that makes “trust the ToS” rational simply isn’t present across a geopolitical boundary the way it is domestically.

The other thing that surprised me pulling these two talks together: the closed-vs-open tradeoff isn’t just about privacy, it’s about who accumulates the compounding asset. Every trace your team generates against a closed API is, per that vendor’s ToS, off-limits for you to train on — you’re improving their model, not yours. The same trace against an open, self-hosted model is a proprietary asset you keep, and two companies running the same product for two years on opposite sides of that line end up with very different long-term positions, independent of which model was “smarter” on day one.

Where this could go

This reads as a personal AI-hygiene checklist but it’s really a procurement and security policy in miniature. Any org running Claude, ChatGPT, or Gemini across multiple teams today almost certainly has a single binary policy — “approved for use” or “not” — applied uniformly regardless of what’s actually flowing through the prompt. The trace test plus the sovereignty ladder is the version of this that scales: classify workflows by IP-density once, route commodity work through the cheapest commercial tier, and route the 20% that’s actually defensible — the proprietary evals, the workflows that encode years of domain logic — through a tier where the org, not the vendor, owns the traces. That’s a natural fit for an LLM gateway that does the classification and routing automatically rather than relying on every engineer to self-police it, and it’s the kind of thing a platform or security team would own rather than leave to individual judgment call by call.

Takeaway

The legal question (“did they train on my data?”) and the competitive question (“did I just tell them where the market is moving?”) have different answers, and most companies are only asking the first one.