A tester at Brightdesk, a fictional helpdesk SaaS, changed one field in the request payload and got draft replies citing another customer's internal articles. Nothing in the model misbehaved. Every customer's articles lived in one vector index with a `tenant_id` field, the assistant's retrieval tool took `tenant_id` as an argument, and the orchestrator filled it from what the web client sent. The tenant boundary was whatever the caller said it was.
That is the usual shape of the problem. A SaaS product arrives with a sound tenancy model, with a tenant column on every table and row-level security behind it, and then the AI feature adds components that were never part of it: an index built once for everyone, a cache shared because hit rates are better, a support agent's tool with one API token for the whole platform. Each is a reasonable shortcut and each is a path between customers. Our position: resolve the tenant once, from the credential, enforce it inside every shared component, and treat isolation as untested until a canary suite in CI says otherwise. The model's weights are the least interesting part of the problem.
To the provider, your whole platform is one customer
Providers isolate their customers from each other, and that isolation stops at your account. OpenAI's prompt caching documentation, for example, states that caches are not shared across organizations [3]. If every tenant of your product is served through one provider organization, the provider sees one customer, and anything shared at that level is shared across all your tenants unless you divide it. The same holds for everything you build around the model. Each component needs its own answer to one question: which tenant is this, and who decided?
Two tenants feed one platform. The prompt cache, vector index, conversation memory, tool credentials, and logs and evals are each a leak point unless tenancy is enforced inside that component.
Seven checks, with the reason for each
1. The tenant comes from the credential, never from the request
Every other check depends on this one. Resolve the tenant at the edge from the session token, API key or runtime key, then carry it as server-side context to retrieval, cache, memory, the tool broker and logs. Ignore any tenant field in the body, a header or model-generated tool arguments for authorization. If a partner genuinely acts across several tenants, check the asserted value against the set the credential may act for and log any mismatch as a security event.
The model in particular must never choose the tenant. A tool schema with a `tenant_id` parameter is an invitation for an injected instruction to change it, which is exactly the Brightdesk bug.
2. The vector filter runs inside the search, and an empty filter is an error
Retrieval is the most common real leak. OWASP's LLM08:2025 Vector and Embedding Weaknesses names it: where several classes of users share one vector database, context can leak between users or queries. Its prevention guidance calls for fine-grained access controls, permission-aware vector stores and strict logical and access partitioning of datasets [4].
The tenant is resolved from the runtime key during authentication and applied by the retrieval service, so results come only from Tenant A's partition. A tenant field supplied by the client is ignored.
Three details decide whether the filter holds. Apply it inside the search, not after: post-filtering a global top-k returns fewer results for small tenants and invites a "fall back to unfiltered" fix that leaks. Reject an empty filter instead of treating it as "match all". And give your most sensitive tenants their own index or namespace, so a missing filter finds nothing instead of everything.
3. Tenant data sits after the shared prompt prefix
Prompt caching reuses computation for a prefix seen before, so a cached prompt returns faster and response time depends on what other requests contained. Gu and colleagues built statistical audits for this, detected prompt caching in 8 of 17 production API providers and global cache sharing across users in 7 of them [1]. The attack relies on cache hits from prefix matches; the researchers disclosed to affected providers in October 2024, and at least five made changes [1]. Self-hosted serving has the same channel. Wu and colleagues showed that KV-cache sharing in frameworks such as SGLang and vLLM, which reuse cache across users with identical token sequences, lets one user reconstruct another's prompts, an attack they call PromptPeek [2].
The preconditions are specific, which is what makes the risk manageable: a cache shared with the victim, a long cacheable prefix that contains tenant data, the ability to submit guessed prefixes, and fine-grained latency measurement. So put the common system prompt first and tenant content after it, and include the tenant ID in the cache key, or run separate pools, wherever you control the cache. Do not mistake routing hints for a boundary: OpenAI documents `prompt_cache_key` as influencing routing, without pinning requests to a machine or guaranteeing a hit [3]. Brightdesk's review found the two platform-side preconditions in place: its system prompt opened with the customer's name and internal escalation policy, and its self-hosted serving stack shared KV cache across all requests. Reordering the prompt and adding the tenant ID to the cache key closed both. Coarser latency reporting raises the cost of measurement without closing the channel, and many requests from one tenant that share a long prefix and vary only a short suffix are the shape of a probe worth an alert.
4. Memory is keyed by tenant and user together
Long-term memory summarizes past conversations and retrieves them later. Keyed by user ID alone, where user IDs are not globally unique, or shared as "team memory" across an organization boundary, it surfaces one tenant's facts in another's session. Use a composite key and the same server-side filter as retrieval.
5. No model is tuned on pooled tenant data
A model fine-tuned on several customers' data is a shared store with no access control, and anything it memorized can be produced for any tenant that prompts well. Tune per tenant, or only on data every tenant has agreed may be pooled, after de-identification.
6. Tools carry per-tenant credentials
An agent that calls a CRM, ticketing system or storage bucket with one platform-wide token can reach every tenant's records, scoped only by the model choosing the right parameters. A prompt injection in one tenant's document then becomes a query against another's data. Issue per-tenant credentials, or per-tenant scoped tokens from a broker, so the downstream system holds the boundary when the agent is wrong. Brightdesk swapped its platform-wide ticketing token for broker-issued per-tenant tokens, so a wrong tool call fails at the ticketing API.
7. Logs, eval sets and rate limits are per tenant
Logs and eval datasets mix tenants by default. A support engineer debugging tenant A sees tenant B's prompts in the same trace view; an eval set built from production traffic goes to a vendor or gets replayed into another tenant's test environment. Tag every record with the tenant, restrict access by it, and build eval sets per tenant or from de-identified data. Capacity counts too: one tenant's runaway agent can exhaust a shared rate limit or token budget for everyone, so enforce per-tenant limits ahead of the shared provider quota.
Prove it with canaries on every build
Isolation that is not tested regresses with the next refactor. Create at least two test tenants, seed each with canary documents, memory entries, tickets and log lines carrying unique random strings, and run the suite on every change to retrieval, caching, memory or tool code. Any canary that crosses fails the build.
| Leak surface | Control | Isolation test |
|---|---|---|
| Prompt or KV cache [1][2] | Tenant data after shared prefix; per-tenant cache keys | Seed a long canary prefix as B, then compare latency for it and a random prefix as A; a consistent gap means the cache crosses tenants |
| Vector index [4] | Server-side filter from credential; per-tenant namespace | Query for B's canary while authenticated as A, directly and through the assistant |
| Conversation memory | Composite tenant and user key | Recall B's canary fact from an A session |
| Pooled fine-tune | Per-tenant tuning or consented, de-identified pools | Prompt for canary strings planted in training data |
| Tool credentials | Per-tenant or brokered scoped credentials | Ask the agent to fetch a B record as A, through each tool |
| Logs and evals | Tenant tags, access by tenant, per-tenant datasets | Search A's views, traces, exports and eval sets for B's canary |
| Rate limits | Per-tenant limits and budgets | Load one tenant and watch another's error rate |
| Every internal service | Tenant from the credential only | Send forged tenant fields in the body, headers and tool arguments, and a request with no tenant context; expect results only from the authenticated tenant, and a rejection rather than a global answer when context is missing |
What a gateway decides, and what stays in your services
Intertrace's gateway derives tenancy from the `itr_rt_*` runtime key during authentication, never from the request body, and that tenant scopes every LLM, MCP, service and agent-to-agent call it decides. It applies access rules and DLP to tool-call arguments, so a call can be held or blocked before it reaches a downstream system, and records each decision with the rule and layer that made it. It does not partition your caches, indexes or memory stores. Those boundaries live in your services, and the canaries are how you prove them.
The search to run on Monday
After its fix, Brightdesk replayed the tester's forged request, confirmed it returned only the tester's own articles, then searched 90 days of retrieval logs for responses whose cited documents belonged to a different tenant than the requester. Run that search on your own logs. If you cannot, because records do not carry both the authenticated tenant and the tenant of every retrieved document, that is the first thing to fix: without it, you cannot tell whether the boundary has ever held.
