Vireo Mutual, a fictional insurer headquartered in the EU, gave its claims handlers an assistant to summarize claim files, draft letters to claimants and find similar past claims. The residency review asked one question: is the model endpoint in the EU? That was the wrong place to stop. Before a prompt reaches the model it has been written to an application log, passed through a proxy and possibly embedded into a vector store. Afterward, the exchange may sit in a provider's abuse-monitoring store, an observability backend and next month's evaluation dataset.
Residency is a property of every copy, not of the endpoint, and most copies are yours to place. This guide follows Vireo's assistant from the first design to the fixed one. It is not legal advice; privacy counsel decides your obligations, and the goal is to hand them an accurate map.
The first design answered one question out of six
The first version used one global model endpoint, logged every request to the application's log service, embedded all past claims into a managed vector database, and copied sample conversations into an evaluation tool. Abuse monitoring was on by default. Claim files routinely held names, policy numbers, bank details and medical reports.
Each spoke is a separate copy with its own location, retention and access list. Every one carries a region question, and the inference endpoint answers only one.
A copy inventory found six locations, four outside the EU or of unknown region: the log service, the vector database, the evaluation tool, and the model endpoint's processing location. The table below generalizes that inventory; build one for each AI feature before deciding anything else. Its "who sets the region" column surprises people: you control most copies, so residency failures are usually your own.
| Copy | What is stored | Who sets the region | Common failure |
|---|---|---|---|
| Client app logs | Request bodies, errors, stack traces with prompt text | You | Debug logging ships prompts to a global log service. |
| Gateway or proxy logs | Full requests and responses, or metadata | You or your gateway vendor | Payload logging enabled by default. |
| Provider inference | Transient processing of prompt and output | Provider, by endpoint and deployment type | A global deployment type processes anywhere. |
| Provider abuse monitoring | Prompts and outputs retained for review | Provider terms | Assumed off; it is on by default for many accounts. |
| Prompt and response caches | Cached prefixes or responses | Provider and you | Extended caching processed outside the region. |
| Provider stateful features | Stored responses, threads, files, batches | Provider, by feature | A feature excluded from residency terms. |
| Vector store and embeddings | Chunks of source documents plus vectors | You | Managed vector database in a default region. |
| Observability and evals | Traces, prompts, labeled datasets | You or your tooling vendor | Production prompts copied into a global eval tool. |
Two copies deserve emphasis. Embeddings are personal data whenever their source text is, and they sit beside the chunks they index. Evaluation datasets are built from production traffic because it is realistic, so they inherit its personal data, then get exported, shared with vendors and kept indefinitely.
In-region processing is not in-region access
Each copy has three locations, and providers often commit to them separately. Processing is where compute touches the data, such as the GPUs running inference. Storage is where data rests: logs, caches, stored conversations, files. Access is where the people and systems that can read it sit, including support staff and abuse reviewers.
The EDPB's recommendations on transfers make the third point explicit: remote access from a third country, for example in support situations, and storage in a cloud outside the EEA are also considered transfers [4]. An endpoint that processes in Frankfurt but lets reviewers elsewhere open stored prompts is not a purely in-region system.
Provider regional terms, read down to the exclusions
The team read each provider's data documentation, exclusions as carefully as headlines. Two examples, as the pages stood when this guide was written; verify against current ones.
OpenAI's API data documentation says abuse monitoring logs are kept for up to 30 days unless longer retention is required by law, and that Zero Data Retention and Modified Abuse Monitoring, which exclude customer content from those logs, need prior approval [1]. Its residency table separates regional storage from regional processing: several regions (including the UK, Canada and Japan) offered storage at rest but not regional processing, and European regional processing requires an approved arrangement such as Zero Data Retention or Modified Abuse Monitoring. Residency does not cover system data such as account, usage and billing information; extended prompt caching in regions without regional processing may process and temporarily store content outside the region; and some endpoints, including Files, are not eligible for zero retention [1].
Microsoft's documentation for models sold by Azure says prompts and responses are processed within the customer-specified geography unless you use a Global or DataZone deployment type [2]. A Global deployment may process in any geography where the model is deployed; an EU DataZone may process in any EU member state. Data at rest, including the abuse monitoring store, stays in the resource's geography, and for EEA deployments the human reviewers are located in the EEA. Batch processing is a Global deployment type [2]. Modified abuse monitoring removes the storage and human review, and the documentation explains how to verify it through a `ContentLogging` capability set to false [2].
The pattern across both: processing and storage commitments are separate, reduced retention is an approved arrangement rather than a default, specific features and deployment types fall outside the commitment, and metadata is often excluded. None of that makes either option unsuitable. It means the setting you rely on must be recorded per feature, deployment and account, and checked.
What counsel needs is a transfer map
For EU personal data, residency questions usually become international transfer questions. Chapter V of the GDPR (Articles 44 to 50) governs transfers to third countries: the general principle in Article 44, then adequacy decisions, appropriate safeguards, binding corporate rules, and derogations for specific situations [3]. A transfer is lawful when one of those bases applies: data leaving the EEA is not automatically a violation, and data staying inside does not settle every obligation.
The EDPB's six-step recommendations start with "know your transfers": map where personal data is located or processed by importers, including onward transfers by your processors to sub-processors, and verify that what you transfer is limited to what is necessary [4]. For LLM traffic that map is the copy inventory, and data minimization does the most work: a prompt that never contained the claimant's name cannot transfer it.
The redesign: route each request by what it contains
One feature mixes data classes, so routing per application would have forced the strictest path onto product questions or the loosest onto medical reports. Vireo routed each request by the class of data it contained, using three classes mapped onto its existing classification policy.
Classification happens before any model is chosen. Public data can use any approved model, internal data goes to an in-region endpoint, restricted data is redacted or blocked, and logs stay in-region on every path.
- Public: published product information, public documents, code you already open-sourced. Any model on your approved list.
- Internal: business data without personal or regulated content, such as procedures and internal notes. Only endpoints whose processing and storage terms match your residency requirement.
- Restricted: personal data, special-category data, credentials, payment data, anything under contractual residency. Redact before it leaves your boundary, or block the request if redaction would break the task.
Classification has to be cheap and conservative. Combine what the application declares (this request carries a claim file) with what detectors find (identifiers, card numbers, health terms), and take the stricter class when they disagree. If classification fails or times out, treat the request as restricted. Classify responses too: a model can echo restricted data from context into the same logs and caches.
- Classify at the gateway. Requests tagged as claim work are restricted by default; detectors confirm identifiers, IBANs and health terms. Policy wording and product questions are public.
- Redact restricted fields before egress. Names, policy numbers and bank details become stable placeholders; the mapping stays in-region and the response is re-identified after it returns.
- Block what cannot be redacted. Free-text medical narratives the task cannot do without are blocked from any endpoint lacking regional processing and approved reduced retention, and handled by an in-region path or a person.
- Send internal and redacted traffic to an in-region endpoint whose processing, storage and abuse-monitoring terms were confirmed and recorded for this account and deployment type.
- Move the vector database and its source chunks to an EU region, and index redacted text where similarity search still works on it.
- Keep payload logs in-region with short retention, export only metadata to observability, and build evaluation sets from redacted traffic kept in the EU.
Redaction brought its own risks
The team tested for three. Placeholders were scoped per conversation, so the same claimant did not get a stable token a provider-side log could link across claims. Detector recall was measured on a labeled sample of real claim files, because a name written in lowercase inside a medical narrative is exactly what pattern detectors miss. And quasi-identifiers such as dates of loss, postcodes and rare diagnoses were reviewed with the privacy team, since together they can identify a claimant with every name removed.
Handlers kept the same assistant, and most requests still reached a capable model, since redacted summaries remain useful. The main cost was letter drafting, where re-identification had to run before a handler saw the draft.
Residency decays unless you test it
A new SDK version enables payload logging, an engineer switches a deployment type for capacity, a vendor adds a sub-processor. Test residency like any other control.
- Plant canary strings in test prompts and search every log, trace store, cache and eval dataset for them on a schedule. A hit outside an approved location is a finding.
- Restrict egress so the application and gateway can reach only approved model hostnames, and alert on any new AI provider destination.
- Record provider settings per account and deployment (endpoint, deployment type, retention arrangement, abuse monitoring) and diff them weekly against the baseline.
- Scan code and CI for direct provider calls and committed keys that bypass the gateway; every bypass is an unmapped copy.
- Re-read provider data pages and sub-processor lists quarterly, and whenever you adopt batch, files or stored conversations.
A gateway decides what leaves, not where it lands
Intertrace does not pin traffic or storage to a region and does not host models; region choices stay with you and your providers. It is the place to classify and enforce what may leave: it detects and redacts PII and secrets in requests and responses, applies DLP to tool-call arguments, and can block a request carrying a restricted class, under an organization data policy agents can tighten but not loosen. Its OpenTelemetry fan-out carries metadata only. It is also a processor that sees prompts, so it belongs in your copy inventory too.
Vireo's last step was the point of the exercise: the team took the finished copy inventory, with processing, storage and access location for each copy and the provider setting behind each, to privacy counsel for the transfer analysis. The question for counsel was no longer whether the endpoint was in the EU. It was whether each of six copies had a basis to be where it was, and most of them now sat in the EU.
