Introducing PII Shield: A Privacy Proxy for Every LLM Call

Why do we need a utility like PII Shield?

Let’s start with an uncomfortable truth: modern AI systems are swimming in sensitive data - often without us fully realizing the risks we’re taking.

• The data is sensitive. Names, emails, phone numbers, credit cards, IBANs, SSNs, Aadhaar, PAN, Driving Licenses, UPI IDs, addresses - all of them frequently appear in user prompts, RAG documents, and tool inputs.

• Prompts leaving trust boundary: Even when the model lives in your own subscription - Azure OpenAI, a Foundry deployment, a self-hosted OSS model - the prompt still travels through SDKs, gateways, observability stacks etc. Before it gets to LLM, the inference layer often logs, evaluates, or fine-tunes against what it sees. Any of those hops could be a potential exposure of raw PII if not handled well.
• Changing regulatory asks: EU AI Act, India's DPDP Act, CCPA, HIPAA, PCI-DSS and a dozen sectoral rules now explicitly cover AI-driven processing. “Not knowing that your data may get stored elsewhere” is not an option anymore.

• The ad-hoc fixes don't scale. Most teams start with a regex or two. Then they add another team, another use case, another language, another ID format — and the regex file becomes a tar pit nobody wants to touch.

The damage from getting this wrong is asymmetric. When controls hold and decisions are sound, nothing remarkable happens. The AI feature launches, threats are contained, and the system does what it was designed to do. But when security assumptions fail, the consequences compound fast. A single misstep can trigger public disclosures, regulatory scrutiny, financial penalties and more. That imbalance is why security can’t be treated as a finishing step. We need a single, well-tested, observable layer that sits between every AI application in the organisation and every model it talks to - and makes sure raw PII never crosses that boundary.

Introducing PII Shield

"PII Shield" is an intelligent anonymization layer that sits between your AI application and the LLM. It detects PII, applies a configurable privacy action per entity type, and - when you want it to - reverses the anonymization after the LLM has done its work. 

At a glance: 

• REST API: /anonymize, /deanonymize and /apps - built with FastAPI.

• PII detection via Microsoft Presidio, with a pluggable NLP backend (spaCy, ONNX, Stanza, HuggingFace Transformers).

• Custom recognisers for India specific identifiers: Aadhaar, PAN, Driving Licence (all 36 states), PIN code, UPI ID, Indian phone numbers — plus 100+ Presidio defaults.

• Per-entity strategies: replace, hash (SHA-256), encrypt (PQC ML-KEM-768 + AES-256-GCM) or fake

• Multi-tenant: register applications via POST /apps, configure strategies per app, route via the X-App-Id header.

• Reversible by design: the "Anonymise → LLM → De-anonymise" sandwich pattern means the LLM only sees anonymized values (e.g. {{PERSON_1}} and {{EMAIL_ADDRESS_2}} etc.), never the real values.

• Library mode: pip install pii_shield and use the engine directly for batch jobs, no FastAPI required.

• Observability built in - Open Telemetry traces/metrics/logs, dual-backend (OTLP for local Grafana LGTM, Azure Monitor for production), pre-built Grafana dashboards

Before we go deep into how this works, let's understand a few key concepts that this utility is centered around. 

Entities and recognisers

An entity is a single, contiguous piece of PII that PII Shield treats as one unit of redaction - for example PERSON, EMAIL_ADDRESS, PHONE_NUMBER, CREDIT_CARD, IBAN_CODE, LOCATION, IN_AADHAAR, IN_PAN, IN_DRIVING_LICENSE, IN_UPI_ID. Each entity has:

• a type (the label above) that determines which placeholder format and which anonymisation strategy applies.
• a span in the original text - start and end character offsets - so the anonymiser can replace exactly the right slice without disturbing the surrounding text.
• a confidence score (0.0–1.0) reported by whichever recogniser found it, which the downstream code uses to drop low-quality hits or break ties between overlapping matches.
• a stable identity within the request - two occurrences of "Rahul Sharma" are the same entity instance, so they collapse to a single {{PERSON_1}} placeholder rather than {{PERSON_1}} and {{PERSON_2}}. That property is what makes coreference survive the round-trip through the LLM.

A recogniser is the component that actually finds entities of a given type in text. PII Shield uses three recognizers, and they typically run together:

• NLP recognisers rely on the language model loaded by the Presidio Analyzer (spaCy / ONNX / Stanza / Transformers). They're the right tool for entities that have no fixed shape such as PERSON, LOCATION, ORGANIZATION, NRP where context and grammar matter more than format.  E.g. "Rahul met Priya in Bengaluru" can't be detected with regex; the model must understand that those tokens are people and a place.
• Pattern recognisers (extension of the PatternRecognizer Class) are small Python classes that combine three things:
  1. one or more regex patterns  (e.g. the 12-digit Aadhaar layout, the AAAAA9999A PAN structure, a state-prefixed Driving Licence number);
  2. a list of context keywords that boost confidence when they appear nearby (aadhaar, uidai, licence, dl, pan, card, dob); and
  3. a base confidence score plus an optional validator function for IDs with a checksum (Aadhaar's Verhoeff digit, PAN's structural rules, IBAN's mod-97).
  The regex says "this could be an Aadhaar"; the context boost says "and the word 'aadhaar' is three tokens to the left, so I'm now very sure" and the validator says "the checksum agrees, accept it." That layering is what keeps false positives in check on noisy CRM notes and chat transcripts.
• Hybrid / deny-list recognisers sit on top, for things like an internal project codename list or a customer-id format that's specific to your business.

When the Analyzer runs, every recogniser scores the same span independently; overlapping hits are reconciled (highest-confidence wins, longer spans absorb shorter ones), and a final list of (entity type, start, end, score) tuples is what the anonymiser sees. 

Adding a new recognizer is easy with the YAML configuration approach. Refer to the below document for details 

how-to-add-a-recognizer

The Sandwich Pattern

We call it a sandwich pattern because two thin layers of PII Shield wrap around the LLM call: a) the top slice (anonymise) replaces every detected entity with a stable, opaque placeholder before the prompt ever leaves your trust boundary, b) the bottom slice (de-anonymise) puts the original values back when the response returns. The LLM in the middle is just the filling - it reasons over {{PERSON_1}} and {{EMAIL_ADDRESS_2}} instead of "Rahul Sharma" and "rahul@example.com", and it has no idea that anything was ever swapped. This is what makes the pattern so easy to retrofit: your application keeps its existing prompts, the LLM keeps its existing API, and the only "PII Shield" component that knows the real values is the very short-lived mapping. Coreference still works end-to-end because {{PERSON_1}} is the same person everywhere in the prompt - and, almost always, everywhere in the response too. Encrypted entities use ML-KEM-768 + AES-256-GCM by default — a NIST-standardised post-quantum KEM. 

High level design

The diagram below depicts key components of “PII Shield”. 

• User / Client App: The end user (or a downstream service) who initiates the request that eventually reaches an AI workload. They are unaware of the redaction layer; from their perspective the AI app responds with full, restored PII.

• AI Application (Agent / RAG / Chat): The customer's GenAI workload (chatbot, agentic flow, RAG pipeline). It owns the business logic but does not call the LLM directly with raw user data. Instead, it routes every prompt through PII Shield first and only forwards the anonymized result. This keeps the AI app free of redaction logic and lets the same shield be reused across multiple workloads.

• PII Shield (FastAPI): The central, stateless service. Every request flows through this single boundary, which makes it the natural place to enforce policy, rate-limit, audit, and instrument. Internally it is composed of several cooperating layers:

  • REST API - the public surface: POST /anonymize_unique (detect + redact in one call, returning a session ID), POST /deanonymize (restore using that session ID), plus /apps and /apps/{id}/config for tenant onboarding and per-app strategy configuration. Health, metrics, and OpenAPI docs are exposed alongside.

  • Presidio Engine: the detection-and-redaction core, built on Microsoft Presidio and split into two stages - 

    • Analyze:  runs the configured NLP backend (spaCy for fastest CPU inference, ONNX for quantized speed, Stanza for higher recall, or  Hugging Face Transformers for state-of-the-art models like IndicNER) and PII Shield's own custom recognizers (Aadhaar, PAN, IFSC, UPI, IN_BANK_ACCOUNT, IN_PHONE, IN_PIN_CODE, CKYC, PRAN, APAAR, GeoCoordinate, NaturalDate, etc.). Each detected span carries a confidence score and is disambiguated using context-keyword boosting.

    • Anonymizer (Operators): applies the per-entity strategy chosen by the app config: replace (placeholder tokens like {{IN_AADHAAR_1}} for the LLM-sandwich pattern), hash (irreversible SHA-256, for analytics on hashed values), encrypt (reversible AES-GCM or post-quantum Kyber for compliance-sensitive flows), or fake (format-preserving synthetic values from Faker for safe demo / synthetic-test datasets).

  • State Stores: there are two logical stores, both backed by Redis but with different lifetimes and access patterns:

    • Session Mapping Store: short-lived UUID→entity-map records that power the sandwich pattern (anonymize → call LLM → de-anonymize the response back to original values). The default TTL is intentionally short (seconds-to-minutes) so plaintext mappings don't linger.

    • App Registry: long-lived per-app configuration: entity-type strategies, allow-lists, entity-type-allow-lists, encryption-key references, and a curated allow-list of fictional brand names. This is what makes the platform multi-tenant with per-tenant policy.

    • Telemetry: Uses OpenTelemetry SDK, and auto-instruments every stage of the pipeline: span per request, metrics for entity-type counts/anonymization latency/cache hit rate, and structured logs with trace correlation. The same OTLP stream can be split between Azure Monitor and a self-hosted Grafana stack.

  • LLM (Azure OpenAI / OSS): The downstream model. PII Shield's contract guarantees it only ever receives anonymized text; raw PII never crosses the trust boundary into the model provider's tenancy. This satisfies most data-residency, DPDP, and GDPR concerns by construction rather than by audit after the fact.

  • Azure Monitor / OTLP → Grafana: This is where OpenTelemetry data lands. Operators get pre-built Grafana dashboards (request volume by entity type, p95/p99 latency per stage, per-app usage, error breakdowns) so production issues — a misbehaving recognizer, a slow NLP backend, a noisy app — are diagnosed from a single pane.

• Playground & Admin UI: Two Streamlit front-ends come bundled with the platform. The Playground lets developers paste arbitrary text, switch NLP backends, and instantly see what gets detected and how it is redacted — invaluable for tuning recognizers and demos. The Admin UI lets operators register applications, set per-entity strategies, manage allow-lists, and view live entity stats. Both call the same REST API, so anything the UI does is reproducible from a script or CI pipeline.

Request flow at a glance - A user types a message into the AI application — say, a chatbot that helps customers with banking queries. Before the application sends anything to the language model, it hands the raw text over to PII Shield through the /anonymize_unique endpoint. PII Shield runs the text through its detection pipeline, swaps every piece of personal information for a stable placeholder token, and returns two things to the AI app: the redacted text and a short-lived session ID. The application then calls the LLM with this safe, placeholder-laden prompt — the model never sees a real Aadhaar number, account number, or phone number.

Once the LLM responds, the AI app posts that response back to PII Shield's /deanonymize endpoint along with the same session ID it received earlier. PII Shield looks up the session, restores every placeholder in the LLM's reply with the original value, and hands the fully restored text back to the application, which delivers it to the user. From the user's perspective the conversation feels completely natural — they see their own details, in their own words, exactly as they'd expect. 

Behind the scenes, the session mapping that made the round-trip possible is automatically dropped from Redis the moment its TTL expires, so plaintext PII never lingers in the system longer than it has to. In parallel, every stage of this flow emits OpenTelemetry traces, metrics, and logs to the observability stack, giving operators a clear, real-time view of what was detected, how long each step took, and how the system is behaving in production.

The entire set-up could be deployed on premises as well as on any of the clouds by replacing components with corresponding cloud offerings. Below is an example in Azure - Container apps is used to host the REST services, Azure Cache for Redis is for the state store, Application Insights, Log Ananlutics and Azure Managed Grafana are used for telemetry. 

Please refer to the codebase here for the implementation and deployment scripts (local & Azure).

code repository pii-shield

Example deployment architecture for Azure

How does it work?

The sandwich pattern explained earlier has three phrases: 

• Anonymize
• LLM Processing
• De-anonymize

Anonymize

When the app sends POST /anonymize with "Call Rahul Sharma at rahul@example.com about Aadhaar 2345 6789 0123.", PII Shield does five things in order:

1. Resolve config. If an X-App-Id header is present, the app's per-entity strategy map is loaded from Redis (with a short in-process cache to keep the hot path fast). If absent, global defaults apply.
2. Detect. The Presidio Analyzer runs the configured NLP backend (spaCy / ONNX / Transformers) plus all built-in and custom pattern recognisers. The output is a list of spans — (entity_type, start, end, score, text) tuples — possibly overlapping, possibly with low-confidence noise.
3. Post-analyse. Adjacent locations and PIN codes get merged into a single ADDRESS, overlapping detections are deduped (longest wins, ties broken by score), context keywords boost confidence, and entries below threshold are dropped. This is the difference between "raw Presidio output" and "something safe to anonymise".
4. Assign unique placeholders. For each distinct PII value, a numbered placeholder is minted: {{PERSON_1}}, {{PERSON_2}}, … The numbering is per entity type, per request, and ordering is stable — the same person mentioned three times in the input gets the same {{PERSON_1}} everywhere. This is what lets the LLM reason about coreference correctly.
5. Apply operators. For each entity, the per-app strategy decides what to write into the output:
   • replace → the placeholder above (reversible; mapping kept).
   • hash → {ENTITY}_<sha256-prefix>> (one-way; not added to entity_mapping — there's nothing to restore).
   • encrypt → {{{ENTITY}_ENC_<base64-ciphertext>}} using ML-KEM-768 + AES-256-GCM (reversible without keeping a mapping; the ciphertext is self-contained).
   • fake → a synthetic value from the locale-aware faker (irreversible; useful for demos and synthetic test data).

The reversible mapping {placeholder → original} is then saved to the session store (in-memory dict keyed by a fresh UUID by default; swappable for Redis or a database). The response carries the id, the anonymized_text, and entity_mapping for inspection:

{
   "id": "a1b2c3d4-...",
   "anonymized_text": "Call {{PERSON_1}} at {{EMAIL_ADDRESS_1}} about Aadhaar {{IN_AADHAAR_1}}.",
   "entity_mapping": {
     "{{PERSON_1}}": "Rahul Sharma",
     "{{EMAIL_ADDRESS_1}}": "rahul@example.com",
     "{{IN_AADHAAR_1}}": "2345 6789 0123"
   }
 }

LLM Processing

The app sends the anonymised text to the LLM exactly as it would have sent the original. The model treats {{PERSON_1}}, {{IN_AADHAAR_1}}, etc. as opaque proper nouns and reasons over them naturally. Because each distinct value got a stable, unique placeholder, the model doesn't lose coreference: "Tell {{PERSON_1}} that their Aadhaar {{IN_AADHAAR_1}} has been verified" still makes perfect sense.

The response usually preserves the placeholders verbatim — sometimes the LLM rewrites half the sentence, but as long as the placeholders are still in there somewhere, de-anonymisation will work.

De-anonymize

The app calls POST /deanonymize with the session id and the LLM's response. PII Shield does three things:

1. Fetch the mapping for that id from the session store (single dict lookup; sub-microsecond).
2. Replace placeholders in the text — longest placeholder first. This matters: a naïve replacement could let {{PERSON_1}} clobber the inner part of {{PERSON_10}}. Sorting by length descending is a small detail with big consequences.
3. Decrypt encrypted entities in-place. Encrypted entities don't appear in the session mapping (the ciphertext is the placeholder), so they're handled separately: any {{ENTITY_ENC_…}} token in the text is base64-decoded and decrypted with the configured backend (PQC by default; Fernet auto-detected for backwards compatibility).

Importantly, the input text to /deanonymize does not have to match what /anonymize produced. The LLM may have rewritten everything around the placeholders - added text, translated, summarised - and PII Shield will still cleanly restore the PII wherever the placeholders appear. Only the placeholders trigger replacement; everything else passes through untouched.

The library mode

The application can also be used as a library if you do not want to deploy it as a service for batch use cases. Here is an example: 

from pii_shield import PiiShieldEngine engine = PiiShieldEngine() # load NLP model once, reuse result = engine.anonymize("Rahul's Aadhaar is 2345 6789 0123") # "<PERSON_1>'s Aadhaar is <IN_AADHAAR_1>" restored = engine.deanonymize(result.anonymized_text, result.entity_mapping)

The BatchProcessor currently handles CSVs and DataFrames with thread/process parallelism - useful for ad-hoc file processing and batch offline pipelines.

What's next

PII Shield is the foundation. Where it really starts to pay off is when you wire it into the rest of your AI stack so that no developer can accidentally bypass it. There are two more blogs in the series:

PII Shield as middleware in Microsoft Agent Framework

Agents are versatile. They invoke tools, fan out to sub-agents, cache intermediate context, and call LLMs from places you didn't expect. The next post will show how to plug PII Shield in as a middleware in Microsoft Agent Framework — automatically anonymising every prompt going out to a model and de-anonymising every response coming back, transparently to the agent author. No more "did we remember to scrub this one?".

PII Shield + Azure API Management (APIM)

For organisations standardising on APIM as their LLM gateway, the natural place for PII Shield is inside the policy pipeline. The follow-up post will cover integrating PII Shield with APIM so that every request to an LLM backend — regardless of which app or team made it — is intercepted, anonymised, forwarded, and de-anonymised on the way back. One policy, organisation-wide privacy.