AI
The Problem: PII Leaking Into LLMs Without Detection
Developers routinely paste customer support cases into LLMs while fixing bugs. A social security number inside that support case can travel through code generation and land in a GitHub commit. Separately, components that share a Python environment can expose database credentials to a code-generation model running in the same process. Neither scenario has a detection step, audit trail, or masking layer. The solution needed is an architecture, not a filter library.
Kubernetes as the Enforcement Layer
The architecture runs 23 containers across seven tiers: infrastructure, privacy, LLM adapters, safety, platform, observability, and UI. Each container image stays under 200 MB. Every container boundary exists because the components on either side carry different trust levels. Kubernetes provides the mechanics: CRDs for policy objects, mutating webhooks for sidecar injection, RBAC for least-privilege service accounts, config map hot-reload so policy updates apply without pod restarts, and horizontal pod autoscalers that scale each tier on CPU percentage, queue depth, or request rate independently.
The Privacy Policy Object and Sidecar Injection
A developer adds one label to a deployment and runs kubectl apply. The Kubernetes API server fires the mutating webhook, which intercepts the create or update event and injects a privacy sidecar. The failure policy is set to ignore, so a downed operator never blocks a deployment. The sidecar listens on port 15080, forwards text to Presidio for PII analysis, and writes every detection event to Postgres and Redis. The policy object specifies per-type actions: hash social security numbers, redact credit cards, replace names with a person tag. Enforce mode blocks requests when anonymization fails; audit mode logs and forwards.
Eight-Stage Pipeline: From Prompt to Sandbox Execution
Each pipeline stage maps to its own Kubernetes pod. Stage one checks the incoming prompt for PII. Stage two normalizes the query into an LLM instruction. Code generation runs on Qwen 3.5 9B. After generation, the pipeline re-scans the output for leakage, then runs a static AST scan drawing patterns from NIST, OWASP, and the Maestro framework. A security reviewer checks against known CVEs. Cisco Foundation SE, an open-source reasoning model built on Llama 3.1 architecture and released February 25th, provides the final security verdict before sandbox execution.
Agent Skill-File Poisoning and the Scanner Response
A poisoned skill MD file can instruct an agent to copy secrets and delete workspace files under the cover of a compliance step. Dasgupta demonstrated this at an OWASP summit: the agent executed copy, delete, and exfiltration commands because the guardrail never fired against instructions embedded in the MD file. The pipeline’s skill scanner detects that pattern, classifies it as secret exposure and destructive action per OWASP agentic categories, corrects the MD file, and removes the unsafe instructions. The corrected file appears on the observability canvas with each removed step labeled explicitly.
Observability Dashboard and Open-Source Release
All 23 pods emit Prometheus metrics. The observability dashboard aggregates pipeline session status, prompt token counts, privacy anonymization events, scanner findings, and threat assessments categorized by OWASP type such as data exfiltration. Models run quantized at 8-bit with a 24K context window that fits a 16 GB VRAM desktop GPU. Scanners and reviewers run on CPU spot nodes; LLM adapters run on GPU nodes. The full stack was scheduled for open-source release on April 15th, with support for Apple and NVIDIA unified memory, Ollama, vLLM, and Llama.cpp inference.
Q&A
What is the deployment overhead and HTTP call latency when running the full sidecar stack? Deployment uses container scripts; GPU assignment is handled through policy configuration, and CPU-bound scanners are separated from GPU-bound models so each scales on its own node pool with KV caching to avoid replicating inference workloads. ▶ 31:41
Notable Quotes
no privacy code, no library imports, just just a label Joaquin Rodriguez · ▶ 08:48
the developer deployed the AI workload and then they got the automatic PII enforcement uh and then of course uh they change zero lines of code. Joaquin Rodriguez · ▶ 09:42
somebody poisoned the MD file and it actually deleted that. Krishnendu Dasgupta · ▶ 18:44
we are releasing this entire thing on April 15th on the shed link that we have here. Krishnendu Dasgupta · ▶ 33:12
Key Takeaways
- A single Kubernetes label triggers automatic PII sidecar injection with zero code changes.
- 23 pods across seven tiers enforce least-privilege boundaries between LLM and data components.
- Cisco Foundation SE, trained on CVEs and released February 25th, provides the security verdict.
- Agent skill MD files are a concrete attack surface; the pipeline scans and patches them before execution.
- The full stack runs on a 16 GB VRAM desktop GPU using 8-bit quantized models and a 24K context window.
About the Speaker(s)
Joaquin Rodriguez is a Senior Software Engineer in Microsoft’s Industry Solutions Engineering organization, where he partners with customers to solve complex technical challenges across cloud and edge environments. With over a decade of experience, he works on problems spanning cloud and edge deployments.
Krishnendu Dasgupta is the founder of AXONVERTEX AI and an engineer with 15+ years in applied machine learning. His work spans healthcare, generative AI, and decentralized AI, with current focus on clinical trials, graph ML, NLP, and privacy-preserving AI.