AI
Agents as Discoverable Web Applications
“an agent is still an application” — Roey Ben Chaim
An agent connects to an LLM and carries tools, but it still has endpoints, APIs, and iframe embeds — the same breadcrumbs any web application leaves behind. That makes it discoverable through standard web recon. What raises the stakes: agents sit inside the enterprise. Many carry embedded credentials, read live data sources, and can write back to them. A public agent is a gateway to your internal systems, not just a chatbot.
Attackers exploit this through predictable design patterns, out-of-the-box configurations that shrink the search space, known platform gaps, and the fact that security by obscurity has never held.
Enumerating Microsoft Copilot Studio Agents at Scale
Every Copilot Studio agent lives at a URL built from predictable parts. The environment slug is always default- plus the tenant ID. The solution prefix looks random — five to eight alphanumeric characters — but most organizations leave the default, shrinking the space to cr plus two or three characters. Fuzz that against common bot names and a valid prefix falls quickly. An undocumented Microsoft API that resolves a tenant ID from a domain (04:21) made tenant resolution trivial — until attackers abused it and the tool was pulled. The Power Platform API flags authentication state per agent. Thousands surfaced this way.
Discovering OpenAI Agent Builder Deployments and Custom GPTs
OpenAI Agent Builder leaves deployment to the developer. OpenAI recommends GitHub starter kits with well-known repo names. Most instances land on Vercel or Render, and some URLs include branch names — shrinking the address space to something fuzzable. Confirming a live agent takes three steps: call create_session, receive the client secret, check verification. A positive result means an agent is running, possibly with confidential data exposed.
Custom GPTs expose their capability manifest to anyone who looks. Default tools (Python, browser, DALL-E) plus custom connectors such as a Gmail reader are visible by design. No exploit required.
OSINT Techniques for Exposed MCP Servers and AI Middleware
Shodan makes MCP servers trivially findable. Search for the well-defined MCP session ID string and Shodan returns hundreds of thousands of exposed MCP servers — some authenticated, many not. An HTML hash dork found an open Copilot API with a Swagger UI exposing a purge-vector-DB endpoint with no authentication. MCP registries compound the problem: each JSON file lists backend URLs and available tools, handing an attacker a map of your services.
Backlinks are the less obvious vector. Embedding an agent in an iframe creates a backlink. Searching for copilotstudio.microsoft.com in a backlink tool returned 2,500 open, unauthenticated agents in a single query. The same technique worked for Amazon agents.
Impact, Guardrail Bypass, and the Power Pawn Toolkit
Guardrails are not a defense against reconnaissance. Once an attacker finds a public agent, they probe it for its knowledge source and tool manifest. Sometimes the agent answers on request. When it doesn’t, extra filler text confuses the guardrail activations enough to get through. Ben Chaim showed tool manifest extraction via guardrail bypass (12:15) live.
Power Pawn packages the recon chain into four modules: Copilot Studio Hunter (tenant ID resolution to full agent enumeration), Agent Builder Hunter (live deployments and exposed MCP tools), Custom GPT Hunter (capability manifests by search term), and LLM Hound (Shodan filters for open AI middleware).
Notable Quotes
an agent is still an application Roey Ben Chaim · ▶ 1:32
Security by obscurity doesn’t really work. Roey Ben Chaim · ▶ 3:17
you don’t know what you don’t know Roey Ben Chaim · ▶ 16:02
Key Takeaways
- Treat every deployed AI agent as a public web application and audit its URL structure for default configurations.
- Run Power Pawn against your domains before attackers enumerate your Copilot, Agent Builder, and MCP endpoints first.
- Add observability to your agents — guardrails alone do not stop tool manifest probing by determined attackers.