AI
Why AI Coding Tools Demand New Guardrails
A developer ran Claude in YOLO mode. The AI agent deleted their entire home directory. The Mac was nuked. Balachandra Shanabhag used this incident to open his talk. Similar problems include code injection in Cursor and prompt injection in Amazon Q. Attackers also abuse AI tools inside breached machines. These tools are smart but lack judgment. They have full access to the user’s machine. Human error can trigger catastrophic results.
Threat Modeling the AI Coding Agent
Shanabhag walked through the components of an AI coding agent. The LLM serves as the brain. It reads internal docs, Jira, and the web. It installs npm or pip packages. It runs shell commands with the user’s privileges. The agent generates code that enters SCM as PRs. It can push to production. Attack vectors include prompt injection from external repos, dependency poisoning, and slop squatting. A malicious insider can use the LLM to build malware inside the enterprise.
Enterprise Risks from AI-Generated Code
AI-generated code can introduce reputational risk if vulnerable code reaches production. Privilege misuse allows agents to act without human authorization. Data exfiltration of IP or secrets can happen. Supply chain attacks are a huge surface, from the tool itself to its dependencies. Accountability is hard to track. Regulatory fines may follow if code is exposed to unapproved models. AI tools are nondeterministic, so traditional deterministic guardrails fail.
Building a Minimum Viable Security Baseline
Shanabhag recommended a minimum viable security baseline. Tools must integrate with enterprise SSO and MFA. Secrets should be scoped to tasks, not full machine access. Data residency and zero data retention must be enforced. Visibility requires hooks into the SIEM. Audit logging and human interrupt for critical operations are mandatory. Models must be pinable to specific versions. Untrusted input boundaries need continuous validation.
Enforcing Guardrails at Scale
The lifecycle approach starts with installation. Restrict what code the tool can read. Use allowlisted models and endpoints. Enforce privilege boundaries. Branch protection, human review, and security pipelines must guard the SCM. Route package downloads through internal proxies. Integrate with EDR. Monitor configuration drift. Scan prompts for secrets. Train developers on secure usage. For agent mode, sandbox and restrict network access.
Demo: Claude Manage Settings
Shanabhag demonstrated Claude’s manage settings. It disables YOLO mode, which the tool calls dangerous mode. It forces enterprise login, preventing personal token use. Permission prompts reduce prompt fatigue. A paranoia list blocks reading secrets or running rm. All tool actions are audited. The settings are enforceable from the top down via hierarchy. This balances security and usability.
Q&A
Should we worry about restrictions on forbidden LLM use cases in licenses? The downstream pipeline must verify license compatibility; the AI tool does not handle that. ▶ 38:41
How do you monitor prompts? Is it automatable or manual? Some tools provide hooks to intercept prompts and enforce rules; web-based tools may lack that capability. ▶ 39:29
Can you share real-world examples of badly written vibe code introducing malware? A startup’s login form displayed the MFA token in the front end; vibe-coded apps often have poor security posture. ▶ 40:10
How would you scale client-side rule files like the .claude file to hundreds of developers? Modern tools have hierarchy and deterministic hooks (e.g., /etc/claude) that take top precedence, but not all tools support that. ▶ 42:03
How do you track AI-generated code as third-party code when eventually all code is touched by AI? Currently AI code is not perfect; treat it as third party until tools reach an ideal state where they build secure code reliably. ▶ 44:10
Notable Quotes
“the AI agent did a mistake here and it went and deleted their entire home area and the whole Mac was nuked” Balachandra Shanabhag · ▶ 4:33
“it’s very very smart. You can ask it anything you ask it not to do something it’s going to find ways basically” Balachandra Shanabhag · ▶ 5:57
“these AI tools are nondeterministic by the nature so the traditionally like the security guard rails are kind of deterministic” Balachandra Shanabhag · ▶ 13:22
“the AI generated code should always be treated as a third party not as a first party” Balachandra Shanabhag · ▶ 26:25
Key Takeaways
- AI coding agents require specific guardrails because they are nondeterministic and have full system access.
- Treat all AI-generated code as third-party code until tools reach maturity.
- Enforce a minimum viable security baseline covering SSO, secret scoping, model pinning, and audit logging.
About the Speaker(s)
Balachandra Shanabhag works as a Staff Security Engineer at Cohesity. He has over 15 years of experience in cybersecurity. He joined Cohesity as a Founding Product Security Engineer and helped bootstrap AppSec and other security initiatives. Before Cohesity, he worked at Juniper Networks for more than 10 years, building network gear from firewalls to core routers.