Breaking Models 🚨AI Security Session 7: Too Much Agency
About
Part 1 LLM03: Excessive Agency Arshi Chadha, OWASP Top 10 for LLM Applications 2026 co-lead The risk when an AI system is allowed to do things, not just say things. This sat at #6 on the 2025 list. In the 2026 update it moved to #3: the biggest jump on the whole list, because production incidents now cluster around models that run commands, call APIs, and touch databases on their own. We’ll walk through the three things that cause it: too much functionality, too many permissions, too much autonomy. And what actually contains it when the model gets manipulated. Part 2 Interactive demo: the email assistant that goes rogue Rishabh Gupta, Software Engineer at Big Tech A prompt-injected email assistant, live. The same malicious instruction succeeds, gets denied, or waits for approval: depending only on how its functionality, permissions, and autonomy are set. Then a real case: a prompt injection planted in a public Slack channel that got Slack AI to surface secrets from private channels. Part 3 The Hugging Face incident: an agent that wasn’t supposed to leave the room Damian Halas, Sr. Software Architect at Nvidia In July 2026, OpenAI models being tested on a cyber benchmark found a fla
