Transcript

Sriram Madapusi Vasudevan: Trustworthy productivity, what could the worst be? What happens when your autonomous AI agent goes rogue? Let’s rewind back to July 2025. Replit had a massive production incident. It all started with a relatively innocuous prompt, clean the database before we rerun. A simple instruction during a code freeze. What could go wrong? Turns out it was everything. Cleanup was the key word that caused a catastrophe. Jason was a SaaS founder who was coding with the AI agent and he had enforced a code freeze at that point in time. He had been vibe coding for nine days straight. He told the agent, clean the database. There was a key misinterpretation at that point. The agent equated clean with dropping the database. The execution resulted in destructive SQL with production credentials. It nuked live data. Remember, this was nine days’ worth of work. The aftermath was all over the Twitterverse.

The agent apologized, saying this was a catastrophic failure on my part. I destroyed all production data. It even said that it could not do any recovery. The CEO went into high gear. He said that we’re rolling out automatic dev and prod separation in a planning only mode. Why is this everyone’s problem? I’m pretty sure everyone is vibe coding to some extent. The outage happened during an enforced code freeze. Guardrails weren’t present. If it happened to Replit, it can easily happen to you.

Your Guide

I’m Sriram Madapusi Vasudevan. I’m a Senior Software Engineer at AWS. I build agentic AI systems at AWS, pushing autonomy into production. Today, I’m going to be sharing the patterns the industry is converging on for securing these AI agents. What you’ll walk away with is an understanding of the ReAct loop, a clear framework to protect your autonomous agents, some field stories across context, reasoning, and tools, mitigation, enterprise-grade strategies rooted in proven principles, and where do you even start? How can you add value from day one?

ReAct Loop: The Agentic Loop We Must Defend

How many of the folks here have heard something called the ReAct loop? The ReAct loop is a reason and act loop that the AI agent can adopt that goes from reason to action followed by observation that goes in a cycle. The agent can now effectively problem solve by breaking down a complex problem into manageable subtasks and gathering more information before it can figure out its goal using tools along the way. This is the agentic loop we must defend. There are three critical stages where vulnerabilities emerge. Here’s our agentic loop. We have context management, reasoning and planning, and tool action execution. These three in a loop is what comprises your agentic loop. At some point when an exit criteria has been met, we short circuit out and go back to the user. Context. Let’s start from the top. Context is what we feed the agent. I’m pretty sure folks here have also heard of the term context engineering.

It’s at the forefront of your agentic loop. It’s story time. Let’s talk about a context corruption story that happened in the wild. IBM documented a Fortune 500 financial firm’s agent memory poisoning incident. Unverified market data entered through the RAG, embedding subtle adversarial cues. So what? The agent cached this to long-term memory, skewing decisions and byp…