AWS Emulator and the engineer's job when agents write the code

AWS Emulator and the engineer's job when agents write the code November 11, 2026 • 1671 words In 2004, barely a month into my new job at Amazon, I got the opportunity to pick an intern to join my team. I had an advantage over my colleagues: I had already heard about a brilliant young man working toward his PhD at the Vrije Universiteit in Amsterdam (where I happened to have received mine as well :-)). I immediately picked Swaminathan Sivasubramanian, whom you all now know as Swami. It was probably one of the best decisions I have made in my 22 years as Amazon’s CTO. After a disastrous Christmas in 2004, when commercial relational databases could not scale to Amazon’s needs, I asked Swami to do a deep dive into how relational databases were actually used at Amazon. It turned out that 70% of database operations were not relational at all. They were key-value. That insight led to Dynamo , which Swami helped architect. When his internship was over, I hired him full-time right away, even before he had finished his PhD . Bringing in another opinionated academic led to many fantastic discussions with Amazon’s practical engineers, who had become scale experts by actually building at scale. Eventual vs. strong consistency, CAP vs. Paxos, and more. Over time, Swami became one of the principal architects of AWS’s data strategy. And as we have seen in recent years, he has also been one of the intellectual drivers behind AI innovation at Amazon. I am honoured that he wanted to write a guest post about the fundamental distributed systems work that sits behind many of AWS’s services, and about how the engineer’s work changes when agents write the code. AWS Emulator, launched today, gives agents the kind of test environment that work relied on. Swami’s story is another great example of invisible engineering . Now, go build. -W AWS Emulator and the engineer's job when agents write the code Today we release AWS Emulator, which gives coding agents a local copy of AWS to test against, and it connects to a project from early in my career. In many ways, this emulator launch reminds me of the effort we put into reliable software engineering to improve the quality of the distributed systems we build. To that end, we wanted to share a story: About twenty years ago, Werner asked me to work with one of Amazon’s earliest engineers, Al Vermeulen, on a distributed lock service we called “Alf”. Alf was built on Paxos, a protocol based on Leslie Lamport’s work that lets a group of servers agree on one value, and a replication engine in it kept copies of the data consistent across servers. Alf, to date, is the service that powers many of our complex distributed systems like S3, DynamoDB and many more (although it has evolved quite a bit with various changes on top – will save it for another blog!). Distributed systems are hard to test Paxos is the gold standard among distributed systems consensus protocols and was well understood in academia. However, building it at the scale Amazon and AWS were growing into was not easy. We had to deal with not just replication across servers and data centers but also tens of thousands of clients asking for locks at the same time, renewing their leases during network partitions all the while with the core requirement: Always be correct! At that time, the best ideas for testing distributed systems were large-scale system testing and chaos-based testing. Both worked, but they were expensive, and they could not reproduce many kinds of conditions on demand. That mattered for Alf, which was meant to be the truth store, giving every client a strongly consistent view of the truth, and which had to keep working through server failures and packet loss as well. Several industry papers had written that Paxos-based locking services were hard to get right, and the difficulty of testing them was part of the reason. We addressed that by simulating Alf’s environment with as much fidelity as we could manage. We ran the servers inside a JVM and simulated the network and the rest of the infrastructure they depended on, so the whole system lived in one program that we controlled. We could drop packets and inject other faults whenever we chose, and we built tools that poked at the simulation for bugs and scalability problems. Concurrency bugs appear only when events happen in a particular order, and in the simulation we could set up many of those orderings on purpose and inspect how Alf handled each one. We also used formal reasoning, through theoretical techniques such as TLA+, and it found many issues as well. By testing this way we ironed out most of Alf’s concurrency bugs before it ran on real servers. Testing in a real environment then turned up different problems, because the real world exposes things a simulation cannot. Outgrowing hand-built test environments Simulation and formal reasoning both improved the software quality of the distributed systems we applied them to. The simulation worked because a small team could build a copy of one system’s environment by hand, and at the time AWS had only a small handful of services. But today AWS has over 200 services, and an application that calls AWS needs a test environment for each one it uses. Building all of those by hand would take years of engineering, and agents now write code far faster than any team can check it in production. So, we wanted to know whether agents could build and test those environments themselves. The idea came from Matt Goldberg, a Senior Principal Engineer on the AWS Developer Tools team. His hypothesis was that, given a written description of how a service should behave and the real service to check against, an agent could build a working version of that service on its own. For AWS, the written description is the set of public API models, written in Smithy, AWS’s format for describing a service’s operations and their inputs, outputs, and errors. The real service to check against is the AWS cloud itself, the same system customers use. The agents work from fidelity tests, which are end-to-end customer scenarios devised from the public API models. A test might check, for example, that in Amazon Simple Queue Service a received message stays hidden from other readers until its visibility timeout runs out. The agents correct each test until it passes against real AWS, so the test records how AWS behaves. The agents then build the emulator until it passes that same test, and a test that still fails points at a gap in the emulator. Matt began with SQS alone, and when he gave Kiro one prompt describing this approach, it returned a passable first version of an SQS emulator in about 15 minutes. Matt Goldberg’s launch post tells how that prototype grew into AWS Emulator. All of AWS Emulator’s code was written and tested by agents, and it is built from more than 400 API models covering more than 200 AWS services. It runs free and offline on your own machine. Teams of agents test it against real AWS and fix the gaps they find. Every change goes through a merge gate, a deep bench of automated checks that every change must pass before it enters the codebase, including several security agents doing adversarial reviews. Infrastructure in the coding loop A coding agent works in a loop: it writes code, runs it, and reads the result before deciding what to change next. In distributed systems that loop runs inside infrastructure, so the services it calls have to behave like the real ones. When they do, an agent can check its own work as it writes and produce correct code on its first attempt far more often, which improves software quality. AWS Emulator puts that infrastructure inside the loop on your own machine. The engineer’s job when agents write the code When agents write the code and check their own work, the engineer’s job is to give them room to work on their own and to build the specs, tests, and merge gates that decide when that work is correct. It was a lesson I learne