Skip to main content

Your browser is older than this site supports, so this page is showing without its design. To see it properly, update to Safari 15.4 or later, or to a current version of Chrome, Edge or Firefox.

Back to Blog
·11 min read·By Emanuele Pugliese

Four Stages, Two Approvals: How We Build Software With AI Agents

Listen to this article

AI made building cheap. It did not make deciding cheap. So we built a process where a person decides twice, reads in their own words both times, and agents do everything else.

We run PuglieseWeb, around sixty repositories across several products, on AI agents. Agents write most of our pull requests. One person decides what ships. The teams that run on SystemDox work the same way, with their own owners, their own rules and the same lane underneath.

The process has been through several revisions since we started building with agents, and each one removed a hand-off. What follows is the current shape: four stages, two approvals, and a set of rails that do not bend.

Why the review moved

A pull request written by an agent is cheap to produce and expensive to read. The expensive part is not the syntax. It is the decisions the agent made silently while writing: which field is optional, what a reply of the wrong word does, whether the new event goes on the bus now or later. A reviewer who meets those decisions in a diff is reviewing them at the worst moment, after they are built, under time pressure, in the agent’s vocabulary rather than the business’s.

The decisions were always the thing worth reviewing. The code was just where they used to be visible. Make them visible before the build, in the owner’s words, and the code review becomes what it should be for agent-written code: a check that the build kept to the design, which a machine does better than a person.

The shape

Stage Who What comes out Approval
Shape the owner and one agent the brief: the decision, the contract, the scenarios, the plan the owner approves the brief
Build agents, one story each stories merged into the integration branch, scenarios turned from proposed to built none: the lane checks that the design was kept
Release the owner what reaches customers the owner approves the release
Learn agents and the owner the record, the decision registered, new rules and checks none

Two approvals. Both are a person reading documents written in their own words, on one page. Nothing in between asks a human to read code.

Stage 1: Shape

A change starts as the owner describes it: a paragraph on SystemDox’s plan page, a voice note, a line in a chat. From that, one agent assembles the brief. It is one page with four parts, and it is written before any code that will ship.

The decision. What changes and why, which repositories it touches, what was considered and rejected, and how we will know it worked. It is the architecture decision record, kept in the company’s decision repository, and it is written for the reviewer: the decision and its consequences first, open questions listed with a proposed answer each.

The contract. What the system will accept, return, publish and consume, written by hand before any handler exists: openapi.yaml for the HTTP surface, every request and response schema, every error code with its status; asyncapi.yaml for the events the change publishes or consumes, only this change’s channels and messages; contract.md for what neither format covers, such as MCP tools, commands and console components. It lives in the change’s own folder in the repository that serves it. The served contract is not touched until the build.

The behaviour. Gherkin scenarios, each in a few lines a business owner, a visitor or a client’s developer could have written. A step says what a person does and sees. It never names a table, a function or a status code; status codes belong to the contract. Every scenario has an id and the tag @proposed. These scenarios are not a description of the tests. They are the tests, which is what makes the brief more than a document.

The plan. The stories, one pull request each in one repository, each naming the scenarios it will turn to @built, its dependencies and the release order. The plan page turns the brief into the board when the owner approves; nothing reaches the board before.

Spikes are free, so we use them. While shaping, the agent may build a throwaway spike against the draft contract to find out whether it holds: a handler that answers the proposed shapes, a consumer that reads the proposed event. The spike never merges. It makes the contract better before a person reads it, which is cheaper than finding the flaw after approval. This is the part of the process that only became possible when building got cheap, and it is why the brief is usually right first time.

Sized to the change. Not every change needs every part. A small change, a new field or a new rule inside an existing capability, carries the contract and the scenarios and no decision record. An ordinary change carries all four parts and is approved in one sitting. A foundational change, one that moves a boundary or changes how every product works, has its decision approved on its own first, because the decision is the contested part and the contract would be wasted work until it is settled. The agent proposes the size; the owner can change it in a word.

Approval one: the brief

The owner opens the change’s page in SystemDox and reads the brief rendered: the decision as a document, the contract as an API reference rather than YAML, the scenarios as a table with their ids and states, the plan as stories. They correct what is wrong in their own words. One scenario usually changes, because the owner knows the customer and the agent does not.

Then they approve, by merging the pull requests that carry the brief. The merge is the approval, so the record of who decided what is the repository’s history and nothing else has to be kept in sync with it.

A brief is not frozen after that. If the build discovers that the contract or a scenario is wrong, the agent does not patch around it. It raises a brief amendment, a short diff on the page, and the owner approves the diff the same way. The build waits for the answer. A decision discovered during a build becomes a decision, not a workaround.

Stage 2: Build

Agents build one story per session, each in its own worktree branched off the integration branch, in parallel across repositories with one writer per repository. The brief for every session is the same, served from SystemDox: the story, the contract, the scenarios, the project rules and checks for the files the story touches.

The rails:

  • The agent may not edit the brief. Flipping a scenario’s tag from @proposed to @built is the only change it may make to the contract or the scenarios. Everything else is an amendment for the owner.
  • A proposed scenario is a strict expected failure. It is collected and must fail until the story builds it. A skipped test is a promise nobody is held to; a strict expected failure is a promise the build must keep, and it turns red if the story forgets to flip the tag.
  • A new test must fail against the old code before it counts. A test written alongside the change it tests passes for the wrong reason surprisingly often.
  • The contract test runs the real handler. Called the way the gateway calls it, over a local database, against the approved schema. Only the edges are faked: the telephony provider, the mail service, the model.
  • Feature branches are not built. The merge is the build. A story merges into the integration branch as soon as its touched tests pass locally. The branch then runs the checks, the full suites, the deploy to the preprod environment, the end-to-end journeys, and a single required status that the release branch will not accept without.
  • One pull request per repository per story, titled with the story id, linking the brief, the scenarios flipped and what was verified.

The change page shows the build as it happens: scenarios turning from proposed to built, stories merged, the preprod status. The owner can look at any time. They do not have to.

Stage 3: Release, approval two

When the stories are merged and every scenario is built, the owner reads the change’s release view: the scenarios now passing, the contract now served, and what a customer will see that they did not see before. Then they approve the release pull request. Production deploys on that merge and on nothing else.

The approval is an approving review on the exact commit. A new push dismisses it. The release of a repository that deploys or publishes anything cannot be merged without it, and it cannot be merged while the integration branch is red. Every merge, into the integration branch or into production, is recorded the same day in a dated entry that names the pull requests and reconciles every decision document the change touched. A hook refuses to let an agent end its session until that entry exists.

Stage 4: Learn

The decision is registered in SystemDox when its record merges, so the next brief is drafted against it. The documents the change graduated into are watched for staleness. And the standards absorb what the change taught: a defect, a near miss or a reviewer’s correction becomes a project rule, a check, or a line in the skill that drafts the next brief, in the same week. The retrospective is not a meeting. It is the standing instruction that every lesson is written where the next agent reads it.

Where SystemDox sits

Most of this process is convention and continuous integration. The part that needs a system is the part every agent and every reviewer must read from at the same moment. That is what SystemDox is for, for us and for the teams that run on it:

  • The change page: the brief rendered, each stage’s state beside it, the release view, the list of what is waiting for the owner.
  • Project rules and checks, served to an agent when it asks for context on the files it is about to edit, so a standard changes once and every session inherits it.
  • The plan page, from a description to a brief to a board.
  • Skills: the process itself is one. The stages, the file layouts, the checklists and the rails are published as a skill that any computer on the team installs with one command, and that a new version replaces everywhere.
  • The decisions, registered as they merge, and the synced documents of every repository, so what the owner approves and what the agent reads are the same files.

What this does not do

A quick approval of a wrong decision is still a wrong decision. The process makes decisions visible, cheap to change and impossible to bypass. It does not make them good.

It needs an owner who reads. Two approvals of a few pages each is a small ask, but it is an ask, and a team where nobody will read the brief before the build should fix that before it fixes its tooling.

And it asks each repository for its runners and lints the first time: the feature-file runner, the contract lint, the contract test harness. After that, a change costs less than it did before, because the stories are small and their acceptance tests exist before the first line.

Monday

For a team that wants the shape without the year of revisions:

  1. Write the contract before the handler. If the framework can generate a spec, keep the generation as a check against the one you approved, not as the approval.
  2. Make proposed scenarios fail strictly. Never skip.
  3. Let agents spike, never merge a spike. The cheapest review of a contract is an agent trying to build against it.
  4. Two approvals, in the owner’s words. The brief and the release. Everything else is the lane’s job.
  5. Put the rules where the agent reads. If your standards live somewhere the agent never opens, the agent is writing to its own.

Interest declared: SystemDox is ours, and this is the process we run it, and the rest of PuglieseWeb, on. It starts free on one repository. If you want the four stages set up in your own estate, with the runners, the lints and the lane, that is what we do.

Related Articles

Comments

Comments are powered by GitHub Discussions. Sign in with GitHub to leave a comment.

Basket is empty