From software engineering to AI Code Factory engineering.
We help your teams build, manage, and measure AI Code Factory with human in the loop principles.
More AI usage hasn’t meant more business value.
Across more than 500,000 developers, AI agents raised commits by 240% and releases by 30%. Three reasons.
240%
more commits
30%
more releases
Across more than 500,000 developers. NBER Working Paper 35275, 2026.
- 1
It still moves at the speed of people.
An AI agent on a laptop works only while its developer prompts it.
- 2
Nobody can manage its cost or its quality.
Everyone sets theirs up their own way, so nobody can say what a change cost or whether it held up.
- 3
There’s no one place to control it.
Tools and keys live on each laptop, so one compromised laptop gives every AI agent on it its developer’s access.
Our AI Code Factory is a loop with engineers in charge.
AI agents take each request from triage all the way to production, and your engineers make the three calls that matter: the spec, the code and the product.
People decide.
Someone agrees each spec and reviews the code and the product, and only a person can merge.
Always running.
Runs happen in the cloud, so nothing waits for a developer to type the next prompt.
Measured from day one.
Every run is traced, so each change has a time, a cost and a quality score.
Changed by pull request.
The factory’s own fixes are tested with evals, and a person approves each one.
A request comes in
From a person in Slack, Teams, Jira or GitHub, or from a failed build, an alert or a schedule. One coordinating AI agent takes it and briefs each stage.
Triage
It turns a bug into a failing test before anyone writes a fix, and it writes only to the issue.
Spec
Only where the fix is a decision. Its checks run: a test, a command or a recording. A person agrees the spec before anything is built.
Implement
A fresh sandbox each run: the network closed, a short-lived token that can only push a branch to one repository, no model keys, and hard limits on time and spend.
Review
A second AI agent on another model family reads the change. What needs judgment goes to your engineer, and a person reviews the code.
Verify
AI agents run the app in their own browser (Amazon Bedrock AgentCore Browser) and attach a recording or test results. The person who asked reviews the product.
Ship
Through your own CI/CD and your repository’s rules. No AI agent can approve or merge; a person always does.
Monitor
It watches production, rolls back past an error line agreed in advance, and sends what it finds back to the start.
No two companies’ loops are the same, so yours is built around how your teams already ship.
Built to fit how your company works, and kept in your own account.
- 1
Adaptable.
- Any model and any AI coding agent, chosen per step: say, Claude Code builds and a model from another family reviews.
- It works in Slack, Teams, Jira and GitHub, reaching each through its MCP server or API, and says in the thread what it will do, its limit and its cost so far.
- It pauses when a decision is yours and picks up from its saved state, even a day later, with a record of every step.
- Anyone on the team can open a run, steer it, or take it over in their own AI coding agent and hand it back.
- 2
Sovereign.
- Every run, transcript and fix stays in your own AWS account, on your compute, with model calls going through your inference gateway.
- Your code is checked out and built only inside a sandbox.
- Through Amazon Bedrock, a model’s maker doesn’t see your prompts.
- 3
Your alpha, protected.
- Your code, your engineers’ rules and skills, and the benchmark of your own work live in your repository and account.
- They stay yours if we leave.
- 4
Defined in code.
- Every AI agent’s instructions, skills, tools, permissions, model, runner and triggers are files in a repository, with the secrets it may use named there and their values kept in your secrets store.
- A change to the factory is a pull request a person merges, checked before it goes live, so a version can be tested or rolled back.
- AI agents can propose the next change too.
Your AWS accountstays yours if we leave
Example.
Then it answers what leadership asks, starting with your people’s time.
Because every run is traced, the dashboard answers four questions: how often a person stepped in for each change, how much is automated, how long a change takes to ship, and what each change costs. Beside them sit the numbers your engineers already trust, like how often a change is reverted or breaks production. Look at where a change’s time goes and most of it is spent waiting for a person; the dashboard shows you at which step.
Each rise is a change to the factory a person approved. How the evals change the factory
One change, two lanes
Asked at 9:00, the spec agreed by a person, and the change ready for review at 9:38. Then it waited 3 h 10 min for a person.
Example figures.
The KPIs, and how they’re built
Human interactions per change
Split in two: decisions (agreeing a spec, reviewing code and product), which stay yours, and rework (chasing, re-asking, fixing by hand), which the factory is tuned to cut.Autonomy
The share of changes merged with no human correction, by kind of work, so you know which kinds can be trusted with more.Cycle time, split
The AI agents' time, the wait for the first human review, and review to merge.Cost per change
Every model call, sandbox minute and platform cost attributed to the run that spent it.Quality
Revert rate, change failure rate and time to restore, the delivery numbers your teams already report, so a factory that gets faster by breaking things shows up straight away.A trace of every run
In OpenTelemetry, sliced by team, person, source, AI agent and model, so it lands in the tools your team already uses.How we build your factory, one part at a time.
Factory engineering is a new job: building the factory, then improving it against the evals and KPIs. Your product engineers keep building your product, now through the factory. Most companies have never had this job, so we do it with your team, in this order.
payments-serviceready for AI agents
make build1 command, from a clean checkoutmake testthe whole suite in 3 min 40 sAGENTS.md214 lines: where things live, how to test themlint: architecture12 rules, one of them: checkout may not import payments internals
checkout imports payments/internalstopped before anyone had to review itExample figures.
Make your code ready for AI agents.
An AI agent works best in a repository that explains itself.
- One command builds it from a clean checkout, and the tests are fast enough to run on every change.
- An AGENTS.md tells AI agents how the code is laid out and how to test it.
- Linters enforce your architecture rules, so a broken rule fails the build before anyone reviews it.
- We untangle a tangled core first: a factory on it only adds changes to the same queue.
Two kinds of work, two pathsone coordinator
Example.
Shape the loop: its stages, gates and fixed paths.
One coordinating AI agent decides which stages each request needs.
- A flag flip goes straight to building; an unclear feature gets a spec and a person’s yes first.
- Which kinds of work may skip the spec is written in the definition: a flag flip, a failed build, a dependency update, within a size limit. For everything else, the build starts only on an approved-spec label that only people can set.
- Branch protection needs an approving review from someone other than whoever pushed last, and a Cedar policy at the gateway stops any AI agent from approving or merging.
- It works to the lowest cost that gets the result, so it asks a question before it builds the wrong thing.
The factory’s definitionv12, reviewed in PR #418
factory/agents/build
- AI agent
- build
- instructions
- Fix the cause, not the symptom. Make the smallest correct change. Never merge.
- skills
- payments-servicetest-first
- tools
- GitHubJiratest runner
- model
- chosen by benchmark
- runner
- sandbox · 4 CPUs · 20 minutes
- triggers
- a labela failed build
Example
Give each AI agent its job, its tools and its limits.
Each AI agent is a folder in the factory’s definition.
- Its instructions, skills, MCP servers, allowed tools and secrets, model and runner.
- An AI agent that doesn’t act for a person gets no secrets by default.
- The coordinator briefs each one with only what its stage needs.
- A follow-up goes back to the AI agent that already has the context.
Two planes, in your accountkeys never enter a sandbox
Draw the boundaries: sandboxes, identity and secrets.
Two planes, both inside your AWS account.
- Control plane: coordination, identity, configuration, observability and model routing.
- Execution plane: a fresh sandbox for each run checks out and builds your code. It takes no inbound connections and reaches out only to an allowlist.
- Model keys stay at the inference gateway, never in a sandbox.
- Each change carries the identity of the person who asked, so it’s attributed to them, or the AI agent’s own for unattended work.
How it’s built
- We write a threat model before it runs.
- Everything an AI agent reads is untrusted, from an issue to a web page.
- Redacting secrets from what it writes is only a backstop.
- Built on Amazon Bedrock AgentCore Runtime, Identity and Gateway, with Cedar policies.
One factory per productanswers go back to the thread
- chat
- the tracker
- the code host
- an alert
- a schedule
- an AI coding agent
customer feedback
Pull request ready for your review.
Found it: the bug from the feedback channel. A fix is in review.
The pricing page copy is updated, with a preview.
Examples.
Connect it to where work starts, and where the answers go.
Work comes in from where people already are, and the answer goes back there.
- Chat, the tracker, the code host, alerts, schedules, or a developer’s own AI coding agent handing a task over and back.
- Support can pass it a bug from the customer feedback channel.
- Each product can have its own factory, marketing included.
The baselinerecorded before the first request
| How you ship today | Now | |
|---|---|---|
| Request to production | 6 days | not yet |
| Waiting for a first review | 19 h | not yet |
| AI cost per change | $24 | not yet |
| Change failure rate | 4.2% | not yet |
first run: tomorrow Now fills in from the evals and the KPIs.
Example figures.
Measure it from the first run.
The KPIs above are switched on before the first request; the evals start with the first run.
- First they record a baseline of how you ship today: DORA’s delivery numbers, review waits, and today’s AI spend divided by the changes it shipped.
- Every later number has something to beat.
- Every night, scorers grade a sample of runs against rubrics your engineers write, each checked against their own grades before its scores are trusted.
- When a mistake keeps turning up, an AI agent proposes a fix as a pull request against the factory’s definition, with its evidence. An instruction, a tool or a model changes only if the scores hold or rise, and only a person can merge it.
- A benchmark of your own past tasks, kept in your account, picks the model for each kind of work.
The factory’s history12 versions, every one a pull request
- Dependency updatesfrom v3
- Failed buildsfrom v6
- Bug fixesfrom v11
- Featuresnext, when the numbers say
Examples.
Roll it out, then hand it over.
One kind of work first, then more as the numbers earn it.
- Start with one low-risk kind of work in one repository, such as dependency updates or failed builds.
- Each kind of work earns autonomy on its own numbers, never by assumption: fewer human touches before review, never a merge without a person. Concurrency and spend limits grow with it.
- We re-run the benchmark when a new model arrives. It looks backward, so A/B tests on live work come next.
- A busy kind of work can move to a model of your own at a fraction of the price, as AI Inference Engineering does.
- Then a handover drill your team passes without us: change the definition, roll back a bad change, rotate a secret, stop the line.
Our engineers build it with yours, then hand it over.
We’re Armakuni, an AWS Premier Tier Partner. After a free workshop, our forward deployed engineers work inside your company to build the factory with your team, start it on one kind of work, measure it and add the next.
Before we step back, your team runs it without us. Then you run it, or we do.
What you keep
- The factory’s definition
- Every AI agent’s instructions and skills
- The dashboards
- The benchmark of your own work
- The record of every run
Questions we’re asked.
Got another? Bring it to the workshop.
We already pay for AI coding agents. Why do we need a factory?
AI coding agents on laptops do make each developer faster, but the work stays ad hoc and stops the moment the developer does. A factory runs the whole loop in the cloud, with your people at the points where they matter. Your developers keep the AI coding agents they like, and the factory can use those same ones.
Can it ship code on its own?
By default, a person reviews the spec, the code and the product, and only a person can merge, because your repository’s branch protection says so. As a kind of work proves itself on the numbers, you can choose to automate more of it.
What stops an AI agent from doing damage in a sandbox?
It has no network beyond the job, a token for one repository that expires, no model keys, and a hard limit on time and spend. What it reads is treated as untrusted, and nothing it writes reaches production without a person’s review.
Does our code or data leave our account?
The sandboxes, the record of every run and the factory’s definition stay in your AWS account. Model calls go to the models you choose, through Amazon Bedrock or your own inference gateway.
Which models and AI coding agents does it use?
Any, such as Claude Code, Codex or Kiro. Your benchmark shows which does each kind of task best for the price, and your team can swap any of them.
Should we buy a factory product instead?
Sometimes, yes, and if one fits your stack we’ll say so and set it up. What we add is your code made ready for AI agents, the factory shaped to how you ship, and all of it in your own account.
See your AI Code Factory before you build it.
Every run in a sandbox, the data kept in your own account, and every change to the factory measured before anyone trusts it. Bring your engineers to a free workshop: we’ll look at how your company ships software today and sketch what your factory would look like on your own code.