Armakuni Contact us
← All case studies

A defect cost £7,924 to fix once it got out.
Caught at the start, it cost £264.

BT was shipping changes that broke things its customers noticed, and taking too long to put them right. It started Building Products Together, a programme across the company with three aims: faster delivery, better engineering quality and healthier teams. We measured where delivery was actually stuck, then worked inside the squads to move defect-finding to the start of the work, with pairing, refactoring and masterclasses taught on their own code.

Before

Changes broke things customers could see.

Nobody could yet say where the time went.

After
£7,924 → £264cost per defect, found at the start of the work
82% → 96.5%code coverage on the classes we worked on

Changes broke things customers could see

Work moved slowly through the delivery process and much of it arrived late, below the on-time target. When a change did reach production, too many failed, which meant frequent service impact and long recovery times for BT's customers on broadband and TV.

None of these is a single team's problem. Slow flow, late delivery and failed changes feed one another, and BT had set up Building Products Together around three pillars (delivery performance, engineering quality and organisational health) to deal with them together.

Nobody could yet say where the time went

A programme that size needs a starting line. Without one, every squad has its own theory of what slows it down, and a year later nobody can show what changed. So the first job was to measure, before recommending anything.

Find defects where they're cheap

The figure that decided the approach was what a defect cost depending on when it was found. Late, it cost thousands of pounds to fix; at the start of the work, a few hundred. Most of the gain was therefore in catching problems earlier, more than in fixing them faster.

How we built it

4 decisions, in the order we made them. Each one says what we did and why, with what it did drawn beside it.

  1. A baseline before any advice
  2. The value stream, drawn and then shortened
  3. Quality built in at the start
  4. Taught on their own work

A baseline before any advice

Armakuni InsightsDelivery performanceDevelopment practiceTeam health one set of numbers
The engineering

We opened with Armakuni Insights, our baselining and diagnostic tool, across three areas: delivery performance, development practice and team health. It combines the delivery measures (how long work takes to flow, how often it ships, how often a change fails) with what the teams themselves report, so the biggest constraints and risks showed up early, in numbers the leadership and the squads could both accept.

Why: the same numbers, taken again later, show what improved and what didn't, so the programme reports results instead of activity.

The value stream, drawn and then shortened

The ideaEvery wait and hand-off, mappedProduction targets of 20% to 38%

“It's great having an expert, outside in perspective, who can help us quickly get up and running, hold us to account, and constructively challenge us to get learning, act on it and drive change. This will help us scale other tribes as frontrunners.”

Tribe Lead, Broadband & TV, BT
The engineering

We mapped how a piece of work travels from idea to production, with every wait and hand-off on it, and then worked on the bottlenecks and waste the map showed. Improvement targets of 20% to 38% were set for flow efficiency, velocity and change failure rate, each against the baseline, and progress against them is under way.

Why: each target is a number on the same dashboard as the baseline, so anyone in the programme can see whether it is being met.

Quality built in at the start

Quality built inDefects found at the startPairing and refactoringNext: mutation testingNext: defect escape rate
The engineering

We brought in advanced quality engineering practices that build quality into development instead of inspecting for it afterwards. Defect detection moved earlier in the lifecycle, and that took the cost per defect from £7,924 to £264. Focused pairing and refactoring raised code coverage on the classes we worked on from 82% to 96.5%.

We also left the next measures in place to keep quality honest: mutation testing (do the tests actually fail when the code is wrong?), defect escape rate and mean time between failures.

Why: coverage alone can be gamed, but mutation testing and escape rate show whether the tests catch real faults.

Taught on their own work

Taught on their own workMasterclassesHands-on sessionsOn the squads' own code
The engineering

We ran masterclasses and hands-on sessions on modern engineering, quality and delivery, with the explanation and the coding side by side. People learned by doing it on their own work, so the practice carried on after each session ended.

What we'd tell another CTO

1We would start where this one started, with a measurement everyone has seen before any recommendation is made. Insights gave the leadership and the squads one set of numbers to argue from, and the targets that followed were believed because both sides had agreed the starting point.
2Find out what a defect costs you at each stage. When the late one costs thirty times the early one, moving the testing earlier is a much better investment than another round of firefighting.
3The coaching worked because it happened on the squads' own code, with the explanation and the keyboard side by side, which is the part one of their engineers thanked us for afterwards.

What else changed

  • A measured baseline for delivery, engineering practice and team health, to track against over time.
  • Targets of 20% to 38% improvement in flow efficiency, velocity and change failure rate, with progress under way.
  • Squads that have done the practices on their own code, and the next quality measures to adopt: mutation testing, defect escape rate and mean time between failures.

In their words

“Thank you for your valuable coaching sessions on modern engineering practices. During sessions the clear explanation with practical coding approach side by side really helped with our learnings and will be very helpful in our future tasks.”

An engineer, Business App Squad, BT

What comes next

The flow, velocity and change-failure targets are being worked towards, and the next squads take the same approach, led by the ones that went first.

How we'd do it today

This project came before our current offers. Here's how we'd approach the same work now.

We'd still start with the baseline, but AK AI Factory would run it continuously from the pipeline's own records rather than as a survey, so the flow and change-failure numbers update every week. AI agents would write the first tests for untested classes and run mutation testing on every change, and an engineer in the squad would review each test before it merges. AK Evolve would find the legacy code with the lowest coverage and the most incidents behind it, so the refactoring goes where defects actually come from.

Modernize with AI AK AI Factory

More work like this

  1. ModernizationSecondsAn ethics and compliance software providerfor a benchmarking report, where it took minutesA big report could freeze the whole application. There was no budget for a rewrite.500+ companies on the platform2 products, 1 pod
  2. Modernization40%A U.S. K–12 staffing providerless manual payroll processing, as reportedA substitute teacher worked the day. Getting paid ran through three separate systems.30% lower cloud costs5× faster deployment cyclesOn AWS Fargate · RDS · Lambda · S3
  3. Modernization5 monthsA global software-delivery platformto cost neutrality, where 36 had been expected, as reportedThree partners proposed a migration. Two years later, nothing had moved.66% less than the proposed migration cost, about $300,0005 weeks ahead of scheduleOn AWS EC2 · SNS · CloudWatch · CloudTrail

All case studies →

Have a problem like this one?

A free workshop: a few hours with the architects who'd do the work, on your own systems. No cost.

Book a free workshop