No auditor is watching. The system still has to behave the same on Monday.

A trusted tester ran the method against a production trading platform — Java 21, Spring Boot, PostgreSQL, around 374 production classes, a suite of some nineteen hundred tests, and a written specification of forty-nine requirements that predated us. Every requirement was marked done or not-done by hand. The question was whether done was true.

The status report was wrong, and nobody could have known

Forty-nine requirements, each marked done or not-done by a person who had read the code. Reviewed twice. The test suite passed. Measurement found that 38 of those requirements had nothing holding them — not badly tested, untested, while the report said otherwise.

The cause was mundane and is everywhere. The project's main test exercised the component on its own, rather than the system that uses it. It passed perfectly and proved nothing about the requirement, because the requirement was about the system.

A test that calls the component proves the component. The requirement is about the system that calls it.the rule, in one line

It shows where requirements are missing, and where they need refining

Some requirements could not be checked because they had never been written as anything checkable — a statement about the system that nobody performs, with no position in any sequence. Those are the ones the box at the top of this page finds, and on a specification written decades ago they are the majority.

Others were written fine and simply had no one asserting them: a reconciliation between two subsystems was exercised by three requirements and asserted by none. A label the reporting team relied on had a test nobody had connected to a requirement, so it counted for nothing.

And a third group needed refining rather than replacing. A single requirement covering every case of a decision table reads as complete and cannot be checked case by case — so it is reported as the one requirement that should become several, with the measurement naming which cases are unsupported. That is the difference between being told a specification is weak and being told which sentence to rewrite.

What makes it credible, in the only terms that matter

Done was measured instead of claimed. That is the entire proposition. Every programme you deliver reports completion as a judgement by someone who looked. This replaces the judgement with a measurement, and the measurement disagreed with experienced people on three quarters of a specification.

And it pointed at the failure before anyone told it where to look. Thirteen limits in one acceptance path had nothing checking them — the boundaries where a value sits exactly at a threshold. That is the same code that had produced a real financial loss earlier that month. The measurement found it from the specification alone, with no knowledge of the incident. For a programme where the acceptance question is "does the new system behave like the old one", that is the whole argument.

The full case study, with the day-by-day timeline, publishes with the repositories. The operator who ran it is available as a reference via contact@plumbspec.org. One limit stated plainly: on a specification with a single enumerating requirement covering many cases, the measurement floors above zero and we report that residue rather than rounding it away.

Every project of any size breaks the work into pieces small enough for one team to build. That is a keyhole per team. Each group sees its slice in detail and the system around it only as rumour, a diagram from six months ago, and whatever came up in the last integration meeting.

AI did not invent that problem. It made the pieces smaller and far more numerous, so a condition engineering has lived with for decades became the binding constraint. The underlying failure is the same either way: work that is locally correct and globally unverified, discovered at integration, when the cost of changing it is highest.

This is the part people expect a process to solve and it never does. Integration meetings, a shared wiki, an architecture review board — all of them move information between silos at the speed somebody remembers to move it. None of them test one team's work against the whole project's requirements.

That is what goldspec does. Every team's piece is checked against the same declared specification rather than against its own understanding of it, so the work comes back cooperative and targeted instead of individually defensible. The silo stays — teams have to specialise — and it stops being opaque.

What that costs

You do not buy a service and you do not buy a body. Your own named resource sits in the seat and gets the tool. We are the AI-enabled tag on a line that is already in your plan.

Three dollars is roughly 7% of a fully-loaded offshore seat. At two thousand hours it is six thousand dollars a year against a line you are already carrying.

What it costs your engineers, which is the number they will ask about

One annotation on a test they were already writing. That is the floor and it is the whole of it:

The highlighted line says which requirement the test is about. Nothing else changes: the suite runs as it did, and the status is derived from the run rather than typed by anyone. A proof that later breaks un-proves its requirement, so a board from last quarter cannot be banked.

And the specification is not a precondition. There is no spec command, no template, and a project with nothing but citations in its test suite gets a complete and correct board. How a programme captures requirements stays the programme's business. The method is recommended, not required, and adopting it later costs nothing done today, because citations are the only thing written by hand and the method neither replaces nor rewrites them.

That matters for an estimate. The thing people price as a methodology rollout has a floor of one annotation per test, and everything above the floor is optional and incremental.

Supported today: Python and Java, by the same command and the same configuration file. .NET is on the roadmap and is not built — if a programme's target runtime is .NET, the measurement does not reach it yet and we would rather you knew that here than in week three.

What you actually buy, and what we can actually count

A seat. One credential per named resource, issued by the hosted service, and the service will not authenticate more of them than you hold. Three dollars a contracted hour, and it scales with the project: one week is the floor, at $120. A full-time year comes to $6,000.

The price scales down because the work does. Most of what this is bought for is a task order rather than a programme, and under an IDIQ there is no annual anything for a seat to attach to. A vendor sizing a two-week task will not put an AI line on a resource for a year to find out whether it helps, so the smallest thing we sell is the smallest engagement that exists.

We say this plainly because the alternative would be dishonest. We cannot see your contracted hours and never will — you would self-report them, we could not check, and a price that depends on a number only the buyer can see is a hope rather than a term. What we can see exactly is how many credentials authenticate against the hosted step, because every one of them comes through us. So that is what the agreement is denominated in.

The tools themselves are Apache-2.0 and need no credential at all — run them on a laptop, in CI, or on-premise, forever, with no call out to us and no seat. What a seat buys is the hosted AI-assisted step.

And the market already prices the problem, which is the useful corroboration: offshore analyst work is the most coordination-dependent staffing there is, and the standing advice is that each team needs a dedicated full-time coordinator or the gaps compound and quality does not hold. That coordinator is a person employed to carry context between keyholes. Their work is exactly what a checkable specification does mechanically, and they cost rather more than three dollars an hour.

What you are actually comparing it against

Not another tool. The alternative is the way the work is done today: more analyst hours, more integration meetings, and a test suite whose green light nobody can convert into a statement about requirements. That approach has a known cost and a known failure mode, and the second is worse — a behaviour that moved and nobody caught surfaces as an incident after cutover, when the programme is closed and the team has rolled off.

There is no productivity multiplier quoted here and we will not invent one. The cost structure above is public; a throughput figure needs measurement that does not exist yet, and the first programme to run it is where that number comes from.

What we do for your requirements gathering

We standardise what you already have onto one structure, and that structure is what makes the rest possible. Put a programme's requirements through it and it shows you:

• Isolated and overlapping processing — which steps nothing feeds into or out of, and which ones are doing the same work in two places under different names.
• Decisions with no outcome — a branch in the process where one path leads somewhere and the other was never written down. These are where a replacement system quietly invents behaviour.
• Missing inputs, guaranteed — every step is held against the business information it needs, so a step that cannot actually be performed with what it is given is reported rather than discovered during acceptance.

None of that requires a line of code to exist yet. It is the work your offshore analysts are doing by reading and asking, done mechanically and repeatably against the documents the programme already holds.

What we will not do

We do not write your requirements. A model that invents the specification is the thing that caused the problem — it fills the vacuum with its own interpretation and generates code against that. The customer owns what the system must do; the machine structures it, challenges it, and measures it. If a programme has no written specification at all, we are not the vendor who will manufacture one.

Usually some of it is. The honest answer has two halves, and the second one is the part most vendors get backwards.

Current behaviour can be read out of the COBOL. Requirements cannot. Extracting what the code does today is tractable and worth doing — the branches, the thresholds, the orders of operation, the cases nobody remembers. That produces a description of behaviour, and a description of behaviour is not a specification. It is a candidate list.

The difference matters commercially, not philosophically. Forty years of a live system contains its defects, its workarounds, and the three places where somebody patched production at two in the morning. Infer your requirements from that code and you have just specified the bugs, formally, with a traceability matrix proving the new system reproduces them. The programme then passes acceptance and carries every original fault into a platform meant to last another thirty years.

We know the shape of this because we hit it. In the system we measured, thirteen unchecked limits sat in the code that had already caused a real financial loss. Had anyone read requirements out of that code, the loss-producing behaviour would have become a requirement — and the measurement would then have confirmed, correctly, that the new system implemented it.

So the sequence is: extract current behaviour mechanically, present it as a list of candidate requirements with the code each one came from, and a person who knows the business rules on each one — keep, change, or drop. That person is your subject-matter expert or the agency's, not us and not a model. What we remove is the reading, the transcription and the cross-checking; what stays with a human is the judgement about which observed behaviour was ever intended.

Where documentation is merely stale rather than absent, the same step is cheaper: the extracted behaviour is held against the document, and what the two disagree about is the shortlist. That disagreement is usually the most valuable artefact a modernisation programme can have, and it is normally discovered one incident at a time after go-live.

Your engineers will want to know what is genuinely different, because tools claiming to measure test quality are not new. Two things are, and the rest of the field has one or neither.

A check only counts if it reaches the requirement the way the real system does

Most tooling scores whether code was executed. This asks whether the check entered through a route the live system actually uses — and refuses to credit the requirement otherwise. That single rule is what turned forty-nine hand-marked requirements into thirty-eight unsupported ones in one run.

When something is unsupported, the report names which requirement

A project-wide percentage tells a team to try harder. Naming the requirement and the line of code tells one person what to go and write. That is the difference between a metric and a work item, and it is why six days of work moved the number instead of a quarter of debate.

Both tools are Apache-2.0 and run on a laptop, in your CI, or on-premise with no call out to us — your reviewer can confirm every claim on this page without talking to anyone. The hosted part is the AI-assisted step, and that is the only thing that bills.

The fastest way to know whether any of this applies is to paste requirements from a document you already hold into the check and see how many could never be tested as written. It costs nothing, it tells us nothing, and it takes about a minute.

If the answer is uncomfortable, that is the conversation worth having.

An evaluation brief for your technical reviewers, the full case study, and the reference contact are available from contact@plumbspec.org.