The control regime

Draft note, Sirsh, delete before publishing. Everything up to "The gap" is description: it is in the repository and I have cited the file. "The gap" onward is a design we have not built, written in the present tense on your instruction. The falsification test at the end of that section is the part I would keep if you cut everything else, because it is the only sentence that can stop us building a second claims audit.

333 commits landed in this repository in six days. 24,000 lines of SQL under specs/, 19,000 lines of tests under dev/tests/, five subsystems, one Postgres extension that other people will eventually install. Almost none of it was typed by a person.

In one review pass over that week, all 31 suites were green and probing found 16 defects.

I have argued elsewhere that trust in an abstraction has never come from the abstraction being perfect. It came from the control regime built around it: differential testing, shared suites, decades of exposure, eventually proof. Generation got cheap and the control regime has not caught up. This is a review of the parts of it somebody else has already built, and a specification of the parts we run ourselves.

What the process has to survive

Percolate is a Postgres extension and a small set of services around it. Five subsystems: retrieval, a workflow engine, RBAC, content ingestion, an agent runtime. Three of those carry nearly all the testing weight, and the reason is the shape of their failures rather than their importance.

A wrong answer from the query layer is data, not an error. Nine retrieval modes over three index families, and when one of them is wrong nothing crashes. The caller gets a different result than the spec promised, and looks at it, and believes it.

Every interesting failure in the workflow engine is a partial one. Retries, sagas, compensation, cancellation, crash recovery. A run reports completed over a set of tasks where one did nothing, and the characteristic tell is that every individual task looks fine.

RBAC is the only surface where a bug is a breach, and its characteristic failure is a query returning more rows than it should. No assertion about happy-path behaviour has ever noticed one.

That shape decides the whole review. Most of what the ecosystem has shipped for agentic development is built for application code in a typed language, where a defect eventually announces itself as a stack trace and a test that goes red. Our defects are silent by construction, and a tool that catches loud ones is worth very little here.

One constraint that makes us unusual and will stop being true: nothing in this collection has been released to anybody. Backwards compatibility is therefore not a constraint, and treating it as one is a mistake with a long tail, since every accommodation made now for a user who does not exist is a shape the design carries after 1.0. Rename the thing, change the signature, let the old caller break. This inverts completely at 1.0 and we have written down which side of the line we are on so that the argument does not have to be had again.

The review: four layers, and only one of them is empty

The spec layer is crowded, and solves the half that was never the problem

By 2026 every major coding tool ships a flavour of spec-driven development. GitHub's Spec Kit is terminal scaffolding, agent-agnostic, the most portable of them. AWS's Kiro is the most product-shaped: a VS Code environment where requirements become user stories become design documents, tracked in a UI. OpenSpec is the lightweight one, and tracks changes as deltas against existing behaviour with ADDED / MODIFIED / REMOVED markers. BMAD runs the whole thing as an agile ceremony with agent personas. Tessl is the most aggressive position in the category, spec-as-source, where the code is a regenerable artifact you never hand-edit, and it has raised $125m to say so. cc-sdd bolts the practice onto Claude Code. There is a live comparison site with a scoring heatmap, which is how you know a category has arrived.

We use none of them, and run the practice they describe.

The reason is specific rather than a preference about tooling. All of these are good at producing a specification and none of them is good at keeping one honest. The failure mode of a spec is not that it was badly written. It is that it was correct in March, the schema moved in April, and nothing in the world was watching. Every previous generation of this idea died the same death: UML round-tripping, model-driven architecture, architecture decision records. Not because the ambition was wrong, but because a hand-maintained artifact drifts from the code and then starts actively lying, at which point it is worse than nothing. Lehman's law of continuing change, operating on the documentation instead of the program.

The instrument for drift is a check against a running system. It is not a document format, a folder convention, or a slash command that writes three markdown files.

Two things are worth taking. OpenSpec's delta markers are the right shape for how a spec actually changes under an agent, because the interesting review question is never "is this document good" but "what did this change promise that the last version did not". And Tessl's registry bet is worth watching closely: 10,000 specs for external libraries, so that an agent writing against a dependency reads a contract rather than remembering one. If that works it removes a whole class of hallucination that no amount of local process touches.

The Tessl position, spec-as-source with regenerated code, is coherent and I do not think it survives contact with our system, for a reason that is about us rather than about them. Our source of truth is a schema. The SQL is the specification, in a language that a database will refuse to accept if it is wrong. Regenerating it from prose would be moving the truth up a layer, into a document nothing can reject. Where that argument does not apply, and for a lot of application code it does not, I think they are pointed at the right thing.

The structure layer is commoditised, and ours was already in the database

CodeGraph launched in January 2026 and was reported at 47,000 stars five months later. GitNexus went from around 1,200 to 42,000 in the same window. Serena wraps language servers and exposes their semantics as MCP tools: find_symbol, find_referencing_symbols, replace_symbol_body, project-wide rename. CodeGraphContext indexes into a graph database. On the research side, RepoGraph, CodexGraph, LocAgent and CGM all describe the same architecture within about a year of each other, which is what convergence looks like from the inside.

All of them hold structure. Files, functions, classes, who calls whom, what imports what, which types flow where, and the transitive closure of that, so an agent can ask which parts of the system a change could possibly reach. They build it by parsing rather than by prompting, which makes it exact and cheap to query, and the measured win is real: mostly tokens and precision rather than correctness.

Adopt, do not build. Specifically: Serena for percolate-core, the Python half, because language-server semantics beat every regex an agent will otherwise invent.

For the half that matters here, the graph is already sitting in the database and nobody has bothered to serve it. pg_depend holds the dependency edges. pg_proc holds every function with its signature and its body. pg_policy holds the row-level security rules. information_schema.role_table_grants holds who may reach what. That is a call graph, a permission graph and a dependency graph, maintained by Postgres itself, exact rather than parsed, and updated transactionally by the act of running the DDL.

The extension worth making is an MCP server over pg_catalog shaped for an agent rather than for a DBA. Not "run this query" but "what would break if I change this function's signature", "what can authenticated actually execute", "which policies mention this column". We use the catalog this way already, by hand, in about ten scripts. Nobody has packaged it.

The query layer is the closest existing thing to what we want

CodeQL extracts a relational database of syntax, control flow and data flow, and gives you a query language over it. Meta's Glean, Google's Kythe, Sourcegraph's SCIP and GitHub's stack-graphs are the same family at repository scale. Treating a program as data you can ask questions of is thirty years old and it works.

What none of them holds is why. A CodeQL query can tell you that this function is called before that one on every path. It cannot tell you that the order matters because of a lock we take in an unrelated module, that we found this out during an incident in March, and that the rule may be broken for the batch importer because it runs single-threaded. That is the gap, and I will come back to it.

The agent-facing layer settled faster than anyone expected

AGENTS.md is project-scoped context, adopted across 60,000 repositories by mid-2026, with governance now under the Agentic AI Foundation at the Linux Foundation, the same body stewarding MCP. The Agent Skills specification published in December 2025, and within two days Microsoft had wired it into VS Code and OpenAI into Codex. By March 2026 more than 30 tools read the same SKILL.md files out of the same folder. The whole specification is two required YAML fields and a markdown body.

We are entirely committed to skills, and ours live in meta/skills/. Two of them are symlinked into .claude/skills/ so they load without being asked for, and the choice of which two is the interesting part. One is the instruction to challenge the process, because a skill you have to remember to invoke cannot tell you that you are about to accept friction. The other is the tracker skill, because "file that" is said in passing rather than as a request to consult documentation, and a session that files without it duplicates an issue.

The design rule that falls out of having both a database and a skills folder is worth stating plainly, because it is the taxonomy the rest of this piece runs on. A rule belongs at the lowest layer that can refuse to violate it.

Everything above the skill line is enforcement. A skill is advice that travels well, which is a different thing. So the question to ask of every convention is not "which skill should this go in" but "how far down can I push this before somebody has to remember it". A convention that lives only in a reviewer's head is one an agent will violate politely and repeatedly, and you will spend your review budget on the same correction forever. A convention encoded as a constraint gets violated once, produces an error, and the agent fixes it without you.

The specification: how we develop

Ten practices. Each of them has a mechanism in the repository and an incident behind it, and I have given both, because a practice with neither is a preference.

1. A capability named in a spec is a row in surface.sql. That file holds 299 declarations: schemas, tables, functions with signatures, and the registrations documents depend on. It checks the database provides each one and exits nonzero. Every row carries the spec sentence it exists to keep honest, so a failure names the promise rather than the missing object: lexical:chunks — USAGE.md tells readers TEXT searches "chunks". Signatures rather than names, because fail_task(uuid,jsonb) and fail_task(uuid,jsonb,boolean) are different promises and the name alone would have passed while every caller broke.

The row is added when the sentence is written, not when the capability is built. A declared-but-missing row is the correct state for work in progress, because it puts the gap in the same place as everything else instead of in somebody's memory.

2. Check in both directions, or the check reads as rigorous and misses everything added since. A list-based check almost always asks "does everything I declared exist?" and almost never "does everything that exists appear in my list?" The first stops your inventory naming ghosts. The second is the one that catches drift, because adding a view is how a feature gets built and adding a line to the inventory is a separate act somebody has to remember.

Ours ran green for months. When we finally asked the second question, 13 of 15 REST-published views were absent from the inventory, including the two over the tables holding API keys and refresh tokens, along with 21 functions any logged-in user may execute, among them the one that grants roles. None of it was broken. All of it was undescribed, and therefore untested.

Scope the reverse check by reachability rather than by type. "Every function" produces a list nobody can finish and a check that gets disabled. "What an application role can execute or select" is the external surface exactly, and it came to 103 objects against a schema of thousands.

3. Push the fact down, and make the extension publish it. One home per fact is the easy half. Which home is the half that decides whether the copies come back, and for us the answer is Postgres, in the order in the table above.

The part that is easy to miss: "down" means into the extension, and the extension has to expose it. percolate-core is a service instance. A deployment may run several, may run none, may run somebody else's worker in another language against the same database. A vocabulary defined in that Python package makes an optional client the only route to knowledge that is not its to own. So the test is not "is this fact in Postgres" but "can a client that has never heard of our Python package discover it?" In practice that means percolate.enums(), percolate.contract(), and workflow.compiler_capabilities(), which report what the catalogue already enforces rather than restating it.

Then check from the database outward: does every closed set the database enforces have an agreeing copy upstream. dev/vocab-audit.py matches copies on their contents rather than through a name map, which is what stops it becoming the next one-way list, and it found workflow.tasks.kind declared three times in one file with only the last one live.

And read a running database, not only the tree. Measured while writing the skill this comes from: of four local databases, the one rebuilt from the tree agreed with the specs, and the three on the published image still admitted completed and done where the tree says succeeded. That is the exact vocabulary split the vocabulary table exists to end, alive in the artifact people install. No check that reads only the source tree can see it.

4. A table an extension owns is not a table whose rows it owns, and Postgres draws that line in one place. pg_extension_config_dump() decides whether pg_dump writes a table's data. Unregistered, it writes CREATE EXTENSION and stops, on the reasoning that an extension recreates its own contents. Correct for a vocabulary. Total data loss for a table holding somebody's users.

52 tables belonged to the extension. 2 were registered. A dump of a fully seeded database contained no users, no agents, no graph nodes and no content. Every backup this collection had ever taken held the schema and nothing else, and the restore came up empty and working.

The check is dev/backup-carries-the-data.sh: on a from-zero install, every extension table that is empty must be registered, because empty at install means every row it will ever hold came from a deployment. We tried to derive the list instead, and it disagreed with a live install on 9 of 52 tables, so the fact is stated once per table and what is derived is the check that you stated it.

5. Every gate, one command, and there is no second list. dev/check.sh runs five tree gates and eight database gates. .github/workflows/check.yml runs that script rather than restating it. The pre-push hook runs the same script. Each gate is named for the question it asks rather than the script that asks it, so that a failure locally and a failure in Actions are recognisably the same failure.

What that replaced was 13 named steps across two workflow files, a list of 11 commands living in prose in a commit message, and a guard script whose job was keeping the two in step. The guard worked. It was the wrong shape: a mechanism for keeping two copies in step is what you build when you have not noticed you could have one copy.

The runner itself failed in the way it exists to prevent. mktemp -t p8check is valid BSD and fails on GNU coreutils, so on the runner the log file was never created, every redirect failed before its gate ran, and twelve gates reported FAILED across 11 milliseconds having executed nothing at all. Green on a Mac, red on every runner, and the whole selling point of the file is that CI runs what you run, which was true of the commands and had never been true of the platform. The header of that file now carries a 30 second Linux container run, and it is the only local check that would have caught it.

6. A verification taken where the defect cannot appear is not a weak verification, it is not one at all. Six instances across three sessions in a single day, and everything about it looks like diligence. The command is right, the output is real, the exit code is read, and the answer means nothing.

Two of the six were the verification of a fix for another of the six, which is how a session ends up three levels deep in confident nonsense while doing everything else right.

The rule that comes out of it: before believing a verification, say out loud what the defect would look like in this subject. If the honest answer is "it could not be here", nothing has been verified yet. The tell is a check that passed for a reason you did not name in advance.

7. Build, then use it like a user, and only then decide what to test. The order is the part people get backwards. Build the thing to spec. Write a short conformance note rather than a suite. Then exercise it by hand, through the surface a caller uses, until it behaves reliably across every edge you can surface by actually using it.

Only after that, reason about failure modes: which of today's behaviours would a plausible future change break silently, without a runtime error, and what is the blast radius when it does. List the candidate tests that analysis produces. Write about a tenth of them, chosen for the guarantees that would decay quietly rather than for coverage.

Tests written before the manual pass encode your assumptions about the implementation rather than the system's failure modes, which is exactly backwards, and they are expensive to unwind because they look like progress.

8. Dogfood under one rule: you may only use what a user can reach. psql as a real role with real claims, PostgREST, the CLI, the HTTP surface, and the capability document the system publishes about itself. That is the whole toolbox. The moment you open schema.sql to find out what argument a function takes, the run has stopped measuring the thing it exists to measure, and the correct output is a finding: the surface could not tell me, and here is where I looked first.

The job has to cross at least two subsystems, because a job inside one is a mechanism test wearing a story, and it has to have a second half, because the first half is what everybody demos.

One run, for calibration. A pharmacovigilance desk wants a workflow to stop until a safety officer decides, and then record what they decided. Five steps. Everything in it is documented and every mechanism in it has tests. It produced four findings, and none of them is subtle:

• The acting step failed with template 'steps.officer.result' resolved to nothing. complete_task merges a task's output into the run context; signal_task does not. The second half of every human-in-the-loop workflow was unreachable from the document, and the worked example does not notice because it never reads the payload.
• timer: 3600 as the first step fired immediately, because the delay is applied by a trigger on status update and a root timer is inserted ready.
• record_chunks silently ignored start_char and end_char, which is the spelling the published example uses under the heading "with spans back into the source".
• claim_task returned one row of NULLs on an empty queue, on the most-called function in the worker contract.

All four survived because every existing assertion is about a mechanism, and each of these lives one call past the end of one.

Every finding becomes a scenario in dev/tests/evals/scenarios.yaml before anything is fixed, with status: open-defect and a because: that states the finding in full. There are 139 of them across 8 categories, each citing which of 28 named failure modes it would notice, and the loader refuses a scenario that cites none. The runner reports a declared-open scenario that starts passing, by name, so a fix nobody recorded is loud rather than silent.

The highest-yield line in any of these runs is the workaround you applied without noticing. A worked example in this repository ages a timestamp by hand before promoting a timer. That line reads as a testing convenience. It is a workaround for the root-timer defect above, applied without noticing, inside the file whose job is to demonstrate that timers work.

9. The quality gate is not PR review. Issues are generated from a spec's phase gates, one issue per gate. An agent picks one up and opens a pull request. Most of them merge without a human approving the diff.

That is a deliberate bet with two ends. Upstream, the spec and the issue are precise enough that an agent builds the right thing first time. Downstream, an ephemeral staging environment spins up inside the existing cluster on merge, and the phase gate is proven there: not asserted in a design document, and not treated as satisfied by a pull-request-time job.

The honest version of the bet is that it fails loudly if either end is weak, and that we would rather find out in staging than pretend a four-minute approval on a nine-hundred-line diff was ever a control. Code review has always been partly theatre. Volume just removed the plausible deniability. Faros AI's 2026 report has median review time up fivefold and 31% more pull requests merging with no review at all, which is the theatre point with a number attached. Building a heavyweight review process on top of that number would be spending effort where the reliability is not.

10. Write up what was expensive, and treat the third instance as a missing gate. Every change that was harder than it should have been gets a file in meta/postmortems/, written at the end while the detail is recoverable. Not a summary of the change, which the commit message and the spec already have. What cost time, why, and what would have prevented it.

The repository already records what is true, what is promised, and what is enforced. None of those records what was expensive, and what was expensive is the only real input to deciding which gate to build next.

Before fixing anything, ask why three times, because the first answer is a symptom and the third is usually where the fix lives. Then two commands: git log on the file to see whether this has been fixed before, and a grep through the postmortems to see whether it has been written up before. One occurrence is history. Three is a missing gate, and the fix is the gate rather than the third instance.

The inverse matters as much and gets forgotten. The bar for a new check is an incident, not a risk. A speculative gate costs on every run forever, catches nothing, and trains people to skim the output, and a gate that is skimmed is one that gets bypassed when it finally does fire. The tax is certain and the benefit is hypothetical. Two proposals were made here on the same afternoon: a deployment-coherence check with three measured instances behind it, one of them live on a published image, and a guard against a case that had occurred once, in the reporter's own fixture, against a design decision that is deliberate. The first was built. The second went to the backlog and is still there.

The directory has an index that reads across all the files and names each recurring class with what was done about it. That index, rather than the individual files, is the mechanism. It is what let us delete a tool rather than repair it, on the strength of two entries that had already recorded its regeneration as a pending chore and one that had already published the verdict on its denominator. The evidence was in the record before anybody went looking.

What we built, measured, and deleted

We ran the claims-graph idea at full scale and killed it, and the measurement is the reason.

dev/claims-audit.py was 711 lines over a committed CSV of 1,312 extracted claims, wired into CI as a coverage ratchet. What the CSV actually held:

The headline number, 43% of claims covered, was computed over a denominator that was mostly prose. It moved when somebody edited a sentence in a spec rather than when coverage changed. The subsystem's own review called it a ratchet whose denominator is mostly prose, and said it was the largest instance of that in the repository. Regenerating the CSV was listed as a pending chore in two separate postmortems, which is the real tell: a gate with a maintenance ritual attached is one people route around.

What survived is one line of shell. Cross-reference every invocable name against the test suite and read the zeroes. That is dev/uncovered.sh now, a report rather than a gate, with no ratchet, no snapshot and no place in CI. It found an entire unexecuted authentication path the day it was written, and the function in question scored one, where the one was its own name inside a list.

The rule for anything that wants to join check.sh came out of that deletion: a gate belongs there if it can name the specific thing that is wrong. A gate that reports a percentage, or that needs an exceptions list to stay green, will be skimmed and then bypassed.

The audit's own numbers said something true about the specs rather than about the tests, and it is worth more than the first finding. 52% ill-defined does not mean 52% untested. It means half of what we had written down named nothing anybody could invoke, and a claim that names nothing cannot be tested by a script or by a person, because there is nothing to look for. When we separated claims about the advertised surface from claims about internals, the two populations looked completely different: interface claims were 13% ill-defined, because a claim about an API names the API, and internal claims were 68%, because they are design reasoning and reasoning does not name a testable subject.

So the alarming coverage number was mostly measuring the presence of prose. That is not a coverage problem and no amount of test writing moves it.

The gap: claims, on the second attempt

The first attempt failed at extraction. It read the specification and tried to turn sentences into claims, which produced a large pile of things that were true, unfalsifiable and unattached. Everything downstream inherited that.

So invert it. A claim is not extracted from prose. A claim is born at the moment something is violated.

We already write down every violation, in four places, and all four are anchored by construction:

• A postmortem names a defect, the mechanism, and what would have prevented it. 8 of them. Every one contains at least one sentence of the form "X must be true and was not".
• An eval scenario carries a because: line that states the guarantee a red line protects, in the user's words, with the SQL that checks it attached. 139 of those exist today, and they are claims that already have their test.
• A revert or a fix commit is a fossil of a claim that was violated. git blame is archaeology we already do manually, and only when something is on fire.
• A surface row carries the spec sentence it keeps honest. 299 of those, each already joined to a database object.

That is the corpus. It is small, it is anchored, and every entry in it is a claim somebody paid for. Nothing in it was generated by asking a model to find assertions in a document.

The node

One row, in the database, in the extension. Not a YAML file, and the change of mind is the point: a fact belongs at the lowest layer that can refuse a violation of it, and a claim about a SQL object should join to pg_proc by oid rather than be matched to it by string.

claim
  statement      what must be true, in the words of whoever it is a promise to
  anchors        oids and names, joined to the catalogue, not guessed
  invoke         the surface you call to make it fail: a route, a function, a command
  exception[]    where the rule does not apply, and why
  incident       what produced this claim: a postmortem, an eval id, a commit
  check          the assertion that would go red, or null
  expires        set only when check is null

Three fields carry the difference from a linter.

exception with its rationale. A node holding nothing but an assertion recreates static analysis and earns the same fate, which is a rule added to the ignore file in week two. What makes this different is the rest: the scope, and the known exceptions with their justifications. The rules you are allowed to break, and what happens when you do. Chesterton's fence with a note nailed to it naming who put it there.

incident. A claim with no incident behind it is a risk somebody imagined, and practice 10 says those go to the backlog. This field is what stops the graph refilling with speculation, and it is why the corpus starts at the postmortems rather than at the specification.

expires. Prose is the rot vector. A claim that cannot be falsified by machine gets a date, and on that date it either acquires a check or it leaves. Not a warning, not a report. It leaves. The alternative is what killed the first attempt: a large and growing population of true, unfalsifiable statements that make every number computed over them meaningless.

The trigger

The value is in the diff, not in the graph. We are not trying to hold the whole picture at once, and we are not asking a model to comprehend the system.

A change lands. The blast radius comes from three joins the database can already answer: pg_depend for what depends on what, the grant graph for who can reach it, and the claim-to-anchor table for what has been promised about any of it. That names a small set of claims. Only those are re-asserted against the new state.

Which is a close description of what a long-tenured engineer does when somebody proposes a change. They do not re-derive the system. They attend to a handful of specific things at exactly the right instant, and that attention is the part that walks out of the building when they leave.

The surface

One MCP tool, with an expressive argument. Not one tool per question, for the reason every tool costs its name, its description and its argument schema in every prompt forever, on the turns it is irrelevant as much as the turns it is not.

claims("REACH aiq.query")           what must stay true if I touch this
claims("HISTORY workflow.tasks")    what did we learn the last time somebody did
claims("OPEN core")                 actionable, weighted by where the claim was made

The middle one is the query nothing in the ecosystem answers today, and it is the one I want most. An agent about to modify a function asks what broke here before, and gets four sentences from three postmortems and a revert, rather than a call graph.

The numbers we will not publish

No coverage percentage. The first attempt died of one, and the failure was not that the number was low.

The ratchet, if there is one, is on the count of actionable claims grouped by where they were made, core before spec, and it fails when the count goes up rather than when a percentage goes down. ill-defined and future are different work and mixing them produces a backlog nobody starts: measured once, 735 untested claims were 71 actionable ones.

And the loop has to be able to terminate. Eventually the largest actionable cluster is two claims, the covered hubs are the big ones, and the correct read is that the next work is not more tests. A method that only describes its loop leaves people running it past the point it pays.

What would make us kill this

Two tests, and I would rather write them now than discover the answer in a year.

The retrospective test, before we build anything. Run the extraction against the 8 postmortems we already have, then ask, for each one, whether the graph would have named the relevant claim in the blast radius of the change that caused the incident, before the incident. If it names fewer than half, the extraction is wrong and no amount of graph is going to fix it.

The live test, six months in. If the claims that fire during a change are the ones the diff already made obvious, and the ones that mattered were never in the graph, this is decoration. The measurement is cheap: every finding from an eval run or a review gets asked whether the graph had it. That ratio is the only number worth reporting about the thing.

A covered claim can still be false, and neither test above moves that limit. Coverage says a test touches what the claim is about. It cannot say the test asserts the right thing. In one review pass here, 7 real defects were found by adversarial review and the claims audit would have surfaced none of them: 4 were behaviour nobody had claimed, and of the 3 that had claims marked covered, the sharpest was an unauthenticated account takeover whose covering test executed the exploit end to end and called it the happy path.

So the graph answers which claims have no test. It does not answer whether the code does what the claim says, and a green audit is not evidence about the second. Run adversarial review alongside it, permanently. On our evidence the two find disjoint sets of bugs.

What we adopt rather than build

Most of this is already somebody else's problem, which is the point of the review.

• Serena, for language-server semantics over the Python services.
• tree-sitter, wherever a parse is needed and no language server exists.
• CodeQL, for the Rust extension and the Python package, on the strength of its data-flow queries rather than its default suite.
• Agent Skills and AGENTS.md as the distribution format for every rule that cannot be pushed into the database. Portable across more than 30 tools, two required YAML fields, no server to keep alive.
• MCP as the transport for anything an agent queries at decision time.
• OpenSpec's delta markers, as the shape a spec change takes under review.
• DORA's four measures, as the outer scoreboard. The AI-era numbers point the other way from the classic result, which makes them the thing being measured rather than a claim being made.

What we will not adopt is any tool whose primary output is a percentage.

The scoreboard we owe

Every argument about agentic development is currently made with anecdotes, and mine is not exempt. The industry numbers are not reassuring: throughput up and stability flat or worse, review time up 91%, AI-assisted pull requests about 154% larger and failing first review at 67% against 15.6% for human-written code, and a median team's main-branch success rate at a five-year low while feature-branch activity climbs.

We cannot report against those yet, and the reason is not flattering. Nothing here has shipped to anybody, so our change failure rate is a number about ourselves. That is a fact about our stage, not a defence, and the moment there is a user the four measures start applying to us the same way they apply to everyone.

What we can report is the thing this whole process is built to move: how many defects were found by a check versus by somebody using the system. Right now the honest answer is that the checks find the loud ones and every serious defect in the last month was found by standing the stack up and using it, on data with more than one tenant in it and more than the happy path in it. The query mode that returned chunk ids and cosine distances for content the caller was not allowed to read. The mode that resolved a seed key fuzzily and reported neither the guess nor the node it walked. The permission policy that scoped through a session, for a run kind that has no session. Each of those reads fine. Each passed review.

If the claims work does anything, that ratio moves. If it does not move, the ratio will say so.

What did not change

I keep expecting one of these practices to fall away and none of them has. The tools are astonishing and the discipline is the same discipline. If anything the classical stuff has appreciated, because the cost of skipping it used to be paid in slow, visible bugs and is now paid in fast, plausible ones.

The one thing that genuinely changed is which documents are dangerous. A stale sentence in a design document used to cost a person ten minutes of confusion. It now costs an agent an afternoon of confidently building the wrong thing, and then it costs you an afternoon of review to find out. Documents that lie are a production issue, and that is the whole reason a claim needs a check attached and an expiry date if it cannot have one.