In my experience, when a model writes most of your code, the first thing to degrade is not code quality. It is your knowledge of what the system already contains.

I discovered this in a surprisingly ordinary question: How friendly must a character be to count as a friend?

My project had five answers. FriendThreshold = 10 appeared as a private constant in three files. Two more files used BestFriendThreshold = 20 and FriendRelationThreshold = 20 for a neighbouring question. Each class was clean. Nothing was obviously broken. Yet the same area of the domain had quietly acquired five separate decisions.

The project is a mod for a strategy game, in production since June 2026, with more than 5,300 unique downloads. I built it entirely with an AI model and have not written a line of its code. I do not review that code either: with my experience as a developer, reviewing every line would cost about as much as writing it, and speed is the reason I work this way. I have never played the mod.

That unusual arrangement changed my relationship with the software. The model begins each session without the knowledge carried through earlier conversations; unless something directs it to an existing concept, it has no reason to look for one it does not know exists. Meanwhile, as I moved from writing code to describing what I wanted, I lost the small design moments that used to happen while typing: pausing over a name, noticing a familiar rule, questioning where something belonged.

I had stopped being the coder of the system and become its customer, while remaining responsible for its architecture.

I needed a way to keep making architectural decisions without relying on having read every implementation.

Why better instructions do not fix it

My first answer was the obvious one: tell the model what good code should look like.

From the beginning, I had detailed instructions about layering, SOLID and Tell, Don’t Ask, alongside specifications and other guardrails. The five thresholds appeared anyway. Each class could follow the rules perfectly while the duplication emerged between classes.

An instruction to a model is advice, not a constraint. It helps only if the model considers it at the right moment, among thousands of other tokens. There is no automatic alarm when it does not. My instructions file eventually described 16 sections of a prompt that had grown to 108.

Generated documentation was not enough either. One file listed every rule guaranteed by a test, with a header telling readers to regenerate it when the tests changed. Ten days passed without regeneration. The document fell about 600 rules behind, but looked as authoritative as ever.

That was the problem with both approaches: nothing forced the description to remain aligned with reality.

A document stays trustworthy only if something compares it with the system whenever the system changes, and refuses to let a difference go unnoticed.

Others have approached the same problem from another direction. In a recent video, Dave Farley describes a growing consensus that prompts and Markdown specifications cannot reliably steer an agent on their own. Specifications need to be executable. His acceptance tests establish what the system must do while leaving the agent freedom over how it does it.

But that freedom is exactly where my five thresholds came from. All five implementations could satisfy the behavioural tests. The behaviour was correct; the structure was drifting. Acceptance tests deliberately avoid prescribing implementation structure, so I needed their counterpart: executable checks on the parts of the how that matter architecturally.

A map: a generated view, held true by a test

The first mechanism was simple: generate a map of the code, commit it, and make a test fail whenever the code and the committed map disagree.

This is approval testing applied to structure rather than output. A small generator reads the source as text, using a syntax parser rather than requiring a build, and produces Markdown.

For example, the top of a real constants map groups names declared in multiple places:

1
2
3
4
5
6

## One name, several homes

| Name            | Where, and what it is worth there                           |
|-----------------|-------------------------------------------------------------|
| ArrivalDistance | AmbushBehavior = 2.5; MissionBehavior = 2.0; PatrolBehavior = 2.0 |
| CooldownDays    | EncounterService = 1; AmbitionPolicy = 30; DuelPolicy = 7   |

Not every repeated name is a defect. Sometimes two declarations represent one concept that should have one home; sometimes they are different concepts with similar names. The map does not decide. It brings the question into view so someone can.

From then on, each change produces two kinds of diff. The code diff shows what was edited. The map diff shows what changed about the system’s structure: perhaps a new dependency between modules, or another declaration of an existing name. Even an unchanged map tells me something about the scope of a change.

Of course, a test helps only if it runs. A pre-commit hook or a pipeline can provide the trigger. My mod needs the game’s libraries and cannot be built on a server, so I run the checks locally. In a team, the checks and their baselines belong in protected continuous integration, beyond the agent’s ability to approve its own exceptions.

The underlying idea is not new. Neal Ford, Rebecca Parsons and Patrick Kua describe architectural fitness functions; ArchUnit, NDepend and PublicApiAnalyzers cover important parts of this territory. My maps address questions specific to the system, including cases where the code cannot be compiled in the test environment.

The closest recent practice is what OpenAI calls harness engineering, described in February 2026: architecture documentation kept consistent through linters and CI, structural tests that enforce layering, and repository maps that help agents navigate the code. That work focuses on the environment in which agents produce a coherent system. I am interested in the other side of the loop: how the person responsible for the architecture continues to judge it without reading all the code.

A failing test is a message to the model

The maps became useful for a reason I had not initially appreciated: when their information appeared.

An instruction says, “Remember this rule.” A failing test says, “Something you just changed affects this rule. Look at it now.”

The second message arrives while the model is still working on the change. The suite remains red until the discrepancy is addressed. A red map test does not necessarily mean the software is broken; it means a structural change requires attention.

That only works if the failure explains itself. My tests show the moved lines and, where a contract blocks a change, the reason it is blocked and how to handle an intentional exception.

Here is a real failure, triggered deliberately by deleting a public API member that had shipped in the previous release:

1
2
3
4
5
6
7

Failed GIVEN_a_released_modder_api_member_WHEN_the_code_no_longer_has_it_THEN_the_suite_refuses_it
  These public members shipped in v2.8.1 and are no longer there:
  CrLetters: static bool Deliver(Hero sender, string body, string sourceMod)
  A bridge built on them breaks on a player's machine. Put the member back (mark it
  [Obsolete] and keep it working if it must go), or, if removing it is truly intended,
  set CR_ALLOW_API_REMOVAL=1, name it under "For modders" in the release notes,
  and say why in the commit.

The failure identifies the missing promise, explains whom the change would affect, and offers two legitimate responses: preserve compatibility or explicitly acknowledge a breaking change.

The timing is not perfect. The test fires after code has been written, so reading a map before editing is still preferable. The test catches what that habit missed, sometimes at the cost of rework.

Still, this is the insight I find most transferable. Farley uses a red test to show that a specification can reject a wrong answer. Here, the failure is also a communication channel. Designing tests around what they tell the agent when they fail matters as much as deciding what they check.

What makes a map worth reading

A map only earns its place if reading it is cheaper than reading the code.

In practice, I aim for one fact per line, with enough context (owner, name and value) for that line to make sense when an agent retrieves it through search rather than reading the document from top to bottom.

There is a limit to what such a map can claim. It gathers what the source declares, and its reliability depends on the names and comments it finds. Missing documentation leaves a visible gap. A misleading name is worse: it can make an incorrect description look complete.

A map cannot be more honest than the names it reads.

I learned this through a detector for hard-coded relationship thresholds. The first version searched for relevant names, and its count appeared stable for months. Then I added a detector for comparisons between relationship values and numeric literals. It found 33 cases the original had missed.

The map now records which detector produced each line, so a reader can distinguish directly extracted facts from inferred matches. Generation makes the output reproducible. It does not make the interpretation true.

When informative maps stop informing

Most maps inform: the test fails, someone inspects the diff, regenerates the approved snapshot, and commits it.

That process can lose its purpose without anyone noticing.

The model had started regenerating maps before checking them. Each test then compared the new map with itself and passed. This went on for weeks. Once I corrected the order (compare first, inspect the changes, then regenerate to approve), the first honest review exposed a blind spot in my dependency detection.

Even with the right order, attention is fragile. A review can slowly become a ritual of “regenerate and commit.” That is why I now treat informative maps as the fallback, not the goal.

For every map, I ask: can a rule decide whether a change is acceptable? If the question is “Did this measure get worse?”, often it can. If the question is “Do these two declarations mean the same thing?”, the answer still needs human judgment.

That distinction led me to two stronger mechanisms: ratchets and contracts.

Ratchets: make yesterday’s debt the ceiling

A ratchet preserves a measure that is allowed to improve but not worsen. The current value becomes a ceiling. If the measure rises, the test fails. If it falls, the ceiling must be lowered in the same commit.

That last step matters. Without it, a cleanup does not protect its own gains: the old ceiling would permit the debt to return.

Today’s mess becomes tomorrow’s ceiling.

This makes ratchets practical on legacy code. I do not have to eliminate every existing problem before I can prevent new ones.

The measure must also reflect the problem. For dependency cycles, I compare the set of namespaces involved, not just the number of cycles. Splitting one namespace can increase the number of counted cycles without actually worsening the architecture.

A ratchet gives an agent a direction it can test. “Improve the architecture” is vague. A ceiling of 46 is concrete: reduce it, confirm the test, and update the baseline. When one of my ratchets reached zero, it became a wall. The debt it measured had turned into a rule prohibiting its return.

That does not mean maps or ratchets improve architecture by themselves. They expose relations and prevent certain regressions. Someone still has to decide what should change and why.

And a guard protects only what it can detect. My first dependency map read using directives, but missed references expressed through fully qualified names. Real cycles escaped detection while the ratchet stayed green.

A good way to test a guard is to make it fail on purpose. Remove a public member. Introduce a cycle through a qualified name. If the suite stays green, the apparent protection is an illusion.

Farley also warns about agents removing tests to make a suite pass. The same applies to baselines: lowering a ceiling as debt improves can be automatic, but raising one, or editing a released contract reference, changes the specification itself and should require a person’s approval.

Contracts: protect promises made outside the codebase

Some structural changes are not merely worth reviewing. They break promises to consumers: a key stored in a player’s save file, for example, or a public member that another mod compiles against.

For these, I use blocking contract maps. If a released member disappears, the suite fails even when someone tries to regenerate the map. The baseline is the last released state, stored separately and updated only at release. Otherwise, accepting a new snapshot could quietly erase the very promise the guard exists to protect.

Sometimes additions break compatibility too. In a second project, each new feature added required fields to an export format. A file written an hour earlier could no longer be imported. The map displayed ten apparently harmless added lines; the model recognized the consequence during review. The guard now rejects new required fields relative to the last release, and an original exported file remains as a regression check.

A useful contract must identify the consumer’s promise, not the implementation detail behind it. During a retrospective replay of Jellyfin, a contract whose identity included the handler method generated 112 alarms on a single commit that only moved files. Keyed on the promise alone, verb and route, the same contract raised two alarms in thirteen and a half years, both on real losses.

A gate’s usefulness depends on its false-alarm rate. If it cries wolf often enough, people switch it off, however sound its underlying logic. The stable identity is the one the consumer actually depends on.

Contract maps have another benefit: they let me review design decisions without opening the code.

After a bug report from another mod author, the model proposed a new public API member: PlayerIsActor, a boolean. I saw the addition in the API map and asked whether other mods should have to understand my mod’s notion of a player’s deed.

They should not. The model replaced the flag with a list of the heroes who acted, leaving my mod to draw its own conclusion. The member had not shipped, so this took one commit rather than a breaking release.

I did not need to read the implementation to make that decision. The contract showed me the shape of the promise at the level where I could judge it. Farley argues that executable specifications can make code reading a choice based on risk. A structural contract can do the same for architecture.

Decision tables: check the behaviour of a refactor

One kind of map goes beyond reading source code. It runs rules.

A decision table sweeps a value across its range, records the output of every rule that depends on that value alone, and commits the result. When a refactor is meant to preserve behaviour, the expected diff is empty. “I only moved code” becomes something that can be checked.

The limitation is equally important: a rule combining the swept value with other inputs remains outside that table’s view.

What maps cannot see

My first maps largely catalogued names. But two rules can represent the same concept without sharing a name.

When the model added a rule based on a clan’s rank, it searched the constants map and found one existing rule. A direct source search found two more, neither represented by a named constant.

So we changed the question. Instead of asking what are the rules called?, we asked what game facts do they read?

The resulting map follows values from the game into pure rules and groups those rules by the facts they consume. Under clan rank, it revealed 28 entries covering roughly fifteen distinct features, where the manual search had found three.

This gave us enough visibility for a refactor: rank became a single type accessed through one gateway, and the map confirmed that rules no longer read the raw value directly.

Then the model reached a harder question: What counts as a ruler? Eighteen rules gave answers that were not equivalent. The map could collect every reader of the concept. It could not decide what the concept was supposed to mean.

Static maps have other boundaries. Because they read source, they cannot tell me what the game engine actually does at runtime. Each feature that interacts with the engine therefore has a self-test, launched through the game console against a real save and the production code path, with game state restored afterwards. One such test revealed that the engine reuses identifiers of destroyed entities, something tests outside the engine had not exposed.

The mod also uses a language model internally. Putting a fact into a prompt does not ensure the model will behave as expected. Where a rule can be checked literally, I check it literally: a secret name never included in the prompt must not appear in the reply. I reserve model-as-judge tests for cases that cannot be verified that way, and run them on demand because they cost money.

These checks complement the maps; they do not make the maps comprehensive.

Where intent comes from

There is an objection worth confronting. If a test is generated from existing code, does it do anything more than confirm the code’s current behaviour? Where does the intended design come from?

Informative maps do not claim to verify intent. They describe what the source currently contains and make changes to that description visible. Intent enters when someone decides which changes are acceptable, which measures must not worsen, and which promises must remain stable.

Behavioural intent needs a home as well. In my project, it lives in tests. Each test file begins by describing how the rule could go wrong, recorded when the rule was decided. For example, the policy allowing a player’s spouse to request a divorce names two opposite failures: a happy marriage disturbed by unsolicited letters, and a miserable one in which nobody ever speaks up. Some tests even cite the question I asked that led to the rule.

Intent is easiest to capture when it is formed. Later, we can only reconstruct it.

One problem remains unsolved. When the same model writes both implementation and tests, the tests can end up confirming whatever the implementation already does. Evidence from outside the loop helps: reports from real users, translated into regression tests, and a human reviewing the structural diffs.

Testing the idea beyond my own project

I wanted to know whether the structural patterns I was seeing were peculiar to AI-generated code. So I turned to software developed in the more traditional way: Jellyfin, an open-source media server with thirteen and a half years of history, about thirty maintainers and mandatory code review.

I replayed the generators over all 11,221 commits. If ordinary review eliminated this kind of structural drift, the maps should have found little worth surfacing. They found considerably more, but what they found needs to be interpreted carefully.

The last row matters as much as the others. Many meaningful changes do not appear in these maps. That is why they need behavioural and runtime checks alongside them.

The replay also has clear limits. It shows what the maps would have displayed, not what a reviewer would have done. And the cycle figures may undercount real dependencies if the replayed generator shared the blind spot I found in my own project.

My forward evidence comes from the mod itself. It has been in production since June 2026, carries more than 6,600 unit tests and over 200 in-game self-tests, and ships updates frequently: a player recently remarked that a new one appears roughly every three days. One project developed by one person demonstrates that the approach can be used, not that it improves outcomes for every team.

I also tried the method on a smaller project in another language. I asked a model to create a Postman-like HTTP client in JavaScript, with maps present from the first commit, and later asked it to evaluate its own architecture. I wrote no code.

In roughly an hour and a half and twelve commits, similar drift appeared: small rules were rewritten beside their originals, and a main file grew from 237 to 353 lines through individually reasonable additions. A subsequent split brought it down to 25 lines. On a fresh project, ratchets can start at zero and act as walls from day one, rather than freezing an existing level of debt.

This was still a short experiment with the same author, model and skill, not an independent team trial. The repository preserves the commit history for inspection.

Where to start, and why this matters now

I would not begin by mapping everything.

Start with one generated map and a test that compares it against the committed version. Protect any contracts with external consumers. Then add maps and guards in response to problems you have actually encountered more than once. Each map should earn its place by making a recurring architectural decision visible when it arises.

I packaged this approach as an agent skill. Its README covers generators, test patterns and a suggested adoption order.

Why do this now? Because the economics of change have shifted. With an AI model, writing or rewriting an implementation has become cheap enough that I often prototype and correct rather than specify everything in advance, but finding out what deserves correction has not become cheaper. Verification matters as much as before; it now lives less in specifications written before implementation and more in checks that run whenever implementation changes.

We used to write code so our colleagues could understand it. Now we also need to preserve why it was written for a reader who has no memory of the conversation that produced it. More and more often, that reader is the model itself.

The developer’s role changes with it. Someone who delegates implementation becomes responsible for an architecture they can no longer keep in their head by reading each line as it is written.

Maps do not restore that lost familiarity. They provide something different: a view of the system that cannot quietly drift without a test noticing, and that arrives when a change makes it relevant. Combined with explicit constraints and human judgment, that can be enough to keep making sound architectural decisions about code we no longer remember writing.

Gabriel Mailhot is a Senior Subject Matter Expert at Desjardins, where he works on development practices, architecture, and complex organizational problems spanning technology, process, and governance. He publishes at scripting.pro.