What actually matters is far less intuitive, and it changes how we should think about LLMs as instruments for creativity.

The Broken Promise of the Creative Prompt

Ask any major LLM for novel solutions to a real problem and you usually get back a polished recombination of known best practices.

Take this question: How can we better prevent technical debt in large enterprise IT projects with fixed dates and budgets?

You’ll typically receive: short Agile sprints, 15-25% refactoring buffers, SonarQube, “stop the line” culture, and KPIs in vendor contracts. Useful and professional, but entirely predictable.

If you’re looking for established best practices, this is exactly what you want. The model is doing what it was trained to do, interpolating the most statistically probable patterns in its training data, and for consolidating known solutions that is a strength.

But if you’re looking for a genuinely new idea, something no one has tried, then that same interpolation is the obstacle. The model is fine; we’re asking it to perform the wrong operation.

Directed Metaphorical Association

Years ago, while researching creative systems, I started experimenting with a different approach: directed metaphorical association. Instead of asking for solutions directly, you identify the strong semantic concepts in a problem, pull rich metaphors from distant domains, transfer properties across the gap, and let the human judge what emerges.

The logic is simple. If you ask an LLM to solve a technical debt problem, it searches its training data for content about technical debt. You get back what the industry already knows. But if you ask it to find structural parallels between technical debt and, say, scuba diving, the model is forced to traverse a completely different region of its knowledge. The connections it finds can’t be recycled best practices, because no one has written about the intersection of nitrogen absorption and code quality.

At the time I developed this, it was all manual. Today, the attention mechanism in Transformer-based LLMs lets us scale this approach dramatically: the model can compute semantic relationships across domains far richer than anything I could code by hand.

But scaling it revealed something I didn’t expect. The results weren’t uniformly creative. Some metaphorical transfers landed immediately and felt actionable. Others were intellectually fascinating but resisted any practical application. At first I assumed this was noise, since some metaphors just work better than others. But the pattern was too consistent to be random, and it turned out to be more interesting than the ideas themselves.

Same Problem, Directed Metaphors

When I applied the method to the technical debt question, directing the LLM to generate metaphorical associations from distant domains rather than recombine known practices, four ideas emerged:

Directed fermentation. In cheesemaking, you don’t fight bacterial activity; you control which bacteria dominate. The distinction that matters is controlled versus uncontrolled, not clean versus dirty. A bad cheese isn’t a cheese with bacteria; it’s a cheese with the wrong bacteria.

Transfer: maintain a catalog of authorized shortcuts (a “starter culture”), each with its documented lifespan and self-replacement rule. Debt that follows the starter is tolerated. Debt outside it is rejected. You stop trying to eliminate debt and start trying to control which debt lives in your system.

Mandatory decompression. In scuba diving, nitrogen dissolves in the blood under pressure. That’s physics, not a negotiation. If the diver ascends too fast, the nitrogen forms bubbles. Since preventing nitrogen absorption is impossible under pressure, the solution is decompression stops, calculated mathematically and non-negotiable.

Transfer: accept that fixed deadlines and budgets will dissolve debt into the codebase. That’s a law of development under pressure, not a failure of discipline. Model debt accumulation as a calculable variable and insert mandatory decompression windows whose length is determined by the model, not by managerial goodwill. You don’t negotiate the laws of physics.

Programmed selective amnesia. The brain actively prunes weak synaptic connections during sleep. Forgetting is a survival mechanism, not a malfunction. Without this pruning, the brain saturates, loses its ability to discriminate, and starts treating noise as signal.

Transfer: at fixed intervals, delete entire modules and force them to be rewritten from their behavioral contract alone. This sounds radical, but the point is what the deletion reveals. Code that cannot be reconstructed from its spec exposes that the spec was insufficient: the real debt was in the specification all along, not in the code.

Unrehearsed ensemble. Jazz bands booked for festival slots with non-negotiable stage times cram complex new arrangements into minimal rehearsal hours. Every player memorizes the charts individually. But onstage, the improvisation falls apart into disjointed solos, because knowing the material individually is not the same as having internalized it as shared group intuition.

Transfer: a team where every developer knows the codebase individually can still produce incoherent architecture, because no one has built the shared intuition of how the system wants to evolve. The debt lies in the absence of collective internalization, which only emerges from working together on the same problems over time, rather than in any individual’s competence.

Something Is Off

Read those four ideas again. Two of them, fermentation and decompression, feel immediately actionable. You could present them to a project director tomorrow and have a productive conversation. They fit inside the existing mental model of how IT projects work; they just reframe a specific aspect of it.

The other two, selective amnesia and the unrehearsed ensemble, are more demanding. They don’t reframe an aspect of the existing model. They ask you to question what you consider the primary artifact (is it the code or the spec?), or what “team competence” even means (is it individual knowledge or collective intuition?). You can’t just “apply” them. You’d need to rethink something basic first.

This isn’t a question of quality. All four are intellectually rich. But they operate at different levels, and that difference is consistent and reproducible enough to be worth understanding.

The Red Herring: Semantic Distance

The obvious explanation would be that some metaphors are simply “too far” from the target domain. That’s the classical thesis from Mednick (1962): close associations are banal, distant associations are incomprehensible, and creativity sits in an optimal zone in between. It’s an elegant idea, and it has dominated the creativity literature for over sixty years.

But it doesn’t explain what we’re seeing here. Fermentation comes from biology. Selective amnesia comes from neuroscience. Both are roughly equidistant from software engineering. Yet the first is immediately applicable and the second is not.

If distance were the relevant variable, ideas from similarly distant domains should behave similarly. They don’t, so something else is at work.

First Finding: Transfer Granularity

What separates the two groups is the size of what gets transferred, not distance, since both draw from domains equally far from software engineering.

Look at what each idea actually imports from its source domain:

Fermentation imports a single rule: “the relevant distinction is not presence versus absence, it’s controlled versus uncontrolled.” One causal relationship, stripped of all cheesemaking context. You can plug it directly into a conversation about technical debt without asking anyone to change how they think about software.

Decompression imports a single rule as well: “accumulation under pressure is a physical law, and the stops are calculable.” The diving context is gone. What remains is a structural insight that maps cleanly onto fixed-deadline projects.

Now look at what the unrehearsed ensemble imports. Instead of a single rule, it brings a system of relationships: the idea that individual competence does not produce collective competence, that shared intuition is a distinct artifact that only emerges from sustained joint practice, and that the absence of this artifact produces incoherence even when every individual performs well. To apply this to a software team, you need to accept all of these premises simultaneously. You’re not plugging in a rule; you’re redefining what “team competence” means (a separate thing that either exists or doesn’t, rather than the sum of individual skills).

Selective amnesia does the same thing. It doesn’t just say “delete code sometimes.” It requires you to reconceptualize what the primary artifacts are, to accept that code is expendable and specs are foundational, to rethink the relationship between implementation and contract. That’s a system of interlocking ideas, not a single transferable rule.

Semantic distance matters: it ensures the pieces are genuinely different from what the target domain already knows. But the variable that predicts whether a metaphorical idea will be immediately useful is the granularity of the transferred unit. Single causal rules are portable. Systems of relationships require a worldview shift.

Both can be valuable, but they serve different purposes, ask different things of the audience, and land on very different timescales.

Second Finding: Compression at the Right Level

If granularity is what matters, the practical question becomes: how do you control it? How do you reliably extract single causal rules from a rich source domain instead of accidentally importing entire systems?

The answer is compression.

When you extract a causal rule from a source domain, you’re performing lossy compression in the information-theoretic sense (keeping only the causal skeleton, discarding everything else). You take a rich domain, scuba diving with its physics, decompression tables, equipment protocols, certification culture, and accident case law, and you compress it down to a bare skeleton: “inevitable accumulation under pressure → calculated mandatory stops.” Everything else gets discarded. What remains is portable.

What I didn’t expect is that this compression doesn’t work as a smooth dial you can turn gradually. It has discrete levels, like quantization of a signal (steps, not a ramp). In practice, I consistently see three distinct levels:

The raw metaphor. “Technical debt is like cancer.” This is barely compressed at all. The metaphor arrives with its full baggage: diagnosis, staging, treatment, remission, relapse, death. You’re not importing a rule; you’re importing a scenario, and the scenario drags in properties that don’t map to the target domain at all. (Does technical debt metastasize? Does it have remission? The metaphor implies yes, but the reality is more complicated.)

The causal rule. “Damage at a threshold triggers an irreversible state transition.” Now the source domain’s vocabulary has disappeared entirely. You can’t tell whether this came from botany, metallurgy, or nuclear physics; it’s a pure structural relationship. It’s abstract enough to travel across any domain boundary, but concrete enough to generate specific ideas when you apply it to a new context.

The philosophical principle. “Systems under constraint reach equilibria.” This is over-compressed. It’s true of everything: thermodynamics, economics, psychology, traffic. Precisely because it applies everywhere, it generates nothing specific anywhere. It’s a tautology dressed as insight.

The intermediate level, the bare causal rule, is the only one that is simultaneously abstract enough to leave the source domain and concrete enough to land in the target domain and generate something actionable.

Too little compression, and you import baggage. Too much, and you import nothing.

Useful creativity, it turns out, is a compression problem, and the sweet spot is a discrete level rather than a point on a continuous dial.

What This Means

These two findings already change how you use an LLM for creative work.

If you want ideas that are immediately actionable, direct the model to extract single causal rules from distant domains, stripped of all source vocabulary. Force compression to the intermediate level. If you can still tell which domain a rule came from, it’s not compressed enough. If you can’t imagine applying it to anything specific, it’s compressed too far.

But when I started applying this systematically, I ran into something I didn’t expect. I had a catalog of beautifully compressed rules. Each one mapped onto the target problem with uncomfortable precision. And yet I still couldn’t turn any of them into a solution. Understanding how a problem works turned out to be a completely different operation from knowing what to do about it. That gap, and what fills it, is what Part II is about.

Gabriel Mailhot is a Senior Subject Matter Expert at Desjardins, where he works on development practices, architecture, and complex organizational problems spanning technology, process, and governance. His 2003–2010 research focused on creative systems and metaphor-based retrieval. He publishes at scripting.pro.

The Limitation I Didn’t See

By the end of Part I, I could reliably extract clean, compressed causal rules from distant domains. Rules like “accumulation under pressure is physics, not a choice” or “the distinction that matters is controlled versus uncontrolled.” Each one a precise diagnosis of how a problem works.

But that’s exactly what they were: diagnoses, X-rays of the pathology rather than solutions.

I spent a while assuming the solution would naturally follow from understanding the cause. It doesn’t. Understanding that nitrogen dissolves in blood under pressure doesn’t tell you to invent decompression stops. The leap from “here’s how the problem works” to “here’s how to break the mechanism” requires a separate act, one that is creative in a way the diagnosis is not.

When I systematized the method by asking an LLM to extract compressed causal rules from many distant domains at once, I could see this limitation clearly. The results were clean and powerful:

“Functional-but-carrying agents propagate the problem because there is no incentive to detect them.” (from epidemiology)

“Repeated sub-critical stress accumulates invisibly until failure occurs without proportional warning.” (from metallurgy)

“Control by outcome alone makes it impossible to distinguish systemic success from accidental success.” (from fermentation science)

Each maps onto technical debt with uncomfortable precision. The epidemiological rule describes how “working” code that carries hidden debt propagates undetected. The metallurgical rule describes how small daily compromises accumulate into sudden, catastrophic failures. The fermentation rule describes how testing only the final product hides whether the process was sound. Yet for all their diagnostic value, not one of them tells you what to do.

The Missing Operation: Inversion

The creative act that turns a diagnosis into a solution is inversion: deliberately breaking the causal chain at a chosen point.

In its simplest form, a diagnosis says “A causes B, which causes C.” Inversion asks what happens if I prevent A from causing B, or if I let B happen but prevent it from causing C. Each break point produces a different family of solutions, often of completely different nature.

In practice this is less obvious than it sounds, because a single causal chain can be broken at multiple points, and the choice of where to break determines not just the solution but its entire character.

Take the epidemiological rule: “functional-but-carrying agents propagate the problem because there is no incentive to detect them.” There are three links in this chain, and each one is a potential target:

Break at link 1: make the carrying agents non-functional. In software terms, design code so that debt-carrying modules stop working after a defined period. The debt itself triggers a forced replacement. This is a radical, architectural solution that changes the infrastructure.

Break at link 2: create incentives to detect, even when everything works. In software terms, reward teams for discovering latent debt rather than for delivering features. Don’t wait for symptoms; actively hunt for carriers. This is an organizational, incentive-based solution that changes the reward structure.

Break at link 3: accept propagation, but make its pathways visible. In software terms, don’t try to prevent debt from spreading; instead, instrument the codebase so that every propagation path is traced in real time. When something breaks, you know exactly which “carrier” transmitted the problem. This is an observational solution that changes what you can see.

One diagnosis yields three completely different solutions. The choice of where to break the chain is itself a creative decision that depends on the context, the constraints, and the appetite for disruption.

This is the operation that was missing from my original method. Extraction gives you a diagnosis. Inversion gives you families of possible solutions. But I wasn’t done, because the most powerful ideas I encountered weren’t single inversions. They were combinations.

Fourth Finding: Novelty Is in the Composition

Here is the central reversal of everything I thought I knew about creativity.

The entire creativity literature since Mednick (1962) has focused on the transfer as the source of novelty, on the assumption that the more distant the connection between domains, the more creative the idea. My own earlier research operated on this assumption.

What experimentation with LLMs at scale revealed is that distance is only a necessary condition. It ensures that the pieces you extract are sufficiently different from one another. But the novelty, the actual Eureka, doesn’t reside in any single extraction or any single inversion. It resides in the composition: the moment you assemble inversions from multiple, unrelated domains into a recipe that is specific to the target problem. The individual pieces aren’t surprising; the assembly is what no one has seen before.

In my own work using structural analogy as a diagnostic tool for organizations, I observe exactly this. When I import rules from biology, law, and neuroeconomics to analyze an organizational dysfunction, no individual rule surprises the domain expert. A biologist knows the first. A lawyer knows the second. An economist knows the third. What produces genuine insight is the combination: the specific way these rules interact when applied simultaneously to a context none of them were designed for.

Let me make this concrete with the technical debt example. Take two inversions from different rules:

From the epidemiological rule, break at link 2: create an incentive to detect even when everything functions normally.

From the fermentation rule: the relevant distinction is not presence versus absence, it’s controlled versus uncontrolled.

Now assemble them. You get something that, as far as I can tell, doesn’t exist in the current IT literature: reproductibility as the metric of code health.

A module is healthy not because it works, nor because its tests pass, but because two independent developers, starting from the same specification, would produce structurally similar code. If they wouldn’t (if the result depends heavily on who wrote it), then the module is a functional-but-carrying agent. It works, but its construction process was uncontrolled. It passed the outcome test while failing the process test.

Think about what this does. You’re no longer looking for bugs or measuring test coverage. You’re measuring whether the process that produced the code was controlled or accidental. The fermentation frame redefines what you’re looking for. The epidemiological inversion tells you when to look: precisely when everything appears fine.

Neither inversion alone produces this idea. The fermentation rule alone gives you a philosophical reframe, interesting but abstract. The epidemiological inversion alone gives you a detection strategy, useful but directionless. The invention is the recipe: reproductibility as a metric, triggered by the absence of alarm signals, designed to catch uncontrolled process masquerading as healthy output. It emerged from the composition, not from either piece.

Fifth Finding: The Solution Exposes the Context

There’s one more thing I got wrong initially, and it matters.

My first instinct, after developing the model, was to add a filtering step: take the composed ideas and check which ones are compatible with the target context. “This wouldn’t work here because of regulatory constraint X; eliminate it.” “This contradicts our governance model; eliminate it.” Standard feasibility analysis.

That instinct is a trap. A sufficiently compelling solution doesn’t get filtered by the context. It exposes the context. When a good idea collides with an organizational, regulatory, or cultural constraint, there are two readings: either the idea is unrealistic, or the constraint itself deserves to be questioned.

The most transformative Eureka is not “this solution works in my world” but “this solution is so compelling that I’m willing to change my world to make it work.” The friction between the idea and the context is a diagnostic of which constraints are actually serving you and which ones have outlived their purpose, not a bug in the creative process.

This has happened before at industry scale. Agile was not “applicable” in the world of waterfall. It didn’t fit the existing governance, contracts, or PMO structure. But it exposed that waterfall rested on a false assumption: that you can fully specify a system before building it. DevOps was not “applicable” in organizations where development and operations were separate silos with separate incentives. But it exposed that the separation itself was the root cause of deployment failures. In both cases, the idea didn’t adapt to the context. The context restructured itself around the idea, because the idea was compelling enough to justify the cost of change. Entire organizations rewrote their processes, their team structures, their governance models. They did so not because a consultant told them to, but because the solution showed the constraints to be obstacles rather than laws.

The idea didn’t adapt to the context. The context restructured itself around the idea.

The same mechanism operates at smaller scales. In my work at a major financial institution regulated by federal and provincial authorities, I’ve seen a creative solution to a development process problem run directly into a regulatory rule that everyone assumed was immutable. But the solution was transformative enough to create the will to engage the regulator in a conversation about whether that rule was still serving its original purpose. Instead of the constraint killing the idea, the idea questioned the constraint. And the list of constraints exposed became a deliverable as valuable as the solution itself.

The Complete Model

Creativity, then, is not a single operation. It is a sequence of three, which I’ll call the Extraction-Inversion-Composition framework (EIC):

Extraction. Compress distant domains into minimal causal rules at the right discrete level: abstract enough to be portable, concrete enough to be operational. This is the preparatory step. LLMs do it extraordinarily well.

Inversion. Identify where each causal chain can be broken. Each breakpoint opens a family of solutions of a different nature. The choice of breakpoint is context-dependent and already involves judgment, but an LLM can enumerate the options systematically.

Composition. Assemble inversions from different rules into a coherent recipe for the target domain. This is where genuine novelty emerges: in the combinatorics of the assembly, not in the individual pieces.

The framework rests on three claims, each of which challenges a different assumption in the existing creativity literature:

First, the causal rule (not the metaphor, the analogy, or the concept) is the correct unit of creative transfer. It is the only level of compression that is simultaneously portable and operational.

Second, inversion is the missing operation between diagnosis and solution. The creativity literature treats the transfer as the creative act, but the transfer only produces a diagnosis. Inversion produces families of solutions. Without it, you have insight without agency.

Third, novelty emerges from composition, not from transfer. The distance between domains ensures that the pieces are different. But the invention is in the assembly, in the specific combination of inversions that no one has tried before.

Each composed proposal is then judged on three levels of strength:

Logical strength: does the inversion actually break the causal chain? This is a structural coherence judgment. An LLM handles it well.

Transformative strength: is the solution compelling enough to justify questioning the constraints that stand in its way? An LLM can partially assess this; it can evaluate whether a proposal is radical and explain why.

Political strength: will the actual actors have the will and power to modify the exposed constraints? This is a situated judgment, anchored in knowledge of power dynamics, culture, and institutional history. Here, and only here, the human is irreplaceable.

What Comes Next

The EIC framework now feels complete on paper. But a model is only as good as its results in practice.

In Part III, I turn the framework into a concrete instrument (a carefully constrained prompt) and test it cold on three different LLMs with the same technical debt problem.

The results reveal something unexpected about the creative signatures of different models, about the limits of prompts as specifications, and about where human judgment remains irreplaceable.

Gabriel Mailhot is a Senior Subject Matter Expert at Desjardins, where he works on development practices, architecture, and complex organizational problems spanning technology, process, and governance. His 2003–2010 research focused on creative systems and metaphor-based retrieval. He publishes at scripting.pro.

I have watched a pattern repeat itself across organizations too many times to believe it is accidental. A technically competent person does careful work, raises legitimate concerns about architectural shortcuts, pushes back on timelines that will generate problems downstream. They are right. Their concerns are documented. And they are quietly sidelined while the project proceeds, delivers on schedule, and the manager who overrode their concerns is promoted.

The promoted manager is not incompetent. In many cases, they are genuinely skilled at something. But skilled at what, exactly? And why does that skill, rather than technical judgment, determine who advances?

This article is an attempt to answer that question rigorously. The answer implicates organizational dynamics, but also the educational institutions that train managers, the governance models that structure IT investment, and the reform movements, Agile among them, that have tried and failed to change the outcome. The system is not broken; it is working exactly as its underlying structure dictates. In 2007, Robert Schaefer, writing in ACM SIGSOFT Software Engineering Notes, asked the same question in more general terms: why do large complex systems so often work against their declared goals? His answer pointed directly at organizational governance, specifically at what he termed the Authoritarian Hierarchy of Control. What follows builds on that diagnosis and extends it into the specific dynamics of IT investment, selection, and reform.

Technical debt is not a failure of discipline. It is a system of incentives working exactly as designed.

The Incentive Structure That Generates Technical Debt

Large organizations govern IT investment through annual budget cycles and delivery metrics. A project is funded for a year, expected to deliver defined functionality by a defined date, and evaluated on whether it did. This structure imposes a specific temporal resolution on decision-making: what matters is what is visible within twelve months.

Architectural quality does not operate on a twelve-month horizon. The decisions that determine whether a system can be evolved cheaply in three years (modularity, decoupling, reversibility, testability) produce no visible output in the current budget cycle. By the logic of the measurement instrument, they are invisible.

Every technical role in a delivery team, whether developer, tech lead, or architect, is in the business of reducing the cost of being wrong. Good design does not predict the future; it ensures that when the system must evolve in ways no one anticipated, the cost of that evolution is bounded rather than catastrophic. But this value cannot be captured by a metric that closes at year end. It has no line in the budget and generates no deliverable.

The project manager who allocates time and money to architectural quality is making an investment whose return will appear in someone else’s budget cycle, under someone else’s name. The project manager who defers that investment delivers on time and on budget, and receives the recognition that comes with it. That is a sensible response to the incentive structure rather than a failure of individual judgment.

Schaefer named this mechanism risk shifting: when direct action to reduce a risk is judged too costly, the less costly tactic is to transfer that risk burden to others. If this transfer can proceed without immediate observation, the conflict it generates will simply be deferred. Annual budget cycles are, structurally, a machine for making risk shifting invisible. The cost lands in a future cycle, under a different manager, attributed to different causes.

Game theory offers a complementary framing. In a prisoner’s dilemma, two actors each face a choice between cooperation, which produces the best collective outcome, and defection, which protects individual interest at collective cost. When both actors reason independently and arrive at the locally coherent choice, both defect. The problem is not irrationality but the fact that individually sensible behavior produces collectively irrational outcomes. The parallel with IT governance runs deep enough to carry predictive weight. Every project manager, every budget cycle, every delivery team faces the same calculus. When every actor optimizes for their own position, the collective result is a technology estate that grows progressively more rigid, more expensive to evolve, and less capable of supporting the business it was built to serve. Nobody made a bad decision; everyone made a good one, given the game they were playing.

The Cascade: From Deferral to Dissimulation to Manipulation

Technical debt is the first and most visible symptom of this structure. But the same logic operates at every level of the organizational hierarchy, producing a cascade of increasingly consequential behaviors.

At the project level, the actor defers architectural quality. This is risk shifting in its most direct form: a cost that is real but diffuse, transferred to a future that belongs to someone else.

As the project encounters difficulty (deferred costs begin to materialize, timelines slip, complexity compounds), the actor at the next level faces a different version of the same calculus. Delivering bad news upward is costly. It damages reputation, invites scrutiny, and moves the problem from the future into the present. Filtering that information, presenting progress in the most favorable available framing, managing perception rather than reality: this is individually sensible for exactly the same reason that deferring architectural debt is individually sensible. The cost is real but diffuse, while the benefit is immediate and personal.

Schaefer formalized this in what he called the Liars’ Axioms. A liar, his term for any actor who strategically manages information to protect their position, suppresses, manufactures, or denies evidence to align observers’ judgments with their own goals. The intent of the lie, he observed, may simply be to stall for time. If the delay runs long enough, the consequences will have had greater impact on the world than the uncovering of the lie. Those who discover it afterward (auditors, commissions of inquiry, historians) matter only to a small audience. The actor has long since moved on.

The calculus that defers architectural quality also defers information. Technical debt accumulates at the bottom of the hierarchy; informational debt accumulates at the top. The mechanism is identical, and the consequences, when they finally surface, are proportional to how long the deferral has continued.

This is a structural observation, not a cynical one about human nature: it describes what happens when the same incentive pattern operates at multiple levels of a hierarchy simultaneously. Each level protects its position by deferring its costs to the next. The progression from deferral to dissimulation is continuous, not discrete. There is no moment at which a sensible actor becomes a dishonest one, only a gradual drift toward behaviors that the structure rewards and away from behaviors that it penalizes.

What Gets Selected

Organizations do not select randomly among the people available to them. Over time, through promotion decisions made by individuals operating within the incentive structure described above, they exercise a form of natural selection. The behaviors that advance are the behaviors that the structure rewards. The behaviors that are penalized (transparency, technical rigor, willingness to deliver unwelcome news) become associated with career stagnation.

Schaefer described this dynamic in terms of the Authoritarian Hierarchy of Control: organizations structured around command and control systematically suppress independent judgment in favor of loyalty and conformance, rewarding those who display alignment with authority over those who bring accurate but unwelcome observations. The result, he noted, is that the quality of products and services tends to be driven to the lowest level of mediocrity the market will bear, and that responsibility for poor performance will be attributed to individual contributors, never to the structure that produced it.

This selection process has a name for each of its outcomes. HEC Montréal, one of Quebec’s leading business schools, administers a political skill assessment that classifies managers into five profiles: Machiavellian, Strategist, Survivor, Good Soldier, and Sacrificial Lamb. The Machiavellian profile, described as driven by a strong thirst for power, politically motivated behavior, and the belief that ends justify means, is listed as having two strengths: highly effective, and achieves objectives. Ethical risks appear as a limitation, alongside friction and mistrust. The profile is presented not as a pathology but as a high-performance option with known trade-offs.

At the other end of the scale, the Sacrificial Lamb, described as believing that merit is sufficient, unable to decode political dynamics, potentially becoming a victim of the system, is listed as having three strengths: integrity, authenticity, and focus on work well done. Its risks include marginalization and incomprehension of real organizational dynamics.

What Gets Trained

HEC is not inventing the reality it describes. It is documenting what it observes in organizations, and then teaching it. HEC is not alone in this. The framing of political skill as a legitimate and measurable managerial competency has been widely adopted across business schools. What was once informal knowledge (how to handle power, manage perception, read the room) has been systematized into curriculum. By naming these profiles, measuring them, and transmitting them to the next generation of managers as legitimate knowledge about how organizations actually work, business education completes a feedback loop that Schaefer identified: the selection pressure that organizations exert is validated, codified, and taught as professional competence.

The Machiavellian who advances in a large organization is not a deviation from the system. They are its product: shaped by the incentive structure, selected by promotion decisions, and trained by the curriculum.

There is an irony here. Schaefer’s 2007 paper received only three citations in the seventeen years following its publication, not because it was wrong, but because the professional and academic structures of software engineering had no established category for an argument that located the root cause of system failure in governance, power, and the suppression of independent judgment. The diagnosis arrived before the vocabulary existed to receive it. That, too, is the system working as designed.

There is a question this article does not pretend to resolve: does the organization produce this selection pressure, which the educational institution then documents? Or does the educational institution legitimate and amplify a pressure that would otherwise remain informal? The honest answer is that causality runs in both directions. The organization and its educational infrastructure have co-evolved into a system that is now very difficult to disturb from within.

Why Agile Was Structurally Condemned

The Agile movement emerged in the early 2000s as a direct response to the dysfunctions described above. Its founding documents, particularly the Agile Manifesto, called for transparency over performance theater, genuine collaboration over contract negotiation, and working software over comprehensive documentation. It identified, correctly, that waterfall project dynamics generated exactly the kind of rigidity and deferral that technical debt represents.

The adoption of Agile in large organizations over the following two decades produced a result that is by now well documented: the vocabulary was adopted; the underlying dynamics were not changed. Sprints became fixed-scope, fixed-date delivery commitments. Velocity became a production output target. Retrospectives became compliance rituals. The Agile transformation was managed by the same people operating within the same incentive structure, producing the same behaviors under a new name.

Agile did not transform governance. Governance transformed Agile.

The standard explanation for this failure centers on the frozen middle: middle managers whose positional authority depends on the existing governance model, who absorbed the pressure for change without transmitting it upward. This explanation is correct but incomplete.

The deeper explanation is that Agile, authentically practiced, requires a specific type of organizational actor. It requires transparency about progress and problems, the willingness to deliver unwelcome news early, and collaboration that is genuine rather than performed. These are the behaviors associated with the Good Soldier and the Sacrificial Lamb in the HEC taxonomy, the profiles described as loyal, competent, and appreciated for their work, but also naive, vulnerable, and easily marginalized in competitive environments.

The Strategist and the Machiavellian, the profiles that large organizations have systematically selected and elevated, are precisely those for whom authentic Agile practice is not a viable strategy. Transparency exposes their deferral. Early bad news undermines their narrative management. Genuine collaboration limits their control of information flows. For these actors, adopting the vocabulary of Agile while hollowing out its content is political competence applied to organizational survival, not resistance to change.

Schaefer anticipated this failure with what he called the Process Paradox: the more a process is commanded, the less independent judgment is permitted, and when something goes wrong (as it inevitably will), the worse the consequences. Agile, commanded from above within an authoritarian hierarchy, does not produce agility. It produces a rigid process that eliminates independent judgment while preserving the appearance of flexibility. The name changes; the structure does not.

Agile did not fail because it was poorly designed or poorly implemented. It failed because it was introduced into organizations that had spent decades selecting against the human dispositions it required. The frozen middle did not block Agile. The selection process that produced the frozen middle made authentic Agile adoption structurally improbable before the first sprint planning meeting.

The Agile coach role, now a standard position in many large organizations, crystallizes this contradiction with particular clarity. It is an officially sanctioned mandate to transform a culture that the incentive structure actively reproduces. The role signals commitment to change without requiring it. In Schaefer’s terms, it is a display directed outward: the organization can point to its coaches as evidence of transformation while the governance model remains untouched. And it produces a specific collateral damage that rarely appears in any measurement. Developers who glimpse a different way of working, who are shown what genuine collaboration and technical quality could look like, find themselves unable to practice any of it within the real constraints of the system. The choices available to them are narrow: exit the organization, remain with a chronic frustration that compounds over time, or disengage quietly and accept the distance between what they know is possible and what the system will permit. The organization does not notice this damage, because what it measures is sprint velocity.

The Normalization of Failure

One final mechanism completes the loop and explains why the system persists without generating the corrective pressure one might expect.

Large IT project failure is not an occasional anomaly. The Standish Group has tracked IT project outcomes for decades; their data consistently show that only around 30% of IT projects are considered fully successful, a figure that drops further for large-scale initiatives, with roughly half of all projects challenged and nearly one in five outright cancelled or abandoned. The pattern is stable across years and geographies. Yet when a specific project fails (costs overrun, the system delivered is less capable than the system it replaced, timelines stretch beyond any defensible estimate), the failure is almost never attributed to the incentive structure that generated it. It is attributed to poor management, to vendor problems, to scope creep, to technical complexity. The postmortem identifies individuals and decisions. It does not identify the game those individuals were playing or the rules that made their decisions predictable.

Schaefer identified this as a structural property of hierarchical governance: the source of all faults will ultimately be whatever legitimate authority claims it is, whether or not such claims are true. Accountability flows downward to individual contributors. The structure that produced the conditions for failure is never the finding. To make it the finding would require those conducting the examination to indict what elevated them.

This misattribution operates at two scales simultaneously, and the two reinforce each other. At the organizational level, the postmortem blames individuals and decisions, protecting the incentive structure. But the same dynamic operates earlier, at the level of the project itself. When timelines slip, as they inevitably do when architectural shortcuts accumulate and deferred complexity begins to surface, the reading is almost always that the technical team is slow, that estimates were poor, that execution was lacking. This interpretation is structurally necessary. If the delay were attributed to its actual cause, the decisions that sacrificed quality to meet an earlier date, accountability would travel upward, toward the people who made those decisions. Attributing it to technical competence severs that causal chain cleanly. The practitioners who lived through the accumulation carry the reputational cost of a problem they did not create. They raised concerns that were overridden and worked inside constraints they did not choose. Some internalize the blame, wondering whether they could have done better, whether they should have pushed back harder, whether they lack something. The structure produces this doubt as a feature, not a side effect.

The system then produces a second deferral: the failed system is declared legacy (too old, too rigid, too expensive to maintain) and replaced. The replacement project is governed by the same annual budget cycle, managed by the same profiles, and evaluated by the same metrics. The cycle restarts. The debt that accumulated in the previous system is not examined as evidence of a structural problem but treated as the natural aging of technology, as entropy rather than design.

Software does not age. The observation is Schaefer’s, made in passing but worth dwelling on: software stays fixed in time. What changes is the context around it: the business it serves, the expectations placed on it, the people who maintain it. When an organization declares a system legacy, it is not describing a property of the technology. It is describing the accumulated consequences of decisions that were never attributed to their real cause.

A system whose failures are consistently misattributed, whose lessons are never learned, and whose replacements reproduce its conditions is not a system that is trying and failing to improve. It is a system that is successfully maintaining itself.

A Final Observation

This analysis is not offered as a counsel of despair to the practitioners who have spent years pushing against these dynamics, advocating for better design, raising concerns that were overridden, watching careful work be undone by delivery pressure. The frustration those practitioners feel does not mean they misunderstood something. It means they understood something correctly and found themselves playing a game whose rules made their understanding a competitive disadvantage.

Schaefer ended his 2007 paper with a question that remains unanswered: what are the limits of the will? He was asking whether individuals operating within deeply coercive systems retain any meaningful capacity for independent judgment. The question is not rhetorical. History suggests that the levers which have actually changed systems of this kind are rarely internal. They tend to be external: competitive pressure severe enough to punish rigidity, regulatory constraint that removes certain decisions from the negotiating table, or the cumulative weight of people who refuse to play the game and vote with their feet. Departures concentrate in exactly the profiles the system needs most, the Good Soldiers and Sacrificial Lambs who carry the technical judgment the Machiavellians cannot replace. Whistleblowing, when it occurs, is almost always punished before it produces change. Smaller organizations built deliberately outside the AHoC model sometimes succeed precisely because they have not yet had time to reproduce its conditions. None of these are solutions. They are the terrain on which change, when it happens, actually occurs.

The technical argument for architectural quality (modularity, decoupling, reversibility) remains correct. The problem is that it is a technical argument being made in an environment where the selection pressure runs in the opposite direction, and technical correctness has never overridden incentive structure.

What this analysis suggests is that the relevant question is not how to make the case for better architecture more persuasively. It is a prior question: in an organization that has systematically selected for political behavior, trained its managers in its legitimacy, and structured its governance to make individual rationality and collective interest incompatible, from what position, and with what leverage, is change actually possible?

That question does not have a technical answer.

Gabriel Mailhot
Software Architect and Software Engineering Quality Advisor
Researcher (2005–2010): Image Processing & Creative AI
Business Expert / Domain Expert (2010–2026): Software Architecture, Software Development and Quality Practices

Reference

Schaefer, R. (2007). The Grand Theory of Everything: What Man-Made Systems Are, and Why They Fail. ACM SIGSOFT Software Engineering Notes, 32(4).