I have watched a pattern repeat itself across organizations too many times to believe it is accidental. A technically competent person does careful work, raises legitimate concerns about architectural shortcuts, pushes back on timelines that will generate problems downstream. They are right. Their concerns are documented. And they are quietly sidelined while the project proceeds, delivers on schedule, and the manager who overrode their concerns is promoted.

The promoted manager is not incompetent. In many cases, they are genuinely skilled at something. But skilled at what, exactly? And why does that skill, rather than technical judgment, determine who advances?

This article is an attempt to answer that question rigorously. The answer implicates organizational dynamics, but also the educational institutions that train managers, the governance models that structure IT investment, and the reform movements, Agile among them, that have tried and failed to change the outcome. The system is not broken; it is working exactly as its underlying structure dictates. In 2007, Robert Schaefer, writing in ACM SIGSOFT Software Engineering Notes, asked the same question in more general terms: why do large complex systems so often work against their declared goals? His answer pointed directly at organizational governance, specifically at what he termed the Authoritarian Hierarchy of Control. What follows builds on that diagnosis and extends it into the specific dynamics of IT investment, selection, and reform.

Technical debt is not a failure of discipline. It is a system of incentives working exactly as designed.

The Incentive Structure That Generates Technical Debt

Large organizations govern IT investment through annual budget cycles and delivery metrics. A project is funded for a year, expected to deliver defined functionality by a defined date, and evaluated on whether it did. This structure imposes a specific temporal resolution on decision-making: what matters is what is visible within twelve months.

Architectural quality does not operate on a twelve-month horizon. The decisions that determine whether a system can be evolved cheaply in three years (modularity, decoupling, reversibility, testability) produce no visible output in the current budget cycle. By the logic of the measurement instrument, they are invisible.

Every technical role in a delivery team, whether developer, tech lead, or architect, is in the business of reducing the cost of being wrong. Good design does not predict the future; it ensures that when the system must evolve in ways no one anticipated, the cost of that evolution is bounded rather than catastrophic. But this value cannot be captured by a metric that closes at year end. It has no line in the budget and generates no deliverable.

The project manager who allocates time and money to architectural quality is making an investment whose return will appear in someone else’s budget cycle, under someone else’s name. The project manager who defers that investment delivers on time and on budget, and receives the recognition that comes with it. That is a sensible response to the incentive structure rather than a failure of individual judgment.

Schaefer named this mechanism risk shifting: when direct action to reduce a risk is judged too costly, the less costly tactic is to transfer that risk burden to others. If this transfer can proceed without immediate observation, the conflict it generates will simply be deferred. Annual budget cycles are, structurally, a machine for making risk shifting invisible. The cost lands in a future cycle, under a different manager, attributed to different causes.

Game theory offers a complementary framing. In a prisoner’s dilemma, two actors each face a choice between cooperation, which produces the best collective outcome, and defection, which protects individual interest at collective cost. When both actors reason independently and arrive at the locally coherent choice, both defect. The problem is not irrationality but the fact that individually sensible behavior produces collectively irrational outcomes. The parallel with IT governance runs deep enough to carry predictive weight. Every project manager, every budget cycle, every delivery team faces the same calculus. When every actor optimizes for their own position, the collective result is a technology estate that grows progressively more rigid, more expensive to evolve, and less capable of supporting the business it was built to serve. Nobody made a bad decision; everyone made a good one, given the game they were playing.

The Cascade: From Deferral to Dissimulation to Manipulation

Technical debt is the first and most visible symptom of this structure. But the same logic operates at every level of the organizational hierarchy, producing a cascade of increasingly consequential behaviors.

At the project level, the actor defers architectural quality. This is risk shifting in its most direct form: a cost that is real but diffuse, transferred to a future that belongs to someone else.

As the project encounters difficulty (deferred costs begin to materialize, timelines slip, complexity compounds), the actor at the next level faces a different version of the same calculus. Delivering bad news upward is costly. It damages reputation, invites scrutiny, and moves the problem from the future into the present. Filtering that information, presenting progress in the most favorable available framing, managing perception rather than reality: this is individually sensible for exactly the same reason that deferring architectural debt is individually sensible. The cost is real but diffuse, while the benefit is immediate and personal.

Schaefer formalized this in what he called the Liars’ Axioms. A liar, his term for any actor who strategically manages information to protect their position, suppresses, manufactures, or denies evidence to align observers’ judgments with their own goals. The intent of the lie, he observed, may simply be to stall for time. If the delay runs long enough, the consequences will have had greater impact on the world than the uncovering of the lie. Those who discover it afterward (auditors, commissions of inquiry, historians) matter only to a small audience. The actor has long since moved on.

The calculus that defers architectural quality also defers information. Technical debt accumulates at the bottom of the hierarchy; informational debt accumulates at the top. The mechanism is identical, and the consequences, when they finally surface, are proportional to how long the deferral has continued.

This is a structural observation, not a cynical one about human nature: it describes what happens when the same incentive pattern operates at multiple levels of a hierarchy simultaneously. Each level protects its position by deferring its costs to the next. The progression from deferral to dissimulation is continuous, not discrete. There is no moment at which a sensible actor becomes a dishonest one, only a gradual drift toward behaviors that the structure rewards and away from behaviors that it penalizes.

What Gets Selected

Organizations do not select randomly among the people available to them. Over time, through promotion decisions made by individuals operating within the incentive structure described above, they exercise a form of natural selection. The behaviors that advance are the behaviors that the structure rewards. The behaviors that are penalized (transparency, technical rigor, willingness to deliver unwelcome news) become associated with career stagnation.

Schaefer described this dynamic in terms of the Authoritarian Hierarchy of Control: organizations structured around command and control systematically suppress independent judgment in favor of loyalty and conformance, rewarding those who display alignment with authority over those who bring accurate but unwelcome observations. The result, he noted, is that the quality of products and services tends to be driven to the lowest level of mediocrity the market will bear, and that responsibility for poor performance will be attributed to individual contributors, never to the structure that produced it.

This selection process has a name for each of its outcomes. HEC Montréal, one of Quebec’s leading business schools, administers a political skill assessment that classifies managers into five profiles: Machiavellian, Strategist, Survivor, Good Soldier, and Sacrificial Lamb. The Machiavellian profile, described as driven by a strong thirst for power, politically motivated behavior, and the belief that ends justify means, is listed as having two strengths: highly effective, and achieves objectives. Ethical risks appear as a limitation, alongside friction and mistrust. The profile is presented not as a pathology but as a high-performance option with known trade-offs.

At the other end of the scale, the Sacrificial Lamb, described as believing that merit is sufficient, unable to decode political dynamics, potentially becoming a victim of the system, is listed as having three strengths: integrity, authenticity, and focus on work well done. Its risks include marginalization and incomprehension of real organizational dynamics.

What Gets Trained

HEC is not inventing the reality it describes. It is documenting what it observes in organizations, and then teaching it. HEC is not alone in this. The framing of political skill as a legitimate and measurable managerial competency has been widely adopted across business schools. What was once informal knowledge (how to handle power, manage perception, read the room) has been systematized into curriculum. By naming these profiles, measuring them, and transmitting them to the next generation of managers as legitimate knowledge about how organizations actually work, business education completes a feedback loop that Schaefer identified: the selection pressure that organizations exert is validated, codified, and taught as professional competence.

The Machiavellian who advances in a large organization is not a deviation from the system. They are its product: shaped by the incentive structure, selected by promotion decisions, and trained by the curriculum.

There is an irony here. Schaefer’s 2007 paper received only three citations in the seventeen years following its publication, not because it was wrong, but because the professional and academic structures of software engineering had no established category for an argument that located the root cause of system failure in governance, power, and the suppression of independent judgment. The diagnosis arrived before the vocabulary existed to receive it. That, too, is the system working as designed.

There is a question this article does not pretend to resolve: does the organization produce this selection pressure, which the educational institution then documents? Or does the educational institution legitimate and amplify a pressure that would otherwise remain informal? The honest answer is that causality runs in both directions. The organization and its educational infrastructure have co-evolved into a system that is now very difficult to disturb from within.

Why Agile Was Structurally Condemned

The Agile movement emerged in the early 2000s as a direct response to the dysfunctions described above. Its founding documents, particularly the Agile Manifesto, called for transparency over performance theater, genuine collaboration over contract negotiation, and working software over comprehensive documentation. It identified, correctly, that waterfall project dynamics generated exactly the kind of rigidity and deferral that technical debt represents.

The adoption of Agile in large organizations over the following two decades produced a result that is by now well documented: the vocabulary was adopted; the underlying dynamics were not changed. Sprints became fixed-scope, fixed-date delivery commitments. Velocity became a production output target. Retrospectives became compliance rituals. The Agile transformation was managed by the same people operating within the same incentive structure, producing the same behaviors under a new name.

Agile did not transform governance. Governance transformed Agile.

The standard explanation for this failure centers on the frozen middle: middle managers whose positional authority depends on the existing governance model, who absorbed the pressure for change without transmitting it upward. This explanation is correct but incomplete.

The deeper explanation is that Agile, authentically practiced, requires a specific type of organizational actor. It requires transparency about progress and problems, the willingness to deliver unwelcome news early, and collaboration that is genuine rather than performed. These are the behaviors associated with the Good Soldier and the Sacrificial Lamb in the HEC taxonomy, the profiles described as loyal, competent, and appreciated for their work, but also naive, vulnerable, and easily marginalized in competitive environments.

The Strategist and the Machiavellian, the profiles that large organizations have systematically selected and elevated, are precisely those for whom authentic Agile practice is not a viable strategy. Transparency exposes their deferral. Early bad news undermines their narrative management. Genuine collaboration limits their control of information flows. For these actors, adopting the vocabulary of Agile while hollowing out its content is political competence applied to organizational survival, not resistance to change.

Schaefer anticipated this failure with what he called the Process Paradox: the more a process is commanded, the less independent judgment is permitted, and when something goes wrong (as it inevitably will), the worse the consequences. Agile, commanded from above within an authoritarian hierarchy, does not produce agility. It produces a rigid process that eliminates independent judgment while preserving the appearance of flexibility. The name changes; the structure does not.

Agile did not fail because it was poorly designed or poorly implemented. It failed because it was introduced into organizations that had spent decades selecting against the human dispositions it required. The frozen middle did not block Agile. The selection process that produced the frozen middle made authentic Agile adoption structurally improbable before the first sprint planning meeting.

The Agile coach role, now a standard position in many large organizations, crystallizes this contradiction with particular clarity. It is an officially sanctioned mandate to transform a culture that the incentive structure actively reproduces. The role signals commitment to change without requiring it. In Schaefer’s terms, it is a display directed outward: the organization can point to its coaches as evidence of transformation while the governance model remains untouched. And it produces a specific collateral damage that rarely appears in any measurement. Developers who glimpse a different way of working, who are shown what genuine collaboration and technical quality could look like, find themselves unable to practice any of it within the real constraints of the system. The choices available to them are narrow: exit the organization, remain with a chronic frustration that compounds over time, or disengage quietly and accept the distance between what they know is possible and what the system will permit. The organization does not notice this damage, because what it measures is sprint velocity.

The Normalization of Failure

One final mechanism completes the loop and explains why the system persists without generating the corrective pressure one might expect.

Large IT project failure is not an occasional anomaly. The Standish Group has tracked IT project outcomes for decades; their data consistently show that only around 30% of IT projects are considered fully successful, a figure that drops further for large-scale initiatives, with roughly half of all projects challenged and nearly one in five outright cancelled or abandoned. The pattern is stable across years and geographies. Yet when a specific project fails (costs overrun, the system delivered is less capable than the system it replaced, timelines stretch beyond any defensible estimate), the failure is almost never attributed to the incentive structure that generated it. It is attributed to poor management, to vendor problems, to scope creep, to technical complexity. The postmortem identifies individuals and decisions. It does not identify the game those individuals were playing or the rules that made their decisions predictable.

Schaefer identified this as a structural property of hierarchical governance: the source of all faults will ultimately be whatever legitimate authority claims it is, whether or not such claims are true. Accountability flows downward to individual contributors. The structure that produced the conditions for failure is never the finding. To make it the finding would require those conducting the examination to indict what elevated them.

This misattribution operates at two scales simultaneously, and the two reinforce each other. At the organizational level, the postmortem blames individuals and decisions, protecting the incentive structure. But the same dynamic operates earlier, at the level of the project itself. When timelines slip, as they inevitably do when architectural shortcuts accumulate and deferred complexity begins to surface, the reading is almost always that the technical team is slow, that estimates were poor, that execution was lacking. This interpretation is structurally necessary. If the delay were attributed to its actual cause, the decisions that sacrificed quality to meet an earlier date, accountability would travel upward, toward the people who made those decisions. Attributing it to technical competence severs that causal chain cleanly. The practitioners who lived through the accumulation carry the reputational cost of a problem they did not create. They raised concerns that were overridden and worked inside constraints they did not choose. Some internalize the blame, wondering whether they could have done better, whether they should have pushed back harder, whether they lack something. The structure produces this doubt as a feature, not a side effect.

The system then produces a second deferral: the failed system is declared legacy (too old, too rigid, too expensive to maintain) and replaced. The replacement project is governed by the same annual budget cycle, managed by the same profiles, and evaluated by the same metrics. The cycle restarts. The debt that accumulated in the previous system is not examined as evidence of a structural problem but treated as the natural aging of technology, as entropy rather than design.

Software does not age. The observation is Schaefer’s, made in passing but worth dwelling on: software stays fixed in time. What changes is the context around it: the business it serves, the expectations placed on it, the people who maintain it. When an organization declares a system legacy, it is not describing a property of the technology. It is describing the accumulated consequences of decisions that were never attributed to their real cause.

A system whose failures are consistently misattributed, whose lessons are never learned, and whose replacements reproduce its conditions is not a system that is trying and failing to improve. It is a system that is successfully maintaining itself.

A Final Observation

This analysis is not offered as a counsel of despair to the practitioners who have spent years pushing against these dynamics, advocating for better design, raising concerns that were overridden, watching careful work be undone by delivery pressure. The frustration those practitioners feel does not mean they misunderstood something. It means they understood something correctly and found themselves playing a game whose rules made their understanding a competitive disadvantage.

Schaefer ended his 2007 paper with a question that remains unanswered: what are the limits of the will? He was asking whether individuals operating within deeply coercive systems retain any meaningful capacity for independent judgment. The question is not rhetorical. History suggests that the levers which have actually changed systems of this kind are rarely internal. They tend to be external: competitive pressure severe enough to punish rigidity, regulatory constraint that removes certain decisions from the negotiating table, or the cumulative weight of people who refuse to play the game and vote with their feet. Departures concentrate in exactly the profiles the system needs most, the Good Soldiers and Sacrificial Lambs who carry the technical judgment the Machiavellians cannot replace. Whistleblowing, when it occurs, is almost always punished before it produces change. Smaller organizations built deliberately outside the AHoC model sometimes succeed precisely because they have not yet had time to reproduce its conditions. None of these are solutions. They are the terrain on which change, when it happens, actually occurs.

The technical argument for architectural quality (modularity, decoupling, reversibility) remains correct. The problem is that it is a technical argument being made in an environment where the selection pressure runs in the opposite direction, and technical correctness has never overridden incentive structure.

What this analysis suggests is that the relevant question is not how to make the case for better architecture more persuasively. It is a prior question: in an organization that has systematically selected for political behavior, trained its managers in its legitimacy, and structured its governance to make individual rationality and collective interest incompatible, from what position, and with what leverage, is change actually possible?

That question does not have a technical answer.

Gabriel Mailhot
Software Architect and Software Engineering Quality Advisor
Researcher (2005–2010): Image Processing & Creative AI
Business Expert / Domain Expert (2010–2026): Software Architecture, Software Development and Quality Practices

Reference

Schaefer, R. (2007). The Grand Theory of Everything: What Man-Made Systems Are, and Why They Fail. ACM SIGSOFT Software Engineering Notes, 32(4).