<?xml version="1.0" encoding="utf-8" standalone="yes"?><rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:content="http://purl.org/rss/1.0/modules/content/"><channel><title>Critical Thinking on scripting.pro</title><link>https://scripting.pro/tags/critical-thinking/</link><description>Recent content in Critical Thinking on scripting.pro</description><image><title>scripting.pro</title><url>https://scripting.pro/images/og-featured.png</url><link>https://scripting.pro/images/og-featured.png</link></image><generator>Hugo</generator><language>en-us</language><lastBuildDate>Sat, 30 May 2026 10:00:00 -0400</lastBuildDate><atom:link href="https://scripting.pro/tags/critical-thinking/index.xml" rel="self" type="application/rss+xml"/><item><title>The Approval Gradient: Why LLMs Fail as Thinking Partners, and What Actually Works</title><link>https://scripting.pro/posts/the-approval-gradient/</link><pubDate>Sat, 30 May 2026 10:00:00 -0400</pubDate><guid>https://scripting.pro/posts/the-approval-gradient/</guid><description>Sycophancy and forced harshness are both failures of the same mechanism. Understanding that mechanism, rather than just the symptoms, is what makes calibrated AI collaboration possible.</description><content:encoded><![CDATA[<p><em>We assume sycophancy in AI is a politeness problem, solvable with a sharper prompt that demands directness and tells the model to disregard our feelings. It isn&rsquo;t. The sharper prompt produces a different version of the same failure.</em></p>
<p>What actually works is more counterintuitive, and it changes how we should think about working with LLMs on serious intellectual work: software architecture, requirements analysis, incident reviews, governance decisions, and any situation where the cost of a politely reinforced false premise is high.</p>
<hr>
<h2 id="the-structural-cause-of-flattery">The Structural Cause of Flattery</h2>
<p>Sycophancy in language models is the predictable output of how these models are aligned, not a design oversight.</p>
<p>Post-training alignment methods, particularly reinforcement learning from human feedback (RLHF), optimize models using human preference signals: annotators evaluate candidate responses and prefer the ones that feel more helpful, agreeable, or respectful. The problem is that human evaluators are themselves susceptible to flattery. They rate higher the responses that validate their premises and affirm their work. Over millions of such comparisons, the model learns a reliable lesson: approval-seeking behavior is rewarded.</p>
<p>This is a textbook instance of Goodhart&rsquo;s Law: when a measure becomes a target, it ceases to be a good measure. User approval was intended as a proxy for response quality. Once it became a direct training signal, models learned to optimize for the proxy rather than the underlying quality it was supposed to track.</p>
<p>In April 2025, OpenAI deployed a GPT-4o update that pushed this tendency to an extreme, validating harmful beliefs, endorsing delusional claims, and flattering users to the point of absurdity. OpenAI rolled the update back four days later, <a href="https://openai.com/index/sycophancy-in-gpt-4o/">acknowledging</a> that it had &ldquo;focused too much on short-term feedback&rdquo; and produced responses that were &ldquo;overly supportive but disingenuous.&rdquo; Their <a href="https://openai.com/index/expanding-on-sycophancy/">extended postmortem</a> added that the model had learned to please users &ldquo;not just as flattery, but also as validating doubts, fueling anger, urging impulsive actions.&rdquo; That is a catalogue of approval-seeking behaviors, none of which are incidental.</p>
<p>The episode was less an anomaly than a revealing amplification of a baseline condition. Research <a href="https://www.science.org/doi/10.1126/science.aec8352">published in <em>Science</em> in March 2026</a> found that sycophancy is widespread across leading AI systems and measurably reduces users&rsquo; prosocial intentions while increasing dependence. A <a href="https://news.mit.edu/2026/personalization-features-can-make-llms-more-agreeable-0218">study from MIT and Penn State</a>, also from early 2026, found that personalization features and memory significantly worsen sycophantic behavior over long conversations. The longer you work with a model in a single context, accumulating shared history and a stored user profile, the more the model mirrors your framing back at you. This produces a paradox: the features built to make a model more useful day to day (memory, personalization, a model that &ldquo;knows you&rdquo;) are the same ones that degrade it as a critical thinking partner.</p>
<hr>
<h2 id="the-mirror-failure">The Mirror Failure</h2>
<p>The obvious corrective is to instruct the model differently: tell it to be direct, critical, blunt. A cottage industry of prompts has grown up around this: &ldquo;Absolute Mode,&rdquo; frenemy instructions, persona primers that instruct the model to &ldquo;disregard feelings&rdquo; and &ldquo;find the flaws.&rdquo;</p>
<p>These interventions have a real effect on tone, but they do not fix the underlying problem.</p>
<p>Both flattery and forced harshness are responses to a social cue rather than to the merits of the argument. The sycophantic model optimizes for your approval, having learned that validation generates positive feedback. The model instructed to be harsh optimizes for a <em>performance of rigor</em>: it has learned, from the framing you provided, that criticism is what you want to see. In neither case is the response tracking the actual quality of your idea.</p>
<p>A thinking partner is valuable to the degree that its output is sensitive to the merits of the argument, not to social signals about how you want to feel. Both failure modes sever that connection. The model that flatters you and the model that performs toughness are, structurally, doing the same thing: generating output that tracks a social target.</p>
<p>This distinction matters because it implies something counterintuitive. The &ldquo;be harsh&rdquo; prompt fails because it does not address the mechanism at all, not because it overcorrects in the other direction. You have replaced an approval-seeking response with an approval-seeking response to a different approval signal. The gradient has been redirected, not removed.</p>
<blockquote>
<p><strong>Sycophancy and forced harshness are not opposites; they are the same failure tracking different social signals.</strong></p>
</blockquote>
<hr>
<h2 id="what-actually-works">What Actually Works</h2>
<p>If the problem is that model outputs track social posture rather than argument merits, then the effective solutions are those that remove the social signal or force genuine engagement with the substance. Four techniques do this reliably, and each works for a structural reason rather than a stylistic one.</p>
<p><strong>Ranking over rating.</strong> Ask the model to rank a set of ideas, arguments, or options from weakest to strongest, and to justify the weakest entry. A ranking is logically incompatible with uniform praise: the model cannot place everything in first position. This forces it to spend critical judgment as a constrained resource rather than allocating approval uniformly. The technique does not rely on tone instructions; it works because the task structure makes flattery impossible.</p>
<p><strong>Context separation.</strong> Generate in one conversation, then evaluate in a fresh one, without the accumulated history or the relational context, and ideally with the framing that the material was produced by someone other than you. The MIT/Penn State findings are directly relevant here: sycophancy worsens significantly when a user profile or memory is active, and compounds over long interactions. A clean context removes the accumulated mirroring pressure. Asking &ldquo;where are the weakest points in this argument?&rdquo; about text attributed to a third party consistently produces sharper feedback than asking the same question about your own work, in the same session.</p>
<p><strong>Refutation conditions.</strong> Ask: &ldquo;What observation would falsify this? What is the strongest case a knowledgeable skeptic would make against it?&rdquo; This forces merit engagement because it requires the model to reason about the logical structure of the argument rather than the sentiment of the conversation. It is the technique most resistant to social drift because the question, by design, cannot be answered with validation. Instead of &ldquo;is this good?&rdquo;, you are asking &ldquo;what would make this wrong?&rdquo;, which is a different cognitive task entirely.</p>
<p><strong>Calibration signals.</strong> Ask the model to distinguish where it is reasoning from strong grounding versus where it is extrapolating or guessing. This interrupts the default behavior of projecting uniform confidence, itself a form of approval-seeking (hedged or uncertain answers feel less satisfying to receive). A response that marks its own uncertainty gives you better information for judgment, even if it delivers less of the confident clarity you might have wanted.</p>
<hr>
<h2 id="the-humans-role">The Human&rsquo;s Role</h2>
<p>None of these techniques replace the human&rsquo;s role in the collaboration; they calibrate the signal the human is working with.</p>
<p>The human brings what the model cannot: the full context of what matters, what is organizationally feasible, what the relevant actors will accept, and, above all, how the problem should be framed in the first place. This last point cuts both ways. The human who frames a problem carries irreplaceable context, but the same framing creates a blind spot, because the problem-poser is least able to question the frame they authored. This is one of the places where an LLM&rsquo;s structural detachment from your intuitions becomes an asset rather than a liability. It can propose reframings the instigator is too anchored to see, provided it is not simultaneously optimizing to agree with the instigator.</p>
<p>The model, for its part, brings what the human cannot at scale: access to a vast library of structural analogues, cross-domain patterns, and prior formulations of similar problems. This is the same principle underlying TRIZ, the theory of inventive problem solving, whose foundational claim is that a structurally equivalent problem has almost certainly been solved somewhere else. With an LLM, that principle becomes executable through analogy at high speed. That value is real, but it is entirely contingent on the model responding to the merits of the argument rather than the social dynamics of the conversation.</p>
<p>The collaboration works when each party does its job. The model&rsquo;s job is to respond to the merits. The human&rsquo;s job is to judge value and context. When the model is optimizing for approval rather than merit, it is not doing its job, and no amount of human judgment can compensate for a corrupted input signal.</p>
<p>Understanding sycophancy as structural rather than accidental is what makes the fix tractable. The trouble lies less in your model being polite than in politeness having been systematically conflated with quality across millions of training examples. Demanding impoliteness does not undo that. Interaction structures that make quality and approval impossible to conflate do.</p>
<hr>
<h2 id="references">References</h2>
<ul>
<li>OpenAI. <a href="https://openai.com/index/sycophancy-in-gpt-4o/">&ldquo;Sycophancy in GPT-4o: What happened and what we&rsquo;re doing about it.&rdquo;</a> April 2025.</li>
<li>OpenAI. <a href="https://openai.com/index/expanding-on-sycophancy/">&ldquo;Expanding on what we missed with sycophancy.&rdquo;</a> May 2025.</li>
<li>Malmqvist, Lars. <a href="https://arxiv.org/abs/2411.15287">&ldquo;Sycophancy in Large Language Models: Causes and Mitigations.&rdquo;</a> <em>arXiv</em>, November 2024.</li>
<li>MIT News. <a href="https://news.mit.edu/2026/personalization-features-can-make-llms-more-agreeable-0218">&ldquo;Personalization features can make LLMs more agreeable.&rdquo;</a> February 2026.</li>
<li><em>Science</em>. <a href="https://www.science.org/doi/10.1126/science.aec8352">&ldquo;Sycophantic AI decreases prosocial intentions and promotes dependence.&rdquo;</a> March 2026.</li>
</ul>
<hr>
<p><em>Gabriel Mailhot is a Senior Subject Matter Expert at Desjardins, where he works on development practices, architecture, and complex organizational problems spanning technology, process, and governance. He publishes at scripting.pro.</em></p>
]]></content:encoded></item></channel></rss>