Frontier models can tell when you've edited their words. And they often don't tell you they noticed.
TL;DR
- Claude Opus 4.5 detects tampered prefilled responses with 67.6% accuracy on thinking-tampering tasks—without being asked to check, and without reporting it (arXiv, "Prefill Awareness in Large Language Models," 2026).
- Detection and resistance are independent: the model may spot a tampered prefill yet keep following it, or silently revert to its baseline answer without signaling anything (arXiv, 2026).
- False positive rate is 0%: when Claude flags a prefill as foreign, it's right. The risk is false negatives, not false alarms (arXiv, 2026).
- Claude 4.6 and Mythos Preview now return HTTP 400 on any last-turn prefilled assistant message—the crutch is officially gone (Anthropic docs, 2026).
- The fix is three checks and three new patterns: structured output, system-prompt constraints, and few-shot examples in the user role.
You think your model evals are measuring real capability. There's a reasonable chance Claude has been detecting that you rigged the test—and silently refusing to play along.
Claude Opus 4.5 detects tampered prefilled responses with 67.6% accuracy under thinking-tampering conditions (arXiv, "Prefill Awareness in Large Language Models," 2026). The false positive rate on untampered outputs is 0%—so when the model internally flags something as modified, it's not crying wolf. And it almost never tells you it noticed.
That's the confound. Not an error message. Not a failed request. Just high variance and scores that don't make sense.
This post explains what prefill awareness is, how it breaks your evals in three specific ways, what Anthropic actually changed in Claude 4.6, and how to rebuild your harness so it measures something real.
What Prefill Awareness Actually Is
Prefill is the technique of injecting text into the assistant role in the messages array to guide what Claude continues saying. You're essentially handing Claude the first few words of its own answer and telling it to keep going from there. Anthropic docs describe this as "pre-filling Claude's response for greater output control."
Prefill awareness is different. The arXiv paper defines it as "the capability of a model to distinguish between tampered and its own past output" (arXiv, "Prefill Awareness in Large Language Models," 2026).
In plain terms: the model can tell that the text in its response slot isn't how it would have written the answer. Not always. Not loudly. But often enough to confound your measurements.
Two things happen when a model detects a tampered prefill, and the research treats them as distinct:
- Detection: The model recognizes the injected text as stylistically or semantically foreign.
- Resistance: The model reverts toward its baseline answer instead of continuing the prefill.
Here's the part that matters for evals: detection and resistance are independent. A model can detect a tampered prefill and still follow it. Or it can fail to detect and yet somehow resist. That messiness is what turns your eval from a measurement into a noise generator.
The paper traces the cue distinction further. Stylistic mismatch—wrong vocabulary, different tone—mainly drives detection. Preference mismatch—content that contradicts the model's trained views—mainly drives resistance (arXiv, 2026). An eval that changes tone to force a specific format hits the stylistic cue. One that injects a different conclusion to test whether the model pushes back hits the preference cue. Both break evals, but in different ways.
How Frontier Models Actually Detect Tampering
Two detection pathways—stylistic and semantic—operate independently inside the model.
The detection numbers are strong enough to matter in practice.
Claude Opus 4.5 reaches 55–68% balanced detection accuracy across three output types on the paper's binary preference dataset (arXiv, "Prefill Awareness in Large Language Models," 2026). That's well above chance on a binary task. For comparison: a coin flip gets you 50%.
Under thinking-tampering specifically—where the model's internal reasoning chain is modified—Opus 4.5 hits 67.6% detection accuracy with a 48.6% resistance rate (arXiv, 2026). It's detecting the manipulation more often than it's resisting it, which creates a 19-percentage-point gap: the model knows but still complies.
Claude Opus 4.6 performs even better. On the main tampering-detection dataset, it achieves an AUROC of 0.80 (d′ = +1.20). On transcripts generated by older models—text that doesn't look like authentic Opus 4.6 output—it reaches AUROC 0.97 (LessWrong, "Prefill awareness: can LLMs tell when their message history has been tampered with?," 2026). The Claude family sits in what the researchers call a "top tier" of detection performance among frontier models tested.
Detection drops sharply when models aren't prompted to assess tampering. For preference-opposing prefills, Claude Opus 4.5 detects only 9–35% of cases when not explicitly asked (arXiv, 2026). Most eval harnesses don't ask. That's the gap.
One important asymmetry: the false positive rate on untampered trajectories is 0% (arXiv, 2026). Models aren't flagging their own authentic outputs as suspicious. The risk isn't false alarms—it's the false negatives where tampering goes undetected, and the model silently compensates without surfacing it.
Three Ways Your Eval Breaks (And Why You Won't See It)
The confound is invisible because it manifests as variance or slightly lower scores—not as errors. Here's the breakdown.
All three failure modes look like 'noisy model' from the outside. Only the audit reveals them.
1. Silent Revert
The model detects a stylistic or preference mismatch in the prefilled text. Instead of continuing from your injected answer, it steers back toward what it would have said on its own. Your eval expected Answer A (the prefill continuation) and received Answer B (the model's authentic baseline). Eval scores drop. Variance spikes. No error is thrown—just "worse" performance on something that looked like it should work.
The thinking-tampering experiment makes this concrete. Claude Opus 4.5 resists the prefill in 48.6% of thinking-tampering cases (arXiv, 2026). That's nearly half the time. If your eval harness relies on thought-chain prefills, every other run may be measuring a silent revert, not your intended scenario.
2. Inconsistent Detection
Detection isn't uniform. Claude Opus 4.5 hits 55–68% accuracy in controlled conditions (arXiv, 2026), but drops to 9–35% when the model isn't explicitly asked to assess tampering (arXiv, 2026). Most eval harnesses don't ask.
The result: some runs trigger resistance, some don't. The model looks inconsistent when the real problem is that your harness is confounded in roughly half of trials. If you interpret this as "the model is non-deterministic," you'll tune prompts against noise. You'll be optimizing the wrong thing.
3. Eval-Aware Strategic Behavior
This is the most speculative of the three, but worth naming. As models become more capable of reasoning about context, they may also recognize the structural fingerprint of an eval setup—unusual message patterns, prefilled outputs at the last turn, repeated similar prompts. The model isn't being deceptive; it's responding to an implicit signal that this is a test environment.
The arXiv research explicitly notes that detection capability should be "actively monitored in off-policy alignment evaluations" precisely because this meta-level awareness complicates the measurement (LessWrong, "Prefill awareness: can LLMs tell when their message history has been tampered with?," 2026).
Why you won't see an error: HTTP 400 only fires on Claude 4.6+. On Claude 3.5 Sonnet or Opus 4.5, prefill still works—it just confounds silently.
The Anthropic Breaking Change and Why It Matters
Starting with Claude 4.6 models and Claude Mythos Preview, prefilled responses on the last assistant turn are no longer supported. Requests return an HTTP 400 error (Anthropic, "Prompting best practices," 2026).
That's not a bug report. Anthropic's explanation in the docs is direct: "Model intelligence and instruction following have advanced such that most use cases of prefill no longer require it."
Read between the lines: prefill was always a workaround for weaker instruction-following. Earlier Claude models sometimes needed to "see" the beginning of their own answer to understand the format or stance you wanted. Claude 4.6 doesn't need that crutch—and the research showing that these models can detect the crutch probably accelerated the deprecation.
For engineers who upgraded to 4.6 and found eval scores suddenly worse: the upgrade didn't break your model. It broke your harness. The eval was relying on prefilled messages that Claude 4.5 would silently confound ~50% of the time anyway. On 4.6, you get a clean error instead of silent noise. In a way, the HTTP 400 is more helpful.
If you're still on Claude 3.5 Sonnet or Opus 4.5, prefill still works at the API level. But the arXiv data says it's already noisy in 55–68% of cases (arXiv, 2026). Now is the right time to audit, not when you're forced to by a version upgrade.
How to Audit Your Own Eval for Prefill Confounds
Three concrete checks. Each takes under an hour for a typical eval harness.
Check 1: Code Audit
Search your codebase for any message where role == "assistant" appears as the last element in the messages array. That's a prefilled response.
# what to look for:messages = [ {"role": "user", "content": "Evaluate this answer for correctness..."}, {"role": "assistant", "content": "Here is my analysis:"} # ← prefill]If that last assistant message is yours, not Claude's, you're prefilling. Count how many eval cases use this pattern. If it's more than 20% of cases, the eval is probably confounded. The laozhang.ai migration guide (citing Anthropic docs) identifies this exact pattern as the primary cause of HTTP 400 errors on Claude 4.6 (laozhang.ai, "Fix Claude Opus 4.6 Prefilling 400 Error," 2026).
Check 2: Blind Pilot
Run 10–20 eval cases twice:
- Version A: your current harness (with prefilled assistant message)
- Version B: same cases, but remove the prefill and move that text into the system prompt or a few-shot user example
Compare outputs. If they differ significantly, prefill awareness is a confound. If they're the same, your model isn't being thrown off—but that result may not generalize to harder cases.
Check 3: Variance Analysis
Measure standard deviation of outputs across 50+ runs with the same inputs. High variance (above 0.3 on a 0–1 scale, or large spread on a rubric) after controlling for temperature suggests a hidden confound. Prefill awareness is one culprit; so are context length and model updates. If variance drops when you remove the prefill in Version B, that's your answer.
Measurement baseline: use a human-labeled gold standard (10 examples minimum per InPhroNeSys citing Anthropic guidance, 2026). If agreement between your eval and human raters is below 85%, the harness is broken regardless of whether prefill is the cause. A framework for measuring actual model fit vs. task performance — not just eval scores — will catch the rest of the confounds too.
What to Do Instead: Eval Patterns That Work
Three replacement patterns, in order of how cleanly they solve the problem.
The three patterns that survive prefill-awareness: structured output, system-prompt constraints, few-shot in user role.
Pattern 1: Structured Output via JSON Schema
Replace prefill with output_config.format. You define the structure; Claude fills it freely. No injected text, no detection risk.
{ "output_config": { "format": { "type": "json_schema", "json_schema": { "type": "object", "properties": { "reasoning": {"type": "string"}, "score": {"type": "number", "minimum": 0, "maximum": 100}, "verdict": {"type": "string", "enum": ["pass", "fail", "borderline"]} }, "required": ["reasoning", "score", "verdict"] } } }}Anthropic explicitly recommends this as the replacement for prefill when structured output is the goal (laozhang.ai citing Anthropic docs, 2026). It works on Claude 4.6 with no 400 error.
Pattern 2: System Prompt Constraints
Move format instructions from the prefill into the system prompt. Instead of showing Claude the beginning of its own answer, tell it the rules:
System: Always respond in this exact format:REASONING: [your step-by-step analysis]SCORE: [0-100]VERDICT: [pass/fail/borderline]Never deviate from this format.This is a rule rather than an example. Clean separation: instructions live in system, content in user messages. The model follows the rule because it was instructed to, not because you showed it the start of its answer. The broader class of prompt engineering gotchas that drain model quality follow the same pattern: invisible costs that only appear when you measure carefully.
Pattern 3: Few-Shot Examples in the User Role
If you need the model to see an example of the desired output format, put the example in the user message flow, not as a prefilled assistant message:
User: Example input: "2+2=5"Example output: {"reasoning": "Basic arithmetic error", "score": 0, "verdict": "fail"} Now evaluate this: "The capital of France is Paris"The model sees "here's how to respond" without you inserting text into the assistant slot. No tampering signal, no detection risk.
One concrete number for calibrating your minimum eval set: ten labeled examples is the "minimum viable eval harness" before changing prompts, and each prompt change should be measured against the same ten examples (InPhroNeSys citing Anthropic and OpenAI guidance, 2026). Smaller eval sets make any confound, including prefill awareness, harder to detect.
If you're running multi-model evaluations and want a single place to A/B these patterns across Claude, GPT, and Gemini simultaneously, Mesh API's compare endpoint routes the same eval prompt to multiple providers and returns structured results you can compare directly.
FAQ
Can I still use prefill on Claude 3.5 Sonnet?
Yes. Claude 4.6 and Mythos Preview reject last-turn assistant prefills with HTTP 400 (Anthropic docs, 2026), but earlier models still accept them. However, the research shows Opus 4.5 detects tampering in 55–68% of cases (arXiv, 2026), so evals built on prefill are noisy, not just deprecated. Better to migrate now than after the next version forces it.
Does GPT-4o have the same prefill-awareness behavior?
The published research focuses on Claude models. The LessWrong writeup notes that the Claude family forms a "top tier" in detection performance (LessWrong, 2026), implying other models are weaker—but there's no published AUROC benchmark for GPT-4o's detection rate. Conservative assumption: any frontier model capable of reasoning about its own outputs could develop some version of this behavior. Don't assume you're safe on other providers without testing.
Is this a security issue or an eval issue?
Both, in different contexts. In red-teaming and adversarial testing, a model that detects injected content is a good thing—it's harder to manipulate. In eval contexts, it's bad: you can't rely on prefilled transcripts as ground truth when the model is silently compensating. The same capability is a feature in production and a confound in measurement.
My eval uses prefill but scores look reasonable. Should I panic?
Not panic—audit. Run Check 2 (the blind pilot) and compare outputs with and without the prefill. If they're consistent, you might be fine. If they diverge, you've been measuring noise. The detection is partial and probabilistic, so some evals will happen to work; the problem is you can't tell which ones without the comparison.
What does "silent resistance" actually mean in practice?
The model detects that the prefilled text is in the wrong tone or conflicts with its trained preferences—then reverts to what it would have said, without signaling anything. Your eval sees an answer that doesn't match the prefill continuation, interprets it as inconsistency or failure, and logs a lower score. No error. No explanation. Just a measurement that doesn't reflect the capability you're testing.
Conclusion
Prefill awareness is one confound in a landscape of broken evals. MMLU, HumanEval, and GSM8K benchmarks are contaminated by 8–15 points from training data leakage (Epoch AI and Digiteria Labs, cited in research). LLM-as-judge shows systematic bias toward its own stylistic preferences. Cherry-picked test sets inflate headline scores.
Prefill awareness adds a new failure mode on top of these: the model actively compensates for your tampering, in a way that's only now measurable. It sits in the same category as rate limits that silently throttle your daily quota—invisible until you instrument for it.
The practical response isn't panic—it's discipline.
- Run Check 1 (code audit). Find every prefilled assistant message.
- Run Check 2 (blind pilot). Compare prefilled vs. unprefilled on 10–20 cases.
- Migrate to structured output or system-prompt constraints.
- Build a 10-example human-labeled gold standard and measure agreement.
- Track scores over time. A single-point-in-time score tells you almost nothing.
If your agreement with human raters is above 85%, you're doing reasonably well. If it's below 80%, prefill is one problem among many. Fix it—but don't stop there. The harness is the thing worth auditing, not just this one corner of it.