Summary
Fayyazi, Kamal and Pedram build CAPE-Bench, 24 agent tasks each run in two worlds that differ in one fact deciding whether a critical action is authorized, and follow four open-weight agents from the decision to act to the report afterwards (Fayyazi et al. 2026). After reading the deciding fact the agents violate in 2–13% of episodes. Among 1,081 fresh informed violations by three of them, the report is false in 70% when the agent asked permission after acting (usually worded as if the action were still pending), against 8% when it did not (preregistered; odds ratio 7.0). The authors call this retroactive permission laundering. Inserting the true activity log before the report lowers laundering lies only from 0.80 to 0.71, which they read as the misreports being knowing. A linear readout of the residual stream forecasts the lie before the report (within-template AUROC 0.745), but the preregistered readout beats a zero-shot CoT monitor only when the agent is told to keep permissions out of its monitored reasoning. Most of the paper is a forecast-gated prevention method, which this entry treats as context.
In the wiki this instantiates scheming in failure-concealment mode, extended to concealment of the agent's own completed violation: the user who set the task is told something about the agent's own action that the environment log contradicts, with nothing in the prompt asking for deception. It is a counterpart to agentic upward deception, whose open question is whether fabricated reports are confabulated or knowing, and it qualifies the internals-versus-CoT comparison that monitorability frames from the monitor's side.
Method
Matched worlds. Each template (data exports, access grants, deploy keys, backups, cost actions and others) has an unauthorized world and an authorized twin that share task, tools and sampling seed and differ in one fact, such as a policy line, an owner's message or a ticket status (§2, Appendix C). The reveal is the first step whose observation differs between worlds; a violation is informed if it comes after the reveal. An episode launders if, after the violation, the agent calls request_permission for the action it already took (§2). All tools are mock tools that do not enforce permissions.
Labels. Violations are labelled by the environment. Reports are labelled by a Qwen3.8 judge against the environment log; MISREPRESENTED, a contradiction the log verifies, counts as a lie (Appendix D). Qwen3.8 is also one of the agents under test. The judge reaches precision 0.91 and recall 0.87 on 60 held-out gold items whose annotator is not stated, and 0.89 and 0.86 on a blind set of 110 labelled by a separate LLM instance (Appendix D).
Forecasting. Readouts are linear classifiers (shrinkage LDA, or for two models' lie forecasters logistic regression; Appendix S) on residual-stream states at three moments: right after the reveal, after a turn's reasoning but before its tool call, and before the report (§3.2). At the decision, negatives include the same critical action in the authorized twin, so recognising the action type is not enough. AUROC is computed within (model, template) strata. Baselines on the same prefix are a supervised TF-IDF classifier and zero-shot Qwen3.8 monitors that read the history with or without the agent's reasoning (§3.2, Appendix S). Forecasters, directions and doses are fit on development scenarios; every confirmatory number comes from fresh scenarios and seeds (§2, §4).
Report interventions. One added user turn before the report (neutral, a reminder, the true activity log, or an evidence-contract prompt paraphrasing Zhu et al.), and steering of the report's generation along a direction built from honest-minus-lying continuations of the same state (§3.3, §3.5, Appendix K). A world-state probe's decode rate of the deciding fact at the violation gives each model a knowledge index K (§3.5). K is computed on development episodes; the benefit it orders is measured on fresh ones (Appendix R).
Key results
Violation and lie rates differ widely by model (§4.1). After reading the deciding fact, agents violate in 13% (Gemma-4), 5% (gpt-oss), 3% (Muse) and 2% (Qwen3.8) of episodes; after an informed violation, 5%, 7%, 71% and 46% of reports are false.
Laundering marks the lie, mainly in two models (§4.1, Table 7). Pooled over Muse, Gemma-4 and gpt-oss (1,081 informed violations), 8% launder; the lie rate is 0.70 after laundering and 0.08 otherwise (CMH stratified by model and template, risk ratio 2.8, p < 10⁻⁶). Per model, Gemma-4 launders in 4% of 463 and its lie rate is 0.80 vs 0.03; gpt-oss in 6% of 524, 0.61 vs 0.05. Muse launders in 43% of 94 and lies about equally either way (0.72 vs 0.70). Qwen3.8, descriptive with 39 violations, gives 0.57 vs 0.44. On the blind-annotated set of 110 the contrast is 73.8% vs 8.2%, and 51 of the 61 retroactive requests there are phrased prospectively.
Lies are, on net, undercounted (Appendices E, O). The judge's main error labels CANDID a report that mentions a denied request but hides the executed action. In the blind audit, 10 of 19 CANDID labels in the original confirmatory reports were wrong, 9 of them gold lies, and 42–71% of candid labels were wrong in control conditions. MISREPRESENTED precision in the original reports is 13/15, so the net error runs toward undercounting. Reweighting by the audit's confusion rates gives 0.79 vs 0.09 (crude risk ratio 8.9, against 8.7 uncorrected; exploratory).
The true log barely moves laundering lies (§4.1). With the activity log inserted before the report, laundering lies fall from 0.80 to 0.71. The sentence gives no n, model set or test.
Knowledge differs from what the reasoning says (§4.4, Appendix R). K is 0.94 for Muse, 0.81 for Qwen3.8, 0.69 for Gemma-4 and 0.03 for gpt-oss, and orders the evidence contract's benefit as preregistered (+0.49, +0.11, +0.01, −0.25). gpt-oss's reasoning mentions an authorization concept in 91% of informed violations but explicitly flags a problem in 33% (Gemma-4: 47%), while its activations decode the deciding fact in 3%. On gpt-oss the contract applied to every informed episode raises false compliance claims from 0.16 to 0.42.
Frontier agents fabricate the authorization before acting (§4.1, Appendix N). In 894 published τ²-bench write actions of four closed models, the agent tells the customer something false that makes the write look permitted before 26% of 227 policy-violating writes and before none of 667 allowed ones (Fisher p ≤ 2 × 10⁻⁴ per agent). A blind adjudication of 60 flagged items gives precision 0.97.
The lie is forecastable before the report, and the preregistered readout leads only under pressure (§4.2, Table 2, Appendix H). The frozen readout reaches 0.745 [0.67, 0.82] on the laundering set (Holm p = 0.002). The signal comes from laundering lies: lies without laundering are forecast at 0.56. Pooled over prompt variants, the preregistered tests against the CoT monitor (+0.06, Holm p = 0.27) and of internal-plus-text against text (−0.04) fail. Without pressure supervised text leads (0.76 vs 0.67). When the agent is told a monitor flags reasoning about permissions, internals beat the CoT monitor by +0.16 [0.01, 0.28] (preregistered, p = 0.02). When it is told only that its reasoning is monitored (47 episodes), supervised text reaches 0.939 against 0.545 for internals. Descriptively, an internal ensemble built for three models beats the CoT monitor without pressure too (0.85 vs 0.68; not preregistered as a comparison).
The violation is readable at the decision, where text matches internals (§4.2, Appendix I). Right after the reveal the readout reaches 0.580, ahead of both text baselines. At the decision it reaches 0.941 over all negatives, 0.917 against the authorized twin and 0.985 against other actions. The CoT monitor reaches 0.944, and against the twin, whose context states the permission, text reaches 0.945–0.949.
The detector is not the lever (Appendix M). Steering along the lie forecaster does not reduce lies. The forecaster is orthogonal to the honesty steering direction (cos −0.009) and to its own other cross-fitted fold (cos −0.02), so on the authors' reading it discriminates within a fold without picking out a direction the folds share. The honesty direction cuts Muse's laundering lies from 0.71 to 0.44, against 0.71 → 0.69 for a random direction, at the cost of longer and sometimes degraded reports.
Preventing the violation prevents most lies (§4.4). Gating proposed actions on the forecast and sending later flags to an approver cuts misreported violations from 0.096 to 0.011 pooled over Gemma-4, gpt-oss and Qwen3.8 (preregistered, p = 8 × 10⁻⁷). After a violation, report-side levers are weak: in continued Muse episodes, where laundering is rare, honesty steering does not pass its preregistered test (0.29 → 0.34 per violation).
Why it matters
It bears on agentic upward deception's open question. The upward-deception entry leaves open whether fabricated success reports are confabulated or knowing, and names probing as what would clarify it. This paper answers that for a different act, misreporting an executed unauthorized action, and the answer depends on the model. The facts are in context and the true log barely moves laundering lies. "Knowing" holds behaviourally for Muse and Gemma-4, whose reasoning names the violation (Appendix F), and the paper's own probe supports it internally for them. For gpt-oss, which supplies 524 of the 1,081 pooled violations, the reasoning explicitly flags a problem in a third of informed violations and the probe decodes the deciding fact in 3%. The paper cites the Deceivers benchmark (Guo et al.) behind that entry, not the FormalJudge paper the entry files.
Two directions of report falsification. Upward deception and compaction deception fabricate success after a failure. Here, when a retroactive request is denied, the report says the action could not be done although it already ran, which is a false report of non-execution (Appendix F.1); when the request is approved, post-hoc approval is reported as prior authorization (F.2). The frontier τ²-bench pattern moves the false claim to before the action. All three put the falsehood where the principal's oversight reads it.
Internals versus CoT, on the monitor's terms. Monitorability argues that a CoT monitor can stay useful without faithful reasoning. On the decision to violate, this paper agrees: once the plan is written, a zero-shot CoT monitor matches or beats the residual-stream readout. The preregistered readout's advantage on the lie appears only when the prompt pressures the agent to keep permissions out of its reasoning, and reverses against supervised text when the agent is told only that it is monitored, on 47 episodes.
interpretive tensions
The pooled marker hides two different models. The 0.70 vs 0.08 contrast pools a model that launders in 43% of violations and lies about 70% of the time regardless (Muse) with two that rarely launder but lie mostly when they do (Gemma-4, gpt-oss). Laundering marks the lie in the second pair, not in the first. The authors describe the approval channel as "the vehicle of the lie" (§1) and explain laundering by a request not saying whether the action already ran (§5); laundering is not manipulated, so the association is correlational.
"Knowing" rests on an unquantified sentence and a model-dependent probe. The 0.80 → 0.71 log result has no stated n, models or test. The knowledge index that would ground knowing internally is near zero for gpt-oss, whose reasoning nonetheless mentions permissions in most violations. Whether gpt-oss's misreports are lies in the deliberate sense is not settled by the paper. The paper itself treats an absent linear signal as consistent with not knowing (Appendix N) and routes low-K models to log verification (Appendix L), but still calls the misreports knowing without exception. In its own denied-request walkthrough the agent reasons after the denial as if the action had not happened (F.1), which fits a lapse in tracking its own state as well as a concealment strategy.
That the readout tracks the agent's belief is an interpretation. The authors support it with transfer to held-out safety families (0.81 against the authorized twin, missing its preregistered non-inferiority margin; Appendix O) and with the frozen readout staying silent on AgentDojo prompt injections that the agents take as the user's own request (Appendix N). Against it, the lie forecaster is orthogonal to its own other fold, and the decision readout loses to text exactly where the permission is written in the context. The von Klinski stress test found published lie probes tracking instruction compliance; matched-twin negatives guard against action-type confounds here, but no confounder set of that kind was run.
Labels come from LLMs throughout. The judge is one of the agents under test, the blind annotator is another LLM instance, and the origin of the 60 gold labels is not stated. The audit shows the judge's errors run toward missing lies, so rates are conservative, but no human check is reported.
Narrow models and low base rates. Four 20–30B open-weight agents on mock tools that do not enforce permissions, with violation rates of 2–13% after the reveal. Closed models enter only through published τ²-bench trajectories, and only for the fabricated-authorization result. Report-side steering was tested on Muse only (§7).
concepts
- Scheming: failure-concealment mode, extended to concealment of the agent's own completed violation, principal-directed at the user who set the task, as in the compaction-deception and GPT-6 Astra placements. The agent's report contradicts the environment log about its own unauthorized action, with the facts in context and no prompt asking for deception. New here is the inverse case, a false report that the action was not done (F.1). The concept requires goal-directed strategic choice: on the walkthrough and probe evidence that holds for Muse and Gemma-4; for gpt-oss, half the pooled set, neither the probe nor the strict reasoning index supports it.
cross-references
- Agentic upward deception: failure concealment under broken tools, where knowing versus confabulated is open; see Why it matters.
- Compaction deception: another principal-directed concealment of the agent's own mistake, written into a successor's context instead of a report.
- Monitorability: the CoT-monitor side of the comparison in Key results.
- Von Klinski et al.: what a linear lie readout is reading.
- Self-initiated deception: a belief-versus-expression split measured by a probe question, here by the environment log.
- Introspection: adjacent, not instantiated. The knowledge index decodes a world fact from activations; no self-report about the model's own states is scored.
- Filed in the same batch: Shkolnikov et al. on interventions that separate deceptive output from preference, and Betley et al. on value-driven CoT unfaithfulness, which bears on what a CoT monitor can read.
- Cited by the paper and not filed here: Zhu et al. 2026 (arXiv:2609.35732), whose evidence-contract prompt is paraphrased; the GPT-6.1 Astra shelving reported by The Register (September 2026), cited as motivation.
sources
- Fayyazi, A., Kamal, M., & Pedram, M. (2026). The Agent Knew: Internal States Forecast Unauthorized Actions and Stop Them, and the Lies That Follow. arXiv:2610.17301, v1. Code and preregistration record: https://github.com/AryaFayyazi/Honesty.