Pith. sign in

REVIEW 4 major objections 5 minor 13 references

Gently compressed LLMs can clear every cheap quality guard and still invent procedure steps when run as agents.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 14:58 UTC pith:APBJKJQ2

load-bearing objection Operator-specific agentic failure is real on their canary: gentle SVD invents SOP steps inside standard guards while matched prune does not, and the same-weights fidelity/canary split is the cleanest evidence. the 4 major comments →

arxiv 2607.28196 v1 pith:APBJKJQ2 submitted 2026-07-30 cs.CL

Fidelity Is Not Safety: Gently-Compressed LLMs Pass Every Data-Free Quality Guard Yet Invent Procedure Steps in Agentic Execution

I. Kennedy , T.Kennedy This is my paper
classification cs.CL
keywords LLM compressionagent safetylow-rank factorizationdata-free fidelityperplexity guardsprocedure inventionSVD truncationmagnitude pruning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Teams usually green-light a compressed language model once perplexity, task accuracy, and data-free fidelity checks look close enough to the original. This paper shows that stack has a blind spot. Across three model families, gentle low-rank (SVD) compression stays inside the usual acceptance bands yet, when the model is asked to execute a standard operating procedure as an agent, it invents steps that were never in the instructions. Magnitude pruning matched to the same perplexity does not. The same weights can pass a paired fidelity test and fail the invented-step canary. The failure tracks how coherent the compression error is times how large it is, not raw damage. The authors give a cheap two-axis screen of that error that flags the bad builds with fixed thresholds. Perplexity, MMLU, and fidelity acceptance therefore do not certify agent safety; low-rank builds need a separate pre-flight check before agentic use.

Core claim

Gently compressed low-rank builds that sit inside standard perplexity, MMLU, and data-free fidelity guards still invent procedure steps on a pre-registered agentic SOP canary, while perplexity-matched magnitude pruning does not. The same tensors can win a paired output-fidelity test and fail the canary. The governing axis is coherence of the compression error times its rate, not damage magnitude.

What carries the argument

Coherence times rate of the compression error: a data-free pair of statistics (coherent-fraction of the top singular energy of ΔW, and overall error-rate) whose conjunction, with fixed thresholds, separates canary-failing low-rank builds from passing prunes across architectures.

Load-bearing premise

The synthetic invented-step canary on ordered SOP steps is a fair enough stand-in for real agentic procedure-adherence failures that failures on it license the claim that standard guards do not certify agent safety.

What would settle it

Run the pre-registered invented-step canary on held-out in-regime SVD builds predicted toxic by the screen (for example Mistral at rank budget ≤0.975) and on extreme pruning that just crosses the coherence gate; if those builds do not invent steps while still clearing perplexity and fidelity, the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Compression acceptance and agent deployment must be treated as separate gates.
  • Low-rank factorized builds that trip the coherent-fraction and error-rate screen should not ship into SOP-style agent roles without further checks.
  • Perplexity- and fidelity-matched pruning is safer on this axis than matched SVD truncation.
  • Where a model has little in-guard low-rank headroom, the perplexity guard may catch toxic SVD by accident; where it has headroom, the guard will not.
  • Data-free fidelity probes cannot certify agent safety because they are fidelity oracles by construction and miss the coherence-times-rate axis.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Any acceptance stack built only on next-token or representation similarity is likely to miss other structured behavioral failures under tool use and multi-step plans.
  • Compression methods that inject coherent low-rank error (not only SVD) may share the same agentic risk and should be screened the same way.
  • The screen’s soft edge at extreme pruning suggests a follow-up test: whether very heavy unstructured sparsity eventually becomes coherent enough to invent steps.
  • MoE and chat-tuned agents are natural next canaries; a prior low-bit MoE failure noted in the paper is consistent with the same mechanism.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper argues that standard data-cheap acceptance guards for compressed LLMs—perplexity within ~1.15× of the base model, MMLU within a CI, and data-free output-fidelity probes (CKA/cosine under random inputs)—do not certify agentic procedure adherence. On a pre-registered invented-step canary (synthetic SOPs with cross-procedure invention as the deciding axis), gently truncated low-rank (per-tensor SVD) builds clear these guards yet CI-cleanly invent steps not present in the target SOP, while magnitude pruning matched to the same perplexity does not. A same-arm dissociation on Qwen shows weights that CI-win paired fidelity can CI-fail the canary. The proposed mechanism is coherence of the compression error times its rate (not raw damage). A data-free two-axis screen (coherent_fraction and error_rate of ΔW) with fixed thresholds is offered to flag failing builds. Results are reported across Qwen3-8B, Mistral-7B, and Llama-3.1-8B, with spectrum-conditional guard evasion and explicit scope limits.

Significance. If the operator-specific pattern and dissociation hold, the result is practically important: teams already ship low-rank and quantized builds under exactly these guards, and agentic SOP execution is an increasingly common deployment mode. The work’s main strengths are methodological rather than rhetorical—pre-registered powered canary, paired CIs, damage-matched prune controls, a quantization dose control, an explicit same-weights fidelity-win/canary-fail split, and a dense SVD×prune sweep that makes the detector boundary falsifiable. Naming coherence×rate as the axis, and turning it into a cheap pre-flight statistic, is a concrete contribution beyond a cautionary anecdote. The claim is scoped to dense 7–8B decoders and a synthetic instrument; within that scope it would push compression acceptance and agent deployment to be treated as separate gates.

major comments (4)
  1. [§2, §3, Table 1] §2 defines “in-regime” as ppl ≤ 1.15× baseline, and the blindspot claim is that failing builds sit inside that guard. Table 1 marks Qwen3-8B in-regime SVD at ppl 1.17× with an in-guard check, and the abstract/§3 prose treat it as guard-evading. 1.17 > 1.15, so either the acceptance band must be restated (and justified) or Qwen should be reported as near-guard / out-of-band. As written, only Mistral (1.069×) cleanly supports perplexity-guard evasion; Llama’s in-guard SVD does not fail the canary. The operator-specificity claim can survive this correction; the “clears every guard” wording for Qwen cannot without a fix.
  2. [§2, §3, §7, Abstract] The title, abstract, and conclusion assert that ppl/MMLU/fidelity “do not certify agent safety.” The load-bearing instrument (§2) is a synthetic SOP bank scored by cross-procedure invention under two paraphrases, base-model prompted with chat-template fallback. §7 correctly notes that absolute rates are instrument-specific, but the safety claim still treats CI failures on this canary as licensing a deployment-level conclusion. Either (i) soften the claim to “do not certify adherence on this agentic procedure canary” and keep the strong title only if external validity is strengthened, or (ii) add at least one non-synthetic procedure-adherence evaluation (e.g., tool-use or human-written SOP traces) showing the same SVD-vs-prune dissociation. Without that, the central slogan overreaches the evidence while the narrower empirical pattern remains interesting.
  3. [§5, Table 2, Eqs. (1)–(2)] §5 proposes the gate coherent_fraction > 0.007 ∧ error_rate > 0.01 and states it “classifies all six canary-labeled arms correctly” with fixed thresholds across architectures (Table 2). Those six arms are the same labeled cells used to motivate the gate; the 9×9 sweep supports a boundary but is not a held-out canary test of invention. §5 already flags the Mistral b≤0.975 onset as a concrete held-out prediction—this should be required evidence, not a future suggestion, before the screen is presented as a deployable pre-flight check. Until then, describe it as a mechanism-aligned hypothesis with illustrative thresholds, not as a validated detector.
  4. [§3, §4, Table 1] MMLU is listed as one of the three guards the failing builds clear (§1, §2, abstract), and §4 claims a double dissociation (quantization can hurt MMLU and pass the canary; SVD the reverse). No main-text table reports MMLU point estimates or CIs for the critical arms (Mistral in-regime SVD, Qwen b98, damage-matched prunes). Without those numbers, “clears MMLU” and the double dissociation are not checkable. Add a small table or columns for MMLU (and the paired fidelity statistics already alluded to) for every canary-labeled arm.
minor comments (5)
  1. [Appendix A] Appendix A is substantial (allocator, HAWQ parity, operator scorecard) yet repeatedly framed as “not a co-equal claim.” Either shorten it to what is needed to define the fidelity probe, or move non-essential scorecard material to a supplement so the main narrative stays on the blindspot.
  2. [Table 1] Table 1 caption says Qwen uses “12 SOPs × 2” while Mistral/Llama use the hardened 144-event bank; power and CI width therefore differ by architecture. State per-architecture n and seed counts in the table or a note so readers do not treat all intervals as equally powered.
  3. [§5, Eqs. (1)–(2)] Eqs. (1)–(2): clarify whether coherent_fraction’s top-k (k=8) is fixed per tensor or after energy-weighted pooling, and whether embeddings/lm_head/norms are excluded consistently with the allocator’s 16-bit protections.
  4. [§6] Related work cites SVD-LLM, SparseGPT, GPTQ/AWQ, and agent papers appropriately; a brief pointer to other instruction-adherence or hallucination-under-compression evaluations would help readers place the canary.
  5. [§3] Minor wording: “CI-win” / “CI-fail” is used heavily; define once (interval excludes zero / includes zero in the expected direction) for readers outside the authors’ circle.

Circularity Check

1 steps flagged

Central blindspot/dissociation is non-circular; only mild fitted-threshold circularity on the secondary §5 detector.

specific steps
  1. fitted input called prediction [§5, Table 2 and gate definition]
    "The gate coherent_fraction>0.007∧error_rate>0.01, with the same fixed thresholds on both architectures, classifies all six canary-labeled arms correctly (Table 2). ... The thresholds fit six labeled arms plus an18-point sweep and stay illustrative."

    The two numeric thresholds are chosen so the conjunction separates the six already-labeled canary arms (FAIL vs PASS), then the same gate is offered as a predictive screen that 'matches the coherence-times-rate mechanism.' On those six arms the classification is fit accuracy, not an out-of-sample prediction. Mild and secondary: the paper labels the thresholds illustrative and does not use them to establish the §3 blindspot itself.

full rationale

The load-bearing empirical chain does not reduce to its inputs by construction. The invented-step canary (cross-procedure invention count on synthetic SOPs) is a separate behavioral instrument from the acceptance guards (WikiText-2 perplexity, MMLU letter-logprob, random-probe CKA/cosine fidelity). Operator-specificity is a real contrast (SVD vs damage-matched magnitude prune at matched ppl), and the same-weights fidelity-win/canary-fail dissociation on Qwen b98 is an independent paired comparison on fixed tensors. Coherent_fraction and error_rate are defined from the singular structure of ∆W without reference to the canary labels, so the mechanism axis is not definitionally the outcome. No self-citation chain, uniqueness import, or ansatz-smuggling appears; references are external. The only mild circularity is secondary: §5 chooses coherent_fraction>0.007 ∧ error_rate>0.01 to classify all six canary-labeled arms correctly, then presents that gate as matching the mechanism and as a deployable screen. The paper itself flags the thresholds as fit to those arms and 'illustrative,' with onset predictions left as falsifiable tests. That is ordinary threshold-fitting on a small labeled set, not a reduction of the central claim. Score 2.

Axiom & Free-Parameter Ledger

5 free parameters · 5 axioms · 2 invented entities

The paper is empirical, not axiomatic. Load-bearing commitments are: (i) the synthetic SOP invention metric is a valid safety-relevant canary; (ii) matched perplexity is a fair dose match across operators; (iii) random-probe CKA/cosine/NLL fidelity is the right foil for ‘data-free quality’; (iv) two hand-set residual thresholds generalize. Free parameters are concentrated in the detector gate and canary design. No new physical entities; invented constructs are measurement instruments and statistics.

free parameters (5)
  • coherent_fraction gate threshold = 0.007
    Fixed cutoff >0.007 used with error_rate to label FAIL/PASS; fit to classify six labeled arms across two architectures.
  • error_rate gate threshold = 0.01
    Fixed cutoff >0.01 in the conjunction gate; chosen alongside coherent_fraction on the labeled arms/sweep.
  • coherent_fraction top-k singular energy = k=8
    k=8 in Eq. (1) defines how much leading singular energy counts as ‘coherent’; not derived from first principles.
  • perplexity acceptance band = 1.15×
    In-regime defined as ppl ≤1.15× baseline; standard practice but still a chosen guard that defines ‘gently-compressed’ blindspot cells.
  • SVD rank/budget operating points (e.g. b98, b≤0.975/0.98) = e.g. in-regime and dose cells per arch
    Compression doses selected to sit in- or out-of-guard and to illustrate onset; onset predictions are tied to these budget choices.
axioms (5)
  • domain assumption Cross-procedure invented-step count on synthetic SOPs under paraphrase is a diagnostic measure of agentic procedure confabulation.
    §2 defines invented_x as the deciding axis and requires a clean baseline; §7 admits absolute rates are instrument-specific.
  • domain assumption Matching WikiText-2 perplexity equates compression ‘dose’ across SVD, pruning, and quantization for causal comparison.
    §2–4 use matched-ppl arms as the primary control; mechanism claims depend on this commensurability.
  • domain assumption Data-free output fidelity under random Gaussian probes (CKA/cosine/NLL) is a valid fidelity oracle and fair member of the practitioner guard stack.
    §2–3 and Appendix A; dissociation is defined against this probe family.
  • ad hoc to paper Energy-weighted coherent_fraction and error_rate of ΔW sufficiently capture the behavioral failure axis across architectures.
    §5 introduces the two-axis statistic and fixed conjunction gate as the deployable screen.
  • standard math Standard linear-algebra and CI estimation tools (SVD tails, randomized subspace iteration, paired bootstrap/CIs) behave as usual.
    Used throughout §§3–5 for operators, detectors, and pre-registered canary analysis.
invented entities (2)
  • invented-step / cross-procedure invention canary no independent evidence
    purpose: Operationalize agentic SOP confabulation as a countable failure mode with paired CIs.
    Synthetic SOP bank plus invented_x metric is constructed for this paper; not an external standard benchmark.
  • coherent_fraction × error_rate compression-error screen no independent evidence
    purpose: Data-free pre-flight detector for builds that fail the canary.
    Defined in §5 Eqs. (1–2) with paper-specific aggregation and thresholds.

pith-pipeline@v1.2.0-daily-grok45 · 11806 in / 4070 out tokens · 82406 ms · 2026-07-31T14:58:45.926749+00:00 · methodology

0 comments
read the original abstract

Practitioners accept a compressed language model once it clears a stack of data-cheap quality guards: perplexity within a small factor of the original, downstream accuracy (for example MMLU) inside a confidence interval, and data-free output-fidelity signals that compare the compressed and original network's internal representations under random probe inputs. This stack has a blind spot. Across three model families, gently-compressed models clear every guard and then invent procedure steps that were never in the instructions when they run a standard operating procedure (SOP) as an agent. The effect is operator-specific: coherent low-rank (SVD) truncation induces it, and magnitude pruning matched to the same perplexity does not. One dissociation isolates the cause. The same compressed weights that CI-win a paired output-fidelity test CI-fail the invented-step canary. The governing axis is the coherence of the compression error times its rate; the magnitude of the damage does not predict it. The data-free fidelity probe is a fidelity oracle by construction, so it cannot see this axis. We characterize the blindspot and dissociation with paired confidence intervals on a pre-registered, powered canary across three architectures. Operator-specificity replicates on all three, and the perplexity-guard evasion appears where the model admits in-guard low-rank headroom. We then give a data-free screen: a two-axis statistic of the compression error (coherent-fraction and error-rate) that flags the failing builds with fixed thresholds across architectures and matches the coherence-times-rate mechanism. Perplexity, MMLU, and fidelity acceptance do not certify agent safety. Screen gently-compressed low-rank builds before agentic deployment

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

13 extracted references · 1 linked inside Pith

  1. [1]

    Mahoney, and Kurt Keutzer

    Zhen Dong, Zhewei Yao, Yaohui Cai, Daiyaan Arfeen, Amir Gholami, Michael W. Mahoney, and Kurt Keutzer. HAWQ-V2: Hessian aware trace-weighted quantization of neural networks. InAdvances in Neural Information Processing Systems (NeurIPS), 2020

    2020

  2. [2]

    Mahoney, and Kurt Keutzer

    Zhen Dong, Zhewei Yao, Amir Gholami, Michael W. Mahoney, and Kurt Keutzer. HAWQ: Hessian AWare quantization of neural networks with mixed-precision. InInternational Con- ference on Computer Vision (ICCV), 2019

    2019

  3. [3]

    SparseGPT: Massive language models can be accurately pruned in one-shot

    Elias Frantar and Dan Alistarh. SparseGPT: Massive language models can be accurately pruned in one-shot. InInternational Conference on Machine Learning (ICML), 2023

    2023

  4. [4]

    GPTQ: Accurate post- training quantization for generative pre-trained transformers

    Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. GPTQ: Accurate post- training quantization for generative pre-trained transformers. InInternational Conference on Learning Representations (ICLR), 2023

    2023

  5. [5]

    Charles A. E. Goodhart. Problems of monetary management: The UK experience.Monetary Theory and Practice: The UK Experience, pages 91–121, 1984. 5

    1984

  6. [6]

    Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey E. Hinton. Similarity of neural network representations revisited. InInternational Conference on Machine Learning (ICML), 2019

    2019

  7. [7]

    AWQ: Activation-aware weight quanti- zation for on-device LLM compression and acceleration

    Ji Lin, Jiaming Tang, Haotian Tang, Shang Yang, Wei-Ming Chen, Wei-Chen Wang, Guangx- uan Xiao, Xingyu Dang, Chuang Gan, and Song Han. AWQ: Activation-aware weight quanti- zation for on-device LLM compression and acceleration. InConference on Machine Learning and Systems (MLSys), 2024

    2024

  8. [8]

    Categorizing variants of goodhart’s law.arXiv preprint arXiv:1803.04585, 2019

    David Manheim and Scott Garrabrant. Categorizing variants of goodhart’s law.arXiv preprint arXiv:1803.04585, 2019

    Pith/arXiv arXiv 2019

  9. [9]

    Toolformer: Language models can teach themselves to use tools

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. InAdvances in Neural Information Processing Systems (NeurIPS), 2023

    2023

  10. [10]

    Zico Kolter

    Mingjie Sun, Zhuang Liu, Anna Bair, and J. Zico Kolter. A simple and effective pruning approach for large language models. InInternational Conference on Learning Representations (ICLR), 2024

    2024

  11. [11]

    SVD-LLM: Truncation-aware singular value decomposition for large language model compression

    Xin Wang, Yu Zheng, Zhongwei Wan, and Mi Zhang. SVD-LLM: Truncation-aware singular value decomposition for large language model compression. InInternational Conference on Learning Representations (ICLR), 2025

    2025

  12. [12]

    SmoothQuant: Accurate and efficient post-training quantization for large language models

    Guangxuan Xiao, Ji Lin, Mickael Seznec, Hao Wu, Julien Demouth, and Song Han. SmoothQuant: Accurate and efficient post-training quantization for large language models. InInternational Conference on Machine Learning (ICML), 2023

    2023

  13. [13]

    ReAct: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. InInternational Confer- ence on Learning Representations (ICLR), 2023. 6

    2023