{"id":"b5e63953-7044-461d-acbc-c2623c5817ad","arxiv_id":"2607.28884","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"ZK-verified LLM inference can be fooled: a provider can serve a small model while producing valid proofs for a much larger declared model by embedding structure-preserving ghost weights.","lead":"The paper shows that zero-knowledge proofs for LLM inference certify correctness of the equations but not the computation actually performed, so a server can declare a large model, embed a small one through crafted 'ghost' weights, and still pass verification. The result matters because it punctures the emerging zkML narrative that verifiable inference also verifies model scale and effort.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Attack B's output projections are ill-typed for the residual stream and the FFN construction is off by a factor of m; as written, width replication fails at the first layer, though Attack A and the core effort-gap claim survive.","rationale":"I read the paper's central claim as: existing zkLLM verification proves consistency of an output with a committed witness under a declared circuit, but does not bind the computational effort actually expended. This is conceptually sound and is already demonstrated by Attack A, which inserts zero-output residual blocks: with zero attention and FFN weights, each added block is algebraically the identity, so an L_in-layer inner model can be embedded in an L_out-layer outer model with identical outputs. That construction is parameter-free, training-free, and does not depend on the flawed width replication. The load-bearing weakness is therefore not the conceptual gap but one of the two concrete instantiations. Attack B's residual-stream dimensionality errors are real and easily checked: the paper itself defines the outer hidden state as m d-dimensional, yet both proposed output projections return d-dimensional vectors, making the residual add ill-typed. The FFN has an additional scale error. This means the paper's advertised 'two concrete families' is currently one solid family plus one broken family, and the A+B evaluation inherits the break. The appended IEEE S&P meta-review's blanket statement that the ghost-weight constructions are 'technically sound' is not supported for Attack B as written. However, because Attack A alone establishes the central claim and Attack B is repairable with modest changes to the output projections, the appropriate disposition is conditional acceptance rather than rejection. My read confirms the reader's verdict, so no adjustment is needed.","tokens_in":23576,"tokens_out":6976,"duration_ms":87253,"concrete_test":"Implement Attack B on a minimal pre-LN transformer (e.g., d_model = 4, m = 2) exactly following the formulas in Section 4.3.1, with tensor-shape assertions at every residual add. If the attention output projection W'_o or the FFN output projection W'_2 produces a vector of dimension d_model instead of m*d_model, the construction as written fails. Also check the FFN equality directly: if [W_2 W_2] applied to replicated activations yields 2*W_2*activation rather than W_2*activation, the missing 1/m factor is confirmed.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The weakest load-bearing point is Attack B (Section 4.3.1): the outer hidden state is defined in R^{m d_model}, but the proposed attention output projections W'_o = W_o S_1^T and W'_o = (1/m)[W_o ... W_o] both map R^{m d_model} -> R^{d_model}, not R^{m d_model}. The residual add X_att = X + H_att W'_o therefore adds a d-dimensional vector to an m d-dimensional stream. The same dimensional break occurs in the FFN: W'_1 = blkdiag(W_1,...,W_1) maps R^{m d_model} -> R^{m d_ff}, but W'_2 = [W_2 ... W_2] maps R^{m d_ff} -> R^{d_model}, so the FFN output is again d-dimensional rather than m d-dimensional. Moreover, even before the dimension mismatch, [W_2 ... W_2] applied to the replicated vector R y equals m W_2 y, not W_2 y; the 'exactly matches the inner FFN' claim in Section 4.3.1 is missing a 1/m factor. Thus the replicated-state invariant fails at the first residual add unless an undeclared broadcast/padding/summation step is inserted. This undermines one of the two advertised ghost-weight families and invalidates the Attack B and A+B rows of Table 2 as written. The issue is internal inconsistency, not merely disagreement with consensus. It is repairable (e.g., by using block-constant output projections that broadcast the inner output across all m blocks), and Attack A remains a valid, training-free depth-inflation construction, so the central 'effort gap' claim is not overturned.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper identifies an 'effort gap' in zero-knowledge LLM inference: a zk proof certifies that an output is consistent with a declared architecture and committed private weights, but does not bind the computational effort actually expended. It introduces the Hollow-LLM attack, in which a provider declares a large outer model, embeds 'ghost weights' that collapse effective computation, serves queries using a much smaller inner model, and still produces valid proofs for the outer circuit. Two algebraic constructions are given: Attack A, which inserts zero-work residual blocks to inflate depth, and Attack B, which replicates the hidden state across m coordinate blocks to inflate width. The paper also analyzes positional-encoding compatibility, reports experiments with a zkGPT-style pipeline, and discusses countermeasures. Attack A is a sound, training-free depth-inflation construction. Attack B, as written, has a dimensionality error in the output projections and a missing 1/m factor in the FFN that break the replicated-state invariant at the first layer.","tokens_in":23914,"tokens_out":6537,"duration_ms":79009,"significance":"The core observation — that proof of correctness of inference is not proof of large-model execution — is conceptually important and timely for the emerging zkML/zkLLM ecosystem. The paper gives a clean formalization of the threat model and a concrete, verifiable instantiation in Attack A, which convincingly demonstrates that zero-weight attention/FFN blocks act as exact residual identities under the verifier's circuit. The evaluation design cleanly separates serve-side from prove-side costs. However, the width-inflation construction (Attack B) is internally inconsistent as stated, and the empirical rows derived from it inherit the flaw. The central effort-gap claim survives, but the paper's advertised 'two training-free algebraic constructions' reduces to one sound construction plus a repairable but currently invalid one.","major_comments":[{"comment":"The outer hidden state is defined as x'_ell in R^{m d_model}, but the proposed output projections W'_o = W_o S_1^T and W'_o = (1/m)[W_o ... W_o] both map from R^{m d_model} to R^{d_model}. The residual add X_att = X + H_att W'_o is therefore ill-typed at the first attention block. A block-diagonal W'_o = blkdiag(W_o, ..., W_o) would preserve the replicated invariant, but as written the invariant fails and Attack B's correctness claim does not hold.","section":"Section 4.3.1, attention output projection"},{"comment":"For the FFN, W'_2 = [W_2 ... W_2] maps R^{m d_ff} to R^{d_model}, not R^{m d_model}. Moreover, applied to a replicated activation R u, W'_2 (R u) = m W_2 u, not W_2 u; the claimed equality 'W'_2 phi(W'_1 x' + b'_1) = W_2 phi(W_1 x + b_1)' is missing a 1/m factor. Biases b'_1 and b'_2 are also not specified in Attack B. The construction should use block-diagonal W'_2 = blkdiag(W_2, ..., W_2) and replicated biases.","section":"Section 4.3.1, FFN sublayer"},{"comment":"Because Attack B is ill-typed, the Attack B and A+B rows of Table 2 do not provide valid evidence for the width-inflation or composition claims. The reported 'Theoretical FLOPs' for Attack B (2.8x/3.9x) are also ambiguous: if these are for the declared outer circuit, they do not measure the executed computation; if they are for the executed model, they contradict the reported 1x serve time. The A+B and IM2 rows cannot be used to conclude that hollowing reproduces honest large-model proving cost until the underlying Attack B construction is repaired and the experiments are re-run.","section":"Table 2 and Section 5.2"}],"minor_comments":[{"comment":"After the definition of LayerNorm, the manuscript contains the stray sentence 'I have no way to know.' This appears to be an editorial artifact and should be removed.","section":"Section 2.1"},{"comment":"Appendix A reproduces a program-committee meta-review, including the statement that Attacks A and B are 'technically sound.' A meta-review is not part of a research submission, and in this case its assertion about Attack B is contradicted by the analysis above. This appendix should be removed.","section":"Appendix A"},{"comment":"The column grouping 'Serve Time' and 'Theoretical FLOPs' should be clarified. A reader could read the 1x serve time and the 2.8x theoretical FLOPs as contradictory; labeling the latter as 'declared-circuit FLOPs' or similar would resolve the ambiguity.","section":"Table 2"},{"comment":"The heading 'Semantic Squivalence' is a typo; it should read 'Semantic Equivalence'.","section":"Section 5.2"},{"comment":"The bulleted list of output-projection options has an empty bullet marker in the formatted text. This is likely a formatting issue from the source; please ensure the list renders correctly.","section":"Section 4.3.1"}],"recommendation":"major_revision","confidential_remarks":"The presence of Appendix A, which is stated to be a program-committee meta-review, is unusual and suggests the manuscript may not have been submitted in a clean form. The editor may wish to confirm the provenance of that appendix. The technical flaw in Attack B is substantial but repairable, and the core effort-gap contribution is independent of it."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague —\n\nThe thing to know about this paper: it has one construction that works and one that doesn't. Attack A, the depth-inflation via zero-work residual blocks, is a clean algebraic trick — zero out the attention and FFN weights in added blocks, and each block becomes x↦x exactly, so the outer model reproduces the inner model's output while the verifier sees extra depth. That alone makes the core point: a ZK proof of LLM inference certifies consistency of the output with a committed witness, not the computational effort spent to find it.\n\nAttack B, the width-replication via coordinate replication, has a type error. The proposed output projections, W'_o = W_o S_1^T or the averaging variant, map R^{m·d} → R^d, not R^{m·d}, so the residual add X + H_att W'_o adds a d-vector to an m·d stream. Same for the FFN output projection W'_2 = [W_2 … W_2], which both drops to d dimensions and, on replicated vectors, computes m·W_2 y instead of W_2 y. The replicated-state invariant fails at the first residual add. The fix is simple — use block-diagonal output projections that broadcast the inner output across all m blocks — but it needs to be stated and verified.\n\nThe paper also does some things well. The threat model is careful, and the discussion of positional encodings is honest: Attack A works with sinusoidal encodings; Attack B doesn't, and they say so. The conceptual framing — proof of correct inference is not proof of large-model execution — is important for the zkML community, and the paper cites the easy-witness literature rather than pretending the observation is new.\n\nThe soft spots beyond Attack B: the evaluation is thin. Table 2 reports normalized ratios with no code, no data, no error bars, and 'unchanged perplexity' is asserted without a number. The appended IEEE S&P meta-review claiming both attacks are technically sound is not supported by the text as written; that passage should go or be substantiated.\n\nBottom line: this deserves a serious referee. The central claim holds, Attack A is a solid instantiation, and the width scheme is repairable. I'd send it to review with a clear request to fix Attack B and open the evaluation.","headline":"Attack A cleanly demonstrates the effort gap; Attack B's width-replication construction has a dimension mismatch that breaks it as written — worth a revision, not a desk reject.","tokens_in":24443,"tokens_out":5863,"would_cite":true,"duration_ms":61819,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Zero-knowledge proofs of LLM inference certify that an output fits the declared equations, but not how much computation actually produced it — so a provider can pass verification while serving a much smaller model.","keywords":["zero-knowledge proofs","LLM inference","ghost weights","effort gap","model substitution","transformer invariances","proof of computation","zkML"],"falsifier":"Take the Attack B construction with m=2 on a standard pre-LN transformer and inspect the first residual add: if the attention output W'_o R x has dimension d while the residual stream has dimension md, the circuit as written has mismatched shapes and the witness editing fails; alternatively, evaluate both f_A_in(W_in, x) and f_A_out(W_ghost, x) under the verifier's quantized arithmetic on random prompts and compare — any mismatch would falsify the exact transcript-alignment claim.","tokens_in":23371,"feed_emoji":"👻","tokens_out":6918,"duration_ms":70654,"temperature":0.7,"pith_summary":"The paper argues that current zero-knowledge verification of LLM inference has an effort gap: a valid proof shows an output is consistent with a public architecture and committed private weights, but it does not show that the prover performed the computational work that architecture implies. On that basis it introduces the Hollow-LLM attack, in which a provider keeps the declared model size and parameter count but loads the architecture with ghost weights — zeroed or replicated blocks that make the larger circuit pass inputs through unchanged or reuse the same computation across replicated coordinates. The served outputs come from a small inner model, yet the verifier's circuit accepts them as coming from the large declared outer model. Two training-free constructions are given: Attack A inflates depth with zero-work residual blocks, and Attack B inflates width by replicating hidden coordinates and collapsing them at the output. If the paper is right, proof validity is not proof of faithful large-model execution, which matters for anyone relying on zero-knowledge-verified LLM services to attest model scale.","feed_headline":"A small LLM can pass zero-knowledge proof for a big model","feed_subtitle":"Valid proofs certify the equations, not the compute behind them — providers pocket the difference.","key_machinery":"The load-bearing object is the ghost-weight witness: a set of weights for the declared outer architecture that exactly reproduces the inner model's input–output function while the actual service runs only the inner model. Four algebraic invariances of the pre-LN transformer carry the construction: the residual identity (a block whose attention and feed-forward outputs are exactly zero acts as x ↦ x), FFN zero-padding (appending zero-weight neurons leaves the FFN output unchanged), LayerNorm replicate invariance (replicated hidden states with replicated LayerNorm parameters normalize identically), and block-diagonal attention (replicated projections with replicated inputs produce identical at","core_discovery":"The central claim is that a zero-knowledge proof of inference certifies membership in an NP relation — that there exist private weights for the declared architecture producing the output — but never the algorithmic path or the amount of computation. The paper shows this gap is exploitable through ghost weights, explicit algebraic structures that make the outer model under ghost weights equal the inner model on every supported prompt. A dishonest provider can therefore serve all queries with the cheap inner model, then, during an audit, produce a valid proof against the expensive outer circuit by supplying the ghost weights and replicated or zeroed witnesses; the proof is indistinguishable fr","pith_inferences":["The same effort gap likely extends beyond LLM inference to any zero-knowledge proof of a deterministic computation with a private witness and no metered resource: if the relation admits an easy witness, proof validity never binds prover cost; the transformer-specific ghost weights are one instance of a general phenomenon.","Because proof transcript alone cannot distinguish hollow from honest runs, countermeasures must come from outside the proof system — behavioral audits, challenge-based probes, ablation tests, or hardware attestation with measured execution become necessary, not optional, if the attack is real.","The real-world viability hinges on whether users and pricing schemes reward declared parameter count more than observed quality; the paper leaves the detection threshold as a perception-management problem, so the practical attack surface is an empirical question.","The paper's Attack B construction, as written, needs a declared dimension-collapsing step at each residual add; until the paper specifies such a step, the width-inflation variant is best read as a design for a modified architecture rather than a drop-in attack on stock pre-LN transformers."],"forward_implications":["Accepted proofs from hollowed deployments are distributed the same way as honest proofs, so neither clients nor verifiers can distinguish a hollowed model from a genuine one by proof transcripts alone.","Serve-time cost tracks only the inner model while proving cost tracks the declared outer circuit, so the provider captures the difference as profit; the paper's experiments show serve latency unchanged at 1× while gate count and prover time scale with declared depth and width.","The attack works without modifying the verifier, the circuit, the commitment scheme, or the proof system; it is a deployment-level assurance gap, not a soundness flaw.","The constructions are training-free and compose: depth expansion and width expansion can be applied together, reproducing the proving cost of a large honest model while executing a small one.","The paper states one concrete limitation: deterministic sinusoidal positional encodings rule out Attack B, since the outer circuit fixes positional values as a public function of width, leaving learned and rotary encodings compatible."],"fun_headline_variants":["ZK proofs verify equations, not compute — Hollow-LLM exploits the gap","Small model can fake big-model ZK proof via ghost weights","Zero-knowledge doesn't prove effort: Hollow-LLM attack explained","Ghost weights let providers pass ZK checks with a tiny model","Proof of correctness isn't proof of compute — Hollow-LLM attack"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"Everything depends on the ghost weights reproducing the inner model's outputs exactly under the verifier's fixed arithmetic, and Attack B additionally assumes the wider residual stream can be collapsed back to the original dimension at each attention and feed-forward block — a step the paper does not explicitly declare — so the replication invariant is preserved through the first residual add.","fun_headline_variants_meta":{"raw":{"variants":["ZK proofs verify equations, not compute — Hollow-LLM exploits the gap","Small model can fake big-model ZK proof via ghost weights","Zero-knowledge doesn't prove effort: Hollow-LLM attack explained","Ghost weights let providers pass ZK checks with a tiny model","Proof of correctness isn't proof of compute — Hollow-LLM attack"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000634,"raw_usage":{"total_tokens":2776,"prompt_tokens":774,"completion_tokens":2002,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":518,"completion_tokens_details":{"reasoning_tokens":1909}},"tokens_in":518,"tokens_out":2002,"duration_ms":14433,"temperature":1.0,"reasoning_tokens":1909,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T01:25:18.052158+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the Attack B construction with m=2 on a standard pre-LN transformer and inspect the first residual add: if the attention output W'_o R x has dimension d while the residual stream has dimension md, the circuit as written has mismatched shapes and the witness editing fails; alternatively, evaluate both f_A_in(W_in, x) and f_A_out(W_ghost, x) under the verifier's quantized arithmetic on random prompts and compare — any mismatch would falsify the exact transcript-alignment claim.","supporting_citations":[],"review_version":1}