{"id":"57bab7c5-ce32-42c3-b661-7af7b9fb9eb1","arxiv_id":"2608.08029","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"Probes trained on final-layer activations detect harmful prompts with similar F1 on four open-weight LLMs, and activations are byte-identical across seeds in this setup.","lead":"This paper re-runs a published study that used small neural networks to read safety signals directly from inside large language models, and checks whether the trick works on three other models. It also finds that, in their test setup, the internal signals they extract are identical no matter the random seed.","discovery_kind":"replication","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Main tables and Appendix B report contradictory per-seed F1 values for identical model/dataset pairs, so the cross-architecture and zero-variance claims are not currently backed by a consistent numerical record.","rationale":"The paper is a good-faith reproducibility study with transparent limitations: it explicitly hedges the zero-variance finding as protocol-specific and acknowledges Doda (2026) as a final-token coverage limitation. I therefore do not attack those claims as primary. The load-bearing problem I find is internal and more basic: the main tables and the per-seed appendix disagree numerically for the same reported settings. The reader's rationale already flagged this as one of three conditional-acceptance issues, but the reader's designated weakest assumption was the final-token signal coverage, not this inconsistency. My stress-test lands on the numerical contradiction because it is the point where the central claim's evidence is least secure: if the per-seed F1 values are actually those in Appendix B, then Table 6's constant rows are misreported, and if Table 6 is correct, then Appendix B is misreported and cannot serve as the raw-results supplement. In either case, the paper's quantitative backbone is not verifiable as written. The condition is resolvable by releasing the raw logs and a single fixed checkpoint-selection rule, so the reader's CONDITIONAL verdict remains appropriate; my concern does not move the verdict but strengthens the condition. I credit the paper for including the SHA-256 diagnostic as a direct check of activation determinism, for reporting architectural diversity, and for refusing to overclaim the non-determinism result in Section 7; those elements are not in question here.","tokens_in":9375,"tokens_out":9300,"duration_ms":94008,"concrete_test":"Request or obtain the raw per-seed logs and the exact checkpoint-selection rule behind Tables 5, 6, and B.1. Recompute F1 for every (model, dataset, seed) using one fixed rule (e.g., best validation epoch per Section 3.1), including the missing seed-42 rows. Then verify whether the five seed F1 values are exactly equal and whether the cross-architecture values stay within one point of the LLaMA baseline. If Appendix B values are correct, Tables 5/6 are misreported; if Tables 5/6 are correct, Appendix B is misreported. Either outcome requires a correction and settles whether the central numbers are trustworthy.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Tables 5/6 and Appendix B report irreconcilable F1 values for the same (model, dataset) pairs. Table 6 lists per-seed F1s equal to Table 5 for all five seeds: LLaMA/BeaverTails 82.90, Gemma/BeaverTails 83.35, Mistral/BeaverTails 83.32, Qwen/BeaverTails 83.39, LLaMA/AEGIS 83.42. Appendix B Table 11 gives per-seed F1s of 82.49, 82.97, 82.92, 83.01, and 83.48 for the same pairs — 0.33 to 0.48 points lower — and omits seed 42 throughout, plus one seed for Qwen. The appendix note that these runs 'might not use the most efficient probe model checkpoint' does not reconcile the discrepancy, because checkpoint selection is part of the pipeline and the main text asserts a single constant value per pair across seeds. Either the main tables or the appendix is not reporting actual per-seed test F1, so the zero-variance observation and the cross-architecture deltas are not backed by a consistent dataset. This is a numerical contradiction that must be resolved before any headline claim can be accepted.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reproduces the latent-space safety-probe pipeline of Khatri et al. (2026), in which a lightweight MLP is trained on final-layer, last-token activations of an LLM to classify harmful prompts, and extends it in two directions: (1) applying the identical pipeline to three additional backbones (Gemma-4-E4B, Mistral-7B-v0.3, Qwen2-7B) on WildJailbreak, BeaverTails, and AEGIS 2.0, and (2) repeating extraction under five seeds to measure variance in F1 and in the extracted activation tensors. The reported reproduction is within 0.37 F1 points of the original LLaMA numbers, the cross-architecture F1 values are within about a point of the original model, and the per-seed F1 values are reported as constant, with SHA-256 digests of the serialized activation tensors identical across seeds. The authors position the work as a reproducibility study with multi-seed statistical controls and release code, activations, and probe checkpoints.","tokens_in":9632,"tokens_out":6158,"duration_ms":62599,"significance":"If the numerical record is reliable, this is a useful and appropriately cautious reproducibility contribution: it provides evidence that linear-probe safety detection transfers across several open-weight model families, and it introduces a concrete SHA-256 diagnostic for distinguishing seed-invariant activations from downstream probe stability. The multi-seed protocol and the explicit discussion of what the zero-variance result does and does not claim are good practices. The main caveat is that the central empirical claims currently rest on an inconsistent set of F1 tables, and several claims in the abstract and contributions are broader than the executed protocol. With those issues resolved, the paper would be a solid contribution to the reproducibility literature, though its incremental nature means it should be judged as a workshop-style reproducibility study rather than a large conceptual advance.","major_comments":[{"comment":"The per-seed F1 values in Table 6 and Appendix B are irreconcilable for the same (model, dataset) pairs. For example, LLaMA/BeaverTails appears as 82.90 in Table 6 but 82.49 in Table 11 for seeds 75/456/789/1024; Gemma/BeaverTails appears as 83.35 versus 82.97; Mistral/BeaverTails as 83.32 versus 82.92; Qwen/BeaverTails as 83.39 versus 83.01; and LLaMA/AEGIS appears as 83.42 versus 83.48. The Appendix B note that these runs 'might not use the most efficient probe model checkpoint' does not reconcile the discrepancy, because checkpoint selection is part of the pipeline and the main text asserts a single constant value per pair across seeds. Either Table 6 or Table 11 is not reporting the actual per-seed test F1, so the zero-variance observation and the cross-architecture deltas are not supported by a consistent numerical record. This must be fixed before the headline claims can be accepted.","section":"§5.3/Table 6 and Appendix B/Table 11"},{"comment":"The contribution bullet stating that the paper quantifies 'what fraction of the original paper's reported margin over baseline guard models survives this noise' is not delivered anywhere in Sections 5 or 6. The manuscript reports probe F1 values and seed variance, but it does not report guard-model F1 values, margin calculations, or any fraction of the margin surviving noise. The authors should either add the missing guard-model comparison or remove this bullet from the contributions.","section":"§1, Contributions"},{"comment":"The non-determinism conclusion is broader than the protocol supports. The protocol varies seed and batch order under a single inference stack (nnsight, batch size 5, no chunked prefill), and Section 6 explicitly acknowledges that these settings may be inadequate to exercise non-deterministic prefill behavior. In addition, Table 7 checks only one dataset (BeaverTails) for each non-LLaMA backbone, plus LLaMA/AEGIS, so the abstract's statement that 'final token latent vectors remained the same for all tested architectures irrespective of the seed values used' should be qualified to 'under the tested extraction configuration and datasets.'","section":"§3.3 and §5.4"},{"comment":"The paper does not disclose that the reproduced work, Khatri et al. (2026), shares its first author with this manuscript. Since one contribution is described as a 'faithful reproduction' used to calibrate the extension, the author overlap should be stated explicitly, and the text should clarify that the LLaMA-3.1-8B calibration is an author-run reproduction rather than an independent replication by a separate group.","section":"§1 and reference list"}],"minor_comments":[{"comment":"The caption says 'First row reproduces the original paper's setting,' but the LLaMA-3.1-8B row shows values of 99.1/82.9/83.5 rather than the reproduced values of 99.47/82.90/83.42 from Table 4. This should be labeled as the original reported values, not the reproduction values, to avoid confusion in the cross-architecture deltas.","section":"Table 5 caption"},{"comment":"Appendix B omits seed 42 for all pairs and one Qwen seed without explanation; the authors should provide the complete per-seed log or state why those runs are excluded.","section":"Appendix B, Table 11"},{"comment":"The text says 'standard deviation is 0 throughout,' but the values are reported to two decimal places; the claim should be phrased as 'zero to the reported precision,' since unrounded F1 values could still have small nonzero variance.","section":"§5.3"},{"comment":"The reference 'Arney Agrawal' should be 'Amey Agrawal' for the chunked-prefill paper; please check spelling against the published version.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The undisclosed author overlap between this manuscript and the reproduced paper should be addressed in revision; it is not disqualifying, but the calibration value is weakened by the lack of independence. The numerical contradiction between Table 6 and Appendix B is the main substantive barrier: if the appendix is a real run log, the main tables need to be regenerated or the appendix corrected. This is within the scope of a reproducibility track, but the paper would need a full consistency pass before acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. The cross-architecture numbers are genuinely new and useful: identical MLP probes on Gemma, Mistral, and Qwen land within a point of the LLaMA numbers on three benchmarks. And the SHA-256 digest check is a clean way to show their extraction stack is seed-invariant. But the paper has a numerical contradiction between the main tables and Appendix B, and the first author is reproducing his own earlier paper without disclosing it. Both need to be fixed before the results can be trusted.\n\nWhat's good: the reproduction deltas are small (within 0.37 F1), the probe architecture is held fixed, and the authors are unusually honest in the limitations section, citing Doda (2026) on final-token probes missing jailbreaks and explicitly saying the zero-variance result is 'no variance detected by our current protocol' rather than a general claim. That's the right framing.\n\nThe soft spots are real. Table 6 reports per-seed F1 of 82.90 for LLaMA/BeaverTails; Appendix B reports 82.49 for the same pair under the same seeds, with a note that the runs 'might not use the most efficient probe model checkpoint.' That doesn't reconcile the numbers. Checkpoint selection is part of the pipeline, and the main text asserts a single constant value per pair. Either the appendix is reporting different checkpoints and the main table is cherry-picking, or one of the tables is wrong. The zero-variance observation itself survives in both tables (constant across seeds), so the determinism claim is less damaged than the stress-test suggests. But the absolute F1 values, and therefore the cross-architecture deltas and the reproduction delta, are not backed by a consistent record. That's load-bearing for a reproducibility study.\n\nSecond, the author overlap. Khatri is first author of the reproduced paper and of this one. The paper never says so. That makes the 'independent reproduction' framing misleading. It's not disqualifying by itself, but it must be disclosed, and the referee should look harder at whether the extension is cherry-picked. The code URL is also malformed (LaTeX escapes), so the promised artifacts aren't actually accessible. For a reproducibility paper, that's embarrassing if not fatal.\n\nWho this is for: people working on latent-space safety probes and guardrail alternatives. The core question—does a linear safety probe transfer across model families—is worth answering, and this is one of the few direct attempts. With the numerical inconsistency fixed, author overlap disclosed, and artifacts actually released, it would be a useful contribution.\n\nRecommendation: send it to peer review, but a referee should insist on a reconciliation of the tables and an explicit disclosure note. If the authors can't explain the discrepancy, the paper shouldn't be published as is.","headline":"Useful cross-family probe numbers and a clean determinism check, but the main tables contradict the appendix and the first author is reproducing his own work without disclosure.","tokens_in":10147,"tokens_out":2945,"would_cite":false,"duration_ms":26432,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A cheap latent-space safety probe works across four LLM families and is seed-invariant in this setup.","keywords":["latent safety probes","harmfulness detection","LLM interpretability","reproducibility study","MLP probe","final-token activations","cross-architecture generalization","seed invariance"],"falsifier":"Take a jailbreak set whose unsafe content is distributed across earlier user-token representations, run the probe and a large guard model on identical prompts, and compare F1; the paper's Section 7 cites evidence that final-token probes can miss such prompts, so a clear F1 collapse on that set would falsify the generality of the high benchmark scores. Separately, rerunning activation extraction through a different inference implementation and comparing SHA-256 digests of the tensor files would settle whether the seed-invariance is a property of the models or of the current serving stack.","tokens_in":9134,"feed_emoji":"🛡️","tokens_out":7301,"duration_ms":69261,"temperature":0.7,"pith_summary":"This paper is a reproducibility study of the claim that a small MLP probe on the final-layer, last-token activation of an LLM can detect harmful prompts about as well as guard models thousands of times larger. The authors re-implement the original single-model pipeline on LLaMA-3.1-8B and match its F1 scores to within 0.37 percentage points. They then train identical probes on Gemma-4-E4B, Mistral-7B-v0.3, and Qwen2-7B, and find F1 scores within roughly one point of the original on WildJailbreak, BeaverTails, and AEGIS 2.0. Repeating extraction under five random seeds shows the final-token latent vectors are byte-for-byte identical across seeds, so probe F1 variance is zero in this setup.","feed_headline":"Safety probes on final-layer activations work across model families","feed_subtitle":"Final-token vectors are identical across seeds, so the probe's high F1 is reproducible on four open-weight LLMs.","key_machinery":"The carrying mechanism is the final-layer, last-token hidden state after prompt prefill, i.e. the vector $h_i = M(x_i)[-1] \\in \\mathbb{R}^d$ for prompt $x_i$ and backbone $M$, followed by a fixed six-layer MLP probe with hidden sizes 2048, 512, 64 and a two-way softmax output. The MLP's input dimension is the only quantity that changes across backbones. This vector is the entire input to the probe; the empirical claim is that it already contains linearly decodable harmfulness information, and the determinism check (SHA-256 hashes of the extracted tensor files) shows the staging step is a pure function of weights and tokens in this setup.","core_discovery":"On the paper's own terms, the central discovery is twofold: the latent-space probing recipe is portable, and the extraction step is deterministic under the tested settings. Identical MLP probes trained on final-layer, last-token activations reach F1 scores of 99.47 on WildJailbreak, 82.90 on BeaverTails, and 83.42 on AEGIS 2.0 for the reproduced LLaMA-3.1-8B backbone, and stay within roughly a point of those values for Gemma-4-E4B, Mistral-7B-v0.3, and Qwen2-7B. Across five random seeds, the serialized activation tensors from Stage 1 are byte-for-byte identical (SHA-256 digests match), and F1, precision, and recall are constant to at least two decimal places. The paper reads this as evidence that harmfulness is linearly represented in the final token across current aligned open-weight chat models, while cautioning that the determinism result is protocol-specific.","pith_inferences":["Beyond the paper, the byte-for-byte seed invariance implies the extraction pipeline is a pure function of weights and tokens in this setup; if that holds on other serving stacks, probe training results become fully deterministic and reproducibility artifacts need only be produced once.","The paper's cited limitation suggests final-token probes can miss jailbreaks whose evidence is spread across earlier tokens, so one testable extension is to pool activations across all user-token positions and check whether F1 rises on adversarial jailbreak sets while staying stable on the standard benchmarks.","A further extension is to measure whether the zero-variance result survives when the same model is run through a different inference backend, a different GPU generation, or with chunked prefills enabled; any digest mismatch would delimit the determinism claim."],"forward_implications":["The same probe hyperparameters transfer across Llama, Gemma, Mistral, and Qwen backbones without per-family tuning, which makes the pipeline a plausible drop-in replacement for a guard model.","Because activation extraction is seed-invariant here, a single extraction run is enough to judge a model's probe performance; multi-seed averaging adds nothing for these backbones and datasets.","The F1 spread between the best and worst backbone is about one point, so alignment recipe differences among open-weight chat models do not materially change the harmfulness information carried by the final token.","If the result holds beyond the 4B-8B open-weight range, latent probing could replace expensive external classifiers in production moderation pipelines."],"supporting_citations":[{"why":"Original two-stage pipeline and the benchmark F1 scores that this study reproduces and extends.","marker":"Khatri et al. (2026)"},{"why":"Supplies the WildJailbreak benchmark used to probe harmful-prompt detection.","marker":"Jiang et al. (2024)"},{"why":"Supplies the BeaverTails safety dataset and its train/validation/test split.","marker":"Ji et al. (2023)"},{"why":"Supplies the AEGIS 2.0 risk-taxonomy dataset used for the third benchmark.","marker":"Ghosh et al. (2025)"},{"why":"Provides the nnsight extraction tool the pipeline uses to read hidden states.","marker":"Fiotto-Kaufman et al. (2024)"},{"why":"Cited in Section 7 as evidence that final-token probes can miss jailbreaks with distributed unsafe evidence; this bounds the reproduced claim.","marker":"Doda (2026)"},{"why":"Shows harmfulness and refusal occupy separate latent directions, supporting the probe's focus on harmfulness.","marker":"Zhao et al. (2025)"}],"fun_headline_variants":["LLM safety probes: portable across families, identical across seeds","Probes on final-token activations transfer across LLMs and seeds","Same latent probe works on 4 LLM families; seed variation changes nothing","Latent safety probes stay accurate across four LLMs, seeds don't matter","Final-token activations are seed-invariant across four LLMs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole result rests on the assumption that the final-layer activation at the last token after prompt processing carries enough harmfulness signal for detection, so prompts whose unsafe evidence is spread across earlier tokens would be invisible to this probe.","fun_headline_variants_meta":{"raw":{"variants":["LLM safety probes: portable across families, identical across seeds","Probes on final-token activations transfer across LLMs and seeds","Same latent probe works on 4 LLM families; seed variation changes nothing","Latent safety probes stay accurate across four LLMs, seeds don't matter","Final-token activations are seed-invariant across four LLMs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001367,"raw_usage":{"total_tokens":5599,"prompt_tokens":1057,"completion_tokens":4542,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":673,"completion_tokens_details":{"reasoning_tokens":4447}},"tokens_in":673,"tokens_out":4542,"duration_ms":34160,"temperature":1.0,"reasoning_tokens":4447,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T00:32:19.853722+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a jailbreak set whose unsafe content is distributed across earlier user-token representations, run the probe and a large guard model on identical prompts, and compare F1; the paper's Section 7 cites evidence that final-token probes can miss such prompts, so a clear F1 collapse on that set would falsify the generality of the high benchmark scores. Separately, rerunning activation extraction through a different inference implementation and comparing SHA-256 digests of the tensor files would settle whether the seed-invariance is a property of the models or of the current serving stack.","supporting_citations":[],"review_version":1}