{"id":"331ff5d5-111e-44e8-b8b3-3c922beca444","arxiv_id":"2509.17932","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":1,"one_line_summary":"TruthV detects true answers by majority-voting the argmax/argmin preferences of a sparse set of MLP value vectors selected on 30 labeled examples.","lead":"Researchers show that a small set of 'value vectors' inside the MLP layers of large language models consistently respond differently to true and false multiple-choice answers. Their method, TruthV, votes among these vectors to detect truthful answers, beating prior training-free detectors on the NoVo benchmark without training a classifier.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"30-example support-set selection among hundreds of thousands of value vectors is likely unstable; Table 2's large full-training-set gains suggest the reported TruthV margins may depend on the fixed support set.","rationale":"The reader's weakest assumption is exactly the load-bearing link: 30 labeled examples are enough to select a top-0.1% subset of value vectors whose preferences transfer to held-out questions. My stress-test found this assumption to be the least secure part of the central claim. The selection pool is enormous (hundreds of thousands of value vectors) compared with NoVo's ~1k attention heads, so the 30-sample accuracy ranking is vulnerable to selection noise; the paper reports no variance estimate or null-label control. Table 2 provides internal evidence that the ranking is not converged, with full-training-set selection giving much larger gains on several datasets. I do not see an internal inconsistency in the method itself, and the held-out evaluations are plausible. The right response is to keep the CONDITIONAL verdict and require the resampling/overlap test (or equivalent) before the 30-sample claim is accepted. The test-set-dependent choice of p in Section 5.3 is a secondary concern: p is a single scalar and the peak is broad, but it should still be reported with a fixed validation-split selection rather than test-set curves. The interpretive claim about 'truthfulness' is also broader than what the MCQ accuracy metric supports, but the most concrete, falsifiable weakness is the stability of the 30-example selection.","tokens_in":12452,"tokens_out":9040,"duration_ms":79315,"concrete_test":"For Gemma-2-2B-it and Llama-3.2-3B-Instruct on QASC and SWAG, draw 100 random 30-example support sets from the training set (stratified by label) and run the full TruthV(argmax) pipeline of Section 5.1 with p=0.001, evaluating on the fixed validation set. Report the mean and standard deviation of accuracy and the average pairwise Jaccard overlap of the selected top-0.1% value-vector index sets. If the accuracy standard deviation exceeds 2 points or the average overlap is below 0.5, the single fixed support set in Table 1 is not a stable basis for the claimed advantage.","verdict_should_be":"UNCHANGED","load_bearing_attack":"TruthV's selection step (Section 5.1) ranks every MLP value vector by its argmax/argmin accuracy on the 30 NoVo examples and keeps the top 0.1%. For Llama-2-7B this is a selection among ~352k scalar predictors (d'=11008 × 32 layers) using 30 labels, while NoVo selects among only ~1k attention heads; the multiple-comparisons burden is orders of magnitude larger. With 30 examples, many vectors will have high accuracy by chance, and the fixed support set gives no variance estimate. The paper's own Table 2 shows the 30-example ranking is not converged: on Llama-3.2-3B-Instruct, QASC accuracy jumps from 50.11 (30 samples) to 71.60 (full training set) and CICv1 from 37.20 to 53.23. Although Table 2's full-set column uses test-set p tuning, the size of the gap indicates the 30-example selection is far from reliable. If the provided 30 examples are an unrepresentative draw, the headline gains (e.g., 70.33 vs 61.67 on Gemma-2-2B-it) may be artifacts of that draw rather than evidence that sparse value vectors encode truthfulness.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes TruthV, a training-free method for detecting the truthfulness of LLM outputs by exploiting individual MLP value vectors. Under the key-value memory interpretation of MLPs, TruthV ranks all value vectors by their argmax/argmin accuracy on 30 labeled multiple-choice examples, selects the top p=0.1% of vectors, and aggregates their per-question predictions by majority voting. The method is evaluated on the NoVo benchmark across four instruction-tuned LLMs (2B-13B) and ten datasets, reporting consistent improvements over NoVo and log-likelihood baselines (e.g., 70.33% vs. 61.67% average accuracy on Gemma-2-2B-it). The paper also presents analyses of the layer-wise distribution, interpretability, and cross-dataset generalization of the selected value vectors, concluding that truthfulness-related variation is captured in a sparse and structured manner in MLP value vectors.","tokens_in":12724,"tokens_out":3551,"duration_ms":35239,"significance":"If the reported results hold, this is a useful contribution: it extends the training-free truthfulness-detection paradigm beyond attention heads to MLP modules, offers a simple and interpretable mechanism (individual value-vector voting), and provides evidence that MLP value vectors encode content-level properties beyond token-level concepts. The method is conceptually clean and does not require classifier training. The main evaluation is broad (4 models, 10 datasets) and the held-out test accuracy is consistently higher than strong baselines. However, the significance is contingent on two load-bearing methodological points: the honest selection of the hyperparameter p, and the stability of vector selection given only 30 labeled examples. Currently both points are insufficiently supported, which tempers confidence in the headline numbers.","major_comments":[{"comment":"The hyperparameter p=0.001 is chosen by peaking accuracy curves that appear to be computed on the same evaluation sets used for Table 1. The text states 'we compute the average accuracy across all four models and find that the performance peaks around p=0.001' without specifying that a separate validation set was used. If the curves in Figure 5 are test-set accuracies, then the hyperparameter is tuned on the test data, and the reported improvements are optimistic. This is load-bearing because all headline results in Table 1 use p=0.001. The authors should either use a validation split for selecting p, or report results across a range of p without claiming the peak as the method's performance, or justify that the choice is stable and not overfit.","section":"§5.3, Figure 5"},{"comment":"The 30-example support set is used to rank up to ~352k value vectors (d'×L), yet the paper provides no stability analysis. Table 2 shows that using the full training set can improve results dramatically: for Llama-3.2-3B-Instruct, QASC accuracy jumps from 50.11 to 71.60 and CICv1 from 37.20 to 53.23. These gaps indicate that the 30-example ranking is far from converged. Since the full-training-set column in Table 2 uses test-set p tuning, the comparison is not apples-to-apples, but the magnitude of the gap still suggests that the fixed 30 examples may be an unrepresentative draw. Without variance estimates, multiple random support sets, or a comparison to random vector selection, the reported gains (e.g., 70.33 vs. 61.67 on Gemma-2-2B-it) could be artifacts of the specific support set. This is a core limitation of the method's claimed low-resource capability.","section":"§5.1, §5.4, Table 2"},{"comment":"The 'existence of truthfulness-related value vectors' analysis is performed on the full training set, as stated in Section 4.1 ('We use the full training set of each dataset except for TQA during this experiment'). Ranking value vectors by training-set accuracy and observing a high-accuracy subset is not evidence of generalization and may reflect overfitting, especially with up to hundreds of thousands of candidate vectors. To support the claim that these vectors genuinely encode truthfulness, the analysis should be repeated on held-out data, or at least the paper should acknowledge that the existence claim is only about training-set correlation. Table 3 partially addresses generalization, but only for the argmax pattern and under full-training selection; the main existence claim and Figure 2 should be re-evaluated or reframed.","section":"§4.1, §4.2, Figure 2"}],"minor_comments":[{"comment":"The caption of Figure 3 says 'top 0.01%' but the body text in Section 4.2 says 'top 0.1%'. This inconsistency should be corrected.","section":"Figure 3 caption vs. §4.2"},{"comment":"Typo: 'the the ARC-Easy training set' should be 'the ARC-Easy training set'.","section":"§4.1"},{"comment":"The baseline NoVo's configuration (e.g., its p value or number of selected heads) is not described in Section 5.1. To ensure reproducibility and fair comparison, the authors should state how the NoVo baseline is instantiated (e.g., which hyperparameters are used and whether they follow the original paper).","section":"General"},{"comment":"The text mentions 'the red dash line' to indicate random guess accuracy, but the figure caption does not explain this. Please add a caption note or ensure the line is visible/described.","section":"§4.2, Figure 2"},{"comment":"Minor typo: 'opposite tend' should be 'opposite trend'.","section":"§4.2"},{"comment":"The paper mentions t-SNE visualizations ('t-SNE visualizations of these selected value vectors reveal no clear clustering'), but no such figure is included. Either add the figure or remove the claim.","section":"§4.2"},{"comment":"Table 2 omits TQA because it has no training set; this should be explicitly noted in the caption or text for clarity.","section":"Table 2"},{"comment":"The 'combined' activation pattern is shown only in Figure 5, not in Table 1. If combining argmax and argmin is claimed to slightly improve accuracy, a quantitative comparison in the main table would strengthen the claim.","section":"§5.3"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Main take: this is a reasonable paper, not a breakthrough. It generalizes NoVo's idea from attention heads to individual MLP value vectors, showing that a sparse subset of them — selected on 30 labeled examples and aggregated by majority vote — beats NoVo's attention-head voting on the same benchmark across four instruction-tuned LLMs. That is a genuinely new result: nobody had looked at value-vector-level argmax/argmin patterns for truthfulness detection.\n\nWhat the paper does well: the experimental setup follows NoVo's benchmark, so the comparison is direct and fair in the sense that all methods get the same 30 examples. The held-out test evaluation is standard, and the improvements are consistent: TruthV wins 9/10 datasets on the argmax pattern for Gemma-2-2B-it, and the average gains are ~8 points over NoVo. The ablations over p are useful, and the cross-dataset generalization results (Table 3) are a good addition — they show above-random transfer, which supports at least a partial claim of truthfulness-related signal rather than pure dataset-specific memorization.\n\nThe soft spots are real, though. First, Figure 5 tunes p on the test-set accuracy curves; the paper then claims p=0.001 is a 'unified setting.' That's peeking, and it inflates the headline numbers. The curves are flat near the peak, so the inflation is probably small — but it should be reported honestly, e.g., by separating a validation split or using a fixed range.\n\nSecond, and more serious: the selection step ranks hundreds of thousands of value vectors on 30 labels. With a pool that large, good-looking accuracy by chance is guaranteed. There are no error bars anywhere, and no code, so we cannot see the variance across different 30-example draws. The paper's own Table 2 is the strongest evidence that the 30-sample selection is not converged: on Llama-3.2-3B-Instruct, QASC jumps from 50.11 (30 examples) to 71.60 (full training set), and CICv1 from 37.20 to 53.23. That's a 20-point swing, which makes me wonder how much of the reported margin is a lucky draw on the default 30 examples.\n\nThird, Section 4.2's 'existence' analysis ranks and evaluates on the same full training set. That's circular. It's fine as an exploratory illustration, but the text presents it as evidence for the phenomenon; it should be labeled as a descriptive property of the training set, not a discovery that transfers.\n\nThe paper's central claim — that sparse MLP value vectors carry truthfulness-related signal and can be aggregated — holds up in a broad sense, but the exact magnitude of the improvement over NoVo is uncertain. I would not trust the absolute numbers without a stability analysis.\n\nBottom line: this deserves a serious referee. The idea is novel enough within the subfield, the benchmark is standard, and the method is simple enough to re-implement. Ask for error bars, a sensitivity analysis over the 30-example support set, and an honest treatment of the p selection.","headline":"TruthV is a sensible incremental extension of NoVo to MLP value vectors with real held-out gains, but the 30-example selection step is fragile and the paper peeks at the test set to set p.","tokens_in":13252,"tokens_out":3217,"would_cite":true,"duration_ms":29426,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A training-free method that votes across sparse MLP value vectors detects truthfulness in LLMs better than attention-head and likelihood baselines.","keywords":["truthfulness detection","MLP value vectors","training-free","hallucination","multiple-choice QA","internal representations","majority voting","sparse selection"],"falsifier":"Run TruthV with the 30 support examples replaced by 30 randomly chosen incorrect answers labeled as 'correct.' If TruthV still scores above random on the real test set, the value-vector ranking is not causal; if accuracy collapses to chance, the selection step is load-bearing. A complementary check: measure the overlap between the top-0.1% vectors selected from the 30 samples and the top-0.1% selected from the full training set; near-zero overlap would indicate the 30-sample ranking is noise.","tokens_in":12282,"feed_emoji":"🗳️","tokens_out":4776,"duration_ms":213246,"temperature":0.7,"pith_summary":"The paper tries to establish that truthfulness-related information in large language models is concentrated in a sparse set of MLP value vectors—columns of the feed-forward down-projection matrix—whose activations rise or fall when the model sees a correct answer. It proposes TruthV, a training-free detector that ranks every value vector by how often its highest or lowest activation lands on the correct answer in just 30 labeled examples, keeps the top 0.1%, and aggregates their votes by majority. On the NoVo benchmark across four LLMs, TruthV outperforms the attention-head-based NoVo method and log-likelihood scoring, often by a wide margin. The paper presents this as evidence that MLP modules, previously neglected in training-free approaches, carry structured and exploitable truthfulness signals.","feed_headline":"Sparse MLP memory vectors vote to catch LLM hallucinations","feed_subtitle":"TruthV ranks value vectors using 30 labeled examples and beats attention and likelihood baselines on 9 of 10 datasets.","key_machinery":"The key machinery is the interpretation of each MLP value vector as an individual truthfulness predictor. Under the key-value memory view of feed-forward layers, a scalar key activation is computed for each candidate answer; a vector is called argmax-aligned if the correct answer yields the highest activation, and argmin-aligned if it yields the lowest. The method ranks every value vector by its argmax (or argmin) accuracy on a 30-example support set, keeps the top 0.1%, and aggregates their per-question votes by majority. This turns a sparse set of internal memory columns into a training-free ensemble classifier.","core_discovery":"The central claim is that a small fraction of MLP value vectors encode truthfulness in a directionally consistent way: for a given question, the correct answer tends to make the corresponding key activation either the maximum or the minimum among candidates. The paper demonstrates this by ranking all value vectors by argmax/argmin accuracy and showing a thin tail of high-accuracy vectors that transfer to held-out data, then builds TruthV on top of that ranking. TruthV selects the top 0.1% of vectors using 30 labeled samples and lets them vote; across four instruction-tuned LLMs and ten datasets it exceeds NoVo and log-likelihood baselines (e.g., 70.33% vs 61.67% average on Gemma-2-2B-it).","pith_inferences":["The same value-vector voting recipe could be applied to other binary content properties (e.g., toxicity, sentiment, style) whenever a small labeled support set for that property is available, since the mechanism does not rely on truthfulness-specific training.","The paper shows that the activation baseline is set by the question prefix and the answer only shifts it; per-question or per-prefix normalization would likely sharpen the signal, a natural next step not run here.","Because the method uses the final-token key activation on candidate answers, it is inherently a multiple-choice scorer; applying it to open-ended generation would require enumerating candidate continuations or a different aggregation, a nontrivial gap.","The small gain from combining argmax and argmin votes, together with the observed symmetry between the two patterns, suggests using their disagreement as an uncertainty signal for abstention or confidence calibration."],"forward_implications":["Truthfulness detection can be done from MLP internal signals without any trained classifier, using only 30 labeled examples and no extra parameters.","The MLP module, not just attention heads, contains statistically reliable truthfulness signals, so future training-free detectors should not restrict themselves to attention.","The sparse top-0.1% subset suffices for strong accuracy, implying that the truthfulness signal is concentrated in a minority of value vectors rather than distributed uniformly.","Using the full training set usually improves over the 30-sample selection, indicating that the ranking is imperfect but already near-saturated; more data mainly refines the list.","Value vectors selected on one dataset transfer to other datasets above random chance, so they carry a mixture of task-general and task-specific truthfulness information."],"fun_headline_variants":["TruthV: sparse MLP vectors vote to expose LLM fabrication","Sparse MLP value vectors cast a truth vote for LLMs","Training-free truthiness: 0.1% of MLP vectors decide","No training needed: sparse MLP vectors catch LLM lies"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The method assumes that the 30 labeled examples are enough to identify value vectors whose per-question preferences keep holding on unseen questions; if the ranking overfits those 30 examples, the voting accuracy collapses.","fun_headline_variants_meta":{"raw":{"variants":["TruthV: sparse MLP vectors vote to expose LLM fabrication","Sparse MLP value vectors cast a truth vote for LLMs","Training-free truthiness: 0.1% of MLP vectors decide","No training needed: sparse MLP vectors catch LLM lies"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000335,"raw_usage":{"total_tokens":1702,"prompt_tokens":763,"completion_tokens":939,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":507,"completion_tokens_details":{"reasoning_tokens":872}},"tokens_in":507,"tokens_out":939,"duration_ms":6512,"temperature":1.0,"reasoning_tokens":872,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T15:47:54.810759+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run TruthV with the 30 support examples replaced by 30 randomly chosen incorrect answers labeled as 'correct.' If TruthV still scores above random on the real test set, the value-vector ranking is not causal; if accuracy collapses to chance, the selection step is load-bearing. A complementary check: measure the overlap between the top-0.1% vectors selected from the 30 samples and the top-0.1% selected from the full training set; near-zero overlap would indicate the 30-sample ranking is noise.","supporting_citations":[],"review_version":1}