{"id":"b469f9a2-95d6-4c0d-b73f-2aaa39b55e7f","arxiv_id":"2602.08159","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"LLM correctness representations are a 3-8 dimensional linear mean-shift signal that simple probes and 25 labels can exploit — but only in the adversarial-misconception regime, not as a universal truth geometry.","lead":"The paper claims that a language model's internal activations encode 'is this answer correct?' as a simple, low-dimensional pattern — about 3-8 dimensions — that a plain linear detector can read, and that two dozen labeled examples are enough to build it. It matters because it promises cheap internal hallucination detectors and a steerable truth direction, with the caveat that the signal clearly works on adversarial misconception questions and the abstract reports experiments","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"TruthfulQA's correct/incorrect pairs may encode negation/assertiveness rather than factual correctness; the Appendix H paraphrase control does not test this axis, and the below-chance HaluEval transfer is consistent with a stylistic confound.","rationale":"The reader's weakest_assumption identifies the same load-bearing concern: the TruthfulQA contrastive pairing may isolate answer style rather than factual correctness. My read agrees and adds specificity: the below-chance HaluEval transfer in Table 8 is a concrete red flag, and the Appendix H paraphrase control is too weak to rule out the negation/assertiveness axis. This concern directly threatens the central claim that the learned low-dimensional direction is a correctness representation and that steering along it is causally about correctness. The paper does many things right in-domain (GroupKFold by question, nested CV, random/orthogonal steering controls), so I do not think the entire empirical contribution collapses; rather, the semantic interpretation and the claimed generality are conditional on resolving this confound. Since the reader already issued a CONDITIONAL verdict on essentially this basis, my stress-test does not move the verdict; it reinforces the condition and sharpens the test that would settle it.","tokens_in":27001,"tokens_out":4443,"duration_ms":55178,"concrete_test":"Recompute the Section 5.1 PLS-dimension sweep and Section 5.3 centroid-parity result on a style-matched subset of TruthfulQA in which correct and incorrect answers share the same polarity markers (e.g., all begin with a direct assertion, or all begin with \"No\") and have matched length distributions. If the probe AUC and centroid-probe parity survive on this matched subset, the confound is not load-bearing; if performance collapses toward chance, the correctness-specific interpretation is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central geometric claim rests on binary labels from TruthfulQA paired answers. These pairs are not balanced for a major stylistic feature: correct answers to misconception questions often refute (\"No...\", \"not true\"), while incorrect answers are fluent, confident assertions. A linear probe or PLS direction can therefore separate on negation/assertiveness rather than factual content. Evidence within the paper is consistent with this: cross-domain transfer to HaluEval is consistently below chance (0.18-0.47 AUC, Table 8), meaning the TruthfulQA-trained probe flags hallucinated text as correct. A genuine correctness direction should not systematically invert on a hallucination-detection task. The paraphrase control (Appendix H) only permutes surface templates (\"The answer is...\", \"To be precise...\") and leaves content and polarity unchanged; it does not control for the negation axis. Since the steering experiment (Section 5.7) uses the same learned direction, the 10.9pp error-rate swing could be a stylistic intervention (making outputs more or less assertive) rather than causal proof about factual correctness. Thus the \"mean shift\" geometry may describe style differences in TruthfulQA's answer sets, not a correctness manifold.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper sets out to characterize the geometry of correctness representations in transformer residual streams. It trains logistic probes and PLS projections on TruthfulQA paired correct/incorrect answers for 9 models (GPT-2 family and instruction-tuned 1B–7B), and claims: (i) the discriminative signal is 3–8 dimensional; (ii) the classes are linearly separable and separated by a mean shift, so centroid distance matches probe AUC; (iii) 25 labels recover ~89% of full-data performance; (iv) internal probes (0.80–0.97 AUC) far outperform output-based methods (0.44–0.64 AUC); (v) steering along the learned direction changes error rates by 10.9pp with random/orthogonal controls null; (vi) the structure is consistent across architectures. The evaluation protocol is unusually careful: GroupKFold by question, nested CV for hyperparameters, length-residualized and length-balanced controls, paraphrase variance decomposition, and random/orthogonal steering controls.","tokens_in":27232,"tokens_out":9228,"duration_ms":97948,"significance":"If the geometric claims survive, they would give a simple, practical detector and a strong constraint on mechanistic accounts of factuality in LLMs: correctness would be a low-rank linear feature in the residual stream, detectable from two centroids. The paper's strengths are its careful controls (Appendix G nested CV; Appendix H paraphrase variance; Appendix F steering holdout) and the breadth of models. Its principal weakness is that the binary label from TruthfulQA's paired answers may be entangled with a negation/assertiveness style axis; the current controls do not exclude this, and the below-chance HaluEval transfer is a red flag. The advertised abstract also contains claims (erasure, DAS, 11 models) that do not appear in the body.","major_comments":[{"comment":"The central claim that the detected signal is factual correctness is not yet separated from a polarity/assertiveness confound. TruthfulQA's paired answers are not matched on negation: correct answers to misconception questions typically refute (\"No...\", \"not true\"), while incorrect answers are fluent assertions. The Appendix H paraphrase control varies only four surface templates (\"The answer is...\", \"To be precise...\") and leaves the polarity axis untouched; the F-ratio 17.40 therefore does not rule out style. Table 8 is consistent with the confound: TruthfulQA-trained probes transfer below chance to HaluEval (0.18–0.47 AUC), meaning hallucinated text is systematically flagged as correct. The remark in §5.5 that HaluEval is excluded because it tests \"summarization faithfulness\" is a post-hoc exclusion; the below-chance transfer is evidence that the direction is not a universal correctne","section":"§5.5, Table 8, Appendix H"},{"comment":"The \"3–8D\" headline is the argmax of a PLS component sweep, not a demonstrated intrinsic rank. For most models the differences between 3D, 4D, 5D, and 8D are within a few thousandths of AUC (e.g., Llama-3B: 0.917/0.917/0.919/0.910 with reported std of 0.02–0.03), and the nested CV in Appendix G selects 5–8D rather than a sharp 3D peak. The claim \"performance degrades with additional dimensions\" is supported only by a noisy trend. I ask for paired significance tests of the selected peak against adjacent dimensions and a statement of how flat the curve is. Without this, \"occupies 3–8 dimensions\" is not an established geometric bound; at most one can say the signal is captured by a small single-digit number of PLS components. The term \"dimensions\" also conflates PLS components with intrinsic signal dimensions.","section":"§5.1, Table 1, Appendix G"},{"comment":"The steering experiment inherits the same confound. The steering vector is the probe weight, so if the probe separates on assertiveness/negation, adding ±α will make outputs more or less assertive and thereby change the error rate on TruthfulQA without any change in factual knowledge. The random/orthogonal controls test direction specificity but not semantic content. Please add controls that measure the stylistic axis (e.g., negation rate, hedging markers) under steering, or repeat steering with a direction trained on negation-matched pairs. Without this, the causal claim \"confidence direction is causally relevant\" is not established over the style alternative.","section":"§5.7, Figure 2"},{"comment":"The submission-level abstract claims \"11 models (124M–14B)\", a \"2–8 dimensional subspace\", \"9.1 points on six models\", \"erasure drops detection to chance\", and \"distributed alignment search ... at most five causal dimensions\". The body reports 9 models up to 7B, a 3–8D peak, a single steering experiment on GPT-2 (Section 5.7), and no erasure or DAS experiments. This is not a minor wording issue; it makes the advertised scope irreproducible. The abstract must either be rewritten to match the body or the missing experiments must be included.","section":"Abstract vs. body"},{"comment":"The \"mean shift\" conclusion is supported by comparing a linear probe with a centroid classifier in the same supervised PLS subspace. Because PLS components are constructed using labels, both classifiers are evaluating the same projection, and under approximately whitened within-class covariance a linear probe is equivalent to a centroid rule. This does not invalidate the practical claim, but it is not a test of whether the original high-dimensional class structure is a mean shift. A test on a label-agnostic subspace (e.g., PCA) or explicit comparison of full-covariance vs diagonal Gaussian models would be needed to support \"making detection geometric rather than learned\".","section":"§5.3, §6"}],"minor_comments":[{"comment":"Procrustes alignment is defined but no cross-model Procrustes results are reported; if the analysis is not used, remove the definition or add the results.","section":"§3.2"},{"comment":"The paper mentions an \"80/20 stratified split\" in §4.1 but the results use 5-fold GroupKFold. Clarify the relationship between these protocols.","section":"§4.1/§4.4"},{"comment":"In-domain HaluEval AUC is above 0.98 for all models while cross-domain transfer from TruthfulQA is below chance. This stark discrepancy deserves a sentence in the main text, not only the appendix discussion.","section":"Table 8"},{"comment":"The baseline error rate of 0.56 should be defined in the caption (is it GPT-2 on the 617 held-out questions?), and the absolute error counts for α=±5 should be reported.","section":"Figure 2"},{"comment":"The term \"confidence manifold\" is used for a supervised discriminative subspace. Consider using a more neutral term such as \"correctness subspace\" to avoid confusion with confidence-calibration literature.","section":"Title/terminology"}],"recommendation":"major_revision","confidential_remarks":"This is a solid experimental paper with careful protocols; the main risk is the factuality/style confound. I recommend major revision, not rejection, because the confound is testable and the protocols are otherwise strong. The abstract/body mismatch should be fixed before any acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a genuinely careful measurements paper, and the two things to know are: (1) the in-domain experiments are well controlled and the 3-8D PLS peak plus centroid parity is a real empirical finding worth taking seriously; and (2) the abstract overstates the body in ways that matter for evaluation.\n\nWhat is actually new: the per-model PLS component sweep peaking at 3-8D with degradation at higher dimensions, the centroid-distance parity with trained probes, the 25-label few-shot result achieving about 89-90% of full-data AUC on GPT-2, and the cross-architectural phase structure. The protocols are careful: GroupKFold grouped by question, nested CV checks with small bias, length and surface-feature confound controls, and steering with random/orthogonal controls. That is real work and useful.\n\nThe main unresolved issue is the negation/assertiveness confound in TruthfulQA's contrastive pairs. Correct answers to misconception questions often refute (\"no\", \"not true\"), while incorrect answers are fluent confident assertions. A linear probe can separate on that axis just as easily as on factual correctness. The paraphrase control does not touch polarity; it only permutes surface templates, so it does not rule this out. The HaluEval transfer result is consistent with the confound: below-chance AUC means the TruthfulQA-trained probe flags hallucinated text as correct. A genuine correctness direction should not systematically invert on a hallucination-detection task. The paper's own explanation (\"confident misconception\" vs \"generation failure\" anti-correlation) is plausible, but it is an explanation, not a control.\n\nThe abstract/body mismatch is not cosmetic. The abstract mentions 11 models up to 14B, concept erasure, distributed alignment search, joint multi-dataset training, and 9.1pp steering on six models. None of those appear in the body, which covers 9 models up to 7B and reports 10.9pp on GPT-2. The abstract also says 2-8 dimensions while the body says 3-8. If those experiments exist in the repo or a future version, fine, but as submitted they are not verifiable.\n\nMore minor: output-based baselines are not reported on the transfer datasets, so the claimed \"regime-specific advantage\" is not quantified. And HaluEval is excluded from the cross-domain average, which is reasonable but needs a clearer justification.\n\nWho this is for: people working on internal correctness probes, linear representation, and hallucination detection. The in-domain characterization is solid, the few-shot method is practically interesting, and the cross-architectural pattern is worth discussing. It deserves a serious referee, but the geometric interpretation needs a polarity/negation control before it can carry the weight of the \"confidence manifold\" claim. If I were handling it, I would send it out with an explicit request to align the abstract with the body and add a control where correct and incorrect answers are balanced for assertion/refusal style.","headline":"A careful in-domain measurement paper whose headline geometric claim is plausible but not yet established; the abstract promises experiments the body does not contain, and the negation/assertiveness confound in TruthfulQA is not ruled out.","tokens_in":27869,"tokens_out":2874,"would_cite":true,"duration_ms":32833,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a language model's knowledge of whether a claim is correct lives in a 3 to 8 dimensional subspace of its internal activations, where separating true from false reduces to a simple mean shift between two class centroid","keywords":["confidence manifold","correctness detection","activation probing","mean shift","low-rank subspace","causal steering","semantic entropy","hallucination detection"],"falsifier":"Train a probe on the same benchmark but with incorrect answers rewritten to match the style of correct answers (same terseness, same hedging, same polarity framing); if detection AUC collapses to chance, the so-called confidence manifold is encoding style rather than factual correctness. A second test: measure the centroid distance between correct answers and confidently asserted false claims generated from outside the benchmark; if the distance vanishes, the manifold does not encode correctness per se.","tokens_in":26775,"feed_emoji":"🎯","tokens_out":2697,"duration_ms":28249,"temperature":0.7,"pith_summary":"Large language models often state falsehoods with the same fluency as facts, so output-level uncertainty cannot tell you when they are wrong. This paper tries to establish that the model's own internal representation of correctness is geometrically simple: the signal that separates correct from incorrect answers occupies only 3 to 8 dimensions of the residual stream, and adding more dimensions actually hurts detection. It further claims that classification within this subspace is dominated by a mean shift between the two classes, so a centroid-based detector built from two mean vectors matches a fully trained probe, and only 25 labeled examples recover most of the full-data accuracy on the smallest model. Causal steering along the learned direction shifts error rates by about ten percentage points while random directions do nothing, implying the direction is not just a correlate but an active lever. The paper also reports a consistent gap between internal probes and output-based uncertainty methods: internal probes score 0.80 to 0.97 AUC while semantic entropy and similar methods hover near chance on confidently asserted misconceptions.","feed_headline":"LLM wrongness lives in a 3-to-8-D subspace","feed_subtitle":"Internal probes beat output uncertainty by 30+ AUC points on confident misconceptions; two mean vectors nearly match a trained classifier.","key_machinery":"The central object is the 'confidence manifold': the low-dimensional subspace of residual-stream activations that separates correct from incorrect claims. It is located with partial least squares regression, which projects activations onto directions maximizing covariance with the correctness labels; sweeping the number of components reveals the 3 to 8 dimensional discriminative rank. The central identity is that the distance between class centroids in this subspace matches the performance of a trained logistic probe, which the paper interprets as class separation being a mean shift. Activation steering (adding a scaled probe direction during generation) serves as the causal lever, and the c","core_discovery":"On the paper's own terms, the central discovery is that the discriminative signal for factual correctness is concentrated in a low-dimensional subspace—3 to 8 dimensions depending on the model—within the transformer residual stream. Using partial least squares to sweep the number of components, the paper finds that performance peaks in this range and degrades when more dimensions are added. Across nine models from five architecture families, no nonlinear classifier (convex hull, Mahalanobis, kernel SVM) beats a linear probe, and the distance between correct and incorrect class centroids matches probe AUC, indicating the classes are separated by a mean shift rather than by covariance structur","pith_inferences":["If the geometry is truly a mean shift, then the same two-vector detector should transfer to other languages and modalities; a cheap test would be to fit centroids on English QA and apply them to multilingual or multimodal answers.","The 3–8 dimensional discriminative rank suggests hallucination detection could be inserted into decoding as a low-cost linear filter, but that would require the steering direction to transfer reliably across prompts and domains, which the paper only partially tests.","Because the paraphrase control only varies surface templates and not the negation axis—correct refutations versus confident assertions—a stronger control that swaps answer polarity while holding the sentence frame would directly test whether the probe is capturing style rather than factuality.","The mean-shift interpretation predicts that a Gaussian-optimal classifier (linear discriminant analysis) should exactly match logistic regression on all models; a direct comparison would confirm or refute the 'identical covariance' assumption implicit in the centroid result."],"forward_implications":["Correctness detection can be reduced to two mean vectors in a learned low-rank subspace, eliminating the need for discriminative training once the subspace is found.","On a small base model, 25 labeled examples recover about 89% of full-data detection accuracy, suggesting few-shot detectors are practical.","Internal probes outperform output-based uncertainty methods on adversarial misconceptions (0.80–0.97 AUC vs 0.44–0.64), so detectors that ignore internal activations are leaving nearly all the signal on the table.","Steering along the learned direction produces a monotonic, roughly 11 percentage point swing in error rate while random and orthogonal controls show no effect, implying the direction is causally relevant for behavior.","Projecting to 5 PLS dimensions improves cross-dataset transfer by 10–14 absolute AUC points on one large model, indicating that removing dataset-specific noise exposes a more universal correctness signal."],"fun_headline_variants":["Wrongness hides in 3-8 model dimensions","LLM correctness is a geometric shift","Two centroids reveal LLM lies","Probes nail wrongness in low-dimensional space"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing assumption is that the paired correct and incorrect answers in the benchmark isolate factual correctness from answer style—specifically, that the negation or refutation register of correct answers is not what the probe is actually detecting.","fun_headline_variants_meta":{"raw":{"variants":["Wrongness hides in 3-8 model dimensions","LLM correctness is a geometric shift","Two centroids reveal LLM lies","Probes nail wrongness in low-dimensional space"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000212,"raw_usage":{"total_tokens":1263,"prompt_tokens":760,"completion_tokens":503,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":504,"completion_tokens_details":{"reasoning_tokens":447}},"tokens_in":504,"tokens_out":503,"duration_ms":6405,"temperature":1.0,"reasoning_tokens":447,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T03:24:15.069794+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a probe on the same benchmark but with incorrect answers rewritten to match the style of correct answers (same terseness, same hedging, same polarity framing); if detection AUC collapses to chance, the so-called confidence manifold is encoding style rather than factual correctness. A second test: measure the centroid distance between correct answers and confidently asserted false claims generated from outside the benchmark; if the distance vanishes, the manifold does not encode correctness per se.","supporting_citations":[],"review_version":1}