{"id":"1e1818e2-be25-425c-bd40-11b07784545a","arxiv_id":"2505.09807","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Truth-direction probes generalize across short conversational formats that end on a lie but fail on longer formats where the lie appears earlier, and a fixed key phrase at the end of the prompt substantially improves this generalization.","lead":"A study of LLM 'truth directions' shows that probes trained to separate true from false statements stop working when the lie is buried earlier in a longer conversation. Adding a fixed closing instruction that asks about honesty significantly restores accuracy on two 8-billion-parameter models.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported generalization failure may be an artifact of always reading out at the final token, which sits after task-irrelevant small talk in long formats.","rationale":"The strongest claim in the paper is about generalization of a truth direction across conversational formats. For that claim to be meaningful, the representation used to measure the direction must be comparable across formats. The method fixes the readout at the conversation-final token, but the semantic content at that token is not held constant: in short formats it is the lie/truth answer, while in long formats it is a generic small-talk phrase. The reported accuracy collapse and the keyphrase rescue are therefore confounded with readout position. This is the single most load-bearing concern because it determines whether the central empirical finding is about the geometry of truth or about probe attachment. The reader's weakest_assumption identifies the same issue, though the reader also lists secondary concerns (error bars, keyphrase added to both sides). My concrete test would settle it: probe at the lie position. If accuracy remains high, the paper's conclusion should be reframed as a caution about readout location; if not, the original conclusion is supported. The paper is otherwise careful: topic-disjoint cross-validation, two models, a control keyphrase, and PCA robustness checks. No internal inconsistency is apparent. Therefore the verdict should remain CONDITIONAL, pending the position-probing check. I do not see grounds for rejection because the empirical finding under the stated protocol is valid and the limitation is partially acknowledged in the manuscript.","tokens_in":7604,"tokens_out":5186,"duration_ms":52539,"concrete_test":"Re-run the F1→F1+L and F1→F2+L generalization experiments in Fig. 3 using activations extracted at the last token of the lie itself (the final token of the assistant's statement containing the lie) for both training and test formats, rather than at the conversation-final token. If cross-topic accuracy recovers to the level of the F→F baselines (e.g., >0.9), the poor generalization is a positional artifact and the keyphrase result is a readout fix; if accuracy stays near chance, the truth-direction failure is real. A secondary check: for F1+K→F1+L+K, ablate the keyphrase at test time only; if accuracy collapses, the improvement depends on standardizing the readout position, not on the model attending to the lie.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 2 states that activations are recorded from 'the last hidden state within a specific layer' for every input. In F1/F2/F3 the final token is the assistant's True/False answer or the lie itself, so the readout position overlaps the lie. In F1+L/F2+L/F3+L the final hidden state is produced by the fixed small-talk exchange ending with 'For sure, go ahead'; the lie occurred several tokens earlier. The large accuracy drops in Fig. 5 (e.g., 0.96 to 0.51 for Llama F1→F1+L at layer 18) therefore conflate two variables: the conversational format and the position of the readout relative to the lie. The keyphrase intervention (+K) places a task-relevant classifier prompt at the very end of both training and test inputs, reinstating an informative readout position. This means the improvement in Fig. 4 may reflect a change in where the probe is attached rather than a change in the geometry of the truth direction. The paper's central claim—that truth directions generalize poorly to longer formats—is load-bearing on the assumption that the last-token hidden state is the correct universal representation. That assumption is not tested; the PCA visualizations in Fig. 6 compare activations at different semantic positions and thus show position differences, not necessarily direction differences. Unless the probe is evaluated at the position of the lie (or via position-averaged or attentive pooling), the 'failure of generalization' is a statement about the probe readout protocol, not about the underlying truth direction.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies whether linear probes trained on LLM hidden states for true/false statements generalize across conversational formats. Using Llama-3-8b-Instruct and Ministral-8b-Instruct, the authors perform topic-disjoint cross-validation and report three main findings: good transfer from statements to short conversations that end in a lie, poor transfer to longer conversations where the lie occurs earlier in the prompt, and substantial accuracy improvements when a fixed keyphrase is appended to both training and test conversations. The paper interprets these results as evidence that truth directions are format-sensitive and that a prompt-level keyphrase can partly restore generalization for lie detection.","tokens_in":7824,"tokens_out":3939,"duration_ms":42355,"significance":"If the central finding is correct, it qualifies earlier claims in [14,15] about the universality of truth directions and has practical relevance for designing LLM lie detectors: the readout position and prompt structure strongly affect probe transfer. The paper has notable strengths: it uses two independent models, topic-disjoint cross-validation, a control keyphrase, a robustness check with PCA-reduced features, and it releases code and data. The main caveat is that the central generalization claim is currently entangled with the choice of readout token, so the significance depends on the follow-up experiments suggested below.","major_comments":[{"comment":"The central claim that truth directions generalize poorly to longer formats is confounded with readout position. For the base formats F1/F2/F3, the recorded last hidden state is produced by the assistant's True/False answer or the lie itself, whereas in F1+L/F2+L/F3+L the last hidden state is produced by the appended small talk ending in 'For sure, go ahead', several tokens after the lie. The large accuracy drops in Figure 5 (e.g., Llama F1 to F1+L from 0.96 to 0.51 at layer 18) therefore conflate two variables: the conversational format and the position of the readout relative to the lie. Adding the keyphrase places a task-relevant classifier prompt at the very end of both train and test inputs, reinstating an informative readout position. To support the claim that the truth direction itself fails to generalize, the probe should also be evaluated at the position of the lie, or with position-averaged or attention-based pooling across tokens; without such an experiment, the reported failure is a statement about the readout protocol rather than about the geometry of the truth direction.","section":"Section 2, 'Collecting activations'; Figures 2, 4, 5"},{"comment":"The PCA visualizations compare activations at different semantic and positional locations. The projections in Figure 6 are computed from activations collected at the final token, but in the F+L formats this token is the small-talk continuation, not the lie or answer token. The 'drastically different' projections may therefore reflect token-position differences rather than differences in the truth direction. The same issue affects Figure 7: the keyphrase changes the final token to a task-relevant position, so the apparent restoration of structure may be due to aligning the readout position. To make these visualizations interpretable as evidence about the truth direction, the authors should either align the readout positions across conditions or include a control condition with matched token positions and no lie.","section":"Section 3, 'Visualizing principal components'; Figures 6 and 7"},{"comment":"The keyphrase intervention is applied to both training and test inputs, so the improvement from F+K to F+L+K may partly result from reducing train/test distribution shift by standardizing the prompt ending, rather than from refocusing the model's attention on the lie. The control keyphrase in Figure 8 addresses this partially, since it also standardizes the ending while asking an unrelated question, and the authors correctly note that the control still outperforms no keyphrase. However, the comparison does not fully separate the 'same suffix in train and test' effect from the 'task-relevant attention refocusing' effect. An additional condition with the keyphrase present only at test time, or a cross-suffix evaluation (train with +K, test with +C and vice versa), would make the mechanism claim much stronger.","section":"Section 3, 'Adding a key phrase'; Figures 4, 5, and 8"}],"minor_comments":[{"comment":"The text states that cross-validation is run 10 times for every experiment, but Figures 2-5 and 8 report only point estimates without error bars or confidence intervals. Given the small and actively balanced training sets, reporting variance or a significance test for the keyphrase improvements would help support the word 'significant' in the conclusions.","section":"Section 2, 'Evaluation'"},{"comment":"The acronym LRC is introduced as a 'linear regression probe' but later the text says 'not limited to logistic regression probe used here'; the notation is inconsistent and should be clarified. Similarly, TTPD is mentioned without definition.","section":"Section 2, 'Evaluation' and Section 3"},{"comment":"The sentence 'For example, we observe similar poor generalization of the TTPD probe from [15] to longer formats that do not end with a lie' reports a result that is not shown in any figure or table. Either include the data or remove the claim.","section":"Section 3, 'From statements to conversations'"},{"comment":"There are typographical inconsistencies such as 'LLama' vs 'Llama' and 'Ministral' vs 'Mistral', and Figure 5 has a label where the column headers run together ('F1+L+KF2+L+KF3+L+K'). A careful proofreading pass is needed.","section":"Throughout"},{"comment":"The paper's own limitation about centering activations per format and topic is clearly stated, but it means the probe cannot be used on a single test instance in a real-world scenario. This should be kept in mind when the abstract and introduction use the phrase 'lie detection in LLM conversations', which may overstate the practical readiness of the method.","section":"Section 4, Conclusion and outlook"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is within scope and the empirical work is honest and clearly presented. The main issue is the readout-position confound, which I believe is fixable with additional experiments rather than a fundamental flaw. If the authors can show that the same generalization failure appears when the probe is read out at the lie position, or that position alignment removes the effect, the paper would make a solid contribution. I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper is a useful, honest empirical follow-up to [15], but its headline claim—that truth directions fail to generalize to longer conversations—is partly an artifact of always reading out at the final token. The keyphrase \"fix\" may work largely by moving the readout to a position where the lie is attended again.\n\nWhat's new and good: The authors construct 12 conversational formats, test two 8B models, and show that probes trained on statements or short conversations drop to near chance on longer formats where the lie is buried earlier. That extends prior work which only tested lie-at-the-end conversations. They also try a simple keyphrase appended to the prompt and find it substantially restores cross-format accuracy, with a control phrase (letter counting) providing a useful baseline. The topic-disjoint cross-validation and the PCA visualizations support the general pattern, and the overfitting check on 100 PCA features is a reasonable robustness test.\n\nWhere it gets soft: The main issue is the readout location, as you flagged. Section 2 says they record \"the last hidden state within a specific layer\" for every input. In F1/F2/F3 the final token is the assistant's answer or the lie itself; in F+L the final token is \"For sure, go ahead,\" after several turns of small talk. So the accuracy drop between F1 and F1+L conflates format change with readout position change. The keyphrase adds a task-relevant prompt at the very end, so the improvement in Fig. 4 may simply reflect that the probe now reads a token that is semantically tied to the lie. The authors even note attention weakening, but they don't test the obvious control: probe at the lie token position, or average over positions. Without that, \"the truth direction fails to generalize\" is a statement about the readout protocol, not necessarily about the geometry of the representation. The paper's own limitations section doesn't mention this, which is a gap.\n\nMinor issues: accuracies are plotted without error bars or significance tests, even though they ran 10 CV repeats; and the keyphrase is present in both train and test, so prompt standardization and attention refocusing are partially entangled—though the control phrase helps separate those.\n\nBottom line: as a practical caution about current lie-detection probes, this is a solid contribution. As evidence about truth directions per se, it's underdetermined. A token-position ablation would settle it. I'd send it to review with that requested, and it's a reasonable paper for a reading group discussion on probe methodology.","headline":"Useful empirical caution about truth-direction lie detectors, but the headline failure is partly an artifact of final-token readout; needs a token-position control.","tokens_in":8370,"tokens_out":3026,"would_cite":false,"duration_ms":30174,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Linear-probe lie detectors generalize poorly to longer conversations where the lie is not at the end, and a fixed closing key phrase largely restores accuracy.","keywords":["truth direction","linear probes","lie detection","conversational generalization","mechanistic interpretability","large language models","prompt engineering","activation space"],"falsifier":"Train the same linear probe on activations taken at the position of the lie-bearing assistant turn instead of at the final token, and evaluate on the F+L formats without any key phrase; if accuracy returns to the level seen with the key phrase, the failure was attachment point, whereas if it stays near chance the truth direction itself does not survive longer contexts.","tokens_in":7363,"feed_emoji":"🕵️","tokens_out":7602,"duration_ms":69898,"temperature":0.7,"pith_summary":"This paper tests whether the 'truth direction' of a large language model—the linear direction in its internal activations that separates true from false statements—can be carried over to conversational lie detection. The authors find that linear probes trained on statements or on short conversations generalize well to other short conversations that end with a lie, but poorly to longer conversations in which the lie appears before a brief closing exchange. Appending a fixed closing key phrase that asks the model to judge whether the assistant was honest substantially improves transfer to those longer formats, and even an unrelated closing phrase helps, though less. The upshot is that current linear-probe lie detectors are sensitive to conversational format, and prompt structure determines how much lie information reaches the final hidden state.","feed_headline":"LLM lie detectors fail on long chats without a closing key phrase","feed_subtitle":"Truth-direction probes break when lies sit earlier; a fixed closing instruction restores most of the accuracy.","key_machinery":"The machinery is a linear probe (logistic regression on centered activations, LRC) attached to the last hidden state of a specific transformer layer, evaluated across layers 12 to 20 of two 8-billion-parameter instruction-tuned models. Activations are centered per format and per topic to isolate the orientation of the truth plane from prompt-dependent bias. The key mechanism behind the failure is the last-token attachment: when the conversation is extended with small talk, the final hidden state is produced by those added tokens, so the lie signal is diluted. The proposed fix is a fixed closing key phrase that instructs the model to judge whether the assistant was honest, which keeps the final hidden state aligned with the truth direction.","core_discovery":"The paper's central claim is that the truth-direction representation is real but format-bound. Probes trained on true/false statements or on short conversations whose final assistant utterance is the lie generalize well across short conversational formats, but fail on longer conversations where the lie sits earlier in the prompt and closing small talk occupies the final tokens, with accuracy dropping toward chance. Adding a fixed key phrase at the end of every input—an instruction to classify whether the assistant was honest or deceptive—drastically improves generalization to the longer formats for both tested models. A control key phrase that asks an unrelated question also improves accuracy relative to no phrase, but less than the lie-focused phrase, showing that standardizing the prompt ending accounts for part of the effect.","pith_inferences":["A direct test of the paper's mechanism would attach the probe at the token position of the lie itself rather than at the final token; if accuracy on the long formats is restored without any key phrase, the failure is a probe-placement artifact rather than a genuine loss of the truth direction.","The control-key-phrase result suggests that prompt standardization alone shifts the hidden-state distribution, so comparing probes trained with a content-neutral standard ending versus no ending could isolate the semantic contribution of the lie-focused phrase.","The paper's per-format and per-topic centering hides prompt-dependent bias, implying that a practical lie detector would need a format-aware bias term or a way to infer centering from a single conversation before it could be deployed on arbitrary inputs."],"forward_implications":["Probes trained on statements or short conversations cannot be trusted as lie detectors on longer conversations where the lie is not the final utterance; accuracy can fall to near chance.","Appending a fixed key phrase that asks the model to assess honesty substantially improves transfer from short to long formats, with the effect visible for both tested models.","Even an unrelated control key phrase improves generalization relative to no key phrase, indicating that standardizing prompt endings contributes to the gain independently of the semantic content.","Generalization between short formats that end on a lie remains strong, so the truth direction is not an artifact, but its reach is format-limited."],"supporting_citations":[{"why":"Introduces the linear truth direction in hidden states and the centering procedure that this paper adopts.","marker":"[14]"},{"why":"Supplies the statement/truth datasets, the probe setup, and the earlier claim of generalization to conversations ending in a lie that this paper tests and challenges.","marker":"[15]"},{"why":"Establishes that LLM internal states can be probed for truthfulness, providing the foundation for linear-probe lie detection.","marker":"[6]"},{"why":"Raises prior generalization concerns about linear probes, motivating the format-generalization question studied here.","marker":"[16]"},{"why":"One of the two instruction-tuned models whose hidden states are probed in the experiments.","marker":"[17]"},{"why":"The second model, providing a replication of the central results across a different architecture.","marker":"[18]"}],"fun_headline_variants":["Closing key phrase fixes LLM lie-detector failures on long chats","LLM lie detectors miss early lies in long chats, until a key phrase","For LLM lie detection, a fixed closing phrase rescues long-conversation accuracy","Truth-direction probes fail on long chats; a fixed closing line helps","Key phrase at prompt end restores LLM lie detection on long chats"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the last token's hidden state is the right representation for lie detection in every conversational format; because in the longer formats the final hidden state is produced by the closing small talk, the measured failure may be a probe-placement artifact rather than a loss of the truth direction.","fun_headline_variants_meta":{"raw":{"variants":["Closing key phrase fixes LLM lie-detector failures on long chats","LLM lie detectors miss early lies in long chats, until a key phrase","For LLM lie detection, a fixed closing phrase rescues long-conversation accuracy","Truth-direction probes fail on long chats; a fixed closing line helps","Key phrase at prompt end restores LLM lie detection on long chats"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001117,"raw_usage":{"total_tokens":4581,"prompt_tokens":806,"completion_tokens":3775,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":422,"completion_tokens_details":{"reasoning_tokens":3677}},"tokens_in":422,"tokens_out":3775,"duration_ms":23836,"temperature":1.0,"reasoning_tokens":3677,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:23:21.771536+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same linear probe on activations taken at the position of the lie-bearing assistant turn instead of at the final token, and evaluate on the F+L formats without any key phrase; if accuracy returns to the level seen with the key phrase, the failure was attachment point, whereas if it stays near chance the truth direction itself does not survive longer contexts.","supporting_citations":[{"cited_title":"Hamprecht, and Boaz Nadler","cited_arxiv_id":null,"evidence_quote":"Supplies the statement/truth datasets, the probe setup, and the earlier claim of generalization to conversations ending in a lie that this paper tests and challenges."},{"cited_title":"Ministral-2410-8b","cited_arxiv_id":null,"evidence_quote":"The second model, providing a replication of the central results across a different architecture."}],"review_version":1}