{"id":"a1e6a0dd-f65a-49dd-a772-4eac863ce093","arxiv_id":"2608.12852","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"In Gemma 3 4B IT, necessary falsehoods are represented separately from contingent falsehoods and closer to semantic anomaly, even though the model verbally labels contingent falsehoods as contradictions.","lead":"This paper probes how an AI language model internally represents impossible statements versus merely false ones. It reports that in Gemma 3 4B's activations, impossibility is carried by a direction nearly orthogonal to the truth direction, even though the model's words treat falsehood and contradiction as the same.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 'necessary falsehood' stimulus set includes an item that is not impossible, so the fitted impossibility direction may not track modality.","rationale":"The reader's weakest_assumption was that the necessary falsehood stimuli might only be pragmatically odd or lexically distinctive. That assumption is not merely hypothetical: the paper's own Methods list 'arrived before it departed' as a necessary falsehood, but this sentence has a natural true reading (a train arriving at a station before departing from it). Since the whole experiment is built on the contrast between contingent falsehood and impossibility, one mislabeled 'impossible' item directly contaminates the target category. The probe's perfect AUC and the orthogonality result therefore cannot be interpreted as evidence about modality until the item set is validated. I agree with the reader's conditional verdict; the requested condition should be an item-level necessity audit rather than only the Table 2 reinterpretation. The AUC 0.20 issue is real but secondary: even if reinterpreted as an inverted separation, it does not rescue the construct validity of the impossibility condition.","tokens_in":8061,"tokens_out":11181,"duration_ms":122375,"concrete_test":"Run an item-level necessity audit on all 15 necessary-falsehood prompts. For each item, attempt to construct a scenario in which the sentence is true while preserving ordinary meanings and background constraints; use three independent annotators blind to the paper's labels. Then refit the impossibility probe and recompute Table 2 and direction cosines with any non-necessary item removed or relabeled. If the held-out AUC of 1.00 and the near-zero cosine survive the corrected item set, the concern is resolved; if performance drops or the geometry changes, the dissociation is an artifact of the contaminated stimulus category.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim requires that the 15 'necessary falsehood' stimuli really are false under every admissible interpretation that preserves ordinary meanings (Methods 4.1). This fails for at least one listed item: 'arrived before it departed' is presented as a temporal-reversal necessary falsehood, but in ordinary English it can be true. A train can arrive at a station and then depart from that same station; under that reading 'it arrived before it departed' is true, not impossible. Thus at least one of the 15 impossibility labels is wrong. The impossibility probe (AUC 1.00 on impossible vs. false) and the cosine between truth and impossibility directions are computed from this contaminated category, so the fitted direction could reflect temporal oddity, narrative sequencing, or pragmatic anomaly rather than modality. The paper is otherwise transparent about limitations, but this is a construct-level error, not a mere interpretive nuance. An independent necessity check on every item is required before the dissociation can be attributed to impossibility.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper reports a probing study of Gemma 3 4B IT. Using 85 philosophical prompts and 75 topic-matched statements in five conditions (true, contingent false, improbable, anomalous, necessary false), the author trains linear probes on residual-stream activations. The main findings are that a truth probe does not separate impossible from false statements (AUC 0.20) while an impossibility probe does (AUC 1.00) with grouped held-out families, and that the truth and impossibility directions are nearly orthogonal, with impossibility closer to semantic anomaly. The SAE features at layer 15 are reported to repeat this geometry. The paper interprets this as evidence that necessary falsehoods are represented as distinct from contingent falsehoods and nearer to anomaly.","tokens_in":8234,"tokens_out":5418,"duration_ms":54293,"significance":"The design has real strengths: held-out-family cross-validation, a permutation test with Bonferroni correction, TF-IDF surface baselines, cross-dataset transfer, and public data and code. If the stimulus validation is repaired, the double dissociation would be a valuable empirical footnote connecting formal semantics and mechanistic interpretability. However, the central quantitative claims rest entirely on the validity of the hand-authored 'necessary falsehood' category, and the SAE feature-selection step is in-sample, so the mechanistic claim is weaker than the probe result.","major_comments":[{"comment":"The temporal-reversal item 'arrived before it departed' is not a necessary falsehood under ordinary English: a train can arrive at a station and then depart from it, making the sentence true. Since this item is one of the 15 impossibility stimuli, the impossibility probe's AUC 1.00 (Table 2) and the orthogonality cosine (Fig. 2c) are computed from a contaminated category. Please validate all 15 items with independent necessity judgments (or formal paraphrase), replace or remove invalid items, and rerun all analyses.","section":"§4.1 (Stimuli), Table 4"},{"comment":"The SAE features are selected for selectivity using the same 160 prompts on which firing prevalence is then reported, so the claim that SAE features 'repeat the probe geometry' is partly guaranteed by construction. The manuscript acknowledges this in §4.4, but the Results section should either present a held-out feature-selection analysis (e.g., select on training families and report on held-out families) or explicitly frame Table 3 as descriptive in-sample data rather than confirmatory evidence.","section":"§4.4 (SAE analysis), Table 3"}],"minor_comments":[{"comment":"The truth-probe AUC of 0.20 for impossible vs. false is reported without a confidence interval; with 15 examples per class, please add bootstrap CIs or a permutation test to confirm that this value is not a stable inversion of the expected ordering.","section":"§2.2, Table 2"},{"comment":"The caption says 'cosine' while the text in §2.2 says 'absolute cosine'; please make the terminology consistent.","section":"Figure 2c caption"},{"comment":"The entry 'True141 0 0' is a formatting issue; add spaces between the counts for readability.","section":"Table 1"},{"comment":"Reporting mean log activations in addition to firing prevalence would make the SAE effect size clearer and would strengthen the descriptive comparison.","section":"§4.4, Table 3"}],"recommendation":"major_revision","confidential_remarks":"I recommend major revision rather than rejection because the stimulus-validity issue is correctable and the core probe design is sound. The central construct-validity fix should be treated as mandatory."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The thing to know about this paper is that it reports a genuinely new empirical dissociation: in Gemma 3 4B's residual stream, the linear direction separating true from false is almost orthogonal to the direction separating impossible from merely false. Prior truth-probing work treated falsehood as one axis; this paper asks whether impossibility is a different direction and finds strong separation with held-out families and a permutation test. That's a real contribution, and the author is careful with limitations—no causal claims, acknowledged correlational probes, surface baselines reported.\n\nThe soft spot is in the stimulus construct. One of the fifteen \"necessary falsehoods\" is \"arrived before it departed.\" In ordinary English that is not impossible—a train can arrive at a station and then depart from it. The Methods claim that every item is false under every admissible interpretation is just false for that item. With only fifteen impossibility items, one bad label means the fitted direction may not be tracking modality. It might be tracking temporal sequencing or something about the specific sentence. The author should audit all items with an independent necessity check before treating the dissociation as established. This doesn't sink the whole geometry, but it's a load-bearing flaw in the category.\n\nAlso, Table 2's AUC 0.20 for the truth probe on impossible vs. false is not \"at or below chance\"—it's well below chance. That's an inverted separation, and it deserves explicit interpretation. The SAE feature selection is same-sample; the author flags it, but it still limits that section.\n\nBottom line: this is a paper for interpretability researchers and anyone interested in how models encode modality. It's a small exploratory result, not a mechanism. I'd send it to peer review—the core observation is novel and the methodology is mostly sound—but I'd require the stimulus audit and the AUC re-interpretation first.","headline":"A careful small-model probe study showing truth and impossibility as near-orthogonal directions, but the 'impossible' category contains at least one genuinely true item and the below-chance AUC is misread as chance.","tokens_in":8753,"tokens_out":3886,"would_cite":false,"duration_ms":39458,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A language model's internal geometry treats impossibility as a separate direction from falsehood.","keywords":["mechanistic interpretability","contradiction","impossibility","truth probing","sparse autoencoders","philosophy of language","semantic anomaly"],"falsifier":"Train the same impossibility probe on a new set of 'necessary falsehoods' built with completely different surface forms (for example, generated by conjoining each proposition with its negation in varied vocabularies) while holding topic vocabulary matched across conditions; if probe AUC on held-out families drops substantially, the direction is capturing lexical templates rather than impossibility. Alternatively, replace the necessary-falsehood items with pragmatically odd but logically possible sentences; if the probe still separates them from contingent falsehood at AUC 1.00, the direction encodes oddity, not modality.","tokens_in":7814,"feed_emoji":"🧠","tokens_out":5820,"duration_ms":55149,"temperature":0.7,"pith_summary":"This paper asks whether a large language model distinguishes statements that are merely false from statements that could not be true. Using 160 prompts built from 17 philosophical families and 15 topic-matched conditions, the author finds a split between the model's words and its internal states: the model verbally labels ordinary falsehoods as 'contradiction', but its activation geometry keeps the two apart. A linear probe trained to detect impossibility separates necessary from contingent falsehood at perfect held-out AUC, while the direction that separates true from false is at chance on that same contrast. The central claim is that necessary falsehoods are not represented as extreme contingent falsehoods; they sit closer to semantic anomaly while remaining distinguishable from it.","feed_headline":"Inside one AI, 'impossible' is a different direction from 'false'","feed_subtitle":"Activation probes find the truth and impossibility axes are nearly orthogonal, even though the model's own words blur the line.","key_machinery":"The load-bearing object is the linear probe readout on residual-stream states, together with the angular comparison of probe weight vectors. Each probe is a regularized logistic regression fitted on grouped, family-held-out folds; the impossibility probe contrasts impossible statements against true, false, and improbable ones, while the truth probe contrasts false against true and the anomaly probe contrasts anomalous against the three possible conditions. The double dissociation—the truth probe cannot order impossible against false, while the impossibility probe can—and the near-orthogonality of the fitted directions carry the argument. A pretrained sparse autoencoder at the impossibility peak (layer 15) supplies a feature-level confirmation.","core_discovery":"The paper's central claim is that in the residual stream of Gemma 3 4B IT, impossibility and falsehood are carried by nearly orthogonal linear directions. The truth direction, which separates true from false statements with balanced accuracy 0.93, fails to separate impossible from merely false statements (AUC 0.20); an impossibility probe fitted on held-out topic families does separate them at AUC 1.00. The absolute cosine between the two directions stays at or below 0.12 beyond depth 10. Sparse autoencoder features at layer 15 repeat this geometry: features selective for impossibility fire on anomalous sentences but rarely on contingent falsehoods. The author states the result as a representational observation, not a claim that impossible statements are intrinsically meaningless.","pith_inferences":["If the near-orthogonality holds in larger models, then a model's tendency to call falsehoods 'contradictions' may be a shallow verbal policy layered on top of a more structured internal geometry; interventions aimed at truthfulness might be adjusting the wrong axis.","The dissociation suggests that ordinary language itself contains enough distributional evidence for a learner to separate the contingent from the impossible without explicit logical supervision, since no labels of this kind were used in pretraining.","A causal test the paper does not run: patching or steering along the impossibility direction should change answers about contradictions while leaving answers about factual lies unchanged; if it does, the direction is used, not merely decodable.","A further extension would replace the handwritten necessary-falsehood stimuli with generated contradictions that share no lexical template, to test whether the direction tracks logical form rather than topic-specific wording."],"forward_implications":["Verbally, the model collapses contingent falsehood into 'contradiction', so behavior alone would not reveal the internal dissociation; activation measurements do.","Truth-probing results cannot be read as locating impossibility: a model can be accurate on factual truth while its truth direction is blind to modal status.","The impossibility direction transfers across topic families and, partially, across the philosophical and modality stimulus sets, so it is not a lexical-template artifact.","Necessary falsehood and semantic anomaly are related but separable in this geometry, so the model does not simply treat contradictions as meaningless.","The study provides a template for asking whether larger models sharpen or dissolve the same separation, and whether the model ever uses this direction when it answers."],"supporting_citations":[{"why":"Prior evidence that truth is linearly decodable from the residual stream; the truth probe replicates and extends this baseline.","marker":"[11]"},{"why":"Prior result that truth values form a linear structure in representations; the paper tests whether impossibility is a separate axis from this structure.","marker":"[12]"},{"why":"Provides the pretrained sparse autoencoder features at layer 15 used to confirm the probe geometry at the feature level.","marker":"[16]"},{"why":"Defines the semantic-anomaly category ('colorless green ideas') used as the experimentally defined comparison condition.","marker":"[3]"},{"why":"Supplies the philosophical distinction between sense, senselessness, and nonsense that the experiment operationalizes as separate stimulus categories.","marker":"[2]"},{"why":"Documentation of the model checkpoint whose activations are measured; without it the experiment's object is undefined.","marker":"[18]"}],"fun_headline_variants":["AI's 'impossible' direction is nearly orthogonal to 'false'","In Gemma 3, impossibility and falsehood are orthogonal directions","Falsehood vs impossibility: AI's internal geometry separates them","Probes find AI keeps impossibility and falsehood in different directions","Impossibility is not extreme falsehood in AI's activation space"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The handwritten 'necessary falsehood' stimuli are assumed to be false under every admissible interpretation that preserves ordinary meanings and background constraints; if they are actually only pragmatically odd or lexically distinctive, the fitted impossibility direction captures a narrower property and the dissociation would not be about modality.","fun_headline_variants_meta":{"raw":{"variants":["AI's 'impossible' direction is nearly orthogonal to 'false'","In Gemma 3, impossibility and falsehood are orthogonal directions","Falsehood vs impossibility: AI's internal geometry separates them","Probes find AI keeps impossibility and falsehood in different directions","Impossibility is not extreme falsehood in AI's activation space"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000517,"raw_usage":{"total_tokens":2533,"prompt_tokens":1000,"completion_tokens":1533,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":616,"completion_tokens_details":{"reasoning_tokens":1444}},"tokens_in":616,"tokens_out":1533,"duration_ms":9039,"temperature":1.0,"reasoning_tokens":1444,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:57:59.346476+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same impossibility probe on a new set of 'necessary falsehoods' built with completely different surface forms (for example, generated by conjoining each proposition with its negation in varied vocabularies) while holding topic vocabulary matched across conditions; if probe AUC on held-out families drops substantially, the direction is capturing lexical templates rather than impossibility. Alternatively, replace the necessary-falsehood items with pragmatically odd but logically possible sentences; if the probe still separates them from contingent falsehood at AUC 1.00, the direction encodes oddity, not modality.","supporting_citations":[{"cited_title":"& Mitchell, T","cited_arxiv_id":null,"evidence_quote":"Prior evidence that truth is linearly decodable from the residual stream; the truth probe replicates and extends this baseline."},{"cited_title":"& Tegmark, M","cited_arxiv_id":null,"evidence_quote":"Prior result that truth values form a linear structure in representations; the paper tests whether impossibility is a separate axis from this structure."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the pretrained sparse autoencoder features at layer 15 used to confirm the probe geometry at the feature level."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the semantic-anomaly category ('colorless green ideas') used as the experimentally defined comparison condition."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the philosophical distinction between sense, senselessness, and nonsense that the experiment operationalizes as separate stimulus categories."}],"review_version":1}