{"id":"ce4515f8-6296-4f06-8819-6cebd62f163a","arxiv_id":"2605.01844","paper_version":2,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":6.0,"correctness_risk":"unknown","formal_verification":"none","parameter_count":0,"one_line_summary":"The Cylindrical Representation Hypothesis (CRH) models LLM representations as a central axis for concept activation surrounded by a normal plane containing sensitive sectors that determine steering sensitivity and introduce intrinsic uncertainty.","lead":"The paper proposes the Cylindrical Representation Hypothesis (CRH) by relaxing the orthogonality assumption of the Linear Representation Hypothesis to explain unstable steering effects in language models. A smart generalist might read it to understand sources of unpredictability when trying to control AI model outputs.","discovery_kind":"extension","skeptic_critique":{"model":"grok-4.3","headline":"Difference vectors may entangle multiple overlapping concepts, undermining reliable isolation of the central axis and normal plane.","rationale":"Reader correctly flags the plane-vs-sector asymmetry as central, but the more fundamental risk is whether the plane itself can be recovered from difference vectors under the overlapping premise the paper uses to motivate CRH. This is a correctness risk internal to the construction rather than an external consensus issue. Experiments claimed in the abstract could address it directly via the proposed check.","tokens_in":1799,"tokens_out":304,"duration_ms":22002,"concrete_test":"For the top-5 concepts used in the paper's experiments, compute pairwise cosine similarities of their difference vectors (as defined in the methods); if average |cos| > 0.15 or if ablating one concept changes another's difference vector norm by >10%, the isolation assumption fails and the cylindrical decomposition is not uniquely recoverable.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"CRH posits that the central axis (from difference vectors) and surrounding normal plane can be reliably recovered while the sensitive sector cannot. This separation is load-bearing: if difference vectors for a target concept are contaminated by contributions from other overlapping concepts (as the hypothesis itself invokes), the recovered axis is not guaranteed to be the 'main difference' and the normal plane is not cleanly identifiable. The abstract asserts reliability for the plane but provides no mechanism ensuring difference vectors remain concept-pure in the overlapping regime the model assumes.","agreement_with_reader":"partial"},"referee_report":{"model":"grok-4.3","summary":"The paper claims that relaxing the orthogonality assumption of the Linear Representation Hypothesis (LRH) while preserving linear representations leads to overlapping concept contributions that naturally produce a sample-specific cylindrical structure. This is formalized as the Cylindrical Representation Hypothesis (CRH), in which a central axis (recovered from difference vectors) captures the main difference between concept absence and presence and drives generation, a surrounding normal plane controls steering sensitivity, and only specific sensitive sectors within that plane facilitate activation while others suppress it. The normal plane is asserted to be reliably identifiable from difference vectors, but the sensitive sector is not, introducing intrinsic uncertainty that explains steering fluctuations. Experiments are reported to verify the cylindrical structure and demonstrate CRH's utility for interpreting real steering behavior, with code provided.","tokens_in":1900,"tokens_out":456,"duration_ms":22765,"significance":"If the cylindrical geometry and the claimed identifiability separation hold, CRH supplies a concrete geometric account of steering instability that remains within the linear-representation paradigm and could guide more stable steering methods. The inclusion of experiments together with a public code repository is a strength that supports empirical scrutiny and reproducibility.","major_comments":[{"comment":"Abstract and the formalization of CRH: the load-bearing assertion that 'the surrounding normal plane can be reliably identified from difference vectors' while the sensitive sector cannot must be justified against the possibility that difference vectors entangle multiple overlapping concepts. The hypothesis itself invokes overlapping contributions, yet no explicit argument or isolation mechanism is supplied showing why the recovered axis and plane remain concept-pure.","section":"Abstract / CRH formalization"},{"comment":"Experiments section: verification of the cylindrical structure should include controls or ablations that test robustness of axis and plane recovery when difference vectors are constructed under controlled concept overlap, directly addressing whether the claimed reliable identification of the normal plane survives the regime the hypothesis assumes.","section":"Experiments"}],"minor_comments":[{"comment":"The GitHub link is mentioned only in the abstract; a stable citation or footnote in the main text would improve traceability.","section":null}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the detailed and constructive review. The comments highlight important points regarding the justification of CRH's identifiability claims and the need for additional experimental controls. We address each below and commit to revisions that strengthen the manuscript without altering its core claims.","responses":[{"response":"We agree that an explicit isolation argument is needed. The CRH formalization defines the axis as the principal direction of difference vectors (concept-present minus concept-absent), which isolates the dominant linear contribution even when secondary concepts overlap in the representation space. The normal plane is then the orthogonal complement, recoverable via the null space of the axis. However, the manuscript does not provide a dedicated lemma or decomposition showing why overlap does not contaminate the plane identification. We will add a short mathematical subsection deriving this from the linear superposition model, demonstrating that the axis remains the leading eigenvector and the plane is identifiable as long as the target concept has non-zero projection on the difference vector.","revision_made":"yes","referee_comment":"[Abstract / CRH formalization] Abstract and the formalization of CRH: the load-bearing assertion that 'the surrounding normal plane can be reliably identified from difference vectors' while the sensitive sector cannot must be justified against the possibility that difference vectors entangle multiple overlapping concepts. The hypothesis itself invokes overlapping contributions, yet no explicit argument or isolation mechanism is supplied showing why the recovered axis and plane remain concept-pure."},{"response":"This is a valid request for stronger validation. Our existing experiments demonstrate the cylindrical geometry on natural activations and steering tasks, but they do not include synthetic ablations with injected concept overlap. We will add a controlled ablation subsection that generates difference vectors under varying overlap levels (via linear combinations of known directions) and reports metrics on axis recovery accuracy and plane stability. This directly tests the identifiability claim under the overlapping regime assumed by CRH.","revision_made":"yes","referee_comment":"[Experiments] Experiments section: verification of the cylindrical structure should include controls or ablations that test robustness of axis and plane recovery when difference vectors are constructed under controlled concept overlap, directly addressing whether the claimed reliable identification of the normal plane survives the regime the hypothesis assumes."}],"tokens_in":1422,"tokens_out":478,"duration_ms":25353,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The main point is that this paper formalizes the Cylindrical Representation Hypothesis to explain why steering directions produce unstable results even when they look well-aligned. By dropping the strict orthogonality assumption in LRH while keeping linear representations, overlapping concept contributions create a sample-specific structure: a central axis from difference vectors that drives the concept, a surrounding normal plane that sets sensitivity, and sensitive sectors inside that plane whose effects cannot be pinned down. That unidentifiability is presented as the source of the observed fluctuations.\n\nWhat is actually new is the specific cylindrical formalization and the explicit separation between the recoverable plane and the unrecoverable sectors. The paper does a reasonable job spelling out the geometry and making the link to practical steering problems, and it ships code so others can check the claims.\n\nThe soft spot is the one flagged in the stress-test. If difference vectors already mix contributions from multiple overlapping concepts, there is no obvious reason the recovered axis will be the pure main difference or the normal plane will be cleanly separable. The abstract asserts reliability for the plane but gives no derivation, error analysis, or mechanism that would keep the vectors concept-pure in the overlapping regime the hypothesis itself assumes. Without seeing quantitative checks that directly test this separation, the central claim rests on an assumption that may not hold.\n\nThis is for people working on interpretability and controllable generation in language models. A reader who wants a new geometric lens on why steering is finicky will get something to think about, even if the evidence is still preliminary. It deserves a serious referee because it engages a real deployed problem with a concrete alternative to LRH, though the experiments will need closer scrutiny on the identifiability question.","headline":"CRH offers a geometric account of steering instability by relaxing LRH orthogonality, but the load-bearing claim that difference vectors cleanly yield an identifiable axis and plane while sectors remain unidentifiable is undercut by likely concept entanglement.","tokens_in":2381,"tokens_out":431,"would_cite":false,"duration_ms":20060,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"Overlapping concepts in language models create a cylindrical structure that accounts for steering fluctuations.","keywords":["cylindrical representation hypothesis","language model steering","linear representation hypothesis","concept representations","steering sensitivity","activation uncertainty","overlapping concepts"],"falsifier":"Finding that steering outcomes remain stable and predictable even without knowing the sensitive sector, or failing to observe the cylindrical structure in activation difference vectors across samples.","tokens_in":2700,"feed_emoji":"🌀","tokens_out":691,"duration_ms":23077,"temperature":0.7,"pith_summary":"The paper proposes that relaxing the orthogonality requirement of the Linear Representation Hypothesis while keeping linear representations leads to a sample-specific cylindrical structure for concepts. This structure consists of a central axis that drives the difference between concept presence and absence, surrounded by a normal plane that determines how sensitive steering is to activation. Within that plane, only certain sectors strongly activate the concept, while others suppress it, creating uncertainty because the sector cannot be reliably identified from difference vectors. A reader would care because this explains why steering large language models often produces unpredictable results even with good direction vectors. The experiments confirm this cylindrical form exists in practice and helps interpret real steering behavior.","feed_headline":"Cylindrical model explains fluctuating LLM steering results","feed_subtitle":"A central axis drives concepts but an unidentifiable sensitive sector in the surrounding plane causes outcome variations.","key_machinery":"The Cylindrical Representation Hypothesis (CRH), a model of concept representation with a central axis for presence difference and a normal plane containing sensitive sectors that control activation ease.","core_discovery":"By relaxing LRH's orthogonality assumption while preserving linear representations, overlapping concept contributions naturally yield a sample-specific axis-orthogonal structure formalized as the Cylindrical Representation Hypothesis (CRH). In CRH, a central axis captures the main difference between concept absence and presence and drives concept generation. A surrounding normal plane controls steering sensitivity by determining how easily the axis can activate the target concept. Within this plane, only specific sensitive sectors strongly facilitate concept activation, while other sectors can suppress or delay it. The surrounding normal plane can be reliably identified from difference vecto","pith_inferences":["Steering methods might be improved by attempting to estimate or average over possible sectors in the plane.","This cylindrical view could apply to other control techniques beyond steering in language models.","Visualizing the normal plane for multiple samples might reveal patterns in sector locations.","Extensions could test whether training affects the size or structure of the sensitive sectors."],"forward_implications":["Steering using the identified axis direction will still vary in outcome depending on the sample's location in the normal plane.","The reliable identification of the normal plane allows consistent axis finding, but sector uncertainty causes observed fluctuations.","Concept activation can be facilitated or delayed based on the sector position relative to the sensitive one.","Linear representations are preserved, but overlaps lead to this cylindrical rather than purely orthogonal form."],"fun_headline_variants":["CRH explains cylindrical structure behind steering variations","Cylindrical hypothesis accounts for unstable model steering","Overlapping concepts yield cylindrical LLM representations","Plane sensitivity leads to fluctuating steering outcomes","Sample-specific cylinders explain steering unpredictability"],"cache_read_input_tokens":2112,"weakest_assumption_plain":"The surrounding normal plane can be reliably identified from difference vectors while the sensitive sector cannot, thereby introducing intrinsic uncertainty at the sector level.","fun_headline_variants_meta":{"raw":{"variants":["CRH explains cylindrical structure behind steering variations","Cylindrical hypothesis accounts for unstable model steering","Overlapping concepts yield cylindrical LLM representations","Plane sensitivity leads to fluctuating steering outcomes","Sample-specific cylinders explain steering unpredictability"]},"model":"grok-4.3","cost_usd":0.00488,"raw_usage":{"total_tokens":2416,"prompt_tokens":713,"num_sources_used":0,"completion_tokens":61,"cost_in_usd_ticks":48799500,"prompt_tokens_details":{"text_tokens":713,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1642,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":713,"tokens_out":61,"duration_ms":18389,"temperature":1.0,"reasoning_tokens":1642,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-01T00:07:18.297839+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"Finding that steering outcomes remain stable and predictable even without knowing the sensitive sector, or failing to observe the cylindrical structure in activation difference vectors across samples.","supporting_citations":[],"review_version":2}