Fluid Reasoning Representations

Alessandro Stolfo; Arth Singh; Arthur Conmy; Dmitrii Kharlapenko; Mrinmaya Sachan; Terry Jingchen Zhang; Zhijing Jin

REVIEW 3 major objections 5 minor 7 references

Fluid Reasoning Representations

T0 review · 3 major / 5 minor · reviewed 2026-08-03 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read During extended reasoning, QwQ-32B's internal representations of actions converge to abstract, naming-independent encodings, and steering those encodings measurably changes problem-solving accuracy.

desk verdict A credible descriptive result about representational convergence during extended reasoning, but the key metric is partly self-referential and the causal evidence is statistically fragile; still worth refereeing. read the letter →

arxiv 2602.04843 v2 pith:F7MRSMUG submitted 2026-02-04 cs.AI

Dmitrii Kharlapenko , Terry Jingchen Zhang , Arth Singh , Alessandro Stolfo , Arthur Conmy , Mrinmaya Sachan , Zhijing Jin This is my paper · ORCID

classification cs.AI

keywords FluidReasoningRepresentationsmechanisticinterpretabilitylanguagemodelsMysteryBlocksWorldactivationsteeringsymbolicabstractionrepresentationconvergencechain-of-thought

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish why reasoning language models outperform ordinary LLMs on abstract tasks whose vocabulary has been deliberately scrambled: not merely because they search longer, but because their internal vector representations of actions and predicates are progressively rebuilt during the reasoning trace. Using Mystery BlocksWorld, where 'pick up' becomes 'attack' and 'on top of' becomes 'craves', the authors show that QwQ-32B's hidden states for the same logical action grow more similar across different wordings as reasoning proceeds, and move toward the representations the model uses on the unobfuscated task. They then show these changes matter: injecting the refined representations into early reasoning improves held-out accuracy beyond random controls, while subtracting or scrambling them degrades accuracy. The paper names this phenomenon Fluid Reasoning Representations and argues it is a general capability present in base models too, with extended thinking supplying the context needed to activate it. A sympathetic reader would care because it offers a concrete, testable mechanism for what 'thinking longer' does inside a model, with implications for both interpretability and intervention.

What carries the argument

The load-bearing object is the cross-naming representation, defined as the average of centered in-naming representations across all namings (Eq. 3): for each action, one subtracts the mean over actions within a naming, then averages the resulting directions across 14 obfuscated wordings to cancel naming-specific surface features and expose an invariant semantic core. This averaged vector serves both as the target for measuring representational convergence and as the steering/patching vector in the causal experiments. The steering rule itself — a norm-preserving convex combination of the token's hidden state with the target vector — is the mechanism by which the paper injects or subtracts the

What would settle it

Compute the convergence metric with a leave-one-out cross-naming average (exclude the target naming from the mean) and check whether similarity still rises during reasoning; if it does not, the observed convergence is an artifact of the estimator. A second check: steer with cross-naming vectors built from namings never used in the steering set; if held-out accuracy does not improve, the vectors are not naming-invariant.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central discovery is that QwQ-32B dynamically constructs context-specific representational spaces during extended reasoning. Over the course of a 15–20k-token chain of thought on semantically obfuscated BlocksWorld puzzles, hidden-state representations of the same action or predicate across different namings become more similar to each other, more similar to the clean BlocksWorld representations, and more separated from representations of different actions. The paper introduces Fluid Reasoning Representations (FRRs) to name this progressive refinement, and reports causal evidence: positive steering with refined representations, especially cross-naming averages,

Load-bearing premise

The cross-naming representation is an average of centered representations that includes the very naming being compared, and the paper assumes this average cancels surface features while preserving semantic content; if instead that average captures shared syntax, trace position, or a generic action-token direction, the convergence and steering results would not demonstrate symbolic abstraction.

Editorial extensions

If this is right

If correct, accuracy gains on obfuscated tasks are partly attributable to in-context representational refinement, not just extended search or memorized shortcuts.
Cross-naming steering transfers task knowledge across wordings, implying learned abstraction is portable across surface forms.
Base and instruction-tuned models already exhibit the refinement dynamic, so reasoning-model training may mainly supply the long context that lets the dynamic unfold.
Steering and patching offer a practical handle: model behavior on abstract tasks can be improved or disrupted by editing these refined directions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

I infer that the same convergence dynamic should appear in other fixed-action-space planning domains (e.g., Towers of Hanoi) and in mathematical reasoning with redefined notation, though the paper only tests BlocksWorld.
A sharper test than the paper's would use leave-one-out cross-naming averages to confirm the convergence is not an artifact of measuring similarity to a mean that includes the measured point.
If the shared direction in late-layer action vectors (seen in shuffled-control gains) is a generic 'action-ness' component, then part of the steering effect may come from injecting task-generic structure rather than naming-invariant semantics; the paper flags but does not resolve this.
One could test transfer by computing cross-naming vectors from one set of namings and steering on a held-out naming; the paper's Table 4 hints at heterogeneous per-naming responsiveness that such a design would quantify.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Short version: this is a well-designed descriptive study with a plausible mechanism story, but the headline convergence metric in Fig 4a is inflated by a self-inclusive average, and the causal effects are small enough that the strong framing in the abstract outruns the evidence. Still, it deserves a serious referee; the authors have done enough independent checks that the central direction likely holds.

What's actually new: applying the in-context representation learning methodology from Park et al. to 15–20k-token reasoning traces in Mystery BlocksWorld with 15 namings, and showing that action/predicate representations converge across wordings as reasoning proceeds. That is a new phenomenon and the paper documents it carefully: Fig 2 gives a leave-one-out pairwise analysis for naming 1, Fig 4b compares mystery representations against clean BlocksWorld, and the appendix includes several other models plus base-model comparisons. The paper is honest about its limitations and discloses exploratory controls. Code is open-sourced; that's real.

Where the soft spots are: the central convergence plot, Fig 4a, measures each naming's similarity to a cross-naming average that includes that naming (Eq. 3). That mechanically inflates the same-action similarity and can also create the increasingly negative different-action similarity. The concern lands. The same average is then used as steering vectors, so the causal experiments inherit the potential contamination. The paper does provide non-self-referential support—Fig 2's leave-one-out and the clean BlocksWorld comparison—so this is not fatal, but the authors should redo the analysis with leave-one-out averaging and report confidence intervals and significance tests for the similarity curves.

The steering statistics are fragile: Table 5 runs one t-test per layer and condition; only three are nominally significant at p<0.05, and no multiple-comparison correction is applied. The effect sizes (1.4–1.8% accuracy on top of a ~30% baseline) are modest, and some early-layer improvements appear even with random vectors. The layer/scale/window choices are post hoc. These concerns are real, but proportionate: the pattern across layers and controls suggests something is there.

Who should read it: people working on mechanistic interpretability of reasoning models, and anyone using steering as evidence for representation-level claims. It is not a home run, and the abstract overstates the strength of the evidence, but it is a genuine empirical contribution.

Recommendation: send to peer review, with both the leave-one-out control and the multiple-comparison issue as mandatory revision points. If the authors fix those faithfully, the paper would be a solid contribution.

Referee Report

3 major / 5 minor

Summary. The paper introduces 'Fluid Reasoning Representations' (FRRs) and studies QwQ-32B on Mystery BlocksWorld, a semantically obfuscated planning domain. The authors extract centered action/predicate representations from hidden states at different reasoning timestamps and define cross-naming representations as averages over 15 namings. They report that (i) same-action representations across different namings become more similar over the reasoning trace and move toward clean BlocksWorld representations; (ii) positive steering with refined in-naming and cross-naming vectors improves held-out accuracy, with cross-naming vectors most effective; (iii) symbolic patching with cross-naming vectors outperforms shuffled controls; and (iv) subtracting refined directions degrades accuracy. They conclude that extended reasoning dynamically constructs context-specific, naming-invariant representational spaces, and that this is a key mechanism behind reasoning models' performance on abstract tasks. The paper includes open-source code, multiple naming variants, and a base-model comparison.

Significance. If the findings hold, this is a valuable mechanistic contribution to interpretability of reasoning LLMs: it identifies a concrete representational process—dynamic refinement toward shared symbolic encodings—that could explain part of the reasoning model advantage on obfuscated tasks. The paper has notable strengths: it provides a codebase, uses 15 naming variants, includes leave-one-out pairwise evidence in Fig. 2, compares against clean BlocksWorld in Fig. 4b, and attempts causal validation through steering and patching with controls. However, the central convergence metric and the causal vectors both rely on a cross-naming average that includes the query naming (Eq. 3), and the statistical support for the causal claims is thin once multiple testing is considered. The core idea is plausible and testable, but the current evidence requires additional analyses before the claims are fully supported.

major comments (3)

[Section 3.2, Eq. (3), Fig. 4a] The main convergence plot measures each naming's centered representation against the cross-naming average defined in Eq. (3), which is computed over all namings including the query naming itself. Since each naming contributes to the average, the similarity at the extraction timestamp has a mechanical self-inclusion component; similarly, the 'different actions' similarity is computed against an average that contains the query action. This inflates the observed convergence and separation. The same cross-naming representations are used as steering vectors in Section 4.1 and as the basis for symbolic patching in Section 4.2, so if the average encodes naming-common surface statistics or trace-position/syntax directions rather than a clean invariant semantic core, the causal conclusions are also affected. Please provide a leave-one-out version of Fig. 4a (and, where feasible, leave-one-out ste
[Section L, Table 5] The statistical evidence for steering improvements is not robust to multiple testing. Table 5 reports 10 one-sided t-tests at alpha=0.05; 3 are significant, but the smallest p-value is 0.021, which is above a Bonferroni threshold of 0.005. Under 10 independent tests, about 0.5 false positives are expected by chance. The claim in Fig. 6 that 'cross-naming > in-naming >> random' is also not supported by Table 5: at layer 30 cross-naming has p=0.139, and at layer 40 in-naming has p=0.274. Moreover, Figs. 2, 4, and 5 show no confidence intervals or significance tests. Please report adjusted p-values (e.g., Benjamini-Hochberg), provide confidence intervals for the mean improvements, and consider pre-registering or limiting the primary comparisons to avoid selection effects from the reported layer/condition search.
[Section 4.2, Eq. (6), Fig. 7] The symbolic patching experiment inherits the self-inclusion issue: the 'symbolic' vectors are constructed from the same cross-naming averages of Eq. (3), which include the naming being patched. This makes it difficult to attribute the observed advantage over shuffled control to naming-invariant abstraction rather than to shared naming-specific statistics. In addition, Fig. 7 provides no statistical test; the error bars (s.e. across namings) are descriptive. A leave-one-out variant of the symbolic vectors (excluding the target naming from the average) would directly address the abstraction claim. The paper's existing non-self-referential evidence (Fig. 2 and Fig. 4b) partially supports convergence, but it does not validate the specific vectors used in the causal experiments.

minor comments (5)

[Section 2.2] The statement that 'some version of Mystery BlocksWorld was present in QwQ-32B's training data' is an inference from shorter traces and manual analysis. This is an important exclusion criterion (naming 3 is dropped), but it is not directly verified. Please soften the claim or provide additional evidence, and discuss how residual partial recognition of other namings could affect the results.
[Section 3.3] The base-model comparison feeds QwQ-generated traces to the base model. Differences in adaptation speed may reflect distribution shift (base model processing unnatural traces) rather than fundamental model capability. A cleaner comparison would use base-model-generated traces (or at least control for trace difficulty), or the claim should be limited to 'base models can adapt in-context when given a long context'.
[Section 2.3 vs. Appendix F.1] The main text says the window size is w=100, while Appendix F.1 states '200 is selected to contain at least 10 action mentions.' Please reconcile this inconsistency.
[Appendix F.1] Minor typo: 'preformed' should be 'performed'.
[Section 4.1] The notation v_type[a] is introduced without a formal definition; it refers to the three vector types (in-naming, cross-naming, random). Please define it explicitly.

Circularity Check

2 steps flagged · score 5.0 of 10

Cross-naming average in Eq. (3) is self-inclusive, so Fig. 4a convergence and cross-naming steering/patching partially reduce to the estimator; independent pairwise and clean-BlocksWorld controls limit the damage.

fitted input called prediction [Eq. (3), Section 3.2, Fig. 4a]
"To extract the abstract, symbolic meaning of an action (predicate) independent of its surface form, we compute cross-naming representations by averaging centered in-naming representations across all namings: ¯r^{L,T}_a = 1/|N| Σ_{N∈N} ˜r^{N,L,T}_a (3) ... (a) Shows average similarities of centered action/predicate representations from all timestamps in Mystery Blocksworld traces with cross-naming representations extracted at 7k tokens."

The target of the convergence measurement is defined as the average of the very representations being measured. Each naming contributes to the cross-naming average, so same-action similarity rises partly because every query vector is compared with a target containing itself; the 7k target also contains the 7k version of each query naming, so the temporal increase from 2k to 7k is partially mechanical. The paper does not report a leave-one-out variant of Fig. 4a. This does not fully determine the result because Fig. 2 (naming 1 vs. others) and Fig. 4b (clean BlocksWorld) are not self-inclusive, but the headline convergence plot is partly built into the estimator.
fitted input called prediction [Section 4.1, Eq. (3)/(4), Figure 6; Section 4.2, Eq. (6)]
"We collect three types of steering vectors at layer L from the 40 correctly solved puzzles: (1) centered in-naming representations..., (2) cross-naming representations ¯r^{L,T}_a averaged across all namings... Cross-naming representations achieve the highest impact across all layers, reinforcing that these adaptations encode abstract problem structure rather than naming-specific artifacts."

The cross-naming steering vector is averaged over all namings including the naming whose held-out rollouts are being steered. Thus the claimed superiority of 'cross-naming' over 'in-naming' does not demonstrate naming-invariant transfer: the vector can carry the target naming's own in-naming statistics plus smoothing. The same self-inclusive average is used as the 'symbolic' replacement in Section 4.2. Steering still uses held-out puzzles and random/shuffled controls, so it is not a pure tautology, but the 'symbolic abstraction' interpretation is not tested leave-one-out.

full rationale

The most load-bearing representational evidence—Fig. 4a, and the cross-naming steering/patching vectors in Sections 4.1 and 4.2—uses cross-naming representations computed by Eq. (3) as an average over all namings, including the naming being compared, steered, or patched. This makes part of the observed convergence and the apparent cross-naming transfer mechanical: the query is compared to or replaced by a vector that contains the query's own contributions. However, the paper is not wholly circular: Fig. 2 uses naming 1 against all other namings (leave-one-out for the query), and Fig. 4b compares mystery-naming representations to clean BlocksWorld representations, giving independent evidence that some convergent, semantics-independent structure exists. The steering experiments also use held-out puzzles with random and shuffled controls, so the causal effect is not purely a definitional artifact. The self-citations (e.g., Venhoff et al. for centering) are methodological and not load-bearing uniqueness claims. Overall, the central 'convergence to a shared symbolic code' claim is partially inflated by the self-inclusive estimator, but it retains independent support; a leave-one-out version of Eq. (3) would directly resolve the concern.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claims rest on two heavy assumptions: that averaged hidden states at token positions isolate action/predicate concepts, and that averaging across namings removes surface form. The hand-fitted quantities are the steering scale, windows, timestamps, and analysis layer. No new physical or architectural entities are introduced; 'Fluid Reasoning Representations' is a descriptive label for the observed dynamics, not a new mechanism with independent falsifiable handles.

free parameters (4)

steering scale s = 2/3 (positive steering); symbolic patching uses s ∈ {10, 20}
Chosen after a hyperparameter sweep on layer 20 (Fig. 10, Sec. J.2) to maximize accuracy improvement, then applied across layers and conditions without per-naming fitting.
steering token window [t_start, t_end) = [1500, 2500] for positive steering; [2000, 4000] for patching/negative steering
Intervention windows are fixed by hand; no sensitivity analysis is reported for the window boundaries.
extraction timestamp for steering vectors = 7k tokens (positive steering); 4k tokens (negative steering)
Selected as 'near convergence' based on the same similarity curves being tested, coupling the intervention target to the descriptive result.
analysis layer L=40 = Layer 40 (layer 30 also checked)
Layer chosen after observing where action representation separation 'converged' in PCA (Sec. D); a post hoc selection.

assumptions (5)

domain assumption Concepts are linearly represented as directions in residual-stream activation space, so additive steering of averaged token hidden states modifies the concept.
Invoked in Eq. (4)-(5) and throughout the linear-representation discussion in Related Work.
domain assumption Averaging hidden states across token occurrences, traces, and namings preserves the semantic content of an action/predicate and cancels surface-form features.
Eqs. (1)-(3); no independent probe validates that the averaged vector isolates the intended concept.
ad hoc to paper Mystery naming 3 was present in QwQ-32B training data, based on shorter traces and manual trace analysis.
Used to justify excluding naming 3 from representational analyses (Sec. 2.2); inferred post hoc and not externally verified.
domain assumption Successfully solved puzzles used to extract steering vectors are representative of the reasoning process for held-out puzzles.
Steering vectors come from 40 correctly solved puzzles; if successful traces differ systematically from all traces, the causal transfer claim weakens.
domain assumption The 15 mystery namings are independent obfuscations of the same domain, and averaging across them gives a stable symbolic estimate.
Naming variants were generated by hand/LLM; no test shows independence or absence of shared surface features across namings.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Fluid Reasoning Representations." pith.science (2026). https://pith.science/paper/F7MRSMUG

@misc{pith2026260204843,
  author       = {Pith},
  title        = {Pith review of: Fluid Reasoning Representations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/F7MRSMUG}},
  note         = {Machine review of arXiv:2602.04843}
}

read the original abstract

Frontier large language models increasingly solve complex tasks involving abstract concepts through extended test-time thinking. Yet we lack a mechanistic account of how extended thinking changes hidden-state representations over the course of a reasoning trace. We introduce \textit{Fluid Reasoning Representations} (FRRs), a representation-level account of how LLMs organize action and predicate concepts during self-generated reasoning, and test them on obfuscated planning, symbolic, and mathematical tasks where task-relevant words are replaced while problem structure is preserved. Across open-weight base, instruction-tuned, and extended-thinking LLMs, representations of the same action or predicate become more similar across wordings and move toward the corresponding unobfuscated concepts over the reasoning trace. Causal probes show that these representations affect behavior: cross-naming steering improves held-out accuracy beyond Gaussian and shuffled controls, symbolic patching retains more action information than shuffled patching, and subtracting refined directions degrades accuracy; together, these results suggest that extended thinking strengthens a representation dynamic already present at lower magnitude in base and instruction-tuned LLMs. Our codebase is open-sourced \href{https://github.com/AI4Collaboration/Fluid-Reasoning-Representation}{here}.

Figures

Figures reproduced from arXiv: 2602.04843 by the authors.

**Figure 1.** Overview of our pipeline. Left: QwQ-32B’s accuracy on Standard BlocksWorld is 96%. Center: Mystery BlocksWorld obfuscates semantics (e.g., “pick up” → “attack”), reducing QwQ’s accuracy to 33%. During extended reasoning traces, the model progressively refines internal representations of obfuscated actions, developing abstract symbolic encodings (vectors v0, . . . , v3, and u0, . . . , u3 are extracted at different … view at source ↗

**Figure 2.** Average similarity of representations from other namings with naming 1 representations, extracted from different timestamps. −40 −20 0 20 −40 −30 −20 −10 0 10 20 30 40 −40 −20 0 20 40 −40 −20 0 20 40 Layer 10 Layer 40 [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗

**Figure 4.** Similarity with cross-naming representations between Mystery and Original BlocksWorld traces. (a) Shows average similarities of centered action/predicate representations from all timestamps in Mystery Blocksworld traces with cross-naming representations extracted at 7k tokens. Note that similarities between different actions become increasingly negative. (b) Shows average similarities of clean BlocksWorld representa… view at source ↗

Figures from the paper (6 more)

**Figure 5.** Figure 5: Average similarity of representations extracted from the 7k timestamp, plotted for both [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Accuracy improvement after positive steering averaged across mystery namings (excluding Naming 3). Takeaways: (i) Even random early-layer interventions (L ≤10) already improve accuracy, suggesting they help remove surfacelevel naming associations. (ii) From L ≥ 20 on…

**Figure 8.** Figure 8: Layer-wise PCA of action representations from different mystery namings extracted at 7k [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]

**Figure 9.** Figure 9: Similarities with cross-naming average representations across different reasoning models. [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]

**Figure 10.** Figure 10: Positive steering results for layer 20 using different scale parameters [PITH_FULL_IMAGE:figures/full_fig_p023_10.png]

**Figure 11.** Figure 11: Accuracy change under shuffled in-naming steering. See Section K. Caveat. These runs covered fewer namings/layers and late-layer effects were heterogeneous. We report them here for transparency; a fuller sweep is left to future work. L STATISTICAL ANALYSIS OF STEERING…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

7 extracted references · 3 linked inside Pith

[3]

with approximately 100 puzzles per naming (indices 200-300). Each mystery naming provides accuracy measurements under multiple conditions: baseline (no steering), in-naming steering, cross- naming steering, and random Gaussian steering at various layers. Statistical Test.We employ one-sample t-tests to assess whether mean accuracy improvements across myst...

2026
[10]

This suggests that certain semantic structures are more amenable to representational refinement than others

demonstrate substantial gains. This suggests that certain semantic structures are more amenable to representational refinement than others. J IMPLEMENTATION DETAILS J.1 STEERING ENGINE We implement steering using PyTorch forward hooks on top of vLLM v0.7.3 v0 Kwon et al. (2023), which provides substantial performance improvements, reducing experiment runt...

2023
[11]

show minimal or no improvement from naming-mean steering, while others (Mystery 1, Mystery
[2013]

Neel Nanda and Joseph Bloom

URLhttps://proceedings.neurips.cc/paper_files/paper/2013/ file/9aa42b31882ec039965f3c4923ce901b-Paper.pdf. Neel Nanda and Joseph Bloom. Transformerlens, 2022. URLhttps://github.com/ TransformerLensOrg/TransformerLens. Neel Nanda, Andrew Lee, and Martin Wattenberg. Emergent linear representations in world mod- els of self-supervised sequence models. In Yon...

2013
[2016]

URLhttps://proceedings.neurips.cc/paper_files/paper/2016/ file/a486cd07e4ac3d270571622f4f316ec5-Paper.pdf. Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, ...

arXiv 2016
[2023]

doi: 10.18653/v1/2023.blackboxnlp-1.2

Association for Computational Linguistics. doi: 10.18653/v1/2023.blackboxnlp-1.2. URL https://aclanthology.org/2023.blackboxnlp-1.2. Christopher Olah. What is a linear representation? what is a multidimensional feature?Trans- former Circuits Thread, 2024. URLhttps://transformer-circuits.pub/2024/ july-update/. OpenAI. Openai o1 system card, 2024. URLhttps...

arXiv 2023
[2025]

11 Subhabrata Dutta, Joykirat Singh, Soumen Chakrabarti, and Tanmoy Chakraborty

URLhttps://arxiv.org/abs/2501.12948. 11 Subhabrata Dutta, Joykirat Singh, Soumen Chakrabarti, and Tanmoy Chakraborty. How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning, 2024. URLhttps: //arxiv.org/abs/2402.18312. Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao B...

arXiv 2024

Pith tools

Reviewed August 3, 2026 · model on record in the stance chip above.

[1] [3]

with approximately 100 puzzles per naming (indices 200-300). Each mystery naming provides accuracy measurements under multiple conditions: baseline (no steering), in-naming steering, cross- naming steering, and random Gaussian steering at various layers. Statistical Test.We employ one-sample t-tests to assess whether mean accuracy improvements across myst...

2026

[2] [10]

This suggests that certain semantic structures are more amenable to representational refinement than others

demonstrate substantial gains. This suggests that certain semantic structures are more amenable to representational refinement than others. J IMPLEMENTATION DETAILS J.1 STEERING ENGINE We implement steering using PyTorch forward hooks on top of vLLM v0.7.3 v0 Kwon et al. (2023), which provides substantial performance improvements, reducing experiment runt...

2023

[3] [11]

show minimal or no improvement from naming-mean steering, while others (Mystery 1, Mystery

[4] [2013]

Neel Nanda and Joseph Bloom

URLhttps://proceedings.neurips.cc/paper_files/paper/2013/ file/9aa42b31882ec039965f3c4923ce901b-Paper.pdf. Neel Nanda and Joseph Bloom. Transformerlens, 2022. URLhttps://github.com/ TransformerLensOrg/TransformerLens. Neel Nanda, Andrew Lee, and Martin Wattenberg. Emergent linear representations in world mod- els of self-supervised sequence models. In Yon...

2013

[5] [2016]

URLhttps://proceedings.neurips.cc/paper_files/paper/2016/ file/a486cd07e4ac3d270571622f4f316ec5-Paper.pdf. Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, ...

arXiv 2016

[6] [2023]

doi: 10.18653/v1/2023.blackboxnlp-1.2

Association for Computational Linguistics. doi: 10.18653/v1/2023.blackboxnlp-1.2. URL https://aclanthology.org/2023.blackboxnlp-1.2. Christopher Olah. What is a linear representation? what is a multidimensional feature?Trans- former Circuits Thread, 2024. URLhttps://transformer-circuits.pub/2024/ july-update/. OpenAI. Openai o1 system card, 2024. URLhttps...

arXiv 2023

[7] [2025]

11 Subhabrata Dutta, Joykirat Singh, Soumen Chakrabarti, and Tanmoy Chakraborty

URLhttps://arxiv.org/abs/2501.12948. 11 Subhabrata Dutta, Joykirat Singh, Soumen Chakrabarti, and Tanmoy Chakraborty. How to think step-by-step: A mechanistic understanding of chain-of-thought reasoning, 2024. URLhttps: //arxiv.org/abs/2402.18312. Nelson Elhage, Neel Nanda, Catherine Olsson, Tom Henighan, Nicholas Joseph, Ben Mann, Amanda Askell, Yuntao B...

arXiv 2024