Pith. sign in

REVIEW 3 major objections 4 minor 26 references

The paper claims that tested language models carry a compact, reusable causal interface—a shared code several future operations read—rather than strictly task-specific circuits.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 04:32 UTC pith:GCGKT2DI

load-bearing objection A genuinely novel, carefully controlled empirical study whose headline numbers rest on a coding convention that the paper itself says must be checked — and the check is missing. the 3 major comments →

arxiv 2607.27617 v1 pith:GCGKT2DI submitted 2026-07-30 cs.AI

Hidden APIs in Language Models: Discovering Reusable Causal Interfaces from Forked Futures

SiYuan Ma , Yiqin Luo , Zhangji , Canran Xiao , Albert Gao , Wei-Hsing Huang , Wei Wang , Qiwei Wu
show 3 more authors
Xinran Li Jinfeng Wei Qixin Zhang
This is my paper
classification cs.AI
keywords forked futurescausal interfaceshidden-state equivalenceprequential causal MDLmechanistic interpretabilitytransplantation analysismediation analysislanguage models
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to show that a language model can harbor a compact, reusable causal interface—a shared 'API'—that multiple future computations read from a hidden state. It defines hidden-state equivalence by 'forked futures': sample the future operation only after the prefix state exists and compare whole response distributions, not the current answer. Four candidate organizations (Shared, Local, Mixture, Distributed) then compete under prequential causal minimum description length with a fidelity constraint. On Qwen2.5-1.5B and Llama-3-8B, Shared has the lowest held-out description length at essentially equal future-signature distortion; role-aligned transplantation and path mediation indicate the original model actually consumes that interface. The claim is explicitly conditional on the tested operation banks, architectures, and held-out futures.

Core claim

On the paper's own terms, the discovery is that a future-bearing hidden state is well described by one operation-agnostic code. Shared wins the held-out prequential causal MDL contest on both detailed models—SG = 0.216 nats (Qwen2.5-1.5B) and 0.294 nats (Llama-3-8B), bootstrap CIs excluding zero—while mean, worst-family, and tail future-signature distortion stay clustered. Shared also carries the strongest joint transplantation, locality, and copy-preservation profile, and API-aligned paths mediate 0.749 of the target effect versus 0.150 for matched nulls. In a blind four-class audit, 14/16 organisms are recovered with one non-Shared-to-Shared error, so the claim is kept explicitly condition

What carries the argument

The load-bearing device is 'forked futures' plus the future causal signature: for a cached prefix state h, an operation k sampled after the fact induces a response distribution σ_k(h), and the collection over the operation bank is Σ_K(h). Two states are empirically future-equivalent when the Jensen-Shannon distortion between such signatures is small (Eq. 3). Competing encoder–reconstructor architectures (Shared, Local, Mixture, Distributed) are then scored by prequential causal MDL subject to a worst-family fidelity constraint, so a compact code can win only if it predicts held-out future distributions without discarding a difficult family. The selected code is validated by role-aligned tran

Load-bearing premise

The entire result rests on the operation bank and the Jensen-Shannon distortion defining which differences between hidden states count as causally relevant; the paper itself notes that states differing only under an unsampled operation will be treated as equivalent, so a richer or decision-weighted bank could split the shared quotient.

What would settle it

Run the same pipeline with an operation bank that adds a rare, decision-critical continuation (or up-weights low-probability futures in the distortion), and check whether Shared still wins with equivalent worst-family FSD. If the added operation reverses the MDL ordering, or if a hidden-state pair the Shared code merges turns out to diverge behaviorally under that operation, the reusable-interface claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If correct, the tested models offer a practical target for causal intervention: editing the shared code should alter many downstream operations at once without destroying unrelated state.
  • The result implies that future-signature fidelity can be held by a low-rank (rank-64) operation-agnostic bottleneck, so interpretability tools can look at one interface instead of hundreds of per-task circuits.
  • Because the effect replicates on two model families and keeps the same sign across five backbones, the architecture preference is a property worth probing in other LLM families.
  • The nonzero 8.3% false-Shared rate in ground-truth organisms means any use of the method must report specificity, not just accuracy.
  • Distribution shifts (unseen operations, compositions, generators, families) preserve the ordering but narrow the gap, so the shared interface is transferable but not domain-invariant.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • A natural next test is to enlarge the operation bank with rare but decision-critical continuations; if the Shared code misses those distinctions, the interface shrinks to the sampled bank.
  • The method could be used as an audit: train the same four-architecture contest on a model suspected of modular or distributed routing and compare false-Shared rates across checkpoints.
  • The paper's capacity-loading and cross-family results hint that real production models may contain a hierarchy of several family-level shared interfaces rather than one global bus—a hypothesis the paper does not itself claim.
  • One could turn the discovery into a training signal: explicitly penalizing per-operation causal description length might encourage models to develop reusable interfaces, making the 'API' a design objective rather than a post-hoc finding.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes 'forked futures' as a way to define hidden-state equivalence: a prefix state is cached before a future operation is sampled, and states are compared through the response distributions induced by a bank of operations. Competing interface architectures (Shared, Local, Mixture, Distributed) are evaluated under prequential causal MDL subject to future-signature fidelity and matched capacity constraints. On Qwen2.5-1.5B and Llama-3-8B, the Shared architecture has the lowest held-out description length (SG = 0.216 and 0.294 nats) at roughly matched FSD. The paper then reports role-aligned transplantation, path mediation versus matched nulls, and a blind model-organism audit (14/16 recovered, one false-Shared error among 12 non-Shared organisms). The conclusion is deliberately conditional: an economical reusable causal interface exists within the tested operation banks, candidate architectures, and interventions.

Significance. If the central claim survives scrutiny, the paper would be a strong contribution: it offers a concrete, falsifiable procedure for distinguishing a genuinely reusable internal state from a low-rank code that merely succeeds on the current answer, and it backs the MDL selection with an unusually broad set of controls. The strengths are real: prequential held-out blocks, semantic/lexical collision controls, matched-capacity alternatives, worst-family and tail FSD reporting, transplantation with locality and copy-preservation checks, mediation against matched null paths, and a blind model-organism test whose false-Shared rate is reported rather than hidden. The principal weakness is that the headline Sharedness Gain is computed under a hand-set MDL coding contract, and the paper itself states in Supplement M.2 that sensitivity to this contract is required but does not provide the analysis. This gap is load-bearing because SG is a difference of MDL values and could reverse under reasonable alternative coding conventions. The claim is otherwise carefully scoped, and the missing sensitivity analysis is fixable within the paper's scope.

major comments (3)
  1. [§5, Eq. (5)–(7), Table 2; Supp. M.2, Table S14] The headline SG values are MDL differences computed under a specific coding contract: parameter cost uses a Gaussian-prior two-part code plus an explicit penalty λ_r·rank + λ_g·gates with (λ_r, λ_g) = (0.02, 0.05). No sensitivity analysis for these weights is reported. Supplement M.2 states that 'coding-weight sensitivity is necessary because MDL comparisons can depend on conventions' and that 'a nominal statement that MDL penalizes complexity is not sufficient if different reasonable coding conventions change the architecture ordering.' This is an explicit acknowledgment that the omitted analysis is load-bearing. Because Shared has no gate cost while Mixture and Distributed are charged for gates/components, and because SG is a minimum over alternatives minus Shared, moderate changes in λ_r or λ_g, or an alternative parameter prior, could change the architecture ordering even with all fi
  2. [Eq. (6), §5, Table S8] The architecture selection rule in Eq. (6) is constrained by a pre-specified family-wise fidelity threshold ε_f, but ε_f is never given numerically in the main text or the supplement. Table S8 reports a TOST margin δ = 0.02 and a non-inferiority margin δ = 0.05, but these are post-selection equivalence tests, not the selection margin used in Eq. (6). Without ε_f, the reader cannot verify that the SG comparison is being made at the declared operating point rather than at a loose margin that allows lossy compression. Report ε_f, how it was selected on validation data, and whether the SG conclusion is stable over a range of ε_f values.
  3. [§Q.1, Eq. (3)] The causal quotient is defined by Jensen–Shannon divergence over a finite, sampled operation bank. §Q.1 openly acknowledges that 'if two hidden states differ only under an operation that is never sampled, the present procedure will treat them as equivalent.' This is a genuine boundary of the method: a future operation outside the bank, or a rare but decision-critical continuation underweighted by JS divergence, could read a distinction that the Shared interface discards. The paper's conditional wording partly addresses this, but the 'reusable' claim would be materially strengthened by an operation-bank expansion analysis: add held-out operation families or substantially enrich the bank and test whether the Shared architecture remains selected or the quotient splits. Without such a test, the reusable-interface claim is as strong as the particular 16+4 operation inventory used here, and th
minor comments (4)
  1. [Table S13] The checkpoint hashes are abbreviated ('sha256:9f3c...a71e', 'sha256:b28d...4c56') and the table itself states they must be replaced. For a reproducibility-focused paper, complete hashes and a software/framework manifest should be part of the final artifact.
  2. [Fig. S3D] The Sharedness-Gain heat map over layer and token position would benefit from a colorbar and explicit numeric scale; as printed it is difficult to read the validation-selected configuration and the surrounding plateau.
  3. [Abstract and Table S5] The abstract says 'one observed non-Shared to Shared error among 12 non-Shared organisms'; Table S5 shows that this was a Local→Shared error and that there was also a Local→Mixture error. The full confusion matrix is clear, but the abstract's phrasing is slightly incomplete.
  4. [Supp. M.2 and V] Several supplement passages use 'should be included' or 'should be reported' for analyses and artifacts (e.g., coding-weight sensitivity, per-seed curves, machine-readable manifests). If these are not yet in the released artifact, the final version should either include them or clearly mark them as deferred; the current text reads as an incomplete reproducibility contract.

Circularity Check

0 steps flagged

No significant circularity: the architecture competition, held-out prequential MDL, and external model-organism falsification keep the derivation self-contained.

full rationale

The central derivation is not circular. Future signatures are generated by the same model being studied, but the causal quotient is defined by an operation bank and Jensen-Shannon distortion before architecture selection, and the Shared/Local/Mixture/Distributed competition is evaluated on held-out prequential blocks under a matched fidelity constraint (Section 3, Eqs. 5-7; Table 2). SG is a measured difference of out-of-sample MDL, not a fitted parameter, and the paper reports confidence intervals, permutation tests, worst-family FSD, and signature-sensitivity sweeps. The model-organism suite provides an external ground-truth calibration: the pipeline observes activations and signatures, not class labels, and it reports 14/16 recovery with one explicit false-Shared error (Table S5), which rules out the discovery rule being forced to Shared by construction. The cited prior work is motivational or benchmark-related and is not load-bearing for the architecture-selection result; there is no chain in which the paper's core claim reduces to a self-citation. The hand-set MDL coding weights (lambda_r=0.02, lambda_g=0.05) and the operation-bank dependence are convention and scope limitations, not definitional identities: Supplement M.2 itself states that coding-weight sensitivity is necessary and the paper explicitly conditions the claim on the tested operation banks (Q.1). These gaps are correctness/robustness concerns, not circular steps. The path-retention rule in Supplement E.2 is outcome-informed, but the reported mediated fraction is a separately measured quantity compared with matched nulls; this is a selection-bias concern rather than a definitional reduction of the headline claim. Overall, the derivation is self-contained and externally anchored.

Axiom & Free-Parameter Ledger

5 free parameters · 4 axioms · 0 invented entities

The central claim rests mainly on the operation bank, the JS-divergence distortion measure, the temporal no-leakage condition, and the MDL coding convention. 'Hidden API' is an interpretive label for the winning Shared architecture, not a new physical or formal entity. The paper itself states several of these assumptions explicitly and treats them as scope limits rather than hiding them.

free parameters (5)
  • MDL complexity penalties (λ_r, λ_g) = λr=0.02, λg=0.05
    Hand-set penalties in prequential causal MDL; they directly influence SG magnitude and ranking, though sensitivity sweeps are reported.
  • Interface rank = 64
    Validation-selected active rank shared across candidates; central to the matched-capacity comparison.
  • FSD fidelity margin ε_f / TOST margin δ = δ=0.02/0.05; ε_f not numerically stated
    Architecture selection is constrained by these pre-specified margins; the choice can alter equivalence/non-inferiority conclusions.
  • Selected layer and token position = layer 19, mid-state position (Qwen)
    Chosen from validation data and frozen before testing; all intervention and mediation results depend on this choice.
  • Number of prequential folds = 20
    Block structure affects the computed description length; matched across candidates but not derived from first principles.
axioms (4)
  • domain assumption Hidden-state equivalence is fully captured by Jensen-Shannon divergence between response distributions over the sampled operation bank (Eq. 3).
    Underpins the causal quotient; if the operation bank or JS divergence misses causally relevant distinctions, states are mislabeled. The paper acknowledges this in Q.1.
  • domain assumption The cached hidden state at layer ℓ/token t is formed before the future operation is revealed, so the encoder cannot read operation identity from the prefix.
    Central temporal control; verified by prompt audits and collision controls, but assumed for the causal interpretation.
  • domain assumption Prequential causal MDL with Gaussian-prior two-part codes and penalties (0.02, 0.05) is a valid architecture-selection criterion.
    The choice is not derived from first principles; the authors argue robustness via sweeps, but other coding conventions could change the ranking.
  • domain assumption Interchange/transplantation and path-blocked mediation identify causal reuse in the original model given matched donor/base and null-path controls.
    Underlies the mechanistic validation; residual confounding from redundant routes and partial overlap are acknowledged in the limitations.

pith-pipeline@v1.3.0-daily-deepseek · 41246 in / 11955 out tokens · 127568 ms · 2026-08-01T04:32:55.418369+00:00 · methodology

0 comments
read the original abstract

Identical language-model answers can arise from hidden states that support different future computations, so current-answer probes do not establish a reusable internal interface. We introduce forked futures: future operations are sampled only after a prefix state has formed, and states are compared through the response distributions induced by those operations. This yields an empirical causal quotient over hidden states without requiring researcher-specified latent labels. Shared, Local, Mixture, and Distributed interfaces then compete under prequential causal description length subject to future-signature fidelity and matched capacity constraints. In the two detailed model evaluations, Shared has the lowest held-out description length, with gains of 0.216 nats on Qwen2.5-1.5B and 0.294 nats on Llama-3-8B, while maintaining tightly clustered mean future-signature distortion; a five-backbone sweep preserves the positive direction of Sharedness Gain. The figure-aligned transplantation analysis gives Shared the strongest joint target-correctness, locality, copy-preservation, and composite profile, and API-aligned paths mediate 0.749 of the target effect versus 0.150 for matched null paths. In the blind four-class model-organism test, 14/16 architectures are recovered, with one observed non-Shared to Shared error among 12 non-Shared organisms. These results support an economical reusable causal interface within the tested operation banks, while keeping the claim explicitly conditional on the candidate architectures, interventions, and held-out futures.

Figures

Figures reproduced from arXiv: 2607.27617 by Albert Gao, Canran Xiao, Jinfeng Wei, Qiwei Wu, Qixin Zhang, Siyuan Ma, Wei-Hsing Huang, Wei Wang, Xinran Li, Yiqin Luo, Zhangji.

Figure 1
Figure 1. Figure 1: Discovery and falsification pipeline. (A–C) Prefix states are cached before future operations [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Ground-truth falsification. (A) Four known routing architectures. (B) Blind recovery [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Architecture competition. (A) Complexity–FSD frontiers. (B) Sharedness Gain across [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Transplantation and mediation. (A) Role-aligned code insertion. (B) Recovered producer– [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 14 linked inside Pith

  1. [5]

    What is in R1? What value is stored in R1? Return the value of R1

    （0, +, 0)Who is left of C Which node is to the left of C? Identify the node located left of C. What is in R1? What value is stored in R1? Return the value of R1. Is C entailed? Does C follow from the premises? Can C be derived? How many have arrived so far? Count arrivals up to now. Number of people arrived until now?Attribute / event world Relation world...

    2026

  2. [7]

    Coordination among neural modules through a shared global workspace.arXiv preprint arXiv:2103.01197,

    Anirudh Goyal, Aniket Didolkar, Alex Lamb, Kartikeya Badola, Nan Rosemary Ke, Nasim Rahaman, Jonathan Binas, Charles Blundell, Michael Mozer, and Yoshua Bengio. Coordination among neural modules through a shared global workspace.arXiv preprint arXiv:2103.01197,

    Pith/arXiv arXiv

  3. [8]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Alex Vaughan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783,

    Pith/arXiv arXiv

  4. [9]

    Verbalizable representations form a global workspace in language models.arXiv preprint arXiv:2607.15495,

    Wes Gurnee, Nicholas Sofroniew, Adam Pearce, Mateusz Piotrowski, Isaac Kauvar, Runjin Chen, Anna Soligo, Paul Bogdan, Euan Ong, Rowan Wang, et al. Verbalizable representations form a global workspace in language models.arXiv preprint arXiv:2607.15495,

    Pith/arXiv arXiv

  5. [11]

    Sleeper agents: Training deceptive llms that persist through safety training.arXiv preprint arXiv:2401.05566,

    Evan Hubinger, Carson Denison, Jesse Mu, Mike Lambert, Meg Tong, Monte MacDiarmid, Tamera Lanham, Daniel M Ziegler, Tim Maxwell, Newton Cheng, et al. Sleeper agents: Training deceptive llms that persist through safety training.arXiv preprint arXiv:2401.05566,

    Pith/arXiv arXiv

  6. [13]

    The linear representation hypothesis and the geometry of large language models.arXiv preprint arXiv:2311.03658,

    Kiho Park, Yo Joong Choe, and Victor Veitch. The linear representation hypothesis and the geometry of large language models.arXiv preprint arXiv:2311.03658,

    Pith/arXiv arXiv

  7. [14]

    The model organism lottery: Model organism interpretability strongly depends on training methodology.arXiv preprint arXiv:2607.01033,

    Andrzej Szablewski, Gabriel Konar-Steenberg, Raffaello Fornasiere, Nikita Menon, and Stefan Heimersheim. The model organism lottery: Model organism interpretability strongly depends on training methodology.arXiv preprint arXiv:2607.01033,

    Pith/arXiv arXiv

  8. [15]

    Proofwriter: Generating implications, proofs, and abductive statements over natural language

    Oyvind Tafjord, Bhavana Dalvi, and Peter Clark. Proofwriter: Generating implications, proofs, and abductive statements over natural language. InFindings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pp. 3621–3634,

    2021

  9. [16]

    Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet—transformer-circuits

    Adly Templeton, Tom Conerly, Jonathan Marcus, Jack Lindsey, Trenton Bricken, Brian Chen, Adam Pearce, Craig Citro, Emmanuel Ameisen, Hoagy Cunningham Andy Jones, et al. Scaling monosemanticity: Extracting interpretable features from claude 3 sonnet—transformer-circuits. pub.https://transformer-circuits. pub/2024/scaling-monosemanticity/index. html,

    2024

  10. [17]

    Steering language models with activation engineering.arXiv preprint arXiv:2308.10248,

    Alexander Matt Turner, Lisa Thiergart, Gavin Leech, David Udell, Juan J Vazquez, Ulisse Mini, and Monte MacDiarmid. Steering language models with activation engineering.arXiv preprint arXiv:2308.10248,

    Pith/arXiv arXiv

  11. [18]

    Model organisms for emergent misalignment.arXiv preprint arXiv:2506.11613,

    11 Published as a conference paper at ICLR 2026 Edward Turner, Anna Soligo, Mia Taylor, Senthooran Rajamanoharan, and Neel Nanda. Model organisms for emergent misalignment.arXiv preprint arXiv:2506.11613,

    Pith/arXiv arXiv 2026

  12. [19]

    Representation engineering: A top-down approach to ai transparency.arXiv preprint arXiv:2310.01405,

    Andy Zou, Long Phan, Sarah Chen, James Campbell, Phillip Guo, Richard Ren, Alexander Pan, Xuwang Yin, Mantas Mazeika, Ann-Kathrin Dombrowski, et al. Representation engineering: A top-down approach to ai transparency.arXiv preprint arXiv:2310.01405,

    Pith/arXiv arXiv

  13. [20]

    12 Published as a conference paper at ICLR 2026 A SUPPLEMENTOVERVIEW This supplement expands the data construction, formal definitions, architecture competition, inter- vention protocol, falsification tests, robustness analyses, and reproducibility record used in the main paper. Figures S1–S5 form five complementary evidence blocks: data validity, optimiz...

    2026

  14. [24]

    Supplementary Fig

    Multi- dimensional controls are matched in rank, sparsity, norm, or intervention load where applicable. Supplementary Fig. S7 compares the recovered API with J-space, SAE/Transcoder, PCA, output- gradient, and random subspaces. The central comparison is not strict API dominance over J-space: the two occupy the same high-performing region and are non-signi...

    2026

  15. [26]

    Density and hub overlap differ from matched null graphs at p <0.001

    Cross-seed Jaccard overlap is 0.83, and threshold-free AUPRC is 0.847±0.031 . Density and hub overlap differ from matched null graphs at p <0.001. The graph has a plausible temporal organization: future-relevant state is assembled in middle layers and consumed later. The high Jaccard overlap and AUPRC show that this structure is not solely the result of o...

    2026

  16. [64]

    The Shared architecture is favored under the tested operation bank and coding contract

    Shared uses one interface, Local six operation-specific interfaces, Mixture four gated components, and Distributed sixteen components. All candidates share the same optimizer, learning-rate schedule, batch size, regularization, early stopping, and twenty-block prequential order. Compute and cache requirements rise substantially from Qwen to Llama, with ap...

    2026

  17. [300]

    Proof/discourse has the smallest operation bank but the highest median difficulty and answer entropy, close to four bits, whereas attribute/event examples are shorter 13 Published as a conference paper at ICLR 2026 B C A D C A D E A Relation world Prompt (example) (+, 0, -) (0, +,

    2026

  18. [512]

    always Shared

    The data generator assigns multiple producer and consumer roles. The architecture controls the route by which producers communicate with consumers: Shared forces all routes through one bottleneck; Local gives each route an independent workspace; Mixture selects shared or local routes; Distributed spreads the relevant variables over components without a pr...

    2026

  19. [1993]

    Model organisms are leaky: Perplexity differencing often reveals finetuning objectives.arXiv preprint arXiv:2605.00994,

    Mohammed Abu Baker, Luca Baroni, and Dan Wilhelm. Model organisms are leaky: Perplexity differencing often reveals finetuning objectives.arXiv preprint arXiv:2605.00994,

    Pith/arXiv arXiv

  20. [2000]

    Principal component analysis

    Ian Jolliffe. Principal component analysis. InInternational Encyclopedia of Statistical Science, pp. 1945–1948. Springer,

    1945

  21. [2021]

    Scaling and evaluating sparse autoencoders

    Leo Gao, Tom Dupre la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. Scaling and evaluating sparse autoencoders. InInternational Conference on Learning Representations, volume 2025, pp. 26721–26754,

    2025

  22. [2022]

    The consciousness prior.arXiv preprint arXiv:1709.08568,

    Yoshua Bengio. The consciousness prior.arXiv preprint arXiv:1709.08568,

    Pith/arXiv arXiv

  23. [2023]

    Sparse autoen- coders find highly interpretable features in language models.arXiv preprint arXiv:2309.08600,

    Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. Sparse autoen- coders find highly interpretable features in language models.arXiv preprint arXiv:2309.08600,

    Pith/arXiv arXiv

  24. [2024]

    Localizing model behavior with path patching.arXiv preprint arXiv:2304.05969,

    10 Published as a conference paper at ICLR 2026 Nicholas Goldowsky-Dill, Chris MacLeod, Lucas Sato, and Aryaman Arora. Localizing model behavior with path patching.arXiv preprint arXiv:2304.05969,

    Pith/arXiv arXiv 2026

  25. [2025]

    Patchscopes: A unifying framework for inspecting hidden representations of language models.arXiv preprint arXiv:2401.06102,

    Asma Ghandeharioun, Avi Caciularu, Adam Pearce, Lucas Dixon, and Mor Geva. Patchscopes: A unifying framework for inspecting hidden representations of language models.arXiv preprint arXiv:2401.06102,

    Pith/arXiv arXiv

  26. [2026]

    In-context learning creates task vectors

    Roee Hendel, Mor Geva, and Amir Globerson. In-context learning creates task vectors. InFindings of the Association for Computational Linguistics: EMNLP 2023, pp. 9318–9333,

    2023