Benchmarking and Understanding Compositional Relational Reasoning of LLMs

Da Xiao; Hongliang Liang; Qingye Meng; Ruikang Ni; Shihui Zheng; Xiangyu Li

REVIEW 4 major objections 6 minor 1 cited by

Benchmarking and Understanding Compositional Relational Reasoning of LLMs

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper claims that two classes of higher-order relating attention heads in Vicuna-33B encode abstract truth values and are reused across models and tasks in compositional relational reasoning.

desk verdict GAR is a genuinely useful benchmark, but the True/False heads claim overreaches from a causal result on one model to a universal mechanism. read the letter →

arxiv 2412.12841 v1 pith:EJJS6CTR submitted 2024-12-17 cs.CL cs.LG

Ruikang Ni , Da Xiao , Qingye Meng , Xiangyu Li , Shihui Zheng , Hongliang Liang This is my paper · ORCID

classification cs.CLcs.LG

keywords compositionalrelationalreasoningmechanisticinterpretabilityattentionheadstruthfulnessbenchmarkassociativerecallin-contextlearninglargelanguagemodels

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper builds a synthetic benchmark, Generalized Associative Recall (GAR), that turns associative recall, knowledge recall, and indirect-object-identification-style tasks into one two-hop relational-reasoning framework, and uses it to argue that current large language models have a fundamental deficiency in compositional relational reasoning (CRR). Even GPT-4 reaches only 71.5% average accuracy on GAR, and the compositionality gap, the gap between solving subproblems and composing them, widens as models get larger. The paper then traces how Vicuna-33B actually solves these tasks with attribution patching and identifies reusable higher-order relating attention heads, including True heads and False heads, whose activations encode whether a statement is true or false. Intervention experiments suggest these heads are causally important for truthfulness judgments and transfer to natural-language inference and truth datasets beyond GAR. If the account holds, LLMs do not reason compositionally from scratch but reuse a small set of truth-encoding attention heads, and GAR supplies a controllable testbed for studying that reuse.

What carries the argument

The load-bearing object is the relational loop, the interleaving of a long-range semantic relation (for example, same or kindOf) with a local syntactic relation (for example, subject-object or adjacent position) between the elements K, V, Q, and A of a statement; GAR tasks are generated by choosing two relational schemas and optionally adding negation, so every task contains such a loop. The paper's circuit analysis then centers on higher-order relating attention heads, which are relating heads whose own attention pattern is formed by composing the outputs of other relating, local, and induction heads; the True and False heads are the two classes of higher-order relating heads that compare the query-side K-V pair with the key-side pair and write a match or mismatch signal. The method that carries the discovery is step-wise attribution patching with integrated gradients, tracing logit contributions back through MLPs and attention heads, and then validating candidates with weak and strong interventions that replace or repattern the heads' attention weights.

What would settle it

A concrete check: run the same GAR tasks on a smaller model (for example, a 1-2 billion parameter model) with an automated circuit-discovery method; if the automatically found circuit for correct answers does not include heads whose activations separate true from false GAR statements, the claim that True/False heads play a fundamental role would be falsified. Alternatively, on correctly answered GAR classification examples, if randomly permuting the attention of heads 14.18, 14.46, 15.51, and 14.0 leaves accuracy unchanged, the causal attribution fails.

Watch

Extended reading notes

Core claim

The central claim is that a handful of attention heads in Vicuna-33B implement the abstract truth value of a GAR statement, and that these heads are reused across tasks and model sizes. In classification tasks, two higher-order relating heads (14.18, 14.46) act as True heads: their query side gathers the information of the attended key-value pair and the key side gathers the candidate answer's pair, so a match writes a true signal into the residual stream. Two other heads (15.51, 14.0) act as False heads, activating when the compared pairs do not match and the statement is false. The paper shows that these heads' activation values separate true from false examples, that knocking out the wrong head class or enforcing the right attention pattern changes accuracy by 6 to 17 percentage points across Vicuna-7B/13B/33B, and that the same four heads' activations, used as features, classify SNLI and Geometry-of-Truth items near or above Vicuna-7B's accuracy. The broader discovery claim is that GAR's relational-loop motif, interleaved semantic and syntactic relations, is the pattern these circuits detect, and that LLMs' failure on hard GAR tasks reflects weak composition of lookup and retrieval relations rather than missing knowledge.

Load-bearing premise

The paper's central conclusion assumes that the heads it found by manually tracing which parts of Vicuna-33B most influence the answer are really the ones doing the reasoning; no automatic circuit finder was used because existing tools could not handle the model's size and complexity, so if the tracing pointed to the wrong heads, the True/False-head claim would collapse.

Editorial extensions

If this is right

GAR provides a controllable difficulty dial: accuracy drops monotonically as the number of non-same semantic relations nr goes from 0 to 2 and when negation is added, so future work can test CRR improvements at fixed difficulty.
Because the compositionality gap grows with model size on generation tasks, scaling alone is unlikely to close the CRR deficiency; targeted data or architecture changes would be needed.
Truthfulness judgments in at least one LLM family are implemented by reusable attention heads rather than by task-specific circuitry, and the same heads transfer to SNLI and Geometry-of-Truth classification.
Intervention on the identified heads can push Vicuna-33B's accuracy toward or above Llama-3-70B on several GAR task types, suggesting a concrete path for steering or improving compositional reasoning without retraining.
The circuit-reuse picture implies that higher-order induction heads play a general role in bridging demonstrations to current-token computations, extending the standard induction-head account of in-context learning.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

One testable extension: if True/False heads really encode abstract truth, then similar pairs of heads should be discoverable in other decoder-only families on the same GAR tasks, and their homologous activations should separate true from false statements; the paper only demonstrates universality across Vicuna sizes.
The paper's own error analysis shows most failures come from applying the retrieval relation to the wrong distractor candidate; a natural next experiment is to measure whether the False heads fire on those erroneous candidate generations, which would make the truth-value heads a monitor for reasoning errors rather than just a classification feature.
GAR's relational-loop view of composition suggests a continuum from associative recall to multi-hop reasoning; extending the benchmark to chains of three or more relations with intermediate variables would test whether the same higher-order relating heads scale to deeper compositions, something the paper does not claim.
Because weak intervention works but less than strong intervention, the model's weights contain the right attention patterns in other heads but cannot compose them into a circuit; this suggests fine-tuning the connectivity between heads, not just their individual attention, might improve compositional generalization.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Start with the punchline: the GAR benchmark is a real contribution, and the paper's evaluation shows an interesting and reproducible performance gap in LLMs on compositional relational reasoning. The head-level finding—True/False heads in Vicuna-33B that respond to statement truth—is intriguing but the paper's central claim, that these heads play a fundamental role in CRR across various models and tasks, is not supported by the evidence actually presented.

What's new and good. GAR integrates AR, KR, and IOI into one framework, and the task design is thoughtful: varying the number of non-same relations and adding negation gives a controlled difficulty axis. The evaluation across 10 models is broad, and the finding that the compositionality gap grows with model size is worth having. The circuit discovery on Vicuna-33B is detailed, and the strong intervention experiments on the eight discovery tasks are convincing: forcing the identified attention patterns to form improves accuracy, sometimes above Llama-3-70B. The authors also deserve credit for shipping code and data.

Where it gets soft. The leap from 'these heads matter on the tasks where they were discovered' to 'they play a fundamental role across models and tasks' is too big. The cross-model evidence uses different head IDs per model, so at best it shows a functional class exists, not that the specific heads are universal. The cross-dataset evidence on SNLI and GoT is an MLP probe trained on four heads' activations—that is a correlational probe, not a causal intervention, and the paper never shows that manipulating these heads changes truth judgments on those datasets. The manual attribution method is acknowledged as a limitation, but it matters here: without automatic circuit discovery or at least a causal test on an out-of-distribution task, the abstract 'truth encoding' could be an artifact of the synthetic template structure. The evaluation also uses a lenient accuracy metric and only 8/16 examples per task, so the performance numbers have more noise than the figures suggest.

Still, the paper is coherent and honest about its own constraints. I would send it to peer review, but I would ask for tempered claims, error bars, and at least one causal intervention outside the GAR tasks. The GAR benchmark I would cite; the universal head claim, not yet. This is a good paper for a reading group—there's plenty to discuss about what counts as evidence in mechanistic interpretability.

Referee Report

4 major / 6 minor

Summary. The paper introduces Generalized Associative Recall (GAR), a synthetic benchmark that unifies associative recall, knowledge recall, and indirect object identification into a relational-loop framework with semantic and syntactic variations (negate, g2c, swapQA, swapKV). It evaluates 10 LLMs on 384 tasks and reports that even GPT-4 reaches only 71.5% average accuracy, with the compositionality gap increasing as models scale. Using attribution patching on Vicuna-33B, the authors identify reusable circuits for classification and generation tasks, including 'True' heads (14.18, 14.46) and 'False' heads (15.51, 14.0) whose activations are claimed to encode abstract truth values. Interventions on these heads improve accuracy on GAR classification tasks, and a supervised MLP trained on their activations separates true/false statements on SNLI and four GoT subsets. The paper concludes that these heads play a fundamental role in compositional relational reasoning across models and tasks.

Significance. If the causal role of True/False heads were established, this would be an important step toward showing that LLMs reuse composable truthfulness abstractions, and GAR could become a useful testbed for mechanistic interpretability. The paper's strengths include a public dataset and code release, a principled schema-based task generator, weak and strong intervention protocols, an inverse-intervention control, and analysis across multiple model sizes. The benchmark itself is likely to be useful even if the circuit-interpretation claims are later refined. However, the strongest conclusions about a 'fundamental role across various models and tasks' are currently supported by a narrower set of interventions than the text claims: cross-model evidence relies on newly identified heads per model, and cross-dataset evidence is a supervised probe rather than a causal manipulation.

major comments (4)

[§The efficacy of True/False Heads in Other Datasets (Table 3, Appendix E)] The claim that True/False heads 'play an important role' on SNLI and GoT is supported only by a supervised MLP classifier trained on the activations of the four GAR-discovered heads. This demonstrates that these activations contain task-relevant information, but it is not a causal or mechanistic test: nothing in the experiment shows that Vicuna-33B actually uses these heads to produce its SNLI/GoT judgments, and the probe is fit to the labels of those datasets. A complementary causal test (e.g., activation patching or knockout on held-out natural-language statements) or a carefully qualified claim is needed before the conclusion that the activation patterns are 'robust across other datasets' is justified.
[§Validating True/False Heads across Models (Table 9, Figure 5a)] The specific heads named in the abstract (14.18, 14.46, 15.51, 14.0) are Vicuna-33B heads; across models, Table 9 lists different heads (e.g., Vicuna-7B True heads 11.31 and 14.23). The universality claim is therefore about a functional class, not about the specific heads advertised in the abstract. Moreover, the cross-model evidence is limited to three Vicuna models on three GAR classification tasks, with no test on non-Vicuna architectures and no test on harder GAR settings such as nr=2 with negate, where Figure 2b reports Vicuna-33B accuracy below 40%. The scope of the 'across various models and tasks' claim should be restricted accordingly or supported by additional experiments.
[§Evaluating LLMs on GAR and Appendix C] The main benchmark conclusions (e.g., GPT-4 at 71.5%, compositionality gap increasing with scale) rest on a lenient accuracy criterion: an answer is counted correct if its probability exceeds the average probability of the alternatives. For generation tasks with three alternatives, a model assigning only 0.34 probability to the correct answer is scored as correct. In addition, each task contains only 8/16 examples, and Figures 2, 4, and 5 report no confidence intervals or significance tests. Given these choices, the precise accuracy numbers and the strength of the 'fundamental deficiency' claim should be treated with caution; reporting error bars and, ideally, top-1/argmax accuracy as a secondary metric would strengthen the benchmark analysis.
[§Discovering and Analyzing the Circuits] The circuit discovery relies on manual step-wise attribution patching, with the query/key/value attribution steps chosen manually, and the authors state that automatic circuit discovery methods were not used because of model size and circuit complexity. Since the identification of True/False heads and the subsequent 'fundamental role' claims depend on the completeness of this attribution, the manual procedure needs additional validation—for example, by comparing against activation patching on held-out tasks or by ablating all other heads on a sample to check for equally important alternative circuits. Without such a check, the central mechanism claim remains vulnerable to attribution error; the paper's own limitation statement in this section should be weighed explicitly in the conclusions.

minor comments (6)

[Figure 1] The figure contains a typo: 'Jonh' should be 'John'.
[Table 3] The column header 'GoT Acc(%)' is based on four selected subsets, not the full Geometry of Truth dataset; the main text should say 'GoT (4 subsets)' to avoid overgeneralization.
[Appendix E] The exclusion of GoT subsets (greater-than, less-than, conjunctions, disjunctions, negations, common claim, likely) is a scope restriction that should appear in the main text next to the 'robust across other datasets' claim, because the excluded subsets are exactly those requiring logical operations.
[Appendix C] The accuracy criterion is described only in the appendix; please define 'average probability of the alternatives' in the main text and discuss its effect on the reported accuracy numbers.
[§Generalized Associative Recall Benchmark] The sentence 'the existence of the loop ensures predictability' is stated as a first-principles guarantee, but no formal argument is provided; as written, it is more a design intuition than a testable claim.
[Figure 5(b)] Please clarify how 'average activation values' are computed across multiple tokens in the hypothesis or statement, since the current definition of activation value refers to a single attending position.

Circularity Check

1 steps flagged · score 2.0 of 10

Minor self-definitional sanity check for True/False heads; central attribution and intervention chain is not circular.

self definitional [Section 'Validating Attention Heads' (definition bullets) and 'Validating True/False Heads across Models' (Figure 5b)]
"True/False Heads: Higher-order relating heads which activate on true/false statements for truthfulness judgement (Figure 3 (b) / (c)). ... It can be seen that the activation values can effectively distinguish true and false statements, indicating that these True/False heads definitely represent the abstract notion of true and false in these tasks."

The head class is defined by the property that its members activate on true/false statements, and the Figure 5(b) evidence is that the activation values of these heads separate true from false statements on the same GAR classification tasks used to identify and name them. This is a consistency check of the naming/definition rather than an independent confirmation of an abstract truth encoding. The plot would be expected by construction for a class selected as 'higher-order relating heads which activate on true/false statements.' The paper's stronger cross-model and cross-task claims rest on separate intervention and probe evidence, so this self-definitional step is not load-bearing for the main causal derivation.

full rationale

The main derivation is not circular. GAR is generated from relational schemas and externally evaluated; the circuit discovery uses attribution patching and is validated by causal interventions, including inverse intervention (Table 10). Cross-model validation re-identifies True/False heads per model rather than reusing the Vicuna-33B heads, and the SNLI/GoT experiments use fixed head activations as features, so those features are not fitted to the target labels. The remaining issues are a minor tautology in the Figure 5(b) 'abstract true/false' claim and an overstatement in calling the supervised MLP probe evidence of an 'important role' on other datasets; the latter is a correlational probe, not a causal test. These are overclaims and a small self-definitional check, not a reduction of the central claim to its inputs; hence a low score.

Assumptions & free parameters 1 free parameters · 3 assumptions · 1 invented entities

The central claims rest on a few design axioms and methodological assumptions rather than fitted parameters. The main free choice is the accuracy threshold. The 'relational loop' is an asserted design principle. Circuit validity is an assumption about attribution patching. GoT subset selection is a post-hoc scope choice.

free parameters (1)

Accuracy threshold = average probability of alternatives (33.3% for generation, 50% for classification)
Hand-chosen evaluation rule in Appendix C; a model is correct if the target's probability exceeds the average of alternatives. This lenient threshold can classify near-random probabilities as correct.

assumptions (3)

ad hoc to paper The existence of a relational loop ensures predictability in CRR tasks
Stated in 'Basic Idea: From AR and KR to GAR' without proof; used to justify GAR's design and later circuit interpretation.
domain assumption Attribution patching, applied manually with query/key/value choices, identifies the true causal circuits in Vicuna-33B
The paper acknowledges automatic circuit discovery was not used ('Currently, we do not use any automatic circuit discovery methods...'); the validity of the circuit analysis depends on this assumption.
ad hoc to paper Excluded GoT subsets (greater-than, less-than, conjunctions, negations, common claim, likely) are outside the scope of attention-head truthfulness
In Appendix E, subsets are excluded with justifications based on MLP mediation or logical operations, but the exclusions are post-hoc and affect the reported transfer accuracy.

invented entities (1)

Relational loop
purpose: Conceptual framework that unifies AR, KR, and IOI and guides task design and circuit interpretation
Introduced as a design principle and used to explain circuit structure; no falsifiable prediction outside the paper is derived from it.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Benchmarking and Understanding Compositional Relational Reasoning of LLMs." pith.science (2026). https://pith.science/paper/EJJS6CTR

@misc{pith2026241212841,
  author       = {Pith},
  title        = {Pith review of: Benchmarking and Understanding Compositional Relational Reasoning of LLMs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EJJS6CTR}},
  note         = {Machine review of arXiv:2412.12841}
}

read the original abstract

Compositional relational reasoning (CRR) is a hallmark of human intelligence, but we lack a clear understanding of whether and how existing transformer large language models (LLMs) can solve CRR tasks. To enable systematic exploration of the CRR capability of LLMs, we first propose a new synthetic benchmark called Generalized Associative Recall (GAR) by integrating and generalizing the essence of several tasks in mechanistic interpretability (MI) study in a unified framework. Evaluation shows that GAR is challenging enough for existing LLMs, revealing their fundamental deficiency in CRR. Meanwhile, it is easy enough for systematic MI study. Then, to understand how LLMs solve GAR tasks, we use attribution patching to discover the core circuits reused by Vicuna-33B across different tasks and a set of vital attention heads. Intervention experiments show that the correct functioning of these heads significantly impacts task performance. Especially, we identify two classes of heads whose activations represent the abstract notion of true and false in GAR tasks respectively. They play a fundamental role in CRR across various models and tasks. The dataset and code are available at https://github.com/Caiyun-AI/GAR.

Figures

Figures reproduced from arXiv: 2412.12841 by the authors.

**Figure 2.** Performance of existing LLMs on GAR. ure 2 (b) and for Llama-3-70B and Vicuna-7B in Appendix C. The accuracies on simple KR tasks with other semantic relations (see 1st example in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

**Figure 3.** Circuits of Vicuna-33B for solving GAR tasks. Explanations on different classes of heads are in Appendix D [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: Analysis and intervention results of some vital heads in Vicuna-33B. In Figure (a), (A) denotes affirmative generation [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]

**Figure 5.** Figure 5: Analysis and intervention of True/False heads. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Accuracies of Llama-3-70B and Vicuna-7B on [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

**Figure 7.** Figure 7: Illustration of intervention methods. Inverse Intervention We randomly sample 500 from all 3072 classification task examples. Vicuna-33B does right on 330 of them. For these 330 examples we do inverse intervention on the True/False heads, i.e. when the answer is Yes/No…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Toward Skill-Native LLMs: Skill Entropy for Benchmarking and Training Long-Horizon Reasoning
cs.CL 2026-08 conditional novelty 7.0 of 10

Skill entropy, a reference-model-based measure of skill-switching difficulty, calibrates a new cross-skill benchmark and serves as an RL reward, more than doubling small models' scores.

Reference graph

Works this paper leans on

13 extracted references · 11 canonical work pages · cited by 1 Pith paper

[1]

synonym of correct answer (can be treated as correct), e.g. ”pet”

work page
[2]

answer by applying a non-same rretrieve to a wrong can- didate V’, e.g. ”fruit”

work page
[3]

vehicle

answer not relevant to any candidates, maybe due to lack of knowledge or hallucination, e.g. ”clothes”. As shown in Table 5, for both models Class 2 errors are dom- inant, indicating that the models fail to correctly compose rlookup and rretrieve and have fundamental deficiency on the tasks. Model Class 1 Class 2 Class 3 GPT-4 21 79 0 Vicuna-33B 6 88 6 Ta...

work page 2023
[10]

affirmative generation) have the same overall structure

The circuits for the same kind of tasks (e.g. affirmative generation) have the same overall structure

work page
[11]

18.13 PPred

For some classes of heads, there are also some heads that are shared across different tasks of the same kind, e.g. 18.13 PPred. head, 15.0 Loc. 2 head, 16.10 Ind. 2 head, 7.39 Rel. head and 14.18 Rel. 2 (True) head for affirma- tive generation tasks

work page
[12]

heads vary with different rretrieve, because they retrieve different attributes according to the semantic re- lation

Pred. heads vary with different rretrieve, because they retrieve different attributes according to the semantic re- lation

work page
[13]

Premise: {premise}. Please answer with Yes or No. Can it be inferred from the premise that {hypothesis}? Answer:→{label}

and 2. indicate an important mechanism of the model which is highly parameter-efficient: re-using circuit struc- tures and attention heads as much as possible for solving similar tasks. E. More on Validating Attention Heads 1.000.000.000.000.000.000.00 0.190.810.000.00.0.000.000.00 0.120.430.440.000.000.000.00 0.040.260.170.530.000.000.00 0.070.190.11 0.2...

work page 2015
[2015]

In Proceedings of the 2015 Conference on Empir- ical Methods in Natural Language Processing (EMNLP)

A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empir- ical Methods in Natural Language Processing (EMNLP). Brinkmann, J.; Sheshadri, A.; Levoso, V .; Swoboda, P.; and Bartelt, C. 2024. A mechanistic analysis of a transformer trained on a symbolic multi-step reasoning task. In Findings of the Asso...

work page 2015

Show all 13 references

[2021]

Transformer Circuits Thread, 1(1): 12

A mathematical framework for transformer circuits. Transformer Circuits Thread, 1(1): 12. Fu, D. Y .; Dao, T.; Saab, K. K.; Thomas, A. W.; Rudra, A.; and R´e, C. 2023. Hungry hungry hippos: Towards language modeling with state space models. In Proceedings of the Eleventh Inter...

2023
[2022]

Countries of cities include Spain, Thailand,

Locating and editing factual associations in GPT. In Advances in Neural Information Processing Systems (NeurIPS), volume 35, 17359–17372. Merullo, J.; Eickhoff, C.; and Pavlick, E. 2024a. Circuit component reuse across tasks in transformer language mod- els. In The Twelfth Int...

2024 arXiv
[2023]

In Conference on Empirical Methods in Natural Language Processing (EMNLP)

Dissecting recall of factual associations in auto- regressive language models. In Conference on Empirical Methods in Natural Language Processing (EMNLP). Hanna, M.; Liu, O.; and Variengien, A. 2023. How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a...

2023
[2024]

In Advances in Neural Information Processing Systems (NeurIPS), volume 36

Faith and fate: Limits of transformers on composition- ality. In Advances in Neural Information Processing Systems (NeurIPS), volume 36. Elhage, N.; Nanda, N.; Olsson, C.; Henighan, T.; Joseph, N.; Mann, B.; Askell, A.; Bai, Y .; Chen, A.; Conerly, T.; et al
[2882]

Liu, S.; Xing, L.; and Zou, J

PMLR. Liu, S.; Xing, L.; and Zou, J. 2024. In-context vectors: Making in context learning more effective and controllable through latent space steering. In Proceedings of Interna- tional Conference on Machine Learning (ICML). Marks, S.; and Tegmark, M. 2023. The geometry of tr...

2024 arXiv

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.

[1] [1]

synonym of correct answer (can be treated as correct), e.g. ”pet”

work page

[2] [2]

answer by applying a non-same rretrieve to a wrong can- didate V’, e.g. ”fruit”

work page

[3] [3]

vehicle

answer not relevant to any candidates, maybe due to lack of knowledge or hallucination, e.g. ”clothes”. As shown in Table 5, for both models Class 2 errors are dom- inant, indicating that the models fail to correctly compose rlookup and rretrieve and have fundamental deficiency on the tasks. Model Class 1 Class 2 Class 3 GPT-4 21 79 0 Vicuna-33B 6 88 6 Ta...

work page 2023

[4] [10]

affirmative generation) have the same overall structure

The circuits for the same kind of tasks (e.g. affirmative generation) have the same overall structure

work page

[5] [11]

18.13 PPred

For some classes of heads, there are also some heads that are shared across different tasks of the same kind, e.g. 18.13 PPred. head, 15.0 Loc. 2 head, 16.10 Ind. 2 head, 7.39 Rel. head and 14.18 Rel. 2 (True) head for affirma- tive generation tasks

work page

[6] [12]

heads vary with different rretrieve, because they retrieve different attributes according to the semantic re- lation

Pred. heads vary with different rretrieve, because they retrieve different attributes according to the semantic re- lation

work page

[7] [13]

Premise: {premise}. Please answer with Yes or No. Can it be inferred from the premise that {hypothesis}? Answer:→{label}

and 2. indicate an important mechanism of the model which is highly parameter-efficient: re-using circuit struc- tures and attention heads as much as possible for solving similar tasks. E. More on Validating Attention Heads 1.000.000.000.000.000.000.00 0.190.810.000.00.0.000.000.00 0.120.430.440.000.000.000.00 0.040.260.170.530.000.000.00 0.070.190.11 0.2...

work page 2015

[8] [2015]

In Proceedings of the 2015 Conference on Empir- ical Methods in Natural Language Processing (EMNLP)

A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empir- ical Methods in Natural Language Processing (EMNLP). Brinkmann, J.; Sheshadri, A.; Levoso, V .; Swoboda, P.; and Bartelt, C. 2024. A mechanistic analysis of a transformer trained on a symbolic multi-step reasoning task. In Findings of the Asso...

work page 2015