Pith. sign in

REVIEW 4 cited by

RAVEL: Evaluating Interpretability Methods on Disentangling Language Model Representations

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2402.17700 v2 pith:SUXZN6UG submitted 2024-02-27 cs.CL cs.LG

Jing Huang , Zhengxuan Wu , Christopher Potts , Mor Geva , Atticus Geiger This is my paper
classification cs.CLcs.LG
keywords raveldistributedinterpretabilitylanguagemethodsmdasmodelmultiple
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Individual neurons participate in the representation of multiple high-level concepts. To what extent can different interpretability methods successfully disentangle these roles? To help address this question, we introduce RAVEL (Resolving Attribute-Value Entanglements in Language Models), a dataset that enables tightly controlled, quantitative comparisons between a variety of existing interpretability methods. We use the resulting conceptual framework to define the new method of Multi-task Distributed Alignment Search (MDAS), which allows us to find distributed representations satisfying multiple causal criteria. With Llama2-7B as the target language model, MDAS achieves state-of-the-art results on RAVEL, demonstrating the importance of going beyond neuron-level analyses to identify features distributed across activations. We release our benchmark at https://github.com/explanare/ravel.

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. One mechanism for many mental spaces: a shared router over a value slot in language models

    cs.CL 2026-07 conditional novelty 7.5 of 10

    A subspace trained to control one mental-space builder also controls others, indicating a shared router/slot mechanism across counterfactual, belief, fictional, and temporal spaces in LMs.

  2. Learning Distribution-Wise Control in Representation Space for Language Models

    cs.CL 2025-06 conditional novelty 6.0 of 10

    Stochastic distribution-wise interventions that learn a mean and variance in representation space improve ReFT-based language model reasoning, with the largest gains from restricting randomness to the first quarter of layers.

  3. Enhancing Chain-of-Thought Reasoning with Critical Representation Fine-tuning

    cs.CL 2025-07 conditional novelty 5.0 of 10

    CRFT selects critical internal representations via attention and saliency scores and fine-tunes only them, improving GSM8K accuracy over ReFT from 29.0% to 32.8% on LLaMA-2-7B.

  4. InverseScope: Scalable Activation Inversion for Interpreting Large Language Models

    cs.LG 2025-06 conditional novelty 5.0 of 10

    A new conditional-generation architecture plus a feature-consistency metric make activation inversion practical for LLMs up to 7B parameters, with experiments on IOI, RAVEL, and in-context learning.

Pith tools