{"id":"68c462ea-3960-4555-bef9-6299fa25ae8e","arxiv_id":"2412.11965","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Attention-head functions can be inferred from parameters alone via vocabulary-space top-k scoring, and these inferences match head outputs and survive causal ablation across six LLMs.","lead":"MAPS infers what an attention head does by reading its weight matrices directly, without running the model on data: it projects the head's output matrix into the model's vocabulary space and checks which token pairs it promotes. Across six LLMs and 20 relations, the static scores correlate with the head's real outputs, and ablating the flagged heads breaks the corresponding tasks, offering a fast map of where skills live in a model.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reported validations of MAPS may be contingent on per-model and per-relation choices of the M variant and the scoring hyperparameters (k, tau), so the central claim is not yet shown to be a robust property of head parameters.","rationale":"The reader's weakest_assumption focuses on the linear single-token approximation and the omission of WQK and biases. That is a real limitation, but the paper partially addresses it with dynamic head-output correlations and causal ablations. A more concrete and under-tested threat to the central claim is the dependence of the whole pipeline on per-model, per-relation hyperparameter choices. The choice of whether to apply the first MLP, the value of k, and the threshold tau are all selected with knowledge of the task and, in one case, explicitly because it gave better results. Since the validation experiments use the same selected configuration, the reported correlations and causal effects could be inflated by tuning. This does not imply any bad faith; it is a standard robustness gap. The proposed sensitivity analysis would settle whether the framework's inferences are stable properties of the head parameters or artifacts of the chosen settings. I therefore keep the reader's CONDITIONAL verdict rather than changing it: the core idea is promising and the validations are substantial, but they should be shown to be robust before treating the mappings as definitive.","tokens_in":34150,"tokens_out":11491,"duration_ms":113819,"concrete_test":"Rerun the correlative (Tables 1 and B.1-B.5) and causal (Tables 2 and B.2) evaluations under one fixed protocol: M = E(WVO)U without the first-MLP variant for all models, k=10 for all non-copying relations, k=1 for copying relations, and tau=15%. In addition, perform a sensitivity sweep over k in {1,5,10,25} and tau in {5%,15%,25%} for at least Llama-3.1 8B and Pythia 12B, reporting the range of correlations and accuracy drops. If the qualitative conclusions survive across the grid, the concern is resolved; if they collapse in some configurations, the reported results are contingent on post hoc selection.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To establish that relation scores in M reflect head functionality, the paper varies the definition of M and the scoring hyperparameters across models and relations. The first MLP is applied to E for most models but explicitly not for Llama-3.1 70B because 'better results' were observed (§A.2); k is set to 1 or 3 for copying relations and 10 or 25 for other relations (§4.1, A.2); and tau=15% is chosen after comparing against randomly initialized heads (§A.3). The correlative and causal experiments are then run under this selected configuration. Because these choices are guided by the same validation signal—agreement with dynamic head outputs and usefulness of the resulting head sets—the reported high correlations and large causal drops may overstate the intrinsic quality of the parameter-only mapping. No sensitivity analysis is reported, so it is unknown whether a single fixed protocol would preserve the results. This matters for the central claim because MAPS is proposed as a parameter-only inference framework; if its success depends on per-model, per-relation tuning, the inferred functionalities are not a robust property of the head parameters. The dynamic-output and causal-removal validations are good evidence when taken at face value, but they cannot be fully interpreted without knowing how much the underlying protocol was selected to make them succeed.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces MAPS, a framework that infers the functionality of attention heads directly from their parameters by interpreting the head's output projection as a vocabulary-to-vocabulary interaction matrix M = E(W_VO)U. For a predefined relation R, MAPS computes a static relation score phi_R(M) as the fraction of source-target pairs for which the target appears among the top-k tokens in the corresponding row of M (Eq. 2). For open-ended analysis, it proposes a saliency score (Eq. 3) to identify the tokens on which a head operates most prominently, then uses an LLM to describe the resulting salient mappings. The paper validates the static scores in two ways: correlations with dynamic head outputs during inference (Eq. 4) and causal ablation experiments in which removing the heads identified by MAPS degrades task accuracy more than removing random heads. It also reports multi-token generalization, comparisons with previously identified heads, and a human study of the automatic descriptions. The experiments span 6 LLMs and 20 relations across four categories, and the authors release code and computed mappings.","tokens_in":34448,"tokens_out":3786,"duration_ms":41637,"significance":"If the central claim is robust, MAPS would be a useful and efficient tool for mechanistic interpretability: it avoids training or inference and scales vocabulary-projection analysis from specific circuits to whole models. The paper's strengths are its breadth (6 models, 20 relations), the combination of correlative and causal validation, the multi-token generalization check, the comparison with previously known heads, and the release of code and data. The human study for the salient-operation pipeline is also a concrete step beyond anecdotal evaluation. However, the main significance depends on whether the reported results are stable properties of head parameters or artifacts of the particular choices of M, k, and tau, which the current manuscript does not yet establish.","major_comments":[{"comment":"The relation score in Eq. (2) is evaluated under a protocol that varies the definition of M and the scoring hyperparameters across models and relations: the first MLP is applied to E for most models but explicitly not for Llama-3.1 70B because 'better results' were observed (§A.2); k is set to 1 or 3 for copying relations and 10 or 25 for other relations (§4.1); and the threshold tau=15% is fixed after comparison with randomly initialized heads (§A.3). Because the same selected protocol is then used in the correlative and causal validations, the reported correlations of 0.71–0.95 and the causal drops do not yet establish that the inference is a stable property of head parameters; they may reflect the selected configuration. Please add a sensitivity analysis that varies k, tau, and the M variant (with and without first MLP, and comparing E(W_VO)U with E(W_VO)E^T) under a fixed protocol, and report how the correlations and the head rankings change.","section":"§4.1, §A.2"},{"comment":"The non-contextualized dynamic score is not a fully independent test of the static score. Both the static and dynamic computations use the same relation pair dataset D_R, the same top-k operation over vocabulary projections, and closely related matrices: the static matrix is E(W_VO)U, while the dynamic score projects a self-attended head output through U. High correlation in this condition is therefore partly measuring internal consistency of the projection rather than independent behavioral evidence. The contextualized condition, the multi-token experiment (§4.3), and the causal ablation provide more independent support and should be presented as the primary validation, or the authors should add a comparison in which the dynamic score is computed with a different projection that includes layer norms and bias terms.","section":"§4.2, Eq. (4)"}],"minor_comments":[{"comment":"There is a typo in the Introduction: 'Predefined relationsignify groups of mappings' should be 'Predefined relations signify groups of mappings'.","section":"Abstract and Introduction"},{"comment":"The word 'Lingustic' appears in several figure labels and table headers; it should be 'Linguistic'.","section":"Tables 6–7 and Figures 5, 9–12"},{"comment":"The main-text causal table omits standard deviations, which are deferred to the appendix; at least ranges or a note that full variance is in §B.2 would help the reader judge the stability of the random-head baseline.","section":"Table 2"},{"comment":"The random-baseline experiment for tau is reported only qualitatively (e.g., 5 random matrices for GPT-2 xl); a statistical comparison of the random and trained classification rates would strengthen the justification for the threshold choice.","section":"§A.3"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and the framework is potentially valuable. The main risk is not the framework itself but the degree to which the validation protocol was selected after inspecting outcomes. I would not reject: the causal and multi-token results are good evidence, and the requested sensitivity analysis and independence check are feasible within the manuscript's scope. I also note that the limitations section is honest about the W_VO-only scope, which is a strength of presentation even though it narrows the title's claim."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read the MAPS paper with the stress-test note in hand. My take: the core framework is real and the paper is a solid contribution to mechanistic interpretability, but the headline correlations probably overstate how robust the parameter-only inference is, because a few key choices were made after peeking at the validation signal.\n\nWhat's genuinely new: taking the vocabulary-space projection M = E(WVO)U, which has been used for single relations or specific circuits, and scaling it to 20 relations across 6 LLMs. The saliency normalization for selecting input tokens (Eq. 3) is a nice fix for the norm-bias problem. The automatic GPT-4o pipeline for describing salient operations is new, and the human study gives an honest error rate. This is a toolkit paper, and it ships code and maps.\n\nThe strongest evidence is the causal experiment: removing heads identified by MAPS for a relation hurts task accuracy more than removing random heads, in most cases by a lot. That is independent of the dynamic-score correlation and it is the right kind of test.\n\nNow the soft spots, in proportion. The stress-test note is correct: the paper varies the definition of M (first MLP applied for all models except Llama-3.1 70B, where 'better results' were observed), sets k to 1/3 for copying but 10/25 for other relations, and picks tau=15% after comparing to random heads. These choices are validated on the same signal used to report success. No sensitivity analysis is shown. That means the 0.71-0.95 correlations are conditional on a per-model, per-relation configuration. I don't think it is fatal - the causal results are broad enough that a fixed protocol would likely still show something - but it weakens the claim that functionality is read off parameters without tuning.\n\nRelatedly, the correlative experiment shares top-k and the same relation datasets with the static score, so part of the correlation is internal consistency. The multi-token generalization and causal tests provide some external anchoring, but they use the same selected heads. I'd like to see a version where the static-to-dynamic comparison uses a held-out set of pairs or a different k.\n\nThe GPT-4o hallucination rate in no-pattern cases is acknowledged and quantified; it is a minor issue for the main mapping claim but worth noting for the pipeline's use.\n\nWho is this for? Anyone doing circuit discovery or cross-model comparisons. It is not a breakthrough, but it is a well-put-together tool that will get used. I would set serious_thinker to yes: the limitations section is honest and the reasoning is clear.\n\nRecommendation: accept for peer review. A good referee can ask for sensitivity analysis and a fairer correlative protocol. These are addressable.","headline":"A useful parameter-only head scanner with solid but partly over-tuned validation; worth refereeing, with a request for sensitivity analysis.","tokens_in":34941,"tokens_out":2881,"would_cite":true,"duration_ms":26629,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper shows that an attention head's token-level operations can be read off its weights alone, and that these readings match real model behavior.","keywords":["attention heads","mechanistic interpretability","vocabulary space projections","token-to-token relations","large language models","parameter-only analysis"],"falsifier":"Take a head MAPS labels as implementing country-to-capital and run prompts that place the country name in different syntactic positions while recording the attention weight the head puts on the country token; if the head no longer promotes the capital whenever that attention weight is low, the operation lives in the attention pattern as much as in the writing weights, and the parameter-only reading is incomplete.","tokens_in":33941,"feed_emoji":"🧠","tokens_out":5240,"duration_ms":45634,"temperature":0.7,"pith_summary":"Attention heads are small building blocks inside large language models, and this paper claims their jobs can be inferred directly from the head's parameters, without running the model or training anything. The authors build MAPS, which reads the head's writing weights as a matrix of scores between every pair of tokens in the vocabulary, then checks which of twenty predefined token-to-token relations (country to capital, word to past tense, English to French, and so on) the head ranks highly. They show these static scores correlate with the tokens the head actually promotes during inference, and that deleting the heads flagged for a relation degrades tasks needing that relation far more than deleting random heads. The same vocabulary-pair matrix lets them find a head's most salient operations automatically, producing plausible human-checked descriptions for most middle- and upper-layer heads in two models. If the claim holds, attention-head analysis becomes cheap and exhaustive: a full map of a model's functions without inference or circuit hunting.","feed_headline":"Head jobs decoded from weights alone, no inference needed","feed_subtitle":"A parameter-only map of token-to-token operations finds overlooked heads in six LLMs.","key_machinery":"The load-bearing object is the vocabulary-pair matrix $M = E(W_{VO})U$, where $E$ is the embedding matrix, $W_{VO}$ the head's output-value weights, and $U$ the unembedding matrix; each entry scores how strongly the head maps source token $s$ to target token $t$. The companion quantity is the relation score $\\varphi_R(M)$, the fraction of a relation's token pairs whose target lands in the top-$k$ of the source row, with a suppression variant computed on $-M$. These convert functionality inference into submatrix reading, and pair with the saliency score $\\sigma_t = \\|e_t W_{VO}\\| / \\|e_t\\|$ to find which tokens the head most affects.","core_discovery":"The central discovery is that the writing part of an attention head, folded into the matrix $M = E(W_{VO})U$, is a readable token-to-token mapping. For a predefined relation $R$ built from pairs $(s,t)$, the head's implementation strength is $\\varphi_R(M)$, the fraction of source tokens for which the target is in the top-$k$ promoted tokens of that row. The paper argues this parameter-only score is not a projection artifact: it correlates 0.36--0.95 with the head's observed outputs across 20 relations and six LLMs, and causally removing heads with high scores drops task accuracy by at least 32% relative to random ablations while leaving control tasks less affected. The matrix also yields salient operations by scoring how much the head changes each token's embedding norm, and an LLM-based pipeline turns those salient mappings into natural-language descriptions that human annotators judge plausible in the large majority of pattern-present cases.","pith_inferences":["If parameter-only inference is as faithful as the correlations suggest, it becomes a practical tool for auditing large models: mapping every head's operations without the cost of corpus-scale inference.","Because the framework ignores the query-key attention matrix, an obvious extension is to condition inferred functions on attention: the same $M$ could be paired with $W_{QK}$ to predict when a head switches operation by context.","The saliency-guided pipeline could serve as an unsupervised circuit-discovery step, generating candidate functions for heads before any task-specific probing.","The relation datasets are limited to single-token pairs; extending to multi-token and cross-lingual relations is the natural stress test of the framework's expressiveness."],"forward_implications":["Given any operation expressible as token pairs, MAPS produces a complete head-by-head map of where the model implements it, for every layer and head, in a single pass over parameters.","Heads flagged as implementing a relation are causally load-bearing for tasks that need it: ablating them drops accuracy much more than ablating random heads.","The same reading method recovers previously known name-mover and capital heads and finds dozens of additional same-function heads that circuit analyses missed.","Function encoding shows regularities across models: similar relations get similar top scores, grouped-query heads in Llama-3.1 share functions, and smaller models pack more relations per head.","Most attention heads in middle and upper layers admit an automatic natural-language description of their salient operation, as judged against human annotations."],"supporting_citations":[{"why":"Supplies the interaction-matrix formulation that lets $W_{VO}$ be interpreted as $M = E(W_{VO})U$.","marker":"Elhage et al. (2021)"},{"why":"Establishes the embedding-space analysis of transformer parameters that MAPS scales into a general framework.","marker":"Dar et al. (2023)"},{"why":"Provides the vocabulary-space projection technique for interpreting model parameters.","marker":"Geva et al. (2021, 2022)"},{"why":"Shows a single relation can be mapped across heads of many LLMs and applies the first MLP to the embedding matrix.","marker":"Gould et al. (2024)"},{"why":"Defines name-mover heads and the final-layer-norm variant of $M$, the key comparison circuit for validation.","marker":"Wang et al. (2023)"},{"why":"Provides mover and capital heads in GPT-2 medium that MAPS recovers and extends.","marker":"Merullo et al. (2024a)"},{"why":"Shows relation decoding is approximately linear and supplies part of the relation datasets.","marker":"Hernandez et al. (2024)"},{"why":"The Llama-3.1 models under analysis, including the grouped-query attention structure studied.","marker":"Dubey et al. (2024)"},{"why":"The Pythia models under analysis, used for correlation, causal, and multi-token generalization tests.","marker":"Biderman et al. (2023)"},{"why":"The GPT-2 models under analysis, used for the salient-operation pipeline and comparison with prior circuits.","marker":"Radford et al. (2019)"}],"fun_headline_variants":["Attention head functions decoded from weights alone","Zero-inference map of attention head operations","Weights reveal attention head roles in LLMs, no inference needed","New tool decodes attention heads without model inference","Parameter-only scoring exposes overlooked attention heads"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a head's function is visible in the linear token-to-token map made from its writing weights alone, so attention patterns, biases, layer norms, and later layers can be ignored when naming what the head does.","fun_headline_variants_meta":{"raw":{"variants":["Attention head functions decoded from weights alone","Zero-inference map of attention head operations","Weights reveal attention head roles in LLMs, no inference needed","New tool decodes attention heads without model inference","Parameter-only scoring exposes overlooked attention heads"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000216,"raw_usage":{"total_tokens":1431,"prompt_tokens":945,"completion_tokens":486,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":561,"completion_tokens_details":{"reasoning_tokens":426}},"tokens_in":561,"tokens_out":486,"duration_ms":5718,"temperature":1.0,"reasoning_tokens":426,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T14:24:46.294979+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a head MAPS labels as implementing country-to-capital and run prompts that place the country name in different syntactic positions while recording the attention weight the head puts on the country token; if the head no longer promotes the capital whenever that attention weight is low, the operation lives in the attention pattern as much as in the writing weights, and the parameter-only reading is incomplete.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows a single relation can be mapped across heads of many LLMs and applies the first MLP to the embedding matrix."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines name-mover heads and the final-layer-norm variant of $M$, the key comparison circuit for validation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The Pythia models under analysis, used for correlation, causal, and multi-token generalization tests."}],"review_version":1}