{"id":"bc81a25e-9057-46b4-ac83-41ac82163709","arxiv_id":"2504.14496","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Knowledge recall in LLMs is abstracted as a function where subject, relation, and object activations act as independent input, body, and return values, and activation patching improves contextual knowledge editing.","lead":"The paper proposes that when a large language model recalls a fact, its hidden activations work like a simple function: the subject supplies the input, the relation supplies the function body, and the final token supplies the returned object. It tests this idea by swapping activation vectors between queries and uses the view to improve short-term knowledge editing with activation patching.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Independence claim is underdetermined: object-only interchange is an architectural tautology (final-token late-layer patch overwrites LM-head input), and component layers are selected on the same data used for evaluation.","rationale":"The paper proposes a functional abstraction of knowledge recall and supports it with localization scores and interchange accuracies. The most load-bearing assumption is that a single activation vector at a chosen token/layer can be cleanly swapped and that this swap demonstrates an independent functional component. The current experiments do not secure this assumption: the object-only condition is an architectural tautology, and the subject/relation conditions are selected on the same data they are used to evaluate. These issues weaken the evidential link between the reported numbers and the conceptual claim. I do not think this makes the hypothesis false; the direction is plausible and the editing application shows a real behavioral effect. But the central claim is underdetermined by the present protocol. This largely matches the reader's identified weakness (circularity of component selection), with the additional observation that the object-only result is not informative and the dual-interchange drop suggests non-compositionality. The appropriate verdict remains conditional, because the requested controls are well-defined and could be supplied in revision; if the held-out control experiment failed, the verdict would need to move toward rejection. I therefore leave the reader's verdict unchanged but emphasize the specific experiments that should be required.","tokens_in":11663,"tokens_out":9027,"duration_ms":85829,"concrete_test":"Re-run the full scoring and interchange procedure on a 50/50 split by relation type: define layer ranges and token positions using only the training half, then compute Table 1 on the held-out half. In each condition, add a control where the source vector is replaced by a random vector of the same norm at the same (layer, token). The claim is supported only if held-out subject/relation/dual accuracies remain high (e.g., >90%) while the random-vector control is substantially lower (e.g., near chance or at least 20 points lower), and if the dual condition does not drop significantly below the single-condition accuracies.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Section 4.2) is that subject, relation, and object activation vectors operate as distinct, separable functional components. The main evidence is Table 1, but the protocol does not actually isolate independence. First, the object vector is defined (Section 4.1.3) as the final-token activation in layers 15-31; by Section 3.1 that vector is the direct input to the classification head. Object-only interchange (Section 3.3) therefore replaces exactly the representation that determines the output distribution, so 99-100% accuracy in Table 1 is expected by construction and cannot validate a 'return value' component. Second, the subject and relation layer ranges (0-14 and 0-10) are chosen from the same scoring heatmaps (Figures 4-6) on the same filtered facts used for evaluation; there is no held-out split, no random-vector control, and no error bar. A random vector with the same norm at the same (layer, token) could plausibly shift the prediction as strongly, in which case the reported accuracies reflect locality of computation rather than a clean functional decomposition. Third, dual interchange (both subject and relation) is 88-93%, several points below object-only; this gap is consistent with interaction between the two vectors and is not addressed. The claim of separable components therefore rests on uncontrolled comparisons.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"This paper proposes that knowledge recall in transformer LLMs can be abstracted as a function execution in activation space, with subject, relation, and object activation vectors acting as independent functional components (input argument, function body, and return value). The authors introduce a patching-based knowledge-scoring algorithm (SES/RES/OES) to localize these vectors, validate them via counter-knowledge interchange experiments, and apply the resulting decomposition to a contextual knowledge editing method. Experiments on Llama2-7b and Mistral-7b use filtered relational knowledge data and two prompt templates; the main evidence for the functional decomposition is the high interchange accuracy reported in Table 1, and the editing results are reported in Table 2.","tokens_in":11859,"tokens_out":5401,"duration_ms":46233,"significance":"The proposed abstraction is attractive: if validated, it would give a sparse, human-readable functional decomposition of factual recall and a cheap, non-parametric knowledge editing intervention. The paper has some strengths: it tests two 7B models, uses two prompt templates with different subject-relation orders, reports score heatmaps and layer/token locality analyses, and provides detailed appendix descriptions of data filtering and interchange construction. The central claim is falsifiable in principle, and the interchange protocol is a standard causal-intervention tool. However, the current evidence does not establish the independence claim because several controls are missing and one of the interventions is, by construction, close to the model's output head. The significance of the paper therefore depends on whether the authors can add the missing controls and address the selection/evaluation overlap.","major_comments":[{"comment":"The object-only interchange result in Table 1 (99–100%) does not provide independent evidence for a separable 'return value' component. Section 4.1.3 defines the object vector as the final-token activation in layers 15–31, and Section 3.1 states that the output distribution is Softmax(phi(h_{L-1,n-1})), so replacing exactly that vector with the reference vector necessarily overwrites the primary input to the classification head. A random vector of matched norm at the same position could plausibly produce a similarly large effect. I ask for two controls: (i) patch the object position with a non-reference activation vector, such as random noise or a vector from an unrelated example, and report the resulting accuracy; and (ii) patch only a subset of the object layer range, for example layers 15–20 and 21–31 separately, to show that the effect is not simply due to overwriting the head input.","section":"§3.1, §4.1.3, Table 1"},{"comment":"The layer ranges for subject (0–14) and relation (0–10) vectors are derived from the SES/RES/OES heatmaps in Figures 4–6, which are computed on the same filtered dataset that is later used for the interchange evaluation in Table 1. Because the component definitions and the evaluation labels come from the same facts, the high interchange accuracy is partly a consequence of circular selection. Please provide a held-out evaluation, for example selecting components on a development split and evaluating on a disjoint test split; report bootstrap or split-level variance for Table 1; and include a random-vector baseline to quantify how much of the interchange effect is due to the locality of computation rather than to the specific semantic content of the vector.","section":"§4.1.3, §4.2, Appendix A.1.1"},{"comment":"The dual interchange accuracies (88.5–92.7%) are consistently 5–10 points below the object-only accuracies (99–100%) and also lower than the subject-only and relation-only accuracies for most rows. Under the proposed independence hypothesis, replacing both input arguments should not degrade performance below the single-component interventions unless the two vectors interact. This gap is direct evidence against the 'distinct, separable components' claim as stated, and it is not discussed. Please analyze the failure cases, report whether the gap is statistically significant, and compare the observed dual accuracy with the accuracy predicted under an independence model, such as the product of single-intervention accuracies.","section":"Table 1, §4.2"},{"comment":"The knowledge editing method is described inconsistently. The first paragraph says to patch 'the late layer activation vectors of the previous token of objects, e.g., the \"is\" token' with the new object mean activation vector, while the next paragraph says to extract 'the mean activation vector at the early layers (0–14) from the object tokens of new knowledge prompting.' This contradicts Section 4.1.3, where object vectors are defined in layers 15–31 at the final token, and it is unclear whether the patched location is the token before the object, an object token, or the final token. Please clarify the exact token positions and layer ranges used for both extraction and patching, and justify why the editing setup differs from the object-vector definition used in the scoring and interchange experiments.","section":"§5.1"},{"comment":"All experiments are run on a filtered subset containing only facts that the model already predicts correctly with both templates, for example 155–547 of 836 examples for Landmark-Country. This filter may select for facts whose recall is unusually easy and localized, so the high interchange accuracies may not generalize to the full knowledge distribution. Please report the filtering rate for each relation and, if feasible, run the scoring and at least the subject-only and relation-only interchanges on the unfiltered data or on a random sample stratified by model confidence, to bound the selection effect.","section":"Appendix A.1.1, Table 3"}],"minor_comments":[{"comment":"There are typos in the appendix headings and text: 'Activation V ectors Interchange' (A.3.2) and 'a the relation' (A.3.1) should be corrected.","section":"Appendix A.3"},{"comment":"The relation name is inconsistent: Table 1 uses 'Fruit-Color' while Table 3 lists 'Food-Color'; please reconcile.","section":"Table 1, Table 3"},{"comment":"The template in Section 4 ('Given <subject>, it's <relation> is') differs from the template in Section 4.1.3 and Figure 4 ('Given <subject>, its <relation> is'); standardize the phrasing.","section":"Section 4, Section 4.1.3, Figure 4"},{"comment":"Section 5.2 says 3000 new triplets are constructed, while Appendix A.4 describes replacing objects for each factual triple; please specify whether 3000 is the total or per relation and report the number of examples behind Table 2.","section":"Section 5.2, Appendix A.4"},{"comment":"The paper does not state whether code or data will be released; for reproducibility, please include a link or an availability statement.","section":"General"},{"comment":"The captions of Figures 4–6 do not define the color mapping for SES, RES, and OES; please add a legend or describe the color scheme in each caption.","section":"Figures 4–6"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope for an interpretability venue, but the central independence claim is currently under-supported. The near-100% object-only accuracy is the most striking number in the paper, but it is largely an artifact of the intervention site; without the requested controls, the paper would be vulnerable to the criticism that the functional decomposition is an interpretive overlay rather than a demonstrated mechanism."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is worth a look but weaker than its headline. The central idea — treating subject, relation, and object vectors as input, function body, and return value — is a clean framing, but most of the empirical content restages known results (causal tracing, relation decoding linearity, function vectors). The genuinely new bit is the activation-patching knowledge editing method, and that part mostly works: efficacy score jumps from ~0.17 to ~0.66 with patching, across two templates and a QA format.\n\nWhat the paper does well: the scoring is systematic, run on two 7B models and two prompt templates, and the interchange effects track the scoring heatmaps. The layer-by-layer plots are clear and consistent. The editing application is a concrete, useful byproduct.\n\nThe soft spots are real and mostly in how the independence claim is supported. The object-only interchange hits 99–100%, but that's nearly by construction: the object vector is defined as the final-token activation in layers 15–31, which is exactly the representation the LM head consumes. Patching that vector with the reference version will move the output wherever the reference object is, regardless of whether the model implements a clean 'return value' component. The subject/relation numbers (92–97%) are more informative, but the layer ranges (0–14 and 0–10) are selected from the same scoring heatmaps on the same filtered facts used for evaluation. There is no held-out split, no random-vector control, and no error bars. Without those, the high accuracies could just reflect that the chosen vectors are causally upstream of the output, not that they are independent functional components. The dual interchange dropping to 88–93% suggests interaction that is never addressed.\n\nThe editing section also has an internal inconsistency: the method patches late-layer activations of the token before the object, but the new object vector is extracted from early layers (0–14). That may be intentional, but it needs justification. The dataset is filtered to facts the model already predicts correctly, which is reasonable for a recall study but should be stated as a bound on generalization. And the unresolved '?' citation in the introduction is sloppy.\n\nThe paper ships no code or data, so the numbers can't be checked independently. That said, the direction of the results is plausible and the framing could be useful. The fixes are straightforward: hold out layer selection, add random-vector and norm-matched controls, report variance, and reconcile the editing extraction/patching layers.\n\nOverall: this deserves a serious referee but not unconditional acceptance. I'd send it back for revisions with those controls requested. Worth discussing in reading group, but I wouldn't cite the independence claim as established.","headline":"A clean functional framing of knowledge recall that mostly restages known results; the editing trick works, but the independence claim is under-supported without controls.","tokens_in":12457,"tokens_out":2606,"would_cite":false,"duration_ms":23356,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Knowledge recall in LLMs is a function call in activation space.","keywords":["knowledge recall","activation patching","functional abstraction","representational interpretability","knowledge editing","interchange intervention","factual association","transformer interpretability"],"falsifier":"Take a subject-only interchange pair and patch only a random half of the subject token positions; if the reference object still wins as often as with full subject patching, the 'subject vector' is not the local carrier claimed. Conversely, patch a late-layer final-token vector the scorer marks as low-OES; if it flips the predicted object at nearly the same rate as the high-OES vector, the scoring step is not identifying a specific return value.","tokens_in":11386,"feed_emoji":"🧠","tokens_out":6102,"duration_ms":49716,"temperature":0.7,"pith_summary":"This paper tries to establish that when a transformer LLM recalls a factual association, its hidden activations implement a function call: the subject token's activation is the input argument, the relation token's activation is the function body, and the final-token activation is the return value. Using a patching-based knowledge-scoring algorithm, the authors locate activation vectors that carry subject, relation, and object information and show that they are localized along specific token positions and layers. Counter-knowledge interchange tests then show that replacing a single identified activation vector with the corresponding vector from another query flips the model's answer to the other query's object most of the time, with roughly 92-100% interchange accuracy depending on the component. The result matters because it offers a concrete functional abstraction of knowledge recall that can be applied to knowledge editing: patching the object vector of a new fact into a context prompt raises the edited fact's success from near zero to about 65-69% on their tests.","feed_headline":"LLM fact recall runs like a function call in activation space","feed_subtitle":"The paper localizes subject, relation, and object to separate vectors and swaps them to flip recalled facts at 92-100% accuracy.","key_machinery":"The load-bearing machinery is activation patching used as both a probe and an intervention. For probing, the paper defines knowledge-encoding scores for subject, relation, and object by corrupting the relevant token embeddings with Gaussian noise, then restoring each corrupted activation one at a time and measuring the recovered object probability; high scores mark the activation vectors that carry each piece of knowledge. For verification, the located vectors are interchanged between a source query and a reference query, and the predicted object is compared with the object that would follow from the corresponding textual swap. The functional abstraction itself, (subject, relation) mapping to object, is the identity that organizes these operations and gives the components their names.","core_discovery":"The paper asserts that knowledge recall in transformer LLMs is, at the level of activations, a function execution with three separable components: subject-token activations in early layers act as the input argument, relation-token activations in early layers act as the function body, and final-token activations in middle-to-late layers act as the return value. The evidence is a knowledge-scoring step that localizes these vectors, followed by interchange interventions: swapping only the subject vector from another query changes the prediction to the other query's object with about 93-97% accuracy; swapping the relation vector gives about 92-95%; swapping the object vector gives about 99-100%; swapping both subject and relation gives about 89-93% on the filtered dataset. The authors conclude that these activation vectors function as distinct, separable components in the knowledge recall process rather than as a distributed tangle.","pith_inferences":["Editorial inference: Because object-only interchange is nearly perfect while subject- and relation-only interchange are slightly lower, the return value may be more cleanly localized than the input argument and function body; a natural test is whether the subject vector's residual error shrinks as more subject-token positions are patched.","Editorial inference: The paper tests six relation types on two 7-billion-parameter models; a stress test for the framework would be a multi-hop or compositional fact, where a relation function must be applied twice, to see whether the same component boundaries hold.","Editorial inference: The knowledge-editing results are short-term and measured on immediate queries; an untested extension is whether repeated patched prompting can consolidate the new object into long-term recall."],"forward_implications":["Factual knowledge can be edited without weight updates by patching the object vector into a context prompt: the paper reports the edited fact's efficacy score rising from about 0.14-0.19 without patching to about 0.66-0.69 with patching.","The same component localization appears under two prompt templates with different subject-relation order, implying the functional roles are tied to token content rather than to template position.","Dual interchange composes a subject from one fact and a relation from another, so if the paper is right, factual recall is modular enough to support compositional knowledge operations at the activation level.","The knowledge-scoring algorithm gives an interpretability tool that locates factual associations in comparable transformers without training probes or parameter access."],"supporting_citations":[{"why":"Supplies the relational knowledge database used for scoring and interchange, and motivates reading relation decoding as a mapping between entities.","marker":"Hernandez et al., 2023"},{"why":"Provides the causal mediation analysis that the knowledge-scoring algorithm adapts to locate knowledge-aware activation vectors.","marker":"Meng et al., 2022a"},{"why":"Establishes causal mediation analysis on transformer activations, the patching basis for the scoring and interchange operations.","marker":"Vig et al., 2020"},{"why":"Supplies the causal abstraction framework behind counterfactual interchange testing.","marker":"Geiger et al., 2021"},{"why":"Extends causal abstraction to align interpretable variables with distributed neural representations, the verification stance of the paper.","marker":"Geiger et al., 2024"},{"why":"Supports the layer-wise account that subject information is transferred in middle layers and factual associations are recalled during generation.","marker":"Geva et al., 2023"},{"why":"Provides the contextual knowledge editing baseline that the patching method improves.","marker":"Zheng et al., 2023"},{"why":"Provides an earlier memory-assisted prompt editing approach that the knowledge editing experiment builds on.","marker":"Madaan et al., 2022"},{"why":"Supplies the Llama-2-7B model used in the experiments.","marker":"Touvron et al., 2023"},{"why":"Supplies the Mistral-7B model used in the experiments.","marker":"Jiang et al., 2023"}],"fun_headline_variants":["LLM recall: subject, relation, object as function parts","Swapping activation vectors flips recalled facts up to 100%","Knowledge recall in LLMs is a separable function call","Activation vectors act as input, body, and return in recall"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument assumes that the knowledge-scoring and interchange procedures isolate a single activation vector that completely and independently carries the subject, relation, or object of a fact; if those representations are distributed across many interacting vectors, high interchange accuracy would not by itself prove the claimed independence.","fun_headline_variants_meta":{"raw":{"variants":["LLM recall: subject, relation, object as function parts","Swapping activation vectors flips recalled facts up to 100%","Knowledge recall in LLMs is a separable function call","Activation vectors act as input, body, and return in recall"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000118,"raw_usage":{"total_tokens":1055,"prompt_tokens":894,"completion_tokens":161,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":510,"completion_tokens_details":{"reasoning_tokens":90}},"tokens_in":510,"tokens_out":161,"duration_ms":2366,"temperature":1.0,"reasoning_tokens":90,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:46:53.951152+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a subject-only interchange pair and patch only a random half of the subject token positions; if the reference object still wins as often as with full subject patching, the 'subject vector' is not the local carrier claimed. Conversely, patch a late-layer final-token vector the scorer marks as low-OES; if it flips the predicted object at nearly the same rate as the high-OES vector, the scoring step is not identifying a specific return value.","supporting_citations":[],"review_version":1}