{"id":"c465a4f0-93cf-4365-9139-ad17da60a558","arxiv_id":"2608.05475","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Task knowledge can be compressed into an external factorized operator that a frozen LLM reads through a trained interface, and this representation outperforms text and matched soft-prefix, prefix, and LoRA baselines on most tested settings.","lead":"This paper introduces KV-Skill, a way to store a task procedure as a small external operator that a frozen language model can read on demand, without adding text to the prompt. The authors show that converting a written skill into this operator often beats using the skill as text or as a soft prompt, and that a compact operator can also be learned directly from task rewards.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Registration lacks a no-content control: gains may come from training an interface on any fixed operator rather than from the text-derived operator's content.","rationale":"The reader identified the fixed-basis representativeness assumption as the weakest point, and I agree that the registration path hinges on what H provides. However, I sharpen the concern: the risk is not simply that H might fail to cover the query distribution, but that the text content of H might be irrelevant altogether. If the interface can learn to use any fixed high-dimensional basis, then the operator's text-derived content is not load-bearing, and the claimed 'compilation' of procedural knowledge from text is unsupported. This is a different failure mode from the reader's 'no amount of interface training can recover' framing, because a sufficiently flexible interface could recover even from an unrepresentative or random basis. The paper's existing random-operator and random-direction controls are performed under a fixed interface trained on the correct operator, so they do not rule out this alternative. The proposed control-text/random-operator registration test would settle the issue. My concern reinforces the reader's CONDITIONAL verdict rather than changing it: the paper needs this additional control before the registration-based half of the central claim can be accepted. The reward-learning path is better controlled via the matched-substrate experiments, so the overall verdict remains CONDITIONAL pending the missing no-content control.","tokens_in":17240,"tokens_out":9228,"duration_ms":91965,"concrete_test":"Fix the Qwen3.5-4B backbone and the registration protocol (Eq. 10, same interface initialization, same training budget and seeds). Derive M_s from three sources: (a) the real LiveMath skill, (b) a length-matched unrelated text (e.g., a Wikipedia article on a different topic), and (c) a random Gaussian matrix with the same shape and Frobenius norm. Train a fresh interface for each source and report LiveMath accuracy on the 124-example test set. If (b) or (c) comes within 5 points of (a), the operator's content is not load-bearing; if (a) clearly exceeds both, the representativeness assumption is supported. Repeat on SearchQA (300 examples) for confirmation.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing weakness is in the registration path. The paper claims that converting a text skill into the operator M_s = H H^T (Eq. 9) makes the same procedural knowledge more effective. But the experimental design does not isolate the operator's content from the addition of a trained interface with gold-label supervision. The interface is trained with CE on gold answers plus KL to the text teacher (Eq. 10), while the Text Skill baseline receives no such training. Because H is a 2560x512 matrix of residual states from a single prefill, it provides a high-dimensional fixed subspace. A length-matched control text or even a random Gaussian matrix of the same shape would also provide such a subspace, and a sufficiently flexible interface could in principle learn to read task-relevant features from any fixed basis. The paper's Table 4 and Figure 4 controls swap operators or directions only after the interface has been trained on the correct operator; they do not test whether the operator's text-derived content is necessary when the interface is trained from scratch. If an unrelated-text or random operator achieves comparable accuracy under the same registration loss, the 'text-derived' mechanism is not what drives the gain, and the headline claim for the registration path collapses to 'training a small interface with gold labels helps.' This is not addressed by the matched reward experiments in Table 5, which concern the reward path only.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces KV-Skill, an external factorized associative operator Ms (with factors U_s, W_s) that a frozen language model reads through a lightweight trainable interface I. The authors propose two construction paths: registration, which converts an authored text skill into a fixed operator Ms = H_s H_s^T using residual states from a single prefill and then trains only the interface with a combination of KL matching to the text teacher and cross-entropy on gold answers; and reward learning, which jointly optimizes a compact operator and interface directly from verifier outcomes. Experiments across ten benchmarks and four backbones report that registration substantially improves over the source text skill, that reward-learned KV-Skills outperform soft prefixes, prefix-KV, and LoRA under matched reward training and parameter budgets, and that text-derived operators can be truncated to one task-aligned direction per layer with little loss. The paper also demonstrates that one shared interface can load multiple skills and that swapping the loaded operator changes task behavior.","tokens_in":17539,"tokens_out":4933,"duration_ms":46988,"significance":"If the central claims hold, the paper offers a genuinely useful design space for storing and loading task knowledge outside both the prompt and the model weights, with modularity benefits and a clear separation between knowledge acquisition and deployment. The matched reward comparisons in Table 5 are well controlled and provide strong evidence for a substrate effect; the swap experiment in Table 4 gives direct evidence that, after training, the loaded operator, not the interface, carries task identity. The rank-one compression result is surprising and mechanistically informative. The paper also ships a thorough resource-accounting appendix and is generally careful with the limitations of its claims. However, the registration path—which anchors the abstract's central claim that 'task knowledge can be acquired from text ... compressed into an external operator'—lacks a no-content control that would separate the contribution of the text-derived operator's content from the addition of a trained interface with gold-label supervision.","major_comments":[{"comment":"The registration comparison is confounded: the KV-Skill condition trains an interface with gold labels plus KL to the text teacher, while the Text Skill and other baselines receive no equivalent training on a fixed operator. To support the claim that converting text to a KV-Skill makes the same procedural knowledge more effective, the authors must run a no-content control: train the same interface with the same registration loss (Eq. 10) on a random fixed operator of the same shape (or an operator derived from an unrelated text skill) and report the resulting performance. Table 4 and Figure 4 are post-hoc controls: they hold an interface that was trained on the correct operator fixed and then swap or truncate the operator. They do not test whether the text-derived content is necessary to achieve the gain when the interface is trained from scratch. If a random operator under the same training protocol reaches comparable accuracy, the registration headline collapses to 'training a small interface with gold labels helps' and the text-derived mechanism is not supported.","section":"§4.1 and §3.4 (Eq. 10)"},{"comment":"The text-derived operator M_s^ℓ = H̄_s^ℓ H̄_s^ℓ⊤ assumes that a single frozen prefill of the authored skill produces residual states that form a sufficient fixed basis for the task. This representativeness assumption is load-bearing for the registration path: if the queries encountered during interface training or test lie outside the span of the collected text-skill states, the operator cannot express the required response regardless of how much the interface is trained. The paper does not test this assumption directly. The random-operator control proposed above would also serve as a partial test of whether the text states' specific content matters, but the authors should additionally consider ablating the number of prefills or the length of the skill document to probe whether the operator improves with more representative states.","section":"§3.3 (Eq. 8) and §3.4"},{"comment":"The rank-correction exponent c_k = (k/N_s)^(1/4) is described as 'analytic' and 'parameter-free', but no derivation is provided. If the fourth-root law was chosen post hoc to make the rank-one results align with the full-rank operator, the claim of no fitted coefficient is misleading. The authors should provide a derivation from the read scale (e.g., from the expected norm of U^⊤ q when U contains N_s normalized states) or, failing that, report the sensitivity of the rank-one accuracy to the exponent (e.g., 1/2, 1/3, 1/5). This matters because the rank-one compression result is one of the paper's headline findings.","section":"§B.1 and §4.2 (c_k = (k/N_s)^(1/4))"},{"comment":"The sequential-registration result, used to support the claim that one shared interface can retain multiple independently loadable KV-Skills without measurable forgetting, is based on a single seed, and the LiveMath intermediate value is reported as a range rather than a point estimate. This is an inherently high-variance claim (small task, one seed, sequential optimization). The replay ablation is also a single-run result. The authors should provide at least three seeds for the sequential registration experiment or explicitly soften the 'no measurable forgetting' claim to a preliminary observation.","section":"§4.5 and Table 10"}],"minor_comments":[{"comment":"The parameter counts are not exactly matched: KV-Skill uses 1.330M and 1.067M parameters on the two backbones, while LoRA uses 1.376M and 1.024M respectively. The text says 'matched parameter count' but the numbers differ by up to 4%; please clarify the matching tolerance or state that the match is approximate.","section":"Table 5"},{"comment":"The 'Random direction' row in Table 3 reports LiveMath accuracy 26.6, which is below the Base accuracy of 22.4 in Table 1; the paper later says 'a magnitude-matched random direction preserves at most 22% and returns SearchQA to Base' but the LiveMath number is not Base. Please add a footnote to Table 3 clarifying the Base value under the same generation budget and example count.","section":"Table 3 and §B.1"},{"comment":"The abstract says 'across ten benchmarks', but the main tables report results on five main tasks, and the remaining benchmarks (STaRK-Prime, STaRK-MAG, OfficeQA, SpreadsheetBench, ALFWorld) are only in the appendix. Consider adding a sentence in the main text pointing to the appendix for the full ten-benchmark list.","section":"Abstract and §4.1"},{"comment":"The resource-accounting section is thorough, but the sentence 'The cached-swap measurement covers changing the operator used for future tokens. It does not recompute representations already stored in the model’s attention KV cache' should be moved earlier in the main text (near §4.3) so that readers do not over-interpret the '0.00 ms' swap time as including history re-encoding.","section":"§C.5 and §C.6"}],"recommendation":"major_revision","confidential_remarks":"The paper is well written and the experiments are extensive, but the registration-path claim is the one most prominently featured in the abstract and introduction, and the missing no-content control is a real experimental gap rather than a presentational issue. If the authors can add the random-operator/unrelated-text-operator control and show that the text-derived operator is necessary for the registration gains, the paper would be a strong candidate for acceptance. I also note that the reward-learning path provides independent evidence for the external-operator idea, so the paper is not fatally undermined if the registration control fails; the claims would simply need to be narrowed. The rank-correction exponent should be justified or at least shown to be insensitive."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is a serious empirical study of a genuinely useful idea: represent task knowledge as an external factorized operator that a frozen model reads through a small trained interface. The operator form is borrowed (they say so), but the two-path framework—registration from a single text prefill and reward learning of a compact latent operator—is new, and the compression finding is interesting.\n\nThe strongest part is the matched reward comparison in Table 5. They match reward, training budget, and parameter count across KV-Skill, soft prefix, prefix KV, and LoRA on two backbones and several tasks, and KV-Skill wins seven of eight settings. That is real evidence that the substrate matters, not just the optimizer. The swap experiment in Table 4 also cleanly shows that after the interface is trained, the loaded operator carries the task identity. And they are unusually honest: they report resource costs, admit the ALFWorld limitation, and cite prior work carefully.\n\nThe soft spot is exactly what the stress-test flags: the registration path never isolates the operator's content from the fact that the interface gets trained with gold labels. The Text Skill baseline is frozen; registration trains the interface with CE plus KL. So the gain could come from giving the model a trainable readout on any fixed high-dimensional subspace, not from the text-derived content. Table 4 swaps operators only after the interface has been trained on the correct one—it does not answer the question. A no-content control—registering the same interface on a random matrix or unrelated text operator under the same loss—is missing. That is a load-bearing gap for the headline claim about registration specifically. The reward path does not have this problem, and the paper would stand on that alone, but the abstract sells both paths.\n\nTwo minor issues: the rank-one correction exponent c_k is asserted without derivation, and most baselines are single runs without error bars. Neither is fatal, but they should be addressed.\n\nFor whom: anyone working on parameter-efficient adaptation, skill libraries, or modular agents. It deserves a serious referee; the registration concern is likely fixable with one control experiment. I would recommend peer review with that request.","headline":"A well-crafted empirical paper on external task operators with a strong matched reward study, but the registration path needs a no-content control before the headline claim fully lands.","tokens_in":687,"tokens_out":823,"would_cite":true,"duration_ms":24247,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that task knowledge can be stored outside both the prompt and the model's weights, as an external factorized operator that a frozen language model reads through a small trained interface.","keywords":["KV-Skill","external factorized operator","frozen language model","task knowledge","text-derived skill","reward learning","residual stream steering","rank-one compression"],"falsifier":"A decisive test is to register a skill whose single-prefill residual states do not cover the held-out query distribution, and compare the rank-one operator against a magnitude-matched random direction: the central claim predicts the task-aligned direction retains most of the gain while the random direction collapses, so observing the opposite pattern would refute the representativeness assumption.","tokens_in":17049,"feed_emoji":"🧠","tokens_out":10402,"duration_ms":87440,"temperature":0.7,"pith_summary":"KV-Skill is a design space for storing task knowledge outside both the prompt and the backbone, in an external factorized operator that a frozen language model reads through a lightweight interface. The paper tries to establish that the same procedural knowledge becomes more effective when converted from text into this operator form, and that an unwritten procedure can be learned from reward directly into a compact latent operator. Across ten benchmarks and four backbones, converting a text skill to a KV-Skill consistently improves over the text skill, and under matched reward and parameter budgets the operator beats soft prefixes, prefix KV, and LoRA in seven of eight settings. A registered text-derived operator can be compressed to one task-aligned direction per injection layer with little loss, and one shared interface can retain several independently loadable skills. The conclusion is that task knowledge can be acquired from text or experience, compressed, and deployed separately from the model's weights.","feed_headline":"Storing skills as external operators beats text prompts","feed_subtitle":"A tiny interface lets a frozen language model read a skill from outside the prompt, winning 7 of 8 matched comparisons.","key_machinery":"The central object is an external factorized associative operator $M_s^{(\\ell)} = W_s^{(\\ell)} U_s^{(\\ell)\\,\\top}$ read at selected layers $\\ell$: the columns of $U$ act as latent keys that measure the signed relevance of each skill slot to the current residual query, and the columns of $W$ return the corresponding response. The query is built by an interface $I$ from the model's residual state, and the operator's response is injected back through a gated residual branch, adding no prompt positions and no attention-KV-cache entries. Registration and reward learning are two endpoints of one design space: registration compiles an authored text skill into a fixed, token-aligned operator from a single prefill, while reward learning develops a compact latent operator shared across depths from task outcomes.","core_discovery":"The paper's central claim is that task knowledge can live outside both the prompt and the weights, as an external factorized associative operator $M_s = W_s U_s^\\top$ that is read from the residual stream through a per-backbone interface $I$. Registration sets the operator's factors to the normalized residual states harvested from a single frozen prefill of an authored text skill, and then trains only the interface so the frozen model can use the fixed operator. Reward learning instead optimizes a compact 32-slot latent operator directly from verifier outcomes, with or without a text-informed initialization. The evidence includes a LiveMath registration accuracy of 77.2 versus 23.4 for the text skill and 64.5 for SoftSkill, best results in seven of eight matched reward-training comparisons against soft prefixes, prefix KV, and LoRA, and a rank analysis showing that the operator retains 90 to 100 percent of its gain at one task-aligned direction per layer while matched random directions fail. This is taken to show that the representation through which task knowledge is delivered, not just the objective, determines how well a frozen model can execute a procedure.","pith_inferences":["The registration result suggests a compiler view of text-to-operator conversion: the paper's evidence implies that the retained object is a query-dependent steering direction, and one testable extension is to build rank-one KV-Skills directly from text without first materializing the full token-aligned operator.","If the operator generalizes across query distributions, this external representation could be used for continual learning without weight updates; the paper only demonstrates retention across three tasks on one backbone, so cross-task scaling remains an open extension.","The sparse-terminal-reward result on ALFWorld implies that the bottleneck is credit assignment rather than operator capacity, so a dense-reward version of reward learning is a natural next test for long-horizon tasks.","The paper does not test transferring one KV-Skill tensor across backbones; a direct extension would ask whether a compact reward-learned operator learned in one model's skill space transfers to another model after only a new interface."],"forward_implications":["Because the operator is read from a separate residual branch, a KV-Skill consumes no prompt positions and does not grow with conversation length, so the context window is preserved.","A KV-Skill can be loaded, swapped, or removed without modifying backbone weights, making each skill an independently deployable capability object.","Reward learning can discover an effective operator from verifier feedback alone, so tasks without an authored procedure can still gain an external skill.","The rank-one compression result implies that a text-derived skill can be stored as roughly one direction per layer after a one-time SVD, provided the interface stays fixed.","Because the shared per-backbone interface can retain several registered skills without measurable forgetting, a single reader can serve a growing skill library."],"supporting_citations":[{"why":"SkillOpt optimizes the text skill in text space; it is the baseline that registration beats, showing the gain is not just extra supervision.","marker":"Yang et al., 2026"},{"why":"SoftSkill compresses a text skill into a continuous prefix; it is the main text-to-prefix baseline that KV-Skill outperforms.","marker":"Tao et al., 2026"},{"why":"LoRA is the weight-adaptation baseline under matched reward and parameter budgets.","marker":"Hu et al., 2022"},{"why":"Prefix tuning is the continuous-prompt substrate used in matched reward comparisons.","marker":"Li and Liang, 2021"},{"why":"Linear attention's associative read motivates the factorized operator form $W_s U_s^\\top$.","marker":"Katharopoulos et al., 2020"},{"why":"In-context learning task vectors are the related rank-one steering mechanism the paper contrasts with query-dependent reads.","marker":"Hendel et al., 2023"},{"why":"Activation steering provides the residual-stream direction baseline that a rank-one KV-Skill extends with query-scaled coefficients.","marker":"Turner et al., 2023"},{"why":"KBLaM's external key-value memory represents declarative knowledge, distinguishing the procedural operator KV-Skill introduces.","marker":"Wang et al., 2024"}],"fun_headline_variants":["Task skills as external operators beat text prompts","KV-Skill: Skills outside prompt and weights win 7 of 8","Frozen models read skills from operators, not prompts","External operator skills outperform text and LoRA tuning","Skill storage outside the prompt beats text, LoRA, prefixes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"Registration assumes that a single frozen pass through the authored text skill produces residual states that already cover the task's query distribution; if those states are unrepresentative, training the interface cannot recover the lost procedure.","fun_headline_variants_meta":{"raw":{"variants":["Task skills as external operators beat text prompts","KV-Skill: Skills outside prompt and weights win 7 of 8","Frozen models read skills from operators, not prompts","External operator skills outperform text and LoRA tuning","Skill storage outside the prompt beats text, LoRA, prefixes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000193,"raw_usage":{"total_tokens":1418,"prompt_tokens":1079,"completion_tokens":339,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":695,"completion_tokens_details":{"reasoning_tokens":259}},"tokens_in":695,"tokens_out":339,"duration_ms":3290,"temperature":1.0,"reasoning_tokens":259,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T12:23:59.538522+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A decisive test is to register a skill whose single-prefill residual states do not cover the held-out query distribution, and compare the rank-one operator against a magnitude-matched random direction: the central claim predicts the task-aligned direction retains most of the gain while the random direction collapses, so observing the opposite pattern would refute the representativeness assumption.","supporting_citations":[],"review_version":1}