{"id":"e52de037-0b86-4cee-ad13-ff0c6dfed679","arxiv_id":"2608.11660","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"HPSE improves knowledge editing by combining the edited model's own rollouts with token-level corrections from a privileged in-context state, yielding better fact decomposition and composition.","lead":"A new training method called HPSE helps language models actually use new facts given to them in a passage, instead of only memorizing the passage. It makes the model practice answering questions while a privileged copy of the same model fills in missing facts, improving single-hop and multi-hop recall after edits.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The privileged teacher's token-level correctness is never verified; if π⋆ hallucinates fact tokens, HPSE trains the student on wrong targets and the composability gains are not attributable to faithful knowledge injection.","rationale":"The reader's weakest assumption—that the privileged model produces reliable, correct next tokens—is exactly the load-bearing condition I identify. The paper's central claim is that HPSE converts unstructured edits into decomposable and composable knowledge; this requires the injected tokens to be correct. The paper provides empirical evidence (consistent gains across editors, backbones, and benchmarks; ablations; step-in dynamics) and a formal proof sketch, but the proof takes teacher correctness as Assumption A.1 rather than verifying it. The step-in gate in Eq. (3) checks confidence, not correctness, and the privileged model is the same base model that the baselines fail with, so hallucination or prior-knowledge reversion is a live risk. This is a correctness-risk concern, not a circularity or soundness objection: the evaluation uses independent gold answers, which keeps the empirical comparison meaningful; nevertheless, without a teacher-accuracy measurement, the gains cannot be attributed to faithful knowledge injection. The concrete token-level audit and oracle-teacher control would settle the concern. Since the reader already reached CONDITIONAL, my analysis reinforces that verdict rather than moving it.","tokens_in":34253,"tokens_out":3199,"duration_ms":37689,"concrete_test":"Run a token-level teacher-accuracy audit before editing: for each benchmark sample, extract the gold fact spans (objects in UnKEBench, edited objects in MQuAKE-uns) and compute the privileged model's top-1 exact-match accuracy on those spans, conditioned on the editing prompt, over all four backbones. If teacher accuracy on fact tokens is below roughly 95%, or if teacher errors align with samples where HPSE fails, the claim that HPSE injects correct knowledge is not established. Additionally, in the LoRA/Qwen2.5 single-edit configuration, compare HPSE against a variant whose teacher is replaced by an oracle prompt containing the gold facts (and against a corrupted-teacher control). If gains vanish with an oracle teacher, the reported improvements are not attributable to the privileged-state mechanism.","verdict_should_be":"UNCHANGED","load_bearing_attack":"HPSE's entire supervision signal is the privileged model π⋆ = π0(·|c,x) (Eq. 1), and the step-in gate (Eq. 3) switches to it whenever the teacher is confident (π⋆ > κ) and far from the student. There is no external verifier. The theory (Assumption A.1, Remark A.1) simply assumes the teacher supplies the correct new-knowledge tokens ('access to the passage makes them a confident continuation'); it does not establish that the base model's in-context reading of the passage is factually correct, especially for the counterfactual edits used in MQuAKE-uns. Confidence is not correctness: a base model can be confidently wrong, reverting to prior knowledge or hallucinating (cf. baseline failures in Figures 6–7 and 13–16). If the teacher's injected tokens are wrong, the student is trained to imitate those wrong tokens at exactly the positions where the gate fires; the 'decomposable and composable' knowledge is then decomposed hallucination. The reported gains against gold answers suggest the teacher is often right on these benchmarks, but the central mechanism—that HPSE injects correct knowledge rather than merely changing the student's output distribution—is not separately measured. Remark A.9 concedes the guarantee covers only teacher/hybrid support, not student-only error states, and nowhere addresses teacher accuracy. Thus the load-bearing condition for the central claim is unverified.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces Hybrid-Policy Self-Editing (HPSE), a training-signal replacement for gradient-based knowledge editors in the unstructured knowledge editing (UKE) setting. HPSE defines a privileged teacher as the frozen base model with the edit passage in context, constructs hybrid rollouts that switch from the student's own tokens to the teacher's greedy token under a confidence-and-disagreement gate (Eqs. 2--3), and trains the student with a forward-KL distillation loss toward the teacher plus an NLL anchor (Eq. 4). The main empirical contribution is a two-benchmark composability evaluation (UnKEBench and MQuAKE-uns) showing consistent gains for FT-M and LoRA across Qwen2.5, Qwen3, Llama-3.1, and Gemma-2 in single-edit and continual-edit settings. Appendix A contains a formal signal-separation theorem under Assumptions A.1--A.3.","tokens_in":34543,"tokens_out":10370,"duration_ms":110229,"significance":"Strengths: the empirical evaluation is grounded in external gold-answer benchmarks (exact-match for MQuAKE-uns; expected-fact judge for UnKEBench), so the main empirical claim is not circular. The paper reports ablations, gate sensitivity, step-in dynamics, and continual-editing results, and provides a code repository. If the reported gains hold, HPSE is a practical plug-in that makes existing editors composability-aware without changing their parameterization. The main caveat is that the theoretical analysis covers the teacher's confident tokens, not independently verified gold facts; the significance of the method depends on the privileged model's in-context correctness, which the paper assumes but does not measure.","major_comments":[{"comment":"The gate in Eq. (3) injects the teacher's greedy token whenever the privileged model is confident, with no external verifier. Assumption A.1 defines the fact span as those same teacher tokens, y_star, and posits that they are the new-knowledge tokens the student lacks. The paper reports no measurement of how often the privileged model's confident tokens agree with the gold-answer facts on either benchmark (e.g., a per-token or per-question agreement rate), nor any analysis of cases where the base model reverts to prior knowledge or hallucinates a passage reading. Because the student is trained to imitate these tokens at exactly the step-in positions, the semantic claim that HPSE injects the intended knowledge is contingent on an unverified premise. The empirical gains suggest the teacher is often right on these datasets, but the paper should either supply this verification or explicitly qualify the central claim.","section":"Appendix A, Assumption A.1; Section 3.2, Eq. (3)"},{"comment":"The informal theorem states that 'the hybrid signal is Omega(ell) while the OPSD signal is O(1)', but the signal is the KL divergence to the privileged model's distribution, and the fact span is the teacher's own greedy token sequence. The theorem is therefore a coverage statement about the teacher's confident continuation, not a statement about the gold facts. The manuscript's abstract and Section 4.2 use 'injects the knowledge' in ways that go beyond what the proof establishes. I request a revised theorem statement and interpretation that makes this conditional explicit, together with either an empirical link between teacher tokens and gold facts or a relaxation of the claim.","section":"Section 3.3, Theorem 3.1; Appendix A.2"},{"comment":"In this cell, HPSE improves Ind. from 2.0 to 5.1 but lowers Cmp. from 12.0 to 8.0 (-33.3%). The aggregate average remains within one point (7.0 vs. 6.5), so the paper's stated 'exceptions within one point on average' is technically correct, but the trade-off between individual and compositional recall in this cell is a direct counterexample to the general claim that HPSE improves composability. The paper should analyze this cell and state the conditions under which HPSE can hurt composition despite improving individual recall.","section":"Section 4.3, Table 7 (MQuAKE-uns, Qwen3, T=10)"}],"minor_comments":[{"comment":"The tau sweep shows Jnt. dropping from 75.0 (tau=2) to 69.6 (tau=4), a 5.4-point change on one metric; the 'robust' conclusion is based on the average varying within 2.2 points. Please report per-metric variability and discuss the practical choice of tau in light of this Jnt.-Dmp. trade-off.","section":"Appendix D.2, Table 8"},{"comment":"The continual-edit results are presented only as filled/outline bars, and the reader cannot see the variance or per-metric dispersion from the figure. Consider showing error bars or per-metric values in the main text for at least one setting, or moving the figure to a more readable format in an appendix.","section":"Section 4.3, Figure 4"},{"comment":"The gate calibration is described as 'inspecting the number of tokens they select on a few sample sequences', which is not sufficiently operationalized for replication. Please specify the target step-in rate or a concrete calibration procedure.","section":"Appendix C.2, Table 4"},{"comment":"CODE (Li et al., 2026a) is a closely related on-policy self-distillation method adapted to knowledge editing; its exclusion from the experiments is reasonable given its dependence on an external frontier model for narrative synthesis, but this should be stated explicitly in the experiment section.","section":"Section 5, Related Work"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid empirical contribution with a coherent proof under explicit assumptions and extensive benchmarking. The major issue is the unverified premise that the privileged model's confident in-context tokens are the intended gold facts. Adding a teacher-accuracy analysis or substantially qualifying the 'knowledge injection' claims would make the paper publishable. The paper fits the journal's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Know this before you read it: HPSE is the first work I've seen that treats unstructured knowledge editing's composability problem as a training-signal problem, and the plug-and-play framing is real. The central idea—use the base model with the edit passage in context as a privileged teacher, and step in with its confident tokens exactly where the student's rollout would diverge—is clean, and the empirical pattern holds up surprisingly well: consistent gains across four backbones, two editors, single and continual editing, with ablations isolating the hybrid rollout.\n\nWhat's genuinely new: the confidence-gated step-in with a gap threshold, the coverage-failure diagnosis, and the untargeted regime with decomposition/composition probes. It's not just \"apply OPSD to UKE\". CODE uses an external teacher for structured KE, SKD uses an external teacher; HPSE uses the model's own in-context state, which is a real difference and also the source of the main soft spot.\n\nThe soft spot is the teacher's correctness. The stress-test note is right: Assumption A.1 simply assumes the teacher supplies the correct new-knowledge tokens, and the theory only shows the hybrid visits those prefixes, not that they are factually right. Confidence is not correctness. On these benchmarks the gains against gold answers suggest the teacher is usually right, but the mechanism isn't separately measured. That's a real gap, though not a fatal one. The paper would be stronger with a per-edit teacher-accuracy check, or at least a discussion of failure cases where the base model confidently reverts to prior knowledge. Remark A.9 already concedes the guarantee doesn't cover student-only error states.\n\nOther soft spots, in order of severity: no variance or seed information anywhere—all tables are single numbers; the theoretical condition ρ < κe^{-τ} is never verified on the actual models; and the benchmark construction leans on Gemini for the untargeted prompts and passage expansion. None of this sinks the main claim, but combined with the teacher-correctness point, the numbers should be treated as promising rather than final.\n\nThe proof itself is coherent under the stated assumptions. The signal-separation result is real math, though it analyzes the algorithm's own definition—the hybrid signal visits fact prefixes by construction. That's a legitimate theoretical contribution, not circular reasoning, because the empirical evaluation is against independent gold answers.\n\nWho should read it: anyone in knowledge editing or LLM updating. It's a well-communicated contribution to a known gap, and it deserves a serious referee. My recommendation: send it to review, and ask the authors for seeds and variance, a teacher-accuracy audit, and a clearer separation of Gemini's role in benchmark construction. With those, it could become a reference point for composability in UKE.","headline":"A genuinely useful plug-in for composable unstructured knowledge editing, with a real but fixable gap: the privileged teacher's correctness is assumed, not verified.","tokens_in":35120,"tokens_out":2731,"would_cite":true,"duration_ms":28307,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Knowledge edits become composable when the student's own rollout is repaired mid-generation.","keywords":["knowledge editing","unstructured knowledge editing","composability","self-distillation","hybrid rollout","multi-hop reasoning","large language models"],"falsifier":"Take an edit passage engineered so the base model's in-context continuation of a fact is confident but wrong (an internal contradiction, a misleading paraphrase, or an entity whose in-context completion conflicts with the passage), run HPSE, and check whether the student adopts the wrong token. Separately, on a sample of UnKEBench edits, measure the student's probability $\\rho$ of each fact token before editing and test whether $\\rho < \\kappa e^{-\\tau}$; if the inequality fails, the theorem's coverage guarantee does not apply even if the method still improves scores.","tokens_in":34050,"feed_emoji":"🧩","tokens_out":7148,"duration_ms":64942,"temperature":0.7,"pith_summary":"This paper claims that unstructured knowledge editing fails because editors passively fit a fixed passage: the edited model can repeat the passage but cannot answer atomic questions about its facts or chain them into multi-hop reasoning. The authors define this missing property as composability, decomposed into decomposition and composition, and benchmark it under an untargeted regime where the editing prompt does not reveal which facts are updated. They then propose HPSE, a plug-and-play training signal in which the student model generates its own response while a privileged in-context copy of the same base model, $\\pi^\\star = \\pi_0(\\cdot \\mid c, x)$, steps in precisely where the student's rollout would stray, placing the missing fact tokens onto the student's trajectory. The hybrid rollout is distilled back into the student with forward KL plus an NLL anchor on the passage. Empirically, HPSE improves both decomposition and composition scores for FT-M and LoRA across four LLM backbones, with composition gains up to +9.9 points for LoRA, and the paper argues this comes without sacrificing locality.","feed_headline":"Hybrid rollout makes edited LLM knowledge decomposable and composable","feed_subtitle":"Step-in injection lifts multi-hop recall by up to 9.9 points on LoRA, without hurting locality.","key_machinery":"The central object is the per-token hybrid rollout policy $\\pi_\\rho$ of Eq. (2), a switch between the student $\\pi_\\theta$ and the privileged model $\\pi^\\star = \\pi_0(\\cdot \\mid c, x)$ that reads the edit passage in context. The step-in gate of Eq. (3) fires when two conditions hold: the privileged model's log-probability of its greedy token exceeds the student's by more than $\\tau$, and that privileged token has confidence above $\\kappa$. The gate converts coverage failure into targeted supervision by placing the missing fact tokens onto the student's own trajectory, and the loss of Eq. (4) matches the student to the privileged distribution over those prefixes via forward KL while an NLL term anchors the passage-level objective. The gate self-terminates: as the student internalizes the facts, the privileged–student gap falls below $\\tau$ and the hybrid rollout converges to the on-policy rollout.","core_discovery":"The central claim is that the gap between passively injected and usable knowledge is a coverage problem: because the new facts are novel, the pre-edited model's own rollouts rarely visit them, so pure on-policy self-distillation (OPSD) has almost no corrective signal to offer. HPSE closes this gap with a hybrid rollout policy that draws student tokens but switches to the privileged model's greedy token when the privileged–student gap exceeds $\\tau$ and the privileged model's confidence exceeds $\\kappa$; the student is then trained to match the privileged distribution on those prefixes. The paper proves that under its novelty assumptions the hybrid rollout visits every fact prefix, giving a fact-signal $S_{\\pi_\\rho} = \\Omega(\\ell)$, while OPSD's signal stays $O(1)$, so the advantage grows with the length of the new fact span. On the empirical side, HPSE is reported to improve composed recall by +2.3 points on average for FT-M and +9.9 points for LoRA, and to lift decomposed recall while preserving answer diversity and MMLU locality.","pith_inferences":["Replace the max-probability confidence gate with an entropy-based or calibrated-uncertainty gate and the behavior should be similar, since the theory's $\\kappa$ floor exists to keep the per-token signal bounded away from zero; this is a cheap ablation.","Because HPSE is token-level and parameter-agnostic, it should transfer to representation-level editors and to non-text token spaces, which the authors list as future work; the coverage argument does not depend on a particular parameterization.","The benchmarking protocol suggests that UKE evaluations should report diversity alongside decomposed recall, since a model that regurgitates the passage can otherwise masquerade as having decomposed knowledge.","A stress test worth running is adversarial passages that make the privileged model confident but wrong; if HPSE amplifies such errors, the method inherits every systematic bias of in-context learning."],"forward_implications":["Existing gradient-based KE editors such as FT-M and LoRA can gain composability without changing which parameters they update, because HPSE replaces only the training signal.","Editing becomes more useful for downstream agents: injected facts can be queried atomically and chained into multi-hop answers, rather than requiring the original passage as a retrieval cue.","The step-in gate self-terminates, so the intervention is a temporary curriculum rather than a permanent auxiliary mechanism.","Under continual editing, the advantage persists as edits accumulate, with 29 of 32 tested editor–model–horizon settings improved.","The theoretical analysis predicts the benefit grows with the length of the new fact span, which is exactly the unstructured-editing regime."],"supporting_citations":[{"why":"Supplies the UnKEBench decomposition probe and the passage-level editing format.","marker":"Deng et al. (2024)"},{"why":"Provides the cleaned MQuAKE multi-hop benchmark that the composition probe is built from.","marker":"Zhong et al. (2025)"},{"why":"Defines the FT-M fine-tuning editor that HPSE plugs into.","marker":"Zhang et al. (2024c)"},{"why":"The LoRA low-rank adaptation baseline that HPSE is applied to.","marker":"Hu et al. (2022)"},{"why":"Defines on-policy self-distillation, the base paradigm HPSE extends with its hybrid rollouts.","marker":"Zhao et al. (2026)"},{"why":"Foundational on-policy distillation result that motivates the self-distillation analysis.","marker":"Agarwal et al. (2024)"},{"why":"Provides the passage-generation prompt used to expand structured MQuAKE edits into unstructured passages.","marker":"Wu et al. (2024a)"},{"why":"FActScore, the LLM-as-judge scoring used for joint recall on UnKEBench.","marker":"Min et al. (2023)"}],"fun_headline_variants":["Hybrid rollout makes edited knowledge composable","Step-in injection lifts multi-hop recall by up to 9.9 points","Coverage gap in knowledge editing fixed by hybrid rollout","Hybrid self-distillation achieves composable knowledge edits"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The method assumes that the base model's in-context reading of the edit passage, $\\pi^\\star = \\pi_0(\\cdot \\mid c, x)$, reliably produces the correct, stylistically compatible next token for every fact it injects, because there is no external verifier and the student will imitate whatever the privileged model emits.","fun_headline_variants_meta":{"raw":{"variants":["Hybrid rollout makes edited knowledge composable","Step-in injection lifts multi-hop recall by up to 9.9 points","Coverage gap in knowledge editing fixed by hybrid rollout","Hybrid self-distillation achieves composable knowledge edits"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000819,"raw_usage":{"total_tokens":3631,"prompt_tokens":1035,"completion_tokens":2596,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":651,"completion_tokens_details":{"reasoning_tokens":2530}},"tokens_in":651,"tokens_out":2596,"duration_ms":21901,"temperature":1.0,"reasoning_tokens":2530,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T00:31:56.886062+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take an edit passage engineered so the base model's in-context continuation of a fact is confident but wrong (an internal contradiction, a misleading paraphrase, or an entity whose in-context completion conflicts with the passage), run HPSE, and check whether the student adopts the wrong token. Separately, on a sample of UnKEBench edits, measure the student's probability $\\rho$ of each fact token before editing and test whether $\\rho < \\kappa e^{-\\tau}$; if the inequality fails, the theorem's coverage guarantee does not apply even if the method still improves scores.","supporting_citations":[],"review_version":1}