{"id":"7529ce7c-e1c5-4437-85e0-77bfa7cbd1db","arxiv_id":"2607.14194","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":11,"one_line_summary":"SIRUS is a training-free, inference-time framework that suppresses target concepts in text-to-video diffusion models via subspace-informed prompt projection and residual subtraction, with a new video-centric unlearning evaluation framework.","lead":"Researchers introduce SIRUS, a training-free method that suppresses a target concept in text-to-video generation by projecting prompt embeddings and subtracting a concept residual during sampling, without retraining the model. It also proposes a video-specific benchmark, VUEF, measuring forgetting, preservation, quality, robustness, and efficiency across five concepts, claiming better forgetting-quality trade-offs than prior video unlearning methods.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing no-op and prompt-deletion controls: the claimed trade-off may be explained by target-word removal rather than by SIRUS's subspace/residual mechanism.","rationale":"The reader's weakest_assumption focuses on whether the concept subspace and residual direction generalize beyond the five tested concepts and on unreported hyperparameters. That is a legitimate generalization/reproducibility worry, but the more load-bearing issue for the headline empirical claim is the absence of minimal controls: without a no-op row, the magnitude of the forgetting scores is unanchored, and without a prompt-deletion row, the specific mechanism cannot be distinguished from simply removing the target words. The paper is internally consistent and the reported arithmetic checks out; the concern is that the central comparison may be confounded. The reader's rationale does mention the missing trivial prompt-deletion baseline, so there is partial agreement, but the reader did not elevate it to the primary threat. Because this is an addressable experimental gap rather than a demonstrated falsehood, the appropriate verdict remains CONDITIONAL as the reader's original; I would keep that verdict rather than accept or reject.","tokens_in":21800,"tokens_out":9157,"duration_ms":101905,"concrete_test":"Run two additional rows under the same VUEF protocol on CogVideoX: (1) no-op control: generate the original prompts with no intervention and compute Fail, AnyHit, FrameHit, LPIPS, CSDR, CLIP Ratio, and VBench; (2) prompt-deletion control: remove/mask all target aliases and anchor terms from the prompt before encoding, with all other sampling settings, seeds, N=16, K=4, and MultiClf unchanged, and compute the same metrics. If the prompt-deletion control reaches average forgetting success >= 70.4% and frame hit <= 25.7% with a quality drop comparable to SIRUS's -0.016, the central claim is not supported; if it is clearly worse on forgetting and/or quality, the concern is resolved. Report per-seed standard deviations for both controls.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that SIRUS's inference-time machinery (concept subspace, hierarchical trigger, positive reference branch, residual subtraction) yields the best forgetting-quality trade-off. But VUEF's forgetting metrics (Fail, AnyHit, FrameHit in Sec. 4) are reported only for unlearned models; Table 1 has no row for unmodified CogVideoX, so we do not know how often the original model already avoids the target concept on these prompts. More important, SIRUS's first trigger level is exact alias/anchor matching and its text-side edit c_t is obtained by projecting triggered tokens off the concept subspace (Sec. 3.2). Because the evaluation prompts are adapted from UnlearnDiffAtk and typically name the target concept explicitly, simple deletion or masking of the alias words could plausibly produce much of the observed forgetting. The reference-branch residual (Sec. 3.4) and the subspace projection are then not shown to be load-bearing. The paper neither reports this natural control nor any variance across seeds, so the headline 70.4% vs 44.4% gap is compatible with a much simpler explanation. This is a first-order evaluation-control gap, distinct from the reader's generalization concern about B and d_t.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes SIRUS, a training-free inference-time method for concept unlearning in text-to-video diffusion models. SIRUS builds a contextualized concept subspace from textual aliases, localizes target-related prompt evidence via a hierarchical trigger, projects triggered token embeddings away from the subspace, retrieves a positive concept-reference branch, and subtracts a capped, momentum-smoothed, time-scheduled residual during classifier-free guidance sampling. The paper also introduces VUEF, a video-oriented evaluation framework with separate forgetting, preservation, quality, robustness, and efficiency axes. On CogVideoX across five safety/object/style concepts, SIRUS is reported to reach 70.4% average forgetting success and 25.7% frame hit versus 44.4%/47.2% for VideoEraser, while reducing the VBench quality drop from -0.043 to -0.016. A Wan2.2 transfer study and several ablations are also presented.","tokens_in":22160,"tokens_out":8944,"duration_ms":94415,"significance":"If confirmed, the paper would make a useful contribution: an inference-time, training-free unlearning mechanism that separates concept localization from suppression, together with a multi-axis T2V evaluation protocol. The per-concept tables, the nudity-specific person-retention analysis, and the component ablation are informative, and the authors are transparent about the parachute difficulty, the preservation/forgetting trade-off, and the Refusal Vector backbone mismatch. The external MultiClf classifier and VBench metrics reduce circularity. However, the central trade-off claim currently rests on missing experimental controls and unreported hyperparameters, so the significance of the headline numbers is conditional.","major_comments":[{"comment":"The central forgetting comparison lacks a no-intervention baseline and a prompt-deletion control. Table 1 shows no row for the unmodified CogVideoX on the same prompts, so the original model's Fail/AnyHit/FrameHit rates are unknown. Because SIRUS's first trigger level is exact alias/anchor matching and its text-side edit is a local projection of triggered tokens, and because the evaluation prompts are adapted from UnlearnDiffAtk and explicitly name the target concept, a trivial control that deletes or masks the target words is a natural competitor and could explain much of the observed forgetting and quality preservation. Without such a control, the 70.4% vs 44.4% gap is not attributable to the subspace/residual mechanism.","section":"§4, Table 1; §3.2"},{"comment":"The method is not reproducible from the manuscript. The profile-dependent coefficients απ, γπ, ηπ, ρπ, βπ, and r_{e,π} are defined but their numerical values or the mapping from evidence profiles to parameter sets are not reported. The same holds for the subspace rank K/energy threshold, similarity threshold, relaxed-fallback token budget, and top-k reference count. These parameters determine the intervention strength and schedule; without them the reported trade-off cannot be independently reconstructed or compared across methods.","section":"§3.2–§3.4"},{"comment":"The experimental tables report point estimates without sample sizes, seed variation, or confidence intervals. N=16 is the number of sampled frames per video, not the number of videos per concept. Several comparisons are modest (church 48 vs 34; parachute 50 vs 28), and the average ΔQ difference (-0.016 vs -0.043) has no error bar. The 'strongest forgetting-quality trade-off' claim is not statistically supported. Report the number of prompts/videos per concept and at least three seeds or confidence intervals.","section":"§5.1, Tables 1–3"},{"comment":"The primary 'success rate' uses a permissive failure threshold: with N=16 sampled frames, a video with three or fewer target-classified frames counts as successfully unlearned. For safety concepts, this conflates residue reduction with forgetting. The choice K=4 is asserted without sensitivity analysis. The appendix reports AnyHit, but the main-text headline still depends on K. Report success for K=0 and at least one intermediate threshold, or justify K=4 more rigorously.","section":"§5.2.1, §B.1"}],"minor_comments":[{"comment":"Copy-editing needed: §3.4 contains the duplicated fragment 'Finally, SIRUS applies a three-part temporal schedule. Third, SIRUS applies a three-part temporal schedule.' and §3.3 has 'the trigger mask is used to from a positive reference prompt bank.'","section":"§3.4, §3.3"},{"comment":"The symbol K is used both for the concept-subspace rank and for the forgetting failure threshold. Rename one of them (e.g., K_sub and K_fail) to avoid confusion.","section":"§3.1, §4"},{"comment":"The comparison to Refusal Vector is not backbone-controlled. The paper acknowledges this in §5.5, but the abstract's 'fully evaluated baselines' phrasing should make explicit that only VideoEraser is a same-backbone full baseline on all five concepts.","section":"Abstract; §5.1"},{"comment":"The Wan2.2 generalization claim is supported in the main text only by forgetting metrics; preservation and VBench quality appear only in the appendix. Add a one-line main-text summary or a Wan2.2 baseline row.","section":"§5.4, Table 5"},{"comment":"The anonymized VUEF implementation is released, but no SIRUS implementation link is given. For an inference-time method, releasing the inference wrapper would materially aid reproducibility.","section":"§4"}],"recommendation":"major_revision","confidential_remarks":"The paper's evaluation framework builds on the authors' prior IGMU/EvalIGMU work and uses the MultiClf evaluator from IGMU. I do not treat this as disqualifying, because the metrics are external to SIRUS itself, but the final version should clearly disclose the relationship. The main weaknesses are experimental controls and reproducibility, not methodological soundness of the core idea."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nHere is my take on the SIRUS paper. The contributions are real: a training-free, inference-time unlearning method that combines a text-embedding subspace, hierarchical triggering, local prompt projection, and a concept-reference residual subtraction, plus a five-axis video unlearning evaluation framework (VUEF) that separates forgetting, preservation, quality, robustness, and efficiency. That framework is genuinely useful—it forces the field to report trade-offs instead of cherry-picking forgetting rates. The ablation study shows each component matters, and the numbers are internally consistent.\n\nThe soft spot is the evaluation's missing controls. The stress-test note has it right: Table 1 has no row for the untouched model, and there is no prompt-deletion baseline where you simply mask or remove the target word. Since the prompts name the concept explicitly and the first trigger is exact alias matching, a trivial deletion baseline could plausibly reproduce a good chunk of the 70.4% success. The paper reports no error bars or seed variation, and the central hyperparameters (α_π, γ_π, η_π, ρ_π, β_π, r_{e,π}) are not reported, so I cannot tell whether the advantage over VideoEraser is robust. The Wan2.2 extension is thinner—forgetting only in the main text—and the duplicated sentence and minor typos are cosmetic.\n\nThis does not sink the paper; the method and framework merit a serious look. But the central claim that the subspace/residual mechanism is what drives the forgetting-quality trade-off is conditional until the authors add the no-op and prompt-deletion baselines and report variance. Without those, the 70.4% vs 44.4% gap is compatible with a simpler explanation.\n\nWho is this for? Anyone working on safe T2V deployment or generative unlearning evaluation. I would send it to peer review and ask for these controls before acceptance. If I were working in this area, I would cite the VUEF framework; I would hold off on citing SIRUS's numbers until the controls are in.","headline":"A useful new training-free T2V unlearning recipe and a genuinely multi-axis evaluation framework, but the headline numbers are undercut by missing prompt-deletion/no-op controls that could explain much of the effect.","tokens_in":22641,"tokens_out":2396,"would_cite":true,"duration_ms":23571,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A training-free method, SIRUS, claims to erase target concepts during video generation — no weight updates — with the best forgetting-versus-quality balance among tested baselines.","keywords":["concept unlearning","text-to-video generation","diffusion models","inference-time intervention","concept subspace","residual guidance","machine unlearning evaluation","video generation safety"],"falsifier":"Take a concept outside the five tested (say 'bicycle') and run SIRUS over a few hundred prompts on the same backbone, scoring video-level forgetting with the same classifier. If success collapses toward the no-intervention baseline, or removing the target also distorts non-target content (large LPIPS jump or object-recall drop), the subspace-faithfulness premise fails. Sharper: vary the alias bank size from one to about twenty on a fixed concept — monotone improvement with more aliases confirms the subspace carries the method, while flat or erratic success means something else does. Or disable","tokens_in":21660,"feed_emoji":"🎬","tokens_out":12793,"duration_ms":107478,"temperature":0.7,"pith_summary":"This paper is trying to establish that concept-level unlearning in text-to-video generation can be done at inference time, with no parameter updates: given textual aliases of a target concept, SIRUS locates the concept inside a prompt and steers the diffusion trajectory away from it during sampling. The paper's core claim is that this localization-versus-suppression split — a text-embedding subspace built from aliases, a hierarchical trigger that marks target tokens, and a scheduled residual subtracted from the guidance output — removes safety, object, and style concepts more reliably than prior inference-time baselines while degrading video quality far less. It also argues that T2V unlearning must be judged at the video level, not on isolated frames, and provides an evaluation framework that reports forgetting, preservation, video quality, robustness, and efficiency separately. If the central claim is right, the practical consequence is that one frozen video generator can serve many forget policies by swapping an alias set and a few profile parameters, at the cost of roughly a quarter more generation time, and the recipe transfers to a second backbone.","feed_headline":"70.4% video-concept forgetting with zero retraining","feed_subtitle":"Targets like nudity, objects, and styles vanish from generated clips while scene and motion quality barely drop.","key_machinery":"The load-bearing object is the concept subspace: an orthonormal row-basis B built by encoding a small bank of templated alias prompts with the frozen text encoder, centering the alias token embeddings, and keeping their top singular directions. The same projection operator does two jobs — measuring per-token similarity for the trigger rule and removing the concept-aligned component from triggered tokens. The second mechanism is the concept residual d_t = ε_c − ε_t, the difference between the denoiser's output under a positive concept-reference condition and under the edited prompt condition, read as the direction that would restore the target, so a controlled subtraction steers generation aw","core_discovery":"The central discovery is that the hard part of T2V unlearning is not how strongly to intervene but where the concept lives in the prompt and what direction would restore it. SIRUS encodes templated alias phrases through the frozen text encoder and takes the singular directions of the centered alias embeddings to form an orthonormal concept subspace B. A hierarchical trigger — exact alias match, then subspace similarity, then a conservative fallback — marks which tokens carry target evidence, and only those tokens are projected away from B. At sampling time, a positive concept-reference condition is mixed from the most similar reference prompts, and the denoiser's prediction under it versus t","pith_inferences":["The machinery is almost entirely concept-generic: subspace localization plus residual subtraction would apply to other generative domains (image, audio, 3D) where a concept can be named by aliases; only the temporal schedule is video-specific, and it could be reinterpreted as a per-timestep intervention profile.","The paper does not vary the alias bank size, template phrasing, or profile parameters; the sharpest unstated test is how forgetting success degrades as aliases become sparser or more figurative, since the entire method rests on the subspace spanning the concept's visual expression.","The per-concept pattern — strong on compact objects like garbage truck, weak on global, persistent targets like parachute — suggests the optimal intervention profile differs by concept type, and a per-concept profile search is a natural extension the paper does not attempt.","Because suppression happens in text-embedding space, one testable hypothesis is that paraphrase and obfuscation jailbreaks that preserve semantic proximity to the alias subspace are deflected, while attacks that route around the embedding entirely may not be — a distinction the robustness study does not yet separate."],"forward_implications":["A single frozen video generator can enforce multiple forget policies by swapping the alias set and profile, without maintaining separately edited model copies; the measured cost is about a 24% increase in per-video generation time on the primary backbone, not offline retraining.","The video-oriented evaluation framework exposes a failure mode that frame-level checks miss: an aggressive baseline reaches high forgetting on nudity by erasing the person itself (person retention 33.4% versus 78.5% for SIRUS), so forgetting and preservation must be read jointly.","The component ablation attributes the largest share of forgetting to the sampling-side concept-reference branch — nudity success drops from 80.0% to 42.0% without it — implying prompt editing alone would not carry the method.","The same recipe transfers to a second, larger backbone with 73.6% average forgetting success, suggesting the approach is not tied to one model's text-embedding geometry.","The paper itself concedes a boundary: visually salient, temporally persistent targets such as parachute keep residual traces, so inference-time suppression is not uniform across concept types."],"fun_headline_variants":["Erase video concepts without retraining, quality intact","70% concept forgetting in videos, near-zero quality drop","Inference-time concept suppression for text-to-video models","Train-free video erasure: targets fade, scenes stay","SIRUS: prompt-driven video unlearning keeps quality"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The load-bearing premise is that the linear subspace built from a small alias-template bank (Section 3.1) reliably spans the target concept across arbitrary prompts, and that the residual d_t = ε_c − ε_t (Section 3.4) faithfully points at what would restore it, so subtracting a capped, momentum-smoothed version suppresses the concept without collateral damage — a premise tested on only five concepts, with profile hyperparameters never reported and the paper itself conceding (","fun_headline_variants_meta":{"raw":{"variants":["Erase video concepts without retraining, quality intact","70% concept forgetting in videos, near-zero quality drop","Inference-time concept suppression for text-to-video models","Train-free video erasure: targets fade, scenes stay","SIRUS: prompt-driven video unlearning keeps quality"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000196,"raw_usage":{"total_tokens":1224,"prompt_tokens":796,"completion_tokens":428,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":540,"completion_tokens_details":{"reasoning_tokens":348}},"tokens_in":540,"tokens_out":428,"duration_ms":4617,"temperature":1.0,"reasoning_tokens":348,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T03:03:41.574351+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a concept outside the five tested (say 'bicycle') and run SIRUS over a few hundred prompts on the same backbone, scoring video-level forgetting with the same classifier. If success collapses toward the no-intervention baseline, or removing the target also distorts non-target content (large LPIPS jump or object-recall drop), the subspace-faithfulness premise fails. Sharper: vary the alias bank size from one to about twenty on a fixed concept — monotone improvement with more aliases confirms the subspace carries the method, while flat or erratic success means something else does. Or disable","supporting_citations":[],"review_version":1}