{"id":"f674c42a-e236-49e1-8f6f-b581fc072b83","arxiv_id":"2607.28829","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"In self-improving federated agent networks, one-time unlearning fails because forgotten data shapes later retained trajectories; MUTE traces, contains, and audits this influence echo to keep deletion valid.","lead":"This paper shows that deleting a user's data from a self-improving network of learning agents fails if the network keeps training on new data collected under the old behavior — the deleted data's influence echoes back later. It proposes MUTE, a method that traces this influence, quarantines risky data, and audits for leakage, cutting communication vs. full retraining.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"MUTE's influence tracing is unvalidated: Eqs. (4)-(5) are the sole basis for containment and scheduling, yet η is both called an upper bound and tuned to 0.6, so the score may miss the very carriers it must catch.","rationale":"The reader's weakest assumption is exactly the influence propagation model, and I agree that it is the most load-bearing spot. The central contribution is not merely Observation 1 (echo exists), but MUTE as a reliable deletion mechanism. Every part of MUTE that goes beyond model-side NPO/projection depends on γ(x): tracing, containment thresholds, audit scores, and the ρ(M_∞)<1 scheduling condition. If Eq. (5) is wrong, the method's claimed advantage over retraining can be attributed to generic unlearning updates, not to the provenance/containment design. The paper's internal inconsistency about η is a strong signal that the model is not principled: an upper bound is not something you fit. The proposed test is feasible because the authors already compute the counterfactual network θ* in simulation, so ground-truth influence can be obtained without new infrastructure. If the test validates γ(x), the concern is resolved; if it fails, the paper's central mechanism is unsupported. Until then, the reader's CONDITIONAL verdict is the right call, and no verdict change is needed.","tokens_in":14851,"tokens_out":8944,"duration_ms":108084,"concrete_test":"In the existing LIBERO simulation, compute ground-truth per-trajectory influence for a held-out set of retained trajectories by running both the forget-shaped policy and the counterfactual policy θ* (already mentioned as an internal reference) in the same environments and measuring each trajectory's change in acceptance probability or action likelihood attributable to D_f. Then rank-correlate this ground-truth influence with Eq. (5)'s γ(x) and check whether the top-γ quarantined set contains the ground-truth top carriers (e.g., Spearman ρ ≥ 0.7 and recall@k ≥ 0.8). Also re-run the main experiments with η set to 1.0; if IRR or BLI breaches the C1/C4 targets at η=1.0, the claimed upper-bound property fails and MUTE's tracing is not reliable.","verdict_should_be":"UNCHANGED","load_bearing_attack":"MUTE's data-side containment and audit scheduling all rest on the influence score γ(x) from Eq. (5), which is fed by the scalar recurrence Eq. (4). If that recurrence is miscalibrated, the quarantine thresholds τ_q/τ_d in Section IV-B are arbitrary, the scheduler's stability condition ρ(M_∞)<1 in Section IV-C is a restatement rather than a guarantee, and the low IRR/BLI numbers in Table II may come from NPO/projection alone rather than from the proposed provenance/containment mechanism. Nothing in the paper validates Eqs. (4)-(5) against a ground-truth influence measure. A concrete symptom is the contradiction: Section IV-A says 'The factor η is an upper bound rather than a fitted constant,' but Section V-B and Fig. 5 describe η as a calibrated constant that 'works best at its fitted value ... near η=0.6.' An upper bound should be set conservatively high (η=1), not tuned downward; tuning to 0.6 means trajectories collected under a heavily-shaped policy can be systematically under-scored, so the highest-influence retained data may escape containment. Without a validation of γ(x), the paper's central mechanism is ungrounded, regardless of how clean the end-to-end results look.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies deletion requests in federated agent networks that continue self-improving after deployment. It reports three observations: one-time retraining on retained data does not silence the influence of forget data, and the forgotten behavior is gradually revived during continued operation (Observation 1, Figs. 2); the regeneration scales with the fraction of forget-shaped retained data (Observation 2, Fig. 3a); and the echo is traceable from server-side collection/aggregation records (Observation 3, Fig. 3b). To address this, the paper proposes MUTE, which computes an influence score from a server ledger (Eqs. 4-5), performs model-side erasure with an NPO-based forget-retain update plus gradient projection, contains high-influence retained trajectories via quarantine/down-weighting (Section IV-B), and schedules future erasures under an uplink budget using a spectral-radius stability condition (Section IV-C). Evaluation on LIBERO with MiniVLA and π0 backbones, three deletion granularities, and a physical Jetson-based testbed is reported, showing lower IRR and BLI and much lower communication cost than full retraining (Table II).","tokens_in":15205,"tokens_out":6107,"duration_ms":69122,"significance":"If the claims hold, the paper identifies a genuine and understudied failure mode: in self-improving federated agent networks, a one-time unlearning update can be undone by policy-driven data collection, so deletion must be sustained over time. The problem formulation (Eq. 3) is useful, and the idea of tracing influence from a lightweight server ledger while keeping raw trajectories local is attractive. The strengths are the clear identification of the echo phenomenon, the use of two VLA backbones, and the attempt to validate on a physical edge testbed. However, the current evidence is not yet sufficient: the experiments are single-run point estimates, the influence propagation model is unvalidated and internally inconsistent about η, the stability condition is asserted rather than proved, and the IRR metric is partially circular with MUTE's own containment mechanism. These issues bear directly on the central claim that MUTE provides reliable deletion, so they must be addressed before the paper can be accepted.","major_comments":[{"comment":"Section V-A states 'We run the whole network once for each backbone,' and Table II reports a single number per cell with no error bars, seeds, or significance tests. All central comparisons (MUTE vs Retrain for IRR, BLI, Comm) are point estimates from one trajectory of a stochastic process (heterogeneous data splits, policy rollouts, training). The empirical claim that MUTE outperforms Retrain is not statistically supported. Please report multiple independent runs with mean and variance, and where possible a paired significance test across the same data/network initializations.","section":"Section V-A, Table II, Fig. 5"},{"comment":"The influence score γ(x)=η γ[t_x] is the backbone of MUTE's tracing, containment thresholds (τ_d, τ_q), and scheduling, but it is never validated against any ground-truth influence measure (e.g., counterfactual retraining or a per-sample attribution method). The text in Section IV-A says 'η is an upper bound rather than a fitted constant,' yet Section V-B and Fig. 5 describe η as a calibrated constant that 'works best at its fitted value ... near η=0.6,' and Table I sets η=0.6. An upper bound should be set conservatively high (η=1); tuning it down means trajectories collected under a heavily shaped policy may be systematically under-scored, so the very carriers that must be contained can escape. Equation (4) is also a heuristic scalar recurrence with no derivation from the actual aggregation and collection process. Without validation of γ(x), the low IRR/BLI numbers in Table II could be","section":"Section IV-A, Eqs. (4)-(5); Section V-B, Fig. 5"},{"comment":"The claim that sustained unlearning is feasible when ρ(M_∞)<1 is an assertion, not a proof. The transition matrix M(a[t]) in Eq. (6) is never defined concretely, no construction of M_∞ is given, and no argument links the spectral-radius condition to constraints (C1)-(C4). The scheduler is described as 'selecting the lowest-communication schedule ... while keeping this stability condition,' but the reader cannot verify how the schedule is computed or why it satisfies the constraint. This is load-bearing for the paper's central guarantee of reliable deletion, and it needs either a concrete proof or an empirically verified construction with the actual aggregation, collection, containment, and erasure operations.","section":"Section IV-C, Eq. (6)"},{"comment":"IRR is measured by continuing self-improvement 'on high-influence retained data' to form θ_unl[K]. MUTE's data-side containment (Section IV-B) quarantines or down-weights exactly those high-influence retained trajectories. Thus MUTE's low IRR is partly by construction: the method removes from the training stream the very trajectories used to evaluate regeneration, while the Retrain baseline keeps them. The comparison is not a fair measure of whether the forgotten behavior would return under continued operation. Please define an evaluation protocol that is not directly manipulated by the containment step — for example, compute IRR on all retained data without applying quarantine/down-weighting during the evaluation rollout, or use a fixed data collection policy that is identical for MUTE and Retrain.","section":"Section V-C3, Eq. (9); Section IV-B"},{"comment":"The paper claims validation on a physical Jetson-based edge testbed, but no testbed-specific quantitative results are reported. The 'Overall Results' subsection refers back to Table II, which is a LIBERO simulation table; there is no table or text reporting actual uplink bytes, deletion response time, or behavioral recurrence measured on the Jetson hardware. The abstract's claim that the method is 'further check[ed] on a physical Jetson-based edge testbed' is unsupported as written. Please include the measured hardware results or remove the claim.","section":"Section VI"}],"minor_comments":[{"comment":"L_forget and L_retain in Eq. (7) are only named, not defined. Please give explicit equations for the NPO term and the retain term, including how θ_ref is used and how the gradient-projection subspace is computed and updated.","section":"Eq. (7)"},{"comment":"Table I lists τ_q but not τ_d, although τ_d is used in Section IV-B. Please add τ_d and explain how both thresholds are derived from the leakage target.","section":"Table I"},{"comment":"The constraints in the problem formulation are not checked against the experimental results: no values for τ_max, δ, ε, or ϱ are given, and Table II does not report whether C1-C4 are satisfied per round. Please state target values and show compliance.","section":"Eq. (3), Table II"},{"comment":"The held-out non-member set D_o is not defined in the experiments. Please specify how large it is, how it is constructed, and whether it is disjoint from both the forget set and the training data.","section":"Section V-C2, Eq. (8)"},{"comment":"The 'counterfactual network θ*' is said to be computed in simulation, but no results using θ* are presented anywhere. Please clarify its role or remove the reference.","section":"Section V-A"},{"comment":"In Section VI, the text says 'we measure the actual uplink bytes, the deletion response time, and the behavioral recurrence,' but these measurements are not reported. Please add this data or delete the sentence.","section":"Section VI"}],"recommendation":"major_revision","confidential_remarks":"The paper's reference list contains many self-citations and 2026-dated preprints. I did not assess novelty, but the editor may wish to verify that MUTE is not a re-branding of the authors' own prior unlearning methods (e.g., SCALE [16], EASE [26]). Also, the paper compares only against Retrain, not against any existing federated unlearning baseline; adding at least one standard FU method would substantially strengthen the empirical contribution."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's the short version: the paper identifies a real failure mode that I haven't seen addressed head-on in federated unlearning. When a self-improving agent network keeps collecting trajectories under a policy that was shaped by the forget data, the retained data itself becomes a carrier of the deleted influence. Retraining on that retained data doesn't fully remove the echo, and the behavior can come back as the network continues to operate. That observation is worth taking seriously, and it's the paper's main contribution.\n\nWhat the paper does well: it formulates a new problem—sustained deletion under policy-driven data collection—and proposes a concrete mechanism, MUTE, that combines ledger-based influence tracing, model-side erasure, and data-side containment. The ledger idea is sensible: estimate influence from server-side records without moving raw trajectories off clients. The authors also run experiments on two VLA backbones, three deletion granularities, and a physical Jetson testbed, which is more than most theory papers in this area.\n\nThe soft spots are where the evidence doesn't match the claims. The influence propagation model in Eqs. (4)-(5) is asserted rather than validated. There's no ground-truth influence measure to check the scalar recurrence against, so the quarantine thresholds and the scheduling stability condition ρ(M_∞)<1 lose their grounding. There's also an internal contradiction: Section IV-A says η is an upper bound, but Section V-B and Fig. 5 treat it as a fitted calibration constant that 'works best' near 0.6. An upper bound set to 0.6 can systematically under-score the very trajectories that need containment.\n\nIRR is partly circular. It measures regeneration by continuing self-improvement on high-influence retained data, which MUTE explicitly quarantines or down-weights. So the mechanism is improving a metric that depends on the same data it removes. The experiments are single-run with no error bars, no code or data release, and no comparison to existing federated unlearning baselines—only retraining and a no-deletion reference. And the citation list leans heavily on the authors' own prior work, which doesn't help the reader situate the novelty.\n\nWho should read this? Anyone working on federated unlearning, privacy in lifelong learning, or deletion in agent networks. The problem formulation and the echo observation are genuinely useful, and a serious referee could push the authors to turn this into a solid paper. My recommendation: send it to peer review with a request for major revision—multi-seed experiments, open-source code, a non-circular regeneration metric, validation of the influence model, and at least one or two existing FU baselines.","headline":"Real new problem in federated unlearning, but MUTE's evidence is weaker than the claims: the influence model is unvalidated, IRR is partly circular, and experiments are single-run.","tokens_in":15644,"tokens_out":3907,"would_cite":true,"duration_ms":41004,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"One-time retraining cannot erase data once an agent network keeps learning.","keywords":["federated unlearning","self-improving agent networks","influence echo","reliable data deletion","influence tracing","behavioral leakage","edge AI privacy","deletion scheduling"],"falsifier":"Run a leave-one-out counterfactual: train a network with and without a forget set, identify which retained trajectories actually revive the forgotten behavior after continued learning, and compare that ground truth to MUTE's influence scores; if a trajectory scored below τ_d still causes IRR to rise above the tolerance when learning resumes, the tracing model is falsified.","tokens_in":14712,"feed_emoji":"🔁","tokens_out":8700,"duration_ms":80863,"temperature":0.7,"pith_summary":"The paper studies data deletion in self-improving federated agent networks — systems where deployed models collect new trajectories and feed them back into later training rounds. It tries to establish that a one-time retraining on retained data does not settle a deletion request: the forgotten data has already shaped later retained trajectories, so the deleted behavior gradually returns as the network keeps operating. To make deletion durable, the paper proposes MUTE, which traces where the forgotten data's influence has flowed using a lightweight server ledger, removes the current model residue with a forget-retain update, contains high-influence retained trajectories by quarantine or down-weighting, and schedules additional erasure under an uplink budget. The claimed payoff is that behavioral leakage and influence regeneration stay near zero while task utility is preserved, at a fraction of the communication cost of full retraining.","feed_headline":"Retraining can't erase data once an agent network keeps learning","feed_subtitle":"The forgotten behavior returns as training continues; the paper's tracing-and-containment method keeps deletion valid.","key_machinery":"The central object is the 'influence echo' — the way forgotten data reshapes later retained trajectories through the deployed policy. It is quantified by a per-trajectory influence score γ(x) = η·γ[t_x], where the version influence γ[t] evolves by a recurrence over aggregation weights and client-side data influence (Eq. 4), and η calibrates how strongly the deployed policy shapes collection. This score lets the server estimate influence without moving raw trajectories. The same state-transition view (Eq. 6) is used to argue that sustained unlearning is feasible when the long-run spectral radius ρ(M_∞) < 1. The method combines a forget-retain update with negative preference optimization for m","core_discovery":"On its own terms, the paper claims that in a closed-loop federated agent network, the influence of forgotten data propagates through two edges: aggregation (from local data to the global model) and collection (from the deployed policy to new trajectories). This 'influence echo' survives retraining on the retain set, grows with the fraction of retained data collected under a forget-shaped policy, and can be traced from deployment, collection, and aggregation records. The proposed solution, MUTE, converts those records into a per-round influence score via a scalar recurrence, uses the score to suppress model-side residue and to quarantine or down-weight high-influence retained trajectories, an","pith_inferences":["The same echo mechanism should appear in any online learning loop where the model's own outputs shape future training data — recommendation systems, autonomous data collection, self-play — so the tracing-and-containment logic may transfer beyond federated agents.","A natural next test: calibrate η per client or per task from a small leave-one-out counterfactual study; if the scalar recurrence cannot rank trajectories correctly, the method may need finer-grained attribution to meet strong privacy guarantees.","The paper's framing implies that regulators and auditors should treat 'deletion' in self-improving systems as a continuous constraint verified over time, not a one-time action verifiable at the moment of the request.","The sensitivity sweep hints that under strong collection shaping (large η or small data heterogeneity α), the scheduler may need to pause collection entirely to keep ρ(M_∞)<1; the paper does not explore that regime in depth."],"forward_implications":["If the influence-echo model is correct, any federated unlearning method that fixes the retain set at request time is insufficient for self-improving networks; deletion guarantees must be re-checked and re-enforced as learning continues.","MUTE's ledger-based tracing shows that raw trajectories can stay on clients while the server still estimates where influence has flowed, at a replay cost of O(T N) for the scalar sequence.","The containment step (quarantine/down-weighting) offers a reversible middle ground: quarantined trajectories remain on the client and can be released once their influence score falls below the threshold.","The stability condition ρ(M_∞)<1 recasts sustained deletion as a scheduling problem: pick the lowest-communication action sequence that makes influence decay, subject to response deadline and utility floor.","The reported numbers — IRR well below retraining, BLI near chance, SR within 0.04 of retraining, and roughly 3-5x lower uplink — suggest the approach is affordable even on edge-like hardware."],"fun_headline_variants":["Influence echo survives retraining in agent networks","Unlearning fails when agent networks keep learning","Agent networks: retraining can't delete forgotten data","MUTE stops forgotten data's echo in agent networks"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"Everything hinges on the scalar influence recurrence (Eqs. 4-5) accurately capturing how forgotten data propagates to later retained trajectories, yet the paper provides no derivation or ground-truth validation and treats η both as an upper bound and as a fitted constant.","fun_headline_variants_meta":{"raw":{"variants":["Influence echo survives retraining in agent networks","Unlearning fails when agent networks keep learning","Agent networks: retraining can't delete forgotten data","MUTE stops forgotten data's echo in agent networks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001078,"raw_usage":{"total_tokens":4348,"prompt_tokens":745,"completion_tokens":3603,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":489,"completion_tokens_details":{"reasoning_tokens":3552}},"tokens_in":489,"tokens_out":3603,"duration_ms":27454,"temperature":1.0,"reasoning_tokens":3552,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-03T01:32:18.526258+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a leave-one-out counterfactual: train a network with and without a forget set, identify which retained trajectories actually revive the forgotten behavior after continued learning, and compare that ground truth to MUTE's influence scores; if a trajectory scored below τ_d still causes IRR to rise above the tolerance when learning resumes, the tracing model is falsified.","supporting_citations":[],"review_version":1}