{"id":"76380ad9-f8e1-4e88-aa3f-fc05bf7bc5d2","arxiv_id":"2505.10941","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"PALL learns tasks in a single network as sparse subnetworks and performs exact task unlearning by resetting the task's trained parameters and briefly retraining shared ones from a rehearsal buffer.","lead":"Privacy-aware lifelong learning (PALL) trains each task as a sparse subnetwork inside one neural network, and removes a task by resetting the parameters that were trained on it. This targets the right-to-be-forgotten: a deployed model can forget a task without retraining from scratch.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Exact-unlearning claim cannot hold as stated: later-task masks and private parameters are optimized using D_t-trained shared weights and are never resampled, so the U output is not distributionally equal to L(θ0, D_without_t) as Eq. (1) requires.","rationale":"The reader's weakest assumption already identifies the load-bearing issue: forward transfer lets later tasks inherit structure and weights influenced by D_t, so resetting the task-t subnetwork is not enough for the distributional guarantee in Eq. (1). My stress-test sharpens this from an unproven assertion to a structural mismatch: Algorithm 1 fixes later-task masks and private parameters that were optimized using D_t-trained shared weights, and the unlearning procedure never redraws them. Hence the post-unlearning model cannot be a fresh sample from L(θ0, D_without_t); the Exact entries in Table 1 and the by-design guarantee in the abstract and contributions are not supported. I keep the reader's CONDITIONAL verdict because the paper can be salvaged by explicitly reframing PALL as approximate or parameter-level unlearning, removing the Eq. (1) guarantee claim, and adding a rigorous privacy audit. The paper's own Section 4.2 caveat already concedes that chance-level Au is not evidence of exact unlearning, which strengthens the need for this revision. No change to the reader's verdict is therefore required, but the revision conditions should be stated as mandatory rather than optional.","tokens_in":24609,"tokens_out":13609,"duration_ms":149512,"concrete_test":"Run a three-task sequence (e.g., split MNIST or CIFAR-10) with PALL: learn D1, D2, D3, then unlearn D1. Independently run the same lifelong learning algorithm L from scratch on {D2, D3} with identical hyperparameters, seeds, and initialization distribution ϕ. Repeat both over at least 200 seeds. Compare the two model populations: (i) two-sample tests on the inference masks m2 and m3 (e.g., Jaccard overlap or connectivity statistics), and (ii) a held-out classifier or MMD two-sample test on the final (θ, masks) vectors. If a distinguisher exceeds chance accuracy, or mask distributions differ significantly, then U(θ, t) is not distributionally equal to L(θ0, D_without_t), falsifying the exact-unlearning-by-design claim. A complementary membership-inference audit treating D1 as the secret would settle whether post-unlearning models remain distinguishable from no-D1 retrained models.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that resetting θ⊙m_t and retraining shared parameters via Eq. (6) yields exact unlearning, i.e., U(θ_{i−1}, t) =_p L(θ0, D_{[τ∈Ω_i]}) in Eq. (1). This cannot hold for Algorithm 1 as stated. During any later task τ > t, the importance scores s_τ and hence the mask m_τ are optimized through Eq. (4)-(5) using frozen shared weights that were trained on D_t; therefore m_τ is a function of D_t, and later-task private parameters are likewise optimized conditional on those D_t-influenced weights. The unlearning branch (Algorithm 1 lines 18-30) resets only θ⊙m_t and retrains the affected shared subset for Nf iterations via Eq. (6). It does not resample m_τ, re-run score optimization, or retrain later-task private parameters from a no-D_t initialization. A fresh run of L on the remaining tasks would draw new random masks and new optimization trajectories; the post-unlearning PALL model instead has fixed masks and private weights from the D_t-including trajectory. Equality in distribution is therefore impossible unless later masks and private parameters are independent of D_t, which is exactly what forward knowledge transfer denies. Resetting all parameters that received D_t gradients is a necessary condition but not a sufficient one for Eq. (1), and no proof of sufficiency is provided. The paper's own Section 4.2 caveat that chance-level Au does not necessarily correspond to exact unlearning, and that a privacy audit is left for future work, directly undermines the Exact entries reported in Table 1.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"PALL is a task-incremental lifelong learning algorithm that trains task-specific sparse subnetworks in a single fixed-capacity network, freezes used parameters to prevent forgetting, permits forward transfer by sharing frozen parameters, and upon a task-unlearning request resets the subnetwork parameters optimized on that task and retrains shared parameters using an episodic buffer. The paper defines exact task unlearning through distributional equality with from-scratch training on the remaining tasks (Eq. 1), claims this guarantee by design, and reports experiments on sequential CIFAR and TinyImageNet with ResNet and ViT architectures, comparing with continual learning and unlearning baselines.","tokens_in":25047,"tokens_out":4386,"duration_ms":44558,"significance":"If the exact-unlearning guarantee were established, this would be a notable advance because it would combine privacy-aware unlearning with fixed-capacity lifelong learning, a combination that is largely absent from the existing literature. The paper's empirical strengths are substantial: 20-seed repetitions, multiple architectures, ablations on memory buffer size and retraining duration, worst-case metrics, training-time comparisons, and released code. The no-forgetting claim (Fl = 0) follows structurally from parameter freezing and is credible. However, the central exact-unlearning claim is not supported by the algorithm as written or by the empirical protocol; the paper's own caveat in Section 4.2 that chance-level Au does not necessarily correspond to exact unlearning, and that a privacy audit is left for future work, underlines this gap.","major_comments":[{"comment":"Eq. (1) requires U(theta_{i-1}, t) and L(theta0, D_{<=t} \\ D_t) to be equal in distribution. For any later task tau > t, the mask m_tau is obtained by optimizing scores s_tau (Eqs. 4-5) on frozen shared parameters inherited from the D_t-trained model, and the private parameters of tau are optimized from that same initialization. The unlearning branch resets theta ⊙ m_t and runs Nf iterations of Eq. (6), but it does not resample m_tau or retrain the later-task private parameters from a D_t-free starting point. Consequently, the output of U is not distributionally equal to a fresh run of L on the remaining tasks unless the later masks and private parameters are independent of D_t, which is exactly the proposition that forward transfer denies. No proof of this independence is supplied, so the 'exact unlearning guarantees by design' claim in the Abstract and Section 1 is not established.","section":"Section 3.3, Eq. (1), Algorithm 1 lines 18-30"},{"comment":"The empirical evaluation does not measure the distributional condition in Eq. (1). Table 1 lists 'Exact' in the Au column for PALL but reports no Au values, and Section 4.2 explicitly states that chance-level Au does not necessarily correspond to exact unlearning and that a privacy audit is left for future work. Since the exact-unlearning guarantee is the paper's load-bearing differentiator, the paper should either provide a concrete test of distributional closeness to the from-scratch retrained model or explicitly downgrade the claim to approximate unlearning and revise the abstract, title, and contribution statements accordingly.","section":"Section 4.2 and Table 1"},{"comment":"The statement 'We can facilitate this exactly by resetting the parameters theta_{i-1} ⊙ m_t' is circular as a justification of exactness: exactness is precisely the property to be proven, not a consequence of resetting. The surrounding argument also relies on two unstated assumptions: (i) all influence of D_t is confined to the parameters directly optimized on D_t, so resetting them removes the task's influence; and (ii) the Nf-step rehearsal in Eq. (6) on other tasks' memory buffers cannot reintroduce task-t information through shared or later-task parameters. These assumptions need to be stated explicitly and argued for, or the exactness claim must be weakened.","section":"Section 3.3, paragraph beginning 'Given a task unlearning request'"}],"minor_comments":[{"comment":"The sentence introducing the submask, 'We indicate the parameter indices which are specifically trained using data from Dt with the submask mt, and mt - mt correspond to the shared, frozen parameter indices from previous tasks', uses the same symbol for the full mask and the trained-only part; please introduce a distinct notation, for example n_t, for the trained-only submask.","section":"Section 3.3, notation"},{"comment":"In Eq. (6), the first expectation draws (x,y,z) from B_tau but only (x,y) is used; either write (x,y) for the first term or define the notation so that the logit z is used only in the second term.","section":"Eq. (6)"},{"comment":"The instruction 'mt ← Retrieve the submask of mt indicating the params trained via Dt or Bt' is confusing because both sides appear identical; please clarify which stored mask is being read and how the learned submask is identified relative to the full task mask.","section":"Algorithm 1, line 21"},{"comment":"The Au column for PALL says 'Exact' rather than reporting a number; please report the observed Au values so the reader can at least check the weak empirical signal that unlearned tasks yield chance-level accuracy.","section":"Table 1"},{"comment":"The paper cites the adaptive unlearning setting of Gupta et al. (2021) in related work but does not discuss how PALL behaves under adaptively chosen unlearning requests; a brief statement on this point would help position the contribution.","section":"Section 2.2 and Section 3.2"}],"recommendation":"major_revision","confidential_remarks":"The exact-unlearning guarantee is the main differentiator of the paper, and it is currently asserted rather than proven or empirically tested at the level required by Eq. (1). If the authors can add a formal argument (under explicit assumptions) or a concrete distributional/auditing test, the paper would be much stronger; otherwise it should be reframed as an approximate-unlearning method, which would change the claimed contribution substantially."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper delivers a genuinely useful engineering contribution: a single fixed-capacity network that combines WSN-style sparse subnetworks with DER++ rehearsal, so a task can be reset and shared weights repaired with a few retraining iterations. The empirical work is extensive and honest: 20 seeds, multiple architectures, ablations on Nf, beta, buffer size, worst-case metrics, and released code. The no-forgetting claim (Fl=0) is structurally guaranteed by parameter freezing, so that holds. The memory comparison against independent-task models is fair and the numbers support it.\n\nThe soft spot is the 'exact task unlearning guarantee.' The abstract and Section 1 assert exact unlearning by design, and Table 1 labels PALL as Exact. But the actual guarantee in Eq. (1) is distributional equality with a from-scratch run on the remaining tasks, and the paper never proves it. The stress-test note is on target: later-task masks m_tau are optimized using weights trained on D_t, and later-task private parameters are initialized from those D_t-influenced weights. Resetting theta⊙m_t and retraining the shared subset for Nf iterations does not resample masks or replay the optimization trajectory. So the post-unlearning model is not distributed like L(theta0, D_without_t). What PALL really provides is approximate unlearning with a clean parameter-reset principle. The paper's own Section 4.2 caveat, that chance-level Au does not imply exact unlearning and a privacy audit is left for future work, is an admission of exactly this gap.\n\nIs this fatal? For the privacy claim as worded, yes. The honest framing would be 'approximate task unlearning with a strong reset-and-rehearsal mechanism.' The practical value remains: a scalable baseline for continual unlearning that likely resists naive membership-inference probes, with trivial retraining cost. The citation pattern is fine; WSN, DER++, and CLPU are credited properly.\n\nWho is this for? Anyone benchmarking continual-learning methods under deletion requests. I would not cite it as an exact-unlearning method without a caveat, but I would cite it as a strong approximate method and a good empirical baseline. The paper deserves a serious referee—the problem is important and the experiments are sharp—but the exact-unlearning claim needs to be proved, weakened, or backed by a real privacy audit. If this were under review now, I would recommend major revision, not acceptance as-is.","headline":"PALL is a solid engineering contribution to continual learning with unlearning, but the 'exact' guarantee is asserted rather than proven, and the stress-test concern about later-task masks and initializations is on point.","tokens_in":25526,"tokens_out":3537,"would_cite":true,"duration_ms":34119,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PALL claims one network can learn tasks sequentially, then delete any task so exactly that it becomes indistinguishable from a model that never saw that task's data, by resetting the task's subnetwork and replaying the others' memories.","keywords":["privacy-aware lifelong learning","exact task unlearning","task-incremental learning","sparse subnetworks","knowledge transfer","catastrophic forgetting","machine unlearning","episodic memory rehearsal"],"falsifier":"Train PALL on task A then task B, unlearn A, and run a from-scratch model on B alone with the same algorithm; if any input-dependent test — a membership-inference classifier over A's data, a linear probe on intermediate activations, or a distributional distance between output logits — tells the unlearned model apart from the from-scratch model at better than chance, the exact-unlearning guarantee of Eq. (1) fails. A cheaper first check: compute whether the reinitialized-and-retrained parameters $\\theta \\odot m_A$ respond to gradients from $A$'s data more than fresh samples from $\\varphi(\\cdot)$ do, which would indicate residual information in the 'deleted' weights.","tokens_in":24421,"feed_emoji":"🧠","tokens_out":16476,"duration_ms":127950,"temperature":0.7,"pith_summary":"The paper claims that a single fixed-capacity neural network can do lifelong learning and machine unlearning at the same time: it acquires tasks one after another without forgetting, and on request it removes a task's influence so completely that the resulting model behaves as if that task's data had never been seen. Its method, PALL, gives each task a sparse subnetwork inside one architecture, selected by learned importance scores on top of a frozen set of previously trained parameters, so the task's information is confined to a known slice of the network. Unlearning resets that slice to its initial random values and briefly retrains the few shared parameters that later tasks borrowed, using stored exemplars and logits. The empirical case is that this yields exact unlearning by construction while matching or beating independent per-task models at a fraction of the memory cost. If the guarantee holds, PALL would give a practical route from right-to-be-forgotten regulations to deployed sequential learners without per-task model copies or full retraining.","feed_headline":"Exact unlearning on demand in one lifelong-learning network","feed_subtitle":"Delete a task's private subnetwork and the model behaves as if that data was never seen.","key_machinery":"The load-bearing object is the task-specific binary subnetwork mask $m_t \\in \\{0,1\\}^d$, built at a per-layer connectivity rate $\\alpha$ by keeping the largest $|s_t|$ entries of a learned importance-score vector $s_t$, trained by a straight-through estimator while parameter updates are masked by the cumulative cover $M_{i-1}$ of all previous tasks. This mask is what claims to confine a task's information to one identifiable slice of the network. The second mechanism is the reset-and-replay unlearning operator: reinitialize $\\theta \\odot m_t$ from $\\varphi(\\cdot)$ and retrain only the affected shared parameters — those in $m_t \\cap m_\\tau$ for later tasks $\\tau > t$ — for $N_f$ iterations on episodic buffers of exemplars and stored logits, following the rehearsal objective of Eq. (6) with $\\beta = 0.5$. The third is bookkeeping: the union mask $M_i = \\bigvee_{j \\in \\Omega_i} m_j$ plus the reinitialization of all unused parameters after each learning request are what the argument relies on to stop information from leaking outside the designated subnetwork.","core_discovery":"The central discovery, stated on the paper's own terms, is that exact task unlearning can be a design property of the learning algorithm rather than a post-hoc repair. Because each task's gradient updates are masked to touch only the parameters not yet used by earlier tasks, and because all unused parameters are reinitialized after every learning request, the paper argues that everything learned from task $t$ resides in the parameters $\\theta \\odot m_t$ selected by that task's binary mask. An unlearning request for $t$ therefore resets $\\theta \\odot m_t$ to fresh values from the initialization distribution $\\varphi(\\cdot)$, deletes the task's mask and memory buffer, and runs $N_f$ rehearsal iterations over the small intersection of $m_t$ with later tasks' masks, which are the only parameters whose reset could hurt surviving tasks. The result, the paper argues, is a model distributed as $L(\\theta_0, D_{[\\tau \\in \\Omega_i]})$ — what a learner that never saw $D_t$ would have produced — so an adversary cannot distinguish it from a from-scratch model. Empirically this yields chance-level accuracy on unlearned tasks, zero catastrophic forgetting ($F_l = 0$) across all settings, accuracy that matches or exceeds the independent-model baseline in the longest scenarios (86.11 versus 86.80 on 100-task TinyImageNet), and a 4.3$\\times$ to 12$\\times$ smaller inference model size than training separate models per task.","pith_inferences":["The exactness guarantee inherits the confinement assumption, and that assumption is testable: later tasks' masks and optimization paths start from weights trained on the forgotten task, so an adversarial audit — a classifier trying to tell the unlearned model apart from a from-scratch model trained only on the surviving tasks — would be a sharper check than the chance-level $A_u$ the paper reports","The same mask bookkeeping suggests a route to finer-grained deletion: if per-example parameter effects could be separated inside a subnetwork, sample-level right-to-be-forgotten requests would follow; as designed, two samples inside one task cannot be told apart, so PALL's granularity is inherently the whole task.","In open-ended task streams with unknown $T$, the fixed connectivity $\\alpha$ trades capacity against forgetting resolution; an adaptive $\\alpha$ that grows subnetworks when unlearning frees parameters is a direct, testable extension the paper leaves open.","The paper's check that retrained parameters differ from their pre-unlearning values (mean absolute differences around 0.002–0.02) shows the weights moved, but not that they carry no exploitable trace of $D_t$; probing the reset-then-retrained weights with $D_t$'s gradients or a linear probe would settle whether deletion is truly complete."],"forward_implications":["An unlearned task's test accuracy drops to chance immediately after the reset, giving a cheap operational signal (the paper's $A_u$ metric) that deletion happened; the paper stresses the real guarantee is by design, not by audit.","Because only 1–4% of the network's parameters need retraining per unlearning and a request completes in 1–3 seconds on the tested setups, exact unlearning becomes cheap enough to invoke repeatedly in adaptive sequences — the paper stresses up to 50 interleaved requests over 100 tasks without collapse.","Inference cost stays at one base model plus one binary mask per surviving task (about 0.69 MB added per task on the 100-task ViT experiment), where the independent-model alternative adds a full network per task and reaches roughly 2 GB.","As the connectivity rate $\\alpha$ shrinks, the method still functions with very small task subnetworks (54K parameters per task at 99% sparsity) and outperforms all independent-subnetwork variants in the 40- and 100-task comparisons, which is what makes the scheme usable when the number of tasks is large or unknown."],"supporting_citations":[{"why":"Supplies the winning-subnetwork mechanism of trainable importance scores with selective weight sharing that PALL adapts to build task-specific sparse subnetworks.","marker":"[Kang et al., 2022]"},{"why":"Provides the per-layer connectivity-rate masking rule (keep the largest |s_t| entries) used to construct each task's binary mask.","marker":"[Ramanujan et al., 2020]"},{"why":"Contributes the dark experience replay objective (Eq. 6 with beta = 0.5) that PALL uses to retrain reset shared parameters after unlearning.","marker":"[Buzzega et al., 2020]"},{"why":"Defines the experience replay baseline and the tiny episodic memory setup that PALL's buffer rehearsal builds on.","marker":"[Chaudhry et al., 2019]"},{"why":"Frames the task-incremental learning-with-unlearning setting and the independent per-task model baseline (CLPU) that PALL generalizes and compares against.","marker":"[Liu et al., 2022]"},{"why":"Anchors the paper's definition of exact unlearning as behavior equivalent to retraining without the deleted data.","marker":"[Bourtoule et al., 2021]"},{"why":"Provides the main inexact selective-forgetting baseline (LSF), whose above-chance Au results contrast with PALL's by-design exact unlearning.","marker":"[Shibata et al., 2021]"}],"fun_headline_variants":["Exact unlearning in one lifelong learning network","Privacy-aware lifelong learning with exact task forgetting","Sparse masks let a single model learn and unlearn exactly","Lifelong learning that also makes data truly forgettable","On-demand exact unlearning without extra models"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole guarantee rests on one assumption: everything a forgotten task taught the network stays inside its own subnetwork's parameters, so resetting those parameters and briefly retraining the few shared ones truly reproduces the model that would have existed without that task — and the paper offers no proof that later tasks, which were built on top of the forgotten task's weights, carry no lingering trace of it.","fun_headline_variants_meta":{"raw":{"variants":["Exact unlearning in one lifelong learning network","Privacy-aware lifelong learning with exact task forgetting","Sparse masks let a single model learn and unlearn exactly","Lifelong learning that also makes data truly forgettable","On-demand exact unlearning without extra models"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000746,"raw_usage":{"total_tokens":3370,"prompt_tokens":1037,"completion_tokens":2333,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":653,"completion_tokens_details":{"reasoning_tokens":2259}},"tokens_in":653,"tokens_out":2333,"duration_ms":15270,"temperature":1.0,"reasoning_tokens":2259,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T21:01:09.291194+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train PALL on task A then task B, unlearn A, and run a from-scratch model on B alone with the same algorithm; if any input-dependent test — a membership-inference classifier over A's data, a linear probe on intermediate activations, or a distributional distance between output logits — tells the unlearned model apart from the from-scratch model at better than chance, the exact-unlearning guarantee of Eq. (1) fails. A cheaper first check: compute whether the reinitialized-and-retrained parameters $\\theta \\odot m_A$ respond to gradients from $A$'s data more than fresh samples from $\\varphi(\\cdot)$ do, which would indicate residual information in the 'deleted' weights.","supporting_citations":[{"cited_title":"Machine unlearning","cited_arxiv_id":null,"evidence_quote":"Anchors the paper's definition of exact unlearning as behavior equivalent to retraining without the deleted data."}],"review_version":1}