Pith. sign in

REVIEW 3 major objections 6 minor 34 references

Trajectory-level unlearning in offline RL cannot be judged by a single membership score.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 08:25 UTC pith:EEFJJJQP

load-bearing objection TOUR is a careful, well-structured unlearning-evaluation benchmark, but its sharpest example—TrajDeleter's Ref AUC 0.853—leans on a non-member matching control that the paper itself shows is fragile. the 3 major comments →

arxiv 2607.21111 v1 pith:EEFJJJQP submitted 2026-07-23 cs.LG cs.AI

TOUR: A Trajectory-Level Unlearning Benchmark for Offline Reinforcement Learning

Chaofan Pan , Lingfei Ren , Xiangyu Jiang , Yanhua Li , Xuemei Cao , Xiangkun Wang , Hao Yu , Wei Wei
show 1 more author
Xin Yang
This is my paper
classification cs.LG cs.AI
keywords offline reinforcement learningmachine unlearningtrajectory-level deletionmembership inferenceprivacy auditingmatched non-membersretraining referencebenchmark
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

TOUR is a benchmark for evaluating whether an offline reinforcement learning agent has truly forgotten selected trajectories. The paper argues that a single likelihood-based membership score is not enough: a lower score can mean real deletion, residual memorization visible to another attack, or policy collapse that destroys useful behavior. To separate these, TOUR combines matched non-member controls, retraining references, retained-performance anchors, and multiple attack families. Its central finding is that common deletion baselines can appear private under one score while leaking under another, so deletion quality should be reported as a privacy-utility profile rather than a single AUC.

Core claim

TOUR demonstrates that trajectory-level deletion quality in offline RL is environment- and attack-dependent, and that no single membership score can certify erasure. The sharpest instance is TrajDeleter in Walker2D medium-expert: the primary per-timestep NLL AUC is 0.512, a forget gap of only 0.012 and practically random, yet the reference-model attack gives a Ref AUC of 0.853, a large residual signal relative to retraining. Retraining and naive fine-tuning often preserve utility while moving the forget-set AUC close to random guessing, whereas uniform gradient ascent plus refitting frequently loses utility without a reliable privacy gain. The paper concludes that unlearning evidence must in

What carries the argument

The matched non-member set: trajectories from a disjoint holdout pool are paired to forget trajectories via k-d tree retrieval on z-normalized episode return, length, and initial-state features, then used as the privacy baseline for all forget-versus-nonmember AUC measurements. Around this, the multi-attack audit adds a reference-model attack (the score is NLL under the retrained reference minus NLL under the target), a threshold attack, an NLL-deviation attack, and TOST equivalence testing, with retained D4RL normalized score as the utility anchor.

Load-bearing premise

The matched non-member set is assumed to be a valid privacy baseline: residual differences between forget trajectories and matched non-members, especially in initial states, could drive the reported AUC ordering independently of memorization.

What would settle it

Take the TrajDeleter Walker2D medium-expert checkpoint and replace the forget set with a random sample of non-member trajectories of equal length and return; if the reference-model attack still yields a Ref AUC near 0.853, the attack is detecting covariate shift rather than memorization.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • If TOUR is right, future offline RL unlearning papers should report a privacy-utility profile, not a single membership AUC.
  • Single-score auditing can overstate deletion: a near-random primary NLL AUC can coexist with a high reference-model AUC, as seen for TrajDeleter in Walker2D medium-expert.
  • Retraining and fine-tuning provide stronger retained-utility references than uniform GA+Refit in several settings, so those baselines should be the calibration point for deletion claims.
  • Policy collapse can masquerade as erasure: forget-set AUC values below 0.5 indicate an inverted separable ordering, not successful forgetting.
  • Conclusions about which deletion method is best are not stable across environments or attack families; method rankings depend on matched non-member construction and retained utility.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The reference-model attack could be adopted as a standard calibration check in other sequential unlearning settings, since it exposes residual signal that a single likelihood score misses.
  • The residual initial-state imbalance after matching (coordinate-level SMD up to 0.365) suggests that a weaker matching feature set may inflate privacy conclusions; richer matching features would be a direct testable extension.
  • The benchmark's environment dependence implies that a deletion method validated on one MuJoCo task should not be assumed to transfer without re-auditing on the target task and data variant.
  • TOUR's protocol could be applied to value-based offline RL agents beyond IQL by replacing the NLL audit with action-error or representation-based attacks, though score equivalence across families remains open.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces TOUR, a benchmark for evaluating trajectory-level deletion/unlearning in offline reinforcement learning. It partitions D4RL locomotion datasets into forget, retain, and matched non-member sets; trains Decision Transformer, LSTM, MLP, and IQL policies under a shared configuration; and compares deletion baselines (retraining, naive fine-tuning, uniform GA+Refit, and the external TrajDeleter). Privacy is measured primarily by a matched likelihood-based membership AUC, complemented by threshold, reference-model, deviation, TOST, action-error, representation-based, and query-limited shadow-model attacks, together with retained-utility anchors. The paper's central empirical claim is that a single likelihood-based membership score can overstate deletion quality: most sharply, TrajDeleter on Walker2D medium-expert has a near-random NLL AUC of 0.512 but a reference-model AUC of 0.853. The paper also reports architecture-associated and component-level sensitivity, explicitly scoping these as diagnostic rather than causal.

Significance. If the findings are robust, TOUR is a valuable contribution to offline RL unlearning evaluation. Its strengths are real: it uses matched non-member controls and reports balance diagnostics; it separates evidence into required, calibrated, and exploratory tiers; it includes matching-robustness variants (Table 18), TOST sensitivity at ε=0.05, and explicit CIs in several tables; and it repeatedly disclaims overreach in architecture and component-level conclusions. The central caveat is that the headline reference-model result — the claim that TrajDeleter leaves residual memorization even when the likelihood score is near random — has not been shown to be robust to the construction of the matched non-member set. Because the paper's main conclusion depends on this example, the contribution is conditionally significant: the protocol is useful, but the sharpest evidence needs an additional control before the load-bearing claim is established.

major comments (3)
  1. [§3.2, §3.4, Table 2, Tables 17–18] The load-bearing evidence for the claim that a single likelihood score overstates deletion quality is the TrajDeleter/Walker2D-ME row in Table 2: NLL AUC 0.512 (gap 0.012) versus Ref AUC 0.853. The reference-model score is NLL(τ;θ_ref) − NLL(τ;θ′); an AUC above 0.5 can arise not only from residual memorization by θ′ but also from residual covariate imbalance between D_f and D_n if θ_ref fits one group better than the other. The paper's own Table 17 reports a post-match maximum coordinate-level state-0 SMD of 0.237 for Walker2D ME, and Table 18 shows that a stronger feature-matched non-member construction materially changes primary forget gaps in Walker2D (R) (e.g., base DT 0.125→0.065; naive FT 0.028→0.005). Ref AUC is not reported under that stronger control. I request that the authors report Ref AUC under the stronger matched D_n for the Table 2 settings, and additionally calibrate the
  2. [§4.1, Table 1, Table 17] The aggregate benchmark comparisons rest on only three seeds, and the paper's own detectable-gap diagnostics in Table 17 exceed several of the observed differences. For example, Walker2D ME has an approximate detectable gap of 0.093, while many reported forget gaps in Table 1 are ≤0.02; HalfCheetah ME has a detectable gap of 0.097. This does not invalidate the large Ref AUC result, but it does undermine finer comparative claims such as “TrajDeleter is not uniformly stronger” if those claims are based on small NLL-gap differences. Please state explicitly which conclusions survive the detectable-gap threshold and avoid reporting three-seed 95% CIs that appear implausibly narrow (e.g., [0.484, 0.487] for Retrain Ref. in Walker2D ME) without clarifying the resampling unit.
  3. [§4.3, Table 3, Appendix B.2] The architecture comparison is explicitly scoped as descriptive rather than causal, and the paper is admirably careful about this. However, Table 3 presents large utility differences across DT, MLP, and LSTM (e.g., Hopper utility 22.35 vs. 15.28 vs. 1.70) alongside the forget-gap differences. Given the acknowledged capacity, optimization, and utility confounds, the architecture section has limited inferential value beyond showing that audit conclusions are not architecture-independent. Since the paper already states this, the issue is mainly one of emphasis: the abstract and introduction should not let readers infer a stronger architecture finding than the evidence supports. A brief sentence in the main text stating that the architecture rows are stress tests of the audit, not measurements of architecture-driven memorization, would avoid overreading.
minor comments (6)
  1. [Table 1 caption] The caption says the 95% intervals summarize seed-level uncertainty, but intervals such as 0.484–0.487 for a three-seed aggregate are implausibly narrow unless the resampling unit is per-trajectory. Clarify the resampling unit and, if the intervals are trajectory-level, label them as such and explain why they differ from the seed-aware intervals in Table 17.
  2. [Table 2] No CIs or bootstrap intervals are reported for the four attack AUCs. Given the small paired counts in some settings, the reader cannot assess whether the Ref AUC 0.853 is statistically distinguishable from, say, 0.5 or from a confounded baseline. Add intervals or at least state the number of paired trajectories used for each cell.
  3. [§3.4 and Table 2] The two columns “TOST” and “All<ε” can disagree, and the text explains this, but a one-line example would help readers: e.g., TrajDeleter/Walker2D-ME passes TOST at ε=0.1 yet fails All<ε because of Ref AUC 0.853. Currently the relation between statistical equivalence and threshold condition is easy to misread as contradictory.
  4. [Appendix D.1, Table 17] For HalfCheetah ME, the post-match maximum coordinate-level state-0 SMD (0.207) is larger than the pre-match value (0.181). Explain why matching can increase a coordinate-level imbalance while reducing the aggregate state-0 norm SMD, or at least acknowledge this explicitly in the matching diagnostic discussion.
  5. [Figure 5] The axis labels in the provided manuscript text appear corrupted (strings such as “/uni0000002b/uni00000044...”). Please ensure the final PDF renders the labels correctly.
  6. [Reproducibility] For a benchmark paper, a public code/data-release link or a clear statement of where the benchmark artifacts will be hosted would be important. The text refers to “benchmark artifacts” but does not provide a URL or repository identifier.

Circularity Check

0 steps flagged

No significant circularity: TOUR's claims are empirical measurements with disclosed assumptions, not predictions derived from fitted inputs.

full rationale

TOUR is an evaluation protocol, not a derivation chain. Its central quantities—forget AUC, reference-model AUC, TOST results, and utility scores—are operationally defined measurements computed from trained policies, the retain set, and the matched non-member set; none is fitted to the conclusion it is used to support. The reference-model attack uses the retraining reference as an external calibration floor, and the paper explicitly disclaims it: 'The retraining reference is an operational floor rather than a certified zero-influence oracle' (§3.4), with the Retrain Ref. row's Ref AUC undefined by construction. The matched non-member construction is an input assumption, not an output: the paper states that matching 'does not eliminate all residual confounds' and reports both balance diagnostics (Table 17) and a stronger-matching robustness check (Table 18). The only self-citation ([19]) appears in a related-work list of representative unlearning approaches and is not load-bearing; no uniqueness theorem or ansatz is imported from it. Architecture and component-level conclusions are explicitly scoped as descriptive diagnostics rather than causal mechanisms (Sections 4.3, 4.4, 5). Residual covariate imbalance in the matched non-member set is a validity threat, not an instance of circularity, because the paper does not define its conclusions in terms of the matching outcome. No step can be exhibited in which a predicted result is equal by construction to its inputs.

Axiom & Free-Parameter Ledger

7 free parameters · 6 axioms · 0 invented entities

TOUR introduces no new physical or mathematical entities; it is an evaluation protocol. The free parameters are hand-chosen configuration values (λ, ε, budgets, K, matching design, baseline hyperparameters), not fits used to manufacture results. The load-bearing assumptions are the validity of NLL as membership signal for DTs, the adequacy of matched non-members as a confound control, and the retraining reference as a privacy floor — all explicitly flagged by the authors as operational rather than certified.

free parameters (7)
  • GA+Refit ascent strength λ = 1.0
    Hand-chosen default in Eq. (2) that sets the deletion-vs-behavior-preservation tradeoff of a central baseline; not fitted, but results (e.g., utility collapse in HalfCheetah ME to 0.00) depend on it.
  • GA+Refit ascent budget / refit steps / ascent LR / clip = 500 steps, 10K steps, 1e-4, 0.25
    Hand-chosen (§3.5); the privacy-utility profile of GA+Refit is sensitive to these.
  • TOST equivalence margin ε = 0.1 (sensitivity at 0.05)
    Hand-chosen threshold that defines the All<ε pass/fail column in Table 2; the paper reports sensitivity at ε=0.05 in Table 19.
  • TrajDeleter configuration α, β, stage steps = α=1.0, β=2.0, 100/1,000 steps
    Hand-chosen external baseline configuration (§3.5); benchmark conclusions about TrajDeleter are contingent on this configuration.
  • Shared training configuration K, steps, batch, LR, dropout, hidden = K=20, 100K steps, batch 64, LR 1e-4, dropout 0.1, hidden 128
    Hand-chosen shared budget; membership signals are configuration-sensitive (context-length probe in Table 8 shows utility swings at fixed AUC).
  • Matching design (k-d tree on z-normalized features, |Dn|=|Df|) = z-norm return, length, init-state; 1-to-1 pairs
    Design choice governing every reported forget gap; robustness variants (Table 18) show gaps shift by up to ±0.06 under stronger/unmatched matching.
  • IQL hyperparameters = expectile 0.7, temperature 3.0, 20K steps, batch 256
    Hand-chosen for the companion IQL analysis (Appendix B.3); affects the action-error audit values.
axioms (6)
  • domain assumption Mean per-timestep NLL under a Gaussian action head is a valid trajectory-level membership signal for DT policies (Eq. 1).
    The primary audit and the forget-gap headline rest on this; the paper itself demonstrates its failure mode (Walker2D ME base DT NLL gap 0.015 vs reference-model gap 0.361).
  • domain assumption The matched non-member set removes the dominant trajectory-level confounds, so residual AUC measures memorization rather than covariate shift.
    Every privacy number is measured against Dn; the paper's Appendix D.1 shows residual s0 mismatch up to 0.365 absolute SMD and concedes 'does not eliminate all residual confounds.'
  • domain assumption The retraining reference is an appropriate privacy floor for calibration.
    Used as the reference in the reference-model attack and the retained-utility anchors; the paper calls it 'an operational floor rather than a certified zero-influence oracle' (§3.4).
  • domain assumption D4RL normalized score (locomotion) and success rate (AntMaze) are adequate utility metrics.
    Utility anchors and the utility-collapse interpretation of baselines depend on these scales; the paper notes AntMaze success rate is not comparable to locomotion returns.
  • domain assumption A 10% uniform-random trajectory split is a representative deletion scenario.
    Defined in §3.2; all forget/retain partitions inherit this choice.
  • standard math Standard statistical machinery (hierarchical bootstrap, paired bootstrap, TOST at α=0.05) is valid for the reported intervals and tests.
    Used for CIs, detectable-gap estimates, and the TOST columns; no novel statistical claims.

pith-pipeline@v1.3.0-alltime-deepseek · 28222 in / 14810 out tokens · 136489 ms · 2026-08-01T08:25:17.746380+00:00 · methodology

0 comments
read the original abstract

Offline Reinforcement Learning (RL) agents are trained on fixed behavioral trajectories, which makes trajectory-level deletion important when selected data must be removed after training. Evaluating such deletion is difficult because a lower membership score can reflect trajectory removal, residual memorization visible to another attack, or policy collapse that destroys useful behavior. We introduce Trajectory-level memOrization and Unlearning in offline RL (TOUR), a benchmark that combines trajectory-level partitioning, matched non-member controls, retraining references, retained-performance anchors, and multi-attack privacy auditing. Across D4RL locomotion experiments and an exploratory AntMaze extension, TOUR shows that common deletion baselines have environment-dependent privacy-utility behavior. Retraining and fine-tuning often provide stronger retained-utility references than uniform GA+Refit, while TrajDeleter remains a useful comparator but is not uniformly stronger under the same audit. Reference-model, threshold, deviation, equivalence, action-error, representation-based, and query-limited attacks further show that a single likelihood-based membership score can overstate deletion quality. In the evaluated settings, conclusions about offline RL unlearning are therefore not stable under single-score auditing. They depend on matched non-member construction, retraining-relative calibration, attack family, retained utility, and explicit scope for diagnostic architecture or component-level evidence.

Figures

Figures reproduced from arXiv: 2607.21111 by Chaofan Pan, Hao Yu, Lingfei Ren, Wei Wei, Xiangkun Wang, Xiangyu Jiang, Xin Yang, Xuemei Cao, Yanhua Li.

Figure 1
Figure 1. Figure 1: Overview of TOUR. The benchmark partitions trajectories into forget, retain, and matched [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Mean forget gap by environment and component-level GA target. Component-level up￾dates expose environment-dependent sensitivity, while smaller relative gaps do not necessarily im￾ply deletion with retained performance. The component-level sweep reveals substantial heterogeneity across environments. In the replay variants analysis, some attention-layer targets re￾duce the forget gap relative to matched unif… view at source ↗
Figure 3
Figure 3. Figure 3: Architecture comparison for the replay variants under the shared training configuration. [PITH_FULL_IMAGE:figures/full_fig_p013_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Hopper replay comparison between component-level and uniform GA across ascent [PITH_FULL_IMAGE:figures/full_fig_p019_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Privacy-utility tradeoff across GA targets and step counts. Component-level updates can [PITH_FULL_IMAGE:figures/full_fig_p020_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: TMI AUC across trajectory return quartiles for HalfCheetah, Hopper, and Walker2D. [PITH_FULL_IMAGE:figures/full_fig_p022_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Cross-environment quartile decay trajectories under naive fine-tuning. Each panel shows [PITH_FULL_IMAGE:figures/full_fig_p023_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Privacy-utility scatter for the diagonal Fisher baseline across seeds [PITH_FULL_IMAGE:figures/full_fig_p027_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

34 extracted references · 5 linked inside Pith

  1. [1]

    A survey on offline reinforcement learning: Taxonomy, review, and open problems.IEEE Transactions on Neural Networks and Learning Systems, 35(8):10237–10257, 2023

    Rafael Figueiredo Prudencio, Marcos ROA Maximo, and Esther Luna Colombini. A survey on offline reinforcement learning: Taxonomy, review, and open problems.IEEE Transactions on Neural Networks and Learning Systems, 35(8):10237–10257, 2023

    2023

  2. [2]

    On the op- portunities and challenges of offline reinforcement learning for recommender systems.ACM Transactions on Information Systems, 42(6):1–26, 2024

    Xiaocong Chen, Siyu Wang, Julian McAuley, Dietmar Jannach, and Lina Yao. On the op- portunities and challenges of offline reinforcement learning for recommender systems.ACM Transactions on Information Systems, 42(6):1–26, 2024

    2024

  3. [3]

    Learning from good trajectories in offline multi-agent reinforcement learning

    Qi Tian, Kun Kuang, Furui Liu, and Baoxiang Wang. Learning from good trajectories in offline multi-agent reinforcement learning. InAAAI, volume 37, pages 11672–11680, 2023

    2023

  4. [4]

    Offline reinforcement learning: Role of state aggregation and trajectory data

    Zeyu Jia, Alexander Rakhlin, Ayush Sekhari, and Chen-Yu Wei. Offline reinforcement learning: Role of state aggregation and trajectory data. InCOLT, pages 2644–2719, 2024

    2024

  5. [5]

    A survey of privacy attacks in machine learning.ACM Computing Surveys, 56(4):1–34, 2023

    Maria Rigaki and Sebastian Garcia. A survey of privacy attacks in machine learning.ACM Computing Surveys, 56(4):1–34, 2023

    2023

  6. [6]

    The right to be forgotten in federated learning: An efficient realization with rapid retraining

    Yi Liu, Lei Xu, Xingliang Yuan, Cong Wang, and Bo Li. The right to be forgotten in federated learning: An efficient realization with rapid retraining. InINFOCOM, pages 1749–1758, 2022

    2022

  7. [7]

    A comprehensive survey of forgetting in deep learning beyond continual learning.IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(3):1464–1483, 2024

    Zhenyi Wang, Enneng Yang, Li Shen, and Heng Huang. A comprehensive survey of forgetting in deep learning beyond continual learning.IEEE Transactions on Pattern Analysis and Machine Intelligence, 47(3):1464–1483, 2024

    2024

  8. [8]

    Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot

    Lucas Bourtoule, Varun Chandrasekaran, Christopher A. Choquette-Choo, Hengrui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. Machine unlearning. InIEEE S&P, pages 141–159, 2021

    2021

  9. [9]

    Hannun, and Laurens van der Maaten

    Chuan Guo, Tom Goldstein, Awni Y . Hannun, and Laurens van der Maaten. Certified data removal from machine learning models. InICML, pages 3832–3842. PMLR, 2020

    2020

  10. [10]

    A survey of machine unlearning.ACM Transactions on Intelligent Systems and Technology, 16(5):1–46, 2025

    Thanh Tam Nguyen, Thanh Trung Huynh, Zhao Ren, Phi Le Nguyen, Alan Wee-Chung Liew, Hongzhi Yin, and Quoc Viet Hung Nguyen. A survey of machine unlearning.ACM Transactions on Intelligent Systems and Technology, 16(5):1–46, 2025

    2025

  11. [11]

    D4RL: datasets for deep data-driven reinforcement learning.arXiv, abs/2004.07219, 2020

    Justin Fu, Aviral Kumar, Ofir Nachum, George Tucker, and Sergey Levine. D4RL: datasets for deep data-driven reinforcement learning.arXiv, abs/2004.07219, 2020

    Pith/arXiv arXiv 2004

  12. [12]

    Decision transformer: Reinforcement learning via sequence modeling

    Lili Chen, Kevin Lu, Aravind Rajeswaran, Kimin Lee, Aditya Grover, Michael Laskin, Pieter Abbeel, Aravind Srinivas, and Igor Mordatch. Decision transformer: Reinforcement learning via sequence modeling. InNeurIPS, pages 15084–15097, 2021

    2021

  13. [13]

    Constrained decision transformer for offline safe reinforcement learning

    Zuxin Liu, Zijian Guo, Yihang Yao, Zhepeng Cen, Wenhao Yu, Tingnan Zhang, and Ding Zhao. Constrained decision transformer for offline safe reinforcement learning. InICML, pages 21611–21630, 2023

    2023

  14. [14]

    Critic-guided decision transformer for offline reinforcement learning

    Yuanfu Wang, Chao Yang, Ying Wen, Yu Liu, and Yu Qiao. Critic-guided decision transformer for offline reinforcement learning. InAAAI, volume 38, pages 15706–15714, 2024

    2024

  15. [15]

    Offline reinforcement learning with implicit q-learning

    Ilya Kostrikov, Ashvin Nair, and Sergey Levine. Offline reinforcement learning with implicit q-learning. InICLR, 2022. 10

    2022

  16. [16]

    Trajdeleter: Enabling trajectory forgetting in offline reinforcement learning agents

    Chen Gong, Kecen Li, Jin Yao, and Tianhao Wang. Trajdeleter: Enabling trajectory forgetting in offline reinforcement learning agents. InNDSS, 2025

    2025

  17. [17]

    Rethinking machine unlearning for large language models.Nature Machine Intelligence, 7(2):181–194, 2025

    Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, et al. Rethinking machine unlearning for large language models.Nature Machine Intelligence, 7(2):181–194, 2025

    2025

  18. [18]

    Eternal sunshine of the spotless net: Selective forgetting in deep networks

    Aditya Golatkar, Alessandro Achille, and Stefano Soatto. Eternal sunshine of the spotless net: Selective forgetting in deep networks. InCVPR, pages 9301–9309, 2020

    2020

  19. [19]

    Erroreraser: Unlearning data bias for improved continual learning

    Xuemei Cao, Hanlin Gu, Xin Yang, Bingjun Wei, Haoyang Liang, Xiangkun Wang, and Tianrui Li. Erroreraser: Unlearning data bias for improved continual learning. InACM SIGKDD, pages 119–130, 2025

    2025

  20. [20]

    Unrolling SGD: understanding factors influencing machine unlearning

    Anvith Thudi, Gabriel Deza, Varun Chandrasekaran, and Nicolas Papernot. Unrolling SGD: understanding factors influencing machine unlearning. InIEEE EuroS&P, pages 303–319, 2022

    2022

  21. [21]

    Lethevit: Selective machine unlearning for vision transformers via attention-guided contrastive learning.arXiv, abs/2508.01569, 2025

    Yujia Tong, Tian Zhang, Jingling Yuan, Yuze Wang, and Chuang Hu. Lethevit: Selective machine unlearning for vision transformers via attention-guided contrastive learning.arXiv, abs/2508.01569, 2025

    Pith/arXiv arXiv 2025

  22. [22]

    Module-aware parameter-efficient machine unlearning on transformers.arXiv, abs/2508.17233, 2025

    Wenjie Bao, Jian Lou, Yuke Hu, Xiaochen Li, Zhihao Liu, Jiaqi Liu, Zhan Qin, and Kui Ren. Module-aware parameter-efficient machine unlearning on transformers.arXiv, abs/2508.17233, 2025

    Pith/arXiv arXiv 2025

  23. [23]

    Subspace control: Turning constrained model steering into controllable spectral optimization.arXiv, abs/2604.04231, 2026

    Yancheng Huang, Changsheng Wang, Chongyu Fan, Yicheng Lang, Bingqi Shang, Yang Zhang, Mingyi Hong, Qing Qu, Alvaro Velasquez, and Sijia Liu. Subspace control: Turning constrained model steering into controllable spectral optimization.arXiv, abs/2604.04231, 2026

    Pith/arXiv arXiv 2026

  24. [24]

    Membership inference attacks against machine learning models

    Reza Shokri, Marco Stronati, Congzheng Song, and Vitaly Shmatikov. Membership inference attacks against machine learning models. InIEEE S&P, pages 3–18, 2017

    2017

  25. [25]

    Privacy risk in machine learning: Analyzing the connection to overfitting

    Samuel Yeom, Irene Giacomelli, Matt Fredrikson, and Somesh Jha. Privacy risk in machine learning: Analyzing the connection to overfitting. InIEEE CSF, pages 268–282, 2018

    2018

  26. [26]

    ML-Leaks: Model and data independent membership inference attacks and defenses on machine learning models

    Ahmed Salem, Yang Zhang, Mathias Humbert, Pascal Berrang, Mario Fritz, and Michael Backes. ML-Leaks: Model and data independent membership inference attacks and defenses on machine learning models. InNDSS, 2019

    2019

  27. [27]

    Systematic evaluation of privacy risks of machine learning models

    Liwei Song and Prateek Mittal. Systematic evaluation of privacy risks of machine learning models. InUSENIX, pages 2615–2632, 2021

    2021

  28. [28]

    Membership inference attacks from first principles

    Nicholas Carlini, Steve Chien, Milad Nasr, Shuang Song, Andreas Terzis, and Florian Tramèr. Membership inference attacks from first principles. InIEEE S&P, pages 1897–1914, 2022

    1914

  29. [29]

    Offline reinforcement learn- ing: Tutorial, review, and perspectives on open problems.arXiv preprint arXiv:2005.01643, abs/2005.01643, 2020

    Sergey Levine, Aviral Kumar, George Tucker, and Justin Fu. Offline reinforcement learn- ing: Tutorial, review, and perspectives on open problems.arXiv preprint arXiv:2005.01643, abs/2005.01643, 2020

    Pith/arXiv arXiv 2005

  30. [30]

    Conservative q-learning for offline reinforcement learning

    Aviral Kumar, Aurick Zhou, George Tucker, and Sergey Levine. Conservative q-learning for offline reinforcement learning. volume 33, pages 1179–1191, 2020

    2020

  31. [31]

    Long short-term memory.Neural Computation, 9(8):1735–1780, 1997

    Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory.Neural Computation, 9(8):1735–1780, 1997

    1997

  32. [32]

    Reinforcement unlearning

    Dayong Ye, Tianqing Zhu, Congcong Zhu, Derui Wang, Kun Gao, Zewei Shi, Sheng Shen, Wanlei Zhou, and Minhui Xue. Reinforcement unlearning. InNDSS, 2025

    2025

  33. [33]

    Language models are unsupervised multitask learners

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. Language models are unsupervised multitask learners. Technical report, OpenAI, 2019

    2019

  34. [34]

    Offline reinforcement learning as one big sequence modeling problem

    Michael Janner, Qiyang Li, and Sergey Levine. Offline reinforcement learning as one big sequence modeling problem. volume 34, pages 1273–1286, 2021. 11 A Environment and Configuration A.1 Environment Overview TOUR spans three offline RL locomotion environments with distinct control and reward structures. HalfCheetah, Hopper, and Walker2D are continuous-co...

    arXiv 2021