REVIEW 5 major objections 5 minor 37 references
FUTURE claims that split thresholds tuned on a differentiable soft-decision-forest surrogate can be copied back into hard tree ensembles to unlearn up to 40% of training data while keeping about 95% of test AUC.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
FUTURE forgets training samples from tree ensembles by optimizing sigmoid-smoothed split thresholds and copying them back to the original discrete trees.
T0 review reviewed 2026-08-05 challenge →
load-bearing objection Model-agnostic unlearning for tree ensembles is a real gap, and this paper has a plausible gradient-based approach, but the headline claim that forgetting occurs isn't actually measured. the 5 major comments →
FUTURE: Flexible Unlearning for Tree Ensemble
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
Core claim
The central claim is that forgetting can be achieved purely by moving split thresholds, and the right threshold moves can be found on a softened surrogate: replace each indicator comparison with a sigmoid, replace the ensemble vote with a softmax, keep the tree structure and leaf one-hot distributions frozen, and optimize the thresholds to maximize entropy on forget samples while minimizing KL divergence and cross-entropy on retain samples. The optimized thresholds are then copied back into the hard ensemble. The paper reports that this preserves about 95% of test AUC on Adult and Diabetes even at 40% deletion and outperforms RF- and boosting-specific baselines, with lower unlearning time th
What carries the argument
The central object is the soft decision forest surrogate: each discrete split test is replaced by a sigmoid, activations multiply along tree paths, each leaf contributes a frozen one-hot prediction, and the ensemble's argmax is replaced by a softmax. This makes the whole ensemble differentiable in the thresholds, so the unlearning objective—maximized predictive entropy on forget samples plus KL and cross-entropy faithfulness on retain samples—can be minimized by gradient descent. The temperature parameters control how close the surrogate is to the hard ensemble; the paper treats them as hyperparameters rather than sending them to infinity, because exactness would kill the gradients.
Load-bearing premise
The method assumes that threshold changes found on the smoothed sigmoid surrogate still erase the target samples' influence when copied back into the original hard trees, even though the leaf class distributions are frozen one-hot estimates.
What would settle it
Poison 5% of training samples with a distinctive trigger, apply FUTURE, then measure trigger success on the unlearned hard model: if poisoned inputs still reach their original leaves and produce the backdoor prediction at the pre-unlearning rate, the threshold-copy step did not erase the forget samples. A direct check is whether forget-set samples still land in the same leaves with the same one-hot class labels after the threshold updates.
If this is right
- Deployed RF, GBDT, and XGBoost models can honor deletion requests without retraining, losing only about 5% of test AUC even when 40% of the training data is removed.
- Any threshold-split tree ensemble can use the same unlearning loop; the method does not depend on how the ensemble was trained.
- Unlearning becomes a standard gradient-descent problem, so its cost scales with the number of thresholds and forget-set size rather than with discrete split-gain bookkeeping.
- Because the surrogate is trained end-to-end on the exact forget list, the same code handles tiny and large unlearning requests without structural modification.
- The reported backdoor-removal results imply the method also weakens deliberate data poisoning, not just incidental memorization.
Where Pith is reading between the lines
- The frozen one-hot leaf distributions suggest FUTURE may leave class-statistic information about forget samples in the leaves; a natural extension is to update leaf predictions alongside thresholds and test with membership-inference attacks.
- The same sigmoid-threshold surrogate could be applied to regression trees, decision rules, or other threshold-based models, since only the split condition needs to be made differentiable.
- Annealing the temperature parameters from small to large values during optimization could improve how faithfully the retuned thresholds transfer to the hard ensemble; the paper fixes them as hyperparameters.
- A complementary audit could measure membership-inference accuracy on the forget set itself; if it drops to chance, the threshold-only update is erasing sample identity rather than just changing predictions.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FUTURE, an approximate unlearning algorithm for tree ensembles. It builds a soft decision forest surrogate of the original hard trees by replacing indicator splits with sigmoids and the ensemble arg max with a temperature-softmax, then optimizes only the split thresholds to maximize predictive entropy on the forget set D_f and to preserve predictions/accuracy on the retain set D_r. The optimized thresholds are copied back to the original hard ensemble. Experiments on Adult and Diabetes with random forest, GBDT, and XGBoost report test AUC-ROC, a backdoor-attack evaluation, and runtime; the authors claim that FUTURE preserves about 95% of test AUC while forgetting up to 40% of training data and is the first model-agnostic tree-ensemble unlearning method.
Significance. If the central claim were established, FUTURE would be a useful contribution: it is the first approach that applies the same unlearning procedure to random forests, GBDT, and XGBoost, and its surrogate-threshold-transfer idea is a plausible alternative to retraining-based and structure-specific methods. The efficiency results (Figure 4) are encouraging, and the formulation is simple enough to build on. However, the paper currently does not directly measure whether D_f is forgotten in the hard model, and the transfer from the soft surrogate to the hard ensemble is not validated. The novelty and potential are real, but the load-bearing empirical claim is unsupported as written.
major comments (5)
- [Section 4.2, Figure 3/Figure 5] The evaluation does not measure forgetting on D_f. Test AUC measures utility, not erasure. The backdoor experiment is described as 'unlearning effectiveness' but the reported result is 'test accuracy on clean data' (Section 4.2), with no attack success rate on poisoned inputs after unlearning, no membership-inference attack, and no accuracy/prediction-entropy on D_f itself. Because Eq. (7) defines forgetting as high predictive entropy on D_f and Eq. (9) maximizes it, any evaluation using the soft model's D_f predictions would be circular; but a direct hard-model metric is required. Please add D_f accuracy/entropy for the hard unlearned model, a membership-inference experiment, or a comparison of predictions on D_f before/after unlearning relative to retraining.
- [Section 3.1, Eq. (5) and Section 3.3] Leaf distributions t_l(y|j) are frozen at the original one-hot estimates, and only thresholds are copied back. A forget sample's hard-model prediction can change only if the updated thresholds reroute it to a leaf with a different one-hot label. Since D_f and D_r share paths, no analysis or experiment quantifies how many D_f samples are actually rerouted, and the original leaf labels may still correspond to the forget class. Please report, on D_f, the fraction of samples whose hard-model leaf path changes after unlearning and whether the final hard predictions differ from the original; alternatively, update or calibrate leaf distributions and justify why this is safe.
- [Section 3.1, Eq. (6), Eq. (9)] Finite sigma and tau make the surrogate only approximate; the limit sigma,tau->infinity recovers the hard model, but finite values are chosen to avoid vanishing gradients. High soft-model entropy on D_f does not imply high hard-model entropy after threshold transfer, and no calibration experiment compares soft vs hard predictions before and after optimization. Please add a sensitivity/calibration study (e.g., prediction agreement between soft surrogate and hard ensemble, and hard-model entropy increase on D_f) to support the threshold-transfer step.
- [Section 4.1, Section 4.2] Key experimental details are missing and reported numbers are not consistent. Hyperparameters sigma, tau, alpha, beta, learning rate, optimizer, number of epochs, and batch size are not provided, so the results are not reproducible. The abstract claims '95% predictive power' while Section 4.2 says FUTURE 'can maintain 98% of the test set performance' at 20%/40% deletion; Figure 3's AUC values do not directly support either number. Please include a full hyperparameter table and exact metric values for each forget-set size and model.
- [Section 4.2, Figure 5] The backdoor experiment is not clearly reported. The text says only that 'the test accuracy on clean data remains comparable', which is not a backdoor-removal metric. If Figure 5 also plots backdoor success rate, explain it; otherwise add the attack success rate before/after unlearning and state the poison rate and target feature/class. As written, the backdoor claim in the abstract and the 'significant and successful unlearning performance' conclusion are not supported.
minor comments (5)
- [Section 4.1] Reference errors: DaRE and HedgeCut are cited as [29], but DaRE is from [3] and HedgeCut is from [27]. The reference list also contains unrelated items (e.g., [29] is a recommender-systems paper).
- [Section 4.2] Typographical issues: 'FUGURE' appears twice, 'AOC-RUC' in the Figure 3 caption should be 'AUC-ROC', and 'forget set site' should be 'forget-set size'.
- [Section 3.1, Eq. (7)] The notation H_f(xf, g_u) is a function of a set but written as if it takes a single sample; clarify that it is an empirical average over D_f. Also define whether H_f is used only as a loss term or as an evaluation metric.
- [Section 4.1] The datasets are described with inconsistent sizes: Diabetes is said to contain 81,412 instances and Adult 48,842; typical Adult has 48,842 rows, but Diabetes is usually smaller. Verify the numbers and add a data-preprocessing reference.
- [General] The paper says FUTURE is 'model-agnostic' and works with 'all kinds of tree-based models', but experiments cover only two tabular datasets and three model families. A brief discussion of expected behavior on regression, multi-class, or missing-feature trees would help.
Circularity Check
No significant circularity: FUTURE's derivation is self-contained and its effectiveness claims are tested against external metrics, not its own training objective.
full rationale
The paper's derivation chain is self-contained. The soft-decision-forest surrogate (Eqs. 4-6) uses a standard sigmoid approximation of the indicator function; the unlearning objective (Eq. 9) directly maximizes predictive entropy on D_f and minimizes KL/classification losses on D_r. The claimed unlearning success is not measured by the optimized entropy itself, but by external behavioral metrics: test AUC (Fig. 3), backdoor attack accuracy (Fig. 5), and runtime. Thus no fitted quantity is renamed as a prediction, and the central result does not reduce to the objective by construction. Self-citations in the reference list (e.g., [4], [6], [10]) support standard definitions and evaluation metrics and are not load-bearing for the method's correctness. The approximation caveat regarding σ and τ and the absence of a formal transfer guarantee are correctness/robustness concerns, not circularity. No circular step can be exhibited from the paper's own equations.
Axiom & Free-Parameter Ledger
free parameters (4)
- sigma (sigmoid temperature) =
not reported
- tau (softmax temperature) =
not reported
- alpha (retain KL weight) =
not reported
- beta (forget entropy weight) =
not reported
axioms (4)
- ad hoc to paper A sigmoid with temperature sigma can approximate the indicator function so closely that thresholds optimized in the soft surrogate transfer back to the original hard tree ensemble.
- ad hoc to paper Softmax over tree votes with temperature tau reproduces the arg max in Eq. 1 closely enough for gradient-based unlearning.
- domain assumption Maximizing predictive entropy on D_f is a valid operationalization of forgetting.
- ad hoc to paper Leaf prediction distributions t_l(y|j) can be kept frozen from the original model while only thresholds are updated.
Cite this review
Pith. "Pith review of FUTURE: Flexible Unlearning for Tree Ensemble." pith.science (2026). https://pith.science/paper/AC2A2IGN
@misc{pith2026250821181,
author = {Pith},
title = {Pith review of: FUTURE: Flexible Unlearning for Tree Ensemble},
year = {2026},
howpublished = {\url{https://pith.science/paper/AC2A2IGN}},
note = {Machine review of arXiv:2508.21181}
}
read the original abstract
Tree ensembles are widely recognized for their effectiveness in classification tasks, achieving state-of-the-art performance across diverse domains, including bioinformatics, finance, and medical diagnosis. With increasing emphasis on data privacy and the \textit{right to be forgotten}, several unlearning algorithms have been proposed to enable tree ensembles to forget sensitive information. However, existing methods are often tailored to a particular model or rely on the discrete tree structure, making them difficult to generalize to complex ensembles and inefficient for large-scale datasets. To address these limitations, we propose FUTURE, a novel unlearning algorithm for tree ensembles. Specifically, we formulate the problem of forgetting samples as a gradient-based optimization task. In order to accommodate non-differentiability of tree ensembles, we adopt the probabilistic model approximations within the optimization framework. This enables end-to-end unlearning in an effective and efficient manner. Extensive experiments on real-world datasets show that FUTURE yields significant and successful unlearning performance.
Figures
Reference graph
Works this paper leans on
-
[1]
Randall Balestriero. 2017. Neural decision trees. arXiv preprint arXiv:1702.07360 (2017)
Pith/arXiv arXiv 2017
-
[2]
Lucas Bourtoule, Varun Chandrasekaran, Christopher A Choquette-Choo, Hen- grui Jia, Adelin Travers, Baiwu Zhang, David Lie, and Nicolas Papernot. 2021. Machine unlearning. In 2021 IEEE symposium on security and privacy (SP) . IEEE, 141–159
work page 2021
-
[3]
Jonathan Brophy and Daniel Lowd. 2021. Machine unlearning for random forests. In International Conference on Machine Learning . PMLR, 1092–1104
work page 2021
- [4]
-
[5]
Ziheng Chen, Jia Wang, Jun Zhuang, Abbavaram Gowtham Reddy, Fabrizio Silvestri, Jin Huang, Kaushiki Nag, Kun Kuang, Xin Ning, and Gabriele Tolomei
-
[6]
Jiali Cheng and Hadi Amiri. 2024. Mu-bench: A multitask multimodal benchmark for machine unlearning. arXiv preprint arXiv:2406.14796 (2024)
Pith/arXiv arXiv 2024
-
[7]
Jiali Cheng and Hadi Amiri. 2025. MultiDelete for Multimodal Machine Un- learning. In Computer Vision – ECCV 2024 , Aleš Leonardis, Elisa Ricci, Stefan Roth, Olga Russakovsky, Torsten Sattler, and Gül Varol (Eds.). Springer Nature Switzerland, Cham, 165–184
work page 2025
-
[8]
Jiali Cheng and Hadi Amiri. 2025. Tool Unlearning for Tool-Augmented LLMs. In Forty-second International Conference on Machine Learning . https://openreview. net/forum?id=7ez7LqHsP5
work page 2025
-
[9]
Jiali Cheng and Hadi Amiri. 2025. Understanding machine unlearning through the lens of mode connectivity. arXiv preprint arXiv:2504.06407 (2025)
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[10]
Jiali Cheng, George Dasoulas, Huan He, Chirag Agarwal, and Marinka Zitnik
-
[11]
Manqing Dong, Lina Yao, Xianzhi Wang, Boualem Benatallah, Shuai Zhang, and Quan Z Sheng. 2021. Gradient boosted neural decision forest. IEEE Transactions on Services Computing 16, 1 (2021), 330–342
work page 2021
-
[12]
Chongyu Fan, Jinghan Jia, Yihua Zhang, Anil Ramakrishna, Mingyi Hong, and Sijia Liu. 2025. Towards LLM Unlearning Resilient to Relearning Attacks: A Sharpness-Aware Minimization Perspective and Beyond. In Forty-second Inter- national Conference on Machine Learning . https://openreview.net/forum?id= zZjLv6F0Ks
work page 2025
-
[13]
Jing Gao. 2024. Online Learning Under Complex and Dynamic Changing Envi- ronments. A vailable at SSRN 5376534 (2024)
work page 2024
-
[14]
Jing Gao, Ankur Mani, and Lavanya Marla. 2020. Sensing in Airspace for Sequen- tial OD Aircraft Routing. A vailable at SSRN (2020)
work page 2020
-
[15]
Jin Huang, Zezhong Fan, Lalitesh Morishetti, Yuchan Guo, Kaushiki Nag, Hong- shik Ahn, Ziheng Chen, and Gabriele Tolomei. 2025. Prompt-Tuning for Recom- mendation Unlearning. In 2025 IEEE Conference on Artificial Intelligence (CAI) . IEEE, 859–863
work page 2025
-
[16]
Ozan Irsoy, Olcay Taner Yıldız, and Ethem Alpaydın. 2012. Soft decision trees. In Proceedings of the 21st international conference on pattern recognition (ICPR2012) . IEEE, 1819–1822
work page 2012
-
[17]
Mayuka Jayawardhana, Samuel Dooley, Valeriia Cherepanova, Andrew Gordon Wilson, Frank Hutter, Colin White, Tom Goldstein, Micah Goldblum, et al. 2025. Transformers boost the performance of decision trees on tabular data across sample sizes. arXiv preprint arXiv:2502.02672 (2025)
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[18]
Jiali Cheng and Hadi Amiri. 2025. Speech Unlearning. In Interspeech 2025. 3209–
work page 2025
-
[19]
Peter Kontschieder, Madalina Fiterau, Antonio Criminisi, and Samuel Rota Bulo
-
[20]
Huawei Lin, Jun Woo Chung, Yingjie Lao, and Weijie Zhao. 2023. Machine unlearning in gradient boosting decision trees. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . 1374–1383
work page 2023
-
[21]
Han Liu, Yuhao Wu, Zhiyuan Yu, and Ning Zhang. 2024. Please tell me more: Privacy impact of explainability through the lens of membership inference attack. In 2024 IEEE Symposium on Security and Privacy (SP) . IEEE, 4791–4809
work page 2024
-
[22]
Varshney, Mohit Bansal, Sanmi Koyejo, and Yang Liu
Sijia Liu, Yuanshun Yao, Jinghan Jia, Stephen Casper, Nathalie Baracaldo, Peter Hase, Yuguang Yao, Chris Yuhao Liu, Xiaojun Xu, Hang Li, Kush R. Varshney, Mohit Bansal, Sanmi Koyejo, and Yang Liu. 2025. Rethinking machine unlearning for large language models. Nature Machine Intelligence 7, 2 (01 Feb 2025), 181–194. https://doi.org/10.1038/s42256-025-00985-0
-
[23]
Weisi Liu, Guangzeng Han, and Xiaolei Huang. 2025. Examining and adapting time for multilingual classification via mixture of temporal experts.arXiv preprint arXiv:2502.08825 (2025)
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[24]
Yingzhou Lu, Minjie Shen, Huazheng Wang, Xiao Wang, Capucine van Rechem, Tianfan Fu, and Wenqi Wei. 2023. Machine learning for synthetic data generation: a review. arXiv preprint arXiv:2302.04062 (2023)
Pith/arXiv arXiv 2023
-
[25]
Ana Lucic, Harrie Oosterhuis, Hinda Haned, and Maarten de Rijke. 2022. FO- CUS: Flexible optimizable counterfactual explanations for tree ensembles. In Proceedings of the AAAI conference on artificial intelligence , Vol. 36. 5313–5322
work page 2022
-
[26]
Haoran Luo, Fan Cheng, Heng Yu, and Yuqi Yi. 2021. SDTR: Soft decision tree regressor for tabular data. IEEE Access 9 (2021), 55999–56011
work page 2021
-
[27]
Sebastian Schelter, Stefan Grafberger, and Ted Dunning. 2021. Hedgecut: Main- taining randomised trees for low-latency machine unlearning. In Proceedings of the 2021 International Conference on Management of Data . 1545–1557
work page 2021
-
[28]
Zhaomin Wu, Junhui Zhu, Qinbin Li, and Bingsheng He. 2023. Deltaboost: Gradient boosting decision trees with efficient machine unlearning. Proceedings of the ACM on Management of Data 1, 2 (2023), 1–26
work page 2023
-
[29]
Xin Xin, Xiangnan He, Yongfeng Zhang, Yongdong Zhang, and Joemon Jose
-
[30]
Yuxin Zhang, Naxi Chen, Yeyubei Zhang, and Wenjun Wu. 2025. Research on business decision support system based on big data and artificial intelligence. A vailable at SSRN 5332298 (2025)
work page 2025
-
[31]
Yigeng Zhang, Mahsa Shafaei, Fabio Gonzalez, and Thamar Solorio. 2021. From none to severe: Predicting severity in movie scripts. arXiv preprint arXiv:2109.09276 (2021)
work page internal anchor Pith review Pith/arXiv arXiv 2021
-
[32]
Yigeng Zhang, Mahsa Shafaei, Fabio Gonzalez, and Thamar Solorio. 2024. Positive and Risky Message Assessment for Music Products. In 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)
work page 2024
-
[2015]
In Proceedings of the IEEE international conference on computer vision
Deep neural decision forests. In Proceedings of the IEEE international conference on computer vision . 1467–1475
-
[2019]
Relational Collaborative Filtering: Modeling Multiple Item Relations for Recommendation. In Proc. of SIGIR’19. ACM, 125–134
-
[2023]
In The Eleventh International Conference on Learning Representations
GNNDelete: A General Strategy for Unlearning in Graph Neural Networks. In The Eleventh International Conference on Learning Representations . https: //openreview.net/forum?id=X9yCkmT5Qrl
-
[2024]
Debiasing machine unlearning with counterfactual examples.arXiv preprint arXiv:2404.15760 (2024)
work page internal anchor Pith review Pith/arXiv arXiv 2024
-
[3213]
https://doi.org/{10.21437/Interspeech.2025-2412}
This paper was first reviewed by deepseek-v4-flash on August 5, 2026.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.