Pith. sign in

REVIEW 2 major objections 5 minor 62 references

Stress-Testing ML Pipelines with Adversarial Data Corruption

T0 review · 2 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Realistic worst-case data errors wreck ML pipelines at just 5% corruption

desk verdict SAVAGE is a real step forward for pipeline red-teaming — a novel dependency-graph search over structured corruptions with a fair, extensive experimental campaign — but the dropped expectation in the optimization and an overstated abstract mean the headline numbers need calibration before publication. read the letter →

arxiv 2506.01230 v1 pith:XASQI6KU submitted 2025-06-02 cs.LG

Jiongli Zhu , Geyang Xu , Felipe Lorenzi , Boris Glavic , Babak Salimi This is my paper
classification cs.LG
keywords datacorruptionadversarialqualityMLpipelinerobustnessdependencygraphsbi-leveloptimizationBayesianmissingvalues
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SAVAGE aims to establish that realistic data-quality failures are structured and interdependent—missing values, label flips, and selection bias follow dependency patterns rather than appearing at random—and that such structured corruptions, even at small fractions around 5 percent, degrade ML pipelines far more than random or manually crafted errors. The paper formalizes this through a Data Corruption Process: a dependency graph that encodes which attributes influence whether and how other attributes are corrupted, paired with pattern-based corruption templates whose parameters control severity. It then frames the search for worst-case corruptions as a bi-level optimization, using beam search over dependency graphs and Bayesian optimization over corruption parameters while treating the entire pipeline as a black box. Experiments across data cleaning, fairness-aware learning, and uncertainty quantification show that existing techniques, including ones designed for specific error types, break under the systematic corruptions SAVAGE discovers. If right, the paper provides both a stress-testing tool and evidence that current robustness evaluations systematically underestimate worst-case data-quality risk.

What carries the argument

The load-bearing object is the data corruption process template: a dependency graph whose nodes are clean attributes, their corrupted counterparts, noise variables, and a selection indicator, together with a pattern-based corruption template that specifies, for each corrupted attribute, a conjunction of range conditions over its parents and a parametric corruption function. Edges express only how the probability or severity of an error in a child depends on parent values, not causal relations among clean attributes. The upper level of the search is beam search over dependency graphs—which attributes appear in the corruption pattern—and the lower level is Tree-Structured Parzen Estimator Bayesian optimization over corruption parameters such as thresholds and noise distributions. A proxy-pipeline strategy transfers dependency graphs found on cheap models to expensive pipelines, and sampling transfers patterns to large datasets, which is what makes the search practical at million-row scale.

What would settle it

Run SAVAGE on a small dataset and exhaustively enumerate every dependency graph and parameter setting within the same template family; if any enumerated corruption with the same error budget degrades the target metric more than SAVAGE's best found corruption, the search is not near-optimal. Alternatively, generate synthetic data from a known corruption process and check whether SAVAGE's recovered dependency graph and parameters match the generating process; recovery failure would show the framework models a proxy rather than the true mechanism.

Watch

Extended reading notes

Core claim

The paper's central claim is that even a small fraction—around 5 percent—of structured corruptions identified by SAVAGE can severely degrade a target performance metric, far beyond the impact of random or manually crafted errors, and that this invalidates core assumptions of existing techniques. The claim is made concrete as a formal problem: given a feasible space of data corruption processes defined by dependency graphs and corruption templates, find the process that minimizes expected pipeline performance. The paper argues this worst case is realistic, not extreme, because the dependency graphs encode plausible mechanisms such as missing-not-at-random, label-dependent missingness, and selection bias, and user-supplied plausibility constraints prune unrealistic scenarios. The empirical sections report that systematic corruptions with a small budget break data-cleaning pipelines, debiasing methods, and conformal-prediction uncertainty guarantees, including methods explicitly designed to handle missing values or label errors.

Load-bearing premise

The load-bearing assumption is that the restricted search space—single target attribute per error type, range-condition patterns, independent noise variables—together with the beam search and TPE heuristics suffices to find the true worst-case corruption; if the real worst case lies outside this space or the heuristics miss it, the central claim that SAVAGE finds maximally degrading corruptions collapses.

Editorial extensions

If this is right

  • Robustness claims based on random or manually specified corruptions are systematically optimistic; SAVAGE-type search can serve as a benchmarking layer for data cleaning, debiasing, and uncertainty quantification methods.
  • Methods tailored to specific error types—imputers built for missing-completely-at-random data, conformal predictors that assume missingness independent of the label, debiasing that assumes label and sensitive-attribute marginals are preserved—lose their guarantees under the structured corruptions SAVAGE finds.
  • Small error budgets around 5 percent are enough to produce severe degradation, so practitioners should treat low corruption rates as a live threat, especially for decision trees and neural networks which show higher variance.
  • Gradient-based data poisoning attacks are less effective than SAVAGE in black-box settings with non-differentiable preprocessing, because SAVAGE exploits structured subpopulation patterns rather than low-level perturbations.
  • The worst corruption for a target subpopulation may lie in a different subpopulation, so stress-testing must search beyond the obvious target.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's worst-case claim is bounded by its search space: only a single target attribute per error type, range-condition patterns, and independent noise variables are considered, and both beam search and TPE are heuristics without near-optimality guarantees; a broader space of multi-attribute or dependent-noise corruptions could contain even worse cases.
  • The discovered dependency graphs could double as diagnostic feedback: a pattern like missingness that depends on the label pinpoints the exact data-collection mechanism that needs fixing, suggesting SAVAGE could be used not just for evaluation but for prioritizing data-quality interventions.
  • A natural testable extension is to run SAVAGE on synthetic data with a known generative corruption process and check whether the recovered dependency graph and parameters match the ground truth; currently the paper validates effectiveness through downstream performance rather than recovery fidelity.
  • The framework's formalism is not restricted to missing values, label errors, and selection bias; outliers, duplication effects, and test-time corruptions fit the same dependency-graph and template machinery and could be explored in future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper introduces SAVAGE, a framework for generating adversarial data corruption scenarios to stress-test ML pipelines. It models realistic data-quality issues through dependency graphs and pattern-based corruption templates, forming Data Corruption Processes (DCPs). The system searches over DCPs using a bi-level optimization: beam search over dependency graph structures at the upper level and TPE-based Bayesian optimization for parameter tuning at the lower level, treating the entire ML pipeline as a black box. Extensive experiments across data cleaning, fairness-aware learning, and uncertainty quantification show that small fractions of structured corruptions cause larger performance drops than random or manually designed corruptions, and that SAVAGE outperforms existing data poisoning baselines in this setting.

Significance. If the results hold, SAVAGE provides a useful practical tool for red-teaming ML pipelines and a benchmark for robustness evaluation. The framework's interpretable corruption patterns, black-box treatment of pipelines, and coverage of multiple error types are notable strengths. The paper includes a broad experimental study, ablations, and comparisons to poisoning attacks, and it claims code availability. However, the evaluation protocol has a load-bearing issue: the optimization objective drops the expectation over noise variables, and the selection of corruption parameters is based on single noisy evaluations, which can inflate the reported degradation. This concern is central to the quantitative claims and requires revision before the results can be fully trusted.

major comments (2)
  1. [Section 4.1, Algorithm 2, Eq. (1)] Eq. (1) defines the objective as minimizing expected performance E[Ψ(A(M(D_train)), D_test)], but the bi-level formulation immediately drops the expectation, and Algorithm 2 evaluates each parameter setting with a single realization (Ψ_t = Ψ(A(M(D_train)), D_test)). Because DCPs are stochastic (e.g., noise variable N_Y ~ Uniform[0,1] in Example 1), the TPE selection is based on the minimum of single noisy draws. This is selection on noise: the selected DCP is the one that produced a favorable Ψ by chance, not necessarily the one with the lowest expected Ψ. The paper's footnote 2 mentions that noisy evaluations are common in BO, but that does not address the bias in selecting an argmin over noisy observations. The reported means over 5 seeds do not remove this selection bias, since the selection itself used one draw per candidate. This can inflate the reported AUC drops and the gap over random baselines, both of which are central claims. Please either (a) re-evaluate the selected DCPs multiple times after search and report the re-estimated expected Ψ with confidence intervals, or (b) use multi-seed evaluation during the search itself and report how the conclusions change. This is empirically checkable and should be straightforward to address.
  2. [Section 5.5.1 and experimental setup] The paper does not report the search hyperparameters required by Algorithm 1 (beam width B, TPE iterations τ, max depth d_max) for any of the experiments. These control the quality and cost of the search, and without them the comparison between SAVAGE and baselines such as 'Random Search' (which evaluates 100 graphs) is not well-defined. Moreover, Table 10 shows that heuristic pruning reduces the graph space from 5619 to 1769, but the main experiments do not state how many graphs were actually evaluated or how many TPE iterations were used per graph. Given that the paper's central quantitative claims rest on SAVAGE finding particularly harmful corruptions, the absence of these search-effort parameters makes it difficult to assess whether the advantage comes from the method itself or from a favorable search budget. Please report these parameters and, ideally, a sensitivity analysis of the main results to them.
minor comments (5)
  1. [Section 3] There are typos: 'corrupdated' should be 'corrupted' and 'stoachstic' should be 'stochastic'; reference [55] has 'Adersarial' instead of 'Adversarial'.
  2. [Section 5] The statement in Section 4.1 that the expectation is dropped is explicit, but the empirical sections do not revisit this choice. Please add a paragraph in Section 5 discussing the implications of this choice for interpreting the reported performance drops.
  3. [Section 5.4] The comparison with shades-of-null would be clearer if it stated the exact evaluation protocol used for the baseline, particularly whether test-data missing values were injected and how the baseline's published numbers were obtained, since the original setting includes missing values in both training and test sets.
  4. [Figure 2] The overview figure is dense; explicitly labeling the upper-level (beam search) and lower-level (TPE) components would help readers map the figure to Section 4.1.
  5. [Table 11] The neural-network results show large standard deviations (e.g., 0.34±0.12); please state the number of seeds and clarify whether the search itself was repeated over seeds or only the evaluation of the selected corruption.

Circularity Check

0 steps flagged · score 2.0 of 10

No load-bearing circularity: SAVAGE is an optimization-based red-teaming tool whose discovered corruptions are the optimizer's outputs rather than predictions, and the paper's external baselines and full-graph ablation provide independent checks.

full rationale

SAVAGE's derivation chain is a search procedure, not a predictive derivation. Equation (1) defines the goal as finding DCPs that minimize expected performance, and Section 4.1 explicitly states 'we have dropped the expectation in the bi-level formulation,' so Algorithm 2 selects parameters on single pipeline runs. This creates a real selection-on-noise risk when interpreting the magnitudes of reported degradation, but it is not circularity: the reported corruption is the optimizer's output, and the paper's effectiveness claims concern the effect of the found corruption, which is the intended red-teaming deliverable. The comparisons to random search, random corruption, GradCancel/BackGrad, and shades-of-null are external checks that a circular argument could not guarantee to pass; a search that optimizes the target metric can still fail to outperform these baselines. The heuristic that forces label-dependent/MNAR patterns is validated in Table 10 against all 5,619 dependency graphs, showing that the pruning rules discard none of the most adverse graphs, so the 'label-dependent missingness is harmful' finding is empirically checked rather than imposed by the search space. Self-citations ([40], [54], [55]) are background pointers and a technical-report reference, not load-bearing support for the central claims. The lack of reported search hyperparameters (beam width, TPE iterations, depth) is a reproducibility and correctness-risk concern, not evidence of circularity. Overall, no step in the paper reduces by construction to its own inputs.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim does not derive a law; it reports the existence of harmful corruption patterns found by optimization. The main load-bearing choices are the restricted search space, the heuristic optimizers with undisclosed hyperparameters, and the noise-distribution assumptions. No new physical or model entities are postulated; the dependency graph and data corruption process are formal modeling constructs, not entities with independent evidence.

free parameters (4)
  • Beam width B = not reported
    Beam search width controls how many dependency-graph candidates are retained each iteration. The paper does not state its value, and the 'worst-case' claim depends on it.
  • TPE iterations tau = not reported
    Number of pipeline evaluations per candidate dependency graph in the lower-level optimization. Not stated in the paper; affects the quality of parameter tuning.
  • Max beam depth d_max = not reported
    Depth limit for pattern expansion in beam search; not specified, and it bounds how many attributes can appear in a corruption pattern.
  • Corruption budget %E = 10%, 30%, 50% in experiments
    User-set constraint on the fraction of rows corrupted. The abstract's 'around 5%' does not correspond to a reported experiment.
assumptions (5)
  • domain assumption Each corrupted attribute is associated with a single independent noise variable with a specified distribution (uniform in the examples).
    Definition 1 and Example 1: the uniform[0,1] noise choice is used across experiments and constrains the space of corruption processes.
  • domain assumption Dependency graphs are acyclic and each corrupted attribute has one noise variable.
    Section 3, Definition 1: excludes corruption processes with cycles or shared noise variables, limiting expressiveness.
  • ad hoc to paper The search is restricted to a single corrupted target attribute per error type (missing values, label errors, or selection bias).
    Section 4.1: 'we assume a predefined error type ... and restrict the search to DCPTs for this error type that corrupt a single target attribute.' Real-world corruption can affect multiple attributes jointly; this restriction is made for tractability.
  • ad hoc to paper Patterns must always include the target attribute and the label attribute.
    Section 4.2.3 heuristics: 'pattern-based templates must always include the target attribute(s) ... and the label attribute.' This biases the search toward label-dependent errors, which later drive the headline findings.
  • domain assumption Training data is corrupted while test data is assumed clean or corrupted independently.
    Section 4 Setting: the pipeline is trained on corrupted D_train and evaluated on clean D_test. This excludes train-test same-corruption scenarios studied by some benchmarks such as Shades-of-Null.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Stress-Testing ML Pipelines with Adversarial Data Corruption." pith.science (2026). https://pith.science/paper/XASQI6KU

@misc{pith2026250601230,
  author       = {Pith},
  title        = {Pith review of: Stress-Testing ML Pipelines with Adversarial Data Corruption},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XASQI6KU}},
  note         = {Machine review of arXiv:2506.01230}
}
read the original abstract

Structured data-quality issues, such as missing values correlated with demographics, culturally biased labels, or systemic selection biases, routinely degrade the reliability of machine-learning pipelines. Regulators now increasingly demand evidence that high-stakes systems can withstand these realistic, interdependent errors, yet current robustness evaluations typically use random or overly simplistic corruptions, leaving worst-case scenarios unexplored. We introduce SAVAGE, a causally inspired framework that (i) formally models realistic data-quality issues through dependency graphs and flexible corruption templates, and (ii) systematically discovers corruption patterns that maximally degrade a target performance metric. SAVAGE employs a bi-level optimization approach to efficiently identify vulnerable data subpopulations and fine-tune corruption severity, treating the full ML pipeline, including preprocessing and potentially non-differentiable models, as a black box. Extensive experiments across multiple datasets and ML tasks (data cleaning, fairness-aware learning, uncertainty quantification) demonstrate that even a small fraction (around 5 %) of structured corruptions identified by SAVAGE severely impacts model performance, far exceeding random or manually crafted errors, and invalidating core assumptions of existing techniques. Thus, SAVAGE provides a practical tool for rigorous pipeline stress-testing, a benchmark for evaluating robustness methods, and actionable guidance for designing more resilient data workflows.

Figures

Figures reproduced from arXiv: 2506.01230 by the authors.

Figure 1
Figure 1. We analyzed 398 public census tables (>1,100 [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of Savage. Different user questions map to varying constraints on the dependency graphs, where ○1 denotes no constraints, ○2 restricts to missing data, and ○3 specifies one unique dependency graph for selection bias. The dependency beam search is conducted to search the dependency graph and its corresponding worst-case concrete corruption (obtained by TPE) that leads to the lowest model utility measured by … view at source ↗
Figure 3
Figure 3. Dependency graphs for missing labels. 𝐷 𝑋 𝑍 𝑌 ∗ 𝑌 𝑆 𝐷: Demographics 𝑋: Socio-cultural traits 𝑍: Zipcode 𝑆: Police patrol indicator 𝑌: Actual crime commitment 𝑌 ∗ : Observed crime commitment [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Dependency graphs for label errors and selection [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Example of beam search (beam size = 1) for finding [PITH_FULL_IMAGE:figures/full_fig_p006_5.png]
Figure 6
Figure 6. Figure 6: AUC of logistic regression when corrupting Adult (a), Employee (b), Credit Card (c) and SQF (d) datasets with MVs. [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: AUC of logistic regression, corrupting Adult (a, c) and Employee (b, d) with selection bias (left) and label errors (right). [PITH_FULL_IMAGE:figures/full_fig_p009_7.png]
Figure 8
Figure 8. Figure 8: AUC of decision trees (first row), random forest [PITH_FULL_IMAGE:figures/full_fig_p010_8.png]
Figure 9
Figure 9. Figure 9: Decrease in AUC of models trained with corrupted [PITH_FULL_IMAGE:figures/full_fig_p012_9.png]
Figure 10
Figure 10. Figure 10: Decrease in subpopulation AUC due to corruption [PITH_FULL_IMAGE:figures/full_fig_p013_10.png]
Figure 12
Figure 12. Figure 12: AUC of models trained with different data corrup [PITH_FULL_IMAGE:figures/full_fig_p014_12.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

62 extracted references · 46 canonical work pages

  1. [1]

    Regulation (EU) 2024/1689 of the European Parliament and of the Council on Artificial Intelligence

    2024. Regulation (EU) 2024/1689 of the European Parliament and of the Council on Artificial Intelligence. https://artificialintelligenceact.eu/article/15/. Article 15: Accuracy, Robustness and Cyber-security

    work page 2024

  2. [2]

    Codebase for SAVAGE

    2025. Codebase for SAVAGE. https://github.com/lodino/savage

    work page 2025

  3. [3]

    Mohamed Abdelaal, Christian Hammacher, and Harald Schoening. 2023. Rein: A comprehensive benchmark framework for data cleaning methods in ml pipelines. arXiv preprint arXiv:2302.04702(2023)

    arXiv 2023

  4. [4]

    James Bergstra, Rémi Bardenet, Yoshua Bengio, and Balázs Kégl. 2011. Algorithms for hyper-parameter optimization.Advances in neural information processing systems24 (2011)

    2011

  5. [5]

    Battista Biggio, Blaine Nelson, and Pavel Laskov. 2012. Poisoning attacks against support vector machines.arXiv preprint arXiv:1206.6389(2012)

    arXiv 2012

  6. [6]

    Lukas Budach, Moritz Feuerpfeil, Nina Ihde, Andrea Nathansen, Nele Noack, Hendrik Patzlaff, Felix Naumann, and Hazar Harmouch. 2022. The effects of data quality on machine learning performance.arXiv preprint arXiv:2207.14529 (2022)

    arXiv 2022

  7. [7]

    Antonio Emanuele Cinà, Kathrin Grosse, Ambra Demontis, Sebastiano Vascon, Werner Zellinger, Bernhard A Moser, Alina Oprea, Battista Biggio, Marcello Pelillo, and Fabio Roli. 2023. Wild patterns reloaded: A survey of machine learning security against training data poisoning.Comput. Surveys55, 13s (2023), 1–39

    2023

  8. [8]

    2016.Practical machine learning with H2O: powerful, scalable techniques for deep learning and AI

    Darren Cook. 2016.Practical machine learning with H2O: powerful, scalable techniques for deep learning and AI. O’Reilly Media, Inc

    work page 2016

Show all 62 references
  1. [9]

    Jimmy Z Di, Jack Douglas, Jayadev Acharya, Gautam Kamath, and Ayush Sekhari

  2. [10]

    Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard S. Zemel. 2012. Fairness through awareness. InITCS. ACM, 214–226

    2012

  3. [11]

    Adrien Ehrhardt, Christophe Biernacki, Vincent Vandewalle, Philippe Heinrich, and Sébastien Beben. 2021. Reject inference methods in credit scoring.Journal of Applied Statistics48, 13-15 (2021), 2734–2754

    2021

  4. [12]

    Chen Xinyun et al. 2017. Targeted backdoor attacks on deep learning systems using data poisoning.arXiv preprint arXiv:1712.05526(2017)

    2017 arXiv

  5. [13]

    Matthias Feurer, Katharina Eggensperger, Stefan Falkner, Marius Lindauer, and Frank Hutter. 2022. Auto-sklearn 2.0: Hands-free automl via meta-learning.The Journal of Machine Learning Research23, 1 (2022), 11936–11996

    2022

  6. [14]

    Milena A Gianfrancesco, Suzanne Tamang, Jinoos Yazdany, and Gabriela Schma- juk. 2018. Potential biases in machine learning algorithms using electronic health record data.JAMA internal medicine178, 11 (2018), 1544–1547

    2018

  7. [15]

    2012.Missing data: Analysis and design

    John Graham. 2012.Missing data: Analysis and design. New York, NY: Springer. https://doi.org/10.1007/978-1-4614-4018-5

    2012 doi

  8. [16]

    Gareth J Griffith, Tim T Morris, Matthew J Tudball, Annie Herbert, Giulia Man- cano, Lindsey Pike, Gemma C Sharp, Jonathan Sterne, Tom M Palmer, George Davey Smith, et al. 2020. Collider bias undermines our understanding of COVID- 19 disease risk and severity.Nature communicat...

    2020

  9. [17]

    Shubha Guha, Falaah Arif Khan, Julia Stoyanovich, and Sebastian Schelter. 2022. Automated Data Cleaning Can Hurt Fairness in Machine Learning-based Deci- sion Making.ICDE(2022)

    2022

  10. [18]

    Luke Haliburton, Sinksar Ghebremedhin, Robin Welsch, Albrecht Schmidt, and Sven Mayer. 2023. Investigating Labeler Bias in Face Annotation for Machine Learning.arXiv preprint arXiv:2301.09902(2023)

    2023 arXiv

  11. [19]

    Moritz Hardt, Eric Price, and Nati Srebro. 2016. Equality of Opportunity in Supervised Learning. InNIPS. 3315–3323

    2016

  12. [20]

    Maliha Tashfia Islam, Anna Fariha, Alexandra Meliou, and Babak Salimi. 2022. Through the data management lens: Experimental analysis and evaluation of fair classification. InProceedings of the 2022 International Conference on Management of Data. 232–246

    2022

  13. [21]

    Matthew Jagielski, Giorgio Severi, Niklas Pousette Harger, and Alina Oprea. 2021. Subpopulation data poisoning attacks. InProceedings of the 2021 ACM SIGSAC Conference on Computer and Communications Security. 3104–3122. 14

    2021

  14. [22]

    Faisal Kamiran and Toon Calders. 2012. Data preprocessing techniques for classification without discrimination.Knowledge and information systems33, 1 (2012), 1–33

    2012

  15. [23]

    Faisal Kamiran and Toon Calders. 2012. Data preprocessing techniques for classification without discrimination.Knowledge and Information Systems33, 1 (2012), 1–33

    2012

  16. [24]

    Falaah Arif Khan, Denys Herasymuk, Nazar Protsiv, and Julia Stoyanovich

  17. [25]

    Sanjay Krishnan, Michael J Franklin, Ken Goldberg, and Eugene Wu. 2017. Boost- clean: Automated error detection and repair for machine learning.arXiv preprint arXiv:1711.01299(2017)

    2017 arXiv

  18. [26]

    Erin LeDell and Sebastien Poirier. 2020. H2o automl: Scalable automatic machine learning. InProceedings of the AutoML Workshop at ICML, Vol. 2020. ICML San Diego, CA, USA

    2020

  19. [27]

    Jing Lei, Max G’Sell, Alessandro Rinaldo, Ryan J Tibshirani, and Larry Wasserman

  20. [28]

    Peng Li, Zhiyi Chen, Xu Chu, and Kexin Rong. 2023. DiffPrep: Differentiable Data Preprocessing Pipeline Search for Learning over Tabular Data.Proceedings of the ACM on Management of Data1, 2 (2023), 1–26

    2023

  21. [29]

    Peng Li, Xi Rao, Jennifer Blase, Yue Zhang, Xu Chu, and Ce Zhang. 2021. Cleanml: A study for evaluating the impact of data cleaning on ml classification tasks. In 2021 IEEE 37th International Conference on Data Engineering (ICDE). IEEE, 13–24

    2021

  22. [30]

    Heng Liu and Gregory Ditzler. 2021. Data poisoning against information- theoretic feature selection.Information Sciences573 (2021), 396–411

    2021

  23. [31]

    Brandon Lockhart, Jinglin Peng, Weiyuan Wu, Jiannan Wang, and Eugene Wu

  24. [32]

    Yiwei Lu, Gautam Kamath, and Yaoliang Yu. 2022. Indiscriminate data poisoning attacks on neural networks.arXiv preprint arXiv:2204.09092(2022)

    2022 arXiv

  25. [33]

    Yiwei Lu, Gautam Kamath, and Yaoliang Yu. 2023. Exploring the limits of model- targeted indiscriminate data poisoning attacks. InInternational Conference on Machine Learning. PMLR, 22856–22879

    2023

  26. [34]

    Qingwei Luo, Sam Egger, Xue Qin Yu, David P Smith, and Dianne L O’Connell

  27. [35]

    Luis Muñoz-González, Battista Biggio, Ambra Demontis, Andrea Paudice, Vasin Wongrassamee, Emil C Lupu, and Fabio Roli. 2017. Towards poisoning of deep learning algorithms with back-gradient optimization. InProceedings of the 10th ACM workshop on artificial intelligence and sec...

    2017

  28. [36]

    2024.Artificial Intelligence Risk Management Framework (1.0) – Generative AI Profile

    National Institute of Standards and Technology. 2024.Artificial Intelligence Risk Management Framework (1.0) – Generative AI Profile. Technical Report NIST AI 600-1. U.S. Dept. of Commerce. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600- 1.pdf

    2024

  29. [37]

    2009.Causality

    Judea Pearl. 2009.Causality. Cambridge university press

    2009

  30. [38]

    Pedregosa, G

    F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cour- napeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine Learning in Python.Journal of Machine L...

    2011

  31. [39]

    Jennifer K Plichta, Christel N Rushing, Holly C Lewis, Marguerite M Rooney, Dan G Blazer, Samantha M Thomas, E Shelley Hwang, and Rachel A Greenup

  32. [40]

    Romila Pradhan, Jiongli Zhu, Boris Glavic, and Babak Salimi. 2022. Interpretable data-based explanations for fairness debugging. InProceedings of the 2022 inter- national conference on management of data. 247–261

    2022

  33. [41]

    Yuji Roh, Kangwook Lee, Steven Whang, and Changho Suh. 2021. Sample selection for fair and robust training.Advances in Neural Information Processing Systems34 (2021), 815–827

    2021

  34. [42]

    Yuji Roh, Kangwook Lee, Steven Euijong Whang, and Changho Suh. 2023. Im- proving fair training under correlation shifts. InInternational Conference on Machine Learning. PMLR, 29179–29209

    2023

  35. [43]

    Sudeepa Roy and Dan Suciu. 2014. A formal approach to finding explanations for database queries. InProceedings of the 2014 ACM SIGMOD international conference on Management of data. 1579–1590

    2014

  36. [44]

    Donald B Rubin. 1978. Multiple imputations in sample surveys-a phenomeno- logical Bayesian approach to nonresponse. InProceedings of the survey research methods section of the American Statistical Association, Vol. 1. American Statistical Association Alexandria, VA, USA, 20–34

    1978

  37. [45]

    Svetlana Sagadeeva and Matthias Boehm. 2021. Sliceline: Fast, linear-algebra- based slice finding for ml model debugging. InProceedings of the 2021 international conference on management of data. 2290–2299

    2021

  38. [46]

    Sebastian Schelter, Tammo Rukat, and Felix Biessmann. 2021. JENGA-A Frame- work to Study the Impact of Data Errors on the Predictions of Machine Learning Models.. InEDBT. 529–534

    2021

  39. [47]

    Avi Schwarzschild, Micah Goldblum, Arjun Gupta, John P Dickerson, and Tom Goldstein. 2021. Just how toxic is data poisoning? a unified benchmark for backdoor and data poisoning attacks. InInternational Conference on Machine Learning. PMLR, 9389–9398

    2021

  40. [48]

    Ali Shafahi, W Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer, Tudor Dumitras, and Tom Goldstein. 2018. Poison frogs! targeted clean-label poisoning attacks on neural networks.Advances in neural information processing systems31 (2018)

    2018

  41. [49]

    Volker Steinbiss, Bach-Hiep Tran, and Hermann Ney. 1994. Improvements in beam search.. InICSLP, Vol. 94. 2143–2146

    1994

  42. [50]

    Alexander Turner, Dimitris Tsipras, and Aleksander Madry. 2018. Clean-label backdoor attacks. (2018)

    2018

  43. [51]

    Xilu Wang, Yaochu Jin, Sebastian Schmitt, and Markus Olhofer. 2023. Recent advances in Bayesian optimization.Comput. Surveys55, 13s (2023), 1–36

    2023

  44. [52]

    Margaux Zaffran, Aymeric Dieuleveut, Julie Josse, and Yaniv Romano. 2023. Conformal prediction with missing values. InInternational Conference on Machine Learning. PMLR, 40578–40604

    2023

  45. [53]

    Zemel, Yu Wu, Kevin Swersky, Toniann Pitassi, and Cynthia Dwork

    Richard S. Zemel, Yu Wu, Kevin Swersky, Toniann Pitassi, and Cynthia Dwork

  46. [54]

    Jiongli Zhu and Babak Salimi. 2024. Overcoming Data Biases: Towards Enhanced Accuracy and Reliability in Machine Learning.IEEE Data Eng. Bull.47, 1 (2024), 18–35

    2024

  47. [55]

    Jiongli Zhu, Geyang Xu, Felipe Lorenzi, Boris Glavic, and Babak Salimi. 2025. Stress-Testing ML Pipelines with Adersarial Data Corruption (extended version). Technical Report. https://github.com/lodino/savage/blob/main/techreport/ techreport.pdf 15

    2025

  48. [2013]

    InICML (3) (JMLR Workshop and Conference Proceedings), Vol

    Learning Fair Representations. InICML (3) (JMLR Workshop and Conference Proceedings), Vol. 28. JMLR.org, 325–333

  49. [2017]

    unknown

    Validity of using multiple imputation for" unknown" stage at diagnosis in population-based cancer registry data.PLoS One12, 6 (2017), e0180033

    2017

  50. [2018]

    Distribution-free predictive inference for regression.J. Amer. Statist. Assoc. 113, 523 (2018), 1094–1111

    2018

  51. [2021]

    Explaining inference queries with bayesian optimization.arXiv preprint arXiv:2102.05308(2021)

    2021 arXiv

  52. [2022]

    InNeurIPS ML Safety Workshop

    Hidden poison: Machine unlearning enables camouflaged poisoning attacks. InNeurIPS ML Safety Workshop

  53. [2023]

    Implications of missing data on reported breast cancer mortality.Breast Cancer Research and Treatment197, 1 (2023), 177–187

    2023

  54. [2024]

    Still More Shades of Null: A Benchmark for Responsible Missing Value Imputation.arXiv preprint arXiv:2409.07510(2024)

    2024 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.