Pith. sign in

REVIEW 4 major objections 7 minor 25 references

Randomly assigning binary predictions and testing whether the two outcome distributions differ is enough to detect outcome performativity, with closed-form sample sizes under three standard models.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 17:22 UTC pith:VCU2NVUA

load-bearing objection Clean offline A/B detector for outcome performativity with usable N formulas; the math is power analysis, not PAC, and the uniformity reduction is the real limit. the 4 major comments →

arxiv 2607.26908 v1 pith:VCU2NVUA submitted 2026-07-29 cs.LG cs.AI

Actions Have Consequences: Detecting Outcome Performativity using Intervention Testing

Brandon Gower-Winter , Georg Krempl This is my paper
classification cs.LG cs.AI
keywords outcome performativityperformative predictionintervention testingA/B testingsample complexityregions of indistinguishabilitycausal detectionbinary classification
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Predictions can change the outcomes they are meant to forecast—granting a loan can keep a borrower solvent, recommending an item can change whether it is clicked. The paper shows that you can detect this causal influence offline, before any model is trained, by randomly assigning the predictions themselves and comparing the resulting label distributions with a standard statistical test. Under three common models of how predictions affect outcomes, the authors derive explicit formulas for how many random assignments per arm are needed for a chi-squared test to succeed. Empirically, detection works once the effect is large enough relative to the sample budget; when the two prediction arms produce nearly identical outcome rates, the required sample size diverges and the effect becomes practically invisible. A case study on real recommender-log data confirms low false-positive rates and the expected growth in power with more interventions. The practical message is that scarcity, cost, or ethics of interventions can leave entire regions of performativity undetectable.

Core claim

Under a do-intervention that randomly assigns binary predictions, a two-group test of the marginal outcome distributions P(Y|do(Ŷ=0)) versus P(Y|do(Ŷ=1)) detects outcome performativity, and the minimum per-arm sample size N admits closed-form expressions (Eqs. 9, 12, 14) for the Simple, Model-based, and Misclassification-based Nature models; these bounds are matched by detection-rate heatmaps and identify regions of indistinguishability where the allotted budget is insufficient.

What carries the argument

OPAB (Outcome Performativity A/B Detection): random Bernoulli assignment of predictions followed by a chi-squared (or Fisher) test on the two resulting contingency tables of observed outcomes; the sample-complexity formulas are obtained by equating the expected chi-squared statistic under each Nature model to the critical value χ²_δ.

Load-bearing premise

Performative effects are assumed uniform across features, so that comparing the two marginal outcome distributions is enough; if effects vary with X, a non-significant test does not prove absence of performativity.

What would settle it

On any of the three Nature models, fix parameters inside a claimed region of indistinguishability for a given N and δ; if the empirical detection rate of OPAB nevertheless approaches 1 over many replicates, the corresponding sample-complexity bound is false.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Practitioners can compute the minimum number of random prediction assignments needed before labelling begins, rather than discovering performativity only after deployment.
  • When the calculated N exceeds what ethics or cost allow, the setting lies in a region of indistinguishability and ordinary supervised learning may silently induce feedback loops.
  • Misclassification-based performativity is typically easier to detect than simple or model-based performativity because only one parameter corner is truly non-performative.
  • Balanced outcome distributions and larger effect sizes sharply reduce the required sample size, as shown on both synthetic heatmaps and the Open Bandits case study.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same intervention budget that fails to detect weak performativity may still be large enough to train a useful predictor, creating a practical tension between detection and modelling.
  • Extending OPAB to continuous or multi-class predictions would require replacing the two-cell contingency test with a higher-dimensional analogue whose sample complexity grows faster.
  • Regions of indistinguishability supply a concrete quantitative criterion for deciding when non-interventional causal methods become necessary despite their lower sample efficiency.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 7 minor

Summary. The paper proposes OPAB, an offline A/B-style procedure that randomly assigns binary predictions (do-interventions) and tests whether the resulting outcome distributions P(Y|do(Ŷ=0)) and P(Y|do(Ŷ=1)) differ. Under three Nature models from prior work (Simple, Model-based, Misclassification-based), it rearranges expected contingency tables under a chi-squared test to obtain closed-form per-arm sample sizes N (Eqs. 9, 12, 14), introduces “regions of indistinguishability,” validates detection-rate heatmaps on UCI-style data imputed with those models, and reports a case study on the Open Bandits Dataset (men vs women). The central claim is that significant marginal dissimilarity under random prediction interventions detects outcome performativity, and that the derived N formulas guide when detection is practically feasible.

Significance. If the result holds in the stated scope, the contribution is practically useful: an offline, pre-deployment test for outcome performativity with explicit sample-size guidance and a clear notion of when allotted interventions are insufficient. Strengths include reproducible code, transparent contingency-table derivations in Appendix B, empirical heatmaps that track the formulas, a real-data case study, and an explicit comparison to the online Adam detector. The work sits at a genuine gap between performative-prediction theory and applied detection. The main scientific value is the formalization of intervention testing for this setting and the indistinguishability framing for scarce or ethically constrained labels—not a high-probability learning-theoretic bound.

major comments (4)
  1. [Methodology after Eq. 6; §6; Abstract] Methodology (text after Eq. 6) and Alg. 1 reduce the causal contrast from H(P(Y|X,do(Ŷ))) to H(P(Y|do(Ŷ))) by assuming performative effects are uniform in X. By the paper’s own definition (Eqs. 1–2), Nature is outcome-performative whenever p*(x,ŷ) depends on ŷ for some x; if effects cancel so E_x[p*(x,0)]≈E_x[p*(x,1)], the two-group test has no power. §6 correctly states that OPAB then cannot establish absence of performativity, but the abstract, RQs, and sample-size claims still present marginal dissimilarity as detecting outcome performativity in general. Please tighten all claim language to “detects average/marginal outcome performativity under random interventions,” and either (i) add a synthetic probe where heterogeneous p*(x,ŷ) cancels in the marginals, or (ii) state as a hard scope limitation that non-significance is inconclusive whenever uniformity may fail.
  2. [§4; Eqs. 9, 12, 14; Appendix B] Appendix B and Eqs. 9, 12, 14 are standard expected-contingency power calculations: plug average cell counts into χ² and rearrange for N at a fixed critical value (e.g. 3.841). They are not high-probability PAC-style sample-complexity bounds over draws of (X,Y), nor do they account for estimation error in p_β or p0, finite-sample χ² approximations, or multiple testing. Calling them “sample complexity bounds” throughout (title of §4, abstract, contributions) overstates the result. Rename to sample-size / power formulas under the expected contingency table, and note that empirical heatmaps validate those formulas rather than distribution-free bounds.
  3. [§4.1–4.3; Figs. 2–5] Empirical validation only imputes Simple/Model-based/Misclassification Nature models that open large marginal gaps (Figs. 2–5, 11–12). Combined with the uniformity reduction, this does not stress-test the failure mode the skeptic identifies (cancellation or X-heterogeneous effects with matched marginals). At least one negative-control configuration—and, for Model-based, a setting where β induces opposing shifts in different regions of X that average out—should be reported so readers can see when OPAB’s detection rate collapses despite true p* depending on ŷ.
  4. [§5; Tables 1–2] Open Bandits case study (Table 1): men detection rate reaches only ~0.52 at N=10^5 with Φ=0.004 and extreme class imbalance (~0.05% clicks). The paper attributes this to small effect size, which is fair, but then the case study only weakly supports “efficacy of OPAB on real data.” Either strengthen with a clearer effect-size / power calculation tied to Eq. 9 using the observed click rates under left vs right placement, or frame the study primarily as a false-positive check (women) plus a low-power real-world stress test, not as positive evidence of reliable detection.
minor comments (7)
  1. [Title page] Title page / affiliation: “8 Hiedelberglaan” appears to be a typo for Heidelberglaan.
  2. [Abstract; §1; §6] Throughout: “performateness,” “Outcome Performativeness,” and similar variants; standardize to “performativity.”
  3. [§3] Eq. 3 invokes Bayes in a form that is not needed for the later do-intervention argument; either use it cleanly or drop it to avoid confusion with the causal diagram in Fig. 1.
  4. [Fig. 2; §4] Fig. 2 caption says 25 replicates while §4 says 100 replicates unless stated otherwise; align the text.
  5. [Alg. 1; Eqs. 9, 12, 14] Alg. 1: groups A and B are length-2 frequency tables; clarify indexing (y∈{0,1}) and that N is random under Bernoulli(0.5) assignment rather than fixed balanced arms (the formulas assume balanced N).
  6. [Appendix D.6] Appendix D.6 Adam comparison: reimplementation without original code is useful but should be labeled as such more prominently in the main text when claiming superiority in sample efficiency.
  7. [Header] arXiv ID in the banner (2607.26908) and “29 Jul 2026” look like placeholders; verify before camera-ready.

Circularity Check

0 steps flagged

No significant circularity: OPAB and the sample-complexity bounds are standard intervention tests plus chi-squared power calculations under externally defined Nature models.

full rationale

The derivation chain does not reduce outputs to inputs by construction. Outcome Performativity is defined via Nature p*(x,ŷ) from prior external work (Kim & Perdomo, Mendler-Dünner, Adam et al.); OPAB is a do-intervention plus a two-group hypothesis test on marginal outcome distributions; and Eqs. 9, 12, and 14 are ordinary rearrangements of the chi-squared statistic on expected contingency tables under those Nature parametrizations (Appendix B). Empirical heatmaps simulate the same stated models and check detection rates against the closed forms—standard theory validation, not fitting a parameter and renaming it a prediction. The Open Bandits case study uses external logged random placements. The uniformity reduction after Eq. 6 is an explicit modeling assumption (and a limitation the paper states in §6), not a circular definition of performativity. Self-citation to the authors’ AAAI performative-drift paper [6] appears only as related online work and is not a premise of the OPAB bounds or detection claim. Score 1 reflects only that minor non-load-bearing self-reference; the central claims are independently derived.

Axiom & Free-Parameter Ledger

5 free parameters · 6 axioms · 2 invented entities

The central detector rests on standard causal intervention and contingency-table testing, plus a paper-specific uniformity reduction and three borrowed Nature models. Free parameters are significance thresholds and the performativity strengths used in simulation; invented notions are the OPAB procedure and regions of indistinguishability as planning objects.

free parameters (5)
  • δ (Type-I threshold) = 0.05 (χ²_δ=3.841, 1 d.f.)
    Fixed at 0.05 for main experiments; drives χ² critical value and thus all N formulas. Varied in appendix only.
  • α0, α1 (Simple Nature) = grid in [0,1]
    Bernoulli outcome probabilities per prediction arm; scanned on a grid to map detection rate and indistinguishability.
  • β0, β1 (Model-based strength) = grid in [-1,1]
    Shift magnitudes along linear-model weights when imputing performativity; chosen by experimenters, not estimated from real OP data.
  • γ0, γ1 (Misclassification response) = grid in [0,1]
    Probability desired outcome still realizes under misclassification; grid-scanned for heatmaps.
  • Intervention budget N̄ / 10% subsample = e.g. N∈{10,100,1000,10000}; 10% of dataset size
    Allotted per-arm samples (or 10% of each dataset) chosen for practicality; defines empirical regions of indistinguishability.
axioms (6)
  • domain assumption Binary outcomes Y∈{0,1} drawn as y~Bernoulli(p*(x,ŷ)) with Nature p*: X×Ŷ→[0,1].
    Stated in §1–3 and used throughout sample-complexity derivations.
  • domain assumption Random do-intervention ŷ~Bernoulli(0.5) breaks dependence of Ŷ on X, enabling a valid two-group comparison.
    Core causal justification (Eq. 6–7, Fig. 1b, Appendix A); standard RCT logic.
  • ad hoc to paper Performative effects are uniform in X, so testing P(Y|do(Ŷ)) suffices instead of P(Y|X,do(Ŷ)).
    Explicit modeling choice after Eq. 6 to enable simple contingency tests and closed-form N; authors flag failure modes in §6.
  • standard math Chi-squared (or Fisher when zeros appear) on the 2×2 outcome×prediction table is the detection rule; N formulas use expected cell counts under that statistic.
    Assumption list in Appendix B; power-analysis rearrangement yields Eqs. 9, 12, 14.
  • domain assumption Three Nature classes (Simple α, Model-based f_θ(x+βŷ·θ), Misclassification γ) adequately represent outcome performativity for complexity analysis.
    Imported from Kim & Perdomo / Mendler-Dünner and Adam et al.; used as the assumption classes in RQ1–3.
  • domain assumption Observed labels are reliable and outcomes are available without verification latency for the batch given to OPAB.
    Stated as limitations in §6; if false, Type I/II error rates of OPAB degrade.
invented entities (2)
  • OPAB (Outcome Performativity A/B Detection) independent evidence
    purpose: Name and algorithm for offline random-prediction intervention testing of outcome performativity.
    Procedural contribution (Alg. 1); not a physical entity, but the paper’s primary invented method object.
  • Regions of indistinguishability no independent evidence
    purpose: Set of Nature parameters where allotted per-arm budget N̄ is below the formula’s required N, so detection is unreliable.
    Definition Eq. 10; organizing concept for practical infeasibility, derived from the sample-size formulas rather than external measurement.

pith-pipeline@v1.2.0-daily-grok45 · 33356 in / 3945 out tokens · 84815 ms · 2026-07-30T17:22:52.567170+00:00 · methodology

0 comments
read the original abstract

In many domains such as Palliative Care, Credit Assignment and Recommender Systems, predictions may causally influence the outcomes they predict. This phenomena is known as Outcome Performativity. This paper formalises an approach for detecting Outcome Performativity using prediction intervention called Outcome Performativity A/B Detection (OPAB). OPAB enables the detection of Outcome Performativity by assessing the dissimilarity in outcome distributions produced by different predictions groups (interventions). If that dissimilarity is significant, Outcome Performativity is detected. We derive sample complexity bounds for OPAB under various Outcome Performative assumption classes which we empirically validate. Results show that detecting Outcome Performativity using OPAB is achievable in numerous cases. Results also show the presence of regions of indistinguishability which describe settings where the allotted number of interventions are insufficient for detecting Outcome Performativity. The results of which have broader practical implications for the detectability of Outcome Performativity in settings where samples are scarce, cost-prohibitive or potentially unethical to obtain. The paper concludes with a case study on the efficacy of OPAB on the Open Bandits dataset, and provides directions for future work.

Figures

Figures reproduced from arXiv: 2607.26908 by Brandon Gower-Winter, Georg Krempl.

Figure 1
Figure 1. Figure 1: Causal diagrams of Outcome Performative with (a) no interventions and (b) OPAB (our method) applied. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: shows the detection rate of OPAB in the Simple Outcome Performativity setting as the allotted interventions [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: A visualization of the Model-based Outcome Performativity Model on a two dimensional feature space. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: shows the detection rate of OPAB in Model-based Outcome Performativity settings for different parameter [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: shows the detection rate of OPAB in Misclassification-based Outcome Performativity settings. Black cells [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: A visualization of the motivation behind OP-AB. The first row shows the true distribution [PITH_FULL_IMAGE:figures/full_fig_p012_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: The estimated number of instances needed per group ( [PITH_FULL_IMAGE:figures/full_fig_p014_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: The estimated number of instances needed per group ( [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: The estimated number of instances needed per group ( [PITH_FULL_IMAGE:figures/full_fig_p016_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Visualizations of Equation 14 across varying values of [PITH_FULL_IMAGE:figures/full_fig_p016_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Results of applying OP-AB to detect misclassification-based Outcome Performativity. These results are [PITH_FULL_IMAGE:figures/full_fig_p018_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Results of applying OP-AB to detect model-based Outcome Performativity. These results are in keeping [PITH_FULL_IMAGE:figures/full_fig_p018_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Results of using various Hypothesis Tests with OP-AB to detect Outcome Performativity. These results [PITH_FULL_IMAGE:figures/full_fig_p019_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Results from exploring various detection thresholds [PITH_FULL_IMAGE:figures/full_fig_p020_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Results demonstrating the importance the number of instances per intervention ( [PITH_FULL_IMAGE:figures/full_fig_p021_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Adam Detection performed on the Breast Cancer (a), Adult Census (b) and Kickstarter (c) datasets. Overall, the detection rate of Adam Detection is inconsistent. This is partly due to the accuracy of the deployed classifiers which when high, rarely produce misclassifications making detection difficult. The number of instances in each dataset also play a role with more instances increasing the likelihood of… view at source ↗
Figure 17
Figure 17. Figure 17: Figure (a) shows the average accuracy achieved by a Linear Model (trained using standard Supervised [PITH_FULL_IMAGE:figures/full_fig_p022_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Figure showing how the same model used in Figure 17b can be used to produce varying results over the same [PITH_FULL_IMAGE:figures/full_fig_p023_18.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

25 extracted references · 1 linked inside Pith

  1. [1]

    Performative prediction

    Juan Perdomo, Tijana Zrnic, Celestine Mendler-Dünner, and Moritz Hardt. Performative prediction. In Hal Daumé III and Aarti Singh, editors,Proceedings of the 37th International Conference on Machine Learning, volume 119, pages 7599–7609. PMLR, 2020

    2020

  2. [2]

    Causal inference out of control: estimating per- formativity without treatment randomization

    Gary Cheng, Moritz Hardt, and Celestine Mendler-Dünner. Causal inference out of control: estimating per- formativity without treatment randomization. InForty-first International Conference on Machine Learning, 2024

    2024

  3. [3]

    Hidden risks of machine learning applied to healthcare: unintended feedback loops between models and future data causing model degradation

    George Alexandru Adam, Chun-Hao Kingsley Chang, Benjamin Haibe-Kains, and Anna Goldenberg. Hidden risks of machine learning applied to healthcare: unintended feedback loops between models and future data causing model degradation. InMachine Learning for Healthcare Conference, pages 710–731. PMLR, 2020. 23 Anonymous Algorithm 2Simple Pseudocode for using O...

    2020

  4. [4]

    When accurate prediction models yield harmful self-fulfilling prophecies.Patterns, 6(4), 2025

    Wouter AC van Amsterdam, Nan van Geloven, Jesse H Krijthe, Rajesh Ranganath, and Giovanni Ciná. When accurate prediction models yield harmful self-fulfilling prophecies.Patterns, 6(4), 2025

    2025

  5. [5]

    Making decisions under outcome performativity

    Michael P Kim and Juan C Perdomo. Making decisions under outcome performativity. In14th Innovations in Theoretical Computer Science Conference (ITCS 2023), pages 79–1. Schloss Dagstuhl–Leibniz-Zentrum für Informatik, 2023

    2023

  6. [6]

    Identifying predictions that influence the future: Detecting performative concept drift in data streams

    Brandon Gower-Winter, Georg Krempl, Sergey Dragomiretskiy, Tineke Jelsma, and Arno Siebes. Identifying predictions that influence the future: Detecting performative concept drift in data streams. InProceedings of the AAAI Conference on Artificial Intelligence, 2025

    2025

  7. [7]

    What are randomised controlled trials good for?Philosophical studies, 147(1):59–70, 2010

    Nancy Cartwright. What are randomised controlled trials good for?Philosophical studies, 147(1):59–70, 2010

    2010

  8. [8]

    Cambridge university press, 2009

    Judea Pearl.Causality. Cambridge university press, 2009

    2009

  9. [9]

    Anticipating performativity by predicting from predictions.Advances in neural information processing systems, 35:31171–31185, 2022

    Celestine Mendler-Dünner, Frances Ding, and Yixin Wang. Anticipating performativity by predicting from predictions.Advances in neural information processing systems, 35:31171–31185, 2022

    2022

  10. [10]

    Error amplification when updating deployed machine learning models

    George Alexandru Adam, Chun-Hao Kingsley Chang, Benjamin Haibe-Kains, and Anna Goldenberg. Error amplification when updating deployed machine learning models. InMachine Learning for Healthcare Conference, pages 715–740. PMLR, 2022

    2022

  11. [11]

    A classification of feedback loops and their relation to biases in automated decision-making systems

    Nicolò Pagan, Joachim Baumann, Ezzat Elokda, Giulia De Pasquale, Saverio Bolognani, and Anikó Hannák. A classification of feedback loops and their relation to biases in automated decision-making systems. InProceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization, pages 1–14, 2023

    2023

  12. [12]

    Model updating after interventions paradoxically introduces bias

    James Liley, Samuel Emerson, Bilal Mateen, Catalina Vallejos, Louis Aslett, and Sebastian V ollmer. Model updating after interventions paradoxically introduces bias. InInternational Conference on Artificial Intelligence and Statistics, pages 3916–3924. PMLR, 2021

    2021

  13. [13]

    Metalearners for estimating heterogeneous treatment effects using machine learning.Proceedings of the national academy of sciences, 116(10):4156–4165, 2019

    Sören R Künzel, Jasjeet S Sekhon, Peter J Bickel, and Bin Yu. Metalearners for estimating heterogeneous treatment effects using machine learning.Proceedings of the national academy of sciences, 116(10):4156–4165, 2019

    2019

  14. [14]

    Double/debiased machine learning for treatment and structural parameters.The Econometrics Journal, 21(1), 2018

    Victor Chernozhukov, Denis Chetverikov, Mert Demirer, Esther Duflo, Christian Hansen, Whitney Newey, and James Robins. Double/debiased machine learning for treatment and structural parameters.The Econometrics Journal, 21(1), 2018

    2018

  15. [15]

    Causal inference with complex treatments: A survey.ACM Computing Surveys, 58(9):1–36, 2026

    Yingrong Wang, Haoxuan Li, Minqin Zhu, Anpeng Wu, Baohong Li, Keting Yin, Ruoxuan Xiong, Fei Wu, and Kun Kuang. Causal inference with complex treatments: A survey.ACM Computing Surveys, 58(9):1–36, 2026

    2026

  16. [16]

    Breast Cancer, 1988

    Matjaz Zwitter and Milan Soklic. Breast Cancer, 1988. Accessed: April 2025

    1988

  17. [17]

    Using the adap learning algorithm to forecast the onset of diabetes mellitus

    Jack W Smith, James E Everhart, William C Dickson, William C Knowler, and Robert Scott Johannes. Using the adap learning algorithm to forecast the onset of diabetes mellitus. InProceedings of the annual symposium on computer application in medical care, page 261, 1988. Accessed: April 2025

    1988

  18. [18]

    Adult census income, 1996

    Barry Becker and Ronny Kohavi. Adult census income, 1996. Accessed: April 2025

    1996

  19. [19]

    Funding successful projects on kickstarter, 2017

    Lathwal. Funding successful projects on kickstarter, 2017. Accessed: April 2025

    2017

  20. [20]

    Titanic - machine learning from disaster, 2012

    Will Cukierski. Titanic - machine learning from disaster, 2012. Accessed: April 2025. 24 Detecting Outcome Performativity Using Intervention Testing

    2012

  21. [21]

    Loan approval prediction dataset, 2023

    Kai. Loan approval prediction dataset, 2023. Accessed: August 2024

    2023

  22. [22]

    Large-scale open dataset, pipeline, and benchmark for bandit algorithms.arXiv preprint arXiv:2008.07146, 2020

    Yuta Saito, Aihara Shunsuke, Matsutani Megumi, and Narita Yusuke. Large-scale open dataset, pipeline, and benchmark for bandit algorithms.arXiv preprint arXiv:2008.07146, 2020

    Pith/arXiv arXiv 2008

  23. [23]

    Open challenges for data stream mining research.ACM SIGKDD Explorations Newsletter, 16(1):1–10, 2014

    Georg Krempl, Indre Žliobaite, Dariusz Brzezi´nski, Eyke Hüllermeier, Mark Last, Vincent Lemaire, Tino Noack, Ammar Shaker, Sonja Sievi, Myra Spiliopoulou, et al. Open challenges for data stream mining research.ACM SIGKDD Explorations Newsletter, 16(1):1–10, 2014

    2014

  24. [24]

    Weakly supervised machine learning.CAAI Transactions on Intelligence Technology, 8(3):549–580, 2023

    Zeyu Ren, Shuihua Wang, and Yudong Zhang. Weakly supervised machine learning.CAAI Transactions on Intelligence Technology, 8(3):549–580, 2023

    2023

  25. [25]

    A survey on concept drift adaptation.ACM computing surveys (CSUR), 46(4):1–37, 2014

    João Gama, Indr˙e Žliobait˙e, Albert Bifet, Mykola Pechenizkiy, and Abdelhamid Bouchachia. A survey on concept drift adaptation.ACM computing surveys (CSUR), 46(4):1–37, 2014. 25

    2014