Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

FairSense: Long-Term Fairness Analysis of ML-Enabled Systems

T0 review · 3 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FairSense simulates feedback loops to expose what drives long-term unfairness

desk verdict FairSense is a solid, honestly-scoped design-time what-if tool for long-term fairness; the case-study conclusions inherit the environment models from prior work, and the consistency checks are partly circular. read the letter →

arxiv 2501.01665 v1 pith:6SYP75RJ submitted 2025-01-03 cs.LG cs.CYcs.SE

Yining She , Sumon Biswas , Christian Kästner , Eunsuk Kang This is my paper · ORCID
classification cs.LGcs.CYcs.SE
keywords long-termfairnessfeedbackloopsMonte-CarlosimulationsensitivityanalysisML-enabledsystemsalgorithmicdistributionshiftcoveringarraysampling
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FairSense is a simulation-based approach to a blind spot in algorithmic fairness: ML systems and the environments they act on can form feedback loops, so a model that looks fair today can drift into unfairness tomorrow. The paper argues that long-term fairness cannot be assessed by static, model-centric testing, and offers a framework that simulates how a system's decisions reshape its inputs over time and then ranks system and environmental parameters by how much they influence long-term fairness. The framework is demonstrated on loan lending, opioid risk scoring, and predictive policing, where it finds that only 10–40% of the parameters matter and that design choices such as the agent policy or the ML model can dominate the outcome. The point of the analysis is practical: developers can spend design and monitoring effort on the few levers that actually change long-term fairness.

What carries the argument

The central object is a feedback-loop model with five components: the ML model M, the decision-maker D, the environment state Q, a stochastic distribution-shift function S : Q × D → Δ(Q), and a projection function P : Q → I that samples the population seen by the system. These components carry the argument because they turn 'long-term fairness' from a static model property into a trace property of a stochastic dynamical system. On top of the model, FairSense runs Monte-Carlo simulation per configuration to produce evolution traces, evaluates them with average-increase and maximum-increase unfairness metrics, and then fits a standardized multiple linear regression with pairwise interaction terms, using ANOVA and η² effect sizes to rank parameters. A covering-array sampling heuristic chooses a subset of configurations whose pairwise combinations cover all parameter values, reducing simulation effort while preserving the regression ranking.

What would settle it

Collect longitudinal data from a deployed loan-lending system over several years, tracking each applicant's credit score change after approval and rejection, and compare the realized distribution shift with the simulated one; if the realized shift reverses the simulated eta-squared ranking of parameters (for instance, making the approval threshold matter more than the agent policy), the central claim that the framework identifies the truly influential parameters fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that long-term unfairness is a property of the whole feedback loop—system decisions, environment response, and future inputs—not of the ML model in isolation, and that this property can be probed before deployment by simulation. FairSense models the loop as a stochastic distribution-shift function S that maps a system decision and current environment state to a distribution over next states, with a projection function P feeding observable population samples back into the model. Monte-Carlo simulation generates many evolution traces per configuration, and two metrics—average and maximum increase in unfairness over a trace—quantify long-term fairness. A standardized multiple linear regression with ANOVA effect sizes then ranks every system and environmental parameter by its influence. In the three case studies the ranking is highly concentrated: the choice of agent explains 76% of variance in loan-lending unfairness, the ML model choice explains 97% in opioid risk scoring, and the non-hotspot discovery rate explains 69% in predictive policing. The paper concludes that design effort can be focused on a small subset of parameters, and that some apparently fair short-term policies, such as the equal-opportunity agent, are less fair in the long run than utility-maximizing ones.

Load-bearing premise

The whole analysis inherits the fidelity of the environment model: the distribution-shift and projection functions are taken from prior work and are not validated against real-world observations, so if credit scores, hospital visits, or crime discovery actually respond to decisions differently than modeled, the parameter rankings and fairness conclusions could change.

Editorial extensions

If this is right

  • In loan lending, max-utility agents are predicted to be fairer over time than equal-opportunity agents, and conservative bank utility parameters improve long-term fairness, so design effort can concentrate on the agent choice rather than ten other parameters.
  • In opioid risk scoring, the choice between XGBoost and MLP dominates long-term fairness, making model selection more consequential than the hospital-visit or prescription shift functions at design time.
  • In predictive policing, raising the crime discovery rate outside predicted hotspots is the strongest lever for long-term fairness, supporting a strategy that distributes patrols beyond the hotspot cells.
  • Covering-array sampling with 2-coverage reduces the number of simulated configurations to 5–33% of the full space while keeping regression fit and term rankings close to the baseline, making the analysis feasible for larger design spaces.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the small-subset finding generalizes beyond the three case studies, design-time fairness analysis should start by identifying the dominant parameter class—agent, model, or environment—rather than tuning all knobs; this is an extension the paper suggests but does not test beyond its own cases.
  • Because the environment models are unvalidated against real-world data, the method is best read as a what-if ranking tool: the parameter rankings are conditional on the shift and projection functions being right, so pairing FairSense with runtime monitoring of observed distribution shifts would be a natural next step.
  • The same simulation-plus-sensitivity pipeline could be applied to other self-reinforcing domains, such as content recommendation, hiring, or dynamic pricing, where feedback loops are suspected but long-term fairness data is scarce; this is an editorial inference, not a claim in the paper.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes FairSense, a simulation-based framework for analyzing long-term fairness of ML-enabled systems that interact with their environment through feedback loops. Given system and environmental parameters, a fairness criterion, and an environmental model consisting of a target dataset, a distribution-shift function, and a projection function, FairSense performs Monte-Carlo simulation to generate evolution traces for each configuration, evaluates long-term fairness metrics (average and maximum increase in unfairness), and then applies standardized multiple linear regression with ANOVA to rank parameters and interactions by their influence on long-term fairness. A covering-array sampling heuristic is introduced to reduce the number of configurations simulated. The framework is demonstrated on three case studies: loan lending, opioid risk scoring, and predictive policing, with reported R2 values between 0.738 and 0.995, Pareto trade-off analyses, and sampling-efficiency comparisons. The authors provide a replication package and explicitly discuss threats to validity, including the lack of real-world validation of the environment models.

Significance. If the framework's conclusions are trustworthy, FairSense fills a genuine gap: most fairness analysis is static and model-centric, while the paper addresses the system-level, dynamic feedback loops that can cause fairness violations to emerge over time. The framework is concrete and actionable for design-time analysis, and the case studies are drawn from real prior work with publicly available data. The paper's strengths include a clear conceptual model, a complete prototype with a replication package, high-variance-explained regression fits on the simulation outputs, and an honest, explicit treatment of the main limitation: the environment models are inherited from prior studies and are not validated against real-world observations. The central risk is that the parameter rankings and the headline 'only 10-40% of parameters matter' summary are conditional on the fidelity of those environment models and on the chosen parameter ranges; the consistency checks in Section IX are not independent of the models used to build the simulations.

major comments (3)
  1. [Section IX] The consistency check against prior work is not an independent validation of the environment models. For loan lending, the environment is built from Liu et al. [2] and D'Amour et al. [19], and the main finding (max-util agent reduces long-term unfairness more than eq-op) is the same result as Liu et al.'s Theorem 3.4. For predictive policing, the model is from Akpinar et al. [29] and the comparison is with the closely related mechanistic analysis of Ensign et al. [3]. This supports internal consistency but not external validity. Because the central value of FairSense is to rank parameters by their influence on long-term fairness, and because Section IX explicitly states that the simulation is not validated against the real world, the paper should either provide a genuinely independent check (e.g., a case where the environment model is not the same as the comparison source) or carefully restrict the claims to statements of the form 'under the assumed environment model.' As written, the abstract's claim to 'detect and analyze long-term unfairness in ML-enabled systems' is stronger than the evidence supports.
  2. [Section VIII-A and Table I] The parameter rankings and the summary that only 10-40% of parameters are impactful depend on the specific ranges and levels chosen for each parameter. For example, in Table I the ranges 'Score update-repay: 8, 12 ... 20', 'Score update-default: -40, -32 ... -16', and 'Bank utility func param: -10, -9 ... -3' appear to be chosen without a stated domain justification. Eta-squared and sum-of-squares effect sizes are scale- and range-dependent: widening the range of one parameter will generally increase its apparent influence relative to others. The paper should justify these ranges from the cited prior work or, more convincingly, conduct a robustness analysis in which the ranges are varied and the ranking stability is reported. Without this, the central sensitivity-analysis conclusions are conditional on arbitrary discretization choices.
  3. [Section VI-A and Eq. (3)] The text states that a 'standardized multiple linear regression model' is used, but Tables II-IV report coefficients such as -2.51E-02 and one-hot dummy variables for categorical parameters, and the description does not specify how standardization was applied or how the categorical reference levels were chosen. These choices affect the comparability of coefficients, the ANOVA sums of squares, and hence the reported eta-squared rankings. The paper should clarify the exact model fitting procedure, including whether numeric parameters were standardized before forming interaction terms, how one-hot encoding interacts with standardization, and which type of ANOVA sums of squares (I, II, or III) was used. This is a load-bearing methodological detail for the main sensitivity-analysis result.
minor comments (6)
  1. [Section II] There is a typo: 'including including demographic parity' should read 'including demographic parity.'
  2. [Section IV-A] The text contains 'mGiven input data' near the beginning of the section; this should be 'Given input data.'
  3. [Section V-A] The stopping rule uses the coefficient of variation, Mean(LF(T)) in the denominator. If the mean long-term fairness is close to zero, the criterion may never be satisfied; the paper should state how this case is handled.
  4. [Section VII-A] The sentence 'Detailed dynamics are described in Section IV.A' is misleading because Section IV-A is the general feedback-loop model, not the loan-specific dynamics; please point to the supplemental material or the specific case-study description.
  5. [Table V] The predictive policing row shows empty entries for the 3-coverage column; this should be explained in the caption or text, since the reason (only three parameters make 3-coverage equivalent to the full enumeration) is not stated.
  6. [Section VIII-A Summary] The claim 'only a small subset (10-40%) of the parameters impact long-term fairness' would be clearer if the denominators were explicit (e.g., 5 of 15 terms for loan lending, 1 of 10 for opioid risk scoring, 2 of 5 for predictive policing), since different baselines are being compared.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the sensitivity analysis is a descriptive fit to simulation traces, and the environment-fidelity caveat is an acknowledged validity limitation rather than a circular derivation.

full rationale

FairSense's derivation chain is self-contained relative to its stated method. Section VI-A states: 'The idea is to learn a regression model that explains how the response is influenced by different options. In our case, the response is the long-term fairness measured in simulation and the options are possible values for the system and environment parameters.' The regression coefficients are therefore a descriptive summary of the simulation output, not a prediction of an external quantity, so reporting them as parameter influences is not a hidden reuse of the target result. The case-study environment models are imported from external prior work (Liu et al., D'Amour et al., Akpinar et al., Ensign et al.), and the paper explicitly acknowledges in Section IX that 'we do not validate the accuracy of the simulation with regard to the real world' and lists environment-model fidelity as a threat to validity. The consistency check against Liu et al. and Ensign et al. is weakened by shared model provenance, but the paper presents the case studies as what-if demonstrations of the framework rather than as independent empirical validation, so this is a fidelity limitation, not a circular step in the framework's logic. The only self-citations ([23], [30], [94], [99]) appear in background and related-work discussions and are not load-bearing for the central sensitivity analysis. No equation is defined in terms of the target result, and no fitted parameter is renamed as a prediction.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

The paper explores ranges of system and environmental parameters rather than fitting constants to a target result. The Monte-Carlo stopping thresholds (95% confidence, 5% relative error) and eta-squared significance thresholds are standard methodological choices, not free parameters of the central claim. No new physical or conceptual entities are postulated; the feedback-loop model is a formalization of known system-environment interactions.

assumptions (5)
  • domain assumption The environment can be modeled as a stateful entity Q with a stochastic distribution-shift function S and a projection function P that determines observable inputs.
    Introduced in Section IV-A; the entire simulation framework depends on this formalization of feedback loops between the ML system and its environment.
  • domain assumption The case-study environment dynamics (credit score updates, hospital visit changes, crime incident discovery) are faithful enough for meaningful what-if analysis.
    Section VII and Section IX; the authors state these are inferred from prior work and that accuracy against the real world is not validated.
  • domain assumption A standardized multiple linear regression with pairwise interaction terms adequately captures the relationship between configuration parameters and long-term fairness.
    Section VI-A; the reported R2 values (0.970, 0.995, 0.738) support fit within the explored configuration space but do not prove the functional form holds elsewhere.
  • standard math Monte-Carlo simulation with the stated stopping rule produces stable estimates of long-term fairness for each configuration.
    Section V-A; the stopping rule is a standard relative-error criterion, though it can be unstable when the mean unfairness increase is near zero.
  • domain assumption Covering array sampling with coverage 2 or 3 preserves the ranking of influential parameters from the full configuration space.
    Section VI-B and RQ3; empirically evaluated on three case studies, not guaranteed in general.

how reviews work

0 comments
Cite this review

Pith. "Pith review of FairSense: Long-Term Fairness Analysis of ML-Enabled Systems." pith.science (2026). https://pith.science/paper/6SYP75RJ

@misc{pith2026250101665,
  author       = {Pith},
  title        = {Pith review of: FairSense: Long-Term Fairness Analysis of ML-Enabled Systems},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/6SYP75RJ}},
  note         = {Machine review of arXiv:2501.01665}
}
read the original abstract

Algorithmic fairness of machine learning (ML) models has raised significant concern in the recent years. Many testing, verification, and bias mitigation techniques have been proposed to identify and reduce fairness issues in ML models. The existing methods are model-centric and designed to detect fairness issues under static settings. However, many ML-enabled systems operate in a dynamic environment where the predictive decisions made by the system impact the environment, which in turn affects future decision-making. Such a self-reinforcing feedback loop can cause fairness violations in the long term, even if the immediate outcomes are fair. In this paper, we propose a simulation-based framework called FairSense to detect and analyze long-term unfairness in ML-enabled systems. Given a fairness requirement, FairSense performs Monte-Carlo simulation to enumerate evolution traces for each system configuration. Then, FairSense performs sensitivity analysis on the space of possible configurations to understand the impact of design options and environmental factors on the long-term fairness of the system. We demonstrate FairSense's potential utility through three real-world case studies: Loan lending, opioids risk scoring, and predictive policing.

Figures

Figures reproduced from arXiv: 2501.01665 by the authors.

Figure 1
Figure 1. A developer using FAIRSENSE specifies three types of inputs: (1) system parameters, which describe the space of configuration options (e.g., type of ML model and agent policies), to be explored, (2) desired fairness criteria, and (3) an environmental model, where the environmental parameters control the dynamics of environmental changes that are induced by the system’s decisions. The latter model itself consists of … view at source ↗
Figure 1
Figure 1. An overview of the FAIRSENSE approach shift model is stochastic, explicitly encoding uncertainty about how the environment may evolve in response to system output. Given these inputs, FAIRSENSE performs Monte-Carlo simulation [26] to systematically generate traces that show how the system and the environment may evolve together over time for a given configuration (i.e., an assignment of values to the system and envi… view at source ↗
Figure 2
Figure 2. A feedback loop created by ML-enabled loan lending system [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figures from the paper (3 more)
Figure 3
Figure 3. Figure 3: Feedback loop model of ML-enabled system [PITH_FULL_IMAGE:figures/full_fig_p003_3.png]
Figure 4
Figure 4. Figure 4: An evolution trace of loan lending system showing [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: The radar plots visualizing trade-offs in three Pareto-optimal configurations for each case study. All values were scaled [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Rethinking Autonomy: Preventing Failures in AI-Driven Software Engineering

    cs.SE 2025-08 reject novelty 3.0 of 10

    A survey-plus-benchmark argues that current code LLMs fail safety thresholds and need stronger governance, but the supporting experiment is incomplete.

Reference graph

Works this paper leans on

107 extracted references · 74 canonical work pages · cited by 1 Pith paper

  1. [2]

    Delayed impact of fair machine learning,

    L. T. Liu, S. Dean, E. Rolf, M. Simchowitz, and M. Hardt, “Delayed impact of fair machine learning,” in ICML, 2018

    2018

  2. [19]

    Fairness is not static: deeper understanding of long term fairness via simulation studies,

    A. D’Amour, H. Srinivasan, J. Atwood, P. Baljekar, D. Sculley, and Y . Halpern, “Fairness is not static: deeper understanding of long term fairness via simulation studies,” in Proceedings of the 2020 Conference on Fairness, Accountability, and Transparency , 2020

    2020

  3. [29]

    The effect of differential victim crime reporting on predictive policing systems,

    N.-J. Akpinar, M. De-Arteaga, and A. Chouldechova, “The effect of differential victim crime reporting on predictive policing systems,” in Proceedings of the ACM FAccT , 2021

    2021

  4. [3]

    Runaway feedback loops in predictive policing,

    D. Ensign, S. A. Friedler, S. Neville, C. Scheidegger, and S. Venkata- subramanian, “Runaway feedback loops in predictive policing,” in Conference on Fairness, Accountability and Transparency FAT , 2018

    2018

  5. [1]

    System safety engineering for social and ethical ML risks: A case study,

    E. Jatho, L. Mailloux, S. Rismani, E. Williams, and J. A. Kroll, “System safety engineering for social and ethical ML risks: A case study,” arXiv preprint arXiv:2211.04602, 2022

    arXiv 2022

  6. [4]

    On adaptive fairness in software systems,

    A. Farahani, L. Pasquale, A. Bennaceur, T. Welsh, and B. Nuseibeh, “On adaptive fairness in software systems,” in International Symposium on Software Engineering for Adaptive and Self-Managing Systems , 2021

    2021

  7. [5]

    Fairness testing: testing software for discrimination,

    S. Galhotra, Y . Brun, and A. Meliou, “Fairness testing: testing software for discrimination,” in ESEC/FSE, 2017

    2017

  8. [6]

    Do the machine learning models on a crowd sourced platform exhibit bias? an empirical study on model fairness,

    S. Biswas and H. Rajan, “Do the machine learning models on a crowd sourced platform exhibit bias? an empirical study on model fairness,” in Proceedings of the ESEC/FSE , 2020

    2020

Show all 107 references
  1. [7]

    Fair preprocessing: Towards understanding compositional fairness of data transformers in machine learning pipeline,

    S. Biswas and H. Rajan, “Fair preprocessing: Towards understanding compositional fairness of data transformers in machine learning pipeline,” in The 29th ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering , 2021

    2021

  2. [8]

    Black box fairness testing of machine learning models,

    A. Aggarwal, P. Lohia, S. Nagar, K. Dey, and D. Saha, “Black box fairness testing of machine learning models,” in Proceedings of the 27th ACM ESEC/FSE , 2019

    2019

  3. [9]

    Automated directed fairness testing,

    S. Udeshi, P. Arora, and S. Chattopadhyay, “Automated directed fairness testing,” in Proceedings of the 33rd ACM/IEEE International Conference on Automated Software Engineering , 2018

    2018

  4. [10]

    White-box fairness testing through adversarial sampling,

    P. Zhang, J. Wang, J. Sun, G. Dong, X. Wang, X. Wang, J. S. Dong, and T. Dai, “White-box fairness testing through adversarial sampling,” in Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, 2020, pp. 949–960

    2020

  5. [11]

    Fairify: Fairness verification of neural networks,

    S. Biswas and H. Rajan, “Fairify: Fairness verification of neural networks,” in ICSE’2023: The 45th International Conference on Software Engineering, 2023

    2023

  6. [12]

    Bias in machine learning software: Why? how? what to do?

    J. Chakraborty, S. Majumder, and T. Menzies, “Bias in machine learning software: Why? how? what to do?” ser. ESEC/FSE 2021, 2021

    2021

  7. [13]

    Fairway: A way to build fair ml software,

    J. Chakraborty, S. Majumder, Z. Yu, and T. Menzies, “Fairway: A way to build fair ml software,” in Proceedings of the 28th ESEC/FSE , 2020

    2020

  8. [14]

    Fix fairness, don’t ruin accuracy: Performance aware fairness repair using automl,

    G. Nguyen, S. Biswas, and H. Rajan, “Fix fairness, don’t ruin accuracy: Performance aware fairness repair using automl,” in ESEC/FSE’2023: The 31st ACM Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering , 2023

    2023

  9. [15]

    J. D. Sterman, Business dynamics: systems thinking and modeling for a complex world . McGraw-Hill, 2000

    2000

  10. [16]

    D. H. Meadows, Thinking in systems: A primer . chelsea green publishing, 2008

    2008

  11. [17]

    O’Neil, Weapons of math destruction: How big data increases inequality and threatens democracy

    C. O’Neil, Weapons of math destruction: How big data increases inequality and threatens democracy . Crown, 2017

    2017

  12. [18]

    Concrete problems in ai safety,

    D. Amodei, C. Olah, J. Steinhardt, P. Christiano, J. Schulman, and D. Man ´e, “Concrete problems in ai safety,” 2016

    2016

  13. [20]

    Runtime monitoring of dynamic fairness properties,

    T. Henzinger, M. Karimi, K. Kueffner, and K. Mallik, “Runtime monitoring of dynamic fairness properties,” in Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency , 2023

    2023

  14. [21]

    Extending the machine learning abstraction boundary: A complex systems approach to incorporate societal context,

    D. Martin Jr, V . Prabhakaran, J. Kuhlberg, A. Smart, and W. S. Isaac, “Extending the machine learning abstraction boundary: A complex systems approach to incorporate societal context,” arXiv preprint arXiv:2006.09663, 2020

    2006 arXiv

  15. [22]

    A classification of feedback loops and their relation to biases in automated decision-making systems,

    N. Pagan, J. Baumann, E. Elokda, G. De Pasquale, S. Bolognani, and A. Hann ´ak, “A classification of feedback loops and their relation to biases in automated decision-making systems,” in Proceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and ...

    2023

  16. [23]

    Towards safe ML-based systems in presence of feedback loops,

    S. Biswas, Y . She, and E. Kang, “Towards safe ML-based systems in presence of feedback loops,” in Proceedings of the 1st International Workshop on Dependability and Trustworthiness of Safety-Critical Systems with Machine Learned Components , 2023

    2023

  17. [24]

    The world and the machine,

    M. Jackson, “The world and the machine,” in International Conference on Software Engineering (ICSE) . ACM, 1995, pp. 283–292

    1995

  18. [25]

    A reference model for requirements and specifications,

    C. A. Gunter, E. L. Gunter, M. Jackson, and P. Zave, “A reference model for requirements and specifications,” IEEE Softw., 2000

    2000

  19. [26]

    C. Z. Mooney, Monte Carlo Simulation . SAGE Publications, 1997

    1997

  20. [27]

    Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models,

    A. McCulloch, “Sensitivity Analysis in Practice: A Guide to Assessing Scientific Models,” Journal of the Royal Statistical Society Series A: Statistics in Society , 2005

    2005

  21. [28]

    Algorithmic fairness in predicting opioid use disorder using machine learning,

    A. E. Kilby, “Algorithmic fairness in predicting opioid use disorder using machine learning,” in Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency , 2021

    2021

  22. [30]

    The role of environmental deviations in engineering robust systems,

    E. Kang, “The role of environmental deviations in engineering robust systems,” in 2021 IEEE 29th International Requirements Engineering Conference Workshops (REW). IEEE, 2021, pp. 435–438

    2021

  23. [31]

    Fairness definitions explained,

    S. Verma and J. Rubin, “Fairness definitions explained,” in Proceedings of the International Workshop on Software Fairness, FairWare , 2018

    2018

  24. [32]

    Fairness through awareness,

    C. Dwork, M. Hardt, T. Pitassi, O. Reingold, and R. Zemel, “Fairness through awareness,” in Proceedings of the 3rd innovations in theoretical computer science conference , 2012

    2012

  25. [33]

    Equality of opportunity in supervised learning,

    M. Hardt, E. Price, and N. Srebro, “Equality of opportunity in supervised learning,” in Advances in neural information processing systems , 2016

    2016

  26. [34]

    Artificial intolerance,

    N. Byrnesarchive, “Artificial intolerance,” MIT Technology Review ,

  27. [35]

    Black loans matter: Distributionally robust fairness for fighting subgroup discrimination,

    M. Weber, M. Yurochkin, S. Botros, and V . Markov, “Black loans matter: Distributionally robust fairness for fighting subgroup discrimination,” arXiv preprint arXiv:2012.01193 , 2020

    2012 arXiv

  28. [36]

    Justice department secures over 31 million from city national bank to address lending discrimination allegations,

    D. of Justice, “Justice department secures over 31 million from city national bank to address lending discrimination allegations,” Office of Public Affairs, 2023

    2023

  29. [37]

    Apple card is accused of gender bias. here’s how that can happen,

    E. Nedlund, “Apple card is accused of gender bias. here’s how that can happen,” CNN Business, 2019. [Online]. Available: https://www. cnn.com/2019/11/12/business/apple-card-gender-bias/index.html

    2019

  30. [38]

    Fairtest: Discovering unwarranted associa- tions in data-driven applications,

    F. Tramer, V . Atlidakis, R. Geambasu, D. Hsu, J.-P. Hubaux, M. Hum- bert, A. Juels, and H. Lin, “Fairtest: Discovering unwarranted associa- tions in data-driven applications,” in 2017 IEEE EuroS&P , 2017

    2017

  31. [39]

    https://www.businessinsider.com/personal-finance/ what-is-hard-inquiry-how-affect-credit-score, 2023

    2023

  32. [40]

    A brief survey of stopping rules in monte carlo simulations,

    M. J. Gilman, “A brief survey of stopping rules in monte carlo simulations,” in Proceedings of the Second Conference on Applications of Simulations. Winter Simulation Conference, 1968

    1968

  33. [41]

    Saltelli, M

    A. Saltelli, M. Ratto, T. Andres, F. Campolongo, J. Cariboni, D. Gatelli, M. Saisana, and S. Tarantola, Global sensitivity analysis: the primer . John Wiley & Sons, 2008

    2008

  34. [42]

    Marrying fairness and explainability in supervised learning,

    P. A. Grabowicz, N. Perello, and A. Mishra, “Marrying fairness and explainability in supervised learning,” in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency , 2022

    2022

  35. [43]

    “how biased are your features?

    B. Ghosh, D. Basu, and K. S. Meel, ““how biased are your features?”: Computing fairness influence functions with global sensitivity analysis,” in Proceedings of the 2023 ACM Conference on FAccT , 2023

    2023

  36. [44]

    John Wiley & Sons, Ltd, 2007, ch

    Elementary Effects Method . John Wiley & Sons, Ltd, 2007, ch. 3

    2007

  37. [45]

    John Wiley & Sons, Ltd, 2007

    Variance-Based Methods. John Wiley & Sons, Ltd, 2007

    2007

  38. [46]

    Statistical methods for research workers,

    R. A. Fisher, “Statistical methods for research workers,” in Break- throughs in statistics: Methodology and distribution . Springer, 1970

    1970

  39. [47]

    Cohen, P

    J. Cohen, P. Cohen, S. G. West, and L. S. Aiken, Applied multiple regression/correlation analysis for the behavioral sciences , 2013

    2013

  40. [48]

    Hartman, Software and Hardware Testing Using Combinatorial Covering Suites

    A. Hartman, Software and Hardware Testing Using Combinatorial Covering Suites. Springer, 2005

    2005

  41. [49]

    https://www.cnn.com/2021/11/17/health/ drug-overdose-deaths-record-high, 2021

    2021

  42. [50]

    https://www.in.gov/pla/inspect/files/Narxcare user guide.pdf, 2020

    2020

  43. [51]

    Dosing discrimination: regulating pdmp risk scores,

    J. D. Oliva, “Dosing discrimination: regulating pdmp risk scores,” Cal. L. Rev., vol. 110, p. 47, 2022

    2022

  44. [52]

    Predictive modelling of susceptibility to substance abuse, mortality and drug-drug interactions in opioid patients,

    R. Vunikili, B. S. Glicksberg, K. W. Johnson, J. T. Dudley, L. Sub- ramanian, and K. Shameer, “Predictive modelling of susceptibility to substance abuse, mortality and drug-drug interactions in opioid patients,” Frontiers in Artificial Intelligence , vol. 4, p. 742723, 2021

    2021

  45. [53]

    Mimic-iv, a freely accessible electronic health record dataset,

    A. E. Johnson, L. Bulgarelli, L. Shen, A. Gayles, A. Shammout, S. Horng, T. J. Pollard, S. Hao, B. Moody, B. Gow et al., “Mimic-iv, a freely accessible electronic health record dataset,” Scientific data, 2023

    2023

  46. [54]

    Compstat and organizational change: A national assessment,

    D. Weisburd, R. Greenspan, S. Mastrofski, and J. J. Willis, “Compstat and organizational change: A national assessment,” National Institute of Justice, 2008

    2008

  47. [55]

    To predict and serve?

    K. Lum and W. Isaac, “To predict and serve?” Significance, 2016. 12

    2016

  48. [56]

    A similarity measure for indefinite rankings,

    W. Webber, A. Moffat, and J. Zobel, “A similarity measure for indefinite rankings,” ACM Transactions on Information Systems (TOIS) , 2010

    2010

  49. [57]

    A new measure of rank correlation,

    M. G. Kendall, “A new measure of rank correlation,” Biometrika, 1938

    1938

  50. [58]

    Long-term fairness with unknown dynamics,

    T. Yin, R. Raab, M. Liu, and Y . Liu, “Long-term fairness with unknown dynamics,” Advances in Neural Information Processing Systems , 2024

    2024

  51. [59]

    A call for better validation of opioid overdose risk algorithms,

    D. C. McElfresh, L. Chen, E. Oliva, V . Joyce, S. Rose, and S. Tamang, “A call for better validation of opioid overdose risk algorithms,” J Am Med Inform Assoc , vol. 30, no. 10, pp. 1741–1746, Sep. 2023

    2023

  52. [60]

    Hidden risks of machine learning applied to healthcare: unintended feed- back loops between models and future data causing model degradation,

    G. A. Adam, C.-H. K. Chang, B. Haibe-Kains, and A. Goldenberg, “Hidden risks of machine learning applied to healthcare: unintended feed- back loops between models and future data causing model degradation,” in Machine Learning for Healthcare Conference , 2020

    2020

  53. [61]

    A comparative study of fairness-enhancing interventions in machine learning,

    S. A. Friedler, C. Scheidegger, S. Venkatasubramanian, S. Choudhary, E. P. Hamilton, and D. Roth, “A comparative study of fairness-enhancing interventions in machine learning,” in Proceedings of the Conference on Fairness, Accountability, and Transparency , 2019

    2019

  54. [62]

    Fairness-aware configuration of machine learning libraries,

    S. Tizpaz-Niari, A. Kumar, G. Tan, and A. Trivedi, “Fairness-aware configuration of machine learning libraries,” in Proceedings of the 44th International Conference on Software Engineering , 2022, pp. 909–920

    2022

  55. [63]

    “ignorance and prejudice

    J. M. Zhang and M. Harman, ““ignorance and prejudice” in software fairness,” in 2021 IEEE/ACM 43rd International Conference on Software Engineering (ICSE). IEEE, 2021, pp. 1436–1447

    2021

  56. [64]

    Fair enough: Searching for sufficient measures of fairness,

    S. Majumder, J. Chakraborty, G. R. Bai, K. T. Stolee, and T. Menzies, “Fair enough: Searching for sufficient measures of fairness,” ACM Transactions on Software Engineering and Methodology , 2021

    2021

  57. [65]

    Learning fair representations,

    R. Zemel, Y . Wu, K. Swersky, T. Pitassi, and C. Dwork, “Learning fair representations,” in ICML, 2013

    2013

  58. [66]

    Certifying and removing disparate impact,

    M. Feldman, S. A. Friedler, J. Moeller, C. Scheidegger, and S. Venkata- subramanian, “Certifying and removing disparate impact,” in proceed- ings of the 21th ACM SIGKDD international conference on knowledge discovery and data mining , 2015

    2015

  59. [67]

    Fairness constraints: Mechanisms for fair classification,

    M. B. Zafar, I. Valera, M. G. Rogriguez, and K. P. Gummadi, “Fairness constraints: Mechanisms for fair classification,” in Artificial Intelligence and Statistics, 2017

    2017

  60. [68]

    Data preprocessing techniques for classifi- cation without discrimination,

    F. Kamiran and T. Calders, “Data preprocessing techniques for classifi- cation without discrimination,” Knowledge and Information Systems , 2012

    2012

  61. [69]

    Mitigating unwanted biases with adversarial learning,

    B. H. Zhang, B. Lemoine, and M. Mitchell, “Mitigating unwanted biases with adversarial learning,” in Proceedings of the 2018 AAAI/ACM Conference on AI, Ethics, and Society , 2018

    2018

  62. [70]

    Fairness-aware classifier with prejudice remover regularizer,

    T. Kamishima, S. Akaho, H. Asoh, and J. Sakuma, “Fairness-aware classifier with prejudice remover regularizer,” in Joint European Con- ference on Machine Learning and Knowledge Discovery in Databases , 2012

    2012

  63. [71]

    Maat: a novel ensemble approach to addressing fairness and performance bugs for machine learning software,

    Z. Chen, J. M. Zhang, F. Sarro, and M. Harman, “Maat: a novel ensemble approach to addressing fairness and performance bugs for machine learning software,” in Proceedings of the ESEC/FSE , 2022, pp. 1122–1134

    2022

  64. [72]

    Are my deep learning systems fair? an empirical study of fixed-seed training,

    S. Qian, V . H. Pham, T. Lutellier, Z. Hu, J. Kim, L. Tan, Y . Yu, J. Chen, and S. Shah, “Are my deep learning systems fair? an empirical study of fixed-seed training,” NIPS, 2021

    2021

  65. [73]

    Adaptive fairness improvement based on causality analysis,

    M. Zhang and J. Sun, “Adaptive fairness improvement based on causality analysis,” in Proceedings of the 30th ACM ESEC/FSE , 2022, pp. 6–17

    2022

  66. [74]

    Improving fairness in machine learning systems: What do industry practitioners need?

    K. Holstein, J. Wortman Vaughan, H. Daum ´e III, M. Dudik, and H. Wallach, “Improving fairness in machine learning systems: What do industry practitioners need?” in Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems , 2019

    2019

  67. [75]

    Neuronfair: Interpretable white-box fairness testing through biased neuron identification,

    H. Zheng, Z. Chen, T. Du, X. Zhang, Y . Cheng, S. Ji, J. Wang, Y . Yu, and J. Chen, “Neuronfair: Interpretable white-box fairness testing through biased neuron identification,” May 21-May 29 2022

    2022

  68. [76]

    Explanation-guided fairness testing through genetic algorithm,

    M. Fan, W. Wei, W. Jin, Z. Yang, and T. Liu, “Explanation-guided fairness testing through genetic algorithm,” in Proceedings of the 44th International Conference on Software Engineering , 2022, pp. 871–882

    2022

  69. [77]

    Information- theoretic testing and debugging of fairness defects in deep neural networks,

    V . Monjezi, A. Trivedi, G. Tan, and S. Tizpaz-Niari, “Information- theoretic testing and debugging of fairness defects in deep neural networks,” in Proceedings of the 45th International Conference on Software Engineering, ser. ICSE ’23, 2023, p. 1571–1582

    2023

  70. [78]

    Astraea: Grammar- based fairness testing,

    E. Soremekun, S. Udeshi, and S. Chattopadhyay, “Astraea: Grammar- based fairness testing,” Transactions on Software Engineering , 2022

    2022

  71. [79]

    Fairsquare: probabilistic verification of program fairness,

    A. Albarghouthi, L. D’Antoni, S. Drews, and A. V . Nori, “Fairsquare: probabilistic verification of program fairness,” Proceedings of the ACM on Programming Languages, 2017

    2017

  72. [80]

    Probabilistic verification of fairness properties via concentration,

    O. Bastani, X. Zhang, and A. Solar-Lezama, “Probabilistic verification of fairness properties via concentration,” Proceedings of the ACM on Programming Languages, 2019

    2019

  73. [81]

    Verifying individual fairness in machine learning models,

    P. G. John, D. Vijaykeerthy, and D. Saha, “Verifying individual fairness in machine learning models,” in Conference on Uncertainty in Artificial Intelligence. PMLR, 2020

    2020

  74. [82]

    Certifying the fairness of knn in the presence of dataset bias,

    Y . Li, J. Wang, and C. Wang, “Certifying the fairness of knn in the presence of dataset bias,” in International Conference on Computer Aided Verification. Springer, 2023

    2023

  75. [83]

    Fairea: A model behaviour mutation approach to benchmarking bias mitigation methods,

    M. Hort, J. M. Zhang, F. Sarro, and M. Harman, “Fairea: A model behaviour mutation approach to benchmarking bias mitigation methods,” in Proceedings of the 29th ACM ESEC/FSE , 2021

    2021

  76. [84]

    Long-term fairness inquiries and pursuits in machine learning: A survey of notions, methods, and challenges,

    U. Gohar, Z. Tang, J. Wang, K. Zhang, P. L. Spirtes, Y . Liu, and L. Cheng, “Long-term fairness inquiries and pursuits in machine learning: A survey of notions, methods, and challenges,” arXiv preprint arXiv:2406.06736, 2024

    2024 arXiv

  77. [85]

    Achieving long-term fairness in sequential decision making,

    Y . Hu and L. Zhang, “Achieving long-term fairness in sequential decision making,” in Proceedings of the AAAI Conference on Artificial Intelligence, 2022

    2022

  78. [86]

    Algorithms for fairness in sequential decision making,

    M. Wen, O. Bastani, and U. Topcu, “Algorithms for fairness in sequential decision making,” in International Conference on Artificial Intelligence and Statistics. PMLR, 2021

    2021

  79. [87]

    From fair decision making to social equality,

    H. Mouzannar, M. I. Ohannessian, and N. Srebro, “From fair decision making to social equality,” in Proceedings of the Conference on Fairness, Accountability, and Transparency, 2019, pp. 359–368

    2019

  80. [88]

    Fairness-aware programming,

    A. Albarghouthi and S. Vinitsky, “Fairness-aware programming,” in Proceedings of the Conference on Fairness, Accountability, and Trans- parency, 2019, pp. 211–219

    2019

  81. [89]

    Monitoring algorithmic fairness,

    T. A. Henzinger, M. Karimi, K. Kueffner, and K. Mallik, “Monitoring algorithmic fairness,” in Computer Aided Verification: 35th International Conference, CAV 2023. Berlin, Heidelberg: Springer-Verlag, 2023

    2023

  82. [90]

    Enforcing delayed-impact fairness guarantees,

    A. Weber, B. Metevier, Y . Brun, P. S. Thomas, and B. C. da Silva, “Enforcing delayed-impact fairness guarantees,” arXiv preprint arXiv:2208.11744, 2022

    2022 arXiv

  83. [91]

    Long-term fairness for real-time decision making: A constrained online optimization approach,

    R. Du, D. Muthirayan, P. P. Khargonekar, and Y . Shen, “Long-term fairness for real-time decision making: A constrained online optimization approach,” arXiv preprint arXiv:2401.02552 , 2024

    2024 arXiv

  84. [92]

    The long arc of fairness: Formalisations and ethical discourse,

    P. Schw¨obel and P. Remmers, “The long arc of fairness: Formalisations and ethical discourse,” in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, 2022

    2022

  85. [93]

    Fairness and abstraction in sociotechnical systems,

    A. D. Selbst, D. Boyd, S. A. Friedler, S. Venkatasubramanian, and J. Vertesi, “Fairness and abstraction in sociotechnical systems,” in In the Proceedings of the Conference on Fairness, Accountability, and Transparency, 2019

    2019

  86. [94]

    Requirements engineering for feedback loops in software-intensive systems,

    E. Kang and R. Meira-G ´oes, “Requirements engineering for feedback loops in software-intensive systems,” in 2022 IEEE 30th International Requirements Engineering Conference Workshops (REW) , 2022

    2022

  87. [95]

    J. L. Hellerstein, Y . Diao, S. Parekh, and D. M. Tilbury, Feedback control of computing systems . John Wiley & Sons, 2004

    2004

  88. [96]

    Data feedback loops: Model-driven amplification of dataset biases,

    R. Taori and T. Hashimoto, “Data feedback loops: Model-driven amplification of dataset biases,” in ICML, 2023

    2023

  89. [97]

    Hidden incentives for auto- induced distributional shift,

    D. Krueger, T. Maharaj, and J. Leike, “Hidden incentives for auto- induced distributional shift,” arXiv preprint arXiv:2009.09153 , 2020

    2009 arXiv

  90. [98]

    Quinonero-Candela, M

    J. Quinonero-Candela, M. Sugiyama, A. Schwaighofer, and N. D. Lawrence, Dataset shift in machine learning . MIT Press, 2022

    2022

  91. [99]

    Self-adaptation for machine learning based systems,

    M. Casimiro, P. Romano, D. Garlan, G. A. Moreno, E. Kang, and M. Klein, “Self-adaptation for machine learning based systems,” 2021

    2021

  92. [100]

    Models for understanding and quantifying feedback in societal systems,

    L. Reader, P. Nokhiz, C. Power, N. Patwari, S. Venkatasubramanian, and S. A. Friedler, “Models for understanding and quantifying feedback in societal systems,” in FAccT ’22, 2022

    2022

  93. [101]

    System dynamics modeling: Tools for learning in a complex world,

    J. D. Sterman, “System dynamics modeling: Tools for learning in a complex world,” California Management Review , pp. 8–25, 2001

    2001

  94. [102]

    A multi-methodology approach to creating a causal loop diagram,

    N. Dhirasasna and O. Sahin, “A multi-methodology approach to creating a causal loop diagram,” Syst., vol. 7, no. 3, p. 42, 2019

    2019

  95. [103]

    Building confidence in causal maps generated from purposive text data: mapping transcripts of the federal reserve,

    H. Kim and D. F. Andersen, “Building confidence in causal maps generated from purposive text data: mapping transcripts of the federal reserve,” System Dynamics Review , vol. 28, no. 4, pp. 311–328, 2012

    2012

  96. [104]

    Structural validation of causal loop diagrams,

    J. R. Burns and P. Musa, “Structural validation of causal loop diagrams,” in Proceedings of the 19th International Conference of the System Dynamics Society, 2001, pp. 23–27

    2001

  97. [105]

    Collecting and analyzing qualitative data for system dynamics: methods and models,

    L. F. Luna-Reyes and D. L. Andersen, “Collecting and analyzing qualitative data for system dynamics: methods and models,” System Dynamics Review, vol. 19, no. 4, pp. 271–296, 2003

    2003

  98. [106]

    Software engineering for sustainability: Find the leverage points!

    B. Penzenstadler, L. Duboc, C. C. Venters, S. Betz, N. Seyff, K. Wnuk, R. Chitchyan, S. M. Easterbrook, and C. Becker, “Software engineering for sustainability: Find the leverage points!” IEEE Softw., 2018. 13

    2018

  99. [2016]

    Available: https://www.technologyreview.com/2016/03/ 28/246328/artificial-intolerance

    [Online]. Available: https://www.technologyreview.com/2016/03/ 28/246328/artificial-intolerance

    2016

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.