arxiv: 2605.11237 · v1 · submitted 2026-05-11 · 💻 cs.LG

Recognition: no theorem link

DeconDTN-Toolkit: A Library for Evaluation and Enhancement of Robustness to Provenance Shift

Yongsen Tan , Zhecheng Sheng , Xiruo Ding , Serguei V. S. Pakhomov , Trevor Cohen

Authors on Pith no claims yet

Pith reviewed 2026-05-13 02:08 UTC · model grok-4.3

classification 💻 cs.LG

keywords provenance shiftcounterfactual invarianceinvariant learningdistribution shiftrobustnessout-of-distribution evaluationtoolkitmachine learning

0 comments

The pith

A formal connection between provenance shift, counterfactual invariance, and invariant learning yields a new objective for model robustness.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper aims to show that provenance shift, the change in how data sources relate to labels at deployment, can be handled by linking it directly to counterfactual invariance and invariant learning. This connection produces a specific learning objective that promotes robustness where standard empirical risk minimization fails. The authors supply DeconDTN-Toolkit as a practical library that generates controlled simulations of these shifts without disrupting existing benchmarks or training protocols. They also define a new out-of-distribution performance indicator and use it to compare mitigation methods. If the connection holds, models trained on the objective should maintain accuracy when source-label relationships change in deployment.

Core claim

We establish a formal connection between provenance shift, counterfactual invariance, and invariant learning to derive a learning objective for robustness. We then introduce DeconDTN-Toolkit, a specialized evaluation and remediation suite designed to simulate provenance shifts of varying degrees while maintaining the training protocol and the infrastructure of existing benchmarks. We reveal the vulnerability of Empirical Risk Minimization under provenance shift, introduce a robust out-of-distribution performance indicator, and conduct a comprehensive evaluation on existing algorithms.

What carries the argument

The derived learning objective obtained by connecting provenance shift to counterfactual invariance, implemented and tested inside DeconDTN-Toolkit.

If this is right

Empirical risk minimization becomes vulnerable once source-label relationships change at deployment.
A robust out-of-distribution performance indicator can be used to measure mitigation success beyond standard accuracy.
Existing invariant-learning algorithms can be re-evaluated and improved using the toolkit's controlled shift simulations.
Methods that mitigate confounding by provenance become implementable and comparable within the same benchmark infrastructure.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The same invariance link might be applied to other distribution shifts that involve hidden confounding variables.
Deployment pipelines could adopt the toolkit as a pre-release stress test before models encounter changing data sources.
The approach may combine with existing invariant-risk-minimization frameworks to produce hybrid objectives without new data collection.

Load-bearing premise

The simulated provenance shifts created by the toolkit accurately represent real deployment changes in source-label relationships while preserving the original training protocol and benchmark infrastructure.

What would settle it

A real-world dataset where the source-label relationship has shifted in deployment shows that models trained with the derived objective retain accuracy while standard ERM models degrade, matching the pattern observed in the toolkit simulations.

Figures

Figures reproduced from arXiv: 2605.11237 by Serguei V. S. Pakhomov, Trevor Cohen, Xiruo Ding, Yongsen Tan, Zhecheng Sheng.

**Figure 2.** Figure 2: WGA is not “on the line” with respect to ID performance, but exhibits a strong linear relationship [PITH_FULL_IMAGE:figures/full_fig_p008_2.png] view at source ↗

**Figure 3.** Figure 3: In-distribution (solid) and OOD (dashed) WGA for ERM in [PITH_FULL_IMAGE:figures/full_fig_p025_3.png] view at source ↗

**Figure 4.** Figure 4: WGA is not “on the line” with respect to ID performance, but exhibits a strong linear relationship [PITH_FULL_IMAGE:figures/full_fig_p025_4.png] view at source ↗

read the original abstract

Despite the burgeoning body of work on distribution shifts, provenance shift-where the relationship between data source and label changes at deployment-remains poorly understood and under-addressed. In this paper, we establish a formal connection between provenance shift, counterfactual invariance, and invariant learning to derive a learning objective for robustness. We then introduce \textsc{DeconDTN-Toolkit}, a specialized evaluation and remediation suite designed to simulate provenance shifts of varying degrees while maintaining the training protocol and the infrastructure of existing benchmarks. We reveal the vulnerability of Empirical Risk Minimization under provenance shift, introduce a robust out-of-distribution performance indicator, and conduct a comprehensive evaluation on existing algorithms. Our work provides both the theoretical grounding and the practical tools necessary to characterize the problem of confounding by provenance, and implementations of methods to mitigate it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

The toolkit for simulating provenance shifts is the practical hook, but the formal link to a new robustness objective via counterfactual invariance looks thin without the derivation details.

read the letter

The main takeaway is a toolkit that lets you generate provenance shifts—changes in the source-label relationship—while keeping existing benchmark setups intact. That fills a gap for testing models in settings like medical data where collection methods differ across sites. They also flag ERM's weakness under these shifts and add a robust OOD metric, plus some evaluation of prior algorithms. Those pieces give researchers concrete ways to probe this type of shift without rebuilding everything from scratch.

Referee Report

1 major / 3 minor

Summary. The paper claims to establish a formal connection between provenance shift (changes in the source-label relationship at deployment), counterfactual invariance, and invariant learning, from which it derives a new learning objective for robustness. It introduces DeconDTN-Toolkit to simulate provenance shifts of varying degrees on existing benchmarks while preserving original training protocols and infrastructure. The work demonstrates the vulnerability of empirical risk minimization (ERM) under such shifts, proposes a robust out-of-distribution performance indicator, and evaluates existing algorithms using the toolkit.

Significance. If the formal connection is rigorously derived without reducing to prior invariant learning objectives and if the simulated shifts are representative, the work would be significant for addressing an under-explored distribution shift with both theoretical grounding and a practical evaluation/remediation suite. The toolkit's design to maintain benchmark infrastructure is a strength for reproducibility and adoption. Credit is given for attempting to link provenance shift to counterfactual concepts and for providing implementations to mitigate confounding by provenance.

major comments (1)

[Theoretical derivation section] The section deriving the learning objective from the formal connection between provenance shift, counterfactual invariance, and invariant learning must explicitly state the modeling assumptions on the joint distribution P(source, features, label) under shift. Without these, it is unclear whether the objective is independent or reduces to a reparameterized version of existing invariant learning methods, as the connection may not be sufficient to guarantee necessity or sufficiency for robustness.

minor comments (3)

[Abstract] The abstract is information-dense; consider breaking the claims into separate sentences for improved readability while retaining all key elements.
[Evaluation section] In the evaluation section, provide explicit equations or pseudocode for the proposed robust out-of-distribution performance indicator and compare it directly to standard metrics such as average accuracy or worst-group accuracy.
[Toolkit description] Ensure the toolkit documentation includes clear descriptions of how simulated shifts preserve the original training protocol, with examples or pseudocode for reproducibility.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for their constructive and detailed review of our manuscript. We have carefully addressed the major comment regarding the theoretical derivation section and have revised the paper to provide the requested clarifications on modeling assumptions.

read point-by-point responses

Referee: [Theoretical derivation section] The section deriving the learning objective from the formal connection between provenance shift, counterfactual invariance, and invariant learning must explicitly state the modeling assumptions on the joint distribution P(source, features, label) under shift. Without these, it is unclear whether the objective is independent or reduces to a reparameterized version of existing invariant learning methods, as the connection may not be sufficient to guarantee necessity or sufficiency for robustness.

Authors: We appreciate the referee's point that the modeling assumptions require explicit statement for clarity. In the revised manuscript, we have added a new subsection immediately preceding the derivation of the learning objective. This subsection explicitly defines the assumptions on the joint distribution P(source, features, label) under provenance shift: specifically, that the shift is induced by an intervention on the source variable that alters P(label | source) while leaving the conditional feature distributions P(features | label, source) and the marginal P(features) unchanged, and that counterfactual invariance is formalized with respect to do-interventions on source. Under these assumptions, we provide a brief argument (with supporting lemmas) showing that the resulting objective does not reduce to a reparameterization of prior invariant learning methods such as IRM, as the penalty term is derived directly from the counterfactual invariance condition specialized to provenance confounding rather than general domain invariance. We have also included a short discussion establishing necessity and sufficiency for robustness to provenance shifts under the stated assumptions. These additions are intended to resolve the ambiguity noted by the referee. revision: yes

Circularity Check

0 steps flagged

No significant circularity; derivation builds on external concepts without reduction to inputs

full rationale

The paper's central claim is establishing a formal connection between provenance shift, counterfactual invariance, and invariant learning to derive a new learning objective. No equations or sections in the provided text demonstrate that this objective reduces by construction to a fitted parameter, self-defined term, or prior self-cited result. The toolkit is presented as a separate evaluation tool that simulates shifts while preserving benchmarks, without using the derived objective as its own validation input. Self-citations, if present, are not load-bearing for the derivation itself. The claim remains independent of its own outputs.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides no explicit free parameters, axioms, or invented entities; the formal connection is asserted without listed assumptions.

pith-pipeline@v0.9.0 · 5455 in / 990 out tokens · 67025 ms · 2026-05-13T02:08:37.247024+00:00 · methodology

discussion (0)

Reference graph

Works this paper leans on

121 extracted references · 121 canonical work pages · 3 internal anchors

[1]

Guyon and A

I. Guyon and A. Elisseeff. An Introduction to Variable and Feature Selection. JMLR

work page
[2]

Guyon and C

I. Guyon and C. Aliferis and A. Elisseeff , title =

work page
[3]

Johnson, Alistair E. W. and Pollard, Tom J. and Shen, Lu and Lehman, Li-wei H. and Feng, Mengling and Ghassemi, Mohammad and Moody, Benjamin and Szolovits, Peter and Celi, Leo Anthony and Mark, Roger G. , journal=. doi:https://doi.org/10.1038/sdata.2016.35 , volume=

work page doi:10.1038/sdata.2016.35 2016
[4]

Johnson, Alistair E. W. and Pollard, Tom J. and Mark, Roger G. , year=. doi:https://doi.org/10.13026/C2XW26 , publisher=

work page doi:10.13026/c2xw26
[5]

Proceedings of the 40th

Garg, Saurabh and Erickson, Nick and Sharpnack, James and Smola, Alex and Balakrishnan, Sivaraman and Lipton, Zachary Chase , year =. Proceedings of the 40th

work page
[6]

JAMIA open , volume=

Rethinking domain adaptation for machine learning over clinical language , author=. JAMIA open , volume=. 2020 , publisher=

work page 2020
[7]

Journal of Clinical and Translational Science , volume=

Development and validation of natural language processing algorithms in the national ENACT network , author=. Journal of Clinical and Translational Science , volume=. 2025 , publisher=

work page 2025
[8]

Cookie Theft

Crossing the “Cookie Theft” corpus chasm: applying what BERT learns from outside data to the ADReSS challenge dementia detection task , author=. Frontiers in Computer Science , volume=. 2021 , publisher=

work page 2021
[9]

Explainable AI in Healthcare and Medicine: Building a Culture of Transparency and Accountability , pages=

Controlling for confounding variables: accounting for dataset bias in classifying patient-provider interactions , author=. Explainable AI in Healthcare and Medicine: Building a Culture of Transparency and Accountability , pages=. 2020 , publisher=

work page 2020
[10]

NeurIPS 2023 Workshop on Distribution Shifts: New Frontiers with Foundation Models , year=

Enhancing Robustness of Foundation Model Representations under Provenance-related Distribution Shifts , author=. NeurIPS 2023 Workshop on Distribution Shifts: New Frontiers with Foundation Models , year=

work page 2023
[11]

Journal of Artificial Intelligence Research , volume=

Robust text classification under confounding shift , author=. Journal of Artificial Intelligence Research , volume=

work page
[12]

Proceedings of the AAAI Conference on Artificial Intelligence , volume=

Robust text classification in the presence of confounding bias , author=. Proceedings of the AAAI Conference on Artificial Intelligence , volume=

work page
[13]

2025 , address =

Ding, Xiruo , title =. 2025 , address =

work page 2025
[14]

Detecting and

Lipton, Zachary and Wang, Yu-Xiang and Smola, Alexander , year =. Detecting and. Proceedings of the 35th

work page
[15]

Unsupervised

Roberts, Manley and Mani, Pranav and Garg, Saurabh and Lipton, Zachary , year =. Unsupervised. Advances in Neural Information Processing Systems , volume =

work page
[16]

Zico , year =

Baek, Christina and Jiang, Yiding and Raghunathan, Aditi and Kolter, J. Zico , year =. Agreement-on-the-Line:. Advances in Neural Information Processing Systems , volume =

work page
[17]

Blanchard, Gilles and Deshmukh, Aniket Anand and Dogan, Urun and Lee, Gyemin and Scott, Clayton , year =. Domain. Journal of Machine Learning Research , volume =

work page
[18]

2017 , month = apr, journal =

Semantics Derived Automatically from Language Corpora Contain Human-like Biases , author =. 2017 , month = apr, journal =

work page 2017
[19]

Advances in

Dranker, Yana and He, He and Belinkov, Yonatan , year =. Advances in

work page
[20]

and Manzoor, Emaad and Pryzant, Reid and Sridhar, Dhanya and

Feder, Amir and Keith, Katherine A. and Manzoor, Emaad and Pryzant, Reid and Sridhar, Dhanya and. Causal. 2022 , month = oct, journal =

work page 2022
[21]

2015 , publisher =

Fundamentals of Clinical Trials , author =. 2015 , publisher =

work page 2015
[22]

Ganin, Yaroslav and Ustinova, Evgeniya and Ajakan, Hana and Germain, Pascal and Larochelle, Hugo and Laviolette, Fran. Domain-. 2016 , journal =

work page 2016
[23]

Gulrajani, Ishaan and. In. International

work page
[24]

and Leskovec, Jure and Kundaje, Anshul and Pierson, Emma and Levine, Sergey and Finn, Chelsea and Liang, Percy , year =

Koh, Pang Wei and Sagawa, Shiori and Marklund, Henrik and Xie, Sang Michael and Zhang, Marvin and Balsubramani, Akshay and Hu, Weihua and Yasunaga, Michihiro and Phillips, Richard Lanas and Gao, Irena and Lee, Tony and David, Etienne and Stavness, Ian and Guo, Wei and Earnshaw, Berton and Haque, Imran and Beery, Sara M. and Leskovec, Jure and Kundaje, Ans...

work page
[25]

Li, Ya and Tian, Xinmei and Gong, Mingming and Liu, Yajing and Liu, Tongliang and Zhang, Kun and Tao, Dacheng , year =. Deep. Proceedings of the

work page
[26]

Learning to

Li, Da and Yang, Yongxin and Song, Yi-Zhe and Hospedales, Timothy , year =. Learning to. Proceedings of the AAAI Conference on Artificial Intelligence , volume =. doi:10.1609/aaai.v32i1.11596 , copyright =

work page doi:10.1609/aaai.v32i1.11596
[27]

Subspace

Li, Zijian and Cai, Ruichu and Chen, Guangyi and Sun, Boyang and Hao, Zhifeng and Zhang, Kun , year =. Subspace. Advances in Neural Information Processing Systems , volume =

work page
[28]

Foundations of Statistical Natural Language Processing , author =

work page
[29]

2009 , month = sep, publisher =

Causality , author =. 2009 , month = sep, publisher =

work page 2009
[30]

Ravfogel, Shauli and Elazar, Yanai and Gonen, Hila and Twiton, Michael and Goldberg, Yoav , editor =. Null. Proceedings of the 58th. 2020 , month = jul, pages =

work page 2020
[31]

and Liang, Percy , year =

Sagawa*, Shiori and Koh*, Pang Wei and Hashimoto, Tatsunori B. and Liang, Percy , year =. Distributionally. International

work page
[32]

Sagawa, Shiori and Raghunathan, Aditi and Koh, Pang Wei and Liang, Percy , year =. An. Proceedings of the 37th

work page
[33]

Sch. Towards. 2021 , month = feb, journal =. doi:10.48550/arXiv.2102.11107 , archiveprefix =. 2102.11107 , primaryclass =

work page doi:10.48550/arxiv.2102.11107 2021
[34]

Sun, Baochen and Saenko, Kate , editor =. Deep. Computer. 2016 , pages =. doi:10.1007/978-3-319-49409-8_35 , isbn =

work page doi:10.1007/978-3-319-49409-8_35 2016
[35]

Counterfactual

Veitch, Victor and D' Amour, Alexander and Yadlowsky, Steve and Eisenstein, Jacob , year =. Counterfactual. Advances in

work page
[36]

Advances in Neural Information Processing Systems , volume =

Wang, Boxin and Chen, Weixin and Pei, Hengzhi and Xie, Chulin and Kang, Mintong and Zhang, Chenhui and Xu, Chejian and Xiong, Zidi and Dutta, Ritik and Schaeffer, Rylan and Truong, Sang and Arora, Simran and Mazeika, Mantas and Hendrycks, Dan and Lin, Zinan and Cheng, Yu and Koyejo, Sanmi and Song, Dawn and Li, Bo , year =. Advances in Neural Information ...

work page
[37]

Measure and

Wang, Xuezhi and Wang, Haohan and Yang, Diyi , editor =. Measure and. Proceedings of the 2022. 2022 , month = jul, pages =

work page 2022
[38]

Yang, Linyi and Song, Yaoxian and Ren, Xuan and Lyu, Chenyang and Wang, Yidong and Zhuo, Jingming and Liu, Lingqiao and Wang, Jindong and Foster, Jennifer and Zhang, Yue , editor =. Out-of-. Proceedings of the 2023. 2023 , month = dec, pages =

work page 2023
[39]

Improving

Yao, Huaxiu and Wang, Yu and Li, Sai and Zhang, Linjun and Liang, Weixin and Zou, James and Finn, Chelsea , year =. Improving. Proceedings of the 39th

work page
[40]

Zhang, Hongyi and Cisse, Moustapha and Dauphin, Yann N. and. Mixup:. International

work page
[41]

Zhou, Kaiyang and Liu, Ziwei and Qiao, Yu and Xiang, Tao and Loy, Chen Change , year =. Domain. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume =. doi:10.1109/TPAMI.2022.3195549 , keywords =

work page doi:10.1109/tpami.2022.3195549 2022
[42]

Examining and

Zhou, Chunting and Ma, Xuezhe and Michel, Paul and Neubig, Graham , year =. Examining and. Proceedings of the 38th

work page
[43]

and Taori, Rohan and Raghunathan, Aditi and Sagawa, Shiori and Koh, Pang Wei and Shankar, Vaishaal and Liang, Percy and Carmon, Yair and Schmidt, Ludwig , year =

Miller, John P. and Taori, Rohan and Raghunathan, Aditi and Sagawa, Shiori and Koh, Pang Wei and Shankar, Vaishaal and Liang, Percy and Carmon, Yair and Schmidt, Ludwig , year =. Accuracy on the. Proceedings of the 38th

work page
[44]

Why Does

Chaudhuri, Kamalika and Ahuja, Kartik and Arjovsky, Martin and. Why Does. Proceedings of the 40th. 2023 , month = jul, pages =

work page 2023
[45]

2024 , month = jun, journal =

Ask. 2024 , month = jun, journal =

work page 2024
[46]

Kirichenko, Polina and Izmailov, Pavel and Wilson, Andrew Gordon , year =. Last. doi:10.48550/arXiv.2204.02937 , archiveprefix =. 2204.02937 , primaryclass =

work page doi:10.48550/arxiv.2204.02937
[47]

Stromberg, Nathan and Ayyagari, Rohan and Welfert, Monica and Koyejo, Sanmi and Nock, Richard and Sankar, Lalitha , year =. For. Transactions on Machine Learning Research , issn =

work page
[48]

and Finn, Chelsea , year =

Yao, Huaxiu and Choi, Caroline and Cao, Bochuan and Lee, Yoonho and Koh, Pang Wei W. and Finn, Chelsea , year =. Wild-. Advances in Neural Information Processing Systems , volume =

work page
[49]

Change Is

Yang, Yuzhe and Zhang, Haoran and Katabi, Dina and Ghassemi, Marzyeh , year =. Change Is. Proceedings of the 40th

work page
[50]

Pretrained

Hendrycks, Dan and Liu, Xiaoyuan and Wallace, Eric and Dziedzic, Adam and Krishnan, Rishabh and Song, Dawn , editor =. Pretrained. Proceedings of the 58th. 2020 , month = jul, pages =. doi:10.18653/v1/2020.acl-main.244 , annotation =

work page doi:10.18653/v1/2020.acl-main.244 2020
[51]

2020 , month = nov, journal =

Shortcut Learning in Deep Neural Networks , author =. 2020 , month = nov, journal =. doi:10.1038/s42256-020-00257-z , copyright =

work page doi:10.1038/s42256-020-00257-z 2020
[52]

Thomas and Pavlick, Ellie and Linzen, Tal , editor =

McCoy, R. Thomas and Pavlick, Ellie and Linzen, Tal , editor =. Right for the. Proceedings of the 57th. 2019 , month = jul, pages =

work page 2019
[53]

, year =

Ruan, Yangjun and Dubois, Yann and Maddison, Chris J. , year =. Optimal. International

work page
[54]

D'Amour, Alexander and Heller, Katherine and Moldovan, Dan and Adlam, Ben and Alipanahi, Babak and Beutel, Alex and Chen, Christina and Deaton, Jonathan and Eisenstein, Jacob and Hoffman, Matthew D. and Hormozdiari, Farhad and Houlsby, Neil and Hou, Shaobo and Jerfel, Ghassen and Karthikesalingam, Alan and Lucic, Mario and Ma, Yian and McLean, Cory and Mi...

work page
[55]

Invariant Risk Minimization

Arjovsky, Martin and Bottou, L. Invariant. 2020 , month = mar, number =. doi:10.48550/arXiv.1907.02893 , archiveprefix =. 1907.02893 , primaryclass =

work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.1907.02893 2020
[56]

Selective

Teney, Damien and Wang, Jindong and Abbasnejad, Ehsan , year =. Selective. Proceedings of the 41st

work page
[57]

Advances in Neural Information Processing Systems , volume=

Learning from failure: De-biasing classifier from biased classifier , author=. Advances in Neural Information Processing Systems , volume=

work page
[58]

Mitigating

Korakakis, Michalis and Vlachos, Andreas and Weller, Adrian , editor =. Mitigating. Proceedings of the 63rd. 2025 , month = jul, pages =. doi:10.18653/v1/2025.acl-long.450 , isbn =

work page doi:10.18653/v1/2025.acl-long.450 2025
[59]

and Haghgoo, Behzad and Chen, Annie S

Liu, Evan Z. and Haghgoo, Behzad and Chen, Annie S. and Raghunathan, Aditi and Koh, Pang Wei and Sagawa, Shiori and Liang, Percy and Finn, Chelsea , year =. Just. Proceedings of the 38th

work page
[60]

, year =

Li, Haoliang and Pan, Sinno Jialin and Wang, Shiqi and Kot, Alex C. , year =. Domain. 2018. doi:10.1109/CVPR.2018.00566 , keywords =

work page doi:10.1109/cvpr.2018.00566 2018
[61]

Probable

Eastwood, Cian and Robey, Alexander and Singh, Shashank and. Probable. 2022 , month = dec, journal =

work page 2022
[62]

Explicit

Moayeri, Mazda and Banihashem, Kiarash and Feizi, Soheil , year =. Explicit. Advances in Neural Information Processing Systems , volume =

work page
[63]

Kumar, Abhinav and Tan, Chenhao and Sharma, Amit , year =. Probing. Advances in Neural Information Processing Systems , volume =

work page
[64]

and Lee, Yoonho and Setlur, Amrith and Levine, Sergey and Finn, Chelsea , year =

Chen, Annie S. and Lee, Yoonho and Setlur, Amrith and Levine, Sergey and Finn, Chelsea , year =. Project and. International

work page
[65]

Parzen, Emanuel , year =. On. The Annals of Mathematical Statistics , volume =. 2237880 , eprinttype =

work page
[66]

Yan, Shen and Song, Huan and Li, Nanxiang and Zou, Lincan and Ren, Liu , year =. Improve. doi:10.48550/arXiv.2001.00677 , archiveprefix =. 2001.00677 , primaryclass =

work page doi:10.48550/arxiv.2001.00677 2001
[67]

Temporal

Agarwal, Oshin and Nenkova, Ani , year =. Temporal. Transactions of the Association for Computational Linguistics , volume =

work page
[68]

2013 , month = feb, journal =

Robust. 2013 , month = feb, journal =. doi:10.1287/mnsc.1120.1641 , keywords =

work page doi:10.1287/mnsc.1120.1641 2013
[69]

Hospedales, Timothy and Antoniou, Antreas and Micaelli, Paul and Storkey, Amos , year =. Meta-. IEEE Transactions on Pattern Analysis and Machine Intelligence , volume =. doi:10.1109/TPAMI.2021.3079209 , keywords =

work page doi:10.1109/tpami.2021.3079209 2021
[70]

Gradient

Shi, Yuge and Seely, Jeffrey and Torr, Philip and N, Siddharth and Hannun, Awni and Usunier, Nicolas and Synnaeve, Gabriel , year =. Gradient. International

work page
[71]

Rame, Alexandre and Dancette, Corentin and Cord, Matthieu , year =. Fishr:. Proceedings of the 39th

work page
[72]

Unbiased look at dataset b ias

Torralba, Antonio and Efros, Alexei A. , year =. Unbiased Look at Dataset Bias , booktitle =. doi:10.1109/CVPR.2011.5995347 , keywords =

work page doi:10.1109/cvpr.2011.5995347 2011
[73]

Advances in

Krizhevsky, Alex and Sutskever, Ilya and Hinton, Geoffrey E , year =. Advances in

work page
[74]

Backdoor

Ding, Xiruo and Sheng, Zhecheng and Yeti. Backdoor. 2024 , month = jan, journal =

work page 2024
[75]

Simple Data Balancing Achieves Competitive Worst-Group-Accuracy , booktitle =

Idrissi, Badr Youbi and Arjovsky, Martin and Pezeshki, Mohammad and. Simple Data Balancing Achieves Competitive Worst-Group-Accuracy , booktitle =. 2022 , month = jun, pages =

work page 2022
[76]

Liu, Zhuang and He, Kaiming , year =. A. The

work page
[77]

2025 , month = aug, journal =

Robustness Tests for Biomedical Foundation Models Should Tailor to Specifications , author =. 2025 , month = aug, journal =. doi:10.1038/s41746-025-01926-2 , copyright =

work page doi:10.1038/s41746-025-01926-2 2025
[78]

2023 , month = jun, journal =

Algorithmic Fairness in Artificial Intelligence for Medicine and Healthcare , author =. 2023 , month = jun, journal =. doi:10.1038/s41551-023-01056-8 , copyright =

work page doi:10.1038/s41551-023-01056-8 2023
[79]

Gichoya, Dina Katabi, and Marzyeh Ghassemi

Yang, Yuzhe and Zhang, Haoran and Gichoya, Judy W. and Katabi, Dina and Ghassemi, Marzyeh , year =. The Limits of Fair Medical Imaging. Nature Medicine , volume =. doi:10.1038/s41591-024-03113-4 , copyright =

work page doi:10.1038/s41591-024-03113-4
[80]

2024 , month = apr, journal =

Demographic Bias in Misdiagnosis by Computational Pathology Models , author =. 2024 , month = apr, journal =. doi:10.1038/s41591-024-02885-z , copyright =

work page doi:10.1038/s41591-024-02885-z 2024

Showing first 80 references.