Pith. sign in

REVIEW 1 cited by

Improving LLM Group Fairness on Tabular Data via In-Context Learning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2412.04642 v1 pith:EHYKH7FJ submitted 2024-12-05 cs.LG cs.AI

Valeriia Cherepanova , Chia-Jung Lee , Nil-Jana Akpinar , Riccardo Fogliato , Martin Andres Bertran , Michael Kearns , James Zou This is my paper · ORCID
classification cs.LGcs.AI
keywords tabulargroupfairnessllmspredictionsapproachesdatasetsexamples
verification ladder T0 review T1 audit T2 compute T3 formal

Signed reviews

No signed human review yet.

0 comments
read the original abstract

Large language models (LLMs) have been shown to be effective on tabular prediction tasks in the low-data regime, leveraging their internal knowledge and ability to learn from instructions and examples. However, LLMs can fail to generate predictions that satisfy group fairness, that is, produce equitable outcomes across groups. Critically, conventional debiasing approaches for natural language tasks do not directly translate to mitigating group unfairness in tabular settings. In this work, we systematically investigate four empirical approaches to improve group fairness of LLM predictions on tabular datasets, including fair prompt optimization, soft prompt tuning, strategic selection of few-shot examples, and self-refining predictions via chain-of-thought reasoning. Through experiments on four tabular datasets using both open-source and proprietary LLMs, we show the effectiveness of these methods in enhancing demographic parity while maintaining high overall performance. Our analysis provides actionable insights for practitioners in selecting the most suitable approach based on their specific requirements and constraints.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Group Fairness Meets the Black Box: Enabling Fair Algorithms on Closed LLMs via Post-Processing

    cs.LG 2025-08 conditional novelty 7.0 of 10

    A prompt-based pipeline lets closed LLMs like GPT-4o be used with classical group-fairness algorithms, without access to weights or embeddings.

Reference graph

Works this paper leans on

73 extracted references · 1 canonical work pages · cited by 1 Pith paper

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Alekh Agarwal, Alina Beygelzimer, Miroslav Dudik, John Langford, and Hanna Wallach. 2018. https://proceedings.mlr.press/v80/agarwal18a.html A reductions approach to fair classification . In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 60--69. PMLR

    2018

  4. [4]

    Carlos Aguirre, Kuleen Sasse, Isabel Cachola, and Mark Dredze. 2023. Selecting shots for demographic fairness in few-shot learning with large language models. arXiv preprint arXiv:2311.08472

    arXiv 2023

  5. [5]

    Nil-Jana Akpinar, Zachary Lipton, and Alexandra Chouldechova. 2024. https://doi.org/10.1145/3630106.3658977 The impact of differential feature under-reporting on algorithmic fairness . In Proceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency, FAccT '24, page 1355–1382, New York, NY, USA. Association for Computing Machinery

    arXiv 2024

  6. [6]

    Nil-Jana Akpinar, Manish Nagireddy, Logan Stapleton, Hao-Fei Cheng, Haiyi Zhu, Steven Wu, and Hoda Heidari. 2022. https://arxiv.org/abs/2204.10233 A sandbox tool to bias(stress)-test fairness algorithms . Preprint, arXiv:2204.10233

    arXiv 2022

  7. [7]

    AI Anthropic. 2024. The claude 3 model family: Opus, sonnet, haiku. Claude-3 Model Card

    2024

  8. [8]

    Solon Barocas, Moritz Hardt, and Arvind Narayanan. 2023. Fairness and machine learning: Limitations and opportunities. MIT Press

    2023

Show all 73 references
  1. [9]

    Ronny Kohavi Barry Becker. 1996. https://doi.org/10.24432/C5XW20 Adult

    1996 doi

  2. [10]

    Emily M Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM conference on fairness, accountability, and transparency, pages 610--623

    2021

  3. [11]

    Richard Berk, Hoda Heidari, Shahin Jabbari, Matthew Joseph, Michael Kearns, Jamie Morgenstern, Seth Neel, and Aaron Roth. 2017. A convex framework for fair regression. arXiv preprint arXiv:1706.02409

    2017 arXiv

  4. [12]

    Emily Black, Manish Raghavan, and Solon Barocas. 2022. Model multiplicity: Opportunities, concerns, and solutions. In Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency, pages 850--863

    2022

  5. [13]

    Su Lin Blodgett, Gilsinia Lopez, Alexandra Olteanu, Robert Sim, and Hanna Wallach. 2021. Stereotyping norwegian salmon: An inventory of pitfalls in fairness benchmark datasets. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 1...

    2021

  6. [14]

    Tolga Bolukbasi, Kai-Wei Chang, James Y Zou, Venkatesh Saligrama, and Adam T Kalai. 2016. Man is to computer programmer as woman is to homemaker? debiasing word embeddings. Advances in neural information processing systems, 29

    2016

  7. [15]

    Aylin Caliskan, Joanna J Bryson, and Arvind Narayanan. 2017. Semantics derived automatically from language corpora contain human-like biases. Science, 356(6334):183--186

    2017

  8. [16]

    Alessandro Castelnovo, Riccardo Crupi, Greta Greco, Daniele Regoli, Ilaria Giuseppina Penco, and Andrea Claudio Cosentini. 2022. A clarification of the nuances in the fairness metrics landscape. Scientific Reports, 12(1):4209

    2022

  9. [17]

    Simon Caton and Christian Haas. 2024. Fairness in machine learning: A survey. ACM Computing Surveys, 56(7):1--38

    2024

  10. [18]

    Banghao Chen, Zhaofeng Zhang, Nicolas Langren \'e , and Shengxin Zhu. 2023. Unleashing the potential of prompt engineering in large language models: a comprehensive review. arXiv preprint arXiv:2310.14735

    2023 arXiv

  11. [19]

    Wenhu Chen. 2022. Large language models are few (1)-shot table reasoners. arXiv preprint arXiv:2210.06710

    2022 arXiv

  12. [20]

    Alexandra Chouldechova. 2017. Fair prediction with disparate impact: A study of bias in recidivism prediction instruments. Big data, 5(2):153--163

    2017

  13. [21]

    Alexandra Chouldechova and Aaron Roth. 2018. The frontiers of fairness in machine learning. arXiv preprint arXiv:1810.08810

    2018 arXiv

  14. [22]

    Zhibo Chu, Zichong Wang, and Wenbin Zhang. 2024. Fairness in large language models: A taxonomic survey. arXiv preprint arXiv:2404.01349

    2024 arXiv

  15. [23]

    Frances Ding, Moritz Hardt, John Miller, and Ludwig Schmidt. 2021. https://proceedings.neurips.cc/paper_files/paper/2021/file/32e54441e6382a7fbacbbbaf3c450059-Paper.pdf Retiring adult: New datasets for fair machine learning . In Advances in Neural Information Processing System...

    2021

  16. [24]

    Sanghamitra Dutta, Dennis Wei, Hazar Yueksel, Pin-Yu Chen, Sijia Liu, and Kush Varshney. 2020. Is there a trade-off between fairness and accuracy? a perspective using mismatched hypothesis testing. In International conference on machine learning, pages 2803--2813. PMLR

    2020

  17. [25]

    Cynthia Dwork, Moritz Hardt, Toniann Pitassi, Omer Reingold, and Richard Zemel. 2012. Fairness through awareness. In Proceedings of the 3rd innovations in theoretical computer science conference, pages 214--226

    2012

  18. [26]

    Alessandro Fabris, Stefano Messina, Gianmaria Silvello, and Gian Antonio Susto. 2022. Algorithmic fairness datasets: the story so far. Data Mining and Knowledge Discovery, 36(6):2074--2152

    2022

  19. [27]

    Xi Fang, Weijie Xu, Fiona Anting Tan, Jiani Zhang, Ziqing Hu, Yanjun Qi, Scott Nickleach, Diego Socolinsky, Srinivasan Sengamedu, and Christos Faloutsos. 2024. Large language models (llms) on tabular data: Predic-tion, generation, and understanding-a survey. arXiv preprint arX...

    2024 arXiv

  20. [28]

    Zahra Fatemi, Chen Xing, Wenhao Liu, and Caiming Xiong. 2021. Improving gender fairness of pre-trained language models without catastrophic forgetting. arXiv preprint arXiv:2110.05367

    2021 arXiv

  21. [29]

    Riccardo Fogliato, Alexandra Chouldechova, and Max G’Sell. 2020. Fairness evaluation in presence of biased noisy labels. In International conference on artificial intelligence and statistics, pages 2325--2336. PMLR

    2020

  22. [30]

    Amit Haim, Alejandro Salinas, and Julian Nyarko. 2024. What's in a name? auditing large language models for race and gender bias. arXiv preprint arXiv:2402.14875

    2024 arXiv

  23. [31]

    Moritz Hardt, Eric Price, and Nati Srebro. 2016. Equality of opportunity in supervised learning. Advances in neural information processing systems, 29

    2016

  24. [32]

    Stefan Hegselmann, Alejandro Buendia, Hunter Lang, Monica Agrawal, Xiaoyi Jiang, and David Sontag. 2023. Tabllm: Few-shot classification of tabular data with large language models. In International Conference on Artificial Intelligence and Statistics, pages 5549--5581. PMLR

    2023

  25. [33]

    Jonathan Herzig, Pawe Krzysztof Nowak, Thomas M \"u ller, Francesco Piccinno, and Julian Martin Eisenschlos. 2020. Tapas: Weakly supervised table parsing via pre-training. arXiv preprint arXiv:2004.02349

    2020 arXiv

  26. [34]

    Hans Hofmann. 1994. https://doi.org/10.24432/C5NC77 Statlog (german credit data)

    1994 doi

  27. [35]

    Enhancing fairness in in-context learning: Prioritizing minority samples in demonstrations

    Jingyu Hu and Mengnan Du. Enhancing fairness in in-context learning: Prioritizing minority samples in demonstrations. In The Second Tiny Papers Track at ICLR 2024

    2024

  28. [36]

    Xin Huang, Ashish Khetan, Milan Cvitkovic, and Zohar Karnin. 2020. Tabtransformer: Tabular data modeling using contextual embeddings. arXiv preprint arXiv:2012.06678

    2020 arXiv

  29. [37]

    Sukriti Jaitly, Tanay Shah, Ashish Shugani, and Razik Singh Grewal. 2023. Towards better serialization of tabular data for few-shot classification. arXiv preprint arXiv:2312.12464

    2023 arXiv

  30. [38]

    Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825

    2023 arXiv

  31. [39]

    Albert Q Jiang, Alexandre Sablayrolles, Antoine Roux, Arthur Mensch, Blanche Savary, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Emma Bou Hanna, Florian Bressand, et al. 2024. Mixtral of experts. arXiv preprint arXiv:2401.04088

    2024 arXiv

  32. [40]

    Keita Kurita, Nidhi Vyas, Ayush Pareek, Alan W Black, and Yulia Tsvetkov. 2019. Measuring bias in contextualized word representations. arXiv preprint arXiv:1906.07337

    2019 arXiv

  33. [41]

    Roman Levin, Valeriia Cherepanova, Avi Schwarzschild, Arpit Bansal, C Bayan Bruss, Tom Goldstein, Andrew Gordon Wilson, and Micah Goldblum. 2022. Transfer learning with deep tabular models. arXiv preprint arXiv:2206.15306

    2022 arXiv

  34. [42]

    Yunqi Li, Lanjing Zhang, and Yongfeng Zhang. 2024. https://arxiv.org/abs/2305.18569 Fairness of chatgpt . Preprint, arXiv:2305.18569

    2024 arXiv

  35. [43]

    Guang Liu, Jie Yang, and Ledell Wu. 2022. Ptab: Using the pre-trained language model for modeling tabular data. arXiv preprint arXiv:2209.08060

    2022 arXiv

  36. [45]

    Yanchen Liu, Srishti Gautam, Jiaqi Ma, and Himabindu Lakkaraju. 2024. https://arxiv.org/abs/2310.14607 Confronting llms with traditional ml: Rethinking the fairness of large language models in tabular classifications . Preprint, arXiv:2310.14607

    2024 arXiv

  37. [46]

    Kristian Lum and James Johndrow. 2016. A statistical framework for fair predictive algorithms. arXiv preprint arXiv:1610.08077

    2016 arXiv

  38. [47]

    Natalia Martinez, Martin Bertran, and Guillermo Sapiro. 2020. Minimax pareto fairness: A multi objective perspective. In International conference on machine learning, pages 6755--6764. PMLR

    2020

  39. [48]

    Ggaliwango Marvin, Nakayiza Hellen, Daudi Jjingo, and Joyce Nakatumba-Nabende. 2023. Prompt engineering in large language models. In International Conference on Data Intelligence and Cognitive Informatics, pages 387--402. Springer

    2023

  40. [49]

    Justus Mattern, Zhijing Jin, Mrinmaya Sachan, Rada Mihalcea, and Bernhard Sch \"o lkopf. 2022. Understanding stereotypes in language models: Towards robust measurement and zero-shot debiasing. arXiv preprint arXiv:2212.10678

    2022 arXiv

  41. [50]

    Chandler May, Alex Wang, Shikha Bordia, Samuel R Bowman, and Rachel Rudinger. 2019. On measuring social biases in sentence encoders. arXiv preprint arXiv:1903.10561

    2019 arXiv

  42. [51]

    Ninareh Mehrabi, Fred Morstatter, Nripsuta Saxena, Kristina Lerman, and Aram Galstyan. 2021. A survey on bias and fairness in machine learning. ACM computing surveys (CSUR), 54(6):1--35

    2021

  43. [52]

    Nikita Nangia, Clara Vania, Rasika Bhalerao, and Samuel R Bowman. 2020. Crows-pairs: A challenge dataset for measuring social biases in masked language models. arXiv preprint arXiv:2010.00133

    2020 arXiv

  44. [53]

    Dana Pessach and Erez Shmueli. 2023. Algorithmic fairness. In Machine Learning for Data Science Handbook: Data Mining and Knowledge Discovery Handbook, pages 867--886. Springer

    2023

  45. [54]

    Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. 2018. Catboost: unbiased boosting with categorical features. Advances in neural information processing systems, 31

    2018

  46. [55]

    Shibani Santurkar, Esin Durmus, Faisal Ladhak, Cinoo Lee, Percy Liang, and Tatsunori Hashimoto. 2023. Whose opinions do language models reflect? In International Conference on Machine Learning, pages 29971--30004. PMLR

    2023

  47. [56]

    Dylan Slack and Sameer Singh. 2023. Tablet: Learning from instructions for tabular data. arXiv preprint arXiv:2304.13188

    2023 arXiv

  48. [57]

    Tony Sun, Andrew Gaut, Shirlyn Tang, Yuxin Huang, Mai ElSherief, Jieyu Zhao, Diba Mirza, Elizabeth Belding, Kai-Wei Chang, and William Yang Wang. 2019. Mitigating gender bias in natural language processing: Literature review. arXiv preprint arXiv:1906.08976

    2019 arXiv

  49. [58]

    Alex Tamkin, Amanda Askell, Liane Lovitt, Esin Durmus, Nicholas Joseph, Shauna Kravec, Karina Nguyen, Jared Kaplan, and Deep Ganguli. 2023. Evaluating and mitigating discrimination in language model decisions. arXiv preprint arXiv:2312.03689

    2023 arXiv

  50. [59]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timoth \'e e Lacroix, Baptiste Rozi \`e re, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  51. [60]

    kelly is a warm person, joseph is a role model

    Yixin Wan, George Pu, Jiao Sun, Aparna Garimella, Kai-Wei Chang, and Nanyun Peng. 2023. " kelly is a warm person, joseph is a role model": Gender biases in llm-generated reference letters. arXiv preprint arXiv:2310.09219

    2023 arXiv

  52. [61]

    Jiaqi Wang, Enze Shi, Sigang Yu, Zihao Wu, Chong Ma, Haixing Dai, Qiushi Yang, Yanqing Kang, Jinru Wu, Huawen Hu, et al. 2023. Prompt engineering for healthcare: Methodologies and applications. arXiv preprint arXiv:2304.14670

    2023 arXiv

  53. [62]

    Tianlu Wang, Rohit Sridhar, Diyi Yang, and Xuezhi Wang. 2021. Identifying and mitigating spurious correlations for improving robustness in nlp models. arXiv preprint arXiv:2110.07736

    2021 arXiv

  54. [63]

    Zilong Wang, Hao Zhang, Chun-Liang Li, Julian Martin Eisenschlos, Vincent Perot, Zifeng Wang, Lesly Miculicich, Yasuhisa Fujii, Jingbo Shang, Chen-Yu Lee, et al. 2024. Chain-of-table: Evolving tables in the reasoning chain for table understanding. arXiv preprint arXiv:2401.04398

    2024 arXiv

  55. [64]

    Kellie Webster, Xuezhi Wang, Ian Tenney, Alex Beutel, Emily Pitler, Ellie Pavlick, Jilin Chen, Ed Chi, and Slav Petrov. 2020. Measuring and reducing gendered correlations in pre-trained models. arXiv preprint arXiv:2010.06032

    2020 arXiv

  56. [65]

    Jules White, Quchen Fu, Sam Hays, Michael Sandborn, Carlos Olea, Henry Gilbert, Ashraf Elnashar, Jesse Spencer-Smith, and Douglas C Schmidt. 2023. A prompt pattern catalog to enhance prompt engineering with chatgpt. arXiv preprint arXiv:2302.11382

    2023 arXiv

  57. [66]

    Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. 2023. Large language models as optimizers. arXiv preprint arXiv:2309.03409

    2023 arXiv

  58. [67]

    Pengcheng Yin, Graham Neubig, Wen-tau Yih, and Sebastian Riedel. 2020. Tabert: Pretraining for joint understanding of textual and tabular data. arXiv preprint arXiv:2005.08314

    2020 arXiv

  59. [68]

    Muhammad Bilal Zafar, Isabel Valera, Manuel Gomez Rogriguez, and Krishna P Gummadi. 2017. Fairness constraints: Mechanisms for fair classification. In Artificial intelligence and statistics, pages 962--970. PMLR

    2017

  60. [69]

    Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2017. Men also like shopping: Reducing gender bias amplification using corpus-level constraints. arXiv preprint arXiv:1707.09457

    2017 arXiv

  61. [70]

    Yilun Zhao, Haowei Zhang, Shengyun Si, Linyong Nan, Xiangru Tang, and Arman Cohan. 2023. Investigating table-to-text generation capabilities of large language models in real-world information seeking scenarios. In Proceedings of the 2023 Conference on Empirical Methods in Natu...

    2023

  62. [71]

    Yongchao Zhou, Andrei Ioan Muresanu, Ziwen Han, Keiran Paster, Silviu Pitis, Harris Chan, and Jimmy Ba. 2022. Large language models are human-level prompt engineers. arXiv preprint arXiv:2211.01910

    2022 arXiv

  63. [72]

    Bingzhao Zhu, Xingjian Shi, Nick Erickson, Mu Li, George Karypis, and Mahsa Shoaran. 2023. Xtab: Cross-table pretraining for tabular transformers. arXiv preprint arXiv:2305.06090

    2023 arXiv

  64. [73]

    Ran Zmigrod, Sabrina J Mielke, Hanna Wallach, and Ryan Cotterell. 2019. Counterfactual data augmentation for mitigating gender stereotypes in languages with rich morphology. arXiv preprint arXiv:1906.04571

    2019 arXiv

  65. [74]

    James Zou and Londa Schiebinger. 2018. Ai can be sexist and racist—it’s time to make it fair

    2018

Pith tools