Pith. sign in

REVIEW 3 major objections 4 minor 69 references

Stairway to Fairness: Connecting Group and Individual Fairness

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Group-fair recommendations can be very unfair to individuals.

desk verdict An honest, systematic comparison of group vs individual fairness measures, but the headline result is largely a built-in property of the measures, not an empirical discovery. read the letter →

arxiv 2508.21334 v1 pith:EGTABRH2 submitted 2025-08-29 cs.IR cs.AIcs.CLcs.CY

Theresia Veronika Rampisela , Maria Maistro , Tuukka Ruotsalo , Falk Scholer , Christina Lioma This is my paper · ORCID
classification cs.IRcs.AIcs.CLcs.CY
keywords groupfairnessindividualevaluationrecommendersystemsLLM-basedrecommendersintersectionalityinequalitymeasuresKendall'stau
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that group fairness and individual fairness in recommender systems are not interchangeable, and that between-group fairness scores can mask large per-user unfairness. It does so by applying the same inequality measures — SD, Gini, and Atkinson — to group-mean effectiveness and to individual-user effectiveness, which makes the two fairness notions directly comparable on a common scale. Across eight runs of four LLM-based recommenders and three datasets, the authors find that fairness scores computed for individuals are always worse than group fairness scores, and no individual fairness measure reliably reproduces the ranking of systems given by a group fairness measure. The finding matters because standard practice evaluates only between-group fairness, so a system can be certified as fair while serving some users far worse than others.

What carries the argument

The central machinery is a two-step evaluation pipeline: first compute an effectiveness base score per user (NDCG at cutoff 10), then apply the same inequality measures — standard deviation, Gini, and Atkinson — either to the group-mean base scores for group fairness or to the per-user base scores for individual fairness. The Atkinson index is load-bearing because it is additively decomposable into between-group and within-group components without residual, which lets the paper trace how individual unfairness splits into a between-group part and a within-group part. This decomposition produces the staircase-like pattern reported in the results — between-group unfairness stays low while within-group and individual unfairness are high — which is the mechanism by which group fairness hides individual unfairness. The paper also uses Kendall's tau with an equivalence threshold of 0.9 to test whether any individual measure could stand in as a proxy for a group measure.

What would settle it

Run a broader sweep of recommender systems, say twenty or more covering both collaborative filtering and LLM-based models, on the same three datasets, and compute the paper's group and individual Gini scores on NDCG@10 together with Kendall's tau at the 0.9 equivalence threshold; if any individual fairness measure consistently agrees with a group fairness measure, or if individual fairness scores are frequently no worse than group fairness scores, the paper's central claim would be contradicted.

Watch

Extended reading notes

Core claim

The paper's central claim is that, measured with the same families of fairness measures on the same effectiveness scores, group fairness and individual fairness are empirically disjoint: a recommender can be highly fair between groups and very unfair to individual users, because group-level scores average away within-group variation. The authors demonstrate this by computing NDCG at cutoff 10 for eight runs of four open-weight LLM recommenders on three datasets, then applying ten between-group fairness measures and three individual fairness measures to those effectiveness scores. They find that individual fairness scores are consistently worse than between-group fairness scores; that within-group unfairness is consistently worse than between-group unfairness and tracks individual unfairness; and that no individual fairness measure ranks systems equivalently to any group fairness measure under Kendall's tau at the 0.9 equivalence threshold. The only group measure showing moderate agreement with individual Gini and Atkinson is the coefficient of variation. The conclusion is that between-group fairness evaluation alone can certify systems that serve some users far worse than others, so evaluation should report between-group, within-group, and individual fairness together.

Load-bearing premise

The load-bearing premise is that comparing fairness scores computed on group-mean effectiveness with fairness scores computed on individual-user effectiveness is meaningful, rather than an artifact of aggregating different numbers of units.

Editorial extensions

If this is right

  • Practitioners who evaluate only between-group fairness may certify systems that are markedly less fair to individual users; the paper shows this occurring for LLM-based recommenders on all three datasets.
  • No existing individual fairness measure can serve as a reliable proxy for a group fairness measure, so evaluation should include both group and individual scores rather than assuming one implies the other.
  • Fairness scores worsen as more sensitive attributes form intersectional groups, so the number of groups itself affects how fair a system appears.
  • Within-group unfairness is consistently larger than between-group unfairness and comparable to individual unfairness, meaning that optimizing between-group fairness alone leaves the largest source of unfairness unaddressed.
  • Using the same measure families for group and individual fairness makes the two scores directly comparable, which previous work could not do because different measure families have different sensitivities and ranges.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A practical audit rule follows: for any recommender, decompose per-user NDCG inequality with the Atkinson index into between- and within-group parts; if the within-group component dominates, the group fairness score is nearly uninformative about user-level fairness. This is an extension the paper does not itself propose as a tool.
  • The same aggregation mechanism plausibly extends to item or provider fairness, where effectiveness or exposure is averaged per provider group; the paper anticipates this but provides no empirical test, so measuring provider-side individual unfairness under the same decomposition is a natural next experiment.
  • A synthetic control study could separate the mathematical necessity from the empirical discovery: generate per-user effectiveness values with controlled within-group variance, and confirm that the gap between group and individual fairness scores is exactly the within-group variance captured by the decomposition, which would show whether the LLM recommenders add anything beyond the general inequali
  • Future mitigation work could add a per-user inequality penalty, such as a Gini term on per-user NDCG, to the loss function; the paper's decomposition suggests such a penalty would primarily reduce within-group unfairness, and whether that also improves or harms between-group fairness is a testable question.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. This paper studies the relationship between group and individual fairness evaluation in recommender systems. Using four LLM-based recommenders, each with a sensitive and a non-sensitive prompt variant, across three datasets, the authors compute per-user and per-group effectiveness (NDCG and Precision) and then apply nine group-fairness measures and three individual-fairness measures to those base scores. They report that group fairness scores are systematically better (lower inequality) than individual fairness scores, that no individual fairness measure reliably agrees with group fairness measures in ranking models (Kendall's tau), and that within-group unfairness is close to individual unfairness. The paper concludes that group fairness can mask individual unfairness and presents this as the first empirical evidence of the disjointness of the two fairness concepts.

Significance. If the empirical claims were properly grounded, the paper would be a useful practical warning for recommender-system practitioners and a systematic comparison of fairness measures under a common base score. The manuscript's strengths include public code, a wide coverage of the group-fairness measure space, and the explicit computation of between- versus within-group decompositions. However, the headline conclusion is largely predetermined by the measurement construction for the decomposable inequality measures, and the ranking-agreement analysis is statistically underpowered, so the significance of the results as stated is currently not established.

major comments (3)
  1. [Section 3 (RQ2, Fig. 3) and Section 2 (Individual fairness)] The result that individual Gini, Atkinson, and CV scores are always worse than the corresponding between-group scores is a mathematical consequence of computing the measures on per-user scores rather than group means. For any partition of users, averaging within groups is a mean-preserving contraction, so the between-group component of a decomposable inequality measure cannot exceed the total inequality; the paper itself notes in Section 2 that Atkind decomposes without residuals into between- and within-group components. The manuscript therefore cannot claim, as it does in the Discussion, that the observed gap is 'empirical evidence' of disjointness. The claim should be reframed as an illustration of a known decomposition property, and the magnitude of the gap should be compared against a null baseline (e.g., random grouping, or a baseline derived from the within-group variance) to establish that the observed values are informative.
  2. [Section 3 (RQ1, Fig. 2)] Kendall's tau is computed from only 8 runs (4 LLMs × 2 prompt types) without repeated data splits or confidence intervals. For n=8, the 95% confidence interval of Kendall's tau is approximately ±0.4, so the conclusion that 'no individual Fair measure consistently has equivalent rankings' is not supported by the reported precision. The authors should report bootstrap or permutation intervals, or increase the number of runs, before drawing the 'no reliable proxy' conclusion.
  3. [Section 2 (Evaluation, LLM matching threshold)] The only relevance signal is fuzzy string matching at a fixed TF-IDF n-gram threshold of 0.75, and per-user NDCG is heavily zero-inflated at this threshold, which mechanically inflates individual inequality relative to group means. A sensitivity analysis across thresholds (e.g., 0.6, 0.7, 0.8) is required to verify that the qualitative gap between group and individual fairness is not an artifact of the matching threshold.
minor comments (4)
  1. [Fig. 3 caption] The caption states that 'the figures should not be used to quantify the gap between two families of measures,' which appears to contradict the text in Section 3 stating that individual Fair scores are 'much less fair' than group scores; please clarify what conclusions the reader may legitimately draw from the figure.
  2. [Abstract] The phrase '8 runs across 3 datasets' is ambiguous; the runs come from four LLMs each used with a sensitive and a non-sensitive prompt, so please phrase it as '4 LLMs with sensitive and non-sensitive prompts across 3 datasets.'
  3. [Section 2 (Atkinson index)] The main text does not specify the inequality-aversion parameter ε used for the Atkinson index; please state it in the main text or point to the exact appendix equation so the decomposition can be reproduced.
  4. [Footnote 10] The statement that similar results for collaborative-filtering recommenders are omitted for brevity should be supported by placing the full results in the repository or an appendix; otherwise the abstract's general claim about recommender systems is too broad for the evidence presented.

Circularity Check

2 steps flagged · score 6.0 of 10

The individual-vs-group fairness gap is a mathematical consequence of aggregating user scores to group means; the headline 'first empirical evidence' restates a known decomposition theorem.

  1. self definitional [Sec. 2 (Evaluation, Individual fairness); Sec. 3 (Intersectional fairness, RQ2)]
    "Group and individual Fair measures are computed in two steps: first, computing an Eff score per user/group as a ‘base score’; and second, aggregating the ‘base score’ between users/groups with a Fair measure. ... while Atkind can be decomposed into between- and within-group fairness with no residuals [3, 4, 6]. ... For all measures and datasets, the individual Fair scores are always worse than groupFair scores."

    Group Fair uses the group-mean Eff as its base score; individual Fair uses each user's Eff. For the Schur-convex inequality measures used (SD, Gini, Atk), replacing individual scores by group means is a mean-preserving contraction that cannot increase inequality. The paper itself states Atkind decomposes into between- and within-group components with no residuals, so Atkind >= Atkb-group is an identity, and the analogous inequality for SD/Gini follows from total-variance/inequality decomposition. Hence 'individual Fair scores are always worse than groupFair scores' is guaranteed for any recommender and any non-degenerate grouping, not an empirical discovery.

  2. renaming known result [Sec. 4 (Discussion and Conclusion)]
    "Our results show that RSs which are fair for groups can still be very unfair for individual users, providing the first empirical evidence on the disjointness of these two RS fairness concepts."

    This sentence presents as novel empirical evidence a conclusion that is entailed by the measurement construction: any positive within-group variance makes the individual-level inequality score exceed the between-group score for SD, Gini, and Atk. The 'disjointness' in score level is a textbook consequence of inequality decomposition (cited by the paper itself via [3, 4, 6]) rather than a property of recommender systems. The genuinely empirical parts (which LLMRec is fairest, Kendall-tau agreement across measures) are separate; but the qualitative disjointness claim does not require the experiments, so calling it 'first empirical evidence' is a renaming of a known mathematical result.

full rationale

The paper's primary stated contribution is the empirical finding that group-fair recommendations can be individually unfair. That qualitative direction is not learned from the experiments: it is entailed by how the two fairness families are computed. Group fairness scores are calculated on group-mean effectiveness; individual fairness scores are calculated on per-user effectiveness. Since SD is the square root of total variance, Gini/Atkinson are Schur-convex inequality measures, and averaging within groups is a mean-preserving contraction, the individual-level score dominates the between-group score whenever groups are internally heterogeneous. The paper explicitly cites the residual-free decomposition of Atkind, which makes Atkind >= Atkb-group an equation, not a finding. The empirical tables and figures therefore instantiate a known decomposition theorem, and the 'first empirical evidence' claim is overstated. The ranking-agreement analysis (RQ1, Fig. 2) and the measured magnitudes are independent empirical content: it is not circular to ask which systems or measures show larger gaps or disagree in rankings. Also, the paper's self-citations [45, 46] are used for framing and prior measure analysis, not as load-bearing support for the central claim, so no additional circularity arises there. Overall, the qualitative headline reduces by construction, while the quantitative/ranking parts remain self-contained, warranting a partial-circularity score of 6.

Assumptions & free parameters 4 free parameters · 3 assumptions · 0 invented entities

The paper does not fit a predictive model, so no model parameters are learned. The listed free parameters are hand-chosen thresholds and cutoffs that affect the empirical results. The main assumptions are about comparability of group-mean and individual-level fairness scores, decomposition properties of inequality measures, and the validity of fuzzy matching for LLM outputs.

free parameters (4)
  • Item matching similarity threshold = 0.75
    LLM recommendations are matched to test items via TF-IDF n-gram similarity if the score is at least 0.75. This threshold determines which recommendations count as hits and therefore affects every effectiveness and fairness score; no sensitivity analysis is provided.
  • Recommendation cutoff k = 10
    All effectiveness and fairness measures are computed at k=10. Results may differ at other cutoffs.
  • Train-set minimum interaction threshold t = 5 for ML-1M and LFM-1B, 2 for JobRec
    Users and items with at most t train interactions are removed; t is chosen so at least 500 test users remain, changing dataset composition.
  • Core filtering = 5-core for ML-1M and JobRec, 50-core for LFM-1B
    Sparsity reduction choices based on prior work; affects which users and items are retained.
assumptions (3)
  • domain assumption Inequality measures such as Gini and Atkinson can be additively decomposed into between-group and within-group components, and individual inequality is at least as large as between-group inequality for these decompositions.
    The paper relies on this to interpret Atk and Gini decomposition in RQ3/RQ4, citing [3, 4, 6, 53].
  • ad hoc to paper Group fairness is appropriately measured by applying an inequality measure to group-mean effectiveness scores, and individual fairness by applying the same measure to per-user scores; the two results are comparable.
    This comparability assumption underlies the central comparison in Section 2 Evaluation; it is a methodological choice rather than an established standard.
  • ad hoc to paper LLM recommendations matched by fuzzy string similarity at threshold 0.75 are valid ground-truth hits for effectiveness evaluation.
    The paper notes that no existing work evaluates the effect of similarity thresholds; all effectiveness and fairness scores depend on this mapping.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Stairway to Fairness: Connecting Group and Individual Fairness." pith.science (2026). https://pith.science/paper/EGTABRH2

@misc{pith2026250821334,
  author       = {Pith},
  title        = {Pith review of: Stairway to Fairness: Connecting Group and Individual Fairness},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/EGTABRH2}},
  note         = {Machine review of arXiv:2508.21334}
}
read the original abstract

Fairness in recommender systems (RSs) is commonly categorised into group fairness and individual fairness. However, there is no established scientific understanding of the relationship between the two fairness types, as prior work on both types has used different evaluation measures or evaluation objectives for each fairness type, thereby not allowing for a proper comparison of the two. As a result, it is currently not known how increasing one type of fairness may affect the other. To fill this gap, we study the relationship of group and individual fairness through a comprehensive comparison of evaluation measures that can be used for both fairness types. Our experiments with 8 runs across 3 datasets show that recommendations that are highly fair for groups can be very unfair for individuals. Our finding is novel and useful for RS practitioners aiming to improve the fairness of their systems. Our code is available at: https://github.com/theresiavr/stairway-to-fairness.

Figures

Figures reproduced from arXiv: 2508.21334 by the authors.

Figure 1
Figure 1. Top: NDCG score distribution of two handpicked, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Agreement (Kendall’s 𝜏) of NDCG-based measures for individual Fair (𝑦-axis) and group Fair (𝑥-axis) in ranking LLMRecs. User groups are based on 3 sensitive attributes. Due to 8-way ties, 𝜏 cannot be computed for Min (JobRec). group fairness. We study: (i) if/how the number of attributes used for grouping affect fairness; and (ii) the difference between individ￾ual and (intersectional) group Fair scores. To this end… view at source ↗
Figure 3
Figure 3. NDCG-based Group (Grp) and individual (Ind) [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: NDCG-based individual, between- and within-group [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

69 extracted references · 20 canonical work pages

  1. [1]

    Enrique Amigó, Yashar Deldjoo, Stefano Mizzaro, and Alejandro Bellogín. 2023. A unifying and general account of fairness measurement in recommender systems. Information Processing & Management 60, 1 (1 2023), 103115. https://doi.org/10. 1016/J.IPM.2022.103115

    work page arXiv 2023

  2. [2]

    Anthony B Atkinson. 1970. On the measurement of inequality. Journal of Economic Theory 2, 3 (1970), 244–263. https://doi.org/10.1016/0022-0531(70) 90039-6

    work page doi:10.1016/0022-0531(70 1970

  3. [3]

    1999.Income Inequality Measurement: The Normative Approach

    Charles Blackorby, Walter Bossert, and David Donaldson. 1999.Income Inequality Measurement: The Normative Approach. Springer Netherlands, Dordrecht, 133–

    work page 1999

  4. [4]

    Francois Bourguignon. 1979. Decomposable Income Inequality Measures. Econo- metrica 47, 4 (1979), 901–920. http://www.jstor.org/stable/1914138

    work page arXiv 1979

  5. [5]

    Team Cohere. 2025. Command A: An Enterprise-Ready Large Language Model. arXiv:2504.00698 [cs.CL] https://arxiv.org/abs/2504.00698

    arXiv 2025

  6. [6]

    Meltem Dayioğlu and Cem Başlevent. 2006. Imputed Rents and Regional Income Inequality in Turkey: A Subgroup Decomposition of the Atkinson Index.Regional Studies 40, 8 (2006), 889–905. https://doi.org/10.1080/00343400600984395

    doi:10.1080/00343400600984395 2006

  7. [7]

    Yashar Deldjoo. 2024. Understanding Biases in ChatGPT-based Recommender Systems: Provider Fairness, Temporal Stability, and Recency.ACM Trans. Recomm. RecSys ’25, September 22–26, 2025, Prague, Czech Republic Rampisela et al. Syst. (Aug. 2024). https://doi.org/10.1145/3690655 Just Accepted

    doi:10.1145/3690655 2024

  8. [8]

    Yashar Deldjoo, Vito Walter Anelli, Hamed Zamani, Alejandro Bellogín, and Tommaso Di Noia. 2019. Recommender Systems Fairness Evaluation via Gen- eralized Cross Entropy. In Proceedings of the Workshop on Recommendation in Multi-stakeholder Environments co-located with the 13th ACM Conference on Rec- ommender Systems (RecSys 2019) . CEUR-WS

    work page 2019

Show all 69 references
  1. [9]

    Yashar Deldjoo, Vito Walter Anelli, Hamed Zamani, Alejandro Bellogín, and Tommaso Di Noia. 2021. A flexible framework for evaluating user and item fairness in recommender systems. User Modeling and User-Adapted Interaction 31 (2021), 457–511. https://doi.org/10.1007/s11257-020-09285-1

    2021 doi

  2. [10]

    Yashar Deldjoo and Tommaso Di Noia. 2025. CFaiRLLM: Consumer Fairness Evaluation in Large-Language Model Recommender System. ACM Trans. Intell. Syst. Technol. (March 2025). https://doi.org/10.1145/3725853 Just Accepted

    2025 doi

  3. [11]

    Yashar Deldjoo, Dietmar Jannach, Alejandro Bellogin, Alessandro Difonzo, and Dario Zanzonelli. 2024. Fairness in recommender systems: research landscape and future directions. User Modeling and User-Adapted Interaction 34, 1 (2024), 59–108. https://doi.org/10.1007/s11257-023-09364-z

    2024 doi

  4. [12]

    Yashar Deldjoo and Fatemeh Nazary. 2024. A Normative Framework for Bench- marking Consumer Fairness in Large Language Model Recommender System. arXiv:2405.02219 [cs.IR] https://arxiv.org/abs/2405.02219

    2024 arXiv

  5. [13]

    Virginie Do. 2023. Fairness in recommender systems: insights from social choice . Theses. Université Paris sciences et lettres. https://theses.hal.science/tel- 04213955

    2023

  6. [14]

    Michael D Ekstrand, Mucun Tian, Ion Madrazo Azpiazu, Jennifer D Ekstrand, Oghenemaro Anuyah, David McNeill, and Maria Soledad Pera. 2018. All The Cool Kids, How Do They Fit In?: Popularity and Demographic Biases in Rec- ommender Evaluation and Effectiveness. In Proceedings of ...

    2018

  7. [15]

    Eurostat. 2018. Young people on the labour market - statistics. https://ec.europa.eu/eurostat/statistics-explained/index.php?title=Young_ people_on_the_labour_market_-_statistics#Half_worked_while_studying

    2018

  8. [17]

    Zuohui Fu, Yikun Xian, Ruoyuan Gao, Jieyu Zhao, Qiaoying Huang, Yingqiang Ge, Shuyuan Xu, Shijie Geng, Chirag Shah, Yongfeng Zhang, and Gerard de Melo. 2020. Fairness-Aware Explainable Recommendation over Knowledge Graphs. In Proceedings of the 43rd International ACM SIGIR Con...

    2020

  9. [18]

    Avijit Ghosh, Tomo Lazovich, Kristian Lum, and Christo Wilson. 2024. Reducing Population-level Inequality Can Improve Demographic Group Fairness: a Twitter Case Study. arXiv:2409.08135 [cs.SI] https://arxiv.org/abs/2409.08135

    2024 arXiv

  10. [19]

    Corrado Gini. 1912. Variabilità e Mutabilità. C. Cuppini, Bologna. Contributo allo Studio delle Distribuzioni e delle Relazioni Statistiche

    1912

  11. [20]

    Team GLM. 2024. ChatGLM: A Family of Large Language Models from GLM-130B to GLM-4 All Tools. arXiv:2406.12793

    2024 arXiv

  12. [21]

    Elizabeth Gómez, David Contreras, Ludovico Boratto, and Maria Salamo. 2024. AMBAR: A dataset for Assessing Multiple Beyond-Accuracy Recommenders. In Proceedings of the 18th ACM Conference on Recommender Systems (Bari, Italy) (RecSys ’24). Association for Computing Machinery, N...

    2024

  13. [22]

    Aaron Grattafiori et al . 2024. The Llama 3 Herd of Models. arXiv:2407.21783 [cs.AI] https://arxiv.org/abs/2407.21783

    2024 arXiv

  14. [23]

    Ben Hamner, Road Warrior, and Wojciech Krupa. 2012. Job Recommendation Challenge. https://kaggle.com/competitions/job-recommendation

    2012

  15. [24]

    Maxwell Harper and Joseph A

    F. Maxwell Harper and Joseph A. Konstan. 2015. The MovieLens Datasets: History and Context. ACM Trans. Interact. Intell. Syst. 5, 4, Article 19 (Dec. 2015), 19 pages. https://doi.org/10.1145/2827872

    2015 doi

  16. [25]

    Zhankui He, Zhouhang Xie, Rahul Jha, Harald Steck, Dawen Liang, Yesu Feng, Bodhisattwa Prasad Majumder, Nathan Kallus, and Julian Mcauley. 2023. Large Language Models as Zero-Shot Conversational Recommenders. In Proceedings of the 32nd ACM International Conference on Informati...

    2023

  17. [26]

    Yupeng Hou, Junjie Zhang, Zihan Lin, Hongyu Lu, Ruobing Xie, Julian McAuley, and Wayne Xin Zhao. 2024. Large Language Models are Zero-Shot Rankers for Recommender Systems. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Le...

    2024 doi

  18. [27]

    Kalervo Järvelin and Jaana Kekäläinen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Trans. Inf. Syst. 20, 4 (Oct. 2002), 422–446. https://doi. org/10.1145/582415.582418

    2002

  19. [28]

    Chumeng Jiang, Jiayin Wang, Weizhi Ma, Charles L. A. Clarke, Shuai Wang, Chuhan Wu, and Min Zhang. 2025. Beyond Utility: Evaluating LLM as Rec- ommender. In Proceedings of the ACM on Web Conference 2025 (Sydney NSW, Australia) (WWW ’25). Association for Computing Machinery, Ne...

    2025

  20. [29]

    Karen Sparck Jones. 1972. A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation 28, 1 (1972), 11–21. https: //doi.org/10.1108/EB026526/FULL/PDF

    1972 doi

  21. [30]

    Walid Krichene and Steffen Rendle. 2022. On sampled metrics for item recom- mendation. Commun. ACM 65, 7 (2022), 75–83. https://doi.org/10.1145/3535335

    2022 doi

  22. [31]

    Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. Efficient Memory Management for Large Language Model Serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Princip...

    2023

  23. [32]

    Jurek Leonhardt, Avishek Anand, and Megha Khosla. 2018. User Fairness in Recommender Systems. In Companion Proceedings of the The Web Conference 2018 (Lyon, France) (WWW ’18). International World Wide Web Conferences Steering Committee, Republic and Canton of Geneva, CHE, 101–...

    2018

  24. [33]

    Yunqi Li, Hanxiong Chen, Shuyuan Xu, Yingqiang Ge, Juntao Tan, Shuchang Liu, and Yongfeng Zhang. 2023. Fairness in Recommendation: Foundations, Methods, and Applications. ACM Trans. Intell. Syst. Technol. 14, 5, Article 95 (Oct. 2023), 48 pages. https://doi.org/10.1145/3610302

    2023 doi

  25. [34]

    Jianxun Lian, Yuxuan Lei, Xu Huang, Jing Yao, Wei Xu, and Xing Xie. 2024. RecAI: Leveraging Large Language Models for Next-Generation Recommender Systems. In Companion Proceedings of the ACM Web Conference 2024 (Singapore, Singapore) (WWW ’24). Association for Computing Machin...

    2024

  26. [35]

    Tingting Liang, Chenxin Jin, Lingzhi Wang, Wenqi Fan, Congying Xia, Kai Chen, and Yuyu Yin. 2024. LLM-REDIAL: A Large-Scale Dataset for Conversational Recommender Systems Created from User Behaviors with LLMs. In Findings of the Association for Computational Linguistics: ACL 2...

    2024 doi

  27. [36]

    Zhiqiang Liu, Xiaoxiao Xu, Jiaqi Yu, Han Xu, Lantao Hu, Han Li, and Kun Gai. 2024. A Self-Adaptive Fairness Constraint Framework for Industrial Rec- ommender System. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management (Boise, ID, USA...

    2024

  28. [37]

    Maria Maistro, Lucas Chaves Lima, Jakob Grue Simonsen, and Christina Lioma

  29. [38]

    Elizaveta Makhneva, Anna Sverkunova, Oleg Lashinin, Marina Ananyeva, and Sergey Kolesnikov. 2023. Make your next item recommendation model time sen- sitive. In Adjunct Proceedings of the 31st ACM Conference on User Modeling, Adap- tation and Personalization (UMAP ’23 Adjunct) ...

    2023

  30. [39]

    Zaiqiao Meng, Richard McCreadie, Craig MacDonald, and Iadh Ounis. 2020. Exploring Data Splitting Strategies for the Evaluation of Recommendation Models. In RecSys 2020 - 14th ACM Conference on Recommender Systems . Association for Computing Machinery, Inc, Virtual Event, Brazi...

    2020

  31. [40]

    Office for National Statistics. 2023. Age classifications: Census 2021. https://www.ons.gov.uk/census/census2021dictionary/variablesbytopic/ demographyvariablescensus2021/age/classifications

    2023

  32. [41]

    Dario Di Palma, Giovanni Maria Biancofiore, Vito Walter Anelli, Fedelucio Nar- ducci, Tommaso Di Noia, and Eugenio Di Sciascio. 2024. Evaluating ChatGPT as a Recommender System: A Rigorous Approach. arXiv:2309.03613 [cs.IR] https://arxiv.org/abs/2309.03613

    2024 arXiv

  33. [42]

    Eliana Pastor and Francesco Bonchi. 2024. Intersectional fair ranking via subgroup divergence. Data Min. Knowl. Discov. 38, 4 (May 2024), 2186–2222. https://doi. org/10.1007/s10618-024-01029-8

    2024 doi

  34. [43]

    Gummadi, and Abhi- jnan Chakraborty

    Gourab K Patro, Arpita Biswas, Niloy Ganguly, Krishna P. Gummadi, and Abhi- jnan Chakraborty. 2020. FairRec: Two-Sided Fairness for Personalized Recom- mendations in Two-Sided Platforms. In Proceedings of The Web Conference 2020 (Taipei, Taiwan) (WWW ’20). Association for Comp...

    2020

  35. [44]

    Giovanni Pellegrini, Vittorio Maria Faraco, and Yashar Deldjoo. 2023. Fairness for All: Investigating Harms to Within-Group Individuals in Producer Fairness Re-ranking Optimization – A Reproducibility Study. arXiv:2309.09277 [cs.IR] https://arxiv.org/abs/2309.09277 Stairway to...

    2023 arXiv

  36. [45]

    Theresia Veronika Rampisela, Maria Maistro, Tuukka Ruotsalo, and Christina Lioma. 2024. Evaluation Measures of Individual Item Fairness for Recommender Systems: A Critical Study. ACM Trans. Recomm. Syst. 3, 2, Article 18 (Nov. 2024), 52 pages. https://doi.org/10.1145/3631943

    2024 doi

  37. [46]

    Theresia Veronika Rampisela, Tuukka Ruotsalo, Maria Maistro, and Christina Lioma. 2024. Can We Trust Recommender System Fairness Evaluation? The Role of Fairness and Relevance. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Informa...

    2024

  38. [47]

    Gummadi, and Mark Crovella

    Bashir Rastegarpanah, Krishna P. Gummadi, and Mark Crovella. 2019. Fighting Fire with Fire: Using Antidote Data to Improve Polarization and Fairness of Recommender Systems. In Proceedings of the Twelfth ACM International Con- ference on Web Search and Data Mining (Melbourne VI...

    2019

  39. [48]

    Mathieu Ravaut, Hao Zhang, Lu Xu, Aixin Sun, and Yong Liu. 2024. Parameter- Efficient Conversational Recommender System as a Language Processing Task. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long...

    2024

  40. [49]

    Markus Schedl. 2016. The LFM-1b Dataset for Music Retrieval and Recommenda- tion. In Proceedings of the 2016 ACM on International Conference on Multimedia Retrieval (ICMR ’16) . Association for Computing Machinery, New York, NY, USA, 103–110. https://doi.org/10.1145/2911996.2912004

    2016

  41. [50]

    Tobias Schumacher, Marlene Lutz, Sandipan Sikdar, and Markus Strohmaier. 2025. Properties of Group Fairness Measures for Rankings. Trans. Soc. Comput. 8, 1–2, Article 2 (Jan. 2025), 45 pages. https://doi.org/10.1145/3674883

    2025 doi

  42. [51]

    A. F. Shorrocks. 1980. The Class of Additively Decomposable Inequality Measures. Econometrica 48, 3 (1980), 613–625. http://www.jstor.org/stable/1913126

    1980

  43. [53]

    Gummadi, Adish Singla, Adrian Weller, and Muhammad Bilal Zafar

    Till Speicher, Hoda Heidari, Nina Grgic-Hlaca, Krishna P. Gummadi, Adish Singla, Adrian Weller, and Muhammad Bilal Zafar. 2018. A Unified Approach to Quan- tifying Algorithmic Unfairness: Measuring Individual & Group Unfairness via Inequality Indices. In Proceedings of the 24t...

    2018

  44. [54]

    Mistral AI team. 2024. Un Ministral, des Ministraux. https://mistral.ai/news/ ministraux. Accessed: 2025-04-23

    2024

  45. [55]

    Qwen Team. 2024. Qwen2.5: A Party of Foundation Models. https://qwenlm. github.io/blog/qwen2.5/

    2024

  46. [56]

    Antonela Tommasel. 2024. Fairness Matters: A look at LLM-generated group recommendations. In 18th ACM Conference on Recommender Systems . ACM, New York, NY, USA, 993–998. https://doi.org/10.1145/3640457.3688182

    2024

  47. [57]

    Bureau of Labor Statistics

    U.S. Bureau of Labor Statistics. 2023. May 2023 National Occupational Employ- ment and Wage Estimates. https://www.bls.gov/oes/current/oes_nat.htm#top

    2023

  48. [58]

    Bureau of Labor Statistics

    U.S. Bureau of Labor Statistics. 2024. College Enrollment and Work Activity of Recent High School and College Graduates Summary. https://www.bls.gov/ news.release/hsgec.nr0.htm

    2024

  49. [59]

    Mengting Wan, Jianmo Ni, Rishabh Misra, and Julian McAuley. 2020. Address- ing Marketing Bias in Product Recommendations. In Proceedings of the 13th International Conference on Web Search and Data Mining (Houston, TX, USA) (WSDM ’20). Association for Computing Machinery, New Y...

    2020

  50. [60]

    Yifan Wang, Weizhi Ma, Min Zhang, Yiqun Liu, and Shaoping Ma. 2023. A Survey on the Fairness of Recommender Systems. ACM Trans. Inf. Syst. 41, 3 (2 2023), 1–43. https://doi.org/10.1145/3547333

    2023 doi

  51. [61]

    Yifan Wang, Peijie Sun, Weizhi Ma, Min Zhang, Yuan Zhang, Peng Jiang, and Shaoping Ma. 2024. Intersectional Two-sided Fairness in Recommendation. In Proceedings of the ACM Web Conference 2024 (Singapore, Singapore) (WWW ’24). Association for Computing Machinery, New York, NY, ...

    2024

  52. [62]

    Yan Wen, Chen Gao, Lingling Yi, Liwei Qiu, Yaqing Wang, and Yong Li. 2023. Efficient and Joint Hyperparameter and Architecture Search for Collaborative Filtering. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Dis- covery and Data Mining (KDD ’23) . Association ...

    2023

  53. [63]

    Yao Wu, Jian Cao, Guandong Xu, and Yudong Tan. 2021. TFROM: A Two-sided Fairness-Aware Recommendation Model for Both Customers and Providers. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval (Virtual Event, Cana...

    2021

  54. [64]

    Chen Xu, Wenjie Wang, Yuxin Li, Liang Pang, Jun Xu, and Tat-Seng Chua. 2024. A Study of Implicit Ranking Unfairness in Large Language Models. InFindings of the Association for Computational Linguistics: EMNLP 2024 , Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Ass...

    2024

  55. [65]

    Jizhi Zhang, Keqin Bao, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. 2023. Is ChatGPT Fair for Recommendation? Evaluating Fairness in Large Language Model Recommendation. In Proceedings of the 17th ACM Conference on Recommender Systems (Singapore, Singapore) (RecSys ’2...

    2023

  56. [66]

    Kesen Zhao, Shuchang Liu, Qingpeng Cai, Xiangyu Zhao, Ziru Liu, Dong Zheng, Peng Jiang, and Kun Gai. 2023. KuaiSim: a comprehensive simulator for recom- mender systems. In Proceedings of the 37th International Conference on Neural Information Processing Systems (New Orleans, L...

    2023

  57. [67]

    Wayne Xin Zhao, Shanlei Mu, Yupeng Hou, Zihan Lin, Yushuo Chen, Xingyu Pan, Kaiyuan Li, Yujie Lu, Hui Wang, Changxin Tian, Yingqian Min, Zhichao Feng, Xinyan Fan, Xu Chen, Pengfei Wang, Wendi Ji, Yaliang Li, Xiaoling Wang, and Ji Rong Wen. 2021. RecBole: Towards a Unified, Com...

    2021

  58. [68]

    Ziwei Zhu, Jianling Wang, and James Caverlee. 2020. Measuring and Miti- gating Item Under-Recommendation Bias in Personalized Ranking Systems. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval (Virtual Event, Chi...

    2020

  59. [161]

    https://doi.org/10.1007/978-94-011-4413-1_4

    doi

  60. [165]

    https://doi.org/10.18653/v1/2024.eacl-long.9

    2024 doi

  61. [2021]

    InProceedings of the 30th ACM International Conference on Information & Knowledge Management (Virtual Event, Queensland, Australia) (CIKM ’21)

    Principled Multi-Aspect Evaluation Measures of Rankings. InProceedings of the 30th ACM International Conference on Information & Knowledge Management (Virtual Event, Queensland, Australia) (CIKM ’21). Association for Computing Machinery, New York, NY, USA, 1232–1242. https://d...

    doi

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.