Recommender Systems Fairness Evaluation via Generalized Cross Entropy

Alejandro Bellogin; Hamed Zamani; Tommaso Di Noia; Vito Walter Anelli; Yashar Deldjoo

REVIEW 3 major objections 5 minor 32 references

Recommender Systems Fairness Evaluation via Generalized Cross Entropy

T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Fairness in recommender systems should be measured as the divergence between the actual distribution of recommendation gains and a designer-specified fair distribution, not as simple equality.

desk verdict Generalized cross entropy with a designer-specified fair distribution is a useful extension of RS fairness metrics, but the paper's empirical support is thin and the fairness verdict reduces to the chosen target, so it is a solid workshop proposal rather than a validated standard. read the letter →

arxiv 1908.06708 v1 pith:56OMW2O2 submitted 2019-08-19 cs.IR

Yashar Deldjoo , Vito Walter Anelli , Hamed Zamani , Alejandro Bellogin , Tommaso Di Noia This is my paper · ORCID

classification cs.IR

keywords recommendersystemsfairnessgeneralizedcrossentropyevaluationuseritemfairdistributionmultistakeholderbeyondparity

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that fairness in recommender systems should not be equated with equal treatment of all user or item groups. Instead, it proposes judging a recommender by how closely its actual distribution of recommendation gains matches a designer-specified 'fair' distribution, measured by generalized cross entropy (GCE). The framework is meant to accommodate any fairness definition expressible as a target distribution, including policies that deliberately favor premium users or other groups based on merit and need. Experiments with item-level and user-level attributes on two real datasets show that the metric can rank recommenders differently from both accuracy measures and standard deviation-based fairness baselines. If the claim holds, evaluating fairness and searching for fair recommenders become a single divergence-minimization problem.

What carries the argument

The central object is the generalized cross entropy divergence $I(M,a)$, defined in Eq. (2) as $\frac{1}{\alpha(1-\alpha)}[\sum_{a_j} p_f(a_j)^\alpha p(a_j)^{1-\alpha} - 1]$ for discrete attributes. It measures how far the system's observed gain distribution $p$ is from the fair target $p_f$, and its minimizer is $p = p_f$. The performance distribution $p$ is built from per-group recommendation gains (Eq. 3 for items, Eq. 4 for users), where the gain function can be constant (count), binary relevance, or DCG-based. Different choices of $p_f$ let the same formula express equality-based fairness, merit-based non-uniform fairness, or any designer-defined target.

What would settle it

Set $p_f$ equal to each system's own empirical $p$ on a fixed dataset; then GCE becomes identically zero and every recommender is declared perfectly fair, regardless of any difference among them. This observation demonstrates that, without an externally justified $p_f$, the metric carries no fairness information on its own.

Watch

Extended reading notes

Core claim

The paper's central claim is that unfairness of a recommender with respect to an attribute $a$ (for users or items) is the generalized cross entropy between the system's actual performance distribution $p$ and an ideal fair distribution $p_f$: $I(M,a) = \frac{1}{\alpha(1-\alpha)}[\sum_{a_j} p_f(a_j)^\alpha p(a_j)^{1-\alpha} - 1]$, which is minimized only when $p = p_f$. The distribution $p$ is estimated from recommendation gains (counts, relevance, or rank-discounted gains such as DCG) aggregated over attribute groups. The distribution $p_f$ is the designer's specification of what fair treatment means: when $p_f$ is uniform, GCE reduces to generalized entropy and fairness means equality; when $p_f$ is non-uniform, fairness encodes merit or need. This single framework is applied to item fairness on the Xing job dataset and user fairness on an Amazon subset, and it reveals over- and under-representation of groups that accuracy metrics and MAD baselines do not surface.

Load-bearing premise

The framework's fairness verdict is only as sound as the designer-supplied fair distribution $p_f$; if $p_f$ is arbitrary or chosen to match the algorithm's behavior, the score measures conformance to a target, not fairness.

Editorial extensions

If this is right

A single numerical score can evaluate user fairness and item fairness for any attribute, sensitive or not, by choosing the appropriate gain function and $p_f$.
Under a uniform $p_f$, the proposed GCE coincides with generalized entropy measures of inequality, connecting recommender fairness evaluation to inequality indices in economics and algorithmic fairness.
The framework can rank recommenders differently from accuracy; the toy and real-data results show that more accurate systems can be less fair under a chosen $p_f$.
By inspecting $p$ versus $p_f$ per group, the metric explains which groups are over- or under-represented, supporting transparency without a separate explanation step.
The same divergence can also serve as an objective to train fair recommenders, not only to evaluate them, a direction the paper states as future work.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The metric's fairness verdict is entirely relative to $p_f$; if $p_f$ is chosen to match an algorithm's behavior, the algorithm scores perfectly fair regardless of any external notion of justice, so a principled and pre-specified choice of $p_f$ is a precondition.
Extending $p_f$ to a joint distribution over multiple attributes (for example, gender and subscription type together) is a natural generalization that would preserve the same divergence formulas.
The parameter $\alpha$, studied here only at $\alpha = -1$, controls the divergence's sensitivity to different parts of the distribution; tuning $\alpha$ by stakeholder preferences or robust-statistics considerations is a testable extension the paper leaves open.
Because any system is 'fair' under the self-referential choice $p_f = p$, the framework is best used comparatively (fixed $p_f$, fixed gain function, multiple recommenders) rather than as an absolute fairness score.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Here is my take on 1908.06708. The paper proposes using generalized cross entropy (GCE) as a fairness metric for recommender systems, where the reference distribution pf encodes the system designer's notion of fair utility allocation. That is genuinely new in the RS fairness literature: most prior work treats fairness as equality, while this allows non-uniform targets based on merit or need. The framework is simple and internally consistent, and when pf is uniform it reduces to the generalized entropy index of Speicher et al., which is a nice link. The authors also show how to compute p for both item and user attributes with different gain functions (count, relevance, DCG). That part is solid and well explained.

Where it gets shaky is the empirical validation and the status of pf. The toy example is explicitly constructed so that choosing pf2 = [1/3, 2/3] makes Rec0's observed p equal to pf, so GCE=0 by construction. That is fine as an illustration, but it demonstrates the core ambiguity: the metric measures conformance to a designer-specified target, not fairness per se. The authors acknowledge that pf is problem-specific and should be set by the designer, which is honest, but it means the metric is only as good as the external choice of pf. Without an independent way to justify pf, the 'fairness' verdict is basically a measure of how well the system matches the designer's prior.

The experiments are thin. Tables 3 and 4 report single GCE values with no variance, confidence intervals, or significance tests. Table 5 has some strange outcomes: the Random recommender, which has near-zero NDCG for all groups, reports a GCE of 1.5000 for uniform pf, far worse than collaborative filtering methods that have similar NDCG per group. That suggests the metric is numerically fragile when p has zero or near-zero components. The paper does not address this. Also, the abstract claims the metric helps identify over- or under-represented groups, but the scalar GCE itself does not; you need to inspect p(aj) vs pf(aj) separately, and the paper does that only in passing.

On the positive side, the citations are appropriate, including Botev and Kroese for GCE and Speicher et al. for the link to inequality indices. The paper does not overstate novelty; it explicitly builds on these works. The writing is clear.

Bottom line: this is a workshop-length proposal with a plausible and useful idea, but the validation is not yet there. The paper deserves a serious referee – a good reviewer could push the authors to add sensitivity analysis for pf, report variance over recommendation runs, and clarify how the metric should be used to diagnose groups. It is not ready to be treated as a validated evaluation standard, but it is a reasonable starting point. I'd cite it if I were working on fairness metrics, and I'd probably bring it up in a reading group, but I would not treat its experimental results as strong evidence.

Recommendation: send it to peer review (e.g., a workshop or short-paper track), not desk reject, but expect revisions.

Referee Report

3 major / 5 minor

Summary. The paper proposes a probabilistic framework, based on generalized cross entropy (GCE), for evaluating fairness in recommender systems. The unfairness measure I(M,a) in Eq. (2) compares an observed performance distribution p over attribute categories with a designer-specified fair distribution p_f, where p_f can encode equality or non-equality notions of fairness. The authors argue that fairness should be based on merits and needs rather than strict equality, and they demonstrate the framework on a toy example (Section 3) and on two real-world datasets: Xing for item-centered fairness and Amazon for user-centered fairness (Section 4). The reported results compare GCE values across recommenders and against MAD-based baselines, and the paper concludes that the metric is flexible, transparent, and helps explain over- or under-representation of user and item groups.

Significance. If the central claim holds, the paper would provide a single evaluation tool that unifies equality-based and merit/need-based fairness definitions for both users and items, with the flexibility to incorporate domain knowledge through p_f. The mathematical definition is simple and internally consistent, and the toy example usefully illustrates how different choices of p_f change the fairness verdict. However, the significance is currently limited by three issues: the lack of a principled procedure for setting p_f, the fact that the scalar GCE does not by itself deliver the claimed explanatory power about over- or under-representation, and the weak empirical support (point estimates without uncertainty or significance testing). These issues are addressable, so the framework has potential, but the manuscript as written does not fully substantiate the abstract's claims.

major comments (3)

[Section 2.1, Eq. (2); Section 3] The framework's fairness verdict is fully determined by the designer-supplied distribution p_f, but Section 2.1 states only that p_f is 'problem-specific and should be determined based on the problem or target scenario in hand,' without providing any method, constraint, or normative principle for deriving it. The toy example in Section 3 illustrates the resulting circularity: p_f2 = [1/3, 2/3] is chosen so that Rec0's observed distribution p equals p_f, yielding GCE ≈ 0 by construction. Under this logic, any recommender can be certified fair by declaring its observed performance distribution to be the fair distribution. To support the central evaluative claim, the paper must specify how p_f should be determined a priori from merits, needs, or an externally justified fairness principle, and should discuss what empirical content GCE has beyond measuring conformance to that target.
[Abstract and Section 5] The abstract and conclusion claim that the framework is explanatory and 'helps on understanding which item or user aspects the recommendation algorithms are over- or under-representing.' However, the GCE value I(M,a) is a single scalar divergence and does not encode the direction or magnitude of per-category deviations. Determining which groups are over- or under-represented requires inspecting p(a_j) − p_f(a_j) separately for each category, which the proposed metric does not output. The paper should either provide a per-category diagnostic as part of the framework or temper the explanatory claim to reflect that GCE only quantifies overall distance from the target.
[Tables 3, 4, and 5] The experimental results report single GCE values with no variance, confidence intervals, or significance tests, so it is impossible to judge whether observed differences between recommenders or between p_f choices are meaningful. Table 5 additionally contains unexplained anomalies: the Random recommender has NDCG@10 values of 0.0000 for three groups and 0.0005 for the fourth, yet its GCE values are 1.5000 for p_f0 and 4.5000 for p_f1...p_f4; similarly, BPRSlim's MAD-rating value of 9.0009 is an order of magnitude larger than all other MAD-rating values. These entries are not discussed and appear inconsistent with the definitions in Section 2. The authors should clarify the computation of p for the user-fairness experiments and should report variability or at least justify why point estimates suffice.

minor comments (5)

[Section 3, Table 2] The notation for the fair distributions is inconsistent: Table 2 defines p_f0 = [1/2, 1/2], p_f1 = [2/3, 1/3], p_f2 = [1/3, 2/3], but the text in Section 3 refers only to 'p_f = [1/3, 2/3]' when discussing Rec0. Please align the notation and explicitly name p_f2 in the text.
[Eqs. (1) and (2)] Equation (1) is written in continuous integral form while Eq. (2) uses a discrete sum; the paper should clarify the domain of the attribute variable and state that Eq. (2) is the discrete analogue. In addition, the absolute value convention for GCE is introduced only in the table captions, so the sign behavior of Eq. (2) should be explained in the text.
[Section 4.2] The user activity groups (VIA, SIA, SA, VA) are defined by quartiles of nR(u), but the quartile thresholds are not reported. Providing these thresholds would improve reproducibility of the Amazon experiments.
[Table 4] The column headers GCE1 and GCE2 are not defined in the text or caption; the caption should state which p_f and α values each column corresponds to.
[Section 4.3] All experiments use α = −1; the paper does not report any sensitivity analysis with respect to α, even though α is presented as a parameter of the generalized framework. A brief discussion or a supplementary table would support the claim of generality.

Circularity Check

2 steps flagged · score 4.0 of 10

The fairness verdict is a divergence from a freely chosen p_f; the toy example and 'best-case' experimental readings are tautological consequences of Eq. (2), though the metric itself is internally consistent and self-contained.

self definitional [Section 2, Eq. (2); Section 2.1; Section 3, Table 2]
"The unfairness measure I is minimized with respect to attribute x = a when p = pf, meaning that the performance of the system is equal to the performance of a fair system. ... The definition of a fair distribution pf is problem-specific and should be determined based on the problem or target scenario in hand. ... if fairness is defined as providing recommendation of higher utility (usefulness) to green users who are users with paid premium membership type, (e.g., by setting pf = [1/3, 2/3]) then since GCE ≈ 0, we can say that recommendations produced by Rec0 are fair."

Eq. (2) defines GCE as a divergence between the observed performance distribution p and the designer-supplied fair distribution p_f, and the paper states it is minimized exactly when p = p_f. Section 2.1 gives no procedure for deriving p_f from merits or needs, leaving it entirely to the designer. In the toy example, Rec0's observed relevant-item distribution is p = [1/3, 2/3] (3 free vs 6 premium relevant hits); the paper chooses p_f2 = [1/3, 2/3] and then reports GCE ≈ 0 and declares Rec0 fair. Thus the fairness verdict is set by the input p_f by construction; any recommender can be made fair by choosing p_f equal to its observed p.
other [Section 4.3, Table 4 and following paragraph]
"After applying the GCE on the winner submission, we observe that with respect to the Country attribute, the lowest value of GCE (best case) is produced for the German companies (GCE = 0.061) while for the Education attribute the category Unknown (GCE = 0.97) produces the best outcome, in both cases, these categories are the most frequently recommended by the analyzed submission."

The p_f variants in Table 4 are hand-specified; in particular pf1 = [0.7, 0.1, 0.1, 0.1] concentrates 70% mass on German. Since Eq. (2) is minimized when p = p_f, the minimum over the five p_f vectors is always attained by the p_f closest to the observed p. The observed recommendation counts are dominated by German (3.9M out of 4.57M) and by the Unknown education category, precisely the 'best case' categories reported. The finding that the winner is 'closer to a fair system' under those p_f is therefore a direct consequence of the chosen p_f set, not independent evidence of fairness.

full rationale

The proposed GCE metric is internally well-defined and the experiments are honest in the sense that p_f variants are stated explicitly; the paper does not hide the dependence on p_f. However, the central 'fairness evaluation' is definitionally a divergence from p_f: Eq. (2) is minimized exactly when p = p_f, and Section 2.1 leaves p_f entirely to the designer. The toy example makes the reduction explicit by choosing p_f2 = [1/3, 2/3] equal to Rec0's observed distribution and then declaring Rec0 fair. The real-data reading is similar: among the hand-picked p_f variants, the lowest GCE is attained by the variant whose mass is concentrated on the dominant observed category, which is exactly what the divergence formula requires. These are not hidden fitted parameters renamed as predictions, but they do mean that the fairness label carries no content beyond conformance to the chosen p_f. The framework is still useful as a configurable conformance measure, and there is no load-bearing self-citation chain or imported uniqueness theorem. Score 4 reflects partial circularity in the evaluative claim while acknowledging the definitional transparency and the utility of the metric when p_f is fixed a priori.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The central metric depends on two probability distributions: p estimated from recommendation gains, and pf chosen by the evaluator. The hand-set numeric inputs are α=-1 and several pf vectors. No parameters are fitted to optimize a prediction, so the framework is an evaluation tool rather than a predictive model. The free-parameter burden is modest but real: the fairness verdict is a function of the evaluator's pf, and the gain-function choice is a domain assumption.

free parameters (5)

α (GCE parameter) = -1
Set to -1 for all reported GCE values. No sensitivity analysis or justification is given for this choice, though α changes the weighting of distribution tails.
fair distribution pf for membership-type item fairness = pf0=[1/2,1/2], pf2=[1/3,2/3]
Chosen by hand to encode equality versus favoring premium users. The metric's output depends directly on this choice.
fair distribution pf for country and education attributes = pf0=[0.25,0.25,0.25,0.25]; pf1-pf4 each put 0.7 on one category and 0.1 on the others
Hand-specified variants used to illustrate how fairness verdicts change when one group is favored. These are inputs to the metric, not learned from data.
fair distribution pf for user activity groups = pf0 uniform; pf1-pf4 put 0.7 on one activity group and 0.1 on the others
Same hand-picked structure as the country and education experiments. The resulting GCE ordering of recommenders is a direct consequence of these pf choices.
user activity group quartile thresholds = quartiles of nR(u) on the Amazon subset
Users are split into very inactive, slightly inactive, slightly active, and very active groups by quartiles of rating count. The thresholds are data-dependent but not reported, which affects comparability.

assumptions (4)

standard math The generalized cross entropy formula in Eqs. (1) and (2) is a valid measure of divergence between two probability distributions.
Taken from Botev and Kroese [6]. The paper does not derive or justify the metric's properties beyond the citation.
domain assumption The performance distribution p, estimated by aggregating recommendation gains over a test set, faithfully represents the utility each attribute group receives.
Introduced in Section 2.2 via recommendation gain equations (3) and (4). The choice of gain function, count versus relevance versus DCG/NDCG, changes p and hence all GCE values.
domain assumption The designer-supplied fair distribution pf is the correct normative fairness target for the scenario.
Section 2.1 states pf is problem-specific. The framework cannot validate pf; any fairness conclusion is relative to this external input. This is the load-bearing assumption.
domain assumption For user fairness, normalized DCG (NDCG) is an appropriate gain function to represent user utility from recommendations.
Section 2.2 suggests DCG/NDCG, and the Amazon experiments report NDCG@10 and GCE values, but the exact mapping from user gains to p(aj) is not written out.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Recommender Systems Fairness Evaluation via Generalized Cross Entropy." pith.science (2026). https://pith.science/paper/56OMW2O2

@misc{pith2026190806708,
  author       = {Pith},
  title        = {Pith review of: Recommender Systems Fairness Evaluation via Generalized Cross Entropy},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/56OMW2O2}},
  note         = {Machine review of arXiv:1908.06708}
}

read the original abstract

Fairness in recommender systems has been considered with respect to sensitive attributes of users (e.g., gender, race) or items (e.g., revenue in a multistakeholder setting). Regardless, the concept has been commonly interpreted as some form of equality -- i.e., the degree to which the system is meeting the information needs of all its users in an equal sense. In this paper, we argue that fairness in recommender systems does not necessarily imply equality, but instead it should consider a distribution of resources based on merits and needs. We present a probabilistic framework based on generalized cross entropy to evaluate fairness of recommender systems under this perspective, where we show that the proposed framework is flexible and explanatory by allowing to incorporate domain knowledge (through an ideal fair distribution) that can help to understand which item or user aspects a recommendation algorithm is over- or under-representing. Results on two real-world datasets show the merits of the proposed evaluation framework both in terms of user and item fairness.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

32 extracted references · 30 canonical work pages

[1]

Amazon product data

2014 (accessed June 5, 2019). Amazon product data . h/t_tp://jmcauley.ucsd.edu/data/amazon/

work page 2014
[2]

Fabian Abel, Yashar Deldjoo, Mehdi Elahi, and Daniel Koh lsdorf. 2017. RecSys Challenge 2017: Oﬄine and Online Evaluation. InProc. RecSys (RecSys ’17). ACM, New York, NY, USA, 372–373

work page 2017
[3]

Jens Adamczak, Gerard-Paul Leyson, Peter Knees, Yashar Deldjoo, Far- shad Bakhshandegan Moghaddam, Julia Neidhardt, Wolfgang W ¨ orndl, and Philipp Monreal. 2019. Session-Based Hotel Recommendatio ns: Challenges and Future Directions. arXiv preprint arXiv:1908.00071 (2019)

arXiv 2019
[4]

Vito Walter Anelli, Tommaso Di Noia, Eugenio Di Sciascio , Azzurra Ragone, and Joseph Tro/t_ta. 2019. Local Popularity and Time in top-N Re commendation. In Advances in Information Retrieval - 41st European Conferen ce on IR Research, ECIR 2019, Cologne, Germany, April 14-18, 2019, Proceedings, Part I (Lecture Notes in Computer Science) , Leif Azzopardi, ...

work page 2019
[5]

Biega, Krishna P

Asia J. Biega, Krishna P. Gummadi, and Gerhard Weikum. 20 18. Equity of A/t_tention: Amortizing Individual Fairness in Rankings. In/T_he 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08-12, 2018 , Kevyn Collins-/T_hompson, Qiaozhu Mei, Brian D. Davison, Yiqun Liu, and Emine...

work page 2018
[6]

Zdravko I Botev and Dirk P Kroese. 2011. /T_he generalized cross entropy method, with applications to probability density estimation. Methodology and Computing in Applied Probability 13, 1 (2011), 1–27

work page 2011
[7]

Breese, David Heckerman, and Carl Myers Kadie

John S. Breese, David Heckerman, and Carl Myers Kadie. 19 98. Empirical Anal- ysis of Predictive Algorithms for Collaborative Filtering . In UAI ’98: Proceedings of the Fourteenth Conference on Uncertainty in Artiﬁcial Intelligence, University of Wisconsin Business School, Madison, Wisconsin, USA, July 2 4-26, 1998, Gregory F. Cooper and Seraf´ ın Moral (...

work page 1998
[8]

Robin Burke. 2017. Multisided fairness for recommendat ion. arXiv preprint arXiv:1707.00093 (2017)

arXiv 2017

Show all 32 references

[9]

Robin Burke, Nasim Sonboli, and Aldo Ordonez-Gauger. 20 18. Balanced Neighborhoods for Multi-sided Fairness in Recommendation . In Conference on Fairness, Accountability and Transparency, FAT 2018, 23-2 4 February 2018, New York, NY, USA (Proceedings of Machine Learning Resear...

2018
[10]

Campos, Fernando D´ ıez, and Iv´ an Cantador

Pedro G. Campos, Fernando D´ ıez, and Iv´ an Cantador. 2014. Time-aware recom- mender systems: a comprehensive survey and analysis of exis ting evaluation protocols. User Model. User-Adapt. Interact. 24, 1-2 (2014), 67–119

2014
[11]

Yashar Deldjoo, Mihai Gabriel Constantin, Hamid Eghba l-Zadeh, Bogdan Ionescu, Markus Schedl, and Paolo Cremonesi. 2018. Audio-v isual encoding of multimedia content for enhancing movie recommendations. I n Proceedings of the 12th ACM Conference on Recommender Systems, RecSys ...

2018
[12]

Yashar Deldjoo, Maurizio Ferrari Dacrema, Mihai Gabri el Constantin, Hamid Eghbal-zadeh, Stefano Cereda, Markus Schedl, Bogdan Iones cu, and Paolo Cremonesi. 2019. Movie genome: alleviating new item cold st art in movie recommendation. User Model. User-Adapt. Interact. 29, 2 (...

2019
[13]

Yashar Deldjoo, Tommaso Di Noia, and Felice Antonio Mer ra. 2019. Assessing the Impact of a User-Item Collaborative A/t_tack on Class of Use rs. In Workshop on the Impact of Recommender Systems (ImpactRecSys’19) - 13th ACM Conference of Recommender Systems

2019
[14]

Yashar Deldjoo, Markus Schedl, Paolo Cremonesi, and Ga briella Pasi. 2018. Content-Based Multimedia Recommendation Systems: Deﬁnit ion and Applica- tion Domains. In Proceedings of the 9th Italian Information Retrieval Worksh op, Rome, Italy, May, 28-30, 2018. (CEUR Workshop Pr...

2018
[15]

Wei Dong, Charikar Moses, and Kai Li. 2011. Eﬃcient k-ne arest neighbor graph construction for generic similarity measures. In Proceedings of the 20th international conference on World wide web. ACM, 577–586

2011
[16]

Michael D Ekstrand, Mucun Tian, Ion Madrazo Azpiazu, Je nnifer D Ekstrand, Oghenemaro Anuyah, David McNeill, and Maria Soledad Pera. 2 018. All /T_he Cool Kids, How Do /T_hey Fit In?: Popularity and Demographic Biases in Recom- mender Evaluation and Eﬀectiveness. In Conference...
[17]

/T_hompson, Srir am Srinivasan, and Lise Getoor

Golnoosh Farnadi, Pigi Kouki, Spencer K. /T_hompson, Srir am Srinivasan, and Lise Getoor. 2018. A Fairness-aware Hybrid Recommender Sys tem. CoRR abs/1809.09030 (2018). arXiv:1809.09030

2018 arXiv
[18]

Asela Gunawardana and Guy Shani. 2015. Evaluating Reco mmender Systems. In Recommender Systems Handbook , Francesco Ricci, Lior Rokach, and Bracha Shapira (Eds.). Springer, 265–308

2015
[19]

Herlocker, Joseph A

Jonathan L. Herlocker, Joseph A. Konstan, and John Ried l. 2002. An Empirical Analysis of Design Choices in Neighborhood-Based Collabor ative Filtering Algorithms. Inf. Retr. 5, 4 (2002), 287–310

2002
[20]

Kalervo J¨ arvelin and Jaana Kek¨ al¨ ainen. 2002. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems (TOIS) 20, 4 (2002), 422–446

2002
[21]

Yehuda Koren. 2008. Factorization meets the neighborh ood: a multifaceted col- laborative ﬁltering model. In Proceedings of the 14th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Las Vega s, Nevada, USA, August 24-27, 2008, Ying Li, Bing Liu, and ...

2008
[22]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Ma trix factorization techniques for recommender systems. Computer 8 (2009), 30–37

2009
[23]

Yehuda Koren, Robert Bell, and Chris Volinsky. 2009. Ma trix Factorization Techniques for Recommender Systems. Computer 42, 8 (2009), 30–37

2009
[24]

Xia Ning and George Karypis. 2011. SLIM: Sparse Linear M ethods for Top-N Recommender Systems. In 11th IEEE International Conference on Data Mining, ICDM 2011, Vancouver, BC, Canada, December 11-14, 2011, Diane J. Cook, Jian Pei, Wei Wang, Osmar R. Za¨ ıane, and Xindong Wu (Ed...

2011
[25]

Steﬀen Rendle, Christoph Freudenthaler, Zeno Gantner , and Lars Schmidt- /T_hieme. 2009. BPR: Bayesian Personalized Ranking from Impli cit Feedback. In UAI 2009, Proceedings of the Twenty-Fi/f_th Conference on Uncer tainty in Artiﬁcial Intelligence, Montreal, QC, Canada, June ...

2009
[26]

Badrul Sarwar, George Karypis, Joseph Konstan, and Joh n Riedl. 2000. Analysis of recommendation algorithms for e-commerce. In Proceedings of the 2nd ACM conference on Electronic commerce. ACM, 158–167

2000
[27]

Ashudeep Singh and /T_horsten Joachims. 2018. Fairness of Exposure in Rankings. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, KDD 2018, London, UK, August 19-23, 2018, Yike Guo and Faisal Farooq (Eds.). ACM, 2219–2228

2018
[28]

Till Speicher, Hoda Heidari, Nina Grgic-Hlaca, Krishn a P Gummadi, Adish Singla, Adrian Weller, and Muhammad Bilal Zafar. 2018. A Uniﬁed Approach to /Q_uantifying Algorithmic Unfairness: Measuring Individual&Group Unfairness via Inequality Indices. In Proceedings of the 24th A...

2018
[29]

Sirui Yao and Bert Huang. 2017. Beyond parity: Fairness objectives for collabora- tive ﬁltering. In Advances in Neural Information Processing Systems. 2921–2930

2017
[30]

Baeza-Yates

Meike Zehlike, Francesco Bonchi, Carlos Castillo, Sar a Hajian, Mohamed Mega- hed, and Ricardo A. Baeza-Yates. 2017. FA *IR: A Fair Top-k Ranking Algorithm. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Man- agement, CIKM 2017, Singapore, November 0...

2017
[31]

Yong Zheng, Tanaya Dave, Neha Mishra, and Harshit Kumar . 2018. Fairness In Reciprocal Recommendations: A Speed-Dating Study. In Adjunct Publication of the 26th Conference on User Modeling, Adaptation and Perso nalization, UMAP 2018, Singapore, July 08-11, 2018 , Tanja Mitrovi...

2018
[32]

Ziwei Zhu, Xia Hu, and James Caverlee. 2018. Fairness-A ware Tensor-Based Recommendation. In Proceedings of the 27th ACM International Conference on Information and Knowledge Management, CIKM 2018, Torino, Italy, October 22-26, 2018, Alfredo Cuzzocrea, James Allan, Norman W. P...

2018

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.

[1] [1]

Amazon product data

2014 (accessed June 5, 2019). Amazon product data . h/t_tp://jmcauley.ucsd.edu/data/amazon/

work page 2014

[2] [2]

Fabian Abel, Yashar Deldjoo, Mehdi Elahi, and Daniel Koh lsdorf. 2017. RecSys Challenge 2017: Oﬄine and Online Evaluation. InProc. RecSys (RecSys ’17). ACM, New York, NY, USA, 372–373

work page 2017

[3] [3]

Jens Adamczak, Gerard-Paul Leyson, Peter Knees, Yashar Deldjoo, Far- shad Bakhshandegan Moghaddam, Julia Neidhardt, Wolfgang W ¨ orndl, and Philipp Monreal. 2019. Session-Based Hotel Recommendatio ns: Challenges and Future Directions. arXiv preprint arXiv:1908.00071 (2019)

arXiv 2019

[4] [4]

Vito Walter Anelli, Tommaso Di Noia, Eugenio Di Sciascio , Azzurra Ragone, and Joseph Tro/t_ta. 2019. Local Popularity and Time in top-N Re commendation. In Advances in Information Retrieval - 41st European Conferen ce on IR Research, ECIR 2019, Cologne, Germany, April 14-18, 2019, Proceedings, Part I (Lecture Notes in Computer Science) , Leif Azzopardi, ...

work page 2019

[5] [5]

Biega, Krishna P

Asia J. Biega, Krishna P. Gummadi, and Gerhard Weikum. 20 18. Equity of A/t_tention: Amortizing Individual Fairness in Rankings. In/T_he 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08-12, 2018 , Kevyn Collins-/T_hompson, Qiaozhu Mei, Brian D. Davison, Yiqun Liu, and Emine...

work page 2018

[6] [6]

Zdravko I Botev and Dirk P Kroese. 2011. /T_he generalized cross entropy method, with applications to probability density estimation. Methodology and Computing in Applied Probability 13, 1 (2011), 1–27

work page 2011

[7] [7]

Breese, David Heckerman, and Carl Myers Kadie

John S. Breese, David Heckerman, and Carl Myers Kadie. 19 98. Empirical Anal- ysis of Predictive Algorithms for Collaborative Filtering . In UAI ’98: Proceedings of the Fourteenth Conference on Uncertainty in Artiﬁcial Intelligence, University of Wisconsin Business School, Madison, Wisconsin, USA, July 2 4-26, 1998, Gregory F. Cooper and Seraf´ ın Moral (...

work page 1998

[8] [8]

Robin Burke. 2017. Multisided fairness for recommendat ion. arXiv preprint arXiv:1707.00093 (2017)

arXiv 2017