Personalized Recommendations via Active Utility-based Pairwise Sampling

Bahar Boroomand; James R. Wright

REVIEW 4 major objections 5 minor 12 references

Personalized Recommendations via Active Utility-based Pairwise Sampling

T0 review · 4 major / 5 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Recommenders learn faster when they ask users the pairs whose answers most improve the final ranking.

desk verdict Utility-driven active sampling for pairwise recommenders is a sensible and well-framed idea, but the paper never specifies how Eq. (9)'s hypothetical retraining is computed, so the experimental gains rest on an unverifiable implementation and deserve a referee's scrutiny, not a desk rejection. read the letter →

arxiv 2508.14911 v1 pith:VQ3XXBZG submitted 2025-08-12 cs.IR cs.LG

Bahar Boroomand , James R. Wright This is my paper

classification cs.IRcs.LG

keywords preferenceelicitationpairwisecomparisonsactivelearningrecommendersystemsPlackett-Lucemodelutility-basedsamplingcoldstartranking

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper claims that recommender systems learn better from a few deliberately chosen pairwise comparisons than from ratings or random questions. The authors model each user's ranking distribution with the Plackett-Luce choice model, define what makes a recommendation good through a task-specific utility function, and select the next pair to ask by estimating which answer would most improve the expected utility of the final menu. In experiments, the active rule outperforms random, entropy-based, and clustering-based querying in both a movie recommendation task and an admissions task. If the claim holds, preference elicitation becomes substantially more data-efficient, which matters when each user answer is costly.

What carries the argument

The load-bearing object is the utility-gain query score $g(\theta)=\arg\max_{(i,j)} E_{\theta}[U(f(\theta|q),\theta|q)-U(f(\theta),\theta|q)]$; it converts 'informative question' into 'question that moves the final recommendation metric.' It sits on top of the Plackett-Luce model, in which each item's probability of being chosen next is proportional to its latent score, allowing Monte Carlo sampling of rankings instead of enumerating $n!$ orders. Two utility functions instantiate the objective: max rank over the menu for media, and fraction of the top-$k$ ranking included for admissions. The critical connective step is $\theta|q$, the hypothetical retrained parameters after a possible answer

What would settle it

Inspect the implementation of Eq. (9): if $\theta|q$ was produced by full retraining for every candidate pair and both outcomes, a single query in the 400-applicant experiment would require roughly $2\times 79{,}800$ model fits; if it was produced by one gradient step or no update, the implemented query rule is not Eq. (9). Re-running the admissions experiment with a one-gradient-step replacement for $\theta|q$ and re-measuring the active-versus-random gap would settle whether the reported gains survive the approximation.

Watch

Extended reading notes

Core claim

The central claim is that query selection should be driven by expected improvement in the utility of the final recommendation rather than by generic uncertainty. For a candidate pair $(i,j)$, the method scores the query by $E_{\theta}[U(f(\theta|q),\theta|q)-U(f(\theta),\theta|q)]$, where $\theta$ is the current model, $q$ ranges over the two possible answers, $\theta|q$ is the model after retraining on that answer, and $f(\theta)$ is the recommendation menu that maximizes expected utility; the pair with the highest score is asked next. With Plackett-Luce as the ranking distribution, the expectation is approximated by Monte Carlo sampling of the top-$k$ positions. The paper reports higher ma

Load-bearing premise

The active-learning formula presumes the system can predict its own model parameters after each possible answer to each candidate question, and the paper does not specify how those imagined retrained parameters are obtained; the reported gains depend on that unspecified step.

Editorial extensions

If this is right

Starting from five random comparisons per user, a handful of deliberately chosen additional pairs can improve the max-rank utility of the final menu, so pairwise elicitation can work in cold-start conditions.
Actively optimizing final utility outperforms entropy-based uncertainty sampling, so resolving uncertainty that does not change the recommended menu is worth less than resolving task-relevant uncertainty.
Because the query rule is written against an arbitrary scoring function $f(u,i;\Theta)$, the same active sampler transfers from matrix factorization to neural-network models.
Swapping the utility function changes what the system optimizes, so the framework can be pointed at different notions of a good outcome, such as maximizing the top item or covering as many top-$k$ candidates as possible.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The reported experiments leave open how $\theta|q$ is obtained; if exact retraining is required, evaluating one query on the 400-applicant dataset would cost about $2\times 79{,}800$ model fits, making the method impractical at scale without an approximation.
If the implementation instead uses one gradient step or a similar shortcut, then Eq. (9) is not being evaluated as written, and the active-versus-random gap may be an artifact of that shortcut rather than of the utility criterion.
A natural extension is to restrict the candidate-pair pool to a shortlist tied to the current recommendation menu or to high-uncertainty items; if the utility-gain ordering is preserved, the same method could scale to large catalogs.
The same active query rule could be applied to diversity-, coverage-, or fairness-based utility functions, since the mechanics only require an expected-utility score for the menu.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

You should know this paper proposes something genuinely useful on paper: a utility-driven active query selection for pairwise recommendations, where the query is chosen to maximize the expected gain in a task-specific utility (top-item rank, admissions coverage) under a Plackett-Luce model. That goes beyond the usual entropy or uncertainty sampling, and the two experimental setups show what the authors are aiming at. The writing is clear, and the related work is honestly positioned; they correctly credit Chajewska et al. and Boutilier for value-of-information.

The main problem is that the engine of the method, Eq. (9), selects the query by computing θ|q, the parameters after hypothetical retraining on each possible outcome, for every candidate pair. The paper never says how θ|q is obtained — full retraining, a single gradient step, an approximation, or nothing. With 400 items you have ~80k pairs, so exact retraining is infeasible. Without an algorithm, the reported gains over random and entropy sampling cannot be attributed to the proposed utility-gain criterion; they may come from whatever shortcut was actually taken. This is not a minor omission; it is the load-bearing step.

There is also an internal mismatch: Algorithm 1 is written for the Admissions utility (Eq. 8), but Experiment 1 claims to use the max-rank utility (Eq. 7). So the provided Monte Carlo approximation does not even match the first experiment's utility. And the appendix's semi-synthetic comparison, which generates ground truth by sampling from a Plackett-Luce model, is stacked in favor of the comparison-based model; the main real-data experiments are not circular, but that appendix does not support the fundamental-advantage claim.

On the plus side, the basic pairwise loss and the Plackett-Luce formulas are correct, and the idea of optimizing an explicit utility rather than a rating objective is the right way to frame the problem. The experiments, if the implementation were specified, would be a useful data point. As is, the paper is an interesting research proposal, not a validated method.

Who should read it: researchers working on preference elicitation and active learning for recommender systems. It deserves a serious referee, but the referee will need to demand a precise description of the active sampling implementation, error bars, and code. I would not cite it until those are provided.

Referee Report

4 major / 5 minor

Summary. The paper proposes a utility-based framework for learning personalized recommendations from pairwise comparisons. A Plackett-Luce model is used to represent distributions over rankings, and recommendation menus are selected by maximizing an arbitrary expected utility (e.g., max-rank for media, top-k coverage for admissions). The central methodological contribution is an active sampling strategy, Eq. (9), which chooses the pairwise query expected to yield the greatest utility gain, approximated by Monte Carlo sampling. Two experiments are reported: a MovieLens matrix-factorization task comparing active, random, and no-sampling regimes, and a graduate-admissions neural-network task comparing active sampling with entropy, random, and cluster-based baselines. The paper claims that the proposed active sampling substantially improves data efficiency and recommendation quality.

Significance. The framework is appealing in principle: it cleanly separates the recommendation objective from the preference model and attempts to align active query selection with the final utility rather than generic uncertainty. The use of Plackett-Luce for sampling ranking distributions is standard and the Monte Carlo utility approximation in Algorithm 1 is sensible for a fixed parameter vector. If the active sampling step were fully specified and the experiments validated, the paper would make a useful contribution to preference elicitation for recommender systems. However, the central active-sampling computation is not actually specified: Eq. (9) requires hypothetical retrained parameters θ|q for every candidate pair and outcome, and no implementation or approximation is given. The reported experimental results therefore cannot be reproduced or even exactly interpreted. In addition, Algorithm 1, as written, computes the top-k coverage utility of Eq. (8) rather than the max-rank utility of Eq. (7) that Experiment 1 claims to use. These issues are load-bearing, so the significance of the empirical claims is not established in the current version.

major comments (4)

[Utility-Based Active Sampling, Eq. (9)] The selection rule g(θ) requires θ|q, the model parameters after retraining on each hypothetical query outcome. The manuscript never states how θ|q is obtained: full retraining, one gradient step, a low-rank update, or an approximation using the current θ. This is not a minor implementational detail: with n=400 candidates there are roughly 79,800 pairs, and if exact retraining is required, one query would require on the order of 160,000 model fits; for MovieLens (n≈1,700) it is worse. The experimental protocols only say that after a real query the model is retrained; they do not indicate how the utility gain in Eq. (9) is evaluated for candidate pairs. The Monte Carlo approximation in Algorithm 1 addresses the expectation over rankings, not the hypothetical-parameter problem. Without this specification, the reported superiority over random and entropy sampling could be an artifact of wha
[Algorithm 1 and Experiment 1] Algorithm 1 computes the utility by counting how many of the sampled top-k items fall in the menu m (lines 13–17), which is exactly the Admissions utility of Eq. (8). Experiment 1 explicitly states that the utility-based active sampling uses the Goods and Media utility function in Eq. (7), where utility is the highest rank (best position) achieved by any recommended item. These are different objectives, and the truncation to the first k sampled items is only without loss of generality for Eq. (8), not for Eq. (7), because the max rank can lie beyond k. The implementation therefore does not match the stated active-sampling criterion in Experiment 1, and the results in Figure 1 may not reflect the claimed max-rank utility.
[Appendix: Semi-synthetic experiment] The appendix's ground-truth preference orderings are generated by sampling from a Plackett-Luce model whose parameters are the shadow ratings u_u^T v_i. The comparison-based model being tested is also a Plackett-Luce model over pairwise comparisons. This means the ground truth is drawn from the same model family as the proposed method, which structurally favors the comparison-based approach over the rating-based MF, whose objective is only indirectly related to the PL generative process. The claim that the comparison-based model fundamentally outperforms rating-based prediction is therefore not supported by this experiment. Additionally, Figure 4 compares training loss of the comparison-based model (a log-likelihood) with MSE of the rating-based model on different scales; the two curves are not directly comparable as evidence about overfitting or objective alignment.
[Experiments: Protocol and reproducibility] Several protocol details needed to reproduce the central results are missing. In Experiment 1, it is unclear how the active pairs are selected given Eq. (9) (see first comment), what Monte Carlo sample size R is used, how many retraining epochs are run after each query, and what 'starting from 10 users per epoch' means for the x-axis 'Number of Queries'. The statement that the no-sampling baseline has different performance for different query counts because the model is re-sampled and re-trained is confusing; if no queries are added, performance should be constant up to randomness. In Experiment 2, the latent random vector z is introduced in the model description but its role in active sampling is not explained. These details matter because the reported differences are large and the central comparison depends on the exact active-selection procedure.

minor comments (5)

[Throughout] Typos and formatting issues: 'pertaining' should be 'pretraining'; 'V olinsky' in the references; 'F ourth' in the RecSys reference; 'Architecture' in the NeuMF description; 'Morgen & Claypool' should be 'Morgan & Claypool'.
[Eq. (1)-(2)] The relationship between θ_ui and s_ui is stated as s_ui = ln θ_ui, but Eq. (2) uses exp(s_ui). This is consistent, but the notation is easy to misread; a short clarification would help.
[Appendix Eq. (10)] The Laplace smoothing formula uses θ_i without defining it in the appendix (presumably the latent score). Since the appendix is self-contained in part, please define θ_i and α.
[Figure 2] The figure caption says 'average over 100 runs' but the main text does not state the number of random seeds or confidence intervals for Experiment 2. Please include this information or state that no error bars are shown.
[Related Work] The discussion of Mallows-based elicitation is useful, but the claim that Plackett-Luce is 'well-suited for the efficient, iterative updates needed to evaluate potential queries' is not substantiated and, given the missing θ|q implementation, appears premature.

Circularity Check

1 steps flagged · score 6.0 of 10

Main active-sampling experiments are not circular, but the appendix's foundational pairwise-vs-rating claim reduces to a Plackett-Luce model evaluating Plackett-Luce-generated ground truth.

self definitional [Appendix, 'Datasets' and 'Experimental Protocol'; Eq. (10) vs Eq. (2)]
"The shadow ratings are then used as the parameters of a Plackett-Luce model, from which each user’s ground truth preference ordering is sampled. ... P ′(i) = θi + αP j∈S(θj + α) (10) ... Pairwise Comparison Model (Preference-Based): Our proposed method optimizes item rankings by learning from pairwise outcomes. For each preference i ≻ j, we use the log-likelihood loss defined in equation (2)."

Eq. (10) defines the semi-synthetic ground-truth preference orderings by sampling from a Plackett-Luce model, and Eq. (2) is the Plackett-Luce pairwise log-likelihood used to train the 'preference-based' model. The data-generating distribution and the compared model are therefore the same family: the comparison-based model is the generative model for the 'ground truth.' The rating-based baseline is trained on the same shadow scores with MSE, which is misspecified for PL-generated rankings. The appendix's conclusion that the comparison-based model 'consistently achieves much higher Kendall-τ' is thus a self-consistency check of the PL assumption, not independent evidence that pairwise preference data is fundamentally better than rating data. The claimed 'foundational advantage' is built int

full rationale

The paper's main contribution—utility-based active sampling (Eq. 9) and its MovieLens/admissions evaluations—is not circular. The query selector optimizes an expected utility gain under a learned model, and the evaluation measures realized utility against external ground-truth rankings (MovieLens preferences and admission scores). That the selection objective matches the evaluation utility is the method's design, not a by-construction guarantee of the reported gains; baselines such as random, entropy, and clustering do not optimize that utility, and the model's expected gain could be miscalibrated. The one circular element is in the appendix's 'foundational advantage' demonstration. There, ground-truth preference orderings are sampled from a Plackett-Luce model (Eq. 10) using the same PL family as the pairwise log-likelihood loss (Eq. 2) of the proposed comparison-based model, while the rating-based baseline is trained on the same shadow scores with MSE—a misspecified objective for PL-generated rankings. The appendix's Kendall-τ superiority is therefore a self-consistency check, not independent evidence that pairwise preference learning outperforms rating-based learning. Because the introduction explicitly relies on this appendix to show that the pairwise approach 'fundamentally outperforms' rating prediction, this is a load-bearing circular step. We also note, as a non-circular weakness, that Eq. (9)'s θ|q is never specified (full retraining vs. one gradient step vs. approximation); this is an omitted implementation detail that affects reproducibility but does not by itself make the derivation circular. No self-citations or imported uniqueness theorems are load-bearing.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central claim rests on the Plackett-Luce modeling assumption and on the availability of ground-truth preferences derived from ratings, despite the paper's critique of ratings. The most consequential unstated premise is that theta|q can be computed for every candidate query; this is both a free design choice and a potential computational barrier.

free parameters (3)

R (Monte Carlo sample count) = not reported
The number of sampled rankings used to approximate expected utility in Algorithm 1 is never given; it controls the variance of the utility estimate and therefore which pairs are selected.
Pretraining comparisons per user (Exp 1) = 5
Models are pretrained with 5 randomly chosen comparisons per user; the choice affects cold-start difficulty and the measured improvements.
Latent dimension K (MF) = 100
Set to 100 without sensitivity analysis; affects capacity of the pairwise model in Exp 1.

assumptions (4)

domain assumption User preferences are generated by a Plackett-Luce model with a fixed score per item.
Used in Eq. (1) to define ranking probabilities; the entire active sampling and utility computation rests on this model.
ad hoc to paper Ground-truth preferences for simulation can be derived from numerical ratings (MovieLens) or a continuous admission score.
The paper argues ratings are unreliable, yet Exp 1 and the Appendix construct ground-truth rankings from ratings or shadow ratings.
ad hoc to paper Truncating sampled rankings to the top k items is a valid approximation for the utility used in Exp 1.
Algorithm 1 only samples the top k items, which is exact for Eq. (8) but not for the max-rank utility (Eq. 7) stated in Exp 1. The paper does not justify the approximation for Eq. (7).
domain assumption The oracle responds consistently with the ground truth and query outcomes are independent of the model's uncertainty.
Simulated user answers are deterministic functions of ground-truth rankings, ignoring the stochasticity and noise the paper motivates.

invented entities (1)

Latent random vector z in the preference embedding
purpose: Provides model uncertainty for active learning, inspired by epistemic neural networks (Dwaracherla et al. 2024).
The paper introduces z into the NeuMF model without specifying its dimension or distribution beyond the standard normal. It does not produce a falsifiable prediction outside the paper.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Personalized Recommendations via Active Utility-based Pairwise Sampling." pith.science (2026). https://pith.science/paper/VQ3XXBZG

@misc{pith2026250814911,
  author       = {Pith},
  title        = {Pith review of: Personalized Recommendations via Active Utility-based Pairwise Sampling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VQ3XXBZG}},
  note         = {Machine review of arXiv:2508.14911}
}

read the original abstract

Recommender systems play a critical role in enhancing user experience by providing personalized suggestions based on user preferences. Traditional approaches often rely on explicit numerical ratings or assume access to fully ranked lists of items. However, ratings frequently fail to capture true preferences due to users' behavioral biases and subjective interpretations of rating scales, while eliciting full rankings is demanding and impractical. To overcome these limitations, we propose a generalized utility-based framework that learns preferences from simple and intuitive pairwise comparisons. Our approach is model-agnostic and designed to optimize for arbitrary, task-specific utility functions, allowing the system's objective to be explicitly aligned with the definition of a high-quality outcome in any given application. A central contribution of our work is a novel utility-based active sampling strategy for preference elicitation. This method selects queries that are expected to provide the greatest improvement to the utility of the final recommended outcome. We ground our preference model in the probabilistic Plackett-Luce framework for pairwise data. To demonstrate the versatility of our approach, we present two distinct experiments: first, an implementation using matrix factorization for a classic movie recommendation task, and second, an implementation using a neural network for a complex candidate selection scenario in university admissions. Experimental results demonstrate that our framework provides a more accurate, data-efficient, and user-centric paradigm for personalized ranking.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

12 extracted references · 10 canonical work pages

[2]

Accessed: 2025-05-08

Kaggle Dataset, https://www.kaggle.com/datasets/mohansacharya/ graduate-admissions. Accessed: 2025-05-08. Acharya, M. S.; Armaan, A.; and Antony, A. S

work page 2025
[8]

In 2011 49th Annual Aller- ton Conference on Communication, Control, and Computing (Allerton), 1143–1150

User rankings from comparisons: Learning per- mutations in high dimensions. In 2011 49th Annual Aller- ton Conference on Communication, Control, and Computing (Allerton), 1143–1150. IEEE. Plackett, R. L

work page 2011
[10]

Your 2 is My 1, Your 3 is My 9: Handling Arbitrary Miscalibrations in Ratings

Your 2 is my 1, your 3 is my 9: Handling arbitrary miscalibrations in ratings. arXiv preprint arXiv:1806.05085. Xia, L

work page Pith review arXiv
[100]

τ = P − Qp (P + Q + T )(Q + Q + U )

Results For these experiments, we measure performance using Kendall-τ, a rank correlation coefficient that quantifies the ordinal association between two ranked lists (Kendall 1938; Marden 1996). τ = P − Qp (P + Q + T )(Q + Q + U ) . Here, P is the number of concordant pairs, Q the number of discordant pairs, T the number of ties only in the first list, a...

work page 1938
[2008]

In Advances in Information Retrieval: 30th Euro- pean Conference on IR Research, ECIR 2008, Glasgow, UK, March 30-April 3,

Here or there: Preference judgments for relevance. In Advances in Information Retrieval: 30th Euro- pean Conference on IR Research, ECIR 2008, Glasgow, UK, March 30-April 3,

work page 2008
[2010]

Busa-Fekete, R.; H ¨ullermeier, E.; and Sz ¨or´enyi, B

ECML/PKDD 2010 Work- shop. Busa-Fekete, R.; H ¨ullermeier, E.; and Sz ¨or´enyi, B

work page 2010
[2011]

In 2011 IEEE/WIC/ACM International Conferences on Web Intelli- gence and Intelligent Agent Technology, volume 1, 451–456

Comparisons in- stead of ratings: Towards more stable preferences. In 2011 IEEE/WIC/ACM International Conferences on Web Intelli- gence and Intelligent Agent Technology, volume 1, 451–456. IEEE. Kalloori, S.; Ricci, F.; and Tkalcic, M

work page 2011
[2012]

arXiv preprint arXiv:1205.2618

BPR: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618. Rubens, N.; Elahi, M.; Sugiyama, M.; and Kaplan, D

arXiv

Show all 12 references

[2017]

shadow ratings

Representation Learning with Pair-wise Constraints for Col- laborative Ranking. In Proceedings of the Tenth ACM In- ternational Conference on Web Search and Data Mining , WSDM ’17, 567–575. New York, NY , USA: Association for Computing Machinery. ISBN 9781450346757. Appendix T...

2015
[2018]

arXiv preprint arXiv:1808.03912

Outer product-based neural collaborative filtering. arXiv preprint arXiv:1808.03912. He, X.; Liao, L.; Zhang, H.; Nie, L.; Hu, X.; and Chua, T.-S

arXiv
[2019]

In 2019 international conference on computa- tional intelligence in data science (ICCIDS) , 1–5

A comparison of regression models for prediction of graduate admissions. In 2019 international conference on computa- tional intelligence in data science (ICCIDS) , 1–5. IEEE. Ammar, A.; and Shah, D

2019
[2024]

arXiv preprint arXiv:2402.00396

Efficient exploration for llms. arXiv preprint arXiv:2402.00396. Harper, F. M.; and Konstan, J. A

arXiv

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.

[1] [2]

Accessed: 2025-05-08

Kaggle Dataset, https://www.kaggle.com/datasets/mohansacharya/ graduate-admissions. Accessed: 2025-05-08. Acharya, M. S.; Armaan, A.; and Antony, A. S

work page 2025

[2] [8]

In 2011 49th Annual Aller- ton Conference on Communication, Control, and Computing (Allerton), 1143–1150

User rankings from comparisons: Learning per- mutations in high dimensions. In 2011 49th Annual Aller- ton Conference on Communication, Control, and Computing (Allerton), 1143–1150. IEEE. Plackett, R. L

work page 2011

[3] [10]

Your 2 is My 1, Your 3 is My 9: Handling Arbitrary Miscalibrations in Ratings

Your 2 is my 1, your 3 is my 9: Handling arbitrary miscalibrations in ratings. arXiv preprint arXiv:1806.05085. Xia, L

work page Pith review arXiv

[4] [100]

τ = P − Qp (P + Q + T )(Q + Q + U )

Results For these experiments, we measure performance using Kendall-τ, a rank correlation coefficient that quantifies the ordinal association between two ranked lists (Kendall 1938; Marden 1996). τ = P − Qp (P + Q + T )(Q + Q + U ) . Here, P is the number of concordant pairs, Q the number of discordant pairs, T the number of ties only in the first list, a...

work page 1938

[5] [2008]

In Advances in Information Retrieval: 30th Euro- pean Conference on IR Research, ECIR 2008, Glasgow, UK, March 30-April 3,

Here or there: Preference judgments for relevance. In Advances in Information Retrieval: 30th Euro- pean Conference on IR Research, ECIR 2008, Glasgow, UK, March 30-April 3,

work page 2008

[6] [2010]

Busa-Fekete, R.; H ¨ullermeier, E.; and Sz ¨or´enyi, B

ECML/PKDD 2010 Work- shop. Busa-Fekete, R.; H ¨ullermeier, E.; and Sz ¨or´enyi, B

work page 2010

[7] [2011]

In 2011 IEEE/WIC/ACM International Conferences on Web Intelli- gence and Intelligent Agent Technology, volume 1, 451–456

Comparisons in- stead of ratings: Towards more stable preferences. In 2011 IEEE/WIC/ACM International Conferences on Web Intelli- gence and Intelligent Agent Technology, volume 1, 451–456. IEEE. Kalloori, S.; Ricci, F.; and Tkalcic, M

work page 2011

[8] [2012]

arXiv preprint arXiv:1205.2618

BPR: Bayesian personalized ranking from implicit feedback. arXiv preprint arXiv:1205.2618. Rubens, N.; Elahi, M.; Sugiyama, M.; and Kaplan, D

arXiv