REVIEW 4 cited by
De-centering the (Traditional) User: Multistakeholder Evaluation of Recommender Systems
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
Multistakeholder recommender systems are those that account for the impacts and preferences of multiple groups of individuals, not just the end users receiving recommendations. Due to their complexity, these systems cannot be evaluated strictly by the overall utility of a single stakeholder, as is often the case of more mainstream recommender system applications. In this article, we focus our discussion on the challenges of multistakeholder evaluation of recommender systems. We bring attention to the different aspects involved -- from the range of stakeholders involved (including but not limited to providers and consumers) to the values and specific goals of each relevant stakeholder. We discuss how to move from theoretical principles to practical implementation, providing specific use case examples. Finally, we outline open research directions for the RecSys community to explore. We aim to provide guidance to researchers and practitioners about incorporating these complex and domain-dependent issues of evaluation in the course of designing, developing, and researching applications with multistakeholder aspects.
Forward citations
Cited by 4 Pith papers
-
A Multistakeholder Approach to Value-Driven Co-Design of Recommender System Evaluation Metrics in Digital Archives
A qualitative study translates stakeholder values from digital archives focus groups into a four-stage research funnel and eight proposed recommender evaluation metric directions.
-
Exploring the Effect of Context-Awareness and Popularity Calibration on Popularity Bias in POI Recommendations
Combining the context-aware LORE model with calibrated popularity re-ranking yields POI recommendations whose popularity distribution most closely matches users' historical preferences.
-
Multistakeholder Fairness in Tourism: What can Algorithms learn from Tourism Management?
A comparative literature review shows tourism management and computer science define multistakeholder fairness differently, and argues algorithmic design should adopt qualitative, participatory methods from tourism research.
-
Recommending With, Not For: Co-Designing Recommender Systems for Social Good
Recommender systems for social good should be co-designed with the communities they affect, across problem definition, design, evaluation, and monitoring.
Discussion (0). Sign in to comment.