REVIEW 3 major objections 4 minor 106 references
Recommending With, Not For: Co-Designing Recommender Systems for Social Good
T0 review · 3 major / 4 minor · reviewed 2026-08-06 · deepseek-v4-flash
Pith's one-line read Recommender systems aimed at social good should be designed with and by the stakeholders who experience their effects, not merely tested on them as research subjects.
desk verdict A clear, honest position paper that makes a strong normative case for participatory recommendation; the feasibility gap in co-evaluation is real but the paper admits it, and the agenda deserves a serious referee. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the participatory design (PD) team, a working group in which diverse stakeholders collaborate across the full recommender lifecycle. The mechanism is the participation spectrum running from consultation, through involvement and participation, to full co-production, where participants have equal say over project goals and outcomes. The argument uses this spectrum to judge design-and-evaluation practices: full co-production, not consultation or focus groups, is what participatory recommendation requires, and evaluation metrics and objective functions are treated as design artifacts that the PD team must co-develop rather than receive from experts.
What would settle it
A field study in which stakeholder co-designers are given full authority over a social-good recommender's evaluation metrics, and the resulting metrics are then shown to be less valid than expert-designed ones on the very outcomes the community cares about, would falsify the feasibility premise. More simply, if a co-design process produces no measurable change in the system's goals, metrics, or deployment decisions compared with a designer-only process, the central claim about power sharing would be refuted.
Extended reading notes
Core claim
The central claim is normative: social-good recommender systems should be co-designed with their stakeholders as full co-designers, not just as user study participants. The paper argues that the entire lifecycle—from defining the problem, to designing objective functions, to interpreting evaluation results—should be a power-sharing collaboration among users, providers, and others affected, with designers contributing technical expertise but not acting as final arbiters. If true, the goals inscribed in such systems would reflect the values of the communities that experience their benefits and harms rather than the platform's alone.
Load-bearing premise
The feasibility premise is that people with widely varying expertise can meaningfully co-design and validate statistically complex evaluation metrics; the paper itself concedes that metrics can be difficult even for experienced recommender designers, and that substantial research is needed to make co-evaluation work.
Editorial extensions
If this is right
- Social goals for a recommender would be selected by the people it affects, so 'success' could not be defined by the platform alone.
- Evaluation metrics and experimental designs would be co-created and co-interpreted, requiring new tools that make statistical results legible to non-experts.
- Designers would shift from final decision-makers to expert contributors, retaining authority only over technical feasibility and consequences.
- The approach would start with small, well-defined systems and scale by amalgamating many local participatory efforts rather than by one global design.
- If stakeholders had a real say, user acceptance and trust in the resulting systems could rise, although the paper presents this as a hypothesis.
Reading between the lines
- The strongest feasibility risk is that participatory processes could be captured by the most organized stakeholders, substituting new power imbalances for old ones; the paper acknowledges power conflicts but offers no guardrails.
- A testable implication is that co-designed systems will match community-defined social outcomes better than designer-only systems, which could be tested in a randomized comparison of design processes.
- The argument implicitly extends beyond recommender systems to search engines and other information access tools, as the paper itself notes, suggesting a broader participatory agenda for information systems.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. This position paper argues that recommender systems aimed at social good should be designed by and with their users, providers, and other affected stakeholders as full co-designers, rather than merely as user study participants. The authors review current recommender-system design and evaluation practice, outline the history and principles of participatory design and co-design, and propose a vision of 'participatory recommendation' that spans problem definition, design, evaluation, and ongoing monitoring. The paper develops this vision through three application case studies (fair marketplaces, social media and online safety, and education) and closes with a research agenda listing needed tools, evaluation methods, and political commitments.
Significance. If the central claim is accepted, this paper gives the recommender-systems community a coherent, well-referenced argument for moving beyond user-centered design toward genuine co-production, and it connects the field to the participatory-design tradition in HCI. The paper is notably honest about its own open problems: it explicitly states that much methodological capability 'has yet to be done' (§4) and that 'substantial research is needed' to make co-evaluation work (§4.3). The three case studies ground the argument in concrete domains, and the authors' extensive prior work on fairness and multistakeholder recommendation gives the proposal credibility. The paper's main contribution at this stage is as a framework and agenda rather than a demonstrated method; its significance will depend on whether researchers can supply the feasibility evidence and power-sharing mechanisms it currently lacks.
major comments (3)
- [§1, §4, §4.3] The paper's central claim is that participatory collaboration is 'necessary to truly achieve the goals of recommendation for social good' (§4). This necessity claim rests on the feasibility premise that nonexpert stakeholders can meaningfully participate in designing and interpreting statistically complex evaluation metrics. The paper itself concedes that metrics 'can be difficult even for experienced recommender system designers to interpret' and that 'substantial research is needed' to make co-evaluation work (§4.3), and it lists evaluation techniques enabling nonexperts as an unmet research need (§6). Given these concessions, the unconditional 'necessary' claim is not supported. The authors should either (a) condition the claim on a stated feasibility assumption and articulate what minimal evidence would test it, or (b) substantially expand the discussion of existing participatory evaluation efforts (e.g., Smith et al. 2024) that show nonexperts can acquire meaningful evaluative authority. Without this, the prescriptive force of the paper's main thesis is weakened.
- [§3.2, §4, §6] The paper acknowledges 'participation-washing' and co-option of participatory work (§3.2) and stresses that PD team collaborations should be 'genuinely power-sharing relationships' (§4), but it offers no mechanism by which power imbalances are actually neutralized. The promised accountability in §6—where people 'hold such efforts accountable'—follows only if the participatory process transfers real decision-rights to affected stakeholders. The paper should address at least one concrete power-sharing mechanism, such as stakeholder veto rights over deployment, independent facilitation, resource redistribution, or the institutional conditions under which such mechanisms might be feasible. As written, the political commitment is stated but the means to prevent capture are not, leaving a gap between the vision and its claimed outcome.
- [§5.1, §5.2, §5.3] The case studies in Section 5 are framed as demonstrating how participatory methods 'could impact' recommender design, but none of them actually instantiates full co-production. The Kiva and journalist examples are interview and focus-group studies (§5.1); the social media section describes potential uses of co-design without reporting a completed participatory process (§5.2); and the education example expresses teacher involvement as an aspiration (§5.3). These cases do not provide empirical support for the claim that participation is necessary for social good; they show that stakeholder perspectives diverge from designer assumptions. The authors should explicitly characterize these as illustrative of the need for participation, not as evidence that participatory recommendation is feasible or effective in its full form. If such evidence is not yet available, the paper should say so and identify what future demonstrations would be needed.
minor comments (4)
- [Abstract] The phrase 'designedbyandwith' is missing spaces; it should read 'designed by and with'.
- [Figure 2] The spectrum labels 'consultation,' 'involvement,' 'participation,' and 'co-production' are defined only in prose; adding a one-sentence example for each level would help readers place recommender-system practices on the spectrum.
- [§4.3] The bullet list includes 'Designing and validating evaluation metrics' as a task for the full PD team, immediately after the sentence stating that substantial research is needed to make co-evaluation work; the apparent tension should be resolved by clarifying which aspects of metric design are delegated to experienced researchers as opposed to decided jointly.
- [References] The citation to 'Gómez Gutiérrez et al. 2021' appears in the text alongside 'Gómez et al. 2021' with no explanation of the distinction between the two works; a brief parenthetical clarification would avoid confusion.
Circularity Check
No load-bearing circularity: the paper is a normative argument; its self-citations are scaffolding, and the feasibility gaps it admits are unsupported premises rather than self-referential reductions.
full rationale
Score 2 rather than 0 because the paper leans on several of the authors' prior works (Ekstrand & Willemsen 2016; Burke & Sylvester 2024; Smith et al. 2024; Ekstrand et al. 2024b) as scaffolding, but none of these citations does load-bearing logical work. The paper is a position paper: its central claim that social-good recommender systems should be designed by and with affected stakeholders follows from the normative premise that 'social goals and operationalizations should be developed through participatory and democratic processes that are accountable to their stakeholders' (Abstract and Section 1), not from a fitted parameter or from a self-citation chain. There are no equations, no fitted inputs renamed as predictions, and no uniqueness theorem imported from the authors. The self-citations are pointers to prior arguments and empirical studies (e.g., Smith et al. 2024 on co-designing fairness metrics with providers), and the paper explicitly frames the missing methodology as future work: Section 4 says 'much of the work on adapting and applying participatory design and co-design ... has yet to be done,' and Section 4.3 says metrics 'can be difficult even for experienced recommender system designers to interpret' and that 'substantial research is needed' for co-evaluation; Section 3.2 also acknowledges risks such as 'participation-washing.' These are evidence and feasibility gaps of the sort the skeptic raises, which weaken the argument's actionability but are not circularity. Per the hard rules, missing support is a correctness risk, not a circularity finding, so the analysis does not escalate the score.
Assumptions & free parameters
assumptions (3)
- domain assumption People who experience the benefits and harms of a recommender system are best positioned to define social good and assess its attainment.
- domain assumption Full co-production-level participation yields systems that better serve stakeholder-defined social goals than designer-led processes.
- domain assumption Non-expert stakeholders can meaningfully participate in designing statistically complex evaluation metrics and reviewing results.
Cite this review
Pith. "Pith review of Recommending With, Not For: Co-Designing Recommender Systems for Social Good." pith.science (2026). https://pith.science/paper/ADVEZLTH
@misc{pith2026250803792,
author = {Pith},
title = {Pith review of: Recommending With, Not For: Co-Designing Recommender Systems for Social Good},
year = {2026},
howpublished = {\url{https://pith.science/paper/ADVEZLTH}},
note = {Machine review of arXiv:2508.03792}
}
read the original abstract
Recommender systems are usually designed by engineers, researchers, designers, and other members of development teams. These systems are then evaluated based on goals set by the aforementioned teams and other business units of the platforms operating the recommender systems. This design approach emphasizes the designers' vision for how the system can best serve the interests of users, providers, businesses, and other stakeholders. Although designers may be well-informed about user needs through user experience and market research, they are still the arbiters of the system's design and evaluation, with other stakeholders' interests less emphasized in user-centered design and evaluation. When extended to recommender systems for social good, this approach results in systems that reflect the social objectives as envisioned by the designers and evaluated as the designers understand them. Instead, social goals and operationalizations should be developed through participatory and democratic processes that are accountable to their stakeholders. We argue that recommender systems aimed at improving social good should be designed *by* and *with*, not just *for*, the people who will experience their benefits and harms. That is, they should be designed in collaboration with their users, creators, and other stakeholders as full co-designers, not only as user study participants.
Figures
Reference graph
Works this paper leans on
-
[1]
Publication date: August 2025
2025
-
[2]
doi:10.1109/WI-IAT.2014.116 Sofie Pilemalm
335–342. doi:10.1109/WI-IAT.2014.116 Sofie Pilemalm
-
[8]
doi:10.4230/DagRep.14.5.58 Nicholas J Belkin and Stephen E Robertson
Evaluation Perspectives of Recommender Systems: Driving Research and Education (Dagstuhl Seminar 24211).Dagstuhl Reports14, 5 (2024), 58–172. doi:10.4230/DagRep.14.5.58 Nicholas J Belkin and Stephen E Robertson
-
[10]
Equity of Attention: Amortizing Individual Fairness in Rankings. InProceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval (SIGIR ’18). ACM, 405–414. doi:10.1145/3209978.3210063 Abeba Birhane, William Isaac, Vinodkumar Prabhakaran, Mark Diaz, Madeleine Clare Elish, Iason Gabriel, and Shakir Mohamed
-
[14]
doi:10.1609/icwsm.v14i1.7320 Gourab K Patro, Abhijnan Chakraborty, Ashmi Banerjee, and Niloy Ganguly
AAAI, 522–533. doi:10.1609/icwsm.v14i1.7320 Gourab K Patro, Abhijnan Chakraborty, Ashmi Banerjee, and Niloy Ganguly
-
[15]
Multisided Fairness for Recommendation. (July 2017). http://arxiv.org/abs/1707.00093 Robin Burke, Gediminas Adomavicius, Toine Bogers, Tommaso Di Noia, Dominik Kowald, Julia Neidhardt, Özlem Özgöbek, Maria Soledad Pera, Nava Tintarev, and Jürgen Ziegler
arXiv 2017
-
[17]
InProceedings of the KDD Workshop on Online and Adapting Recommender Systems (OARS)
A Performance-Preserving Fairness Intervention for Adaptive Microfinance Recommendation. InProceedings of the KDD Workshop on Online and Adapting Recommender Systems (OARS). https://par.nsf.gov/biblio/10351344 Robin Burke and Morgan Sylvester
-
[18]
Post-Userist Recommender Systems: A Manifesto. arXiv:2410.11870 doi:10.48550/ arXiv.2410.11870 Robin Burke, Amy Voida, Nicholas Mattei, and Nasim Sonboli
Show all 106 references
-
[19]
InHarvard CRCS Workshop on AI for Social Good at the 29th International Joint Conference on Artificial Intelligence (IJCAI 2020)
Algorithmic Fairness, Institutional Logics, and Social Choice. InHarvard CRCS Workshop on AI for Social Good at the 29th International Joint Conference on Artificial Intelligence (IJCAI 2020). https://projects.iq.harvard.edu/files/crcs/files/ai4sg_2020_paper_29.pdf ACM Trans. ...
2020
-
[20]
arXiv:2211.12434 doi:10.48550/arXiv.2211.12434 Vicky Charisi, Stéphane Chaudron, Rosanna Di Gioia, Riina Vuorikari, Marina Escobar-Planas, Ignacio Sanchez, and Emilia Gomez
Expansive Participatory AI: Supporting Dreaming within Inequitable Institutions. arXiv:2211.12434 doi:10.48550/arXiv.2211.12434 Vicky Charisi, Stéphane Chaudron, Rosanna Di Gioia, Riina Vuorikari, Marina Escobar-Planas, Ignacio Sanchez, and Emilia Gomez. 2022.Artificial Intell...
-
[21]
InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI ’24)
From Fitting Participation to Forging Relationships: The Art of Participatory ML. InProceedings of the 2024 CHI Conference on Human Factors in Computing Systems (CHI ’24). ACM, New York, NY, USA, 1–9. doi:10.1145/3613904.3642775 Eric Corbett, Emily Denton, and Sheena Erete
2024
-
[23]
doi:10.1007/s11257-023-09364-z Fernando Delgado, Stephen Yang, Michael Madaio, and Qian Yang
Fairness in Recom- mender Systems: Research Landscape and Future Directions.User Modeling and User-Adapted Interaction(April 2023). doi:10.1007/s11257-023-09364-z Fernando Delgado, Stephen Yang, Michael Madaio, and Qian Yang
2023 doi
-
[24]
InProceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO ’23)
The Participatory Turn in AI Design: Theoretical Foundations and the Current State of Practice. InProceedings of the 3rd ACM Conference on Equity and Access in Algorithms, Mechanisms, and Optimization (EAAMO ’23). ACM, New York, NY, USA, 1–23. doi:10.1145/3617694.3623261 Gianl...
-
[25]
2020), 65–74
Human-in-the-Loop Artificial Intelligence for Fighting Online Misinformation: Challenges and Opportunities.The Bulletin of the Technical Committee on Data Engineering43, 3 (Sept. 2020), 65–74. http://sites.computer.org/debull/A20sept/p65.pdf Fernando Diaz, Bhaskar Mitra, Micha...
2020
-
[26]
InProceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM ’20)
Evaluating Stochastic Rank- ings with Expected Exposure. InProceedings of the 29th ACM International Conference on Information and Knowledge Management (CIKM ’20). ACM. doi:10.1145/3340531.3411962 Danica Dillion, Niket Tandon, Yuling Gu, and Kurt Gray
-
[27]
doi:10.1016/j.tics.2023.04.008 Ana Maria Bustamante Duarte, Nina Brendel, Auriol Degbelo, and Christian Kray
Can AI Language Models Replace Human Participants? Trends in Cognitive Sciences27, 7 (July 2023), 597–600. doi:10.1016/j.tics.2023.04.008 Ana Maria Bustamante Duarte, Nina Brendel, Auriol Degbelo, and Christian Kray
2023 doi
-
[28]
2018), 3:1–3:39
Participatory Design and Participa- tory Research: An HCI Case Study with Young Forced Migrants.ACM Transactions on Computer-Human Interaction25, 1 (Feb. 2018), 3:1–3:39. doi:10.1145/3145472 Nell K. Duke and Susan V. Bennett-Armistead. 2003.Reading & Writing Informational Text...
2018 doi
-
[29]
InParticipatory Design
Scandinavian Design: On Participation and Skill. InParticipatory Design. CRC Press, 41–77. Michael D. Ekstrand, Lex Beattie, Maria Soledad Pera, and Henriette Cramer. 2024a. Not Just Algorithms: Strategically Addressing Consumer Impacts in Information Retrieval. InProceedings ...
-
[30]
Foundations and Trends®in Information Retrieval16, 1-2 (2022), 1–177
Fairness in Information Access Systems. Foundations and Trends®in Information Retrieval16, 1-2 (2022), 1–177. doi:10.1561/1500000079 Michael D Ekstrand and Martijn C Willemsen
2022 doi
-
[32]
2020), 725–744
Enhancing Classroom Instruction with Online News.Aslib Journal of Information Management72, 5 (Jan. 2020), 725–744. doi:10.1108/AJIM-11-2019-0309 Michael Feffer, Michael Skirpan, Zachary Lipton, and Hoda Heidari
2020 doi
-
[33]
InProceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society (AIES ’23)
From Preference Elicitation to Participatory ML: A Critical Survey & Guidelines for Future Research. InProceedings of the 2023 AAAI/ACM Conference on AI, Ethics, and Society (AIES ’23). ACM, New York, NY, USA, 38–48. doi:10.1145/3600211.3604661 Andres Ferraro, Xavier Serra, an...
2023
-
[35]
doi:10.1287/mnsc.1080.0974 Ursula M Franklin
Blockbuster Culture’s next Rise or Fall: The Impact of Recommender Systems on Sales Diversity.Management Science55, 5 (May 2009), 697–712. doi:10.1287/mnsc.1080.0974 Ursula M Franklin. 2004.The Real World of Technology(revised edition ed.). House of Anansi Press, Toronto, Ont....
2009
-
[36]
Recommender System-Induced Eating Disorder Relapse: Harmful Content and the Challenges of Responsible Recommendation.ACM Trans. Intell. Syst. Technol.16, 1 (Jan. 2025), 15:1–15:13. doi:10.1145/3675404 ACM Trans. Recomm. Syst., Vol. TBA, No. TBA, Article
2025 doi
-
[37]
InSIGIR ’21
The Winner Takes It All: Geographic Imbalance and Provider (Un)Fairness in Educational Recommender Systems. InSIGIR ’21. ACM, New York, NY, USA, 1808–1812. doi:10.1145/3404835.3463235 Emilia Gómez Gutiérrez, Vicky Charisi, and Stephane Chaudron
-
[38]
InProceedings of the Perspectives on the Evaluation of Recommender Systems Workshop 2021, Vol
Evaluating Recommender Systems with and for Children: Towards a Multi-Perspective Framework. InProceedings of the Perspectives on the Evaluation of Recommender Systems Workshop 2021, Vol
2021
-
[39]
2018), 2:1–2:34
Amplifying Quiet Voices: Challenges and Opportunities for Participatory Design at an Urban Scale.ACM Transactions on Computer- Human Interaction25, 1 (Jan. 2018), 2:1–2:34. doi:10.1145/3139398 Judith Gregory
2018 doi
-
[41]
InRecommender Systems Handbook(third ed.), Francesco Ricci, Lior Rokach, and Bracha Shapira (Eds.)
Evaluating Recommender Systems. InRecommender Systems Handbook(third ed.), Francesco Ricci, Lior Rokach, and Bracha Shapira (Eds.). Springer US, New York, NY, 547–601. doi:10.1007/978-1-0716-2197-4_15 Francisco Gutiérrez, Nyi Nyi Htun, Vero Vanden Abeele, Robin De Croon, and K...
-
[42]
InProceedings of the 27th International Conference on Intelligent User Interfaces (IUI ’22)
Explaining Call Recommendations in Nursing Homes: A User-Centered Design Approach for Interacting with Knowledge-Based Health Decision Support Systems. InProceedings of the 27th International Conference on Intelligent User Interfaces (IUI ’22). ACM, New York, NY, USA, 162–172....
-
[43]
InProceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20)
Designing Trans Technology: Defining Challenges and Envisioning Community-Centered Solutions. InProceedings of the 2020 CHI Conference on Human Factors in Computing Systems (CHI ’20). ACM, New York, NY, USA, 1–13. doi:10.1145/3313831.3376669 Jaron Harambam, Dimitrios Bountouri...
2020
-
[45]
2024), 2603–2605
AI Language Models Cannot Replace Human Research Participants.AI & SOCIETY39, 5 (Oct. 2024), 2603–2605. doi:10.1007/s00146-023-01725-x William Hill, Larry Stead, Mark Rosenstein, and George Furnas
2024 doi
-
[46]
InCHI ’95: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems
Recommending and Evaluating Choices in a Virtual Community of Use. InCHI ’95: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. ACM Press/Addison-Wesley Publishing Co., New York, NY, USA, 194–201. doi:10.1145/223904.223929 Velislava Hillman and Molly Esquivel
-
[47]
https://progressive.org/api/content/6ac9c2dc-e8fa-11ec-a11d-12274efc5439/ Dietmar Jannach and Christine Bauer
How Surveillance Capitalism Is Spreading to Schools.The Progressive Magazine (June 2022). https://progressive.org/api/content/6ac9c2dc-e8fa-11ec-a11d-12274efc5439/ Dietmar Jannach and Christine Bauer
2022
-
[48]
2020), 79–95
Escaping the McNamara Fallacy: Toward More Impactful Recommender Systems Research.AI Magazine41, 4 (Dec. 2020), 79–95. doi:10.1609/aimag.v41i4.5312 Maria Soledad Pera Jason Hall
2020 doi
-
[49]
InProceedings of the Thirtieth International Florida Artificial Intelligence Research Society Conference (FLAIRS 2017)
UBR: A Book Search — Recommender Hybrid. InProceedings of the Thirtieth International Florida Artificial Intelligence Research Society Conference (FLAIRS 2017). AAAI. https://aaai.org/papers/665-flairs-2017- 15422/ Nis Johannsen and Finn Kensing
2017
-
[51]
InProceedings of the 2020 Conference on Fairness, Accountability, and Transparency
Toward Situated Interventions for Algorithmic Equity: Lessons from the Field. InProceedings of the 2020 Conference on Fairness, Accountability, and Transparency. ACM, New York, NY, USA, 45–55. doi:10.1145/3351095.3372874 Finn Kensing and Jeanette Blomberg
2020
-
[53]
In RecSys ’16
Recommender Systems for Self-Actualization. In RecSys ’16. ACM, New York, NY, USA, 11–14. doi:10.1145/2959100.2959189 Bart P Knijnenburg, Martijn C Willemsen, Zeno Gantner, Hakan Soncu, and Chris Newell
-
[54]
2012), 441–504
Explaining the User Experience of Recommender Systems.User Modeling and User-Adapted Interaction22, 4 (Oct. 2012), 441–504. doi:10. 1007/s11257-011-9118-4 Ron Kohavi, Diane Tang, and Ya Xu. 2020.Trustworthy Online Controlled Experiments: A Practical Guide to A/B Testing. Cambr...
2012 doi
-
[55]
2021), 31–42
Human-Centered Recommender Systems: Origins, Advances, Challenges, and Opportunities.AI Magazine42, 3 (Nov. 2021), 31–42. doi:10.1609/aimag.v42i3.18142 Margit Kristensen, Morten Kyng, and Leysia Palen
2021 doi
-
[58]
So, Tell Me What Users Want, What They Really, Really Want!
"So, Tell Me What Users Want, What They Really, Really Want!". InExtended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, New York, NY, USA, alt04:1–alt04:10. doi:10.1145/3170427.3188397 Beth Maloch
2018
-
[60]
In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization
Personalized Recommendation of PoIs to People with Autism. In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization. ACM, New York, NY, USA, 163–172. doi:10.1145/3340631.3394845 Noemi Mauro, Liliana Ardissono, and Federica Cena
-
[61]
ACM65, 2 (Jan
Supporting People with Autism Spectrum Disorders in the Exploration of PoIs: An Inclusive Recommender System.Commun. ACM65, 2 (Jan. 2022), 101–109. doi:10.1145/3505267 Rishabh Mehrotra, James McInerney, Hugues Bouchard, Mounia Lalmas, and Fernando Diaz
2022 doi
-
[62]
InProceedings of the 27th ACM International Conference on Information and Knowledge Management (CIKM ’18)
Towards a Fair Mar- ketplace: Counterfactual Evaluation of the Trade-off between Relevance, Fairness & Satisfaction in Recommendation Systems. InProceedings of the 27th ACM International Conference on Information and Knowledge Management (CIKM ’18). ACM, 2243–2251. doi:10.1145...
-
[63]
2020), 957–967
Recommender Systems and Their Ethical Challenges.AI & Society35, 4 (Dec. 2020), 957–967. doi:10.1007/s00146-020-00950-y Andrew D. Miller, Sonali R. Mishra, Logan Kendall, Shefali Haldar, Ari H. Pollack, and Wanda Pratt
2020 doi
-
[64]
InProceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW ’16)
Partners in Care: Design Considerations for Caregivers and Patients during a Hospital Stay. InProceedings of the 19th ACM Conference on Computer-Supported Cooperative Work & Social Computing (CSCW ’16). ACM, New York, NY, USA, 756–769. doi:10.1145/2818048.2819983 Ashlee Milton...
-
[65]
In Human Computer Interaction Handbook
Participatory Design: The Third Space in Human-Computer Interaction. In Human Computer Interaction Handbook. CRC Press, 1125–1153. doi:10.1201/B11963-57 Michael J. Muller and Sarah Kuhn
-
[67]
Taxonomy of PD Practices: A Brief Practitioner’s Guide. Commun. ACM36, 6 (1993),
1993
-
[68]
doi:10.1145/153571.255960 Emiliana Murgia, Monica Landoni, Theo Huibers, Jerry Alan Fails, and Maria Soledad Pera
-
[72]
Recommender Systems
1:22 Ekstrand et al. Recommender Systems. ACM, 358–367. doi:10.1145/3383313.3412251 Maria Soledad Pera, Federica Cena, Monica Landoni, Cataldo Musto, and Alain D. Starke
-
[73]
InA Human-Centered Perspective of Intelligent Personalized Environments and Systems, Bruce Ferwerda, Mark Graus, Panagiotis Germanakos, and Marko Tkalčič (Eds.)
Human Factors in User Modeling for Intelligent Systems. InA Human-Centered Perspective of Intelligent Personalized Environments and Systems, Bruce Ferwerda, Mark Graus, Panagiotis Germanakos, and Marko Tkalčič (Eds.). Springer Nature Switzerland, Cham, 3–42. doi:10.1007/978-3-...
-
[75]
2018), 5:1–5:26
Participatory Design in Emerging Civic Engagement Initiatives in the New Public Sector: Applying PD Concepts in Resource-Scarce Organizations.ACM Transactions on Computer-Human Interaction25, 1 (Jan. 2018), 5:1–5:26. doi:10.1145/3152420 Victoria Purcell-Gates, Nell K Duke, and...
2018 doi
-
[77]
In2022 ACM Conference on Fairness, Accountability, and Transparency
Data Cards: Purposeful and Transparent Dataset Documentation for Responsible AI. In2022 ACM Conference on Fairness, Accountability, and Transparency. ACM, Seoul Republic of Korea, 1776–1826. doi:10.1145/3531146.3533231 Amifa Raj and Michael D Ekstrand
-
[78]
InProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval
Measuring Fairness in Ranked Results: An Analytical and Empirical Comparison. InProceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM, 726–736. doi:10.1145/3477495.3532018 Chand Rajendra-Nicoluccis, Michael Sugarma...
-
[80]
2021), 465:1–465:38
A Human-Centered Systematic Literature Review of the Computational Approaches for Online Sexual Risk Detection.Proceedings of the ACM on Human-Computer Interaction5, CSCW2 (Oct. 2021), 465:1–465:38. doi:10.1145/3479609 Elizabeth B.-N. Sanders and Pieter Jan Stappers
2021 doi
-
[81]
doi:10.1080/15710880701875068 Olga C Santos, Jesus G Boticario, and Diana Pérez-Marín
Co-Creation and the New Landscapes of Design.CoDesign4, 1 (March 2008), 5–18. doi:10.1080/15710880701875068 Olga C Santos, Jesus G Boticario, and Diana Pérez-Marín
2008 doi
-
[83]
Online Music Listening Culture of Kids and Adolescents: Listening Analysis and Music Recommendation Tailored to the Young. (Dec. 2019). http://arxiv.org/abs/1912.11564 Roan Schellingerhout, Francesco Barile, and Nava Tintarev
2019 arXiv
-
[84]
InExplainable Artificial Intelligence, Luca Longo (Ed.)
A Co-Design Study for Multi-Stakeholder Job Recommender System Explanations. InExplainable Artificial Intelligence, Luca Longo (Ed.). Springer Nature Switzerland, Cham, 597–620. doi:10.1007/978-3-031-44067-0_30 Nick Seaver
-
[85]
Captivating Algorithms: Recommender Systems as Traps.Journal of Material Culture24, 4 (Dec. 2018). doi:10.1177/1359183518820366 John S. Seberger, Marissel Llavore, Nicholas Nye Wyant, Irina Shklovski, and Sameer Patil
2018 doi
-
[86]
InProceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI ’21)
Empowering Resignation: There’s an App for That. InProceedings of the 2021 CHI Conference on Human Factors in Computing Systems (CHI ’21). ACM, New York, NY, USA, 1–18. doi:10.1145/3411764.3445293 Joshua M. Sharfstein
2021
-
[87]
Stakeholders
Banishing “Stakeholders”.The Milbank Quarterly94, 3 (Sept. 2016), 476–479. doi:10.1111/1468- 0009.12208 Ben Shneiderman. 2022.Human-Centered AI. Oxford University Press, Oxford, United Kingdom. Ashudeep Singh and Thorsten Joachims
2016
-
[88]
InProceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining
Fairness of Exposure in Rankings. InProceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. ACM, New York, NY, USA, 2219–2228. doi:10.1145/ 3219819.3220088 Mona Sloane, Emanuel Moss, Olaitan Awomolo, and Laura Forlano
-
[89]
ACM, New York, NY, USA, 1–6
Recommending With, Not For 1:23 (EAAMO ’22). ACM, New York, NY, USA, 1–6. doi:10.1145/3551624.3555285 Jessie J. Smith, Anas Buhayh, Anushka Kathait, Pradeep Ragothaman, Nicholas Mattei, Robin Burke, and Amy Voida
-
[90]
In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’23)
The Many Faces of Fairness: Exploring the Institutional Logics of Multistakeholder Microlending Recommendation. In Proceedings of the 2023 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’23). ACM, New York, NY, USA, 1652–1663. doi:10.1145/3593013.3594106 J...
2023
-
[91]
InProceedings of the 16th ACM Conference on Recommender Systems (RecSys ’22)
Recommender Systems and Algorithmic Hate. InProceedings of the 16th ACM Conference on Recommender Systems (RecSys ’22). ACM, New York, NY, USA, 592–597. doi:10.1145/3523227.3551480 Jessie J. Smith, Aishwarya Satwani, Robin Burke, and Casey Fiesler
-
[92]
InThe 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’24)
Recommend Me? Designing Fairness Metrics with Providers. InThe 2024 ACM Conference on Fairness, Accountability, and Transparency (FAccT ’24). ACM, New York, NY, USA, 2389–2399. doi:10.1145/3630106.3659044 Jessie J. Smith, Nasim Sonbolil, Casey Fiesler, and Robin Burke
2024
-
[93]
doi:10.1002/aaai.12054 Nasim Sonboli, Jessie J
The Multisided Complexity of Fairness in Recommender Systems.AI Magazine43, 2 (June 2022), 164–176. doi:10.1002/aaai.12054 Nasim Sonboli, Jessie J. Smith, Florencia Cabral Berenfus, Robin Burke, and Casey Fiesler
2022 doi
-
[94]
InProceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization
Fairness and Transparency in Recommendation: The Users’ Perspective. InProceedings of the 29th ACM Conference on User Modeling, Adaptation and Personalization. ACM, New York, NY, USA, 274–279. doi:10.1145/3450613.3456835 Alain Starke, Martijn Willemsen, and Chris Snijders
-
[95]
InProceedings of the Eleventh ACM Conference on Recommender Systems
Effective User Interface Designs to Increase Energy-Efficient Behavior in a Rasch-based Energy Recommender System. InProceedings of the Eleventh ACM Conference on Recommender Systems. ACM, 65–73. doi:10.1145/3109859.3109902 Caroline Stockman and Emma Nottingham
-
[96]
2022), 1–15
Surveillance Capitalism in Schools: What’s the Problem?Digital Culture and Education14, 1 (Jan. 2022), 1–15. https://www.digitalcultureandeducation.com/volume-14-1/stockman-2022 Jonathan Stray, Alon Halevy, Parisa Assar, Dylan Hadfield-Menell, Craig Boutilier, Amar Ashar, Chlo...
2022
-
[97]
doi:10.1145/3632297 Özge Sürer, Robin Burke, and Edward C Malthouse
Building Human Values into Recommender Systems: An Interdisciplinary Synthesis.ACM Transactions on Recommender Systems2, 3 (June 2024), 20:1–57. doi:10.1145/3632297 Özge Sürer, Robin Burke, and Edward C Malthouse
2024 doi
-
[98]
In Proceedings of the 12th ACM Conference on Recommender Systems
Multistakeholder Recommendation with Provider Constraints. In Proceedings of the 12th ACM Conference on Recommender Systems. ACM, New York, NY, USA, 54–62. doi:10.1145/3240323. 3240350 Lida Theodorou, Daniela Massiceti, Luisa Zintgraf, Simone Stumpf, Cecily Morrison, Edward Cu...
-
[99]
InProceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’21)
Disability-First Dataset Creation: Lessons from Constructing a Dataset for Teachable Object Recognition with Blind and Low Vision Data Collectors. InProceedings of the 23rd International ACM SIGACCESS Conference on Computers and Accessibility (ASSETS ’21). ACM, New York, NY, U...
-
[100]
InRecommender Systems Handbook, Francesco Ricci, Lior Rokach, and Bracha Shapira (Eds.)
Explaining Recommendations: Design and Evaluation. InRecommender Systems Handbook, Francesco Ricci, Lior Rokach, and Bracha Shapira (Eds.). Springer US, Boston, MA, 353–382. doi:10.1007/978- 1-4899-7637-6_10 Fabiana Tomasini Giannini and Ingrid Mulder
-
[101]
InProceedings of the Participatory Design Conference 2022 - Volume 2 (PDC ’22, Vol
Towards a Power-Balanced Participatory Design Process. InProceedings of the Participatory Design Conference 2022 - Volume 2 (PDC ’22, Vol. 2). ACM, New York, NY, USA, 111–117. doi:10.1145/ 3537797.3537819 Sabina Tomkins, Steven Isley, Ben London, and Lise Getoor
2022
-
[102]
InProceedings of the 12th ACM Conference on Recommender Systems
Sustainability at Scale: Towards Bridging the Intention- Behavior Gap with Sustainable Recommendations. InProceedings of the 12th ACM Conference on Recommender Systems. ACM, 214–218. doi:10.1145/3240323.3240411 Antonela Tommasel, Daniela Godoy, and Arkaitz Zubiaga
-
[103]
2021), 171–201
Recommender Systems in the Healthcare Domain: State-of-the-Art and Research Issues.Journal of Intelligent Information Systems57, 1 (Aug. 2021), 171–201. doi:10.1007/s10844-020-00633-6 Robin Ungruh, Karlijn Dinnissen, Anja Volk, Maria Soledad Pera, and Hanna Hauptmann
2021 doi
-
[104]
InProceedings of the 18th ACM Conference on Recommender Systems (RecSys ’24)
Putting Popularity Bias Mitigation to the Test: A User-Centric Evaluation in Music Recommenders. InProceedings of the 18th ACM Conference on Recommender Systems (RecSys ’24). ACM, New York, NY, USA, 169–178. doi:10.1145/3640457.3688102 Robin Ungruh and Maria Soledad Pera
- [105]
-
[106]
InProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining
Fairness in Networks: Social Capital, Information Access, and Interventions. InProceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. ACM, New York, NY, USA, 4078–4079. doi:10.1145/3447548.3470821 Torben Volkmann, Markus Dresel, and Nicole Jochems
-
[107]
InExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (CHI EA ’23)
Balancing Power Relations in Participatory Design: The Importance of Initiative and External Factors. InExtended Abstracts of the 2023 CHI Conference on Human Factors in Computing Systems (CHI EA ’23). ACM, New York, NY, USA, 1–6. doi:10.1145/3544549.3585864 Yifan Wang, Weizhi...
2023
-
[108]
ACM Transactions on Information Systems41, 3 (Feb
A Survey on the Fairness of Recommender Systems. ACM Transactions on Information Systems41, 3 (Feb. 2023), 52:1–52:43. doi:10.1145/3547333 Jason C. Yip, Kiley Sobel, Caroline Pitt, Kung Jin Lee, Sijin Chen, Kari Nasu, and Laura R. Pina
2023 doi
-
[109]
InProceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI ’17)
Examining Adult-Child Interactions in Intergenerational Participatory Design. InProceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI ’17). ACM, New York, NY, USA, 5742–5754. doi:10.1145/3025453.3025787 Eva Zangerle and Christine Bauer
2017
-
[110]
Surveys55, 8 (Dec
Evaluating Recommender Systems: Survey and Framework.Comput. Surveys55, 8 (Dec. 2022), 170:1–170:38. doi:10.1145/3556536 Meike Zehlike, Ke Yang, and Julia Stoyanovich
2022 doi
-
[111]
Surveys (April 2022)
Fairness in Ranking, Part I: Score-Based Ranking.Comput. Surveys (April 2022). doi:10.1145/3533379 Douglas Zytko, Pamela J. Wisniewski, Shion Guha, Eric P. S. Baumer, and Min Kyung Lee
2022 doi
-
[112]
InExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (CHI EA ’22)
Participatory Design of AI Systems: Opportunities and Challenges across Diverse Users, Relationships, and Application Domains. InExtended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems (CHI EA ’22). ACM, New York, NY, USA, 1–4. doi:10.1145/3491101.3...
2022
-
[1976]
InProceedings of the ASIS Annual Meeting
Some Ethical and Political Implications of Theoretical Research in Information Science. InProceedings of the ASIS Annual Meeting. https://www.researchgate.net/publication/255563562 Asia J Biega, Krishna P Gummadi, and Gerhard Weikum
-
[1993]
ACM36, 6 (June 1993), 24–28
Participatory Design.Commun. ACM36, 6 (June 1993), 24–28. doi:10.1145/153571. 255960 Michael J. Muller, Daniel M. Wildman, and Ellen A. White
1993 doi
-
[1995]
https://aisel.aisnet.org/sjis/vol7/iss1/1 Linda Golson Bradley and Carol A Donovan
User Participation and Democracy: A Discussion of Scandinavian Research on System Development.Scandinavian Journal of Information Systems7, 1 (1995), 73–98. https://aisel.aisnet.org/sjis/vol7/iss1/1 Linda Golson Bradley and Carol A Donovan
1995
-
[1998]
1998), 167–185
Participatory Design: Issues and Concerns.Computer Supported Cooperative Work (CSCW)7, 3 (Sept. 1998), 167–185. doi:10.1023/A:1008689307411 Bart P Knijnenburg, Saadhika Sivakumar, and Daricia Wilkinson
1998 doi
-
[2003]
https://www.ijee.ie/articles/Vol19-1/IJEE1353.pdf Asela Gunawardana, Guy Shani, and Sivan Yogev
Scandinavian Approaches to Participatory Design.International Journal of Engineering Education19, 1 (2003), 62–74. https://www.ijee.ie/articles/Vol19-1/IJEE1353.pdf Asela Gunawardana, Guy Shani, and Sivan Yogev
2003
-
[2005]
InProceedings of the 4th Decennial Conference on Critical Computing: Between Sense and Sensibility (CC ’05)
Empowerment Reconsidered. InProceedings of the 4th Decennial Conference on Critical Computing: Between Sense and Sensibility (CC ’05). ACM, New York, NY, USA, 203–206. doi:10.1145/1094562.1094599 Michael Katell, Meg Young, Dharma Dailey, Bernease Herman, Vivian Guetler, Aaron ...
-
[2006]
InProceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’06)
Participatory Design in Emergency Medical Service: Designing for Future Practice. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’06). ACM, New York, NY, USA, 161–170. doi:10.1145/1124772.1124798 ACM Trans. Recomm. Syst., Vol. TBA, No. TBA, Article
-
[2007]
doi:10.1598/RRQ.42.1.1 Mahima Pushkarna, Andrew Zaldivar, and Oddur Kjartansson
Learning to Read and Write Genre-Specific Text: Roles of Authentic Experience and Explicit Teaching.Reading Research Quarterly42, 1 (2007). doi:10.1598/RRQ.42.1.1 Mahima Pushkarna, Andrew Zaldivar, and Oddur Kjartansson
2007 doi
-
[2008]
jstor:40171756 https://www.jstor.org/stable/40171756 Noemi Mauro, Liliana Ardissono, and Federica Cena
Beyond Exposure: The Uses of Informational Texts in a Second Grade Classroom.Research in the Teaching of English42, 3 (2008), 315–362. jstor:40171756 https://www.jstor.org/stable/40171756 Noemi Mauro, Liliana Ardissono, and Federica Cena
2008
-
[2009]
doi:10.1598/RT.62.6.3 Liam Bannon, Jeffrey Bardzell, and Susanne Bødker
Making Science Trade Book Choices for Elementary Classrooms.The Reading Teacher62, 6 (2009), 484–497. doi:10.1598/RT.62.6.3 Liam Bannon, Jeffrey Bardzell, and Susanne Bødker
2009 doi
-
[2010]
The Reading Teacher64, 4 (2010), 246–260
Information Book Read-Alouds as Models for Second-Grade Authors. The Reading Teacher64, 4 (2010), 246–260. doi:10.1598/RT.64.4.3 Eva Brandt, Thomas Binder, and Elizabeth B-N Sanders
2010 doi
-
[2012]
InProceedings of the 12th Participatory Design Conference: Research Papers - Volume 1 (PDC ’12)
Disentangling Power and Decision-Making in Participatory Design. InProceedings of the 12th Participatory Design Conference: Research Papers - Volume 1 (PDC ’12). ACM, New York, NY, USA, 41–50. doi:10.1145/2347635.2347642 Robin Burke
-
[2013]
InProceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’13)
Why Interactive Learning Environments Can Have It All: Resolving Design Conflicts between Competing Goals. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’13). ACM, New York, NY, USA, 109–118. doi:10.1145/2470654.2470670 Afsaneh Razi, Seunghy...
-
[2014]
doi:10.1016/j.scico.2013.12.004 Markus Schedl and Christine Bauer
Extending Web-Based Educational Systems with Personalised Support through User Centred Designed Recommendations along the e-Learning Life Cycle.Science of Computer Programming88 (2014), 92–109. doi:10.1016/j.scico.2013.12.004 Markus Schedl and Christine Bauer
2014 doi
-
[2015]
2015), 1–38
Sketching Awareness: A Participatory Study to Elicit Designs for Supporting Ad Hoc Emergency Medical Teamwork.Computer Supported Cooperative Work (CSCW)24, 1 (Feb. 2015), 1–38. doi:10.1007/s10606-014-9210-5 Ulrik Lyngs, Reuben Binns, Max Van Kleek, and Nigel Shadbolt
2015 doi
-
[2017]
InProceedings of the 25th Conference on User Modeling, Adaptation and Personalization (UMAP ’17)
Recommender Systems as Multistakeholder Environments. InProceedings of the 25th Conference on User Modeling, Adaptation and Personalization (UMAP ’17). ACM, 347–348. doi:10.1145/3079628.3079657 William Agnew, A. Stevie Bergman, Jennifer Chien, Mark Díaz, Seliem El-Sayed, Jayle...
-
[2018]
2018), 26–32
Reimagining Participatory Design.Interactions26, 1 (Dec. 2018), 26–32. doi:10.1145/3292015 Alisha Banskota and Yiu-Kai Ng
2018 doi
-
[2019]
In Proceedings of the 13th ACM Conference on Recommender Systems
Designing for the Better by Taking Users into Account: A Qualitative Evaluation of User Control Mechanisms in (News) Recommender Systems. In Proceedings of the 13th ACM Conference on Recommender Systems. ACM, New York, NY, USA, 69–77. doi:10.1145/3298689. 3347014 Jacqueline Ha...
-
[2020]
doi:10.1007/s11257-019-09256-1 Himan Abdollahpouri, Robin Burke, and Bamshad Mobasher
Multistakeholder Recommendation: Survey and Research Directions.User Modeling and User-Adapted Interaction30, 1 (March 2020), 127–158. doi:10.1007/s11257-019-09256-1 Himan Abdollahpouri, Robin Burke, and Bamshad Mobasher
2020 doi
-
[2021]
InProceedings of the 18th IFIP International Conference on Human-Computer Interaction (INTERACT 2021)
What Is Fair? Exploring the Artists’ Perspective on the Fairness of Music Streaming Platforms. InProceedings of the 18th IFIP International Conference on Human-Computer Interaction (INTERACT 2021). http://arxiv.org/abs/2106.02415 Daniel M Fleder and Kartik Hosanagar
2021 arXiv
-
[2023]
It’s Just a Robot That Looks at Numbers
“It’s Just a Robot That Looks at Numbers”: Restoring Journalistic Voice in News Recommendation.Proceedings of the International Workshop on News Recommendation and Analytics(Sept. 2023). https://ceur-ws.org/Vol-3561/paper1.pdf Hannah Arendt. 2019.The Human Condition. Universit...
2023
-
[2025]
arXiv:2501.05170 [cs] doi:10.48550/arXiv.2501.05170 Robin Burke, Pradeep Ragothaman, Nicholas Mattei, Brian Kimmig, Amy Voida, Nasim Sonboli, Anushka Kathait, and Melissa Fabros
De-Centering the (Traditional) User: Multistakeholder Evaluation of Recommender Systems. arXiv:2501.05170 [cs] doi:10.48550/arXiv.2501.05170 Robin Burke, Pradeep Ragothaman, Nicholas Mattei, Brian Kimmig, Amy Voida, Nasim Sonboli, Anushka Kathait, and Melissa Fabros
Reviewed August 6, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.