ConSearcher: Supporting Conversational Information Seeking in Online Communities with Member Personas
Pith reviewed 2026-05-15 08:44 UTC · model grok-4.3
The pith
ConSearcher improves conversational search in online communities by generating member personas from queries to surface diverse perspectives.
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
ConSearcher dynamically generates member personas from user queries so that people can clarify their interests by checking what a simulated similar member might ask and can obtain responses that reflect diverse community perspectives, yielding significantly higher information-seeking outcomes and user engagement than two standard conversational search baselines.
What carries the argument
Dynamically generated member personas based on user queries, which simulate community members' perspectives to let users explore interests and receive viewpoint-diverse answers within a conversational interface.
If this is right
- Users reach better information-seeking outcomes when they can preview questions from simulated similar members.
- Engagement rises when responses are framed through multiple member personas rather than a single generic chat.
- Over-personalization becomes a visible risk that can distort the range of perspectives users encounter.
- The same persona-generation approach can be applied to other community topics such as travel planning or product advice.
Where Pith is reading between the lines
- Replacing LLM personas with real-time member data could reduce hallucination risks while preserving the engagement gains.
- Over-personalization may create echo-chamber effects if users repeatedly interact only with personas that match their initial queries.
- Scaling the system to very large communities would require testing whether persona quality remains stable across thousands of members.
Load-bearing premise
LLM-generated personas based on queries will accurately and helpfully represent real community members' perspectives without introducing systematic biases or hallucinations.
What would settle it
A direct comparison study in which participants rate ConSearcher responses against actual replies posted by real community members on the same queries, measuring accuracy, bias, and perceived helpfulness.
Figures
read the original abstract
Many people browse online communities to learn from others' experiences and opinions, e.g., for constructing travel plans. Conversational search powered by large language models (LLMs) could ease this information-seeking task, but it remains under-investigated within the online community. In this paper, we first conducted an exploratory study (N=10) that indicated the helpfulness of a classic conversational search tool and identified room for improvement. Then, we proposed ConSearcher, an LLM-powered tool with dynamically generated member personas based on user queries to facilitate conversational search in the community. In ConSearcher, users can clarify their interests by checking what a simulated member similar to them may ask and get responses from diverse members' perspectives. A within-subjects study (N=27) showed that compared to two conversational search baselines, ConSearcher led to significantly higher information-seeking outcome and user engagement but raised concerns about over-personalization. We discuss implications for supporting conversational information seeking in online communities.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces ConSearcher, an LLM-powered conversational search tool for online communities that dynamically generates member personas based on user queries to provide diverse perspectives. Following an exploratory study (N=10) highlighting needs, a within-subjects study (N=27) demonstrates that ConSearcher yields significantly higher information-seeking outcomes and user engagement compared to two baselines, though it raises issues of over-personalization.
Significance. If the persona mechanism can be shown to drive the gains without artifacts from LLM generation, this could advance design of multi-perspective conversational tools in online communities, with implications for HCI and CSCW. The two user studies provide empirical grounding for the claims.
major comments (1)
- [N=27 within-subjects study] N=27 within-subjects study: The attribution of significantly higher information-seeking outcomes and engagement to the member personas rests on the untested assumption that LLM-generated personas faithfully represent real community members' perspectives. No ground-truth validation (e.g., overlap with actual posts, expert fidelity ratings, or hallucination audit) is reported, so measured gains could arise from generation artifacts rather than the intended multi-perspective mechanism.
minor comments (2)
- [Abstract] Abstract: The claim of 'significantly higher' outcomes omits any mention of the statistical tests, effect sizes, measures, or controls, which would allow immediate assessment of evidence strength.
- [Method] Method: More detail on the implementation of the two conversational search baselines would aid replication and clarify what aspects of ConSearcher drive the differences.
Simulated Author's Rebuttal
We thank the referee for their constructive feedback. We address the major comment below and will revise the manuscript accordingly.
read point-by-point responses
-
Referee: [N=27 within-subjects study] N=27 within-subjects study: The attribution of significantly higher information-seeking outcomes and engagement to the member personas rests on the untested assumption that LLM-generated personas faithfully represent real community members' perspectives. No ground-truth validation (e.g., overlap with actual posts, expert fidelity ratings, or hallucination audit) is reported, so measured gains could arise from generation artifacts rather than the intended multi-perspective mechanism.
Authors: We thank the referee for this observation. Both baselines in the N=27 within-subjects study rely on the identical LLM backbone, so any general generation artifacts are controlled for; the only systematic difference is the dynamic member-persona mechanism. The exploratory study (N=10) also provided qualitative support that users perceived the personas as reflecting plausible community viewpoints. We nevertheless agree that explicit fidelity validation (e.g., overlap metrics with real posts or expert ratings) is absent. In the revised manuscript we will expand the Limitations section to acknowledge this gap and outline concrete future validation steps, including expert fidelity audits and post-hoc comparison against community archives. revision: yes
Circularity Check
No significant circularity detected
full rationale
The paper is an empirical HCI contribution that proposes ConSearcher and evaluates it via two independent user studies (exploratory N=10 and within-subjects N=27). No equations, parameter fitting, or derivation chain exist; the central claims rest on measured outcomes from participant interactions rather than any self-referential reduction. Self-citations, if present, are not load-bearing for the reported results, which are externally falsifiable through the study protocols.
Axiom & Free-Parameter Ledger
Reference graph
Works this paper leans on
-
[1]
Butler, Elisabeth Joyce, Robert Kraut, Kimberly S
Jaime Arguello, Brian S. Butler, Elisabeth Joyce, Robert Kraut, Kimberly S. Ling, Carolyn Rosé, and Xiaoqing Wang. 2006. Talk to me: foundations for successful individual-group interactions in online communities. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems(Montréal, Québec, Canada) (CHI ’06). Association for Computing Mach...
-
[2]
2012.Thematic analysis.American Psycho- logical Association
Virginia Braun and Victoria Clarke. 2012.Thematic analysis.American Psycho- logical Association
work page 2012
-
[3]
Ricardo JGB Campello, Davoud Moulavi, and Jörg Sander. 2013. Density-based clustering based on hierarchical density estimates. InPacific-Asia conference on knowledge discovery and data mining. Springer, 160–172
work page 2013
-
[4]
Ana Paula Chaves and Marco Aurelio Gerosa. 2018. Single or multiple conversa- tional agents? An interactional coherence comparison. InProceedings of the 2018 CHI Conference on Human Factors in Computing Systems. 1–13
work page 2018
-
[5]
Longfei Chen, Qianyu Liu, Chenyang Zhang, Yangkun Huang, Zhenhui Peng, Haipeng Zeng, Zhida Sun, Xiaojuan Ma, and Quan Li. 2024. Amplifying the music listening experience through song comments on music streaming platforms. Journal of Visualization27, 3 (2024), 401–419
work page 2024
-
[6]
Yoonseo Choi, Eun Jeong Kang, Seulgi Choi, Min Kyung Lee, and Juho Kim. 2025. Proxona: Supporting Creators’ Sensemaking and Ideation with LLM-Powered Audience Personas. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Association for Computing Machinery, New York, NY, USA, Article 149, 32 pages. doi:10.1145/3706598.3714034
-
[7]
R Maria del Rio-Chanona, Nadzeya Laurentsyeva, and Johannes Wachs. 2024. Large language models reduce public knowledge sharing on online Q&A plat- forms.PNAS Nexus3, 9 (Sept. 2024). doi:10.1093/pnasnexus/pgae400
-
[8]
Elina Eriksson, Henrik Artman, and Anna Swartling. 2013. The secret life of a persona: When the personal becomes private. InProceedings of the SIGCHI conference on human factors in computing systems. 2677–2686
work page 2013
-
[9]
1992.ANOV A: Repeated measures
Ellen R Girden. 1992.ANOV A: Repeated measures. Number 84. sage
work page 1992
-
[10]
Juhye Ha, Hyeon Jeon, Daeun Han, Jinwook Seo, and Changhoon Oh. 2024. CloChat: Understanding How People Customize, Interact, and Experience Per- sonas in Large Language Models. InProceedings of the CHI Conference on Human Factors in Computing Systems. 1–24
work page 2024
- [11]
-
[12]
SG Hart. 1988. Development of NASA-TLX (Task Load Index): Results of empirical and theoretical research.Human mental workload/Elsevier(1988)
work page 1988
-
[13]
Kevin Lane Keller and Richard Staelin. 1987. Effects of quality and quantity of information on decision effectiveness.Journal of consumer research14, 2 (1987), 200–213
work page 1987
-
[14]
Chinmay Kulkarni, Koh Pang Wei, Huy Le, Daniel Chia, Kathryn Papadopoulos, Justin Cheng, Daphne Koller, and Scott R. Klemmer. 2013. Peer and self assessment in massive online classes.ACM Trans. Comput.-Hum. Interact.20, 6, Article 33 (dec 2013), 31 pages. doi:10.1145/2505057
-
[15]
Yuxuan Lei, Tianfu Wang, Jianxun Lian, Zhengyu Hu, Defu Lian, and Xing Xie
- [16]
-
[17]
Brenna Li, Amy Wang, Patricia Strachan, Julie Anne Séguin, Sami Lachgar, Karyn C Schroeder, Mathias S Fleck, Renee Wong, Alan Karthikesalingam, Vivek Natarajan, et al. 2024. Conversational AI in health: Design considerations from a Wizard-of-Oz dermatology case study with users, clinicians and a medical LLM. InExtended Abstracts of the CHI Conference on H...
work page 2024
-
[18]
Chengzhong Liu, Zeyu Huang, Dingdong Liu, Shixu Zhou, Zhenhui Peng, and Xiaojuan Ma. 2022. PlanHelper: Supporting Activity Plan Construction with Answer Posts in Community-Based QA Platforms.Proceedings of the ACM on Human-Computer Interaction6, CSCW2 (2022), 1–26
work page 2022
-
[19]
Chengzhong Liu, Shixu Zhou, Dingdong Liu, Junze Li, Zeyu Huang, and Xiaojuan Ma. 2023. CoArgue: Fostering Lurkers’ Contribution to Collective Arguments in Community-based QA Platforms. InProceedings of the 2023 CHI Conference on Human Factors in Computing Systems. 1–17
work page 2023
-
[20]
Qiaoling Liu, Eugene Agichtein, Gideon Dror, Evgeniy Gabrilovich, Yoelle Maarek, Dan Pelleg, and Idan Szpektor. 2011. Predicting web searcher satisfaction with existing community-based answers. InProceedings of the 34th international ACM SIGIR conference on Research and development in Information Retrieval. 415–424
work page 2011
-
[21]
Tianjian Liu, Hongzheng Zhao, Yuheng Liu, Xingbo Wang, and Zhenhui Peng
-
[22]
ComPeer: A Generative Conversational Agent for Proactive Peer Support. InProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology(Pittsburgh, PA, USA)(UIST ’24). Association for Computing Machinery, New York, NY, USA, Article 117, 22 pages. doi:10.1145/3654777. 3676430
-
[23]
Yiren Liu, Pranav Sharma, Mehul Oswal, Haijun Xia, and Yun Huang. 2025. Per- sonaFlow: Designing LLM-Simulated Expert Perspectives for Enhanced Research Ideation. InProceedings of the 2025 ACM Designing Interactive Systems Conference. ConSearcher: Supporting Conversational Information Seeking in Online Communities with Member Personas DIS ’26, June 13–17,...
work page 2025
-
[24]
Hayat Malik, Sonali Bhandari, Elizabeth L Fonseca, Chiara Bonaccorsi, and David Lee. 2024. Towards integrated learning experiences on social media: An explo- ration of# DayInTheLife videos for career exploration. InProceedings of the 2024 ACM Designing Interactive Systems Conference. 1722–1740
work page 2024
-
[25]
Nicola Marsden, Monika Pröbster, Mirza Ehsanul Haque, and Julia Hermann. 2017. Cognitive styles and personas: designing for users who are different from me. InProceedings of the 29th Australian Conference on Computer-Human Interaction. 452–456
work page 2017
-
[26]
Lene Nielsen and Kira Storgaard Hansen. 2014. Personas is applicable: a study on the use of personas in Denmark. InProceedings of the SIGCHI Conference on Human Factors in Computing Systems. 1665–1674
work page 2014
-
[27]
Zihan Niu, Zheyong Xie, Shaosheng Cao, Chonggang Lu, Zheyu Ye, Tong Xu, Zuozhu Liu, Yan Gao, Jia Chen, Zhe Xu, et al. 2025. Part: Enhancing proactive social chatbots with personalized real-time retrieval. InProceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval. 4269–4274
work page 2025
-
[28]
Jonas Oppenlaender, Elina Kuosmanen, Andrés Lucero, and Simo Hosio. 2021. Hardhats and Bungaloos: Comparing Crowdsourced Design Feedback with Peer Design Feedback in the Classroom. InProceedings of the 2021 CHI Conference on Human Factors in Computing Systems(Yokohama, Japan)(CHI ’21). Association for Computing Machinery, New York, NY, USA, Article 570, 1...
-
[29]
Ilja Ormel, Charles C Onu, Mona Magalhaes, Terence Tang, John B Hughes, Susan Law, et al. 2021. Using a mobile app–based video recommender system of patient narratives to prepare women for breast cancer surgery: development and usability study informed by qualitative data.JMIR formative research5, 6 (2021), e22970
work page 2021
-
[30]
Heather O’Brien. 2016. Theoretical perspectives on user engagement.Why engagement matters: Cross-disciplinary perspectives of user engagement in digital media(2016), 1–26
work page 2016
-
[31]
Srishti Palani, Zijian Ding, Austin Nguyen, Andrew Chuang, Stephen MacNeil, and Steven P Dow. 2021. CoNotate: Suggesting queries based on notes promotes knowledge discovery. InProceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–14
work page 2021
-
[32]
Joon Sung Park, Lindsay Popowski, Carrie Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. 2022. Social simulacra: Creating populated prototypes for social computing systems. InProceedings of the 35th Annual ACM Symposium on User Interface Software and Technology. 1–18
work page 2022
-
[33]
1993.The adaptive decision maker
John W Payne, James R Bettman, and Eric J Johnson. 1993.The adaptive decision maker. Cambridge university press
work page 1993
-
[34]
Zhenhui Peng, Qiaoyi Chen, Zhiyu Shen, Xiaojuan Ma, and Antti Oulasvirta
-
[35]
DesignQuizzer: A Community-Powered Conversational Agent for Learning Visual Design.Proceedings of the ACM on Human-Computer Interaction8, CSCW1 (2024), 1–40
work page 2024
-
[36]
Zhenhui Peng, Qingyu Guo, Ka Wing Tsang, and Xiaojuan Ma. 2020. Exploring the Effects of Technological Writing Assistance for Support Providers in Online Mental Health Community. InProceedings of the 2020 CHI Conference on Human Factors in Computing Systems(Honolulu, HI, USA)(CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–15. doi:10.1...
-
[37]
Zhenhui Peng, Yunhwan Kwon, Jiaan Lu, Ziming Wu, and Xiaojuan Ma. 2019. Design and Evaluation of Service Robot’s Proactivity in Decision-Making Support Process. InProceedings of the 2019 CHI Conference on Human Factors in Computing Systems(Glasgow, Scotland Uk)(CHI ’19). Association for Computing Machinery, New York, NY, USA, 1–13. doi:10.1145/3290605.3300328
-
[38]
Zhenhui Peng, Xiaojuan Ma, Diyi Yang, Ka Wing Tsang, and Qingyu Guo. 2021. Effects of support-seekers’ community knowledge on their expressed satisfaction with the received comments in mental health communities. InProceedings of the 2021 CHI Conference on Human Factors in Computing Systems. 1–12
work page 2021
-
[39]
Filip Radlinski and Nick Craswell. 2017. A theoretical framework for conversa- tional search. InProceedings of the 2017 conference on conference human informa- tion interaction and retrieval. 117–126
work page 2017
-
[40]
Sercan Şengün, Joni Salminen, Soon-Gyo Jung, Kholoud Khalil Aldous, and Bernard J Jansen. 2024. “There’s Something About Noura”: Exploring Think- Aloud Reasonings for Users’ Persona Choice in a Design Task. InProceedings of the 2024 ACM Designing Interactive Systems Conference. 1234–1247
work page 2024
-
[41]
Nikhil Sharma, Q Vera Liao, and Ziang Xiao. 2024. Generative Echo Chamber? Effect of LLM-Powered Search Systems on Diverse Information Seeking. In Proceedings of the CHI Conference on Human Factors in Computing Systems. 1–17
work page 2024
-
[42]
Donghoon Shin, Daniel Lee, Gary Hsieh, and Gromit Yeuk-Yin Chan. 2025. Poster- Mate: Audience-driven Collaborative Persona Agents for Poster Design. InPro- ceedings of the 38th Annual ACM Symposium on User Interface Software and Technology. 1–20
work page 2025
-
[43]
Joongi Shin, Michael A Hedderich, Bartłomiej Jakub Rey, Andrés Lucero, and Antti Oulasvirta. 2024. Understanding human-AI workflows for generating personas. InProceedings of the 2024 ACM Designing Interactive Systems Conference. 757–781
work page 2024
-
[44]
Sangho Suh, Bryan Min, Srishti Palani, and Haijun Xia. 2023. Sensecape: En- abling multilevel exploration and sensemaking with large language models. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology. 1–18
work page 2023
-
[45]
Lev Tankelevitch, Viktor Kewenig, Auste Simkute, Ava Elizabeth Scott, Advait Sarkar, Abigail Sellen, and Sean Rintel. 2024. The metacognitive demands and opportunities of generative AI. InProceedings of the CHI Conference on Human Factors in Computing Systems. 1–24
work page 2024
-
[46]
Thitaree Tanprasert, Sidney S Fels, Luanne Sinnamon, and Dongwook Yoon. 2024. Debate Chatbots to Facilitate Critical Thinking on YouTube: Social Identity and Conversational Style Make A Difference. InProceedings of the CHI Conference on Human Factors in Computing Systems. 1–24
work page 2024
- [47]
-
[48]
Isaac Waller and Ashton Anderson. 2019. Generalists and specialists: Using community embeddings to quantify activity diversity in online platforms. InThe World Wide Web Conference. 1954–1964
work page 2019
-
[49]
Isaac Waller and Ashton Anderson. 2021. Quantifying social organization and political polarization in online platforms.Nature600, 7888 (2021), 264–268
work page 2021
-
[50]
Ben Wang, Jiqun Liu, Jamshed Karimnazarov, and Nicolas Thompson. 2024. Task supportive and personalized human-large language model interaction: A user study. InProceedings of the 2024 Conference on Human Information Interaction and Retrieval. 370–375
work page 2024
-
[51]
1995.Sensemaking in organizations
Karl E Weick and Karl E Weick. 1995.Sensemaking in organizations. Vol. 3. Sage publications Thousand Oaks, CA
work page 1995
-
[52]
Shiwei Wu, Mingxiang Wang, Chuhan Shi, and Zhenhui Peng. 2025. ComViewer: An Interactive Visual Tool to Help Viewers Seek Social Support in Online Mental Health Communities.Proc. ACM Hum.-Comput. Interact.9, 2, Article CSCW006 (May 2025), 31 pages. doi:10.1145/3710904
- [53]
-
[54]
Yitian Yang, Yugin Tan, Yang Chen Lin, Jung-Tai King, Zihan Liu, and Yi- Chieh Lee. 2025. Understanding How Psychological Distance Influences User Preferences in Conversational versus Web Search. InProceedings of the 2025 CHI Conference on Human Factors in Computing Systems (CHI ’25). Associa- tion for Computing Machinery, New York, NY, USA, Article 582, ...
-
[55]
Han Zhang, Jiaqi Zhang, Yuxiang Zhou, Ryan Louie, Taewook Kim, Qingyu Guo, Shuailin Li, and Zhenhui Peng. 2025. MentalImager: Exploring Generative Images for Assisting Support-Seekers’ Self-Disclosure in Online Mental Health Communities.Proc. ACM Hum.-Comput. Interact.9, 2, Article CSCW133 (May 2025), 35 pages. doi:10.1145/3711031
-
[56]
Xiaoyue Zhang and Chang Liu. 2023. Examination of Information Problem De- composition Strategies: A New Perspective for Understanding Users’ Information Problems in Search as Learning. InProceedings of the Annual International ACM SIGIR Conference on Research and Development in Information Retrieval in the Asia Pacific Region. 84–94
work page 2023
-
[57]
Yu Zhang, Jingwei Sun, Li Feng, Cen Yao, Mingming Fan, Liuxin Zhang, Qianying Wang, Xin Geng, and Yong Rui. 2024. See Widely, Think Wisely: Toward Design- ing a Generative Multi-agent System to Burst Filter Bubbles. InProceedings of the CHI Conference on Human Factors in Computing Systems. 1–24
work page 2024
-
[58]
Yuanhao Zhang, Xiaoyu Wang, Jiaxiong Hu, Ziqi Pan, Zhenhui Peng, and Xiao- juan Ma. 2025. Designing AI-Infused Interactive Systems for Online Communities: A Systematic Literature Review.arXiv preprint arXiv:2509.23309(2025)
work page internal anchor Pith review Pith/arXiv arXiv 2025
-
[59]
Chengbo Zheng, Yuanhao Zhang, Zeyu Huang, Chuhan Shi, Minrui Xu, and Xiaojuan Ma. 2024. Disciplink: Unfolding interdisciplinary information seeking process via human-ai co-exploration. InProceedings of the 37th Annual ACM Symposium on User Interface Software and Technology. 1–20
work page 2024
-
[60]
Li Zhou, Jianfeng Gao, Di Li, and Heung-Yeung Shum. 2020. The design and im- plementation of xiaoice, an empathetic social chatbot.Computational Linguistics 46, 1 (2020), 53–93
work page 2020
-
[61]
Haining Zhu, Hongjian Wang, and John M Carroll. 2019. Creating Persona Skeletons from Imbalanced Datasets-A Case Study using US Older Adults’ Health Data. InProceedings of the 2019 on designing interactive systems conference. 61–70. A Questionnaire Items Used in User Study Figure 6 and Figure 7 provide the detailed items and statistical results of user ta...
work page 2019
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.