Pith. sign in

REVIEW 3 major objections 6 minor 85 references

WaLLM -- Insights from an LLM-Powered Chatbot deployment via WhatsApp

T0 review · 3 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A WhatsApp-based LLM chatbot, run for six months in developing-region communities, draws mostly factual and health-related queries, with daily push messages and leaderboard access tied to higher engagement.

desk verdict A useful deployment study whose headline engagement numbers are not supported by the analysis; the descriptive topic findings are the real contribution. read the letter →

arxiv 2505.08894 v1 pith:YQPED27R submitted 2025-05-13 cs.HC cs.AIcs.CY

Hiba Eltigani , Rukhshan Haroon , Asli Kocak , Abdullah Bin Faisal , Noah Martin , Fahad Dogar This is my paper · ORCID
classification cs.HCcs.AIcs.CY
keywords WhatsAppchatbotlargelanguagemodelsdevelopingregionsuserengagementhealthinformationseekinggamificationdigitaldividedeploymentstudy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that a general-purpose LLM chatbot delivered over WhatsApp can serve as a practical information and engagement platform in developing regions, and that deployment logs can reveal what users actually seek. Across six months and roughly 100 users in Pakistan, Sudan, and the US diaspora, it reports that 55% of freely typed queries request factual information, that health and well-being is the leading topic at 28%, and that two-thirds of user activity falls within 24 hours of a daily pushed "Top Question" message. The stakes, if true, are that a familiar chat interface plus low-cost nudges and simple gamification can measurably increase sustained interaction with AI services where app stores and web interfaces are barriers.

What carries the argument

The operating mechanism is a WhatsApp-native chatbot that wraps off-the-shelf LLMs behind plain text plus button-based interactive messages. The features doing the argumentative work are the daily "Top Question of the Day" push, curated "Trending" and "Recent" query lists, AI-generated suggested follow-ups, and a points-and-leaderboard reward system. These features turn a one-shot Q&A tool into a recurring engagement loop, and the timestamps captured by the WhatsApp Business API let the paper attribute activity spikes to specific nudges, such as the sharp rise in sessions within the first hour after the daily push.

What would settle it

A direct test: run the same WhatsApp chatbot for six months with a sample whose in-country location is verified rather than inferred from phone numbers, recruited through community organizations instead of personal networks, and compare the topic mix and the size of the post-push activity spike; if health falls well below 28% or the 24-hour spike disappears, the paper's central generalizations fail.

Watch

Extended reading notes

Core claim

The paper's central claim is that deploying an LLM assistant inside WhatsApp produces a usable picture of what nonexpert users in developing-region communities want from generative AI, and that design choices measurably shape how they engage. The data show factual information-seeking dominates (55% of freeform queries), health and well-being is the most common topic (28%), users treat the chatbot as a trusted source for nutrition and disease questions, daily push messages are followed by a statistically significant doubling of active users, and users who repeatedly accessed the leaderboard interacted with the service about three times as much as those who did not. The paper also reports that roughly 9% of sampled responses had accuracy issues and that about two-thirds of those inaccurate responses were not contested by users.

Load-bearing premise

The load-bearing assumption is that the recruited sample—people who accepted direct invitations or snowball referrals, were fluent in English, and included many US-based diaspora members—behaves like the broader developing-region population the deployment is meant to inform.

Editorial extensions

If this is right

  • If these patterns hold, WhatsApp is a viable channel for general-purpose AI access in low-bandwidth, low-literacy settings, reducing the need to build separate apps.
  • Daily push messages produce measurable engagement, so a cheap, non-intrusive reminder may be enough to sustain use over months.
  • A health-dominated query mix implies that LLM services aimed at developing regions should prioritize accurate, localizable health content rather than assuming use will be mostly entertainment or casual chat.
  • The roughly 3x interaction gap for leaderboard users, even if partly self-selected, suggests that social visibility and gamification can be an effective engagement lever for a subset of users.
  • Because about two-thirds of inaccurate responses go uncontested, trust calibration and hallucination awareness are immediate design responsibilities for any such deployment.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Inference: The 28% health share likely understates health-related use, since advice and nutrition queries classified outside "health" still concern bodily and medical matters; combining topic and intent labels could reveal an even stronger health orientation.
  • Inference: The study cannot fully separate whether the daily push causes activity or arrives when activity would occur anyway; a randomized A/B design on push timing would distinguish a reminder effect from selection.
  • Inference: If leaderboard access is largely a marker of already-active users rather than a driver, gamification features should be evaluated causally before being credited with engagement gains.
  • Inference: The findings suggest a testable extension: adding local-language buttons and retrieval-grounded health answers should increase both factual accuracy and trust, and that can be measured directly in a follow-up deployment.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper describes the design, deployment, and log analysis of WaLLM, a WhatsApp-based LLM chatbot targeted at users in Pakistan, Sudan, and the US diaspora. Over a six-month deployment, 112 registered users generated approximately 14.7K interactions, which the authors analyze with a mixed-methods approach: quantitative analysis of sessions and interactive features, plus qualitative coding of a 10% sample of freeform queries. The paper reports that 55% of user queries seek factual information, that health and well-being is the most popular topic (28%), that two-thirds of user activity occurred within 24 hours of the daily Top Question message, and that users who accessed the leaderboard interacted with the service three times more than those who did not. The authors present design implications about trust, cultural customization, and user interface for generative AI in developing regions.

Significance. The deployment itself is a valuable design probe: it provides rare in-the-wild log data on LLM use over a familiar chat interface in developing-region contexts, and the appendix includes the actual prompts, giving the work a reproducible implementation core. The descriptive findings about query topics, use cases, and accuracy are plausible and potentially useful to ICTD and HCI researchers. However, the headline engagement claims are not supported by the reported analysis: the TopQ statistic is vulnerable to a calendar-time confound, and the leaderboard comparison is based on a post hoc subset rather than a comparison against non-users. The paper's main contribution is therefore best framed as descriptive log-based insights and design lessons, not causal evidence for nudging or gamification effects. If the claims are revised to match the evidence, the paper could make a solid contribution; in its current form, the abstract and central narrative overstate what the data show.

major comments (3)
  1. [§5.2.1 and Abstract] The headline engagement claim—'Two-thirds of users' activity occurred within 24 hours of the daily top question'—is not established by the reported analysis. Because TopQ messages were sent daily for 62 consecutive days (§3.1.4), every day in that two-month window is within 24 hours of a TopQ message; the 62.93% figure therefore measures the fraction of users' active days that fell inside the TopQ period, not an activity response to a nudge. The supporting comparison between days with and without a TopQ is confounded with calendar time and tenure, since non-TopQ days fall outside the TopQ period and are later in the deployment when churn is high. No difference-in-differences, placebo test, or pre/post comparison is provided. The abstract and §1 causal wording should be softened or the analysis redesigned before this finding is presented as evidence that the daily nudge increased engagement.
  2. [§5.2.2, Table 4, and Abstract] The abstract's claim that 'Users who accessed the Leaderboard interacted with WaLLM 3x as those who did not' is not supported by Table 4. Table 4 compares frequent leaderboard users (n=5, selected by repeated access) with occasional users (n=9), not with non-users. The 3:1 ratio in the text refers to frequent users' sessions with Leaderboard access versus their sessions without it (10.8 vs 3.8 interactions per session), a within-user correlation that is subject to self-selection. A direct comparison between users who ever used the leaderboard and matched non-users is missing, so the group-level '3x' claim should be removed or re-derived from a suitable comparison.
  3. [§4.3.2, §5.1.1] The topic and intent percentages, including the headline 55% factual and 28% health figures, rest on a qualitative coding of a 10% sample (360 freeform queries) by the authors without reported inter-rater reliability, a codebook, or a disagreement-resolution procedure. Treating these thematic labels as quantitative population estimates is therefore fragile. Please add reliability statistics (e.g., Cohen's kappa on a subsample), describe the coding process in more detail, and report the percentages as sample-based descriptive statistics rather than as population estimates.
minor comments (6)
  1. [§5.2.1] The reported 'Chi-square Test: U = 165.78' is mislabeled; U conventionally denotes a Mann-Whitney statistic, while a chi-square test should report χ² with degrees of freedom and the contingency table on which it is based.
  2. [§5.2.2] Similarly, the 'Wilcoxon Signed-Rank Test: U = 0.0' should report W (or V) rather than U, and should state the number of pairs on which the signed-rank test is computed.
  3. [§4.3.1] The user categories are defined by session counts, so statements such as 'Regular users have maintained 10 times the number of active days' are partly definitional; these should be presented as descriptive consequences of the category definitions rather than as independent behavioral discoveries.
  4. [§4.1, §7, Abstract] The limitations around self-selected, English-fluent, diaspora-inclusive recruitment are acknowledged in §7 but should be reflected in the abstract and conclusion whenever the topic-mix results are stated for 'users in developing regions,' to avoid overgeneralization.
  5. [§5.1.3] The text says 'approximately 22%' of freeform queries need additional information, but the two stated components are 8% and 13%, which sum to 21%; please reconcile these numbers.
  6. [§4.3.1 and Table 1] The percentages reported in the text (76%, 17%, 7%) do not match Table 1 counts (17, 74, 6 out of 97 users, i.e., approximately 17.5%, 76.3%, 6.2%); please correct the percentages and their ordering.

Circularity Check

2 steps flagged · score 6.0 of 10

TopQ engagement statistic is a calendar artifact of the daily push schedule; leaderboard 'frequent user' comparisons partly restate their own grouping rule.

  1. self definitional [§5.2.1 'Top Question of the Day' (Figure 7 analysis)]
    "A total of 62 push messages were sent, beginning three weeks after the launch of the service and continuing daily for two months. ... On average, 62.93% of the days a user was active fell within the 24 hour period after one of 62 topQ messages were sent."

    Because topQ messages were sent daily for 62 consecutive days, the 24-hour window after a topQ covers essentially the entire two-month topQ period. The 62.93% figure therefore measures the fraction of a user's active days that fell inside the topQ period, not the fraction of activity caused by receiving a topQ. The accompanying comparison (days with topQ had twice as many active users as days without) is identically confounded: all non-topQ days lie outside the topQ window (mostly before it, or after churn), so the statistic is determined by the deployment calendar rather than by user response. The conclusion in §1 that 'this feature has encouraged users to engage more' is thus equivalent to the input schedule by construction.

  2. self definitional [§5.2.2 'Rewards', Table 4]
    "We use repeated access to 'Leaderboard' in one session and viewership in more than 10% of the sessions as an indication of interest. Accordingly, we categorize the users into occasional users and frequent users. ... frequent users have on average 15 times the interactions of occasional users."

    The 'frequent' group is selected by repeated leaderboard access and leaderboard views in more than 10% of sessions. Since leaderboard access is itself an interaction and the selection rule targets users for whom leaderboard use is a substantial share of sessions, high session and interaction totals are partly baked into the grouping. The finding that frequent users have 15x the interactions of occasional users is therefore in part a restatement of the selection rule rather than an independent effect. The abstract's broader 3x leaderboard-vs-non-leaderboard comparison is also a self-selection contrast with no control for the baseline activity that predicts leaderboard use.

full rationale

The paper is primarily a descriptive deployment study: the topic distribution (health 28%), factual-intent share (55%), and response-accuracy analysis are derived from the log data through standard qualitative coding and manual verification, and these are not circular. The circularity is concentrated in the engagement-feature claims. The headline 'two-thirds of activity within 24h of topQ' reduces by construction to the fraction of active days falling inside the daily-push period, because topQ was sent every day for two months; no control period, placebo, or difference-in-differences is provided. Similarly, the frequent-vs-occasional leaderboard comparison defines the predictor (repeated leaderboard access in >10% of sessions) using the same interaction stream that is then reported as the outcome, so the 15x interaction gap is partly an artifact of the categorization rule. The self-citations in the paper (e.g., TwIPS and prior misinformation work) are not load-bearing for these claims. Overall, several central engagement conclusions are partially constructed by definition, though the descriptive usage findings remain independently meaningful; hence a score of 6 rather than higher.

Assumptions & free parameters 5 free parameters · 3 assumptions · 0 invented entities

The central findings rest on analytic choices (session threshold, group cutoffs, exclusion rules, sampling fraction) and on assumptions about population representativeness and coding reliability. There are no invented entities or fitted physical parameters; the LLM outputs and user behaviors are the measured data.

free parameters (5)
  • Session timeout threshold = 15 minutes
    Interactions separated by less than 15 minutes are grouped into a session; chosen from prior work (Guy 2016; Trippas et al. 2024), but no sensitivity analysis is provided.
  • User activity categories = 1 session (one-time), 1-100 sessions (casual), >100 sessions (regular)
    Heuristic thresholds based on observed usage distribution; they determine all group-level comparisons and hence the 'regular users contribute 80%' finding.
  • TopQ analysis exclusion criteria = Exclude registrations after topQ started and registration sessions
    Post hoc exclusion leaves 47 users; the resulting 62.93% active-days-within-24h metric is the basis for the abstract's 'two-thirds' claim.
  • Leaderboard frequent-user threshold = Repeated access in >10% of sessions
    Used to split leaderboard users into occasional vs frequent; the frequent group's higher interaction counts drive the 3x claim in the abstract.
  • Qualitative sample size = 10% of freeform queries
    Stratified sample used for topic, intent, and accuracy analysis; no confidence intervals or inter-rater reliability reported.
assumptions (3)
  • domain assumption Diaspora users can serve as surrogates for users in home countries
    Invoked in §4.1 to justify including US-based diaspora in the developing-regions sample; cited to Best et al. but not validated here.
  • domain assumption The 15-minute session definition from prior work transfers to WhatsApp chatbot interactions
    Used throughout §4.3.1 and §5 to define sessions and user groups; no validation for WhatsApp context.
  • domain assumption Manual labeling of query intent, topic, and response accuracy is reliable
    §4.3.2 and §5.1 report single-coder qualitative analysis without inter-rater reliability, so category percentages may be unstable.

how reviews work

0 comments
Cite this review

Pith. "Pith review of WaLLM -- Insights from an LLM-Powered Chatbot deployment via WhatsApp." pith.science (2026). https://pith.science/paper/YQPED27R

@misc{pith2026250508894,
  author       = {Pith},
  title        = {Pith review of: WaLLM -- Insights from an LLM-Powered Chatbot deployment via WhatsApp},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/YQPED27R}},
  note         = {Machine review of arXiv:2505.08894}
}
read the original abstract

Recent advances in generative AI, such as ChatGPT, have transformed access to information in education, knowledge-seeking, and everyday decision-making. However, in many developing regions, access remains a challenge due to the persistent digital divide. To help bridge this gap, we developed WaLLM - a custom AI chatbot over WhatsApp, a widely used communication platform in developing regions. Beyond answering queries, WaLLM offers several features to enhance user engagement: a daily top question, suggested follow-up questions, trending and recent queries, and a leaderboard-based reward system. Our service has been operational for over 6 months, amassing over 14.7K queries from approximately 100 users. In this paper, we present WaLLM's design and a systematic analysis of logs to understand user interactions. Our results show that 55% of user queries seek factual information. "Health and well-being" was the most popular topic (28%), including queries about nutrition and disease, suggesting users view WaLLM as a reliable source. Two-thirds of users' activity occurred within 24 hours of the daily top question. Users who accessed the "Leaderboard" interacted with WaLLM 3x as those who did not. We conclude by discussing implications for culture-based customization, user interface design, and appropriate calibration of users' trust in AI systems for developing regions.

Figures

Figures reproduced from arXiv: 2505.08894 by the authors.

Figure 1
Figure 1. Example of a user interaction with the WhatsApp chatbot. The user asks a question and the chatbot responds with an answer [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Workflow of “Suggest Follow-ups” messages [PITH_FULL_IMAGE:figures/full_fig_p005_2.png] view at source ↗
Figure 3
Figure 3. Examples of the rewards service views. The leaderboard highlights active users on a daily and long-term basis. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Example of “Top Question of the Day” message [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: WaLLM System Architecture in the “Suggest Follow-ups”, “Trending” and “Recent” lists are generated in advance, or pre-fetched, to minimize latency and enhance usability. 3.2 Implementation WaLLM was hosted on Amazon Web Services (AWS) [35] and integrated with a cloud-b…
Figure 6
Figure 6. Figure 6: Details of user activity and usage patterns [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Users activity during topQ message events [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Examples of different interactive messages [PITH_FULL_IMAGE:figures/full_fig_p028_8.png]
Figure 9
Figure 9. Figure 9: Examples of different queries lists messages [PITH_FULL_IMAGE:figures/full_fig_p028_9.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

85 extracted references · 49 canonical work pages

  1. [1]

    2025. Meta AI. https://www.meta.ai/

    2025

  2. [3]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023)

    arXiv 2023

  3. [4]

    Saleh Afroogh, Ali Akbari, Emmie Malone, Mohammadali Kargar, and Hananeh Alambeigi. 2024. Trust in AI: progress, challenges, and future directions. Humanities and Social Sciences Communications 11, 1 (Nov. 2024). doi:10.1057/s41599-024-04044-8

    doi:10.1057/s41599-024-04044-8 2024

  4. [5]

    Ayesha Ali, Agha Ali Raza, and Ihsan Ayyub Qazi. 2023. Validated digital literacy measures for populations with low levels of internet experiences. Development Engineering 8 (Jan. 2023), 100107. doi:10.1016/j.deveng.2023.100107

    arXiv 2023

  5. [6]

    Oghenemaro Anuyah, Ruyuan Wan, Cornelius Adejoro, Tom Yeh, Ronald Metoyer, and Karla Badillo-Urquiola. 2023. Cultural Considerations in AI Systems for the Global South: A Systematic Review. In Proceedings of the 4th African Human Computer Interaction Conference . 125–134

    2023

  6. [7]

    Michael L Best, Thomas N Smyth, Daniel Serrano-Baquero, and John Etherton. 2009. Designing for and with diaspora: A case study of work for the truth and reconciliation commission of Liberia. In CHI’09 Extended Abstracts on Human Factors in Computing Systems . 2903–2918. WaLLM - Insights from an LLM-Powered Chatbot deployment via WhatsApp 21

    2009

  7. [8]

    Sébastien Bubeck, Varun Chandrasekaran, Ronen Eldan, Johannes Gehrke, Eric Horvitz, Ece Kamar, Peter Lee, Yin Tat Lee, Yuanzhi Li, Scott Lundberg, Harsha Nori, Hamid Palangi, Marco Tulio Ribeiro, and Yi Zhang. 2023. Sparks of Artificial General Intelligence: Early experiments with GPT-4. arXiv:2303.12712 [cs.AI] https://arxiv.org/abs/2303.12712 arXiv preprint

    arXiv 2023

  8. [9]

    Wanling Cai and Li Chen. 2020. Predicting user intents and satisfaction with dialogue-based conversational recommendations. In Proceedings of the 28th ACM Conference on User Modeling, Adaptation and Personalization . 33–42

    2020

Show all 85 references
  1. [10]

    B Barla Cambazoglu, Leila Tavakoli, Falk Scholer, Mark Sanderson, and Bruce Croft. 2021. An intent taxonomy for questions asked in web search. In Proceedings of the 2021 Conference on Human Information Interaction and Retrieval . 85–94

    2021

  2. [11]

    Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. 2024. A survey on evaluation of large language models. ACM Transactions on Intelligent Systems and Technology 15, 3 (2024), 1–45

    2024

  3. [12]

    Ana Paula Chaves and Marco Aurelio Gerosa. 2021. How should my chatbot interact? A survey on social characteristics in human–chatbot interaction design. International Journal of Human–Computer Interaction 37, 8 (2021), 729–758

    2021

  4. [13]

    Simran Chopra and Shruthi Chivukula. 2017. My phone assistant should know I am an Indian: influencing factors for adoption of assistive agents. In Proceedings of the 19th International Conference on Human-Computer Interaction with Mobile Devices and Services . 1–8

    2017

  5. [14]

    De Angelis, F

    L. De Angelis, F. Baglivo, G. Arzilli, G. P. Privitera, P. Ferragina, A. E. Tozzi, and C. Rizzo. 2023. ChatGPT and the rise of large language models: the new AI-driven infodemic threat in public health. Frontiers in Public Health 11 (April 2023), 1166120. doi:10.3389/fpubh.202...

    2023

  6. [15]

    Joost CF de Winter. 2023. Can ChatGPT pass high school exams on English language comprehension? International Journal of Artificial Intelligence in Education (2023), 1–16

    2023

  7. [16]

    Mingqi Gao, Jie Ruan, Renliang Sun, Xunjian Yin, Shiping Yang, and Xiaojun Wan. 2023. Human-like summarization evaluation with chatgpt. arXiv preprint arXiv:2304.02554 (2023)

    2023 arXiv

  8. [17]

    Meghna Gupta, Devansh Mehta, Anandita Punj, and Indrani Medhi Thies. 2022. Sophistication with limitation: Understanding smartphone usage by emergent users in india. In Proceedings of the 5th ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies . 386–400

    2022

  9. [18]

    Ido Guy. 2016. Searching by talking: Analysis of voice queries on mobile web search. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval . 35–44

    2016

  10. [19]

    Rukhshan Haroon and Fahad Dogar. 2024. TwIPS: A Large Language Model Powered Texting Application to Simplify Conversational Nuances for Autistic Users. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibility (St. John’s, NL, Canada) (AS...

    2024

  11. [20]

    Hedderich, Natalie N

    Michael A. Hedderich, Natalie N. Bazarova, Wenting Zou, Ryun Shim, Xinda Ma, and Qian Yang. 2024. A Piece of Theatre: Investigating How Teachers Design LLM Chatbots to Assist Adolescent Cyberbullying Education. In Proceedings of the 2024 CHI Conference on Human Factors in Comp...

    2024

  12. [21]

    International Telecommunication Union (ITU). 2024. Measuring digital development: Facts and Figures 2024 . Technical Report. International Telecommunication Union, Geneva, Switzerland. Accessed: January 29, 2025

    2024

  13. [22]

    Mohit Jain, Pratyush Kumar, Ishita Bhansali, Q Vera Liao, Khai Truong, and Shwetak Patel. 2018. FarmChat: a conversational agent to answer farmer queries. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 2, 4 (2018), 1–22

    2018

  14. [23]

    Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. Survey of hallucination in natural language generation. Comput. Surveys 55, 12 (2023), 1–38

    2023

  15. [24]

    That courage to encourage

    Naveena Karusala, David Odhiambo Seeh, Cyrus Mugo, Brandon Guthrie, Megan A Moreno, Grace John-Stewart, Irene Inwani, Richard Anderson, and Keshet Ronen. 2021. “That courage to encourage”: Participation and Aspirations in Chat-based Peer Support for Youth Living with HIV. In P...

    2021

  16. [25]

    Majeed Kazemitabaar, Runlong Ye, Xiaoning Wang, Austin Zachary Henley, Paul Denny, Michelle Craig, and Tovi Grossman. 2024. CodeAid: Evaluating a Classroom Deployment of an LLM-based Programming Assistant that Balances Student and Educator Needs. In Proceedings of the 2024 CHI...

    2024

  17. [26]

    Jan Kocoń, Igor Cichecki, Oliwier Kaszyca, Mateusz Kochanek, Dominika Szydło, Joanna Baran, Julita Bielaniewicz, Marcin Gruza, Arkadiusz Janz, Kamil Kanclerz, et al. 2023. ChatGPT: Jack of all trades, master of none. Information Fusion 99 (2023), 101861

    2023

  18. [27]

    Nikola Kovačević, Christian Holz, Markus Gross, and Rafael Wampfler. 2024. The personality dimensions gpt-3 expresses during human-chatbot interactions. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 8, 2 (2024), 1–36

    2024

  19. [28]

    Daniel Lambton-Howard, Robert Anderson, Kyle Montague, Andrew Garbett, Shaun Hazeldine, Carlos Alvarez, John A Sweeney, Patrick Olivier, Ahmed Kharrufa, and Tom Nappey. 2019. Whatfutures: Designing large-scale engagements on Whatsapp. In Proceedings of the 2019 CHI Conference ...

    2019

  20. [29]

    Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2021. Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks. arXiv preprint ...

    2021 arXiv

  21. [30]

    Akhil Mathur, Lakshmi Manasa Kalanadhabhatta, Rahul Majethia, and Fahim Kawsar. 2017. Moving beyond market research: Demystifying smartphone user behavior in india. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 1–27. 22 Hiba E...

    2017

  22. [31]

    Federico Milana, Enrico Costanza, and Joel E Fischer. 2023. Chatbots as advisers: The effects of response variability and reply suggestion buttons. In Proceedings of the 5th International Conference on Conversational User Interfaces . 1–10

    2023

  23. [32]

    Wen Mo, Aneesha Singh, and Catherine Holloway. 2024. From Information Seeking to Empowerment: Using Large Language Model Chatbot in Supporting Wheelchair Life in Low Resource Settings. In Proceedings of the 26th International ACM SIGACCESS Conference on Computers and Accessibi...

    2024

  24. [33]

    Muhammad Muzammil, Abisheka Pitumpe, Xigao Li, Amir Rahmati, and Nick Nikiforakis. 2025. The Poorest Man in Babylon: A Longitudinal Study of Cryptocurrency Investment Scams. In Proceedings of The Web Conference (WWW)

    2025

  25. [34]

    n.d. 2024. 1-800-ChatGPT - Calling and Messaging ChatGPT with your phone. https://help.openai.com/en/articles/10193193-1-800-chatgpt-calling- and-messaging-chatgpt-with-your-phone/

    2024

  26. [35]

    n.d. 2024. Cloud Computing Services - Amazon Web Services (AWS). https://aws.amazon.com

    2024

  27. [36]

    n.d. 2024. Meet Claude Anthropic. https://www.anthropic.com/claude

    2024

  28. [37]

    n.d. 2024. Meta AI is Now Multilingual, More Creative and Smarter. https://about.fb.com/news/2024/07/meta-ai-is-now-multilingual-more-creative- and-smarter/

    2024

  29. [38]

    n.d. 2025. Get Started. https://faq.whatsapp.com/497209988909970/?helpref=popular_topics. Accessed: January 29, 2025

    2025

  30. [39]

    n.d. 2025. Get Started. https://business.whatsapp.com/products/business-platform. Accessed: January 29, 2025

    2025

  31. [40]

    Nguyen, Anna Sidorova, and Russell Torres

    Quynh N. Nguyen, Anna Sidorova, and Russell Torres. 2021. User interactions with chatbot interfaces vs. Menu-based interfaces: An empirical study. Computers in Human Behavior 128 (Nov. 2021), 107093. doi:10.1016/j.chb.2021.107093

    2021

  32. [41]

    Udit Paul, Jiamo Liu, David Farias-llerenas, Vivek Adarsh, Arpit Gupta, and Elizabeth Belding. 2022. Characterizing Internet Access and Quality Inequities in California M-Lab Measurements. In Proceedings of the 5th ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Soci...

    2022

  33. [42]

    Perplexity AI. 2024. Introducing the PPLX API. https://www.perplexity.ai/hub/blog/introducing-pplx-api Accessed: 2025-05-01

    2024

  34. [43]

    Anthony Poon, Sarah Giroux, Parfait Eloundou-Enyegue, François Guimbretiere, and Nicola Dell. 2019. Engaging High School Students in Cameroon with Exam Practice Quizzes via SMS and WhatsApp. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (Glasg...

    2019

  35. [44]

    Vishnu Prasad, Richard Shallam, Alok Sharma, Delvin Varghese, and Devansh Mehta. 2021. A Hybrid Multi-Modal System for Conducting Virtual Workshops Using Interactive Voice Response and the WhatsApp Business API. In Extended Abstracts of the 2021 CHI Conference on Human Factors...

    2021

  36. [45]

    Pragnya Ramjee, Mehak Chhokar, Bhuvan Sachdeva, Mahendra Meena, Hamid Abdullah, Aditya Vashistha, Ruchit Nagar, and Mohit Jain. 2024. ASHABot: An LLM-Powered Chatbot to Support the Informational Needs of Community Health Workers. arXiv preprint arXiv:2409.10913 (2024)

    2024 arXiv

  37. [46]

    Amon Rapp, Lorenzo Curti, and Arianna Boldi. 2021. The human side of human-chatbot interaction: A systematic literature review of ten years of research on text-based chatbots. International Journal of Human-Computer Studies 151 (2021), 102630

    2021

  38. [47]

    Corbin Rosset, Chenyan Xiong, Xia Song, Daniel Campos, Nick Craswell, Saurabh Tiwary, and Paul Bennett. 2020. Leading conversational search by suggesting useful questions. In Proceedings of the web conference 2020 . 1160–1170

    2020

  39. [48]

    Priya Fatima Sajjad, Rukhshan Haroon, Ayesha Naeem, Uswah-E-Fatima, and Zartash Afzal Uzmi. 2022. Unpacking Misinformation Amid the COVID-19 Pandemic: A Mixed Methods Study. IEEE Internet Computing 26, 2 (2022), 7–18. doi:10.1109/MIC.2022.3154318

    2022

  40. [49]

    Rijurekha Sen, Sohaib Ahmad, Amreesh Phokeer, Zaid Ahmed Farooq, Ihsan Ayyub Qazi, David Choffnes, and Krishna P Gummadi. 2017. Inside the walled garden: Deconstructing facebook’s free basics program. ACM SIGCOMM Computer Communication Review 47, 5 (2017), 12–24

    2017

  41. [50]

    Vidya Setlur and Melanie Tory. 2022. How do you converse with an analytical chatbot? revisiting gricean maxims for designing analytical conversational behavior. In Proceedings of the 2022 CHI conference on human factors in computing systems . 1–17

    2022

  42. [52]

    Arthur Tang, Neo Tung, Huy Quang Nguyen, Kin On Kwok, Stanley Luong, Nhat Bui, Giang Nguyen, and Wilson Tam. 2024. Health information for all: do large language models bridge or widen the digital divide? BMJ (Oct. 2024), e080208. doi:10.1136/bmj-2024-080208

    2024 doi

  43. [53]

    David R Thomas. 2003. A general inductive approach for qualitative data analysis. (2003)

    2003

  44. [54]

    Johanne R Trippas, Sara Fahad Dawood Al Lawati, Joel Mackenzie, and Luke Gallagher. 2024. What do users really ask large language models? an initial log analysis of google bard interactions in the wild. InProceedings of the 47th International ACM SIGIR Conference on Research a...

    2024

  45. [55]

    Mark Twain. 2008. Eve’s Diary, Complete. Standard Publications, Incorporated

    2008

  46. [56]

    Rama Adithya Varanasi, Aditya Vashistha, and Nicola Dell. 2021. Tag a teacher: A qualitative analysis of WhatsApp-based teacher networks in low-income Indian schools. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems . 1–16

    2021

  47. [57]

    Rama Adithya Varanasi, Aditya Vashistha, and Nicola Dell. 2021. Tag a Teacher: A Qualitative Analysis of WhatsApp-Based Teacher Networks in Low-Income Indian Schools. In Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems (Yokohama, Japan) (CHI ’21). A...

    2021

  48. [58]

    Aditya Vashistha, Richard Anderson, and Shrirang Mare. 2019. Examining the use and non-use of mobile payment systems for merchant payments in India. In Proceedings of the 2nd ACM SIGCAS Conference on Computing and Sustainable Societies . 1–12

    2019

  49. [59]

    Gomez, Łukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Proceedings of the 31st International Conference on Neural Information Processing Systems (Long Beach, California, ...

    2017

  50. [60]

    María Guadalupe Veytia-Bucheli, José Gómez-Galán, and Diego Vergara. 2020. Presence of new forms of intercultural communication in higher education: Emojis and social interactions through whatsapp among graduate students. Education Sciences 10, 11 (2020), 295

    2020

  51. [61]

    Jiayin Wang, Weizhi Ma, Peijie Sun, Min Zhang, and Jian-Yun Nie. 2024. Understanding User Experience in Large Language Model Interactions. arXiv preprint arXiv:2401.08329 (2024)

    2024 arXiv

  52. [62]

    Chi, Quoc V

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. Chain-of-thought prompting elicits reasoning in large language models. In Proceedings of the 36th International Conference on Neural Information Processi...

    2022

  53. [63]

    As an AI language model, I cannot

    Joel Wester, Tim Schrills, Henning Pohl, and Niels van Berkel. 2024. “As an AI language model, I cannot”: Investigating LLM Denials of User Requests. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’24). Association for ...

    2024

  54. [64]

    Should I visit the clinic

    Deepika Yadav, Kirti Dabas, Prerna Malik, Anushka Bhandari, and Pushpendra Singh. 2022. “Should I visit the clinic”: Analyzing WhatsApp- mediated Online Health Support for Expectant and New Mothers in Rural India. In Proceedings of the 2022 CHI Conference on Human Factors in C...

    2022

  55. [65]

    Nur Yildirim, Hannah Richardson, Maria Teodora Wetscherek, Junaid Bajwa, Joseph Jacob, Mark Ames Pinnock, Stephen Harris, Daniel Coelho De Castro, Shruthi Bannur, Stephanie Hyland, Pratik Ghosh, Mercy Ranjit, Kenza Bouzid, Anton Schwaighofer, Fernando Pérez-García, Harshita Sh...

    2024

  56. [66]

    Imama Zahoor, Shiza Ihtsham, Muhammad Umar Ramzan, Agha Ali Raza, and Basmaa Ali. 2024. AI-Driven Healthcare Delivery in Pakistan: A Framework for Systemic Improvement. In Proceedings of the 7th ACM SIGCAS/SIGCHI Conference on Computing and Sustainable Societies (New Delhi, In...

    2024

  57. [67]

    Denny Zhou, Nathanael Scharli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Olivier Bousquet, Quoc Le, and Ed H. Chi

  58. [72]

    User Prompt: {user-query} A.2 Prompt for generating a more detailed answer to a user’s query using a high-quality model

    Your responses should be visually appealing for WhatsApp users, so use emojis, short paragraphs, etc. User Prompt: {user-query} A.2 Prompt for generating a more detailed answer to a user’s query using a high-quality model. System Prompt: Instructions You are an advanced AI ass...

  59. [73]

    You can use the context provided but only to get the information needed to address the user’s current message

  60. [74]

    of the messages in the context

    No need to follow the length of responses, verbosity etc. of the messages in the context

  61. [76]

    User Prompt: {user-query} WaLLM - Insights from an LLM-Powered Chatbot deployment via WhatsApp 25 A.3 Prompt for generating follow-up questions based on a user’s query

    Your responses should be visually appealing for WhatsApp users, so use emojis, short paragraphs, etc. User Prompt: {user-query} WaLLM - Insights from an LLM-Powered Chatbot deployment via WhatsApp 25 A.3 Prompt for generating follow-up questions based on a user’s query. System...

  62. [77]

    Each question statement should be visually appealing for WhatsApp users, so you must use emojis in the question statement

  63. [78]

    q1": "question statement

    Strictly respond in the following JSON format: {"q1": "question statement", "q2": "question statement", "q3": "question statement", "q4": "question statement", "q5": "question statement", "q6": "question statement"} User Prompt: {user-query} A.4 Prompt for deciding whether or ...

  64. [79]

    Return None if the statement is not a question statement

  65. [80]

    Return None if the question statement is in any language other than English

  66. [81]

    that is not explicitly defined within the question statement

    Return None if the question statement refers to any object/idea/thing/text etc. that is not explicitly defined within the question statement. 26 Hiba Eltigani, Rukhshan Haroon, Asli Kocak, Abdullah Bin Faisal, Noah Martin, and Fahad Dogar

  67. [82]

    Convert it to a SINGLE question statement

  68. [83]

    Fix any typos in it and make it less than 125 words

  69. [84]

    ONLY return None or the rephrased question statement

    Add a relevant emoji to it to make it visually appealing. ONLY return None or the rephrased question statement. A.5 Prompt for rating a question to decide whether or not to include it in Trending Questions list. System Prompt: Instructions You are an advanced AI data analyst d...

  70. [85]

    Fix any typos or punctuation errors

  71. [86]

    Add one emoji to the question statement to make it visually appealing

  72. [87]

    Do NOT answer the question and do NOT add unnecessary details to the question statement

  73. [88]

    Strictly return ONLY the rephrased question

    ONLY if the question is longer than 150 words, shorten it. Strictly return ONLY the rephrased question. A.7 Prompt for generating answers to follow-up, trending and recent questions. System Prompt: Instructions You are an advanced AI assistant designed to provide informative a...

  74. [89]

    For example, factual queries can be answered in a few sentences

    If the query can be answered concisely, do not be verbose. For example, factual queries can be answered in a few sentences

  75. [90]

    Engage in a conversational manner and use humor if applicable

  76. [91]

    Continue Reading

    Your responses should be visually appealing for WhatsApp users, so use emojis, short paragraphs, etc. User Prompt: {user-query} 28 Hiba Eltigani, Rukhshan Haroon, Asli Kocak, Abdullah Bin Faisal, Noah Martin, and Fahad Dogar B Examples of WaLLM Message Types (a) Menu Message (...

  77. [2022]

    ArXiv abs/2205.10625 (2022)

    Least-to-Most Prompting Enables Complex Reasoning in Large Language Models. ArXiv abs/2205.10625 (2022). https://api.semanticscholar. org/CorpusID:248986239 24 Hiba Eltigani, Rukhshan Haroon, Asli Kocak, Abdullah Bin Faisal, Noah Martin, and Fahad Dogar A Prompts Each subsecti...

    2022 arXiv

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.