Prototypical Human-AI Collaboration Behaviors from LLM-Assisted Writing in the Wild

Bahareh Sarrafzadeh; Debarati Das; Hancheng Cao; Sheshera Mysore

REVIEW 4 major objections 6 minor 4 cited by

Prototypical Human-AI Collaboration Behaviors from LLM-Assisted Writing in the Wild

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A compact set of seven collaboration behaviors explains most of how users steer LLM-assisted writing sessions.

desk verdict A solid first map of how users steer LLM writing sessions in the wild, with a transparent pipeline that deserves refereeing despite a real shared-labeler weakness. read the letter →

arxiv 2505.16023 v3 pith:AB6MXKIA submitted 2025-05-21 cs.CL cs.HC

Sheshera Mysore , Debarati Das , Hancheng Cao , Bahareh Sarrafzadeh This is my paper

classification cs.CLcs.HC

keywords human-AIcollaborationLLM-assistedwritingin-the-wildloganalysisfollow-upbehaviorsprototypicalprincipalcomponentintentsLLMalignment

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the way people steer an LLM after their first request is not idiosyncratic: follow-up messages in writing sessions cluster into a small set of prototypical behaviors, which the authors call PATHs, and seven of these explain 80-85% of the variation across millions of real sessions. The claim matters because most prior log analyses stop at what task users are doing or whether they are satisfied, whereas this work characterizes the interaction itself. Using two assistants with different models, interfaces, and user bases, Bing Copilot and WildChat, the paper finds the same recurring behaviors: revising the request, asking questions, requesting more outputs, changing style, and adding content. It then shows that a user's writing goal predicts which behavior they use, for example brainstorming catchy text leads to requesting more outputs while drafting professional documents leads to asking questions. If correct, this gives a data-driven map of collaboration that points toward aligning LLMs from natural, session-level feedback rather than single-turn explicit preferences.

What carries the argument

The central object is the PATH, Prototypical Human-AI Collaboration Behavior, a cluster of sessions defined by co-occurring follow-up types. The method works in three steps: a GPT-4o classifier labels each user utterance as an original request or one of eleven follow-up types, such as restates request, elaborates request, requests answers, requests more outputs, changes style, adds content, and removes content; each session becomes a tf-idf vector over these follow-up types, with the tf-idf weighting preventing the most common behaviors from dominating; and PCA on the session-by-type matrix yields principal components, each interpreted as a PATH, retaining the components that jointly explain 80-85% of variance. The same classifier pipeline labels original requests with one of eighteen writing intents, and logistic regressions with intents as predictors and PATH membership as the target turn the components into testable correlations. The PCA is doing the load-bearing work of converting individual utterance labels into a small set of repeatable collaboration styles.

What would settle it

Manually annotate a fresh random sample of roughly 2,000 sessions per dataset with follow-up types and writing intents, retrain or correct the classifiers, rerun the PCA and the intent-PATH regressions, and check whether seven components still explain 80-85% of variance and whether the key correlations, catchy text with requesting more outputs and professional documents with asking questions, reappear; if the structure shifts or dissolves, the discovered PATHs are artifacts of the labeling model.

Watch

Extended reading notes

Core claim

The central discovery is that follow-up utterances in LLM-assisted writing form a compact behavioral structure rather than noise. After classifying every follow-up into one of eleven high-level types with a GPT-4o labeler and applying PCA to the per-session type frequencies, seven principal components, the PATHs, capture 80-85% of the variance in both datasets, with the same dominant behaviors, restating the request and requesting more outputs, leading in each. Explicit satisfaction signals are rare, appearing in only 1-5% of sessions, so the PATHs are built from the implicit steering users actually do. The paper further claims statistically significant, cross-deployment correlations between writing intents and specific PATHs: generating eye-catching text correlates with requesting more outputs for brainstorming, drafting professional documents, summaries, and technical text correlates with asking follow-up questions to learn domain norms, and adding content appears when generations lack information known only to the user, such as personal experience in resumes or messages. These correlation patterns appear in both Bing Copilot and WildChat logs, which the paper takes as evidence that the behaviors reflect how users collaborate with LLMs rather than how one particular system is built.

Load-bearing premise

Everything downstream depends on GPT-4o classifiers correctly labeling every session's task, every follow-up type, and every writing intent, but those classifiers were validated on only 110 or 220 manually annotated sessions with accuracies between 74% and 89%, so systematic labeler errors could produce the PATHs and intent correlations rather than real user behavior.

Editorial extensions

If this is right

Explicit positive or negative feedback is rare, so aligning LLMs to implicit, natural feedback across multi-turn sessions becomes a tractable and necessary target.
Because intent predicts behavior, alignment can be intent-specific: a model serving a brainstorming request should expect 'give me more' follow-ups, while a professional-drafting session should expect knowledge and norm questions.
Staged generation of long creative texts with follow-ups that grow more specific suggests models should handle under-specified feedback that sharpens over turns.
Content-addition behavior shows professional and communicative drafts commonly lack user-only knowledge, pointing to personalization or proactive question-asking as the fix.
The same PATH structure appearing in two different deployments implies the behaviors are shared styles of human-AI collaboration, not artifacts of one interface.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension left implicit in the paper: because PATHs come from co-occurrence counts, they ignore turn order, and a sequence-aware model of follow-ups might split the staged long-generation behavior into finer prototypes or reveal that order itself is a collaboration signal.
The intent-PATH correlations suggest a real-time adaptation loop the paper never states: an assistant that recognizes a brainstorming session could proactively supply more alternatives, and one that recognizes professional drafting could volunteer domain norms before being asked.
The same pipeline could be run on non-writing logs, such as coding, image generation, or theorem proving, to see whether the same skeleton behaviors, restate, ask, request-more, add, remove, restyle, reappear, which would make PATHs a general taxonomy of human-AI steering.
Because the classifier accuracies range from 74% to 89%, the cleanest confirmation would be a small lab study where users self-report their intent; if self-reported intents correlate with PATHs more strongly than classifier-derived intents do, the reported effect sizes are lower bounds in the wild.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The headline is simple: this is the first large-scale, in-the-wild look at how users actually follow up on LLM writing requests, and the core descriptive findings are believable. The PATH construct—seven principal components that account for 80-85% of the variance in follow-up types—is a useful way to organize behavior, and the specific correlations (e.g., 'generate catchy text' → 'request more outputs', 'professional documents' → 'ask questions/add content') make intuitive sense and are backed by example conversations. It does what it claims: goes beyond task classification and satisfaction to characterize multi-turn co-construction.

The strengths are real. The two datasets are large, the qualitative spot-checks are genuinely informative, and the paper is honest about its limitations—the appendix contains the validation numbers, the label definitions, and the prompts. That level of transparency is exactly what an empirical HCI/NLP paper should look like.

Now the soft spots. The biggest one is also the one the stress-test flags: a single model family (GPT-4o) labels both the writing intents (predictors) and the follow-up types (outcomes). If GPT-4o systematically associates certain phrasings with certain labels—e.g., 'give me more' with 'REQUESTS MORE OUTPUTS' and 'catchy/name/slogan' with 'GENERATE CATCHY TEXT'—then the same correlation appears in both datasets because the labeler is shared, not because user behavior is shared. The validation in Appendix C.3 (roughly 110 sessions per classifier) is small and can't detect that kind of systematic bias. I don't think this is fatal—the qualitative examples make the correlations look real and the paper doesn't overclaim causality—but it is a load-bearing weakness. The paper should report effect sizes, correct for multiple comparisons, test threshold robustness for the PCA binarization, and ideally re-label a sample with a different model or human annotators to check for annotator-driven artifacts.

A smaller issue: the proprietary Bing Copilot data limits full reproducibility, although the WildChat pipeline is public and that part is a step in the right direction. The paper also does not release code for the WildChat analysis, which would help.

Overall verdict: this is a solid, important-for-the-subfield paper that deserves a serious referee. It is not a foundational result yet—the label-bias concern needs to be addressed before I'd treat the intent-PATH correlations as the basis for alignment research—but it is a good contribution that will generate discussion and follow-up work. If I were the editor, I would send it to review with the expectation of a major revision on the statistical rigor.

Referee Report

4 major / 6 minor

Summary. The paper analyzes two large in-the-wild log corpora of LLM-assisted writing sessions, one from Bing Copilot and one from WildChat. Using GPT-4o classifiers, it labels each user utterance as an original request or one of eleven follow-up types, and labels original requests with fine-grained writing intents. It then applies PCA to the tf-idf representation of follow-up types to define seven Prototypical Human-AI Collaboration Behaviors (PATHs), and runs logistic regressions to correlate writing intents with PATHs. The central claims are that a small set of PATHs accounts for 80-85% of the variance in follow-up behavior in both datasets, and that specific writing intents are significantly associated with specific PATHs, with qualitative examples illustrating behaviors such as brainstorming, staged generation, question asking, and content addition.

Significance. If the empirical findings hold, this is a valuable large-scale characterization of human-LLM collaboration in writing. The two-dataset design, the use of a public dataset for reproducibility, and the mixed-methods grounding in qualitative examples are genuine strengths. The paper also makes concrete, falsifiable descriptive claims about follow-up behavior that go beyond coarse task classification and satisfaction prediction. The main contribution is empirical rather than theoretical, and its value depends heavily on the reliability of the automated labels; the current evidence for that reliability is too thin to support the strength of the downstream claims.

major comments (4)

[§4, §C.3 (Table 8)] The central regression results depend on labels produced by the same GPT-4o model family for both the predictors (writing intents via fWritingI) and the outcomes (follow-up types via fFollowU). The validation in Appendix C.3 uses only 110 sessions per classifier, with accuracies of 74-89% and kappa of 0.74-0.82. This sample is too small to detect systematic label bias, and a consistent lexical association in GPT-4o labeling--for example, the word "more" appearing both in requests labeled GENERATE CATCHY TEXT and in follow-ups labeled REQUESTS MORE OUTPUTS--could manufacture the very correlations reported in §6. Cross-dataset replication does not rule this out because the same labeling model is applied to both datasets. The authors should address this load-bearing concern directly, for example by re-running the full pipeline on a larger human-annotated subset, using an independently trained or different labeler, or providing explicit label-bias diagnostics that show the correlations survive under alternative labels.
[§C.2 (logistic regressions) and §4] The paper does not correct for multiple comparisons in the regression analysis. With roughly 18 intent predictors and seven PATH targets, running independent logistic regressions at the p≤0.05 level yields an expected number of false positives in the double digits per dataset, even before considering cross-dataset dependence. The statement that an association is meaningful because it is significant in both BCPWr and WCWr does not address this, since a shared labeler can induce correlated errors in both datasets. The authors should report adjusted p-values, false-discovery-rate corrections, or permutation-based null distributions. The thresholding of PCA scores into binary PATH membership, described as "selected to ensure the accuracy of our logistic regressions," is also a modeling choice that can inflate apparent associations; sensitivity of the results to this threshold should be reported.
[§4, §5 (variance claim)] The abstract and Section 5 state that seven PATHs "capture 80-85% of the variance," but this is true by construction because the number of principal components l was chosen precisely so that the retained components explain 80-85% of the variance. The substantive claim is that a relatively small l suffices, which is reasonable, but the current wording implies that the variance proportion was discovered rather than selected. The authors should present a scree plot or an equivalent explanation of how l was chosen and should report the variance explained as a function of l. The binarization threshold for PATH membership is a second free parameter that should receive the same sensitivity treatment.
[§C.2, Table 1] The logistic regressions treat sessions as independent observations, but the datasets contain many sessions per user; BCPWr has 202k users and 250k sessions, and WCWr has 22k users and 68k sessions. Repeated sessions from the same user are likely correlated, so the reported p-values may be anticonservative. The authors should use cluster-robust standard errors by user or otherwise account for within-user dependence. This is directly relevant to the strength of the statistical significance claims that support RQ2.

minor comments (6)

[§1] There is a grammatical error in the phrase "a important and increasingly prevalent use case"; it should be "an important."
[§C.2] The possessive in "the users original requests" should be "the user's original requests."
[§4, Table 5] The paper should clarify the unit of analysis for the regression. Table 5 reports a mean of 1.6 original requests per session, but §C.2 describes one-hot intent features for a session. It would be helpful to state explicitly how sessions with multiple original requests are represented in the one-hot encoding and whether the target PATH is defined at the session level.
[Figure 3b] The heatmap in Figure 3b is difficult to read in the current rendering, especially for distinguishing the loading magnitudes for PATHs 5-7; the accompanying text is clear, but the figure labels should be enlarged or the figure split into two panels.
[Prompt 3, Table 3] The mapping from prompt labels (GENERATE TITLE, GENERATE CASUAL BIO) to the paper's displayed labels (GENERATE CATCHY TEXT, GENERATE BIO) is shown only in the prompt appendix; the main text should use one consistent naming convention throughout.
[§8] The Limitations section acknowledges English-only data, order-insensitivity, and light-weight qualitative analysis, but it does not mention the shared-labeler concern or the absence of multiple-comparison correction. The sentence "We ensure the reliability of our analysis through manual validation of all automated components" overstates the certainty provided by the small validation samples in Appendix C.3.

Circularity Check

1 steps flagged · score 6.0 of 10

The 'seven PATHs explain 80-85% of variance' claim restates the PCA retention threshold rather than an empirical discovery; the intent-PATH regressions themselves are not circular.

self definitional [Section 4 'Identifying PATHs' and Section 5 'High-level trends in PATHs' (also reflected in the abstract)]
"We retain the first l dimensions of P that explain 80-85% variance in F, treating the rest as noise. ... Seven PATHs explain 80-85% of the variance, showing that a small set of collaboration behaviors explains the bulk of variance in follow-up behaviors in BCPWr and WCWr."

The reported variance-explained figure is set by the analysis criterion, not discovered from the data. The paper defines the number of retained principal components as exactly those needed to explain 80-85% of variance, then presents 'Seven PATHs explain 80-85% of the variance' as a substantive answer to RQ1. Saying that the retained components explain the chosen threshold is a restatement of the selection rule; the only data-dependent fact is that l=7, which the paper does not foreground. The abstract's claim that a small group of PATHs 'explain a majority of the variation' therefore inherits the threshold choice rather than being an independent empirical result.

full rationale

The paper's main RQ2 derivation—logistic regression of writing intents on PATH scores—is a genuine, non-circular empirical analysis: the predictors (fWritingI intent labels) and targets (PATH/follow-up labels) are distinct variables, and the regression coefficients are not forced by construction. There is no load-bearing self-citation, no uniqueness theorem imported from the authors' prior work, and no ansatz smuggled in via citation. The shared GPT-4o labeling of both intents and follow-ups is a real measurement threat to the intent-PATH correlations, but it is a validity/annotation-bias concern rather than a reduction of an output to its inputs, and Appendix C.3's small manual validation is evidence about accuracy rather than a circular step. The one clear circularity is the variance-explained headline: the 80-85% figure is the retention threshold used to choose the number of PATHs, so reporting it as a finding is self-definitional. Because that variance claim is a central takeaway but the rest of the analysis has independent content, a score of 6 reflects partial circularity rather than a fully forced derivation.

Assumptions & free parameters 2 free parameters · 4 assumptions · 1 invented entities

The central claim rests on the validity of the LLM-based labeling pipeline, the simplified session representation, and the assumption that the filtered logs represent in-the-wild behavior. The free parameters (number of PATHs, binarization threshold) add flexibility to the analysis and should be tested for robustness.

free parameters (2)

Number of principal components l = 7
Chosen to explain 80-85% of the variance in follow-up type frequencies. This determines the number of PATHs and is a hand-selected cutoff rather than a value determined by a theory.
PATH binarization threshold = Not stated; selected to retain 15-20% of sessions per PATH
The continuous PCA scores for each PATH are thresholded to create binary targets for logistic regression. The threshold is tuned to make the regressions accurate, which is a data-dependent choice that can inflate apparent correlations.

assumptions (4)

domain assumption GPT-4o based classifiers (fCoarseT, fFollowU, fWritingI) are accurate enough to label the large-scale data without systematic bias.
Validation on 110-220 manually annotated sessions shows accuracies of 74-89%, but with hundreds of thousands of sessions, even a small systematic error could alter the derived PATHs and regressions.
domain assumption The tf-idf bag-of-follow-up-types representation preserves sufficient behavioral information for PCA to find meaningful patterns.
This representation ignores the order of follow-ups and treats each session as a bag of types. The paper acknowledges this in limitations, but the PATHs are defined entirely within this simplified representation.
domain assumption The taxonomies of follow-up types and writing intents are discrete, distinct, and cover the space of user behaviors.
The labels were generated iteratively from clusters and manually checked, but annotators noted confusion between similar categories, such as story vs. character vs. script, and style vs. add content.
domain assumption The filtered datasets (English, PC-based, 2+ utterances) are representative of in-the-wild writing collaboration.
The paper limits analysis to English sessions and, for Bing Copilot, to personal computers. This excludes mobile and non-English behaviors that could differ, as acknowledged in the limitations section.

invented entities (1)

Prototypical Human-AI Collaboration Behaviors (PATHs)
purpose: To summarize the dominant co-occurring follow-up behaviors in writing sessions into a small number of interpretable patterns.
PATHs are derived from the same data used for the correlation analysis via PCA. They have no external validation and are not shown to predict any independent outcome, so they are an internal construct.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Prototypical Human-AI Collaboration Behaviors from LLM-Assisted Writing in the Wild." pith.science (2026). https://pith.science/paper/AB6MXKIA

@misc{pith2026250516023,
  author       = {Pith},
  title        = {Pith review of: Prototypical Human-AI Collaboration Behaviors from LLM-Assisted Writing in the Wild},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AB6MXKIA}},
  note         = {Machine review of arXiv:2505.16023}
}

read the original abstract

As large language models (LLMs) are used in complex writing workflows, users engage in multi-turn interactions to steer generations to better fit their needs. Rather than passively accepting output, users actively refine, explore, and co-construct text. We conduct a large-scale analysis of this collaborative behavior for users engaged in writing tasks in the wild with two popular AI assistants, Bing Copilot and WildChat. Our analysis goes beyond simple task classification or satisfaction estimation common in prior work and instead characterizes how users interact with LLMs through the course of a session. We identify prototypical behaviors in how users interact with LLMs in prompts following their original request. We refer to these as Prototypical Human-AI Collaboration Behaviors (PATHs) and find that a small group of PATHs explain a majority of the variation seen in user-LLM interaction. These PATHs span users revising intents, exploring texts, posing questions, adjusting style or injecting new content. Next, we find statistically significant correlations between specific writing intents and PATHs, revealing how users' intents shape their collaboration behaviors. We conclude by discussing the implications of our findings on LLM alignment.

Figures

Figures reproduced from arXiv: 2505.16023 by the authors.

**Figure 2.** Our analysis methods aim to discover PATHs (above) and identify how PATHs differ across writing intents in original requests (ORs) (below). had 2 follow-ups on average and that half of all sessions contain only one writing intent. 4 Log Analysis with PATHs Our analysis centers around two research questions: (RQ1) What high-level collaboration behaviors emerge from user interactions in LLM-assisted writing? And (RQ… view at source ↗

**Figure 3.** (a) The fraction of sessions which contain a follow-up type. (b) The correlation between follow-up types [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗

Figures from the paper (11 more)

**Figure 4.** Figure 4: (a) Large positive values in coefficient plots indicate strong correlations. The intents discussed in § [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: (a) The intents discussed in §6.2 are highlighted in color. Coefficients for WCWr are plotted in [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: (a) The intents discussed in §6.3 are highlighted in color. Coefficients for WCWr are plotted in [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

**Figure 7.** Figure 7: The proportion of coarse tasks in BCPAll (2.8M sessions) and WCAll (160k sessions). The sessions are in English and likely to have at least 1 user follow-up. A session can have multiple coarse tasks. On average, BCPAll and WCAll had 1.25 and 1.43 tasks per session. ta…

**Figure 8.** Figure 8: Frequencies of writing intents in BCPWr and WCWr identified by multi-label fWritingI. From the top-3 intents, we see that Bing Copilot was used for creating communication or professional texts, while WildChat was used for creative texts such as stories, scripts, or cha…

**Figure 9.** Figure 9: (a) The intents discussed in §D.1 are highlighted in color. Coefficients for WCWr are plotted in [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]

**Figure 10.** Figure 10: Coefficient plots for writing intents vs PA [PITH_FULL_IMAGE:figures/full_fig_p020_10.png]

**Figure 11.** Figure 11: Coefficient plots for writing intents vs PA [PITH_FULL_IMAGE:figures/full_fig_p020_11.png]

**Figure 12.** Figure 12: Supplementary coefficient plots for Figure [PITH_FULL_IMAGE:figures/full_fig_p021_12.png]

**Figure 13.** Figure 13: Increasing similarity between the follow-up utterance and the previous model generation in WC [PITH_FULL_IMAGE:figures/full_fig_p022_13.png]

**Figure 14.** Figure 14: (a) Increasing similarity between user follow-up and the next model generation across follow-up types [PITH_FULL_IMAGE:figures/full_fig_p022_14.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Training-Free Token-Level Steering for LLM Personalized Co-Writing
cs.CL 2026-08 conditional novelty 6.0 of 10

A token-level, training-free steering framework that improves LLM personalized co-writing by mixing the base model's posterior with a kernel-density estimate from a small user dataset.
Componentization: Decomposing Monolithic LLM Responses into Manipulable Semantic Units
cs.HC 2025-09 conditional novelty 6.0 of 10

Generative model outputs can be decomposed into typed, linkable components that users edit, toggle, and regenerate before recomposition, as implemented in the MAODchat prototype.
Evaluating Style-Personalized Text Generation: Challenges and Directions
cs.CL 2025-08 reject novelty 6.0 of 10

A new style-discrimination benchmark for personalized text generation shows ensemble metrics give only a marginal, possibly test-fitted, edge over the best single judge.
Oldie but Goodie: Re-illuminating Label Propagation on Graphs with Partially Observed Features
cs.LG 2025-08 unverdicted novelty 5.0 of 10

A hybrid label-and-feature propagation framework with attention and pseudo-label contrastive learning claims state-of-the-art node classification on graphs with partially missing features.

Reference graph

Works this paper leans on

17 extracted references · 9 canonical work pages · cited by 4 Pith papers

[1]

Output one of more of the correct labels for the CONVERSATION

work page
[2]

yes please

Output the labels in decreasing order of their relevance to the CONVERSATION CONVERSATION : {{conversation_text}} Output a explanation and then one or more of the labels for the CONVERSATION . 23 Prompt 2The prompt for GPT-4o used to label user utterances as original requests and or one among eleven follow-up types in BCPWr and WCWr. This is discussed in ...

work page
[3]

it makes you think

" it makes you think": Provocations help re- store critical thinking to ai-assisted knowledge work. arXiv preprint arXiv:2501.17247. Susan Dumais, Robin Jeffries, Daniel M. Russell, Diane Tang, and Jaime Teevan. 2014.Understanding User Behavior Through Log Data and Analysis, pages 349–

arXiv 2014
[8]

Weixin Liang, Yuhui Zhang, Hancheng Cao, Binglu Wang, Daisy Yi Ding, Xinyu Yang, Kailas V odrahalli, Siyu He, Daniel Scott Smith, Yian Yin, Daniel A

The widespread adoption of large language model-assisted writing across society.Preprint, arXiv:2502.09747. Weixin Liang, Yuhui Zhang, Hancheng Cao, Binglu Wang, Daisy Yi Ding, Xinyu Yang, Kailas V odrahalli, Siyu He, Daniel Scott Smith, Yian Yin, Daniel A. McFarland, and James Zou. 2024b. Can large lan- guage models provide useful feedback on research pa...

arXiv 2024
[9]

In Proceedings of the 21st International Conference on Natural Language Processing (ICON), pages 230– 242, AU-KBC Research Centre, Chennai, India

Exploring user dissatisfaction: Taxonomy of implicit negative feedback in virtual assistants. In Proceedings of the 21st International Conference on Natural Language Processing (ICON), pages 230– 242, AU-KBC Research Centre, Chennai, India. NLP Association of India (NLPAI). Chaitanya Malaviya, Joseph Chee Chang, Dan Roth, Mohit Iyyer, Mark Yatskar, and Ky...

arXiv 2024
[11]

Search Information

Bias in ai autocomplete suggestions leads to attitude shift on societal issues.PsyArXiv. Shujin Wu, Yi R. Fung, Cheng Qian, Jeonghwan Kim, Dilek Hakkani-Tur, and Heng Ji. 2025. Aligning LLMs with individual preferences via interaction. In Proceedings of the 31st International Conference on Computational Linguistics, pages 7648–7662, Abu Dhabi, UAE. Associ...

arXiv 2025
[14]

Output one best label for every USER UTTERENCE

work page
[15]

Msitu Africa is requesting for funding through crowdfunding websites. Write for me a small compelling write up requesting for funds

Output the labels in the order of the USER UTTERENCES . Here is the conversation : {{conversation_text}} Output a label for each USER UTTERENCE . 24 Follow-up Type Description Examples (original request and follow-up) RESTATES REQUESTReformulates their request“Msitu Africa is requesting for funding through crowdfunding websites. Write for me a small compe...

work page

Show all 17 references

[16]

Output one of more of the correct labels for the USER REQUEST
[17]

say better and more professional: Beckey, really appreciate your advocacy during

Output the labels in decreasing order of their relevance to the USER REQUEST . USER REQUEST :{{user_request}} Output a explanation and then one or more of the labels for the USER REQUEST . 26 Writing intent type Example original requests IMPROVE TEXT “say better and more profe...

2024
[372]

Nathan Eagle and Alex Sandy Pentland

Springer New York, New York, NY . Nathan Eagle and Alex Sandy Pentland. 2009. Eigenbe- haviors: Identifying structure in routine.Behavioral ecology and sociobiology, 63:1057–1066. Jonas Frich, Lindsay MacDonald Vermeulen, Christian Remy, Michael Mose Biskjaer, and Peter Dalsgaard

2009
[2009]

it felt like having a second mind

Eigenplaces: analysing cities using the space– time structure of the mobile phone network.En- vironment and planning B: Planning and design, 36(5):824–836. Vasile Rus and Panayiota Kendeou. 2025. Are llms actually good for learning?AI & SOCIETY, pages 1–2. Rupak Sarkar, Bahare...

2025 arXiv
[2013]

Pattern Anal

Representation learning: A review and new perspectives.IEEE Trans. Pattern Anal. Mach. Intell., 35(8):1798–1828. Advait Bhat, Saaket Agashe, Parth Oberoi, Niharika Mohile, Ravi Jangir, and Anirudha Joshi. 2023. Inter- acting with next-phrase suggestions: How suggestion systems...

2023 arXiv
[2019]

InProceedings of the 2019 CHI Confer- ence on Human Factors in Computing Systems, CHI ’19, page 1–18, New York, NY , USA

Mapping the landscape of creativity support tools in hci. InProceedings of the 2019 CHI Confer- ence on Human Factors in Computing Systems, CHI ’19, page 1–18, New York, NY , USA. Association for Computing Machinery. Yue Fu, Sami Foell, Xuhai Xu, and Alexis Hiniker

2019
[2020]

It was 80% me, 20% AI

Ai-mediated communication: Definition, re- search agenda, and ethical considerations.Journal of Computer-Mediated Communication, 25(1):89–100. Kunal Handa, Alex Tamkin, Miles McCain, Saffron Huang, Esin Durmus, Sarah Heck, Jared Mueller, Jerry Hong, Stuart Ritchie, Tim Belonax...

2025 arXiv
[2024]

In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI ’24, New York, NY , USA

From text to self: Users’ perception of aimc tools on interpersonal communication and self. In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems, CHI ’24, New York, NY , USA. Association for Computing Machinery. 10 Ge Gao, Alexey Taymanov, Eduardo Sa...

2024
[2025]

Preprint, arXiv:2409.11360

Ai suggestions homogenize writing to- ward western styles and diminish cultural nuances. Preprint, arXiv:2409.11360. Marwah Alaofi, Luke Gallagher, Dana Mckay, Lau- ren L. Saling, Mark Sanderson, Falk Scholer, Dami- ano Spina, and Ryen W. White. 2022. Where do queries come fro...

2022 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

Output one of more of the correct labels for the CONVERSATION

work page

[2] [2]

yes please

Output the labels in decreasing order of their relevance to the CONVERSATION CONVERSATION : {{conversation_text}} Output a explanation and then one or more of the labels for the CONVERSATION . 23 Prompt 2The prompt for GPT-4o used to label user utterances as original requests and or one among eleven follow-up types in BCPWr and WCWr. This is discussed in ...

work page

[3] [3]

it makes you think

" it makes you think": Provocations help re- store critical thinking to ai-assisted knowledge work. arXiv preprint arXiv:2501.17247. Susan Dumais, Robin Jeffries, Daniel M. Russell, Diane Tang, and Jaime Teevan. 2014.Understanding User Behavior Through Log Data and Analysis, pages 349–

arXiv 2014

[4] [8]

Weixin Liang, Yuhui Zhang, Hancheng Cao, Binglu Wang, Daisy Yi Ding, Xinyu Yang, Kailas V odrahalli, Siyu He, Daniel Scott Smith, Yian Yin, Daniel A

The widespread adoption of large language model-assisted writing across society.Preprint, arXiv:2502.09747. Weixin Liang, Yuhui Zhang, Hancheng Cao, Binglu Wang, Daisy Yi Ding, Xinyu Yang, Kailas V odrahalli, Siyu He, Daniel Scott Smith, Yian Yin, Daniel A. McFarland, and James Zou. 2024b. Can large lan- guage models provide useful feedback on research pa...

arXiv 2024

[5] [9]

In Proceedings of the 21st International Conference on Natural Language Processing (ICON), pages 230– 242, AU-KBC Research Centre, Chennai, India

Exploring user dissatisfaction: Taxonomy of implicit negative feedback in virtual assistants. In Proceedings of the 21st International Conference on Natural Language Processing (ICON), pages 230– 242, AU-KBC Research Centre, Chennai, India. NLP Association of India (NLPAI). Chaitanya Malaviya, Joseph Chee Chang, Dan Roth, Mohit Iyyer, Mark Yatskar, and Ky...

arXiv 2024

[6] [11]

Search Information

Bias in ai autocomplete suggestions leads to attitude shift on societal issues.PsyArXiv. Shujin Wu, Yi R. Fung, Cheng Qian, Jeonghwan Kim, Dilek Hakkani-Tur, and Heng Ji. 2025. Aligning LLMs with individual preferences via interaction. In Proceedings of the 31st International Conference on Computational Linguistics, pages 7648–7662, Abu Dhabi, UAE. Associ...

arXiv 2025

[7] [14]

Output one best label for every USER UTTERENCE

work page

[8] [15]

Msitu Africa is requesting for funding through crowdfunding websites. Write for me a small compelling write up requesting for funds

Output the labels in the order of the USER UTTERENCES . Here is the conversation : {{conversation_text}} Output a label for each USER UTTERENCE . 24 Follow-up Type Description Examples (original request and follow-up) RESTATES REQUESTReformulates their request“Msitu Africa is requesting for funding through crowdfunding websites. Write for me a small compe...

work page