Information Retrieval · arXiv:2610.29216 v1 · submitted 21 Oct 2026 · 64 references
Preference Is Not Permission: Personalizing the Auto-Purchase Line in Agentic Commerce
The paper says
“Personalized shopping benchmarks ask which item fits the user; we ask how far an agent may commit on that item, and pose learning the individual's auto-purchase line as a task.”
more of the abstract
The boundary is evidence-conditioned and three-valued: whether committing is appropriate depends on the sufficiency of product information (warrant), reversibility, price against the user's norm, and urgency, not only on fit; and the natural output is don't buy, confirm first, or buy automatically. Across sixteen instruction-tuned models, autonomous commitment is misaligned with a human authorization reference in which fifteen annotators, role-playing the delegating principal from a given profile, choose the furthest action they would allow without confirmation: models commit even when fit is low and when their own evidence flag reports the warrant missing. This is overreach. The line is individual: these raters disagree, yet the disagreement is structured and partly predictable from a person's own other labels. A shallow learner on a few dozen of a person's own labels predicts their held-out auto-purchase line at a median of 0.79 by 30-40 labels, outperforming every evaluated prompt×model configuration, including a per-annotator oracle over 80 configurations; the learned boundary does not transfer across domains. We release the counterfactual benchmark, human authorization reference, and annotation protocol after acceptance.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 22 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up. Solid empirical paper formalizing a real gap (preference ≠ permission) and showing that personalized authorization boundaries are learnable from few labels; limitations are openly stated and the central claims hold up. This line is wrong
5 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 64 references say.
5 minor objections
The paper says, at §3.4, Table 3
Pith says: minor objection 1 of 5
The cell 'price-anomaly' is described as a two-factor construct (unexplained discount + unrated seller). The follow-up 'seller-unrated' cell is mentioned only in passing. Consider adding a brief note in the table or caption to clarify the relationship between these cells and why the additional cell was needed, to avoid reader confusion.
The paper says, at §4.2, para 2
Pith says: minor objection 2 of 5
The sentence 'Relative spend is largely ignored within the user's stated ceiling' is supported only for the L3–L5 dial levels; the L1–L2 levels show a downward trend (commit 0.72 at 0.12B vs. 0.49 at 0.90B). The text acknowledges this but the headline 'ignored' may be slightly misleading. Consider qualifying the statement to reflect that sensitivity appears only under cautious instructions.
show 3 more
The paper says, at §6.1, Fig. 3 and Table 1
Pith says: minor objection 3 of 5
The held-out accuracy metric in Fig. 3(b) is not explicitly defined as binary 'auto vs. not' (it is described in the caption caption but the text could state it earlier in the section). Also, the partial-pooling logistic regression (D and D+) is introduced only in §6.1 but referenced in Table 1; a brief description of the shrinkage prior and how it compares to the unpooled version used in Fig. 3 would improve clarity.
The paper says, at §7, robustness section
Pith says: minor objection 4 of 5
The Kendall's tau median of 0.82 for the paraphrase sweep is reported, but the text does not specify which variable's rank ordering is being correlated (it should be model ranking by commit@weak). Clarify explicitly.
The paper says, at §8, Limitations
Pith says: minor objection 5 of 5
The test–retest limitation (sentence b213) is well stated. However, the paper could briefly note in this section that session-specific heuristics (consistent but ephemeral labeling strategies) would still be problematic for real-world personalization if those heuristics do not reflect the user's stable boundary. The current argument ('pure noise would not be predictable') addresses noise but not systematic but temporary strategies. A short sentence acknowledging this would strengthen the limitation's framing.
What would settle it
Re-label 30–50 of the original cases (stratified by factor level) by 5–10 of the same annotators after a 2-week washout. Compute per-annotator test-retest ordinal agreement (e.g., weighted Cohen's kappa or Krippendorff's alpha). If the mean test-retest alpha is below 0.5, the within-annotator predictability reported in §6 could be driven by session-consistent annotation heuristics rather than stable personal authorization preferences, qualifying the interpretation of RQ3.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 22 Oct 2026
- Reader holds up
- Skeptic partly agrees
- Editor wrote the verdict above
- Referee minor revision: 5 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
This is a well-executed empirical study that identifies and measures a genuine blind spot in agentic-commerce evaluation. The core insight—that preference fit does not authorize autonomous purchase—is simple but consequential, and the paper treats it seriously. The counterfactual benchmark (minimal-edit cells grounded in real histories, with a human authorization reference from 15 annotators) is carefully constructed. The three-RQ structure is clean: models overcommit (RQ1), a lightweight warrant check can correct the worst overreach (RQ2), and the personalized line is learnable from a person's own labels, outperforming any prompt×model configuration (RQ3). The evaluation is thorough: sixteen models across two domains, a five-level autonomy dial, 24 paraphrases, 8 placebos, robustness checks including leave-one-family-out and author exclusion. The results are credible—models do ignore fit and commit on weak warrant, and the personalization result (median 0.79 at 30–40 labels) is striking against the oracle configuration baseline.
The paper is honest about its limits. The human reference comes from role-playing annotators, not real users on live storefronts; the authors flag this. The low inter-annotator agreement (α≈0.25) is discussed, and the claim of individual structure rests on held-out predictability from that annotator's own labels, not on high agreement—a reasonable argument. The stress-test concern about test–retest reliability is real but the paper already acknowledges it (b213) and offers the argument that noise would attenuate rather than create the own-label effect. That is a fair point, but it does leave the practical interpretation somewhat open: the learned boundary may reflect a stable labeling heuristic rather than an enduring personal authorization line. For the central overreach claims (RQ1, RQ2), this matters less because they depend on the pooled reference. For RQ3, it is a genuine caveat, but one the authors state and frame as future work.
The paper is not perfect: the C3-P warrant check is a proof of concept and the gate's high agreement with the injected warrant label is partly built in. The 15 base families and 16 models limit invariance claims. But these are scoped clearly.
Who is this for? Researchers in agentic commerce, AI safety for delegated transactions, and personalization. It deserves a serious referee; the editing should check that the test–retest caveat is prominent enough and that the distinction between stable preference and stable heuristic is not glossed over.
I would take this paper to my reading group, cite it, and recommend acceptance with minor revisions that ask the authors to emphasize the test–retest limitation in the abstract or conclusion rather than only in §8.
The strongest objection
The paper's RQ3 claims that the auto-purchase line is individual and learnable from a few labels, based on within-annotator held-out prediction. However, the paper lacks a test-retest reliability study: annotators labeled each case only once. The authors acknowledge (sentence b213) that 'without test–retest judgments we cannot fully separate stable individual variation from scenario ambiguity or instability'. While they argue that 'pure noise would not be predictable from an annotator's other labels', this argument assumes that any consistent annotation pattern reflects stable personal preference. An annotator could apply a consistent but transient heuristic (e.g., 'always confirm on final-sale items') that is reliable within a session but not stable over time. If test-retest agreement were low, the 'personalized boundary' would predict an ephemeral labeling strategy rather than a user's enduring authorization line, undermining the practical interpretation of the learning result. The paper's other claims (model overreach, factor effects) depend less on the stability of individual annotators and more on the pooled human reference, which is less sensitive to this issue. For the personalized learning claim, however, this is a load-bearing gap.
The reader identified the human reference as the weakest assumption, noting test-retest was missing. My concern focuses specifically on the implications for the personalized learning claim, which the reader did not isolate as the primary issue. The paper explicitly acknowledges the lack of test-retest and qualifies its claims, so I recommend no change to the ACCEPT verdict. The internal validity of the learning results stands; the concern is about the stability interpretation, which is a scope limitation the paper already states.
The referee's report
If the results hold, this work is significant for several reasons. First, it identifies and measures a gap that existing preference-centric benchmarks miss: the distinction between 'what the user would like' and 'what the agent may commit to without asking'. Second, the methodology is rigorous and reproducible: minimal-edit counterfactuals, a multi-rater human reference (kept as distributions, not majorities), extensive robustness checks (prompt sweep across five autonomy dial levels, 24 meaning-preserving paraphrases, 8 placebo edits, leave-one-family-out, author removal), and clear separation of measurements. Third, the finding that a simple personalized boundary fit to a few dozen labels outperforms all prompt/model configurations, including a per-annotator oracle over 80 configurations, is novel and practically consequential. The benchmark and human reference will be valuable resources for future work on agent authorization. The paper's explicit acknowledgment of limitations (vignette design, low agreement, no test–retest, synthetic manipulations, small base families) strengthens rather than weakens the contribution by bounding its scope.
Pith's reading of the paper (a machine's summary, not the authors')
This paper asks how far an agentic shopping system may go on its own after finding a suitable item. The authors show that current models routinely overcommit: they auto-buy even when product evidence is weak and even when the model itself flags that evidence as missing. The authors collect a human authorization reference from fifteen annotators role-playing the delegating customer across 270 synthetic scenarios. Against that reference, every evaluated model overreaches on at least some cells. Crucially, the boundary between appropriate autonomy and overreach is personal: annotators disagree, but the disagreement is structured. A simple logistic regression fit to a person's own labels on 30–40 cases predicts their held-out authorization line at a median of 0.79, outperforming every prompt×model configuration tested, including a per-annotator oracle over 80 policies. The learned boundary does not transfer across domains, confirming it captures individual sensitivities to authorization factors rather than generic patterns.
What carries the argument. The central object is the auto-purchase line, an evidence-conitioned three-valued boundary (don't buy < confirm first < buy automatically) that determines how far an agent may commit on a particular product. The learning mechanism is a partial-pooling logistic regression that predicts this boundary from four factor features (warrant, reversibility, relative price, urgency) and is fit to a person's own labels, shrunk toward a prior from other annotators. This setup identifies that the disagreement among annotators is systematic and learnable, not noise.
Core claim. The paper establishes that preference fit (whether the product matches the user) and authorization permission (whether the agent may commit to purchase based on evidence) are distinct functions. High fit does not imply high permission. Across sixteen instruction-tuned models, autonomous commitment is misaligned with a human reference: models buy even when fit is low and when their own evidence flag reports the warrant missing. However, an individual's authorization boundary — a three-level ladder of don't buy, confirm first, or buy automatically — can be learned from a few dozen of that person's own labels. A partial-pooling logistic regression on factor features (warrant, reversibility, relative price, urgency) predicts held-out cases at a median 0.79, beating every evaluated prompt and model choice. This shows that personalization of the authorization line is not only necessary but feasible.
The premise it rests on. The human authorization reference, collected from 15 annotators role-playing principals on synthetic evidence cards, reflects the actual authorization preferences of real users in real shopping contexts.
If the paper is right (Pith's inferences, not the authors')
Shopping agents should be evaluated on authorization safety separately from preference accuracy; a system can rank the right item and still overreach.
Current models are only partially calibrated: they ignore relative price within the user's stated ceiling, respond sharply only when that ceiling is crossed, and show little sensitivity to poor fit.
A lightweight, population-level warrant check (C3-P) can eliminate all warrant-driven overcommit in the benchmark, but real-storefront validation remains missing.
Personalized authorization boundaries require in-domain labels; they do not transfer across domains.
Overcommit is largely a single-weak-signal failure: two adverse cues can even reduce overcommit compared with one alone.
The predictability of personal authorization lines from a few labels suggests that a short calibration questionnaire at account setup could collect the necessary signal for a deployed agent.
Because the human reference has low inter-annotator agreement (Krippendorff α ≈ 0.25), any deployed system should treat the authorization line as a per-user latent variable rather than a population norm, reinforcing the need for personalization.
The factor structure (warrant, reversibility, price, urgency) is domain-independent, but individual sensitivities are not; this implies that cross-domain transfer may require adapting the learned factor weights, not just the features.
For safety certification, a two-layer design (population-level warrant check plus user-specific boundary) could provide a testable guarantee: the lower layer catches evident overreach while the upper layer tailors permission to the individual.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Personalized shopping benchmarks ask which item fits the user; we ask how far an agent may commit on that item, and pose learning the individual's auto-purchase line as a task. The boundary is evidence-conditioned and three-valued: whether committing is appropriate depends on the sufficiency of product information (warrant), reversibility, price against the user's norm, and urgency, not only on fit; and the natural output is don't buy, confirm first, or buy automatically. Across sixteen instruction-tuned models, autonomous commitment is misaligned with a human authorization reference in which fifteen annotators, role-playing the delegating principal from a given profile, choose the furthest action they would allow without confirmation: models commit even when fit is low and when their own evidence flag reports the warrant missing. This is overreach. The line is individual: these raters disagree, yet the disagreement is structured and partly predictable from a person's own other labels. A shallow learner on a few dozen of a person's own labels predicts their held-out auto-purchase line at a median of 0.79 by 30-40 labels, outperforming every evaluated prompt×model configuration, including a per-annotator oracle over 80 configurations; the learned boundary does not transfer across domains. We release the counterfactual benchmark, human authorization reference, and annotation protocol after acceptance.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-22
The authors' main claim, as the classifier picked it out 0.54
“Across sixteen instruction-tuned models, autonomous commitment is misaligned with a human authorization reference in which fifteen annotators, role-playing the delegating principal from a given profile, choose the furthest action they would allow without confirmation: models commit even when fit is low and when their own evidence flag reports the warrant missing. This is overreach.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.09 · states limitation 0.92 · human participants 0.93 · controlled comparison 0.70 · randomized assignment 0.02 · states quantitative result 0.98 · compares with prior results 0.91 · mentions code or data availability 0.80
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Personalized shopping benchmarks ask which item fits the user; we ask how far an agent may commit on that item, and pose learning the individual's auto-purchase line as a task. | contribution 0.78 | 0.61 | 0.08 | asserted 0.89 |
| The boundary is evidence-conditioned and three-valued: whether committing is appropriate depends on the sufficiency of product information (warrant), reversibility, price against the user's norm, and urgency, not only on fit; and the natural output is don't buy, confirm first, or buy automatically. | contribution 0.72 | 0.48 | 0.17 | asserted 1.00 |
| Across sixteen instruction-tuned models, autonomous commitment is misaligned with a human authorization reference in which fifteen annotators, role-playing the delegating principal from a given profile, choose the furthest action they would allow without confirmation: models commit even when fit is low and when their own evidence flag reports the warrant missing. This is overreach. | result 0.96 | 0.92 | 0.48 | asserted 1.00 |
| The line is individual: these raters disagree, yet the disagreement is structured and partly predictable from a person's own other labels. | result 0.90 | 0.87 | 0.07 | asserted 0.97 |
| A shallow learner on a few dozen of a person's own labels predicts their held-out auto-purchase line at a median of 0.79 by 30-40 labels, outperforming every evaluated prompt$\times$model configuration, including a per-annotator oracle over 80 configurations; the learned boundary does not transfer across domains. | result 0.99 | 0.94 | 0.38 | asserted 1.00 |
| We release the counterfactual benchmark, human authorization reference, and annotation protocol after acceptance. | contribution 0.95 | 0.80 | 0.03 | asserted 0.96 |
What kind of paper the classifier reports this is: dataset or benchmark 0.98 · evidence from observation or experiment 1.00
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-22: 1 request, 5,240 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (4)
Figures are the authors', reproduced from the paper.
References (64)
- [1] Herman Aguinis and Kyle J. Bradley. 2014. Best Practice Recommendations for Designing and Implementing Experimental Vignette Methodology Studies.Orga- nizational Research Methods17, 4 (2014), 351–371. doi:10.1177/1094428114547952 on Pithdoi:10.1177/1094428114547952
- [2] Gati Aher, Rosa I. Arriaga, and Adam Tauman Kalai. 2023. Using Large Language Models to Simulate Multiple Humans and Replicate Human Subject Studies. In Proceedings of the 40th International Conference on Machine Learning, Vol. 202. 337–371. https://proceedings.mlr.press/v202/... on Pith
- [3] Cheryl S. Alexander and Henry Jay Becker. 1978. The Use of Vignettes in Survey Research.The Public Opinion Quarterly42, 1 (1978), 93–104. doi:10.1086/268432 on Pithdoi:10.1086/268432
- [4] Amine Allouah, Omar Besbes, Josué D. Figueroa, Yash Kanoria, and Akshit Kumar on Pith
- [5] Norah Alzahrani, Hisham Alyahya, Yazeed Alnumay, Sultan AlRashed, Shaykhah Alsubaie, Yousef Almushayqih, Faisal Mirza, Nouf Alotaibi, Nora Al-Twairesh, Areeb Alowisheq, M Saiful Bari, and Haidar Khan. 2024. When Benchmarks are Targets: Revealing the Sensitivity of Large Langua... on Pithdoi:10.18653/v1/2024.acl-long.744
- [6] Belinda Barton, Natalina Zlatevska, and Harmen Oppewal. 2022. Scarcity tactics in marketing: A meta-analysis of product scarcity effects on consumer purchase intentions.Journal of Retailing98, 4 (2022), 741–758. doi:10.1016/j.jretai.2022.06. 003 on Pithdoi:10.1016/j.jretai.2022.06
- [7] Nicolas Bougie, Gian Maria Marconi, Tony Yip, and Narimasa Watanabe. 2026. AlignUSER: Human-Aligned LLM Agents via World Models for Recommender System Evaluation.arXiv preprint arXiv:2601.00930(2026) on PitharXiv
- [8] Margaret C. Campbell, Justin Pomerance, and Erin L. Percival Carter. 2025. Painful Prices: The Moral Harm Model of Price Fairness.Journal of Consumer Research53 (2025), 444–466. doi:10.1093/jcr/ucaf045 on Pithdoi:10.1093/jcr/ucaf045
- [9] Jiacheng Chen, Tao Zhang, Manxi Lin, Dunxian Huang, et al. 2026. ShopX: A Foundation Model for Intent-to-Item Fulfillment in Agentic Shopping.arXiv preprint arXiv:2606.31693(2026) on PithPith's reviewarXiv
- [10] Manuel Cherep, Chengtian Ma, Abigail Xu, Maya Shaked, Patricia Maes, and Nikhil Singh. 2026. A Framework for Studying AI Agent Behavior: Evidence from Consumer Choice Experiments. InProceedings of the 14th International Conference on Learning Representations. https://openrevie... on Pith
- [11] Mengyi Deng, Zhiwei Li, Xin Li, Tingyu Zhu, Ying Zhao, Zhijiang Guo, and Wei Wang. 2026. Uncertainty-Aware Clarification in LLM Agents with Information Gain. InProceedings of the 43rd International Conference on Machine Learning. https://openreview.net/forum?id=7N2ydbDahV on Pith
- [12] Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Samuel Stevens, Boshi Wang, Huan Sun, and Yu Su. 2023. Mind2Web: Towards a Generalist Agent for the Web. InAdvances in Neural Information Pro- cessing Systems. https://papers.nips.cc/paper_files/paper/2023/hash/ 5950bf290a1570ea401... on Pith
- [13] Zeyao Du, Tong Li, and Haibo Zhang. 2026. EComAgentBench: Benchmarking Shopping Agents on Long-Horizon Tasks with Distributed Hidden Intent.arXiv preprint arXiv:2606.17698(2026) on PithPith's reviewarXiv
- [14] Quang Huy Duong, Li Zhou, Meng Meng, Le Thuy An Dang, and Tiep Duy Nguyen. 2025. Striking the Right Balance: Customising Return Policy Leniency for Managing Customer Online Return Proclivity and Satisfaction.Journal of Retailing and Consumer Services85 (2025), 104315. doi:10.1... on Pithdoi:10.1016/j.jretconser.2025
- [15] Nicholas Edwards and Sebastian Schuster. 2026. Ask or Assume? Uncertainty- Aware Clarification-Seeking in Coding Agents.arXiv preprint arXiv:2603.26233 (2026) on PithPith's reviewarXiv
- [16] Kathleen M. Eisenhardt. 1989. Agency Theory: An Assessment and Review.Acad- emy of Management Review14, 1 (1989), 57–74. doi:10.5465/amr.1989.4279003 on Pithdoi:10.5465/amr.1989.4279003
- [17] Shicheng Fan, Mingdai Yang, Duohao Wang, Canyu Chen, Yongfeng Zhang, Hua Wei, Manling Li, Julian McAuley, Kun Zhang, Philip S. Yu, Kejing Yu, and Zhiwei Liu. 2026. Agentic Commerce World: An Auditable and Verifiable Environment for Vibe Commerce.arXiv preprint arXiv:2608.02441(2026) on PithPith's reviewarXiv
- [18] Andrew Gelman and Jennifer Hill. 2007.Data Analysis Using Regression and Multilevel/Hierarchical Models. Cambridge University Press on Pith
- [19] Jason Goldberg. 2026. Why OpenAI’s Checkout Retreat Spells Trouble For Its Commerce Strategy. Forbes. Accessed September 2026 on Pith
- [20] Friederike Groschupp, Daniele Lain, Aritra Dhar, Lara Magdalena Lazier, and Srdjan Čapkun. 2025. Can LLMs Make (Personalized) Access Control Decisions? arXiv preprint arXiv:2511.20284(2025) on PithPith's reviewarXiv
show 44 more
- [21] Jens Hainmueller, Dominik Hangartner, and Teppei Yamamoto. 2015. Vali- dating Vignette and Conjoint Survey Experiments against Real-World Behav- ior.Proceedings of the National Academy of Sciences112, 8 (2015), 2395–2400. doi:10.1073/pnas.1416587112 on Pithdoi:10.1073/pnas.1416587112
- [22] Yuren Hao, Shuhaib Mehri, ChengXiang Zhai, and Dilek Hakkani-Tür. 2026. User Preference Modeling for Conversational LLM Agents: Weak Rewards from Retrieval-Augmented Interaction.arXiv preprint arXiv:2603.20939(2026) on PitharXiv
- [23] Majed El Helou, Benjamin Ryder, Chiara Troiani, Jean Diaconu, Hervé Muyal, and Marcelo Yannuzzi. 2026. Hybrid Inspection and Task-Based Access Control in Zero-Trust Agentic AI.arXiv preprint arXiv:2605.02682(2026) on PithPith's reviewarXiv
- [24] Majed El Helou, Chiara Troiani, Benjamin Ryder, Jean Diaconu, Hervé Muyal, and Marcelo Yannuzzi. 2025. Delegated Authorization for Agents Constrained to Semantic Task-to-Scope Matching.arXiv preprint arXiv:2510.26702(2025) on PitharXiv
- [25] Yupeng Hou, Jiacheng Li, Xiangjun Fu, Zhankui He, An Yan, Xiusi Chen, and Julian McAuley. 2026. Bridging Language and Items for Retrieval and Recom- mendation: Benchmarking LLMs as Semantic Encoders. InProceedings of the 64th Annual Meeting of the Association for Computational... on Pithdoi:10.18653/v1/2026.acl-long.147
- [26] Narayan Janakiraman, Holly A. Syrdal, and Ryan Freling. 2016. The Effect of Return Policy Leniency on Consumer Purchase and Return Decisions: A Meta- analytic Review.Journal of Retailing92, 2 (2016), 226–235. doi:10.1016/j.jretai. 2015.11.002 on Pithdoi:10.1016/j.jretai
- [27] Divyansh Kaushik, Eduard Hovy, and Zachary Lipton. 2020. Learning The Difference That Makes A Difference With Counterfactually-Augmented Data. In Proceedings of the International Conference on Learning Representations (ICLR). https://openreview.net/forum?id=Sklgs0NFvr on Pith
- [28] Klaus Krippendorff. 2011. Computing Krippendorff’s Alpha-Reliability. Depart- mental Papers (ASC), University of Pennsylvania. https://repository.upenn.edu/ handle/20.500.14332/2089 on Pith
- [29] Kaiqu Liang, Julia Kruk, Shengyi Qian, Xianjun Yang, Shengjie Bi, Yuanshun Yao, Shaoliang Nie, Mingyang Zhang, Lijuan Liu, Jaime Fernández Fisac, Shuyan Zhou, and Saghar Hosseini. 2026. Learning Personalized Agents from Human Feedback.arXiv preprint arXiv:2602.16173(2026) on PitharXiv
- [30] Bin Liu, Mads Schaarup Andersen, Florian Schaub, Hazim Almuhimedi, Shikun Zhang, Norman Sadeh, Alessandro Acquisti, and Yuvraj Agarwal. 2016. Fol- low My Recommendations: A Personalized Privacy Assistant for Mobile App Permissions. InProceedings of the Twelfth USENIX Conferenc... on Pith
- [31] Jiashen Liu. 2017. 515K Hotel Reviews Data in Europe. Kaggle. CC0 Public Domain. https://www.kaggle.com/datasets/jiashenliu/515k-hotel-reviews-data- in-europe, Accessed September 2026 on Pith
- [32] Alexandra E. Michael and Franziska Roesner. 2026. How Agents Ask for Per- mission: User Permissions for AI Agents, from Interfaces to Enforcement.arXiv preprint arXiv:2607.13718(2026) on PithPith's reviewarXiv
- [33] Moran Mizrahi, Guy Kaplan, Dan Malkin, Rotem Dror, Dafna Shahaf, and Gabriel Stanovsky. 2024. State of What Art? A Call for Multi-Prompt LLM Evaluation. Transactions of the Association for Computational Linguistics12 (2024), 933–949. doi:10.1162/tacl_a_00681 on Pithdoi:10.1162/tacl_a_00681
- [34] Raja Parasuraman, Thomas B. Sheridan, and Christopher D. Wickens. 2000. A Model for Types and Levels of Human Interaction with Automation.IEEE Transactions on Systems, Man, and Cybernetics - Part A: Systems and Humans30, 3 (2000), 286–297. doi:10.1109/3468.844354 on Pithdoi:10.1109/3468.844354
- [35] Ralph Peeters, Aaron Steiner, Luca Schwarz, Julian Yuya Caspary, and Christian Bizer. 2026. WebMall - A Multi-Shop Benchmark for Evaluating Web Agents. InProceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval. 3342–3348.... on Pithdoi:10.1145/3805712.3808592
- [36] Ethan Perez, Sam Ringer, Kamile Lukosiute, Karina Nguyen, Edwin Chen, et al on Pith
- [37] Conor Plunkett. 2026. PayBench: A Benchmark for Unsafe Commercial Auton- omy in AI Agents with Delegated Payment Authority. https://paybench.org/. Accessed September 2026 on Pith
- [38] Felipe Maia Polo, Ronald Xu, Lucas Weber, Mírian Silva, Onkar Bhardwaj, Leshem Choshen, Allysson Flavio Melo de Oliveira, Yuekai Sun, and Mikhail Yurochkin on Pith
- [39] Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin, and Sameer Singh. 2020. Beyond Accuracy: Behavioral Testing of NLP Models with CheckList. InProceed- ings of the 58th Annual Meeting of the Association for Computational Linguistics. 4902–4912. doi:10.18653/v1/2020.acl-main.442 on Pithdoi:10.18653/v1/2020.acl-main.442
- [40] Bibek Guha Sarkar and Ritu Mehta. 2026. Time Pressure and Shopping Behaviour in Retailing: A Review and Research Agenda.Spanish Journal of Marketing - ESIC(2026), 1–22. doi:10.1108/SJME-01-2025-0028 on Pithdoi:10.1108/sjme-01-2025-0028
- [41] Chinmay Savadikar, Mingyu Zhao, Yuanzheng Zhu, Han Li, Shuang Xie, Alberto Castelo, Tianfu Wu, and Lingyun Wang. 2026. ShopGym: An Integrated Frame- work for Realistic Simulation and Scalable Benchmarking of E-Commerce Web Agents. InProceedings of the ICML 2026 Workshop on RL ... on Pith
- [42] Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2024. Quantifying Language Models’ Sensitivity to Spurious Features in Prompt Design or: How I learned to start worrying about prompt formatting. InProceedings of the 12th International Conference on Learning Represent... on Pith
- [43] Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R. Bowman, Esin DURMUS, Zac Hatfield-Dodds, Scott R Johnston, Shauna M Kravec, Timothy Maxwell, Sam McCandlish, Kamal Ndousse, Oliver Rausch, Nicholas Schiefer, Da Yan, Miranda Zhang, and Ethan Pere... on Pith
- [44] Thomas B. Sheridan and William L. Verplank. 1978.Human and Computer Control of Undersea Teleoperators. Technical Report. MIT Man–Machine Systems Laboratory on Pith
- [45] Guanquan Shi, Haohua Du, Zhiqiang Wang, Xiaoyu Liang, Weiwenpei Liu, Song Bian, and Zhenyu Guan. 2025. SoK: Trust-Authorization Mismatch in LLM Agent Interactions.arXiv preprint arXiv:2512.06914(2025) on PitharXiv
- [46] Ryotaro Shimizu, Takashi Wada, Yu Wang, Johannes Kruse, Sean O’Brien, Sai HtaungKham, Linxin Song, Yuya Yoshikawa, Yuki Saito, Fugee Tsung, Masayuki Goto, and Julian McAuley. 2025. Disentangling Likes and Dislikes in Personalized Generative Explainable Recommendation. InProcee... on Pithdoi:10.1145/3696410.3714583
- [47] Manan Suri, Puneet Mathur, Nedim Lipka, Franck Dernoncourt, Ryan A. Rossi, and Dinesh Manocha. 2026. Structured Uncertainty guided Clarification for LLM Agents.arXiv preprint arXiv:2511.08798(2026) on PithPith's reviewarXiv
- [48] Emre Turan. 2026. Oversight Has a Capacity: Calibrating Agent Guards to a Subjective, Fatiguing Human.arXiv preprint arXiv:2606.08919(2026) on PithPith's reviewarXiv
- [49] Jiangyuan Wang, Kejun Xiao, Qi Sun, Huaipeng Zhao, Tao Luo, Jian Dong Zhang, and Xiaoyi Zeng. 2026. ShoppingBench: a real-world intent-grounded shopping benchmark for LLM-based agents. InProceedings of the 40th AAAI Conference on Artificial Intelligence and 38th Conference on ... on Pithdoi:10.1609/aaai.v40i39.40640
- [50] Lan Xia, Kent B. Monroe, and Jennifer L. Cox. 2004. The Price is Unfair! A Conceptual Framework of Price Fairness Perceptions.Journal of Marketing68, 4 (2004), 1–15. doi:10.1509/jmkg.68.4.1.42733 on Pithdoi:10.1509/jmkg.68.4.1.42733
- [51] Zhen Xiang, Linzhi Zheng, Yanjie Li, Junyuan Hong, Qinbin Li, Han Xie, Jiawei Zhang, Zidi Xiong, Chulin Xie, Carl Yang, Dawn Song, and Bo Li. 2025. GuardA- gent: safeguard LLM agents via knowledge-enabled reasoning. InProceedings of the 42nd International Conference on Machine... on Pith
- [52] Yue Xu, Qian Chen, Zizhan Ma, Dongrui Liu, Wenxuan Wang, Xiting Wang, Li Xiong, and Wenjie Wang. 2026. Toward Personalized LLM-Powered Agents: Foundations, Evaluation, and Future Directions.arXiv preprint arXiv:2602.22680 (2026) on PitharXiv
- [53] Ting Yan. 2026. Do User-Authored Permission Policies Improve Protection Against AI Agent Overreach?arXiv preprint arXiv:2608.27443(2026) on PithPith's reviewarXiv
- [54] Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. 2022. WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents. InAdvances in Neural Information Processing Systems, Vol. 35. 20744–20757. doi:10.52202/068431-1508 on Pithdoi:10.52202/068431-1508
- [55] Zijian Yu, Kejun Xiao, Huaipeng Zhao, Tao Luo, and Xiaoyi Zeng. 2026. Shop- ping Companion: Benchmarking and Training LLM Agents for Long-Horizon Preference-Grounded E-Commerce Tasks.arXiv preprint arXiv:2603.14864(2026) on PithPith's reviewarXiv
- [56] Weizhi Zhang, Wooseong Yang, Yuxin Cui, Zhaohui Guo, Hins Hu, Liangwei Yang, Henry Peng Zou, Qifei Wang, Hanqing Zeng, Jiayi Liu, Yinglong Xia, and Philip S. Yu. 2025. Toward User Preference Alignment in LLM Recommendation via Explicit Context Feedback. InProceedings of the 20... on Pith
- [57] Boyuan Zheng, Zeyi Liao, Scott Salisbury, Zeyuan Liu, Michael Lin, Qinyuan Zheng, Zifan Wang, Xiang Deng, Dawn Song, Huan Sun, and Yu Su. 2025. We- bGuard: Building a Generalizable Guardrail for Web Agents.arXiv preprint arXiv:2507.14293(2025) on PithPith's reviewarXiv
- [58] Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. 2024. WebArena: A Realistic Web Environment for Building Autonomous Agents. InProceedings of the 12th International Con... on Pith
- [59] Jinhao Zhu, Kevin Tseng, Gil Vernik, Xiao Huang, Shishir G. Patil, Vivian Fang, and Raluca Ada Popa. 2025. MiniScope: A Least Privilege Framework for Autho- rizing Tool Calling Agents.arXiv preprint arXiv:2512.11147(2025) on PithPith's reviewarXiv
- [60] Muhammad Zulhusni. 2026. Accenture: Consumers show growing trust in AI shopping agents. TechForge. https://www.artificialintelligence-news.com/ news/ai-shopping-agents-consumer-trust-accenture-report/, Accessed Septem- ber 2026 on Pith
- [61] David Mellafe Zuvic. 2026. Capability Gates Are Not Authorization: Confused- Deputy Failures in LLM Agent Frameworks.arXiv preprint arXiv:2606.28679 (2026). Table 3: The counterfactual cells. The top block changes one factor from the base (product and instruction held fixed), ... on PithPith's reviewarXiv
- [2023] Discovering Language Model Behaviors with Model-Written Evaluations. InFindings of the Association for Computational Linguistics: ACL 2023. 13387– 13434. doi:10.18653/v1/2023.findings-acl.847 on Pithdoi:10.18653/v1/2023.findings-acl.847
- [2024] Efficient multi-prompt evaluation of LLMs. InAdvances in Neural Informa- tion Processing Systems. https://openreview.net/forum?id=jzkpwcj200 on Pith
- [2026] What Is Your AI Agent Buying? Evaluation, Biases, Model Dependence, & Emerging Implications of Agentic E-Commerce. InProceedings of the ACM Web Conference 2026. 8697–8700. doi:10.1145/3774904.3792943 on Pithdoi:10.1145/3774904.3792943
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment