Pith. sign in

REVIEW 3 major objections 2 minor 3 cited by

TripTailor: A Real-World Benchmark for Personalized Travel Planning

T0 review · 3 major / 2 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read TripTailor introduces a real-world benchmark for personalized travel planning and reports that fewer than 10% of LLM-generated itineraries reach human-level quality.

desk verdict Abstract promises a travel-planning benchmark; the body is a Type Ia supernova paper, so there is nothing to referee. read the letter →

arxiv 2508.01432 v1 pith:JHZCNBZE submitted 2025-08-02 cs.AI

Yuanzhe Shen , Kaimin Wang , Changze Lv , Xiaoqing Zheng , Xuanjing Huang This is my paper
classification cs.AI
keywords TripTailorpersonalizedtravelplanningLLMbenchmarkitineraryevaluationpointsofinteresthuman-levelperformancefeasibilityrationalitypersonalization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces TripTailor, a benchmark for personalized travel planning built from over 500,000 real-world points of interest and nearly 4,000 human-written itineraries. Its headline result is that fewer than 10% of itineraries produced by state-of-the-art large language models meet human-level quality, with feasibility, rationality, and personalization identified as the main weak points. A sympathetic reader would take the intended contribution to be a realistic evaluation resource that lets the field measure travel-planning agents against actual human plans instead of synthetic constraints. A notable caveat is that the manuscript body is not this paper: the submitted full text is an unrelated Type Ia supernova analysis, so the dataset, annotation protocol, and experimental details that would support TripTailor are currently absent from the text.

What carries the argument

The paper's central object, as described in the abstract, is the TripTailor benchmark dataset: over 500,000 real-world points of interest with detailed information, paired with nearly 4,000 diverse human-written travel itineraries that serve as ground truth for 'human-level' quality. The working idea is that comparing LLM outputs against these human itineraries, rather than against synthetic constraint-satisfaction tasks, reveals the dimensions of quality that current systems miss—feasibility of the plan, rationality of the route and timing, and personalized customization to the user's stated needs. In the submitted manuscript, however, this machinery is described only in the abstract; the full text does not contain the dataset construction or evaluation details.

What would settle it

Open the linked repository and verify the claimed counts and contents: whether there are indeed 500,000+ POIs and nearly 4,000 human itineraries, whether the human plans are independent of the LLM outputs used in evaluation, and whether a fresh run of a state-of-the-art model yields a human-level rate below 10%. The clearest falsifier already on the record is the text mismatch: the full manuscript body is a supernova study, so the claimed benchmark's construction, annotation, and experiments are not present to be checked.

Watch

Extended reading notes

Core claim

On its own terms, the paper asserts that existing travel-planning benchmarks fail because they use simulated data and measure only constraint satisfaction, and that a benchmark built from real points of interest and diverse human itineraries gives a truer picture. The central discovery reported is the large gap: fewer than 10% of itineraries from current state-of-the-art LLMs are judged to reach human-level quality on this benchmark. The paper's intended contribution is the corpus itself—hundreds of thousands of real POIs with detailed metadata plus thousands of human itineraries—together with an evaluation design that scores feasibility, rationality, and personalization. Whether the assertions are supported by evidence cannot be checked from the supplied text, because the body of the manuscript is a separate astronomy paper.

Load-bearing premise

The load-bearing premise is that the TripTailor resource exists as described—500,000+ real points of interest, about 4,000 human itineraries, and a consistent scoring rubric—and that human-written itineraries are a valid ground truth for 'human-level' quality; the body of the paper currently shows none of this.

Editorial extensions

If this is right

  • If the benchmark works as described, travel-planning LLMs should be measured against human itineraries rather than by constraint-satisfaction scores alone.
  • The target implied by the paper is that a good itinerary must be simultaneously feasible, rational, and personalized; systems that only satisfy hard constraints will score poorly.
  • The reported under-10% human-level rate would imply that current LLMs are not ready to be deployed as travel planners without substantial additional mechanisms.
  • The benchmark could become a standard reproducible testbed for comparing future travel-planning agents, provided the dataset and code are made available.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the manuscript body is an unrelated astronomy paper, the public record should be treated as an unverified claim: the dataset statistics, human-annotation protocol, and the 10% result have not yet been shown in the text itself.
  • A useful check, were the dataset released, would be to see whether the human-rater judgments behind 'human-level quality' are stable across annotators; if not, the 10% threshold would be hard to interpret.
  • If reproducible, the result suggests travel planning needs hybrid systems—retrieval of real POI data and constraint checking around a generator—rather than end-to-end generation alone.
  • The text mismatch also raises a practical editorial point: readers and reviewers cannot confirm even the existence of the benchmark from this submission, so the contribution is properly assessed only after a corrected manuscript appears.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 2 minor

Summary. The manuscript as submitted consists of an abstract that introduces 'TripTailor,' a benchmark for personalized travel planning claimed to contain over 500,000 real-world points of interest and nearly 4,000 itineraries, with a headline result that fewer than 10% of itineraries generated by state-of-the-art LLMs achieve human-level performance. The full text, however, is an unrelated astrophysics paper on the Type Ia supernova SN 2024gy. No section of the body mentions TripTailor, travel planning, points of interest, itineraries, LLM baselines, human evaluation, or the benchmark's evaluation protocol. The central claims of the abstract are therefore entirely unsupported by the submitted manuscript.

Significance. If TripTailor existed as described, it could be a valuable community resource: a large-scale, real-world dataset with human-written reference itineraries would address a recognized gap in travel-planning evaluation, and the claim that current LLMs rarely match human-level quality would be a substantive, falsifiable result. However, none of this is demonstrated in the submitted text. There is no dataset description, no annotation protocol, no rubric definition for feasibility, rationality, and personalization, no experimental tables or figures, and no error bars or inter-annotator agreement statistics. The abstract's GitHub link is not a substitute for verifiable manuscript content. The potential significance of the claimed contribution cannot be assessed because the manuscript does not actually present the benchmark.

major comments (3)
  1. [Abstract vs. full text] The abstract's central empirical claim—that 'fewer than 10% of the itineraries generated by the latest state-of-the-art LLMs achieve human-level performance'—and the dataset statistics (over 500,000 POIs and nearly 4,000 itineraries) have no corresponding support in the body. The full text, from its title through its conclusion, is an unrelated paper on SN 2024gy and never mentions TripTailor, travel planning, POIs, itineraries, or LLM evaluation. This is the load-bearing claim of the submission, and it is unsupported by the submitted text.
  2. [Entire body] The manuscript contains no description of the evaluation methodology: no rubric for feasibility, rationality, or personalization, no human evaluation procedure, no number of generated itineraries per model, no baseline model names, no comparison tables or figures, and no uncertainty estimates. Even a generous reading of the abstract cannot make the reported '<10% human-level' result checkable, reproducible, or interpretable.
  3. [Abstract, final sentence] The abstract identifies 'several critical challenges in travel planning, including feasibility, rationality, and personalized customization,' but no section of the body presents experiments or analyses that yield these conclusions. These challenges are asserted rather than established.
minor comments (2)
  1. [Footer of full text] The full text carries the footer 'arXiv:2508.01428v2 [astro-ph.HE] 30 Oct 2025,' whereas the submission is under arXiv:2508.01432 (cs.AI). This identifier mismatch is a clear indication that the body of the submission is not the paper described in the abstract.
  2. [Figures and tables] All figure and table captions in the body pertain to photometric and spectroscopic observations of SN 2024gy; none refer to benchmark construction, model outputs, or evaluation results, further confirming the absence of any TripTailor content in the manuscript.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the submitted body is an unrelated supernova paper, so there is no TripTailor derivation chain whose inputs could be shown to equal its outputs.

full rationale

The abstract of the submitted paper claims that TripTailor is a benchmark with over 500,000 POIs, nearly 4,000 human-written itineraries, and that fewer than 10% of LLM-generated itineraries reach human-level performance. However, the supplied full text is an unrelated astrophysics paper, 'SN 2024gy: Multi-epoch Spectroscopic Features Suggestive of Delayed Detonation in a Type Ia Supernova,' carrying the inner footer 'arXiv:2508.01428v2 [astro-ph.HE] 30 Oct 2025.' No section of the body describes TripTailor, the POI collection, the itinerary annotation protocol, the evaluation rubric, or the experiments that produce the reported statistic. Consequently, there is no derivation chain in the submitted document for the benchmark claims, and no step can be exhibited in which a predicted quantity equals a fitted input by construction, a parameter is renamed as a prediction, or a load-bearing conclusion rests on a self-citation. Under the hard rule that circularity may only be claimed when the paper can be quoted to exhibit the specific reduction, no circular step is identifiable here. The absence of supporting text is a serious correctness and integrity concern, but it is not circularity. The structural risk that the benchmark authors might define the quality rubric and then measure 'human-level performance' against that same rubric remains hypothetical because the rubric does not appear in the submitted manuscript, so it cannot be used to support a circularity finding. The supernova analysis itself is outside the scope of this review and is not implicated in any circularity claim. Therefore the appropriate score is 0, with no circular steps reported.

Assumptions & free parameters 2 free parameters · 2 assumptions · 0 invented entities

The manuscript provides only the abstract for the claimed TripTailor work, so the ledger entries are the structural assumptions any benchmark of this type must carry: a hand-defined human-level reference, a rubric for itinerary quality, and representative real-world data. No explicit free parameters, axioms, or invented entities are observable in the submitted text because the benchmark's methodology section is absent entirely. The supernova body text contributes nothing to the ledger for the travel-planning claim.

free parameters (2)
  • human-level performance threshold
    The abstract's headline claim that fewer than 10% of LLM itineraries reach human-level quality depends on a rubric deciding what counts as human-level. That rubric is absent from the submitted text, so the threshold is a hand-defined criterion rather than an externally grounded one.
  • evaluation metric weights for feasibility, rationality, and personalization
    The abstract lists these as the critical challenges the benchmark targets, implying a composite scoring scheme, but no weights or scoring functions appear anywhere in the manuscript.
assumptions (2)
  • domain assumption Human-written itineraries in the dataset are valid ground truth for human-level travel plan quality.
    The 'human-level performance' comparison presumes that the collected human itineraries are a fair and correct reference standard; this enters through the abstract's framing and is not substantiated in the manuscript body.
  • domain assumption The 500,000 POIs and 4,000 itineraries accurately represent real-world travel-planning demand and are correctly curated.
    The benchmark's claim to be 'real-world' rests on the quality and representativeness of the underlying data collection, which is described only as headline counts in the abstract.

how reviews work

0 comments
Cite this review

Pith. "Pith review of TripTailor: A Real-World Benchmark for Personalized Travel Planning." pith.science (2026). https://pith.science/paper/JHZCNBZE

@misc{pith2026250801432,
  author       = {Pith},
  title        = {Pith review of: TripTailor: A Real-World Benchmark for Personalized Travel Planning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/JHZCNBZE}},
  note         = {Machine review of arXiv:2508.01432}
}
read the original abstract

The continuous evolution and enhanced reasoning capabilities of large language models (LLMs) have elevated their role in complex tasks, notably in travel planning, where demand for personalized, high-quality itineraries is rising. However, current benchmarks often rely on unrealistic simulated data, failing to reflect the differences between LLM-generated and real-world itineraries. Existing evaluation metrics, which primarily emphasize constraints, fall short of providing a comprehensive assessment of the overall quality of travel plans. To address these limitations, we introduce TripTailor, a benchmark designed specifically for personalized travel planning in real-world scenarios. This dataset features an extensive collection of over 500,000 real-world points of interest (POIs) and nearly 4,000 diverse travel itineraries, complete with detailed information, providing a more authentic evaluation framework. Experiments show that fewer than 10\% of the itineraries generated by the latest state-of-the-art LLMs achieve human-level performance. Moreover, we identify several critical challenges in travel planning, including the feasibility, rationality, and personalized customization of the proposed solutions. We hope that TripTailor will drive the development of travel planning agents capable of understanding and meeting user needs while generating practical itineraries. Our code and dataset are available at https://github.com/swxkfm/TripTailor

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. TREK: A Travel Reasoning and Evaluation Kit for LLM Agents in Complex Trip Planning

    cs.CL 2026-07 conditional novelty 6.5 of 10

    On 800 jointly constrained trip tasks with a deterministic scorer and achievable gold, the best of 15 LLM agents fully solves only 46.2% of feasible plans, with unstated persona needs as the universal bottleneck.

  2. AlterAtlas: Shifting Travel Planning from AI Generation to Validation via Persona-Driven Simulations

    cs.HC 2026-07 conditional novelty 6.0 of 10

    AlterAtlas replaces one-shot AI itinerary generation with an interactive validation loop where persona-driven simulations expose route-level constraints and guide iterative revision.

  3. Agentic AI for Trip Planning Optimization Application

    cs.AI 2026-04 unverdicted novelty 5.0 of 10

    An orchestrated multi-agent AI framework for trip planning optimization paired with a new ground-truth dataset achieves 77.4% accuracy on the TOP Benchmark, outperforming single-agent and workflow baselines.

Reference graph

Works this paper leans on

37 extracted references · 15 canonical work pages · cited by 3 Pith papers

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  4. [4]

    Maciej Besta, Nils Blach, Ales Kubicek, Robert Gerstenberger, Michal Podstawski, Lukas Gianinazzi, Joanna Gajda, Tomasz Lehmann, Hubert Niewiadomski, Piotr Nyczyk, et al. 2024. Graph of thoughts: Solving elaborate problems with large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 17682--17690

    work page 2024

  5. [5]

    Aili Chen, Xuyang Ge, Ziquan Fu, Yanghua Xiao, and Jiangjie Chen. 2024. Travelagent: An ai assistant for personalized travel planning. arXiv preprint arXiv:2409.08069

    arXiv 2024

  6. [6]

    Xiang Deng, Yu Gu, Boyuan Zheng, Shijie Chen, Sam Stevens, Boshi Wang, Huan Sun, and Yu Su. 2024. Mind2web: Towards a generalist agent for the web. Advances in Neural Information Processing Systems, 36

    work page 2024

  7. [7]

    Yu Du, Fangyun Wei, and Hongyang Zhang. 2024. Anytool: Self-reflective, hierarchical agents for large-scale api calls. arXiv preprint arXiv:2402.04253

    arXiv 2024

  8. [8]

    Atharva Gundawar, Mudit Verma, Lin Guan, Karthik Valmeekam, Siddhant Bhambri, and Subbarao Kambhampati. 2024. Robust planning with llm-modulo framework: Case study in travel planning. arXiv preprint arXiv:2405.20625

    arXiv 2024

Show all 37 references
  1. [9]

    Yilun Hao, Yongchao Chen, Yang Zhang, and Chuchu Fan. 2024. Large language models can plan your travels rigorously with formal verification tools. arXiv preprint arXiv:2404.11891

    2024 arXiv

  2. [10]

    Subbarao Kambhampati, Karthik Valmeekam, Lin Guan, Mudit Verma, Kaya Stechly, Siddhant Bhambri, Lucas Paul Saldyt, and Anil B Murthy. 2024. https://proceedings.mlr.press/v235/kambhampati24a.html Position: LLM s can’t plan, but can help planning in LLM -modulo frameworks . In P...

    2024

  3. [11]

    Layla AI, LLC . 2025. Ai travel agent | free itineraries | 2025. https://layla.ai/. Accessed on January 27, 2025

    2025

  4. [12]

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. 2024. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437

    2024 arXiv

  5. [13]

    Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, et al. 2023 a . Agentbench: Evaluating llms as agents. arXiv preprint arXiv:2308.03688

    2023 arXiv

  6. [14]

    Yiqi Liu, Nafise Sadat Moosavi, and Chenghua Lin. 2023 b . Llms as narcissistic evaluators: When ego inflates evaluation scores. arXiv preprint arXiv:2311.09766

    2023 arXiv

  7. [15]

    Adian Liusie, Potsawee Manakul, and Mark Gales. 2024. https://aclanthology.org/2024.eacl-long.8/ LLM comparative assessment: Zero-shot NLG evaluation through pairwise comparisons using large language models . In Proceedings of the 18th Conference of the European Chapter of the...

    2024

  8. [16]

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. 2023. Toolllm: Facilitating large language models to master 16000+ real-world apis. arXiv preprint arXiv:2307.16789

    2023 arXiv

  9. [17]

    Jianing Qiu, Kyle Lam, Guohao Li, Amish Acharya, Tien Yin Wong, Ara Darzi, Wu Yuan, and Eric J Topol. 2024. Llm-based agentic systems in medicine and healthcare. Nature Machine Intelligence, 6(12):1418--1420

    2024

  10. [18]

    Vyas Raina, Adian Liusie, and Mark Gales. 2024. Is llm-as-a-judge robust? investigating universal adversarial attacks on zero-shot llm assessment. arXiv preprint arXiv:2402.14016

    2024 arXiv

  11. [19]

    Roadtrippers, LLC . 2025. Road trip planner – build your itinerary and find the best stops. https://roadtrippers.com/. Accessed on January 27, 2025

    2025

  12. [20]

    Jingqing Ruan, Yihong Chen, Bin Zhang, Zhiwei Xu, Tianpeng Bao, Hangyu Mao, Ziyue Li, Xingyu Zeng, Rui Zhao, et al. 2023. Tptu: Task planning and tool usage of large language model-based ai agents. In NeurIPS 2023 Foundation Models for Decision Making Workshop

    2023

  13. [21]

    Jie-Jing Shao, Xiao-Wen Yang, Bo-Wen Zhang, Baizhi Chen, Wen-Da Wei, Lan-Zhe Guo, and Yu-feng Li. 2024. Chinatravel: A real-world benchmark for language agents in chinese travel planning. arXiv preprint arXiv:2412.13682

    2024 arXiv

  14. [22]

    Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. 2023. Hugging GPT : Solving AI tasks with chat GPT and its friends in hugging face. In Proceedings of NeurIPS

    2023

  15. [23]

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik R Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. In Proceedings of NeurIPS

    2023

  16. [24]

    Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg. 2023. Progprompt: Generating situated robot task plans using large language models. In 2023 IEEE International Conference on Robotics and Aut...

    2023

  17. [25]

    Lei Wang, Wanyu Xu, Yihuai Lan, Zhiqiang Hu, Yunshi Lan, Roy Ka-Wei Lee, and Ee-Peng Lim. 2023 a . Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models. In Proceedings of the 61st Annual Meeting of the Association for Computational ...

    2023

  18. [26]

    Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Qi Liu, Tianyu Liu, and Zhifang Sui. 2023 b . Large language models are not fair evaluators. arXiv preprint arXiv:2305.17926

    2023 arXiv

  19. [27]

    Ruoyao Wang, Peter Jansen, Marc-Alexandre C \^o t \'e , and Prithviraj Ammanabrolu. 2022. Scienceworld: Is your agent smarter than a 5th grader? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 11279--11298

    2022

  20. [28]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824--24837

    2022

  21. [29]

    Jian Xie, Kai Zhang, Jiangjie Chen, Tinghui Zhu, Renze Lou, Yuandong Tian, Yanghua Xiao, and Yu Su. 2024. Travelplanner: A benchmark for real-world planning with language agents. In Proceedings of the 41st International Conference on Machine Learning

    2024

  22. [30]

    Frank Xing. 2024. Designing heterogeneous llm agents for financial sentiment analysis. ACM Transactions on Management Information Systems

    2024

  23. [31]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. 2024. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115

    2024 arXiv

  24. [32]

    Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Tom Griffiths, Yuan Cao, and Karthik Narasimhan. 2024. Tree of thoughts: Deliberate problem solving with large language models. Advances in Neural Information Processing Systems, 36

    2024

  25. [33]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. In Proceedings of ICLR

    2022

  26. [34]

    Kechi Zhang, Jia Li, Ge Li, Xianjie Shi, and Zhi Jin. 2024 a . Codeagent: Enhancing code generation with tool-integrated agent systems for real-world repo-level coding challenges. arXiv preprint arXiv:2401.07339

    2024 arXiv

  27. [35]

    Zheyuan Zhang, Daniel Zhang-Li, Jifan Yu, Linlu Gong, Jinchang Zhou, Zhiyuan Liu, Lei Hou, and Juanzi Li. 2024 b . Simulating classroom education with llm-empowered agents. arXiv preprint arXiv:2406.19226

    2024 arXiv

  28. [36]

    Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, et al. 2023. Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems, 36:46595--46623

    2023

  29. [37]

    Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, et al. 2023. Webarena: A realistic web environment for building autonomous agents. arXiv preprint arXiv:2307.13854

    2023 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.