Pith. sign in

REVIEW 4 major objections 5 minor 1 cited by

AutoData: A Multi-Agent System for Open Web Data Collection

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read AutoData claims that a coordinated multi-agent system can turn a one-sentence data request into an accurate, ready-to-use dataset collected live from open web sources, outperforming general-purpose agents, programming assistants, and…

desk verdict Useful system and a genuinely new benchmark, but the headline comparison on Instruct2DS is not matched-set, so the central claim isn't established yet. read the letter →

arxiv 2505.15859 v1 pith:7K6BBF67 submitted 2025-05-21 cs.IR cs.AI

Tianyi Ma , Yiyue Qian , Zheyuan Zhang , Zehong Wang , Xiaoye Qian , Feifan Bai , Yifan Ding , Xuwei Luo
show 4 more authors
Shinan Zhang Keerthiram Murugesan Chuxu Zhang Yanfang Ye
This is my paper
classification cs.IRcs.AI
keywords multi-agentsystemopenwebdatacollectionnaturallanguageinstructionorientedhypergraphcacheLLMagentsscrapingInstruct2DSbenchmarktokencostreduction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

AutoData is designed to prove that open web data collection can be fully automated: the user gives a single natural-language instruction describing the dataset they want, and the system returns a ready-to-use dataset. The paper argues that existing solutions are either brittle wrappers requiring manual per-site engineering or expensive LLM-based agents that burn tokens, and it introduces a multi-agent system with eight specialized agents split into research and development squads, coordinated by a manager and a communication layer called OHCache. On a new benchmark, Instruct2DS, spanning academic, finance, and sports domains, AutoData reports the highest F1, precision, and recall in every domain while taking less implementation time and lower monetary expense than every baseline, including human programmers and the strongest general-purpose agents. If these results hold, dataset acquisition becomes much cheaper and more accessible for researchers, educators, journalists, and other non-experts.

What carries the argument

The carrying mechanism is OHCache, the oriented hypergraph cache system for agent communication. In OHCache, messages are modeled as oriented hyperedges of an eight-node hypergraph, each hyperedge directed from a single source agent to a targeted subset of receivers, so no message is broadcast to everyone and no agent processes an unbounded history. A formatter turns each agent output into a structured message before insertion, and a local cache node stores large artifacts, broadcasting only a cache identifier so raw HTML or API payloads never clog the message channel. This coordination layer is what keeps the multi-agent pipeline cheap enough to run, and the paper's ablations show that removing OHCache, the formatter, or the cache degrades performance and raises cost.

What would settle it

Independently rebuild the Instruct2DS ground truth for a random 50-task sample using the paper's published instruction templates, run AutoData on those same instructions, and check whether the reported F1 scores reproduce; separately, include tasks where the target web source is sparse or locked down, since the 'valid task' definition removes such cases from the reported averages.

Watch

Extended reading notes

Core claim

The central claim is that a single instruction can be turned end-to-end into an accurate, ready-to-use dataset from live web sources, and that the proposed system, AutoData, does this better and more cheaply than the alternatives it is compared with. The workflow is a two-stage division of labor: a research squad plans, browses the web, and consolidates a development blueprint, then a development squad implements, debugs, executes, and validates the collection program. The innovation that makes this efficient is OHCache, which replaces broadcast messaging among agents with an oriented message hypergraph—each message is a directed hyperedge from one source agent to a chosen set of recipients—and moves bulky artifacts such as HTML files into a local cache that agents fetch on demand. The paper reports that on its new Instruct2DS benchmark across academic, finance, and sports domains, AutoData surpasses every baseline in every domain with less implementation time and lower expense, and it presents supporting results on SWDE, EXTENDEDSWDE, HUMANEVAL, and two case studies to show the approach transfers to legacy extraction benchmarks, code generation, and specialized collection tasks.

Load-bearing premise

The entire evaluation assumes that the manually constructed Instruct2DS ground truth is complete and correct, and that restricting the benchmark to 'valid tasks' whose ground truth is non-empty does not systematically exclude precisely the hard cases where AutoData's advantage could shrink.

Editorial extensions

If this is right

  • Routine dataset requests—such as all accepted papers of a conference in a given year—can be executed by non-programmers, removing per-site wrapper engineering.
  • On Instruct2DS, the reported average is 5.58 minutes and $0.57 per task, versus 15.37 minutes and $2.49 for the runner-up general agent, making large-scale collection plausible.
  • Instruct2DS provides a public live-web benchmark with 234 sampled tasks over three domains, giving future agent systems a standard test rather than ad hoc scrapes.
  • The same architecture achieves 92.5 Pass@1 on HUMANEVAL with a GPT-4o backbone, suggesting the development squad is a capable general-purpose coder, not just a scraper.
  • Case studies on picture-book collection and survey-paper BibTeX show the system handles multi-level HTML crawling and per-source quirks with higher human-evaluated accuracy and lower cost than the best general agent.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the paper defines a 'valid task' as one with non-empty ground truth, the reported F1 numbers cover only cases where the web source contains the requested data; a stress test that deliberately includes sparse, login-walled, or JavaScript-rendered sources could expose a different ranking.
  • The oriented-hypergraph routing idea is general: any multi-agent workflow where a few agents share an artifact—software development, literature review, data curation—could adopt the targeted-hyperedge plus local-cache pattern to cut token costs, independent of web scraping.
  • The cost figures are tied to proprietary LLM pricing; the paper's own ablations show an open-weight backbone performs markedly worse, so the dollar-and-minute advantages should be treated as conditional on the chosen model and vendor rates.
  • Releasing the Instruct2DS database construction algorithm and the full sampled task list would let others independently recompute the ground truth; the paper asserts manual cross-validation but publishes neither the database nor the algorithm.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes AutoData, a multi-agent system that takes a natural-language instruction and produces a dataset by orchestrating eight specialized agents in research and development squads, coordinated by a central manager and communicating through a novel 'oriented hypergraph cache system' (OHCache). The authors also introduce Instruct2DS, a new benchmark for open web data collection with 234 tasks in academic, finance, and sports domains. The system is evaluated on Instruct2DS, SWDE, EXTENDED SWDE, and HumanEval, with additional case studies on picture-book collection and paper collection from surveys. The central claim in Section 5.1 is that AutoData surpasses all baseline methods in every domain of Instruct2DS while requiring less implementation time and lower expense.

Significance. If the reported results hold, the paper would make a useful contribution: a low-cost, instruction-driven data-collection system, a new live-web benchmark, and a communication architecture that addresses token-cost and information-overload problems in multi-agent systems. The authors are to be credited for evaluating against external ground truth, comparing with a wide range of baselines including human programmers and commercial agents, and providing source-code and dataset links. The case studies, ablation experiments, and LLM-backbone sensitivity analysis (Appendix E.1) also add useful evidence. However, the significance is currently contingent on the fairness and completeness of the evaluation, which has several load-bearing weaknesses discussed below.

major comments (4)
  1. [Section 5.1, Table 1, Appendix C, Appendix D.4] The headline comparison is not made on a matched task set. AutoData is scored on all 234 Instruct2DS tasks (Appendix D.4), while Appendix C states that Manus and the human-programmer baseline were run on 'a representative subset of tasks' with no subset size, task IDs, or sampling procedure. Since the central claim in Section 5.1 is explicitly relative ('surpasses all baseline methods in every domain'), the comparison is unverified unless all rows of Table 1 are computed over the same task set. The time and expense comparisons are affected equally because they are aggregate quantities. The authors should either evaluate all baselines on the full 234-task set or restrict all methods, including AutoData, to a single matched subset, and report the exact tasks used.
  2. [Appendix D.4] The Instruct2DS evaluation samples only 'valid tasks,' defined as instructions that yield non-empty ground truth, which excludes instructions for which the web source does not contain the requested data or for which extraction is difficult. This is a form of selection bias in the benchmark itself: the reported F1 scores are conditional on data being accessible, and the excluded cases could be systematically harder for AutoData than for other methods. The authors should quantify how many generated instructions were discarded as invalid, report task-level success/failure rates, and show that the conclusions are robust when invalid tasks are included or when the validity criterion is applied equally across all methods.
  3. [Tables 1, 2, 4, 5 and Figure 3] All results are reported as single point estimates without error bars, confidence intervals, or significance tests. Because AutoData and the baselines are LLM-based and stochastic, and because the reported differences are often small (for example, Table 2 shows AutoData and Manus within 0.03 F1 on SWDE), single-run numbers do not support the claim of superiority. The authors should provide multiple runs (at least three to five) with means and standard deviations, or bootstrap confidence intervals, and where feasible paired tests over the same task set. The same issue applies to the ablation study in Figure 3, which is presented only graphically and without numerical values or variance information.
  4. [Section 5.4, Figure 3] The ablation study is not reported quantitatively. The text states that removing components or agent squads leads to 'notable decline in performance' or 'marginal' changes, but Figure 3 contains no numerical values, no task-level breakdown, and no indication of how many tasks were used. Since the OHCache architecture is a central claimed contribution, its contribution should be documented with a table of F1, precision, recall, time, and cost for each ablation condition on the same task set used in Table 1.
minor comments (5)
  1. [Table 1] The Human row reports a time value but no expense, while the surrounding text claims AutoData incurs 'lower expenses' than existing methods; please clarify the expense basis for the Human baseline and for each method.
  2. [Table 2] Several numeric cells appear to be malformed, for example the COT row shows '2.447.32', which is likely two numbers (2.44 and 7.32) concatenated without a separator. Please fix the table formatting.
  3. [Appendix A.4] There are typos such as 'Baseketball' for 'Basketball' and, in Figure 6, 'corporate' appears as 'corporte' and 'distributed' as 'distrbuted'. Please proofread the appendices.
  4. [Appendix D.1] The experiments are said to use four NVIDIA A40 GPUs, but the method is implemented via LLM API calls with a GPT-4o backbone; please clarify what computation is actually performed on the local GPUs.
  5. [Appendix D.2] The definition of 'time' excludes program execution time, so the reported time values reflect only the interval from instruction release to the start of data collection; this should be stated prominently in the main text because it materially affects the interpretation of the efficiency comparison.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: AutoData's central claims rest on external benchmark comparisons and independently constructed ground truth, not on its own outputs or self-citations.

full rationale

The paper contains no derivation chain in which a predicted quantity is equivalent to an input by construction. AutoData is a multi-agent system whose outputs (collected datasets) are compared against ground truth that is constructed independently of the system: Section 4.1 states that the database D serves as the authoritative source for GT-DS and that 'MAS methods are provided only with the task instruction Instruct and have no access to the underlying database.' Appendix A.1 repeats that 'all models merely take Instruct as input, and have no access to the data in Instruct2DS in the collection process.' The evaluation is therefore self-contained with respect to external evidence rather than circular. Additional evaluation on SWDE, EXTENDEDSWDE, and HumanEval uses pre-existing benchmarks and standard metrics, and the paper reports following prior experimental setups (AutoScraper, MetaGPT), which further anchors the comparisons outside the paper's own fitted values. The several self-citations in the paper (e.g., references to the authors' earlier graph and hypergraph learning papers, and to their previous data collection work) appear only as background or motivation and do not carry the central effectiveness claim; there is no invocation of a uniqueness theorem, no ansatz smuggled in via self-citation, and no renaming of a known result as a new derivation. The main validity concern identified by the skeptic is that Manus and human baselines were run on 'a representative subset of tasks' (Appendix C) while Table 1 appears to report single aggregate numbers, so the matched-task-set comparison may be incomplete. That is an experimental-fairness and reproducibility concern, not a circularity concern: even if the baseline subset differs, AutoData's reported outputs are still measured against external ground truth rather than being forced by the system's own definitions or fitted parameters. No passage in the manuscript asserts a circular step, a missing supporting derivation, or a load-bearing unverified self-citation, and Appendix G candidly lists limitations (proprietary LLM dependence, inaccessible websites, legal risk, text-only scope) without any admission of circularity. Therefore no significant circularity is present, and the appropriate score is 0.

Assumptions & free parameters 2 free parameters · 4 assumptions · 0 invented entities

No free parameters are fitted to the evaluation data in the model sense; the listed items are experimental design choices that affect the reported numbers. The axioms are the background assumptions the system's performance depends on, mainly about LLM reliability, web stability, and ground truth correctness. No new physical or conceptual entities with independent evidence are introduced; OHCache is a software component.

free parameters (2)
  • valid task criterion = non-empty ground truth
    Appendix D.4: only tasks yielding non-empty ground truth are included; this hand-chosen inclusion rule affects reported F1 and could inflate performance.
  • human evaluation sample size = 500
    Appendix D.3: randomly draw 500 samples for evaluation; chosen by hand, affects certainty of reported human scores.
assumptions (4)
  • domain assumption LLM backends (GPT-4o) reliably follow ReAct prompts and generate correct code
    Section 3.1: each agent executes via Eq. 1; if LLM reasoning/code generation is unreliable, the system's performance collapses.
  • domain assumption Target web sources are publicly accessible and stable
    Section 4.2 and Appendix A.1: authors choose domains with 'static web sources' to ensure reproducibility; if sites change, ground truth and collection become stale.
  • domain assumption Ground truth construction algorithm g is correct
    Section 4.1, Eq. 6: GT-DS = g(T_i, P, D); correctness of g is asserted via manual cross-validation, but no formal proof or release of the algorithm is provided.
  • domain assumption Evaluation metrics (F1, precision, recall) computed on rounded numerical attributes
    Appendix D.2: all numerical attributes rounded to four decimals; rounding may affect exact-match F1.

how reviews work

0 comments
Cite this review

Pith. "Pith review of AutoData: A Multi-Agent System for Open Web Data Collection." pith.science (2026). https://pith.science/paper/7K6BBF67

@misc{pith2026250515859,
  author       = {Pith},
  title        = {Pith review of: AutoData: A Multi-Agent System for Open Web Data Collection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7K6BBF67}},
  note         = {Machine review of arXiv:2505.15859}
}
read the original abstract

The exponential growth of data-driven systems and AI technologies has intensified the demand for high-quality web-sourced datasets. While existing datasets have proven valuable, conventional web data collection approaches face significant limitations in terms of human effort and scalability. Current data-collecting solutions fall into two categories: wrapper-based methods that struggle with adaptability and reproducibility, and large language model (LLM)-based approaches that incur substantial computational and financial costs. To address these challenges, we propose AutoData, a novel multi-agent system for Automated web Data collection, that requires minimal human intervention, i.e., only necessitating a natural language instruction specifying the desired dataset. In addition, AutoData is designed with a robust multi-agent architecture, featuring a novel oriented message hypergraph coordinated by a central task manager, to efficiently organize agents across research and development squads. Besides, we introduce a novel hypergraph cache system to advance the multi-agent collaboration process that enables efficient automated data collection and mitigates the token cost issues prevalent in existing LLM-based systems. Moreover, we introduce Instruct2DS, a new benchmark dataset supporting live data collection from web sources across three domains: academic, finance, and sports. Comprehensive evaluations over Instruct2DS and three existing benchmark datasets demonstrate AutoData's superior performance compared to baseline methods. Case studies on challenging tasks such as picture book collection and paper extraction from surveys further validate its applicability. Our source code and dataset are available at https://github.com/GraphResearcher/AutoData.

Figures

Figures reproduced from arXiv: 2505.15859 by the authors.

Figure 1
Figure 1. The overall framework of AutoData. With the input instruction, agents in the research squad collaborate to generate a development blueprint by browsing the web pages. Afterward, the development squad builds the program based on the blueprint and executes the program to obtain the desired dataset. To ensure efficient and effective multi-agent collaboration, we introduce a novel oriented hypergraph cache system for in… view at source ↗
Figure 2
Figure 2. A sample in Instruct2DS and examples of instruction templates [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Ablation studies for AutoData. We conduct an ablation study to analyze the contributions of agents and components in OHCache, as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figures from the paper (7 more)
Figure 4
Figure 4. Figure 4: Example of instruction Instruct and a sample in corresponding GT-DS . There are several concerns in benchmark dataset construction: • Static of Web Sources: Due to the continuous evolution of web content, ground truth data may change over time, potentially undermining …
Figure 5
Figure 5. Figure 5: To systematically evaluate the performance of models for data collection tasks in academic [PITH_FULL_IMAGE:figures/full_fig_p016_5.png]
Figure 6
Figure 6. Figure 6: Stock data structure and instruction templates. [PITH_FULL_IMAGE:figures/full_fig_p017_6.png]
Figure 7
Figure 7. Figure 7: NBA data structure at the team level and instruction templates. [PITH_FULL_IMAGE:figures/full_fig_p017_7.png]
Figure 8
Figure 8. Figure 8: NBA data structure at player level and instruction template. [PITH_FULL_IMAGE:figures/full_fig_p018_8.png]
Figure 9
Figure 9. Figure 9: MLB hitting data structure and instruction templates. [PITH_FULL_IMAGE:figures/full_fig_p019_9.png]
Figure 10
Figure 10. Figure 10: MLB pitching data structure and instruction templates. [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Source or It Didn't Happen: A Multi-Agent Framework for Citation Hallucination Detection

    cs.CL 2026-05 conditional novelty 7.0 of 10

    CiteTracer detects hallucinated citations with a 12-code field-level taxonomy and a cascading multi-agent retrieval pipeline, reporting 97.1% accuracy on a synthetic benchmark.

Reference graph

Works this paper leans on

73 extracted references · 49 canonical work pages · cited by 1 Pith paper

  1. [1]

    Drivingdojo dataset: Advancing interactive and knowledge-enriched driving world model

    Yuqi Wang, Ke Cheng, Jiawei He, Qitai Wang, Hengchen Dai, Yuntao Chen, Fei Xia, and Zhao- Xiang Zhang. Drivingdojo dataset: Advancing interactive and knowledge-enriched driving world model. InNeurIPS, 2024

    work page 2024

  2. [2]

    Olympicarena: Benchmarking multi-discipline cognitive reasoning for superintelligent ai

    Zhen Huang, Zengzhi Wang, Shijie Xia, Xuefeng Li, Haoyang Zou, Ruijie Xu, Run-Ze Fan, Lyumanshan Ye, Ethan Chern, Yixin Ye, et al. Olympicarena: Benchmarking multi-discipline cognitive reasoning for superintelligent ai. InNeurIPS, 2024

    work page 2024

  3. [3]

    Adaptive graph enhancement for imbalanced multi-relation graph learning

    Yiyue Qian, Tianyi Ma, Chuxu Zhang, and Yanfang Ye. Adaptive graph enhancement for imbalanced multi-relation graph learning. InCIKM, 2025

    work page 2025

  4. [4]

    The fineweb datasets: Decanting the web for the finest text data at scale

    Guilherme Penedo, Hynek Kydlí ˇcek, Anton Lozhkov, Margaret Mitchell, Colin A Raffel, Leandro V on Werra, Thomas Wolf, et al. The fineweb datasets: Decanting the web for the finest text data at scale. InNeurIPS, 2024

    work page 2024

  5. [5]

    Diet-odin: A novel framework for opioid misuse detection with interpretable dietary patterns

    Zheyuan Zhang, Zehong Wang, Shifu Hou, Evan Hall, Landon Bachman, Jasmine White, Vincent Galassi, Nitesh V Chawla, Chuxu Zhang, and Yanfang Ye. Diet-odin: A novel framework for opioid misuse detection with interpretable dietary patterns. InKDD, 2024

    work page 2024

  6. [6]

    Openwebmath: An open dataset of high-quality mathematical web text

    Keiran Paster, Marco Dos Santos, Zhangir Azerbayev, and Jimmy Ba. Openwebmath: An open dataset of high-quality mathematical web text. InICLR, 2023

    work page 2023

  7. [7]

    Dual-level hypergraph contrastive learning with adaptive temperature enhancement

    Yiyue Qian, Tianyi Ma, Chuxu Zhang, and Yanfang Ye. Dual-level hypergraph contrastive learning with adaptive temperature enhancement. InWWW, 2024

    work page 2024

  8. [8]

    Hypergraph contrastive learning for drug trafficking community detection

    Tianyi Ma, Yiyue Qian, Chuxu Zhang, and Yanfang Ye. Hypergraph contrastive learning for drug trafficking community detection. InICDM, 2023

    work page 2023

Show all 73 references
  1. [9]

    Ngqa: A nutritional graph question answering benchmark for personalized health-aware nutritional reasoning.arXiv preprint arXiv:2412.15547, 2024

    Zheyuan Zhang, Yiyang Li, Nhi Ha Lan Le, Zehong Wang, Tianyi Ma, Vincent Galassi, Keerthiram Murugesan, Nuno Moniz, Werner Geyer, Nitesh V Chawla, et al. Ngqa: A nutritional graph question answering benchmark for personalized health-aware nutritional reasoning.arXiv preprint a...

    2024 arXiv

  2. [10]

    Can llms convert graphs to text-attributed graphs? InNACCL, 2025

    Zehong Wang, Sidney Liu, Zheyuan Zhang, Tianyi Ma, Chuxu Zhang, and Yanfang Ye. Can llms convert graphs to text-attributed graphs? InNACCL, 2025

    2025

  3. [11]

    Automatic dataset construction (adc): Sample collection, data curation, and beyond.arXiv preprint arXiv:2408.11338, 2024

    Minghao Liu, Zonglin Di, Jiaheng Wei, Zhongruo Wang, Hengxiang Zhang, Ruixuan Xiao, Haoyu Wang, Jinlong Pang, Hao Chen, Ankit Shah, et al. Automatic dataset construction (adc): Sample collection, data curation, and beyond.arXiv preprint arXiv:2408.11338, 2024

    2024 arXiv

  4. [12]

    Tool-planner: Task planning with clusters across multiple tools

    Yanming Liu, Xinyue Peng, Jiannan Cao, Shi Bo, Yuwei Zhang, Xuhong Zhang, Sheng Cheng, Xun Wang, Jianwei Yin, and Tianyu Du. Tool-planner: Task planning with clusters across multiple tools. InICLR, 2025

    2025

  5. [13]

    Tool learning with foundation models.ACM Comput

    Yujia Qin, Shengding Hu, Yankai Lin, Weize Chen, Ning Ding, Ganqu Cui, Zheni Zeng, Xuanhe Zhou, Yufei Huang, Chaojun Xiao, Chi Han, Yi Ren Fung, Yusheng Su, Huadong Wang, Cheng Qian, Runchu Tian, Kunlun Zhu, Shihao Liang, Xingyu Shen, Bokai Xu, Zhen Zhang, Yining Ye, Bowen Li,...

    2024

  6. [14]

    Llm-empowered class imbalanced graph prompt learning for online drug trafficking detection

    Tianyi Ma, Yiyue Qian, Zehong Wang, Zheyuan Zhang, Shinan Zhang, Chuxu Zhang, and Yanfang Ye. Llm-empowered class imbalanced graph prompt learning for online drug trafficking detection. InACL, 2025

    2025

  7. [15]

    From one tree to a forest: a unified solution for structured web data extraction

    Qiang Hao, Rui Cai, Yanwei Pang, and Lei Zhang. From one tree to a forest: a unified solution for structured web data extraction. InSIGIR, 2011

    2011

  8. [16]

    Synthesis of forgiving data extractors

    Adi Omari, Sharon Shoham, and Eran Yahav. Synthesis of forgiving data extractors. InWSDM, pages 385–394, 2017. 10

    2017

  9. [17]

    Adaptive expansion for hypergraph learning.arXiv preprint arXiv:2502.15564, 2025

    Tianyi Ma, Yiyue Qian, Shinan Zhang, Chuxu Zhang, and Yanfang Ye. Adaptive expansion for hypergraph learning.arXiv preprint arXiv:2502.15564, 2025

    2025 arXiv

  10. [18]

    Towards learning generalities across graphs via task-trees

    Zehong Wang, Zheyuan Zhang, Tianyi Ma, Nitesh V Chawla, Chuxu Zhang, and Yanfang Ye. Towards learning generalities across graphs via task-trees. InICML, 2025

    2025

  11. [19]

    Neural graph pattern machine

    Zehong Wang, Zheyuan Zhang, Tianyi Ma, Nitesh V Chawla, Chuxu Zhang, and Yanfang Ye. Neural graph pattern machine. InICML, 2025

    2025

  12. [20]

    Toolllm: Facilitating large language models to master 16000+ real-world apis

    Yujia Qin, Shihao Liang, Yining Ye, Kunlun Zhu, Lan Yan, Yaxi Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, et al. Toolllm: Facilitating large language models to master 16000+ real-world apis. InICLR, 2024

    2024

  13. [21]

    O’Reilly Media, Inc., 2018

    Ryan Mitchell.Web scraping with Python: Collecting more data from the modern web. O’Reilly Media, Inc., 2018

    2018

  14. [22]

    Web scraping technologies in an api world.Briefings in bioinformatics, 15(5):788–797, 2014

    Daniel Glez-Peña, Anália Lourenço, Hugo López-Fernández, Miguel Reboiro-Jato, and Flo- rentino Fdez-Riverola. Web scraping technologies in an api world.Briefings in bioinformatics, 15(5):788–797, 2014

    2014

  15. [23]

    Web scraping or web crawling: State of art, techniques, approaches and application.International Journal of Advances in Soft Computing & Its Applications, 13(3), 2021

    Moaiad Ahmad Khder. Web scraping or web crawling: State of art, techniques, approaches and application.International Journal of Advances in Soft Computing & Its Applications, 13(3), 2021

    2021

  16. [24]

    Datasets for large language models: A comprehensive survey.arXiv preprint arXiv:2402.18041, 2024

    Yang Liu, Jiahuan Cao, Chongyu Liu, Kai Ding, and Lianwen Jin. Datasets for large language models: A comprehensive survey.arXiv preprint arXiv:2402.18041, 2024

    2024 arXiv

  17. [25]

    ChatDev: Communicative agents for software development

    Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, Juyuan Xu, Dahai Li, Zhiyuan Liu, and Maosong Sun. ChatDev: Communicative agents for software development. InACL, 2024

    2024

  18. [26]

    MetaGPT: Meta programming for a multi-agent collaborative framework

    Sirui Hong, Mingchen Zhuge, Jonathan Chen, Xiawu Zheng, Yuheng Cheng, Jinlin Wang, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, Lingfeng Xiao, Chenglin Wu, and Jürgen Schmidhuber. MetaGPT: Meta programming for a multi-agent collaborative fr...

  19. [27]

    Self-collaboration code generation via chatgpt

    Yihong Dong, Xue Jiang, Zhi Jin, and Ge Li. Self-collaboration code generation via chatgpt. ACM Transactions on Software Engineering and Methodology, 33(7):1–38, 2024

    2024

  20. [28]

    Improving factuality and reasoning in language models through multiagent debate

    Yilun Du, Shuang Li, Antonio Torralba, Joshua B Tenenbaum, and Igor Mordatch. Improving factuality and reasoning in language models through multiagent debate. InICML, 2023

    2023

  21. [29]

    Chateval: Towards better llm-based evaluators through multi-agent debate

    Chi-Min Chan, Weize Chen, Yusheng Su, Jianxuan Yu, Wei Xue, Shanghang Zhang, Jie Fu, and Zhiyuan Liu. Chateval: Towards better llm-based evaluators through multi-agent debate. In ICLR, 2024

    2024

  22. [30]

    Agentcf: Collaborative learning with autonomous language agents for recommender systems

    Junjie Zhang, Yupeng Hou, Ruobing Xie, Wenqi Sun, Julian McAuley, Wayne Xin Zhao, Leyu Lin, and Ji-Rong Wen. Agentcf: Collaborative learning with autonomous language agents for recommender systems. InWWW, 2024

    2024

  23. [31]

    Mopi-hfrs: A multi-objective personalized health-aware food recommendation system with llm-enhanced interpretation

    Zheyuan Zhang, Zehong Wang, Tianyi Ma, Varun Sameer Taneja, Sofia Nelson, Nhi Ha Lan Le, Keerthiram Murugesan, Mingxuan Ju, Nitesh V Chawla, Chuxu Zhang, et al. Mopi-hfrs: A multi-objective personalized health-aware food recommendation system with llm-enhanced interpretation. ...

    2025

  24. [32]

    React: Synergizing reasoning and acting in language models

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. InICLR, 2023

    2023

  25. [33]

    Reflective multi-agent collaboration based on large language models

    Xiaohe Bo, Zeyu Zhang, Quanyu Dai, Xueyang Feng, Lei Wang, Rui Li, Xu Chen, and Ji-Rong Wen. Reflective multi-agent collaboration based on large language models. InNeurIPS, 2024

    2024

  26. [34]

    Camel: Communicative agents for" mind" exploration of large scale language model society

    Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for" mind" exploration of large scale language model society. InNeurIPS, 2023. 11

    2023

  27. [35]

    Are longer prompts always bet- ter? prompt selection in large language models for recommendation systems.arXiv preprint arXiv:2412.14454, 2024

    Genki Kusano, Kosuke Akimoto, and Kunihiro Takeoka. Are longer prompts always bet- ter? prompt selection in large language models for recommendation systems.arXiv preprint arXiv:2412.14454, 2024

    2024 arXiv

  28. [36]

    Multi-agent discussion mechanism for natural language generation

    Xu Li, Mingming Sun, and Ping Li. Multi-agent discussion mechanism for natural language generation. InAAAI, 2019

    2019

  29. [37]

    Autogen: Enabling next-gen llm applications via multi-agent conversation.arXiv preprint arXiv:2308.08155, 2023

    Qingyun Wu, Gagan Bansal, Jieyu Zhang, Yiran Wu, Beibin Li, Erkang Zhu, Li Jiang, Xiaoyun Zhang, Shaokun Zhang, Jiale Liu, et al. Autogen: Enabling next-gen llm applications via multi-agent conversation.arXiv preprint arXiv:2308.08155, 2023

    2023 arXiv

  30. [38]

    Generative agents: Interactive simulacra of human behavior

    Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. InUIST, 2023

    2023

  31. [39]

    Building cooperative embodied agents modularly with large language models

    Hongxin Zhang, Weihua Du, Jiaming Shan, Qinhong Zhou, Yilun Du, Joshua B Tenenbaum, Tianmin Shu, and Chuang Gan. Building cooperative embodied agents modularly with large language models. InICLR, 2024

    2024

  32. [40]

    Openceres: When open information extraction meets the semi-structured web

    Colin Lockard, Prashant Shiralkar, and Xin Luna Dong. Openceres: When open information extraction meets the semi-structured web. InNAACL, 2019

    2019

  33. [41]

    Zeroshotceres: Zero-shot relation extraction from semi-structured webpages

    Colin Lockard, Prashant Shiralkar, Xin Luna Dong, and Hannaneh Hajishirzi. Zeroshotceres: Zero-shot relation extraction from semi-structured webpages. InACL, 2020

    2020

  34. [42]

    Leave it to manus, 2025

    Manus Team. Leave it to manus, 2025. URLhttps://manus.im

    2025

  35. [43]

    Autoagent: A fully-automated and zero-code framework for llm agents.arXiv e-prints, pages arXiv–2502, 2025

    Jiabin Tang, Tianyu Fan, and Chao Huang. Autoagent: A fully-automated and zero-code framework for llm agents.arXiv e-prints, pages arXiv–2502, 2025

    2025

  36. [44]

    Openmanus: An open-source framework for building general ai agents, 2025

    Xinbin Liang, Jinyu Xiang, Zhaoyang Yu, Jiayi Zhang, Sirui Hong, Sheng Fan, and Xiao Tang. Openmanus: An open-source framework for building general ai agents, 2025. URL https://doi.org/10.5281/zenodo.15186407

    2025 doi

  37. [45]

    Autoagents: The automatic agents generation framework

    Guangyao Chen, Siwei Dong, Yu Shu, Ge Zhang, Sesay Jaward, Karlsson Börje, Jie Fu, and Yemin Shi. Autoagents: The automatic agents generation framework. InIJCAI, 2024

    2024

  38. [46]

    Cline, the collaborative ai coder, 2024

    Cline Team. Cline, the collaborative ai coder, 2024. URLhttps://cline.bot/

    2024

  39. [47]

    Cursor ide, 2025

    Cursor Team. Cursor ide, 2025. URLhttps://cursor.com. Web-based IDE

    2025

  40. [48]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. In NeurIPS, 2022

    2022

  41. [49]

    Reflexion: Language agents with verbal reinforcement learning

    Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. InNeurIPS, 2023

    2023

  42. [50]

    Autoscraper: A progressive understanding web agent for web scraper generation

    Wenhao Huang, Zhouhong Gu, Chenghao Peng, Zhixu Li, Jiaqing Liang, Yanghua Xiao, Liqian Wen, and Zulong Chen. Autoscraper: A progressive understanding web agent for web scraper generation. InEMNLP, 2024

    2024

  43. [51]

    Palm: Scaling language modeling with pathways.Journal of Machine Learning Research, 24(240): 1–113, 2023

    Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. Palm: Scaling language modeling with pathways.Journal of Machine Learning Research, 24(240): 1–113, 2023

    2023

  44. [52]

    Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021

    Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. Evaluating large language models trained on code.arXiv preprint arXiv:2107.03374, 2021

    2021 arXiv

  45. [53]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023. 12

    2023 arXiv

  46. [54]

    Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card.arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  47. [55]

    Verbal and visual literacy: the role of picturebooks in the reading experience of young children.Handbook of early childhood literacy, pages 235–248, 2003

    Maria Nikolajeva. Verbal and visual literacy: the role of picturebooks in the reading experience of young children.Handbook of early childhood literacy, pages 235–248, 2003

    2003

  48. [56]

    Mirrors, windows, and sliding glass doors

    Rudine Sims Bishop. Mirrors, windows, and sliding glass doors. perspectives: Choosing and using books for the classroom, 6 (3).Perspectives: Choosing and using books for the classroom, 6(3):ix–xi, 1990

    1990

  49. [57]

    https://dlp.lib.miamioh.edu/ picturebook/

    Children’s picture book database at miami university. https://dlp.lib.miamioh.edu/ picturebook/

  50. [58]

    A survey on the memory mechanism of large language model based agents

    Zeyu Zhang, Xiaohe Bo, Chen Ma, Rui Li, Xu Chen, Quanyu Dai, Jieming Zhu, Zhenhua Dong, and Ji-Rong Wen. A survey on the memory mechanism of large language model based agents. arXiv preprint arXiv:2404.13501, 2024

    2024 arXiv

  51. [59]

    From llms to llm-based agents for software engineering: A survey of current, challenges and future.arXiv preprint arXiv:2408.02479, 2024

    Haolin Jin, Linghan Huang, Haipeng Cai, Jun Yan, Bo Li, and Huaming Chen. From llms to llm-based agents for software engineering: A survey of current, challenges and future.arXiv preprint arXiv:2408.02479, 2024

    2024 arXiv

  52. [60]

    Large language model based multi-agents: A survey of progress and challenges.arXiv preprint arXiv:2402.01680, 2024

    Taicheng Guo, Xiuying Chen, Yaqi Wang, Ruidi Chang, Shichao Pei, Nitesh V Chawla, Olaf Wiest, and Xiangliang Zhang. Large language model based multi-agents: A survey of progress and challenges.arXiv preprint arXiv:2402.01680, 2024

    2024 arXiv

  53. [61]

    Multi-agent coordination across diverse applications: A survey.arXiv preprint arXiv:2502.14743, 2025

    Lijun Sun, Yijun Yang, Qiqi Duan, Yuhui Shi, Chao Lyu, Yu-Cheng Chang, Chin-Teng Lin, and Yang Shen. Multi-agent coordination across diverse applications: A survey.arXiv preprint arXiv:2502.14743, 2025

    2025 arXiv

  54. [62]

    The landscape of emerging ai agent architectures for reasoning, planning, and tool calling: A survey.arXiv preprint arXiv:2404.11584, 2024

    Tula Masterman, Sandi Besen, Mason Sawtell, and Alex Chao. The landscape of emerging ai agent architectures for reasoning, planning, and tool calling: A survey.arXiv preprint arXiv:2404.11584, 2024

    2024 arXiv

  55. [63]

    Accelerating scientific breakthroughs with an ai co-scientist, 2024

    Google Research. Accelerating scientific breakthroughs with an ai co-scientist, 2024

    2024

  56. [64]

    Webarena: A realistic web environment for building autonomous agents

    Shuyan Zhou, Frank F Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Yonatan Bisk, Daniel Fried, Uri Alon, et al. Webarena: A realistic web environment for building autonomous agents. InNeurIPS, 2023

    2023

  57. [65]

    AutoScraper: A smart, automatic, fast and lightweight web scraper for python,

    Alireza Mika. AutoScraper: A smart, automatic, fast and lightweight web scraper for python,

  58. [66]

    Span-oriented information extraction–a unifying perspective on information extraction.arXiv preprint arXiv:2403.15453, 2024

    Yifan Ding, Michael Yankoski, and Tim Weninger. Span-oriented information extraction–a unifying perspective on information extraction.arXiv preprint arXiv:2403.15453, 2024

    2024 arXiv

  59. [67]

    A survey on deep learning for named entity recognition.IEEE transactions on knowledge and data engineering, 34(1):50–70, 2020

    Jing Li, Aixin Sun, Jianglei Han, and Chenliang Li. A survey on deep learning for named entity recognition.IEEE transactions on knowledge and data engineering, 34(1):50–70, 2020

    2020

  60. [68]

    Entity linking with a knowledge base: Issues, techniques, and solutions.IEEE Transactions on Knowledge and Data Engineering, 27(2): 443–460, 2014

    Wei Shen, Jianyong Wang, and Jiawei Han. Entity linking with a knowledge base: Issues, techniques, and solutions.IEEE Transactions on Knowledge and Data Engineering, 27(2): 443–460, 2014

    2014

  61. [69]

    End-to-end sequence labeling via bi-directional LSTM-CNNs- CRF

    Xuezhe Ma and Eduard Hovy. End-to-end sequence labeling via bi-directional LSTM-CNNs- CRF. In Katrin Erk and Noah A. Smith, editors,ACL, 2016

    2016

  62. [70]

    Rodney L

    Jeremy J Siegel and Jeremy D Schwartz.The Long-term Returns on the Original S & P 500 Firms. Rodney L. White Center for Financial Research, 2004

    2004

  63. [71]

    V olume” string, e.g., “Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics 22 (V olume 1: Long Papers)

    Xiao-Yang Liu, Hongyang Yang, Qian Chen, Runjia Zhang, Liuqing Yang, Bowen Xiao, and Christina Dan Wang. Finrl: A deep reinforcement learning library for automated stock trading in quantitative finance. InNeurIPS, 2022. 13 Appendix: Table of Contents A New Dataset Instruct2DS ...

    2022

  64. [2022]

    URLhttps://github.com/alirezamika/autoscraper

  65. [2024]

    URLhttps://openreview.net/forum?id=VtmBAGCN7o

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.