Pith. sign in

REVIEW 4 major objections 6 minor 1 cited by

Creating an LLM-based AI-agent: A high-level methodology towards enhancing LLMs with APIs

T0 review · 4 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This thesis presents a seven-step methodology for giving large language models API access and an on-device architecture that runs recurring tasks as fast classifications.

desk verdict A sincere but under-supported diploma thesis: a useful survey of LLM-API techniques, a plausible but unvalidated on-device macro architecture, and a toy proof-of-concept the author himself stops short of defending. read the letter →

arxiv 2412.13233 v2 pith:HNWMFK53 submitted 2024-12-17 cs.SE

Ioannis Tzachristas This is my paper · ORCID
classification cs.SE
keywords largelanguagemodelsAIagentsAPIintegrationsemanticvectorspacewordembeddingson-devicearchitecturetaskdecompositionselection
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Large language models are powerful but sealed off from the external digital world: their knowledge freezes at training time and they cannot take actions. This thesis argues that wiring them to application programming interfaces fixes that, and it distills the wiring process into seven steps—model selection, tool-knowledge enhancement, multi-stage pipeline design, API selection, API-call generation, task decomposition, and feedback-driven improvement. Its sharper proposal is an on-device architecture: reusable tasks are stored locally as macros that map a user description to a fixed series of API calls, so the agent answers recurring requests by classifying them into the right macro instead of generating a fresh response. If correct, this would give phones and tablets a fast, offline, privacy-preserving assistant for frequent jobs, with the cloud model reserved for new or complex ones.

What carries the argument

The load-bearing mechanism is the locally stored custom database of macro-functions: each entry pairs a task title and description with the ordered series of API calls that fulfils the task, so the agent can remember exactly what to execute. Matching a user's phrasing to the right entry is done in a semantic vector space—a representation in which similar meanings are close together—using word embeddings and cosine similarity. That turns every recurring request into a classification problem, replacing a slow generative step with a fast lookup.

What would settle it

Take genuinely new user requests—including paraphrases, multilingual inputs, and voice-transcribed queries—run them through the local matcher, and count how often the correct macro or API sequence is selected; a substantial drop in accuracy, or frequent user failures while teaching new macros, would refute the paper's central premise.

Watch

Extended reading notes

Core claim

The central discovery is a high-level recipe for turning a text-only LLM into an API-calling AI agent, plus a concrete architectural proposal for running that agent on a portable device. The seven steps walk a builder from choosing a suitable model and injecting external tool knowledge, through a multi-stage pipeline that identifies intent, selects and calls an API, and processes the response, to task decomposition and iterative user feedback. The on-device architecture replaces the generative step with a classification step: a locally stored custom database holds macro-functions—each with a title, a description, and an ordered series of API calls—and at run time the agent matches the user's request to an entry through a semantic vector space of word embeddings. In the proof of concept, one hundred synthetic user requests were classified into the right macro category by all three tested models, and the two generative models could also combine macros or offer extra explanations; the thesis itself describes the experiment as a strong indication rather than a solid proof.

Load-bearing premise

The architecture's speed and offline availability rest on the assumption that matching the user's wording to a stored task description by meaning will always pick the correct macro, and that users can reliably teach the agent new macros through the training interface.

Editorial extensions

If this is right

  • Recurring requests can be answered with minimal latency, because a semantic lookup over a small local database is far cheaper than generating a response from a cloud LLM.
  • The agent keeps working offline for tasks that do not require live data, since both the database and the small models live on the device.
  • Users can personalize the agent by teaching it new tasks through the training interface, and the macro library can grow through sharing.
  • Large, multi-step goals can be split into dependent API calls, so a single request can trigger a coordinated sequence such as booking flights and hotels in order.
  • Iterative feedback and retraining allow the agent to adapt its API choices and call formats to observed success rates.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Beyond the paper: testing the semantic matcher on real user utterances—including multilingual paraphrases, typos, and voice transcription—would likely lower the reported 100 percent classification rate, and that drop would mark the architecture's true reliability boundary.
  • Beyond the paper: a hybrid design that falls back to a cloud LLM when no local macro scores above a threshold would trade a little privacy for much lower failure rates on novel requests, extending the architecture's reach without abandoning its speed.
  • Beyond the paper: a direct latency and energy benchmark comparing classification-only service with on-device generation for the same task list would make the claimed speed advantage quantitative and testable.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes a 7-step methodology for augmenting LLMs with APIs, surveys existing frameworks and tools, and introduces an on-device architecture in which a local custom database stores macros/functions that map user requests to pre-defined API-call sequences. The authors argue that, because the agent only needs to classify a user request rather than generate a response, frequent tasks can be executed very fast and offline. The experimental part consists of a piano-sheet demonstration using ChatGPT-4 and a proof-of-concept classification experiment in which BERT, Llama-3-8B, and Phi-2 classify 100 ChatGPT-generated user inputs into use cases from a custom database.

Significance. If the central claims were rigorously established, the on-device macro-database architecture would be a useful contribution to the growing field of tool-augmented LLMs, particularly for low-latency and privacy-preserving personal assistants. The paper also provides a broad, readable synthesis of existing work on API selection, task decomposition, and feedback, and it is honest about several limitations. However, the claimed contributions are not supported by the evidence presented: the only quantitative experiment reports no accuracy values, no comparison baselines, and no end-to-end API execution, and the piano-sheet example is anecdotal. The value of the paper as submitted is mainly expository, not a validated research contribution.

major comments (4)
  1. [Section 3.2.1 and Section 4.2.2] The architecture's central claim is that replacing generation with classification yields very fast execution of frequent tasks, but the proof-of-concept in Section 4.2.2 only reports that classification was "successful" without giving accuracy, a confusion matrix, or any comparison between BERT, Llama-3, and Phi-2. More importantly, the experiment evaluates only use-case label classification; it does not evaluate the extraction of arguments such as X and Y in ORDER_FROM_NEAR-BY_MARKET(X, Y), nor the chaining of multiple macros into a series of executable API calls. These are load-bearing for the claimed speed and offline functionality, so the central claim is untested.
  2. [Section 4.2.1] The 100 user inputs are generated by ChatGPT-4, the same model family used to author the macro descriptions, so the evaluation is biased toward phrasings that match the database entries. There is no test on independent human utterances, paraphrases, ambiguous or multilingual requests, or unseen macro combinations. Section 4.2.2 itself concedes that the experiment "does not constitute a solid proof" of the method's effectiveness; given this concession, the on-device architecture's robustness claims in Sections 3.2.4 and 3.2.6 remain unsupported.
  3. [Sections 3.2.3-3.2.6] The claimed advantages of reduced latency, offline availability, and scalability are stated without any quantitative evidence. No end-to-end system is implemented, no latency or memory measurements on a real device are reported, and no comparison with a cloud-based LLM baseline is provided. The example database in Figure 3.3 is explicitly labeled as illustrative, so the feasibility of the matching and execution flow is not demonstrated beyond a synthetic classification exercise.
  4. [Section 4.1] The piano-sheet example is anecdotal: it shows ChatGPT-4 outputs and states that the melody is correct and the fingering is reasonable, but it provides no evaluation rubric, no quantitative measure of correctness, no comparison with alternative approaches, and no evidence that the generated sheet is usable by a pianist. The claim that this demonstrates the effectiveness of the proposed methodology is therefore not supported, and the section does not exercise the proposed on-device architecture.
minor comments (6)
  1. [Manuscript structure] The paper contains both a Greek and an English version with duplicated content and some inconsistencies between the two versions; for example, the Greek Chapter 3 title differs from the English Chapter 3 title. The manuscript should be consolidated into a single coherent article.
  2. [Section 2.4.2] The loss-function equation has malformed typesetting and unclear notation; the subscripts and the meaning of n and j should be defined precisely.
  3. [Figure 4.9] Figure 4.9 appears to be an OCR-like rendering of the 100 user inputs with numerous typos and artifacts (e.g., "SUmmariz:e", "wori:.out"); it should be replaced with a clean, machine-readable table, and the exact model versions and prompts should be provided for reproducibility.
  4. [Section 4.1.3] The text says the LLM was connected to the MuseScore API, but no actual API call, endpoint, authentication method, or raw response is shown; the reader cannot verify whether the integration was real or simulated.
  5. [Section 2.5.2] The claim that tools are designed to be resistant to adversarial attacks and jailbreaking is made without citation or supporting evidence and should be either substantiated or removed.
  6. [Bibliography] Several references are cited incompletely or inconsistently (e.g., [18] and [5,9] in Section 2.1), and figure sources are not uniformly credited; the reference list needs to be checked carefully for completeness.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the thesis is a survey-plus-design proposal whose claims rest on external citations and explicitly conditional architecture arguments, not on self-referential derivation.

full rationale

The 7-step methodology is presented as a synthesis of cited prior frameworks (e.g., [3], [5], [18]) rather than as a first-principles derivation, so there is no input/output equivalence to expose. The API-selection mechanism uses cosine similarity and a borrowed loss function only as descriptive tools, not as fitted parameters that are later renamed as predictions. The on-device architecture's speed claim is conditional and architectural: because macros are stored in a local database, "the AI-agent only needs to execute a classification task instead of a generative task"; this is a design assertion, not a reduction of the claimed benefit to its own definition. The proof of concept does generate its own evaluation inputs with ChatGPT-4, but the paper explicitly concedes "this experiment does not constitute a solid proof of the effectiveness of the aforementioned method"; that is a data-quality/validation limitation, not circularity, since the method's effectiveness is not defined in terms of those 100 examples and no parameter is fitted to them before the 'prediction'. There are no self-citations, no imported uniqueness theorem, and no ansatz smuggled in via the author's prior work. The central claims therefore retain independent content, even though their empirical support is thin.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claim of the on-device architecture rests on hand-crafted macro definitions and embedding-based intent matching, both assumed to be reliable without quantitative validation. The experimental section adds a second assumption: that ChatGPT-generated user inputs resemble real user behavior. No new physical or formal entities are introduced, and the only free parameter counted is the hand-chosen use-case taxonomy that the classification experiment depends on.

free parameters (1)
  • Use-case taxonomy in the custom database = not reported
    The custom database in Figure 3.3 defines a fixed set of use cases, scenario descriptions, and task macros. The classification experiment assumes this hand-chosen taxonomy is sufficient and representative, but no procedure is given for selecting or validating these categories.
assumptions (4)
  • domain assumption Semantic vector space matching is sufficient to map user intent to predefined macros or functions.
    Section 3.2.1 states that matching user text input to functions is done through a semantic vector space or word embeddings, with no experimental evidence that this reliably handles ambiguous or out-of-vocabulary requests.
  • domain assumption Small on-device LLMs can carry out the required classification reliably.
    Section 4.2.2 claims BERT, Llama 3 8B, and Phi-2 all classify successfully, but no accuracy metrics or error cases are provided, so the reliability of small models on diverse user inputs is assumed.
  • domain assumption The 7-step pipeline generalizes across domains.
    Throughout Chapter 2, the methodology is presented as broadly applicable, but the only demonstrations are a piano sheet example and a synthetic classification task, offering limited evidence of generalization.
  • domain assumption Users can correctly define macros through the Training Mode Interface with continuous feedback.
    Section 3.2.1 assumes a user and the AI-agent can communicate through feedback to produce a detailed series of API calls, but the interface is not implemented and no usability study is presented.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Creating an LLM-based AI-agent: A high-level methodology towards enhancing LLMs with APIs." pith.science (2026). https://pith.science/paper/HNWMFK53

@misc{pith2026241213233,
  author       = {Pith},
  title        = {Pith review of: Creating an LLM-based AI-agent: A high-level methodology towards enhancing LLMs with APIs},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/HNWMFK53}},
  note         = {Machine review of arXiv:2412.13233}
}
read the original abstract

Large Language Models (LLMs) have revolutionized various aspects of engineering and science. Their utility is often bottlenecked by the lack of interaction with the external digital environment. To overcome this limitation and achieve integration of LLMs and Artificial Intelligence (AI) into real-world applications, customized AI agents are being constructed. Based on the technological trends and techniques, we extract a high-level approach for constructing these AI agents, focusing on their underlying architecture. This thesis serves as a comprehensive guide that elucidates a multi-faceted approach for empowering LLMs with the capability to leverage Application Programming Interfaces (APIs). We present a 7-step methodology that begins with the selection of suitable LLMs and the task decomposition that is necessary for complex problem-solving. This methodology includes techniques for generating training data for API interactions and heuristics for selecting the appropriate API among a plethora of options. These steps eventually lead to the generation of API calls that are both syntactically and semantically aligned with the LLM's understanding of a given task. Moreover, we review existing frameworks and tools that facilitate these processes and highlight the gaps in current attempts. In this direction, we propose an on-device architecture that aims to exploit the functionality of carry-on devices by using small models from the Hugging Face community. We examine the effectiveness of these approaches on real-world applications of various domains, including the generation of a piano sheet. Through an extensive analysis of the literature and available technologies, this thesis aims to set a compass for researchers and practitioners to harness the full potential of LLMs augmented with external tool capabilities, thus paving the way for more autonomous, robust, and context-aware AI agents.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Guided Persona-based AI Surveys: Can we replicate personal mobility preferences at scale using LLMs?

    cs.CL 2025-01 reject novelty 4.0 of 10

    A persona-based LLM survey generator given MiD 2017 response statistics reproduces those statistics almost exactly, but the evaluation is circular and the main claim is unsupported.

Reference graph

Works this paper leans on

55 extracted references · 24 canonical work pages · cited by 1 Pith paper

  1. [1]

    The rise and potential of large language model based agents: A survey,

    Z. Xi, W. Chen, X. Guo, W. He, Y. Ding, B. Hong, M. Zhang, J. Wang, S. Jin, E. Zhou, et al., “The rise and potential of large language model based agents: A survey,” arXiv preprint arXiv:2309.07864, 2023

    arXiv 2023

  2. [2]

    “Plato.” https://en.wikipedia.org/wiki/Plato, 2024

    work page 2024

  3. [3]

    Api -bank: A benchmark for tool- augmented llms,

    M. Li, F. Song, B. Yu, H. Yu, Z. Li, F. Huang, and Y. Li, “Api -bank: A benchmark for tool- augmented llms,” arXiv preprint arXiv:2304.08244, 2023

    arXiv 2023

  4. [4]

    A survey on large language model based autonomous agents,

    L. Wang, C. Ma, X. Feng, Z. Zhang, H. Yang, J. Zhang, Z. Chen, J. Tang, X. Chen, Y. Lin, et al., “A survey on large language model based autonomous agents,” Frontiers of Computer Science, vol. 18, no. 6, pp. 1–26, 2024

    work page 2024

  5. [5]

    Toolllm: Facilitating large language models to master 16000+ real-world apis,

    Y. Qin, S. Liang, Y. Ye, K. Zhu, L. Yan, Y. Lu, Y. Lin, X. Cong, X. Tang, B. Qian, et al., “Toolllm: Facilitating large language models to master 16000+ real-world apis,” arXiv preprint arXiv:2307.16789, 2023

    arXiv 2023

  6. [6]

    Musescore

    “Musescore.” https://musescore.org/en, 2024

    work page 2024

  7. [7]

    Gpt-4 technical report,

    J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al., “Gpt-4 technical report,” arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  8. [8]

    Bert: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “Bert: Pre-training of deep bidirectional transformers for language understanding,” arXiv preprint arXiv:1810.04805, 2018

    arXiv 2018

Show all 55 references
  1. [9]

    Llama 2: Open foundation and fine-tuned chat models,

    H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, et al., “Llama 2: Open foundation and fine-tuned chat models,” arXiv preprint arXiv:2307.09288, 2023

    2023 arXiv

  2. [10]

    Human-level control through deep re- inforcement learning,

    V. Mnih, K. Kavukcuoglu, D. Silver, A. A. Rusu, J. Veness, M. G. Bellemare, A. Graves, M. Riedmiller, A. K. Fidjeland, G. Ostrovski, et al., “Human-level control through deep re- inforcement learning,” nature, vol. 518, no. 7540, pp. 529–533, 2015

    2015

  3. [11]

    Continuous control with deep reinforcement learning,

    T. P. Lillicrap, J. J. Hunt, A. Pritzel, N. Heess, T. Erez, Y. Tassa, D. Silver, and D. Wierstra, “Continuous control with deep reinforcement learning,” arXiv preprint arXiv:1509.02971, 2015

    2015 arXiv

  4. [12]

    Proximal policy optimization algorithms,

    J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov, “Proximal policy optimization algorithms,” arXiv preprint arXiv:1707.06347, 2017

    2017 arXiv

  5. [13]

    Language models are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al., “Language models are few-shot learners,” Advances in neural infor- mation processing systems, vol. 33, pp. 1877–1901, 2020

    1901

  6. [14]

    Metagpt: Meta programming for multi-agent collaborative framework,

    S. Hong, X. Zheng, J. Chen, Y. Cheng, J. Wang, C. Zhang, Z. Wang, S. K. S. Yau, Z. Lin, L. Zhou, et al., “Metagpt: Meta programming for multi-agent collaborative framework,” arXiv preprint arXiv:2308.00352, 2023. 102

    2023 arXiv

  7. [15]

    Gpt is an effective tool for multilingual psychological text analysis,

    S. Rathje, D.-M. Mirea, I. Sucholutsky, R. Marjieh, C. Robertson, and J. J. Van Bavel, “Gpt is an effective tool for multilingual psychological text analysis,” 2023

    2023

  8. [16]

    Memgpt: Towards llms as operating systems,

    C. Packer, V. Fang, S. G. Patil, K. Lin, S. Wooders, and J. E. Gonzalez, “Memgpt: Towards llms as operating systems,” arXiv preprint arXiv:2310.08560, 2023

    2023 arXiv

  9. [17]

    Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face,

    Y. Shen, K. Song, X. Tan, D. Li, W. Lu, and Y. Zhuang, “Hugginggpt: Solving ai tasks with chatgpt and its friends in hugging face,” Advances in Neural Information Processing Systems, vol. 36, 2024

    2024

  10. [18]

    Toolformer: Language models can teach themselves to use tools,

    T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Can- cedda, and T. Scialom, “Toolformer: Language models can teach themselves to use tools,” Ad- vances in Neural Information Processing Systems, vol. 36, 2024

    2024

  11. [19]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al., “Language models are unsupervised multitask learners,” OpenAI blog, vol. 1, no. 8, p. 9, 2019

    2019

  12. [20]

    Gptvoicetasker: Llm- powered virtual assistant for smartphone,

    M. D. Vu, H. Wang, Z. Li, J. Chen, S. Zhao, Z. Xing, and C. Chen, “Gptvoicetasker: Llm- powered virtual assistant for smartphone,” arXiv preprint arXiv:2401.14268, 2024

    2024 arXiv

  13. [21]

    Opendevin: A github repository

    “Opendevin: A github repository.” https://github.com/OpenDevin/OpenDevin, 2024

    2024

  14. [22]

    Devika: A github repository by stition ai

    “Devika: A github repository by stition ai.” https://github.com/stitionai/devika, 2024

    2024

  15. [23]

    Hugging face: Open source machine learning tools

    “Hugging face: Open source machine learning tools.” https://huggingface.co, 2024

    2024

  16. [24]

    A survey of large language models,

    W. X. Zhao, K. Zhou, J. Li, T. Tang, X. Wang, Y. Hou, Y. Min, B. Zhang, J. Zhang, Z. Dong, et al., “A survey of large language models,” arXiv preprint arXiv:2303.18223, 2023

    2023 arXiv

  17. [25]

    Language model behavior: A comprehensive survey,

    T. A. Chang and B. K. Bergen, “Language model behavior: A comprehensive survey,” Compu- tational Linguistics, pp. 1–58, 2024

    2024

  18. [26]

    Harnessing the power of llms in practice: A survey on chatgpt and beyond,

    J. Yang, H. Jin, R. Tang, X. Han, Q. Feng, H. Jiang, S. Zhong, B. Yin, and X. Hu, “Harnessing the power of llms in practice: A survey on chatgpt and beyond,” ACM Transactions on Knowledge Discovery from Data, 2023

    2023

  19. [27]

    Aligning large language models with human: A survey,

    Y. Wang, W. Zhong, L. Li, F. Mi, X. Zeng, W. Huang, L. Shang, X. Jiang, and Q. Liu, “Aligning large language models with human: A survey,” arXiv preprint arXiv:2307.12966, 2023

    2023 arXiv

  20. [28]

    Towards reasoning in large language models: A survey,

    J. Huang and K. C.-C. Chang, “Towards reasoning in large language models: A survey,” arXiv preprint arXiv:2212.10403, 2022

    2022 arXiv

  21. [29]

    Augmented language models: a survey,

    G. Mialon, R. Dessì, M. Lomeli, C. Nalmpantis, R. Pasunuru, R. Raileanu, B. Rozière, T. Schick, J. Dwivedi-Yu, A. Celikyilmaz, et al., “Augmented language models: a survey,” arXiv preprint arXiv:2302.07842, 2023

    2023 arXiv

  22. [30]

    A survey on evaluation of large language models,

    Y. Chang, X. Wang, J. Wang, Y. Wu, L. Yang, K. Zhu, H. Chen, X. Yi, C. Wang, Y. Wang, et al., “A survey on evaluation of large language models,” ACM Transactions on Intelligent Systems and Technology, 2023

    2023

  23. [31]

    Rapid’s api hub

    “Rapid’s api hub.” https://rapidapi.com, 2024

    2024

  24. [32]

    Chain of hindsight aligns language models with feedback,

    H. Liu, C. Sferrazza, and P. Abbeel, “Chain of hindsight aligns language models with feedback,” arXiv preprint arXiv:2302.02676, 2023

    2023 arXiv

  25. [33]

    React: Synergizing reasoning and acting in language models,

    S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao, “React: Synergizing reasoning and acting in language models,” arXiv preprint arXiv:2210.03629, 2022. 10 3

    2022 arXiv

  26. [34]

    Tree of thoughts: Deliberate problem solving with large language models,

    S. Yao, D. Yu, J. Zhao, I. Shafran, T. Griffiths, Y. Cao, and K. Narasimhan, “Tree of thoughts: Deliberate problem solving with large language models,” Advances in Neural Information Pro- cessing Systems, vol. 36, 2024

    2024

  27. [35]

    Revolutionizing mobile interaction: Enabling a 3 billion parameter gpt llm on mobile,

    S. Carreira, T. Marques, J. Ribeiro, and C. Grilo, “Revolutionizing mobile interaction: Enabling a 3 billion parameter gpt llm on mobile,” arXiv preprint arXiv:2310.01434, 2023

    2023 arXiv

  28. [36]

    Llm in a flash: Efficient large language model inference with limited memory,

    K. Alizadeh, I. Mirzadeh, D. Belenko, K. Khatamifard, M. Cho, C. C. Del Mundo, M. Raste - gari, and M. Farajtabar, “Llm in a flash: Efficient large language model inference with limited memory,” arXiv preprint arXiv:2312.11514, 2023

    2023 arXiv

  29. [37]

    Llm as a system service on mobile devices,

    W. Yin, M. Xu, Y. Li, and X. Liu, “Llm as a system service on mobile devices,” arXiv preprint arXiv:2403.11805, 2024

    2024 arXiv

  30. [38]

    Mrkl systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning,

    E. Karpas, O. Abend, Y. Belinkov, B. Lenz, O. Lieber, N. Ratner, Y. Shoham, H. Bata, Y. Levine, K. Leyton -Brown, et al., “Mrkl systems: A modular, neuro-symbolic architecture that combines large language models, external knowledge sources and discrete reasoning,” arXiv prepri...

    2022 arXiv

  31. [39]

    Camel: Communicative agents for

    G. Li, H. A. A. K. Hammoud, H. Itani, D. Khizbullin, and B. Ghanem, “Camel: Communicative agents for” mind” exploration of large scale language model society,” 2023

    2023

  32. [40]

    Communicative agents for software development,

    C. Qian, X. Cong, C. Yang, W. Chen, Y. Su, J. Xu, Z. Liu, and M. Sun, “Communicative agents for software development,” arXiv preprint arXiv:2307.07924, 2023

    2023 arXiv

  33. [41]

    Generative agents: Interactive simulacra of human behavior,

    J. S. Park, J. O’Brien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein, “Generative agents: Interactive simulacra of human behavior,” in Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, pp. 1–22, 2023

    2023

  34. [42]

    Nemo guardrails: A toolkit for controllable and safe llm applications with programmable rails,

    T. Rebedea, R. Dinu, M. Sreedhar, C. Parisien, and J. Cohen, “Nemo guardrails: A toolkit for controllable and safe llm applications with programmable rails,” arXiv preprint arXiv:2310.10501, 2023

    2023 arXiv

  35. [43]

    Reflexion: Language agents with verbal reinforcement learning,

    N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao, “Reflexion: Language agents with verbal reinforcement learning,” Advances in Neural Information Processing Systems , vol. 36, 2024

    2024

  36. [44]

    Think-on-graph: Deep and responsible reasoning of large language model with knowledge graph,

    J. Sun, C. Xu, L. Tang, S. Wang, C. Lin, Y. Gong, H.-Y. Shum, and J. Guo, “Think-on-graph: Deep and responsible reasoning of large language model with knowledge graph,” arXiv preprint arXiv:2307.07697, 2023

    2023 arXiv

  37. [45]

    Voyager: An open-ended embodied agent with large language models,

    G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar, “Voyager: An open-ended embodied agent with large language models,” arXiv preprint arXiv:2305.16291, 2023

    2023 arXiv

  38. [46]

    Proof-carrying code,

    G. C. Necula, “Proof-carrying code,” in Proceedings of the 24th ACM SIGPLAN-SIGACT sym- posium on Principles of programming languages, pp. 106–119, 1997

    1997

  39. [47]

    Roberta: A robustly optimized bert pretraining approach,

    Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” arXiv preprint arXiv:1907.11692, 2019

    1907 arXiv

  40. [48]

    Xlnet: Gener- alized autoregressive pretraining for language understanding,

    Z. Yang, Z. Dai, Y. Yang, J. Carbonell, R. R. Salakhutdinov, a nd Q. V. Le, “Xlnet: Gener- alized autoregressive pretraining for language understanding,” Advances in neural information processing systems, vol. 32, 2019. 104

    2019

  41. [49]

    Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes,

    C.-Y. Hsieh, C.-L. Li, C.-K. Yeh, H. Nakhost, Y. Fujii, A. Ratner, R. Krishna, C.-Y. Lee, and T. Pfister, “Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes,” arXiv preprint arXiv:2305.02301, 2023

    2023 arXiv

  42. [50]

    Prompting is all you need: Automated android bug replay with large language models,

    S. Feng and C. Chen, “Prompting is all you need: Automated android bug replay with large language models,” in Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, ICSE ’24, (New York, NY, USA), Association for Computing Machinery, 2024

    2024

  43. [51]

    Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,

    T. Haarnoja, A. Zhou, P. Abbeel, and S. Levine, “Soft actor-critic: Off-policy maximum entropy deep reinforcement learning with a stochastic actor,” in International conference on machine learning, pp. 1861–1870, PMLR, 2018

    2018

  44. [52]

    Generative adversarial user mod el for reinforcement learning based recommendation system,

    X. Chen, S. Li, H. Li, S. Jiang, Y. Qi, and L. Song, “Generative adversarial user mod el for reinforcement learning based recommendation system,” in International Conference on Machine Learning, pp. 1052–1061, PMLR, 2019

    2019

  45. [53]

    Reflexion: an autonomous agent with dynamic memory and self-reflection,

    N. Shinn, B. Labash, and A. Gopinath, “Reflexion: an autonomous agent with dynamic memory and self-reflection,” arXiv preprint arXiv:2303.11366, 2023

    2023 arXiv

  46. [54]

    Instruction tuning for large language models: A survey,

    S. Zhang, L. Dong, X. Li, S. Zhang, X. Sun, S. Wang, J. Li, R. Hu, T. Zhang, F. Wu, et al., “Instruction tuning for large language models: A survey,” arXiv preprint arXiv:2308.10792, 2023

    2023

  47. [55]

    Langchain: A github reposit ory by langchain ai

    “Langchain: A github reposit ory by langchain ai.” https://github.com/langchain-ai/ langchain, 2024

    2024

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.