Pith. sign in

REVIEW 4 major objections 5 minor 2 cited by

Get Experience from Practice: LLM Agents with Record & Replay

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Replay recorded runs to make LLM agents reliable

desk verdict A coherent and honestly scoped vision paper for record-replay in LLM agents, but the reliability claims rest on an unconstructed check-function TCB and there is no empirical support. read the letter →

arxiv 2505.17716 v1 pith:WJFUAE6B submitted 2025-05-23 cs.LG cs.MA

Erhu Feng , Wenbo Zhou , Zibin Liu , Le Chen , Yunpeng Dong , Cheng Zhang , Yisheng Zhao , Dong Du
show 3 more authors
Zhichao Hua Yubin Xia Haibo Chen
This is my paper
classification cs.LGcs.MA
keywords recordandreplayLLMagentsmulti-levelexperiencecheckfunctionstrustedcomputingbaseagentreliabilitycostreductionprivacy
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that the main problems of LLM-based agents—unreliable outputs, privacy exposure, high cost, and slow execution—are inherent to letting a large model reason from scratch on every task. It proposes AgentRR, which records a proven interaction trace, summarizes it into a structured multi-level experience, and replays that experience on similar future tasks. Check functions act as a trusted boundary during replay, confining the agent to actions that stay within the validated experience. The paper claims this preserves the generalization of LLMs while adding the determinism and efficiency of a scripted procedure, and it outlines deployment modes such as a large model recording for a small local model to replay.

What carries the argument

The central mechanism is the multi-level experience: a summarized collection of trajectories from similar tasks, stored at low and high levels of abstraction, together with check functions that serve as the system's trusted computing base (TCB). Low-level experiences capture concrete action sequences such as clicks and API calls; high-level experiences capture abstract plans like 'select city, select hotel, choose dates, query.' Check functions verify execution-flow integrity, state preconditions, data and parameter constraints, and safety invariants during replay, thereby bounding the agent's behavior. The paper also models executions as state-transition diagrams, where experience reduces the space of trajectories the model must search.

What would settle it

A replay session in which the agent performs an action that violates a user-defined safety rule (for example, submitting a payment without confirmation) and the check function does not flag it would directly refute the claim that check functions guarantee adherence to safety requirements throughout replay.

Watch

Extended reading notes

Core claim

AgentRR's central claim is that agent intelligence and agent execution can be decoupled: the expensive, fallible reasoning that plans a task needs to happen only once, during recording, and can then be distilled into structured experience that a cheaper and more controlled replay process executes. The discovery is that experience must live at multiple levels of abstraction—low-level experiences preserve exact action sequences for fast, faithful replay, while high-level experiences capture procedural knowledge that a local model can adapt to new contexts—and that each level needs a check function as a trust anchor. The paper contends that this combination addresses reliability by bounding actions with validated traces, privacy by keeping raw data local during replay, cost by reducing large-model calls, and performance by making replay closer to scripted execution than to step-by-step reasoning.

Load-bearing premise

The load-bearing assumption is that the check function can act as a complete and trustworthy safety boundary during replay, catching every illegal action, bad precondition, and unsafe parameter.

Editorial extensions

If this is right

  • Agents can solve repetitive tasks at near-scripted speed and with greatly reduced LLM API cost, because most steps are replayed rather than re-reasoned.
  • Privacy improves because the replay phase can run locally with a small model, and raw user data need not be sent to a cloud LLM for planning.
  • Reliability improves because hallucinations and unrecoverable error states are caught by check functions that confine the agent to validated behavior.
  • The record-and-replay paradigm opens a design space where recording and replaying can be performed by different entities, such as a large model recording and a small edge model replaying, letting expensive planning be done once and executed many times.
  • An experience repository could let users share vetted task experiences, so that effective agents do not require every user to perform expensive reasoning from scratch.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If check functions are the practical trust anchor, the value of AgentRR will hinge on how much of a task's safety requirements can be expressed in them; a testable extension would measure how often hand-written checks miss edge cases that a recorded trace happens to avoid.
  • The multi-level idea suggests a continuum between RPA-style scripting and autonomous agents, so AgentRR could be compared against incremental improvements in prompt caching and workflow engines to see where the boundary of 'replay' and 'reasoning' actually lies.
  • Experiences could be used as fine-tuning data: after a small model replays a task successfully several times, the recorded experience might be distilled into the model's weights, gradually reducing even the local model's reliance on high-level reasoning.
  • The experience repository could become a learning system if replays that diverge from an experience are recorded and analyzed, refining the experience's check functions over time; the paper does not explore this feedback loop.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes AgentRR, a record-and-replay paradigm for LLM agents in which an agent's interaction trace is recorded, summarized into multi-level 'experiences' (low-level concrete operations and high-level procedural knowledge), and replayed under the supervision of 'check functions' that are claimed to act as a trusted computing base. The authors argue that this decouples intelligence from execution and thereby improves reliability, privacy, cost, and performance compared with both pure LLM agents and traditional record-and-replay tools. The paper also sketches several application modes (human-record/model-replay, large-model-record/small-model-replay, and untrusted-model-record/trusted-model-replay) and an Experience Store, and concludes that AgentRR effectively addresses the four challenges. No quantitative evaluation, implementation, or formal correctness analysis is reported.

Significance. If the proposed mechanisms worked as claimed, AgentRR would be a useful conceptual contribution to the emerging area of dependable LLM agents: the taxonomy of record/replay modes is clear, the related-work survey is broad, and the discussion of the reliability-privacy-cost-performance trade-off space is thoughtful. However, the central claims are not supported by any measurement, implementation, or formal argument. The paper itself, in Section 6, concedes that 100% reliable replay remains elusive and that summarization can be incomplete, which directly undermines the safety guarantee advertised in Section 3.2.2. At present the contribution is therefore a position paper, not a validated system; its significance depends entirely on a future implementation and evaluation.

major comments (4)
  1. [Section 6] The sentence 'Our design and evaluation demonstrate that AgentRR effectively addresses the challenges of agent reliability, privacy, and performance through its innovative record-and-replay mechanism' is unsupported by the contents of the manuscript. Section 5 is an anecdotal case study: it reports that OpenAI's CUA takes approximately three minutes and still fails, and that Chrome Recorder requires manual parameter input, but it gives no AgentRR success rate, no latency or cost measurements, no privacy metric, and no controlled comparison with the baselines. The central claim of the paper is therefore an assertion, not a demonstrated result.
  2. [Section 3.2.2] The guarantee that once a user audits and trusts the check functions, 'it guarantees that the agent will adhere to the user's safety requirements throughout the replay process' is load-bearing but unsubstantiated. The check functions are called a trusted computing base, yet the paper provides no construction, formal semantics, completeness condition, or verification method. Section 4.2 allows check functions to be 'generated by the user-provided description and ML-based summary,' so they can inherit the very hallucination and incompleteness problems the framework is meant to remove. Section 6 then concedes that 'achieving 100% reliable replay remains elusive' and that summarization can be incomplete. This internal contradiction removes the support for the reliability and safety half of the central claim.
  3. [Table 2] The entries for AgentRR in Table 2—'High (Exceeds human speed)', 'High' accuracy, and 'High (Generalized for repetitive tasks)'—are presented as comparative facts, but no experiment or quantitative model in the paper justifies them. A comparison table requires either measurements or a clearly stated analytical derivation; neither is provided. In particular, 'Exceeds human speed' is a quantitative claim with no supporting data.
  4. [Section 3.4] The state-transition-diagram formalism is introduced as a foundation for the replay guarantee, but it is underspecified: the paper does not define when a check function is complete with respect to a state-transition diagram, nor does it give an algorithm or proof that the replay phase produces a valid trajectory. Section 4.3 similarly describes selecting 'the lowest-level experience that still maintains the highest success rate' without defining a selection criterion. These omissions make the core replay mechanism difficult to assess or reproduce.
minor comments (5)
  1. [Section 2.3] In the paragraph comparing AgentRR with traditional R&R, the word 'Experienc' should be 'Experience'.
  2. [Figure 2] The figure contains the typo 'informantion'; it should be 'information'.
  3. [Section 5] In the Summary Phase description, 'Vaild' should be 'Valid'.
  4. [References] Reference [75] is cited for RnR-Safe, but the listed title, 'Density-Aware Routing in Highly Dynamic DTNs: The RollerNet Case', appears unrelated to the cited security-hardening mechanism; please verify and correct this citation.
  5. [Table 1] The row 'LLM Agent | Human' with primary use case 'Understanding agent behavior, teaching' is confusing; clarify whether the human is the replayer in the record-and-replay sense or the end user of the replayed experience.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: AgentRR is an architectural proposal with no fitted parameters, no quantitative predictions, and no load-bearing self-citation chain.

full rationale

The paper makes no quantitative predictions and fits no parameters to data, so there is no fitted-input-called-prediction chain to examine. Its central claims about reliability, privacy, cost, and performance are qualitative consequences of the proposed record-and-replay architecture: replaying recorded experiences reduces LLM calls and constrains behavior, which is an argument from mechanism rather than a derivation that reduces to its inputs. The check-function guarantee in Section 3.2.2 ('Once the user audits and places trust in the given check functions, it guarantees that the agent will adhere to the user's safety requirements throughout the replay process') is a load-bearing trust assumption, but it is explicitly an axiom of the design — the paper states that check functions serve as the TCB and that the user must audit and trust them. This is an unsupported assertion and a correctness/safety risk, not circularity: the paper does not claim to derive that guarantee from anything else, and it candidly concedes in Section 6 that 'achieving 100% reliable replay remains elusive' and that summarization can be incomplete, which confirms the guarantee is a design goal rather than a proven equivalence. No self-citation, uniqueness theorem, or renamed known result is used to force the conclusion. Hence no circular step meets the evidentiary standard of quoting a specific reduction of a claimed result to its own inputs.

Assumptions & free parameters 0 free parameters · 3 assumptions · 2 invented entities

The paper introduces two abstract constructs, multi-level experiences and check functions, and relies on assumptions about trace summarization, task similarity, and the completeness of safety checks. There are no fitted parameters because no quantitative evaluation is performed.

assumptions (3)
  • domain assumption Recorded interaction traces can be summarized into experiences that preserve the essential workflow while dropping instance-specific details.
    The summary phase in Section 4.2 assumes commonalities can be identified across traces and generalized into a reusable experience; if this summarization loses critical constraints, replay safety fails.
  • domain assumption Check functions can serve as a complete and trustworthy safety boundary (TCB) for replay.
    Section 3.2.2 states that once a user audits and trusts the check functions, the agent will adhere to safety requirements; this requires that check functions are correct and complete, with no construction or verification method supplied.
  • domain assumption Similar tasks share sufficient structural overlap for a recorded experience to be applicable.
    The replay phase (Section 4.3) selects experiences based on task similarity; if tasks differ in ways not captured by the experience abstraction, replay will fail.
invented entities (2)
  • Multi-level Experience
    purpose: Abstract representation of recorded agent traces at varying granularities to balance reliability and generalization.
    The paper defines high-level and low-level experiences conceptually (Section 3.2.1) but provides no formal representation, data structure, or implementation.
  • Check Function
    purpose: A safety verification mechanism (TCB) that checks execution flow, preconditions, parameter constraints, and invariants during replay.
    It is a hypothesized component with no code or specification; the paper states it can be defined by users or generated by models but gives no concrete examples of correct construction.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Get Experience from Practice: LLM Agents with Record & Replay." pith.science (2026). https://pith.science/paper/WJFUAE6B

@misc{pith2026250517716,
  author       = {Pith},
  title        = {Pith review of: Get Experience from Practice: LLM Agents with Record & Replay},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WJFUAE6B}},
  note         = {Machine review of arXiv:2505.17716}
}
read the original abstract

AI agents, empowered by Large Language Models (LLMs) and communication protocols such as MCP and A2A, have rapidly evolved from simple chatbots to autonomous entities capable of executing complex, multi-step tasks, demonstrating great potential. However, the LLMs' inherent uncertainty and heavy computational resource requirements pose four significant challenges to the development of safe and efficient agents: reliability, privacy, cost and performance. Existing approaches, like model alignment, workflow constraints and on-device model deployment, can partially alleviate some issues but often with limitations, failing to fundamentally resolve these challenges. This paper proposes a new paradigm called AgentRR (Agent Record & Replay), which introduces the classical record-and-replay mechanism into AI agent frameworks. The core idea is to: 1. Record an agent's interaction trace with its environment and internal decision process during task execution, 2. Summarize this trace into a structured "experience" encapsulating the workflow and constraints, and 3. Replay these experiences in subsequent similar tasks to guide the agent's behavior. We detail a multi-level experience abstraction method and a check function mechanism in AgentRR: the former balances experience specificity and generality, while the latter serves as a trust anchor to ensure completeness and safety during replay. In addition, we explore multiple application modes of AgentRR, including user-recorded task demonstration, large-small model collaboration and privacy-aware agent execution, and envision an experience repository for sharing and reusing knowledge to further reduce deployment cost.

Figures

Figures reproduced from arXiv: 2505.17716 by the authors.

Figure 1
Figure 1. Conceptual comparison of human, R&R tools, LLM agents, and AgentRR in task execution. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Multi-level experience: High-level experience describes the task planning process without being bound to specific platforms or UI layouts. Low-level experience contains more detailed action decomposition and may be coupled with specific platforms and UI layouts. 10 [PITH_FULL_IMAGE:figures/full_fig_p010_2.png] view at source ↗
Figure 3
Figure 3. The overall architecture of AgentRR: The AgentRR system consists of three core components: the Record module, Summary module, and Replay module. Additionally, to facilitate experience sharing across different users, AgentRR incorporates an experience store. 4.1 Record Phase: Capturing User Traces The Record phase requires capturing a complete sequence of actions, which can be accomplished by logging either GUI inter… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: The online form filling example: During the record phase, users capture multiple trace behaviors. [PITH_FULL_IMAGE:figures/full_fig_p015_4.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Industrial Practice of LLM-Based Test Case Carving and Assertion Generation (Experience Paper)

    cs.SE 2026-07 conditional novelty 6.0 of 10

    NL2Test generates executable API regression tests from a natural-language scenario and a recorded traffic capture, achieving 82.4% exact-match on 51 industrial scenarios and 85.4% adoption across 3,196 deployed tests.

  2. MobiAgent: A Systematic Framework for Customizable Mobile Agents

    cs.MA 2025-08 conditional novelty 6.0 of 10

    A full-stack mobile agent framework reports state-of-the-art task completion on its own DAG-based benchmark and 2-3x speedups from replaying recorded trajectories.

Reference graph

Works this paper leans on

97 extracted references · 6 canonical work pages · cited by 2 Pith papers

  1. [1]

    Workflow Use

    2025. Workflow Use. https://github.com/browser-use/workflow-use. Referenced May 2025

    2025

  2. [2]

    Anthropic. 2024. Introducing the Model Context Protocol. https://www.anthropic.com/news/ model-context-protocol. Accessed: 2025-05-19

    2024

  3. [3]

    Anthropic. 2025. Computer use (beta).https://docs.anthropic.com/en/docs/agents-and-tools/ computer-use. Accessed: 2025-05-19

    2025

  4. [4]

    Eugene Bagdasarian, Ren Yi, Sahra Ghalebikesabi, Peter Kairouz, Marco Gruteser, Sewoong Oh, Borja Balle, and Daniel Ramage. 2024. AirGapAgent: Protecting Privacy-Conscious Conversational Agents. arXiv:2405.05175 [cs.CR] https://arxiv.org/abs/2405.05175 17

    arXiv 2024

  5. [5]

    Sourav Banerjee, Ayushi Agarwal, and Saloni Singla. 2024. LLMs Will Always Hallucinate, and We Need to Live With This. arXiv:2409.05746 [stat.ML]https://arxiv.org/abs/2409.05746

    arXiv 2024

  6. [6]

    Chang and Longling Geng

    Edward Y. Chang and Longling Geng. 2025. SagaLLM: Context Management, Validation, and Transac- tion Guarantees for Multi-Agent LLM Planning. arXiv:2503.11951 [cs.AI]https://arxiv.org/abs/ 2503.11951

    arXiv 2025

  7. [7]

    Brendan McMahan, Nicole Mitchell, Krishna Pil- lutla, and Keith Rush

    Zachary Charles, Arun Ganesh, Ryan McKenna, H. Brendan McMahan, Nicole Mitchell, Krishna Pil- lutla, and Keith Rush. 2024. Fine-Tuning Large Language Models with User-Level Differential Privacy. arXiv:2407.07737 [cs.LG] https://arxiv.org/abs/2407.07737

    arXiv 2024

  8. [8]

    Chaoran Chen, Zhiping Zhang, Bingcan Guo, Shang Ma, Ibrahim Khalilov, Simret A Gebreegziabher, Yanfang Ye, Ziang Xiao, Yaxing Yao, Tianshi Li, and Toby Jia-Jun Li. 2025. The Obvious Invisible Threat: LLM-Powered GUI Agents’ Vulnerability to Fine-Print Injections. arXiv:2504.11281 [cs.HC] https://arxiv.org/abs/2504.11281

    arXiv 2025

Show all 97 references
  1. [9]

    Chaoran Chen, Daodao Zhou, Yanfang Ye, Toby Jia jun Li, and Yaxing Yao. 2025. CLEAR: Towards Contextual LLM-Empowered Privacy Policy Analysis and Risk Generation for Large Language Model Applications. arXiv:2410.13387 [cs.HC] https://arxiv.org/abs/2410.13387

    2025 arXiv

  2. [10]

    Pan, Wen Zhang, Huajun Chen, Fan Yang, Zenan Zhou, and Weipeng Chen

    Mingyang Chen, Tianpeng Li, Haoze Sun, Yijie Zhou, Chenzheng Zhu, Haofen Wang, Jeff Z. Pan, Wen Zhang, Huajun Chen, Fan Yang, Zenan Zhou, and Weipeng Chen. 2025. ReSearch: Learning to Reason with Search for LLMs via Reinforcement Learning. arXiv:2503.19470 [cs.AI] https://arxi...

    2025 arXiv

  3. [11]

    Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory

    PrateekChhikara, DevKhant, SaketAryan, TaranjeetSingh, andDeshrajYadav.2025. Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory. arXiv:2504.19413 [cs.CL] https: //arxiv.org/abs/2504.19413

    2025 arXiv

  4. [12]

    Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. 2024. Safe RLHF: Safe Reinforcement Learning from Human Feedback. InThe Twelfth Interna- tional Conference on Learning Representations . https://openreview.net/forum?id=TyFrPOKYXw

    2024

  5. [13]

    Daniela A. S. de Oliveira, Jedidiah R. Crandall, Gary Wassermann, S. Felix Wu, Zhendong Su, and Frederic T. Chong. 2006. ExecRecorder: VM-based full-system replay for attack analysis and system recovery.In Proceedings of the 1st Workshop on Architectural and System Support for...

    2006

  6. [14]

    Christian Schroeder de Witt. 2025. Open Challenges in Multi-Agent Security: Towards Secure Systems of Interacting AI Agents. arXiv:2505.02077 [cs.CR]https://arxiv.org/abs/2505.02077

    2025 arXiv

  7. [15]

    Yihe Deng, Yu Yang, Junkai Zhang, Wei Wang, and Bo Li. 2025. DuoGuard: A Two-Player RL-Driven Framework for Multilingual LLM Guardrails. arXiv:2502.05163 [cs.CL] https://arxiv.org/abs/ 2502.05163

    2025

  8. [16]

    Nolan Dey, Gurpreet Gosal, Zhiming, Chen, Hemant Khachane, William Marshall, Ribhu Pathria, Mar- vin Tom, and Joel Hestness. 2023. Cerebras-GPT: Open Compute-Optimal Language Models Trained on the Cerebras Wafer-Scale Cluster. arXiv:2304.03208 [cs.LG]https://arxiv.org/abs/2304...

    2023 arXiv

  9. [17]

    Yi Dong, Ronghui Mu, Gaojie Jin, Yi Qi, Jinwei Hu, Xingyu Zhao, Jie Meng, Wenjie Ruan, and Xiaowei Huang. 2024. Building Guardrails for Large Language Models. arXiv:2402.01822 [cs.CL] https://arxiv.org/abs/2402.01822

    2024 arXiv

  10. [18]

    Haojie Duanmu, Xiuhong Li, Zhihang Yuan, Size Zheng, Jiangfei Duan, Xingcheng Zhang, and Dahua Lin. 2025. MxMoE: Mixed-precision Quantization for MoE with Accuracy and Performance Co-Design. arXiv:2505.05799 [cs.LG] https://arxiv.org/abs/2505.05799

    2025 arXiv

  11. [19]

    Jiazhan Feng, Shijue Huang, Xingwei Qu, Ge Zhang, Yujia Qin, Baoquan Zhong, Chengquan Jiang, Jinxin Chi, and Wanjun Zhong. 2025. ReTool: Reinforcement Learning for Strategic Tool Use in LLMs. arXiv:2504.11536 [cs.CL] https://arxiv.org/abs/2504.11536

    2025 arXiv

  12. [20]

    Sidong Feng, Haochuan Lu, Ting Xiong, Yuetang Deng, and Chunyang Chen. 2023. Towards Efficient Record and Replay: A Case Study in WeChat. arXiv:2308.06657 [cs.SE] https://arxiv.org/abs/ 2308.06657

    2023 arXiv

  13. [21]

    Rogov, Elena Tu- tubalina, and Ivan Oseledets

    Andrey Galichin, Alexey Dontsov, Polina Druzhinina, Anton Razzhigaev, Oleg Y. Rogov, Elena Tu- tubalina, and Ivan Oseledets. 2025. I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders. arXiv:2503.18878 [cs.CL] http...

    2025 arXiv

  14. [22]

    Aoran Gan, Hao Yu, Kai Zhang, Qi Liu, Wenyu Yan, Zhenya Huang, Shiwei Tong, and Guoping Hu. 2025. Retrieval Augmented Generation Evaluation in the Era of Large Language Models: A Comprehensive Survey. arXiv:2504.14891 [cs.CL] https://arxiv.org/abs/2504.14891

    2025 arXiv

  15. [23]

    Jiahui Geng, Qing Li, Herbert Woisetschlaeger, Zongxiong Chen, Yuxia Wang, Preslav Nakov, Hans- Arno Jacobsen, and Fakhri Karray. 2025. A Comprehensive Survey of Machine Unlearning Techniques for Large Language Models. arXiv:2503.01854 [cs.CL]https://arxiv.org/abs/2503.01854

    2025 arXiv

  16. [24]

    Anna Goldie, Azalia Mirhoseini, Hao Zhou, Irene Cai, and Christopher D. Manning. 2025. Synthetic Data Generation & Multi-Step RL for Reasoning & Tool Use. arXiv:2504.04736 [cs.AI] https:// arxiv.org/abs/2504.04736

    2025 arXiv

  17. [25]

    Google. 2025. Gemini Developer API Pricing. https://ai.google.dev/gemini-api/docs/pricing. Accessed: 2025-05-19

    2025

  18. [26]

    Google Chrome. 2024. Chrome DevTools Recorder.https://developer.chrome.com/docs/devtools/ recorder?hl=zh-cn. Accessed: 2024-03-19

    2024

  19. [27]

    Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang. 2024. MiniLLM: Knowledge Distillation of Large Language Models. arXiv:2306.08543 [cs.CL] https://arxiv.org/abs/2306.08543

    2024 arXiv

  20. [28]

    Idan Habler, Ken Huang, Vineeth Sai Narajala, and Prashant Kulkarni. 2025. Building A Secure Agentic AI Application Leveraging A2A Protocol. arXiv:2504.16902 [cs.CR] https://arxiv.org/abs/2504. 16902

    2025 arXiv

  21. [29]

    Shanshan Han, Salman Avestimehr, and Chaoyang He. 2025. Bridging the Safety Gap: A Guardrail Pipeline for Trustworthy LLM Inferences. arXiv:2502.08142 [cs.AI] https://arxiv.org/abs/2502. 08142 19

    2025 arXiv

  22. [30]

    Bairu Hou, Qibin Chen, Jianyu Wang, Guoli Yin, Chong Wang, Nan Du, Ruoming Pang, Shiyu Chang, and Tao Lei. 2025. Instruction-Following Pruning for Large Language Models. arXiv:2501.02086 [cs.CL] https://arxiv.org/abs/2501.02086

    2025 arXiv

  23. [31]

    Xinyi Hou, Yanjie Zhao, Shenao Wang, and Haoyu Wang. 2025. Model Context Protocol (MCP): Landscape, Security Threats, and Future Research Directions. arXiv:2503.23278 [cs.CR] https: //arxiv.org/abs/2503.23278

    2025 arXiv

  24. [32]

    Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen

    Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. LoRA: Low-Rank Adaptation of Large Language Models. arXiv:2106.09685 [cs.CL] https://arxiv.org/abs/2106.09685

    2021 arXiv

  25. [33]

    Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and Ting Liu. 2025. A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions. ACM Transactions ...

    2025 doi

  26. [34]

    Tenghao Huang, Kinjal Basu, Ibrahim Abdelaziz, Pavan Kapanipathi, Jonathan May, and Muhao Chen. 2025. R2D2: Remembering, Reflecting and Dynamic Decision Making for Web Agents. arXiv:2501.12485 [cs.AI] https://arxiv.org/abs/2501.12485

    2025 arXiv

  27. [35]

    2024.R2-Guard: Robust Reasoning Enabled LLM Guardrail via Knowledge- Enhanced Logical Reasoning

    Mintong Kang and Bo Li. 2024.R2-Guard: Robust Reasoning Enabled LLM Guardrail via Knowledge- Enhanced Logical Reasoning. arXiv:2407.05557 [cs.AI] https://arxiv.org/abs/2407.05557

    2024 arXiv

  28. [36]

    Kelsey Kinzer. 2025. LLM Hallucination Detection in App Development. https://www.comet.com/ site/blog/llm-hallucination/. Accessed: 2025-05-19

    2025

  29. [37]

    Hanna Kim, Minkyoo Song, Seung Ho Na, Seungwon Shin, and Kimin Lee. 2025. When LLMs Go Online: The Emerging Threat of Web-Enabled LLMs. arXiv:2410.14569 [cs.CR]https://arxiv.org/ abs/2410.14569

    2025 arXiv

  30. [38]

    Rui Kong, Qiyang Li, Xinyu Fang, Qingtian Feng, Qingfeng He, Yazhu Dong, Weijun Wang, Yuanchun Li, Linghe Kong, and Yunxin Liu. 2024. LoRA-Switch: Boosting the Efficiency of Dynamic LLM Adapters via System-Algorithm Co-design. arXiv:2405.17741 [cs.AI] https://arxiv.org/abs/2405. 17741

    2024 arXiv

  31. [39]

    Owen Kwon, Abraham George, Alison Bartsch, and Amir Barati Farimani. 2025. RT-cache: Efficient Robot Trajectory Retrieval System. arXiv:2505.09040 [cs.RO]https://arxiv.org/abs/2505.09040

    2025 arXiv

  32. [40]

    Wing Lam, Zhengkai Wu, Dengfeng Li, Wenyu Wang, Haibing Zheng, Hui Luo, Peng Yan, Yuetang Deng, and Tao Xie. 2017. Record and replay for Android: are we there yet in industrial cases?. In Proceedings of the 2017 11th Joint Meeting on Foundations of Software Engineering (Paderb...

    2017

  33. [41]

    Fung, and Paul Pu Liang

    Jaewoo Lee, Keyang Xuan, Chanakya Ekbote, Sandeep Polisetty, Yi R. Fung, and Paul Pu Liang. 2025. TAMP: Token-Adaptive Layerwise Pruning in Multimodal Large Language Models. arXiv:2504.09897 [cs.CV] https://arxiv.org/abs/2504.09897 20

    2025 arXiv

  34. [42]

    Evan Li, Tushin Mallick, Evan Rose, William Robertson, Alina Oprea, and Cristina Nita-Rotaru. 2025. ACE: A Security Architecture for LLM-Integrated App Systems. arXiv:2504.20984 [cs.CR] https: //arxiv.org/abs/2504.20984

    2025 arXiv

  35. [43]

    Siran Li, Linus Stenzel, Carsten Eickhoff, and Seyed Ali Bahrainian. 2025. Enhancing Retrieval- Augmented Generation: A Study of Best Practices. arXiv:2501.07391 [cs.CL] https://arxiv.org/ abs/2501.07391

    2025 arXiv

  36. [44]

    Xuechen Li, Florian Tramèr, Percy Liang, and Tatsunori Hashimoto. 2022. Large Language Models Can Be Strong Differentially Private Learners. arXiv:2110.05679 [cs.LG] https://arxiv.org/abs/ 2110.05679

    2022 arXiv

  37. [45]

    Yubo Li, Xiaobin Shen, Xinyu Yao, Xueying Ding, Yidi Miao, Ramayya Krishnan, and Rema Pad- man. 2025. Beyond Single-Turn: A Survey on Multi-Turn Interactions with Large Language Models. arXiv:2504.04717 [cs.CL] https://arxiv.org/abs/2504.04717

    2025 arXiv

  38. [46]

    Gonzalez

    Kevin Lin, Charlie Snell, Yu Wang, Charles Packer, Sarah Wooders, Ion Stoica, and Joseph E. Gonzalez

  39. [47]

    Gui Ling, Ziyang Wang, Yuliang Yan, and Qingwen Liu. 2024. SlimGPT: Layer-wise Structured Pruning for Large Language Models. arXiv:2412.18110 [cs.AI]https://arxiv.org/abs/2412.18110

    2024 arXiv

  40. [48]

    Bang Liu, Xinfeng Li, Jiayi Zhang, Jinlin Wang, Tanjin He, Sirui Hong, Hongzhang Liu, Shaokun Zhang, Kaitao Song, Kunlun Zhu, Yuheng Cheng, Suyuchen Wang, Xiaoqiang Wang, Yuyu Luo, Haibo Jin, Peiyan Zhang, Ollie Liu, Jiaqi Chen, Huan Zhang, Zhaoyang Yu, Haochen Shi, Boyan Li, ...

    2025 arXiv

  41. [49]

    Yue Liu, Jiaying Wu, Yufei He, Hongcheng Gao, Hongyu Chen, Baolong Bi, Jiaheng Zhang, Zhiqi Huang, and Bryan Hooi. 2025. Efficient Inference for Large Reasoning Models: A Survey. arXiv:2503.23077 [cs.CL] https://arxiv.org/abs/2503.23077

    2025 arXiv

  42. [50]

    Qianren Mao, Qili Zhang, Hanwen Hao, Zhentao Han, Runhua Xu, Weifeng Jiang, Qi Hu, Zhijun Chen, Tyler Zhou, Bo Li, Yangqiu Song, Jin Dong, Jianxin Li, and Philip S. Yu. 2025. Privacy-Preserving Fed- erated Embedding Learning for Localized Retrieval-Augmented Generation. arXiv:...

    2025 arXiv

  43. [51]

    Microsoft. 2024. Playwright Codegen. https://playwright.dev/docs/codegen. Accessed: 2024-03- 19

    2024

  44. [52]

    Microsoft. 2025. Playwright. https://playwright.dev/ Accessed: 2025-05-20

    2025

  45. [53]

    Magnus Müller and Gregor Žunič. 2024. Browser Use: Enable AI to control your browser . https: //github.com/browser-use/browser-use 21

    2024

  46. [54]

    Narek Galstyan. 2024. Application-Integrated Record-Replay of Distributed Systems . Technical Re- port. University of California, Berkeley. https://www2.eecs.berkeley.edu/Pubs/TechRpts/2024/ EECS-2024-4.pdf Accessed: 2025-05-19

    2024

  47. [55]

    Wanli Ni, Haofeng Sun, Huiqing Ao, and Hui Tian. 2025. Federated Intelligence: When Large AI Models Meet Federated Fine-Tuning and Collaborative Reasoning at the Network Edge. arXiv:2503.21412 [cs.AI] https://arxiv.org/abs/2503.21412

    2025 arXiv

  48. [56]

    OpenAI. 2025. OpenAI Platform Pricing. https://platform.openai.com/docs/pricing. Accessed: 2025-05-19

    2025

  49. [57]

    Konstantinos Parasyris, Giorgis Georgakoudis, Esteban Rangel, Ignacio Laguna, and Johannes Doerfert

  50. [58]

    Pedro Pinacho-Davidson, Fernando Gutierrez, Pablo Zapata, Rodolfo Vergara, and Pablo Aqueveque

  51. [59]

    Scott Reed, Konrad Zolna, Emilio Parisotto, Sergio Gomez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley Edwards, Nicolas Heess, Yutian Chen, Raia Hadsell, Oriol Vin...

    2022 arXiv

  52. [60]

    Z. Z. Ren, Zhihong Shao, Junxiao Song, Huajian Xin, Haocheng Wang, Wanjia Zhao, Liyue Zhang, Zhe Fu, Qihao Zhu, Dejian Yang, Z. F. Wu, Zhibin Gou, Shirong Ma, Hongxuan Tang, Yuxuan Liu, Wenjun Gao, Daya Guo, and Chong Ruan. 2025. DeepSeek-Prover-V2: Advancing Formal Mathematic...

    2025 arXiv

  53. [61]

    arXiv:2505.04784 [cs.CR] https://arxiv.org/abs/2505.04784

    A Proposal for Evaluating the Operational Risk for ChatBots based on Large Language Models. arXiv:2505.04784 [cs.CR] https://arxiv.org/abs/2505.04784

    arXiv

  54. [62]

    rr-debugger. 2025. Record and Replay Framework.https://github.com/rr-debugger/rr. Accessed: 2025-05-19

    2025

  55. [63]

    Lee, and Josep Torrellas

    Yasser Shalabi, Mengjia Yan, Nima Honarmand, Ruby B. Lee, and Josep Torrellas. 2018. Record- Replay Architecture as a General Security Framework. In2018 IEEE International Symposium on High Performance Computer Architecture (HPCA). 180–193. https://doi.org/10.1109/HPCA.2018.00025

    2018

  56. [64]

    Gleb Rodionov, Roman Garipov, Alina Shutova, George Yakushev, Vage Egiazarian, Anton Sinitsin, De- nis Kuznedelev, and Dan Alistarh. 2025. Hogwild! Inference: Parallel LLM Generation via Concurrent Attention. arXiv:2504.06261 [cs.LG] https://arxiv.org/abs/2504.06261

    2025

  57. [65]

    FromMindtoMachine: TheRiseofManusAIasaFullyAutonomous Digital Agent

    MinjieShenandQikaiYang.2025. FromMindtoMachine: TheRiseofManusAIasaFullyAutonomous Digital Agent. arXiv:2505.02024 [cs.AI] https://arxiv.org/abs/2505.02024

    2025 arXiv

  58. [66]

    Gonzalez, and Ion Stoica

    Ying Sheng, Shiyi Cao, Dacheng Li, Coleman Hooper, Nicholas Lee, Shuo Yang, Christopher Chou, Banghua Zhu, Lianmin Zheng, Kurt Keutzer, Joseph E. Gonzalez, and Ion Stoica. 2024. S-LoRA: Serving Thousands of Concurrent LoRA Adapters. arXiv:2311.03285 [cs.LG] https://arxiv.org/ ...

    2024 arXiv

  59. [67]

    Chenyang Shao, Xinyuan Hu, Yutang Lin, and Fengli Xu. 2025. Division-of-Thoughts: Harnessing Hybrid Language Model Synergy for Efficient On-Device Agents. arXiv:2502.04392 [cs.CL] https: //arxiv.org/abs/2502.04392 22

    2025 arXiv

  60. [68]

    Aditi Singh, Abul Ehtesham, Saket Kumar, and Tala Talaei Khoei. 2025. Agentic Retrieval-Augmented Generation: A Survey on Agentic RAG. arXiv:2501.09136 [cs.AI] https://arxiv.org/abs/2501. 09136

    2025 arXiv

  61. [69]

    Zihe Song, S M Hasan Mansur, Ravishka Rathnasuriya, Yumna Fatima, Wei Yang, Kevin Moran, and Wing Lam. 2025. Can You Mimic Me? Exploring the Use of Android Record & Replay Tools in Debugging. arXiv:2504.20237 [cs.SE] https://arxiv.org/abs/2504.20237

    2025 arXiv

  62. [70]

    Yuchen Shi, Siqi Cai, Zihan Xu, Yuei Qin, Gang Li, Hang Shao, Jiawei Chen, Deqing Yang, Ke Li, and Xing Sun. 2025. FlowAgent: Achieving Compliance and Flexibility for Workflow Agents. arXiv:2502.14345 [cs.AI] https://arxiv.org/abs/2502.14345

    2025 arXiv

  63. [71]

    ASimpleandEffectivePruningApproach for Large Language Models

    MingjieSun, ZhuangLiu, AnnaBair, andJ.ZicoKolter.2024. ASimpleandEffectivePruningApproach for Large Language Models. arXiv:2306.11695 [cs.CL]https://arxiv.org/abs/2306.11695

    2024 arXiv

  64. [72]

    Yiliu Sun, Yanfang Zhang, Zicheng Zhao, Sheng Wan, Dacheng Tao, and Chen Gong. 2025. Fast-Slow- Thinking: Complex Task Solving with Large Language Models. arXiv:2504.08690 [cs.CL] https: //arxiv.org/abs/2504.08690

    2025 arXiv

  65. [73]

    Yang Sui, Yu-Neng Chuang, Guanchu Wang, Jiamu Zhang, Tianyi Zhang, Jiayi Yuan, Hongyi Liu, Andrew Wen, Shaochen Zhong, Hanjie Chen, and Xia Hu. 2025. Stop Overthinking: A Survey on Efficient Reasoning for Large Language Models. arXiv:2503.16419 [cs.CL]https://arxiv.org/abs/ 2503.16419

    2025 arXiv

  66. [74]

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, ...

    2025 arXiv

  67. [75]

    Pierre-Ugo Tournoux, Jeremie Leguay, Farid Benbadis, John Whitbeck, Vania Conan, and Marcelo Dias de Amorim. 2011. Density-Aware Routing in Highly Dynamic DTNs: The RollerNet Case.IEEE Transactions on Mobile Computing 10, 12 (2011), 1755–1768. https://doi.org/10.1109/TMC.2010. 247

    2011 doi

  68. [76]

    Yingshui Tan, Yilei Jiang, Yanshi Li, Jiaheng Liu, Xingyuan Bu, Wenbo Su, Xiangyu Yue, Xiaoyong Zhu, and Bo Zheng. 2025. Equilibrate RLHF: Towards Balancing Helpfulness-Safety Trade-off in Large Language Models. arXiv:2502.11555 [cs.AI] https://arxiv.org/abs/2502.11555

    2025 arXiv

  69. [77]

    Yu, Qingsong Wen, and Yang Liu

    Kun Wang, Guibin Zhang, Zhenhong Zhou, Jiahao Wu, Miao Yu, Shiqian Zhao, Chenlong Yin, Jinhu Fu, Yibo Yan, Hanjun Luo, Liang Lin, Zhihao Xu, Haolang Lu, Xinye Cao, Xinyun Zhou, Weifei Jin, Fanci Meng, Junyuan Mao, Hao Wu, Minghe Wang, Fan Zhang, Junfeng Fang, Chengwei Liu, Yif...

    2025 arXiv

  70. [78]

    Shangshang Wang, Julian Asilis, Ömer Faruk Akgül, Enes Burak Bilgin, Ollie Liu, and Willie Neiswanger. 2025. Tina: Tiny Reasoning Models via LoRA. arXiv:2504.15777 [cs.CL] https: //arxiv.org/abs/2504.15777

    2025 arXiv

  71. [79]

    Hongyu Wang, Shuming Ma, and Furu Wei. 2025. BitNet v2: Native 4-bit Activations with Hadamard Transformation for 1-bit LLMs. arXiv:2504.18415 [cs.CL]https://arxiv.org/abs/2504.18415

    2025 arXiv

  72. [80]

    Yiping Wang, Qing Yang, Zhiyuan Zeng, Liliang Ren, Lucas Liu, Baolin Peng, Hao Cheng, Xuehai He, Kuan Wang, Jianfeng Gao, Weizhu Chen, Shuohang Wang, Simon Shaolei Du, and Yelong Shen

  73. [81]

    Zhenhailong Wang, Haiyang Xu, Junyang Wang, Xi Zhang, Ming Yan, Ji Zhang, Fei Huang, and Heng Ji. 2025. Mobile-Agent-E: Self-Evolving Mobile Assistant for Complex Tasks. arXiv:2501.11733 [cs.CL] https://arxiv.org/abs/2501.11733

    2025 arXiv

  74. [82]

    Yongdong Wang, Runze Xiao, Jun Younes Louhi Kasahara, Ryosuke Yajima, Keiji Nagatani, At- sushi Yamashita, and Hajime Asama. 2025. DART-LLM: Dependency-Aware Multi-Robot Task Decomposition and Execution using Large Language Models. arXiv:2411.09022 [cs.RO] https: //arxiv.org/a...

    2025 arXiv

  75. [83]

    Haoqi Wu, Wei Dai, Li Wang, and Qiang Yan. 2025. Cape: Context-Aware Prompt Perturbation Mechanism with Differential Privacy. arXiv:2505.05922 [cs.CR]https://arxiv.org/abs/2505.05922

    2025 arXiv

  76. [84]

    arXiv:2504.20571 [cs.LG] https://arxiv.org/abs/2504.20571

    Reinforcement Learning for Reasoning in Large Language Models with One Training Example. arXiv:2504.20571 [cs.LG] https://arxiv.org/abs/2504.20571

    arXiv

  77. [85]

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. arXiv:2210.03629 [cs.CL] https: //arxiv.org/abs/2210.03629

    2023 arXiv

  78. [86]

    Peter West and Christopher Potts. 2025. Base Models Beat Aligned Models at Randomness and Cre- ativity. arXiv:2505.00047 [cs.CL] https://arxiv.org/abs/2505.00047

    2025 arXiv

  79. [87]

    Tianyi Zhang, Yang Sui, Shaochen Zhong, Vipin Chaudhary, Xia Hu, and Anshumali Shrivastava. 2025. 70% Size, 100% Accuracy: Lossless LLM Compression for Efficient GPU Inference via Dynamic-Length Float. arXiv:2504.11651 [cs.LG] https://arxiv.org/abs/2504.11651

    2025

  80. [88]

    Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli. 2025. Hallucination is Inevitable: An Innate Limitation of Large Language Models. arXiv:2401.11817 [cs.CL]https://arxiv.org/abs/2401.11817

    2025 arXiv

  81. [89]

    Andrew Zhao, Yiran Wu, Yang Yue, Tong Wu, Quentin Xu, Yang Yue, Matthieu Lin, Shenzhi Wang, Qingyun Wu, Zilong Zheng, and Gao Huang. 2025. Absolute Zero: Reinforced Self-play Reasoning with Zero Data. arXiv:2505.03335 [cs.LG] https://arxiv.org/abs/2505.03335

    2025 arXiv

  82. [90]

    Chaoyun Zhang, He Huang, Chiming Ni, Jian Mu, Si Qin, Shilin He, Lu Wang, Fangkai Yang, Pu Zhao, Chao Du, Liqun Li, Yu Kang, Zhao Jiang, Suzhen Zheng, Rujia Wang, Jiaxu Qian, Minghua Ma, Jian-Guang Lou, Qingwei Lin, Saravan Rajmohan, and Dongmei Zhang. 2025. UFO2: The Desktop ...

    2025 arXiv

  83. [91]

    Xingyu Zheng, Yuye Li, Haoran Chu, Yue Feng, Xudong Ma, Jie Luo, Jinyang Guo, Haotong Qin, Michele Magno, and Xianglong Liu. 2025. An Empirical Study of Qwen3 Quantization. arXiv:2505.02214 [cs.LG] https://arxiv.org/abs/2505.02214

    2025 arXiv

  84. [92]

    Yuxuan Zhang et al. 2023. MobileGPT: Augmenting LLM with Human-like App Memory for Mobile Task Automation. arXiv preprint arXiv:2312.03003 (2023). https://arxiv.org/pdf/2312.03003

    2023 arXiv

  85. [93]

    Yifei Zhou, Song Jiang, Yuandong Tian, Jason Weston, Sergey Levine, Sainbayar Sukhbaatar, and Xian Li. 2025. SWEET-RL: Training Multi-Turn LLM Agents on Collaborative Reasoning Tasks. arXiv:2503.15478 [cs.LG] https://arxiv.org/abs/2503.15478 26

    2025 arXiv

  86. [94]

    Xinran Zhao, Hanie Sedghi, Bernd Bohnet, Dale Schuurmans, and Azade Nova. 2025. Improving Large Language Model Planning with Action Sequence Similarity. arXiv:2505.01009 [cs.AI] https: //arxiv.org/abs/2505.01009 25

    2025 arXiv

  87. [96]

    Wangchunshu Zhou, Yuchen Eleanor Jiang, Long Li, Jialong Wu, Tiannan Wang, Shi Qiu, Jintian Zhang, Jing Chen, Ruipu Wu, Shuai Wang, Shiding Zhu, Jiyu Chen, Wentao Zhang, Xiangru Tang, Ningyu Zhang, Huajun Chen, Peng Cui, and Mrinmaya Sachan. 2023. Agents: An Open-source Frame-...

    2023 arXiv

  88. [2023]

    InProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (Denver, CO, USA)(SC ’23)

    Scalable Tuning of (OpenMP) GPU Applications via Kernel Record and Replay. InProceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis (Denver, CO, USA)(SC ’23). Association for Computing Machinery, New York, NY, USA, Article ...

  89. [2025]

    arXiv:2504.13171 [cs.AI] https: //arxiv.org/abs/2504.13171

    Sleep-time Compute: Beyond Inference Scaling at Test-time. arXiv:2504.13171 [cs.AI] https: //arxiv.org/abs/2504.13171

    arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.