Pith. sign in

REVIEW 2 major objections 5 minor 57 references

Compiling repeated SOP steps into offline-validated tools cuts production alarm-triage latency by 42% and error rate by up to 53%.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-10 13:46 UTC pith:AJG6HUZA

load-bearing objection Solid production systems paper: offline-compiled SOP tools cut real FC alarm latency and error, with ablations that actually hold up. the 2 major comments →

arxiv 2607.08010 v1 pith:AJG6HUZA submitted 2026-07-09 cs.CL cs.LGcs.SE

Tool-Making and Self-Evolving LLM Agents in Low-Latency Systems

Kalle Kujanp\"a\"a , Ning Liu , Shahnawaz Alam , Yeshwanth Reddy Sura , Tianyu Yang , Kristina Klinkner , Shervin Malmasi This is my paper
classification cs.CL cs.LGcs.SE
keywords LLM agentstool makingself-evolving agentsstandard operating procedureslow-latency systemsalarm triageCodeActfulfillment center
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Production coding agents keep rewriting the same metric queries every time they walk a standard operating procedure, which burns latency and injects run-to-run variance. This paper shows that an offline tool-making pipeline can compile each repeated SOP node into a single, environment-grounded, versioned tool: a data-collector first runs against the live backend to capture schemas and traces, a tool-maker writes a candidate, and a reflector-repair loop fixes it against labeled cases. At runtime the production agent simply calls those tools and falls back to code generation only when a tool is missing or fails. Deployed on a 44-node fulfillment-center alarm-triage agent over heterogeneous metric backends, the approach cuts p50 latency 42% in production and reduces end-to-end error rate by up to 53% on 1,500 historical alarms. Compact structured verdicts further let the main agent call tools directly, removing a sub-agent layer and cutting p50 another 62% in a controlled ablation. The same versioned tools also surface upstream schema drift and underspecified SOP steps that free-form code generation had previously hidden.

Core claim

Repeated inference-time code generation for stable SOP steps can be replaced by offline-compiled, environment-grounded tools that return deterministic structured verdicts. In a live 44-node fulfillment-center alarm-triage system this substitution reduces production p50 latency by 42% and end-to-end error rate by up to 53% on 1,500 historical alarms, while also enabling a simpler direct-call architecture that cuts p50 a further 62% in ablation.

What carries the argument

The agentic tool-making pipeline: for each SOP node a data-collector sub-agent gathers live MCP traces and schemas, a tool-maker LLM synthesizes a candidate function with a uniform (warehouse, timestamp, context) to verdict signature, and a reflector-repair loop iterates against labeled cases until the tool passes the held-out set; the resulting versioned tools are then invoked at runtime with CodeAct fallback.

Load-bearing premise

That a small set of SME-labeled cases plus a few live execution traces is a faithful enough stand-in for the full SOP intent that tools validated offline will stay correct under live schema drift and edge cases the labels never saw.

What would settle it

On a fresh temporal hold-out of production alarms, measure whether tools that passed offline evaluation still match expert-labeled verdicts after real upstream schema or value-format changes; a large rise in silent mismatches or forced CodeAct fallbacks would falsify the claim that offline grounding transfers.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Industrial agents that repeatedly walk written procedures can move most of their latency and variance off the critical path by compiling those steps once.
  • Compact structured tool verdicts remove the need for multi-turn sub-agent layers, enabling simpler direct-call architectures with large further latency gains.
  • Versioned tools with stable inputs and outputs turn previously hidden upstream data drift and SOP underspecification into monitorable, auditable signals.
  • Once tools exist, residual failures concentrate on specification quality rather than the agent's ability to rediscover schemas, so SOP clarification becomes the highest-leverage remaining fix.
  • A single tool library amortizes generation cost across models and sites when site-specific thresholds are supplied as runtime parameters.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same offline compile-and-repair pattern should transfer to other stable-backend workflows (incident runbooks, policy checks, text-to-SQL) where inference-time coding currently dominates cost.
  • Label-free variants already within a few points of the labeled ceiling suggest the approach can scale to new nodes without full expert labeling once voting and judge filters are mature.
  • Because tools expose specification gaps that free-form code hid, organizations may need new processes for owning and versioning SOPs as first-class software artifacts.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes an offline agentic tool-making pipeline that compiles repeated SOP decision nodes into environment-grounded, validated, versioned tools, replacing per-request CodeAct-style code generation at inference. A data-collector sub-agent gathers live MCP traces; a tool-maker synthesizes candidates; a reflector–repair loop tests against labeled cases. Deployed in a 44-node fulfillment-center outbound-dock alarm-triage agent, tool calls cut production p50 latency by 42% versus a sub-agent CodeAct baseline (Table 3), reduce end-to-end error rate by up to 53% on 1,500 historical alarms (Table 4), and enable a direct-call architecture that further cuts p50 by 62% in a controlled ablation holding model fixed. Component ablations (Tables 1, 5–6, 9) show both the data-collection trace and the test–repair loop are necessary for high per-node pass@1; residual failures concentrate on four underspecified nodes that improve to 99.9% after SOP/data fixes (Table 2). Versioned tools also surface upstream schema and drift issues in production (§6).

Significance. If the reported production and offline results hold, the work supplies a concrete, deployable operating pattern for self-evolving industrial agents: amortize repeated procedural coding offline under test, keep CodeAct as fallback, and use stable tool I/O for auditability and drift detection. Strengths include multi-seed pass@1 ablations (30 seeds), bootstrap significance on end-to-end accuracy (Tables 12–14), a controlled architecture ablation, mixed-trajectory fine-tuning that preserves fallback, and post-deployment evidence that tools expose real upstream inconsistencies. The contribution is primarily systems/empirical rather than theoretical, but it is well-scoped to latency-sensitive SOP workflows and is more thoroughly evaluated in production than most related tool-making or SOP-agent papers.

major comments (2)
  1. The central production claim rests on an offline SME-specified labeler as the surrogate for SOP intent (§3.1, Appendix A). Residual errors concentrate on four underspecified nodes (§4.3, Table 2), and Limitations correctly note that full automation of the improvement loop remains open. The paper should more explicitly quantify how often tools disagree with the labeler under live shadow traffic, and state the promotion criteria (held-out pass rate + human review) as a first-class reliability bound rather than only a process note, so readers can judge transfer risk beyond this FC setting.
  2. Table 3 row 3 confounds architecture (main-agent direct calls) with model change (GLM-4.5-Air). The offline ablation that holds model fixed and reports a further 62% p50 cut is load-bearing for the architectural claim (§5.1) but is only described in text. That ablation should be elevated to a table (with p50/p99 and seed variance) so the architecture effect is as inspectable as the tool-vs-CodeAct comparison.
minor comments (5)
  1. Absolute production latencies are withheld for competitive reasons (Appendix A); normalized ratios are acceptable, but state the measurement window (number of alarms / time span) for Table 3 so p50/p99 stability can be assessed.
  2. Figure 1 is clear; consider adding a short callout that tools return structured (verdict, value, threshold, explanation) so the direct-call redesign is motivated visually as well as in §5.1.
  3. Appendix D’s repair-loop “unlearning” example is valuable; a one-sentence pointer in §4.3 would help main-text readers find the concrete failure mode.
  4. Related Work could more sharply contrast Pei et al. (2025) and SOP-Bench-style evaluations with the production deployment and temporal held-out splits used here.
  5. Typos/style: “adata-collection” / “atest–repair” spacing in §3.1; ensure consistent “pass@1” vs “pass@1 (%)” labeling across Figure 2 and Table 1.

Circularity Check

0 steps flagged

No significant circularity: latency and accuracy claims are external operational measurements, not tautologies of the tool-making objective.

full rationale

This is an empirical systems paper whose load-bearing claims are production p50/p99 latency (Table 3), end-to-end accuracy on 1,500 historical alarms with bootstrap significance (Tables 4, 12–14), and component ablations of the offline tool-maker (Tables 1, 5–6, 9, 15). Tools are synthesized and repaired on labeled training cases, then scored by per-node pass@1 on a temporal held-out evaluation set; production promotion further uses shadow traffic and human review. Latency is measured on live alarms; accuracy is scored against SME-specified ground-truth observations, root causes, and actions. None of these outcomes is forced by construction of the training objective or by renaming a fitted parameter as a prediction. Self-citation of Eluna (Liu et al., 2026b) supplies only the hierarchical baseline architecture, not a uniqueness theorem or ansatz that forces the reported deltas. Residual failures on underspecified SOP nodes and reliance on an offline labeler are validity/scope caveats (Limitations; §4.3), not circular reductions. The derivation chain is therefore self-contained against external operational benchmarks.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 2 invented entities

This is an empirical systems paper. The central claims rest on domain assumptions about stable backends and SOP structure, on SME-derived labels, and on engineering hyperparameters of the tool-making loop—not on free physical constants or invented particles. The main invented construct is the versioned per-node tool library produced by the pipeline.

free parameters (4)
  • repair_loop_budget
    Deployed repair loop uses a fixed budget of three rounds (with early stop on full training-set pass); alternative round counts and multi-hypothesis branching are ablated but the production choice is a hand-set hyperparameter.
  • data_collector_sample_count
    Three labeled training cases per node (balanced across verdicts) are used to build the data-collection trace; N is a design choice shown to be data-efficient but still free.
  • tool_dropout_mixture_for_distillation
    Teacher trajectories use 75% with tools (25% random tool dropout) and 25% fully tool-free so the student retains fallback behavior; mixture weights are chosen by the authors.
  • LoRA_rank_and_learning_rate
    Student fine-tunes use rank/α 256 and lr 2e-4 for Qwen3 32B, and 32 / 1e-4 for GLM-4.5-Air; these are fitted training choices that affect the deployed agent.
axioms (5)
  • domain assumption Repeated SOP decision nodes against a relatively stable metric backend are the dominant source of latency and run-to-run variance in the baseline CodeAct sub-agent loop.
    Stated in §1–§3 as the motivation for compiling nodes into tools; the production gains depend on this workload structure.
  • domain assumption An offline SME-specified labeler produces correct (input, verdict) pairs for each decision node.
    §3.1 and Appendix A: training and evaluation cases are labeled by an offline labeler whose logic was specified by subject-matter experts.
  • domain assumption Live MCP execution traces (code, schemas, values, graded verdicts) supply the execution details omitted by human-authored SOP text.
    Core grounding claim of the tool-making pipeline (§3.1); ablations show +D is necessary for high pass@1.
  • domain assumption When a tool is unavailable or throws, falling back to the baseline CodeAct sub-agent preserves task completeness.
    Runtime design in §3.1–3.2; mixed-trajectory fine-tuning is used to keep fallback accuracy statistically comparable.
  • standard math Standard supervised distillation / rejection-sampling / LoRA training is a valid way to obtain a small production agent that follows the SOP with tools.
    §3.2 cites Yuan et al. (2023) rejection sampling and Hu et al. (2022) LoRA; treated as established ML practice.
invented entities (2)
  • Per-SOP-node versioned tool library with uniform (warehouse, timestamp, context) → verdict signature independent evidence
    purpose: Replace multi-turn inference-time code generation for each decision node with a single deterministic call returning structured true/false/no-data plus detail.
    The tools are synthesized artifacts of the pipeline, not pre-existing APIs; they are the main engineered object the paper introduces and deploys.
  • Agentic tool-making pipeline (data-collector + tool-maker + reflector repair loop) independent evidence
    purpose: Compile underspecified SOP text into environment-grounded tools offline before production use.
    Composition of known LLM roles into a specific offline compile-test-repair system for industrial SOPs; evidence is the measured pass@1 and production metrics.

pith-pipeline@v1.1.0-grok45 · 22967 in / 3579 out tokens · 29843 ms · 2026-07-10T13:46:49.561191+00:00 · methodology

0 comments
read the original abstract

Production LLM agents often waste latency and reliability by regenerating code for the same procedural steps on every request. We replace this inference-time coding loop with an agentic tool-making pipeline that compiles repeated SOP steps into validated, versioned tools before deployment. The tool-maker grounds synthesis in the live environment as it collects execution traces, observes backend schemas and values, generates candidate tools, and repairs them against labeled cases. At runtime, the production agent calls these tools directly and falls back to code generation only when needed. We deploy the approach in a Fulfillment Center alarm-triage system, where an agent diagnoses alarms against a 44-node SOP over heterogeneous metric backends. In production, tool calls reduce p50 latency by 42%. On 1,500 historical alarms, they reduce end-to-end error rate by up to 53% by suppressing run-to-run variance in repeated steps. Because tools return compact structured verdicts, they also enable a simpler direct-call architecture, reducing p50 latency by a further 62% in a controlled ablation. Versioned tools also improve auditability and expose specification gaps and upstream data drift. Our results show that self-evolving agents can make industrial LLM systems faster, more reliable, and easier to operate.

Figures

Figures reproduced from arXiv: 2607.08010 by Kalle Kujanp\"a\"a, Kristina Klinkner, Ning Liu, Shahnawaz Alam, Shervin Malmasi, Tianyu Yang, Yeshwanth Reddy Sura.

Figure 1
Figure 1. Figure 1: Offline (top), each SOP node is compiled into a tool: a data-collector sub-agent runs against MCP, [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Per-node pass@1 (left) and generation cost in [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: pass@1 across repair rounds for each repair strategy, without the data-collection trace (SOP+R, Opus). [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: pass@1 vs. training-set size, with and without data grounding (Opus). [PITH_FULL_IMAGE:figures/full_fig_p012_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Per-node pass@1 (left bars) and generation cost in tokens per seed (right bars) for each ablation [PITH_FULL_IMAGE:figures/full_fig_p013_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

57 extracted references · 57 canonical work pages · 18 internal anchors

  1. [1]

    Aho and Jeffrey D

    Alfred V. Aho and Jeffrey D. Ullman , title =. 1972

    work page 1972

  2. [2]

    Publications Manual , year = "1983", publisher =

    work page 1983

  3. [3]

    Chandra and Dexter C

    Ashok K. Chandra and Dexter C. Kozen and Larry J. Stockmeyer , year = "1981", title =. doi:10.1145/322234.322243

    work page doi:10.1145/322234.322243 1981

  4. [4]

    Scalable training of

    Andrew, Galen and Gao, Jianfeng , booktitle=. Scalable training of

    work page

  5. [5]

    Dan Gusfield , title =. 1997

    work page 1997

  6. [6]

    Tetreault , title =

    Mohammad Sadegh Rasooli and Joel R. Tetreault , title =. Computing Research Repository , volume =. 2015 , url =

    work page 2015

  7. [7]

    A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =

    Ando, Rie Kubota and Zhang, Tong , Issn =. A Framework for Learning Predictive Structures from Multiple Tasks and Unlabeled Data , Volume =. Journal of Machine Learning Research , Month = dec, Numpages =

    work page

  8. [8]

    Proceedings of the Nineteenth European Conference on Computer Systems , pages=

    Automatic Root Cause Analysis via Large Language Models for Cloud Incidents , author=. Proceedings of the Nineteenth European Conference on Computer Systems , pages=

    work page

  9. [9]

    Proceedings of the 33rd ACM international conference on information and knowledge management , pages=

    Rcagent: Cloud root cause analysis by autonomous agents with tool-augmented large language models , author=. Proceedings of the 33rd ACM international conference on information and knowledge management , pages=

    work page

  10. [10]

    SOP-Agent: Empower General Purpose AI Agent with Domain-Specific SOPs

    Sop-agent: Empower general purpose ai agent with domain-specific sops , author=. arXiv preprint arXiv:2501.09316 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  11. [11]

    Forty-first International Conference on Machine Learning , year=

    Executable code actions elicit better llm agents , author=. Forty-first International Conference on Machine Learning , year=

    work page

  12. [12]

    arXiv preprint arXiv:2506.08119 , year=

    SOP-Bench: Complex Industrial SOPs for Evaluating LLM Agents , author=. arXiv preprint arXiv:2506.08119 , year=

    work page arXiv

  13. [13]

    arXiv preprint arXiv:2510.08942 , year=

    SOP-Maze: Evaluating Large Language Models on Complicated Business Standard Operating Procedures , author=. arXiv preprint arXiv:2510.08942 , year=

    work page arXiv

  14. [14]

    arXiv preprint arXiv:2601.22208 , year=

    Stalled, Biased, and Confused: Uncovering Reasoning Failures in LLMs for Cloud-Based Root Cause Analysis , author=. arXiv preprint arXiv:2601.22208 , year=

    work page arXiv

  15. [15]

    Companion Proceedings of the ACM on Web Conference 2025 , pages=

    Flow-of-Action: SOP Enhanced LLM-Based Multi-Agent System for Root Cause Analysis , author=. Companion Proceedings of the ACM on Web Conference 2025 , pages=

    work page 2025

  16. [16]

    CREATOR: Tool Creation for Disentangling Abstract and Concrete Reasoning of Large Language Models

    Creator: Disentangling abstract and concrete reasonings of large language models through tool creation , author=. arXiv preprint arXiv:2305.14318 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  17. [17]

    International Conference on Learning Representations , volume=

    Large language models as tool makers , author=. International Conference on Learning Representations , volume=

    work page

  18. [18]

    International Conference on Learning Representations , volume=

    Craft: Customizing llms by creating and retrieving from specialized toolsets , author=. International Conference on Learning Representations , volume=

    work page

  19. [19]

    TroVE: Inducing Verifiable and Efficient Toolboxes for Solving Programmatic Tasks

    Trove: Inducing verifiable and efficient toolboxes for solving programmatic tasks , author=. arXiv preprint arXiv:2401.12869 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  20. [20]

    Voyager: An Open-Ended Embodied Agent with Large Language Models

    Voyager: An open-ended embodied agent with large language models , author=. arXiv preprint arXiv:2305.16291 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  21. [21]

    Agent Workflow Memory

    Agent workflow memory , author=. arXiv preprint arXiv:2409.07429 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  22. [22]

    SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills

    SkillWeaver: Web Agents can Self-Improve by Discovering and Honing Skills , author=. arXiv preprint arXiv:2504.07079 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  23. [23]

    Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Llm agents making agent tools , author=. Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    work page

  24. [24]

    Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track , pages=

    Towards Enforcing Company Policy Adherence in Agentic Workflows , author=. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: Industry Track , pages=

    work page 2025

  25. [25]

    StepFly: Agentic Troubleshooting Guide Automation for Incident Diagnosis

    StepFly: Agentic Troubleshooting Guide Automation for Incident Diagnosis , author=. arXiv preprint arXiv:2510.10074 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  26. [26]

    GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models

    GLM-4.5: Agentic, Reasoning, and Coding (ARC) Foundation Models , author=. arXiv preprint arXiv:2508.06471 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  27. [27]

    International Conference on Learning Representations , volume=

    Is self-repair a silver bullet for code generation? , author=. International Conference on Learning Representations , volume=

    work page

  28. [28]

    Advances in neural information processing systems , volume=

    Toolformer: Language models can teach themselves to use tools , author=. Advances in neural information processing systems , volume=

    work page

  29. [29]

    Advances in Neural Information Processing Systems , volume=

    Gorilla: Large language model connected with massive apis , author=. Advances in Neural Information Processing Systems , volume=

    work page

  30. [30]

    International conference on machine learning , pages=

    Pal: Program-aided language models , author=. International conference on machine learning , pages=. 2023 , organization=

    work page 2023

  31. [31]

    ReAct: Synergizing Reasoning and Acting in Language Models

    React: Synergizing reasoning and acting in language models , author=. arXiv preprint arXiv:2210.03629 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  32. [32]

    SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints

    SOPBench: Evaluating Language Agents at Following Standard Operating Procedures and Constraints , author=. arXiv preprint arXiv:2503.08669 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  33. [33]

    arXiv preprint arXiv:2602.02585 , year=

    Agentic Observability: Automated Alert Triage for Adobe E-Commerce , author=. arXiv preprint arXiv:2602.02585 , year=

    work page arXiv

  34. [34]

    Bian Que: An Agentic Framework with Flexible Skill Arrangement for Online System Operations

    Bian Que: An Agentic Framework with Flexible Skill Arrangement for Online System Operations , author=. arXiv preprint arXiv:2604.26805 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  35. [35]

    Advances in neural information processing systems , volume=

    Self-Refine: Iterative Refinement with Self-Feedback , author=. Advances in neural information processing systems , volume=

    work page

  36. [36]

    Advances in neural information processing systems , volume=

    Reflexion: Language agents with verbal reinforcement learning , author=. Advances in neural information processing systems , volume=

    work page

  37. [37]

    International Conference on Learning Representations , volume=

    Teaching large language models to self-debug , author=. International Conference on Learning Representations , volume=

    work page

  38. [38]

    Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    Self-edit: Fault-aware code editor for code generation , author=. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages=

    work page

  39. [39]

    Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    Debug like a human: A large language model debugger via verifying runtime execution step by step , author=. Findings of the Association for Computational Linguistics: ACL 2024 , pages=

    work page 2024

  40. [40]

    Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineering

    Code generation with alphacodium: From prompt engineering to flow engineering , author=. arXiv preprint arXiv:2401.08500 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  41. [41]

    Advances in Neural Information Processing Systems , volume=

    Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs , author=. Advances in Neural Information Processing Systems , volume=

    work page

  42. [42]

    Proceedings of the 31st International Conference on Computational Linguistics , pages=

    MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL , author=. Proceedings of the 31st International Conference on Computational Linguistics , pages=

    work page

  43. [43]

    APEX-SQL: Talking to the data via Agentic Exploration for Text-to-SQL

    APEX-SQL: Talking to the data via Agentic Exploration for Text-to-SQL , author=. arXiv preprint arXiv:2602.16720 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  44. [44]

    Assessing the Impact of Requirement Ambiguity on LLM-based Function-Level Code Generation

    Assessing the Impact of Requirement Ambiguity on LLM-based Function-Level Code Generation , author=. arXiv preprint arXiv:2604.21505 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  45. [45]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    Benchmarking and improving text-to-sql generation under ambiguity , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    work page 2023

  46. [46]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    Evaluating cross-domain text-to-SQL models and benchmarks , author=. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages=

    work page 2023

  47. [47]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages=

    Understanding the effects of noise in text-to-sql: An examination of the bird-bench benchmark , author=. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers) , pages=

    work page

  48. [48]

    arXiv preprint arXiv:2601.08778 , year=

    Pervasive Annotation Errors Break Text-to-SQL Benchmarks and Leaderboards , author=. arXiv preprint arXiv:2601.08778 , year=

    work page arXiv

  49. [49]

    Scaling Relationship on Learning Mathematical Reasoning with Large Language Models

    Scaling relationship on learning mathematical reasoning with large language models , author=. arXiv preprint arXiv:2308.01825 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  50. [50]

    GLM-5: from Vibe Coding to Agentic Engineering

    GLM-5: from Vibe Coding to Agentic Engineering , author=. arXiv preprint arXiv:2602.15763 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  51. [51]

    Claude Opus 4.6 System Card , year =

    work page

  52. [52]

    Proceedings of the 29th symposium on operating systems principles , pages=

    Efficient memory management for large language model serving with pagedattention , author=. Proceedings of the 29th symposium on operating systems principles , pages=

    work page

  53. [53]

    Qwen3 Technical Report

    Qwen3 Technical Report , author=. arXiv preprint arXiv:2505.09388 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  54. [54]

    The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence

    The MiniMax-M2 Series: Mini Activations Unleashing Max Real-World Intelligence , author=. arXiv preprint arXiv:2605.26494 , year=

    work page internal anchor Pith review Pith/arXiv arXiv

  55. [55]

    Hu, Edward J and Shen, Yelong and Wallis, Phillip and Allen-Zhu, Zeyuan and Li, Yuanzhi and Wang, Shean and Wang, Lu and Chen, Weizhu , booktitle=

    work page

  56. [56]

    2026 , journal =

    Ning Liu and Kalle Kujanp\"a\"a and Zhaoxuan Zhu and P Aditya Sreekar and Kaiwen Liu and Chuanneng Sun and Jorge Marchena Menendez and Matthew Bales and Tianyu Yang and Shahnawaz Alam and Rose Yu and Baoyuan Liu and Kristina Klinkner and Shervin Malmasi , title =. 2026 , journal =

    work page 2026

  57. [57]

    2024 , eprint=

    SWIFT:A Scalable lightWeight Infrastructure for Fine-Tuning , author=. 2024 , eprint=

    work page 2024