Pith. sign in

REVIEW 3 major objections 6 minor 52 references

Reviewed by Pith at T0; open to challenge.

T0 means a machine referee read the full paper against a public rubric. The mark states how deep the mechanical check went, never who wrote it. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · glm-5.2

Make the environment smart, not just the agent

2026-07-09 11:57 UTC pith:DQQC2IFT

load-bearing objection Useful framing with two genuinely strong empirical results (DFC, BranchBench), but the integrated system claim is unvalidated — the safety thesis rests on cross-component DFC that is acknowledged as unsolved. the 3 major comments →

arxiv 2607.07397 v1 pith:DQQC2IFT submitted 2026-07-08 cs.AI cs.DB

Agentic Data Environments

Elaine Ang , Chenxi Huang , Georgios Liargkovas , Jerry Liu , Jinhui Liu , Nikos Pagonas , Charlie Summers , Haonan Wang
show 8 more authors
Jiakai Xu Tianle Zhou Yusen Zhang Zhou Yu Zhuo Zhang Tianyi Peng Kostis Kaffes Eugene Wu
This is my paper
classification cs.AI cs.DB
keywords agentic data environmentsinformation managementdata flow controlbranchingagent safetydata lakesexploratory question answeringprovenance
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that the central bottleneck in deploying autonomous agents is not the agent itself but the data environment it operates within. Today's data systems are passive stores: they serve data on request but do not actively organize information for agent consumption, nor do they constrain the consequences of agent actions. The paper proposes Agentic Data Environments as active substrates that simultaneously amplify agent capabilities and bound failure risk. On the capability side, three mechanisms are introduced: Agentic Information Management (AIM), which automatically transforms raw data sources into task-specific structured representations (e.g., relational schemas) rather than generic vector embeddings; Agentic Information Retrieval (AIR), which enables agents to discover relevant evidence across massive, heterogeneous data lakes through exploratory question answering; and Agentic Data Elicitation (ADE), which surfaces latent signals from the environment through observation or controlled experimentation and materializes them as reusable artifacts. On the safety side, two mechanisms are proposed: branching, which allows agents to explore speculative state changes in isolated copies, and Data Flow Control (DFC), which enforces deterministic policies on how data may be derived, combined, and released. The paper presents empirical evidence for individual components: AIM achieves 49.8% higher accuracy than Mem0 and 15.82% higher than Octen on the LoCoMo benchmark while using approximately 10% of the context length; no existing branchable database system completes the BranchBench benchmark at modest scale; and DFC enforcement incurs approximately zero overhead on TPC-H queries across five database engines. The paper frames the composition of these five mechanisms into a single deployable system with a virtuous flywheel where agents improve the environment that in turn improves future agents, but acknowledges that end-to-end integration and cross-component DFC are not yet demonstrated.

Core claim

The paper identifies that agent failures are increasingly information failures rather than reasoning failures, and that the data environment, not the agent model, is the leverage point for both amplifying capability and bounding risk. The specific empirical contributions are: (1) AIM, a multi-agent system that automatically designs, builds, and evolves task-specific data representations, achieving comparable accuracy to full-context approaches on LoCoMo at roughly 10% of the context length and outperforming specialized memory systems by 15.82-49.8%; (2) LakeQA, a benchmark over a 9.5 TB data lake showing that frontier models achieve at most 23% end-to-end accuracy on exploratory question-ans

What carries the argument

The central mechanism is the shift from passive data stores to active data environments with five complementary components. AIM transforms known data sources into agent-ready capabilities through a four-stage pipeline (Learning, Schema Modeling, Data Loading, Refinement) that generates and evolves structured representations. AIR addresses discovery across massive data lakes via a semantic layer that summarizes lake contents to enable efficient source retrieval. ADE elicits latent signals through passive observation or active controlled experimentation, materializing reusable artifacts. Branching provides state safety through isolated speculative copies, with the Checkpoint-lite (Chkpt) abstr

Load-bearing premise

The paper assumes that five independently benchmarked components (AIM, AIR, ADE, branching, DFC) compose into a single coherent, deployable system, but no end-to-end integration is demonstrated and cross-component data flow control is explicitly acknowledged as unimplemented.

What would settle it

Demonstrate that the five components cannot operate simultaneously without prohibitive performance degradation or correctness violations, or that the virtuous flywheel does not hold because artifacts produced by one component (e.g., AIM schemas) are invalidated by the actions of another (e.g., ADE experiments or branching state divergence).

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If AIM-style automatic representation management proves generalizable, the role of data engineering shifts from manual pipeline construction to specifying high-level guidance for agent-driven pipeline generation and maintenance.
  • The LakeQA result that frontier models achieve at most 23% accuracy, with the dominant failure mode being source discovery rather than reasoning, suggests that scaling model capability alone will not close the performance gap without corresponding advances in data lake navigation infrastructure.
  • If DFC can be extended from per-query enforcement to cross-workflow tracking, it would provide a deterministic alternative to probabilistic LLM-based safety checks, which the paper shows achieve only 0.4 F1 on trivial policy checks.
  • The finding that no branchable database completes BranchBench at modest scale implies that agentic exploration patterns like Monte Carlo Tree Search, which require hundreds or thousands of speculative states, are currently infeasible on production database infrastructure.
  • The virtuous flywheel concept, where agent-produced artifacts compound to improve future task performance, implies that the value of an agentic data environment grows non-linearly with usage, creating potential lock-in effects for early-deployed environments.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The paper benchmarks each component independently, but the claimed virtuous flywheel requires that AIM, AIR, ADE, branching, and DFC operate simultaneously on the same data environment. If component-level overheads compose additively or interfere (e.g., DFC policy checks inside branched speculative states, or ADE experiments triggering branching overhead), the combined system's performance profile
  • The ADE concept of agents running controlled experiments within the environment resembles active learning and system identification. A testable extension would be to measure whether ADE-elicted artifacts transfer across different agent architectures or model families, or whether they overfit to the specific agent that produced them.
  • The paper's framing of data environments as active substrates implies a natural boundary between agent responsibility and environment responsibility. A productive research direction would be to formalize this boundary: which safety guarantees are best enforced by the environment (deterministic, high-overhead) versus by the agent (probabilistic, low-overhead), and how the two layers should coordina
  • If branching and DFC are successfully composed, the resulting environment would allow agents to explore policy-compliant alternatives within branched states, creating a search space constrained by both state isolation and data flow rules. The efficiency of search within this constrained space, relative to unconstrained exploration followed by post-hoc filtering, is an open question with implicatio

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper proposes the concept of Agentic Data Environments: data systems that shift from passive stores to active substrates that both amplify agent capabilities (via AIM, AIR, ADE) and bound failure consequences (via branching and Data Flow Control). The paper presents preliminary empirical results for each component—AIM on LoCoMo, LakeQA for AIR, BranchBench for branching, and TPC-H for DFC—and outlines a research agenda for their composition. The framing of agent safety as a systems problem rather than an agent-design problem is well-motivated and timely.

Significance. The paper's central reframing—treating the data environment as the locus of both capability amplification and safety enforcement—is a valuable contribution to the emerging agentic systems literature. The taxonomy of information availability (known but unstructured, discoverable, latent) into AIM/AIR/ADE is clean and useful. The DFC work (§3.2.1) ships a concrete rewrite-based enforcement mechanism evaluated across five DBMS engines with deterministic guarantees, which is a substantive technical contribution. BranchBench identifies a real gap in current branchable DBMSes. The LakeQA benchmark (1000+ tasks, 9.5 TB data lake, multi-annotator validation) represents significant benchmarking effort. The paper is honest about open problems, which is appropriate for a vision paper.

major comments (3)
  1. §3.2.1 vs §3.2.2: The paper's strongest safety claim is that DFC enforcement incurs ~0 overhead on TPC-H across five engines (§3.2.1). However, this result covers only single-query, DBMS-internal provenance policies. The paper's central thesis—that agentic data environments bound failure consequences—requires cross-component DFC that tracks information flows across SQL, Python, files, prompts, and external APIs (§3.2.2, Example 7). The authors acknowledge this gap explicitly ('enforcing DFC requires tracking how information flows across the entire agentic workflow rather than within a single query'), but the framing in the introduction and §4 presents the ~0 overhead result as evidence for the safety thesis without adequate qualification. The gap between what is evaluated (per-query SQL provenance rewrites) and what the thesis requires (representation-invariant provenance across semantic
  2. §2.1, Example 3: The AIM accuracy claims (49.8% over Mem0, 15.82% over Octen, 4.18× faster than GAM, 13.54% higher relative accuracy) are presented as specific quantitative results but without methodology details, error bars, confidence intervals, or statistical tests. For a paper making comparative performance claims, this is insufficient. At minimum, the evaluation protocol (number of runs, model versions, prompt templates, evaluation metrics) should be specified, and variance should be reported. Without this, the reader cannot assess whether the claimed margins are statistically meaningful.
  3. §4, Figure 6: The 'virtuous flywheel'—where agents improve the environment which in turn improves future agents—is a central architectural claim of the paper, but no end-to-end system integrates all five components (AIM, AIR, ADE, branching, DFC). Each component is evaluated independently on a different benchmark (LoCoMo, LakeQA, BranchBench, TPC-H). The composition assumption—that component-level results will compose into environment-level guarantees—is load-bearing for the paper's thesis but is not demonstrated or even partially validated. The paper would benefit from at least a concrete scenario or small-scale integration showing that, e.g., AIM outputs can be governed by DFC policies, or that branching can be used during AIM pipeline refinement.
minor comments (6)
  1. §3.1.1: BranchBench results (5–4000× latency degradation, 3% completion on Neon, 17% on DoltgreSQL) are presented without full experimental details (scale factors, concurrency settings, timeout thresholds, hardware). A table or appendix with per-system, per-workload results would strengthen the claims.
  2. §3.1.2: The Chkpt results (66 ms filesystem checkpoint, 1.46 s for 1 GB) are presented without comparison methodology details (what exactly is being checkpointed, how many trials, variance).
  3. The paper cites several of its own prior works ([42] for BranchBench/Chkpt, [50] for DFC, [36–38] for ADE examples) as building blocks. The paper should more clearly delineate what is new in this paper versus what is summarized from prior work.
  4. §2.2: LakeQA is described as having '1000+ tasks' created by '5 database Ph.D. students and 4 senior undergraduates' but the abstract of the figure says '4 independent annotators including one database PhD.' These should be reconciled.
  5. §1.1, Eq. (1): The Value = Benefits − Costs framing is intuitive but informal. If it is meant to be more than a motivating analogy, the paper should specify how benefits and costs are measured; otherwise, it should be explicitly labeled as illustrative.
  6. §3.2: The LLM-based policy checking baseline (F1=0.4, 0.8–2.2s latency) is a useful motivating datapoint but lacks details on the prompt template, number of queries, and model versions used.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive review. The referee correctly identifies that the paper's central thesis—data environments as the locus of both capability amplification and safety enforcement—creates a gap between the per-component results we evaluate and the cross-component guarantees the full vision requires. We agree with all three major comments and will revise the manuscript accordingly: (1) we will add explicit qualifications to the introduction and §4 scoping the DFC results to per-query SQL provenance and clearly marking cross-component DFC as future work; (2) we will expand the AIM evaluation description in §2.1 to include the evaluation protocol, model versions, number of runs, and variance; (3) we will add a concrete integration scenario to §4 illustrating how AIM outputs can be governed by DFC policies and how branching can be used during AIM pipeline refinement. We appreciate the referee's recognition of the DFC mechanism, BranchBench, and LakeQA as substantive contributions, and agree that the composition assumption should be made more explicit rather than left implicit.

read point-by-point responses

  1. Referee: §3.2.1 vs §3.2.2: The paper's strongest safety claim is that DFC enforcement incurs ~0 overhead on TPC-H across five engines. However, this covers only single-query, DBMS-internal provenance policies, while the thesis requires cross-component DFC tracking information flows across SQL, Python, files, prompts, and external APIs. The framing in the introduction and §4 presents the ~0 overhead result as evidence for the safety thesis without adequate qualification.

    Authors: The referee is correct. The ~0 overhead result applies specifically to per-query SQL provenance policies enforced via rewrite-based compilation within the DBMS, and the manuscript does not adequately scope this when presenting it in the introduction and §4. We will revise the introduction and §4 to explicitly qualify that the evaluated DFC results cover single-query, DBMS-internal provenance policies, and that cross-component DFC (§3.2.2, Example 7) remains an open research direction. The gap between per-query provenance rewrites and representation-invariant provenance across heterogeneous tools is real and we do not claim to have solved it; we will make this scoping clearer. revision: yes

  2. Referee: §2.1, Example 3: The AIM accuracy claims (49.8% over Mem0, 15.82% over Octen, 4.18× faster than GAM, 13.54% higher relative accuracy) are presented without methodology details, error bars, confidence intervals, or statistical tests. The evaluation protocol should be specified and variance reported.

    Authors: The referee is right that the comparative claims in Example 3 lack the methodological detail needed for the reader to assess statistical significance. We will expand the description to include the evaluation protocol: number of runs, model versions used for the target agent, prompt templates, the specific LoCoMo question categories and evaluation metrics, and reported variance across runs. We will also clarify which LoCoMo question subsets each comparison covers. revision: yes

  3. Referee: §4, Figure 6: The 'virtuous flywheel' is a central architectural claim but no end-to-end system integrates all five components. Each component is evaluated independently on a different benchmark. The composition assumption is load-bearing for the thesis but not demonstrated or even partially validated. The paper would benefit from at least a concrete scenario or small-scale integration showing that component-level results compose.

    Authors: The referee correctly identifies that the composition assumption is load-bearing and currently undemonstrated. We do not have an end-to-end integration of all five components, and we agree that the paper should not present the flywheel as validated. We will add a concrete worked scenario to §4 illustrating how the components would compose—for example, showing that AIM-generated databases can be governed by DFC policies (AIM outputs as DFC-governed sources), and that branching can be used during AIM pipeline refinement to evaluate alternative schemas without corrupting the production environment. This scenario will be framed as illustrative rather than empirically validated, and we will explicitly state that demonstrating composition guarantees is a primary item on the research agenda. revision: yes

Circularity Check

0 steps flagged

No circularity detected — empirical results are genuine measurements against external benchmarks; self-citations provide context but do not force outputs by construction

full rationale

This is a position/vision paper with preliminary empirical results. I examined every load-bearing claim: (1) AIM accuracy on LoCoMo is an empirical comparison against external baselines (Mem0, Octen, GAM) — no fitted parameter is renamed as a prediction. (2) BranchBench results are empirical measurements of existing DBMS systems; the benchmark methodology is self-cited [42] but the results are measurements, not derivations from [42]'s claims. (3) DFC's ~0 overhead on TPC-H is an empirical measurement across five engines; the optimizer-invariance principle from [50] is a design constraint, not a fitted input that forces the overhead result. (4) Chkpt timing comparisons and LakeQA accuracy are straightforward empirical evaluations. The self-citations ([42] Xu et al., [50] Summers et al., [39] Liargkovas et al.) are normal for a position paper building on the authors' prior work, but none create a circular derivation chain where a result reduces to its own inputs by construction. The skeptic's concern about DFC scope (single-query enforcement vs. cross-component data flow control) is a correctness/completeness risk — the paper explicitly acknowledges this gap in §3.2.2 — not a circularity issue. No step exhibits self-definitional reduction, fitted-input-as-prediction, or ansatz-smuggling via self-citation.

Axiom & Free-Parameter Ledger

0 free parameters · 5 axioms · 4 invented entities

The paper introduces no free parameters (no fitted constants). It relies on five domain assumptions, two of which (information failures dominate; agents treat data as means to end) are unstated premises that motivate the framework. The invented entities are conceptual frameworks (Agentic Data Environment, EQA) and artifacts (BranchBench, Chkpt) with partial empirical backing but no released implementations.

axioms (5)
  • domain assumption Agent failures are increasingly information rather than reasoning failures
    Section 2 opens with this claim, which motivates the entire AIM/AIR/ADE framework. No evidence is provided that information failures dominate reasoning failures; it is stated as given.
  • domain assumption Benefits accumulate gradually but costs are abrupt, catastrophic, and difficult to reverse
    Section 1.1, Eq. 1 (Value = Benefits - Costs). This asymmetry assumption, supported by prospect theory citation [17], motivates the entire safety framework. It is reasonable but not formally derived.
  • domain assumption DFC policies must be optimizer-invariant (depend only on contributing input tuples, not execution plan)
    Section 3.2.1. This is a design principle for DFC enforcement. It is justified by the observation that optimizers rewrite plans, but it is assumed rather than proven necessary in all cases.
  • domain assumption The data consumer is no longer a human; agents treat data as a means to an end (task success) rather than faithfully representing reality
    Section 2. This assumption drives the shift from 'representing reality' to 'preparing task-relevant signals.' It is a framing choice that motivates AIM but is not empirically validated as universally true.
  • ad hoc to paper Representation Invariance: the same DFC policy should apply irrespective of whether data moves through SQL, Python, files, or prompts
    Section 3.2.2. This is proposed as a design principle for cross-component DFC but is not formally defined or proven. It is a desideratum that the paper acknowledges is not yet implemented.
invented entities (4)
  • Agentic Data Environment no independent evidence
    purpose: The overarching concept: a data environment that actively prepares information and governs agent interactions
    The concept is defined and motivated but no complete implementation exists. Individual components (AIM, AIR, ADE, branching, DFC) have partial implementations, but the integrated entity is a research vision.
  • Exploratory Question Answering (EQA) independent evidence
    purpose: A new problem setting: agents must iteratively infer needed evidence and search a data lake to find sources
    LakeQA benchmark provides a falsifiable evaluation: ≤23% accuracy across seven frontier models. The benchmark is described in enough detail to be independently assessed, though not yet released.
  • BranchBench independent evidence
    purpose: Benchmark modeling the branch-mutate-evaluate loop for agentic exploration
    Five application patterns are described, and results across Neon, Xata, Tiger, Dolt, PostgreSQL are reported. The benchmark provides falsifiable claims (no system completes at modest scale), though no code is released.
  • Checkpoint-lite (Chkpt) independent evidence
    purpose: OS-level state branching abstraction using copy-on-write
    Preliminary performance numbers are provided (66ms filesystem checkpoint, 1.46s for 1GB state vs 8.84s for Podman/CRIU). These are falsifiable claims, though implementation is not released.

pith-pipeline@v1.1.0-glm · 17924 in / 3474 out tokens · 535839 ms · 2026-07-09T11:57:50.290138+00:00 · methodology

0 comments
read the original abstract

Autonomous agents promise substantial gains in speed, scale, and labor efficiency, but their failures can impose abrupt and often irreversible costs. The central challenge for agentic automation is therefore to increase the benefits of automation while bounding the consequences of failure. While databases remain central to modern computing, agents operate over a broader data environment spanning files, APIs, applications, and system state. In this talk, I will outline early work on Agentic Data Environments -- the execution substrate in which agents operate -- that both amplify agent capabilities and enforce safety guarantees. This perspective reframes data systems from passive stores of state into active substrates for safe, reliable execution.

Figures

Figures reproduced from arXiv: 2607.07397 by Charlie Summers, Chenxi Huang, Elaine Ang, Eugene Wu, Georgios Liargkovas, Haonan Wang, Jerry Liu, Jiakai Xu, Jinhui Liu, Kostis Kaffes, Nikos Pagonas, Tianle Zhou, Tianyi Peng, Yusen Zhang, Zhou Yu, Zhuo Zhang.

Figure 1
Figure 1. Figure 1: This paper describes a shift from systems for [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: AIR finds relevant data sources for EQA or target agents, AIM turns sources into capabilities, and ADE materializes missing signals into new data sources. curates information to improve . The main variable across settings is how available the required information already is. In some cases the relevant data source is known but poorly structured for the task. In others the information exists but must be disc… view at source ↗
Figure 3
Figure 3. Figure 3: AIM takes a target agent, guidance, a set of [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: LakeQA is the first exploratory QA bench [PITH_FULL_IMAGE:figures/full_fig_p005_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Agentic Data Environments must sup￾port full state branching. The core challenge is that branches have transient de￾pendencies. The correct branch state is the closure of ob￾jects a live session depends on, including open files, cached metadata, mutated tables, and shell variables. In [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: The virtuous agent￾environment flywheel. Automation requires a shift from passive databases to agentic data en￾vironments to create a virtuous cycle where agents rely on and also im￾prove the environment. Each task produces answers and reusable artifacts (e.g., schemas, extracted content, indexes, performance models, policies) that make future tasks more accurate, cheaper, and faster. Over time, these bene… view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

52 extracted references · 52 canonical work pages · 10 internal anchors

  1. [1]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin, “Attention is all you need,” inNeural Information Processing Systems, 2017. [Online]. Available: https://api.semanticscholar.org/CorpusID:13756489

    work page 2017

  2. [2]

    Terminal-Bench: Benchmarking Agents on Hard, Realistic Tasks in Command Line Interfaces

    M. A. Merrill, A. G. Shaw, N. Carlini, B. Li, H. Raj, I. Bercovich, L. Shi, J. H. Shin, T. Walshe, E. K. Buchanan, J. Shen, G. Ye, H. Lin, J. Poulos, M. Wang, M. Nezhurina, J. Jitsev, D. Lu, O. M. Mastromichalakis, Z. Xu, Z. Chen, Y . Liu, R. Zhang, L. L. Chen, A. Kashyap, J.-L. Uslu, J. Li, J. Wu, M. Yan, S. Bian, V . Sharma, K. Sun, S. Dillmann, A. Anan...

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  3. [3]

    Model context protocol (mcp),

    Anthropic, “Model context protocol (mcp),” https://github.com/modelcontextprotocol/ modelcontextprotocol, 2024, open protocol for integrating LLM applications with external tools and data sources

    work page 2024

  4. [4]

    Claude skills,

    ——, “Claude skills,” https://github.com/anthropics/skills, 2025, reusable capability packages for Claude Code agents

    work page 2025

  5. [5]

    Agent2agent (a2a) protocol,

    Google and Linux Foundation, “Agent2agent (a2a) protocol,” https://github.com/a2aproject/A2A, 2025, open protocol for communication and interoperability between AI agents. 12

    work page 2025

  6. [6]

    OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

    T. Xie, D. Zhang, J. Chen, X. Li, S. Zhao, R. Cao, T. J. Hua, Z. Cheng, D. Shin, F. Lei, Y . Liu, Y . Xu, S. Zhou, S. Savarese, C. Xiong, V . Zhong, and T. Yu, “Osworld: Benchmarking multimodal agents for open-ended tasks in real computer environments,”ArXiv, vol. abs/2404.07972, 2024. [Online]. Available: https://api.semanticscholar.org/CorpusID:269042918

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  7. [7]

    Vipergpt: Visual inference via python execution for reasoning,

    D. Sur’is, S. Menon, and C. V ondrick, “Vipergpt: Visual inference via python execution for reasoning,” 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 11 854–11 864, 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID:257505358

    work page 2023

  8. [8]

    Can LLM Already Serve as A Database Interface? A BIg Bench for Large-Scale Database Grounded Text-to-SQLs

    J. Li, B. Hui, G. Qu, B. Li, J. Yang, B. Li, B. Wang, B. Qin, R. Cao, R. Geng, N. Huo, C. Ma, K. C. Chang, F. Huang, R. Cheng, and Y . Li, “Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls,”ArXiv, vol. abs/2305.03111, 2023. [Online]. Available: https://api.semanticscholar.org/CorpusID:258547040

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  9. [9]

    Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows

    F. Lei, F. Lei, J. Chen, Y . Ye, R. Cao, D. Shin, H. Su, Z. Suo, H. Gao, H. Gao, P. Yin, V . Zhong, C. Xiong, R. Sun, Q. Liu, S. Wang, and T. Yu, “Spider 2.0: Evaluating language models on real-world enterprise text-to-sql workflows,”ArXiv, vol. abs/2411.07763, 2024. [Online]. Available: https://api.semanticscholar.org/CorpusID:273970164

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  10. [10]

    (2025) Vibe coding fiasco: Replit ai agent goes rogue, deletes com- pany database

    PCMag Staff. (2025) Vibe coding fiasco: Replit ai agent goes rogue, deletes com- pany database. Accessed: 2026-03-05. [Online]. Available: https://www.pcmag.com/news/ vibe-coding-fiasco-replite-ai-agent-goes-rogue-deletes-company-database

    work page 2025

  11. [11]

    A. Down. (2026, Feb.) Amazon cloud outages linked to ai tools at aws. Ac- cessed: 2026-03-05. [Online]. Available: https://www.theguardian.com/technology/2026/feb/20/ amazon-cloud-outages-ai-tools-amazon-web-services-aws

    work page 2026

  12. [12]

    S. Ray. (2023, May) Samsung bans chatgpt and other chatbots for employees after sensitive code leak. Accessed: 2026-03-05. [Online]. Available: https://www.forbes.com/sites/siladityaray/2023/05/02/ samsung-bans-chatgpt-and-other-chatbots-for-employees-after-sensitive-code-leak/

    work page 2023

  13. [13]

    (2025) Chatgpt gmail shadow leak

    The Verge Staff. (2025) Chatgpt gmail shadow leak. Accessed: 2026-03-05. [Online]. Available: https://www.theverge.com/news/781746/chatgpt-gmail-shadow-leak

    work page 2025

  14. [14]

    (2025, Sep.) Salesforce ai data leak via prompt injection

    The Register Staff. (2025, Sep.) Salesforce ai data leak via prompt injection. Accessed: 2026-03-05. [Online]. Available: https://www.theregister.com/2025/09/19/salesforce ai data leak prompt injection/

    work page 2025

  15. [15]

    (2025) Notion ai security incident

    Code Integrity. (2025) Notion ai security incident. Accessed: 2026-03-05. [Online]. Available: https://www.codeintegrity.ai/blog/notion

    work page 2025

  16. [16]

    Humans and automation: Use, misuse, disuse, abuse,

    R. Parasuraman and V . Riley, “Humans and automation: Use, misuse, disuse, abuse,”Human Factors, vol. 39, no. 2, pp. 230–253, 1997

    work page 1997

  17. [17]

    Prospect theory: An analysis of decision under risk,

    D. Kahneman and A. Tversky, “Prospect theory: An analysis of decision under risk,”Econometrica, vol. 47, no. 2, pp. 263–291, 1979

    work page 1979

  18. [18]

    Principles of dataspace systems,

    A. Y . Halevy, M. J. Franklin, and D. Maier, “Principles of dataspace systems,”Proceedings of the twenty-fifth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, 2006. [Online]. Available: https://api.semanticscholar.org/CorpusID:7325481

    work page 2006

  19. [19]

    LongMemEval: Benchmarking Chat Assistants on Long-Term Interactive Memory

    D. Wu, H. Wang, W. Yu, Y . Zhang, K.-W. Chang, and D. Yu, “Longmemeval: Benchmarking chat assistants on long-term interactive memory,”arXiv preprint arXiv:2410.10813, 2024. 13

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  20. [20]

    Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory

    P. Chhikara, D. Khant, S. Aryan, T. Singh, and D. Yadav, “Mem0: Building production-ready ai agents with scalable long-term memory,”arXiv preprint arXiv:2504.19413, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  21. [21]

    Agentic Context Engineering: Evolving Contexts for Self-Improving Language Models

    Q. Zhang, C. Hu, S. Upasani, B. Ma, F. Hong, V . Kamanuru, J. Rainton, C. Wu, M. Ji, H. Liet al., “Agentic context engineering: Evolving contexts for self-improving language models,”arXiv preprint arXiv:2510.04618, 2025

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  22. [22]

    Effective context engineering for ai agents,

    Anthropic, “Effective context engineering for ai agents,” 2025, anthropic Engineer- ing Blog, September 29, 2025. [Online]. Available: https://www.anthropic.com/engineering/ effective-context-engineering-for-ai-agents

    work page 2025

  23. [23]

    Evaluating very long-term conversational memory of llm agents,

    A. Maharana, D.-H. Lee, S. Tulyakov, M. Bansal, F. Barbieri, and Y . Fang, “Evaluating very long-term conversational memory of llm agents,” inProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2024, pp. 13 851–13 870

    work page 2024

  24. [24]

    Octen series: Optimizing embedding models to #1 on rteb leaderboard,

    O. Team, “Octen series: Optimizing embedding models to #1 on rteb leaderboard,” 2025. [Online]. Available: https://octen-team.github.io/octen blog/posts/octen-rteb-first-place/

    work page 2025

  25. [25]

    General agentic memory via deep research,

    B. Yan, C. Li, H. Qian, S. Lu, and Z. Liu, “General agentic memory via deep research,”arXiv preprint arXiv:2511.18423, 2025

    work page arXiv 2025

  26. [26]

    Know What You Don't Know: Unanswerable Questions for SQuAD

    P. Rajpurkar, R. Jia, and P. Liang, “Know what you don’t know: Unanswerable questions for squad,”arXiv preprint arXiv:1806.03822, 2018

    work page internal anchor Pith review Pith/arXiv arXiv 2018

  27. [27]

    Natural questions: a benchmark for question answering research,

    T. Kwiatkowski, J. Palomaki, O. Redfield, M. Collins, A. Parikh, C. Alberti, D. Epstein, I. Polosukhin, J. Devlin, K. Leeet al., “Natural questions: a benchmark for question answering research,”Transactions of the Association for Computational Linguistics, vol. 7, pp. 453–466, 2019

    work page 2019

  28. [28]

    Looking beyond the surface: A chal- lenge set for reading comprehension over multiple sentences,

    D. Khashabi, S. Chaturvedi, M. Roth, S. Upadhyay, and D. Roth, “Looking beyond the surface: A chal- lenge set for reading comprehension over multiple sentences,” inProceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tech- nologies, Volume 1 (Long Papers), 2018, pp. 252–262

    work page 2018

  29. [29]

    DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs

    D. Dua, Y . Wang, P. Dasigi, G. Stanovsky, S. Singh, and M. Gardner, “Drop: A reading comprehension benchmark requiring discrete reasoning over paragraphs,”arXiv preprint arXiv:1903.00161, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1903

  30. [30]

    Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension,

    M. Joshi, E. Choi, D. Weld, and L. Zettlemoyer, “Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension,” inProceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2017, pp. 1601–1611

    work page 2017

  31. [31]

    Dense passage re- trieval for open-domain question answering,

    V . Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W.-t. Yih, “Dense passage re- trieval for open-domain question answering,” inProceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, pp. 6769–6781

    work page 2020

  32. [32]

    Compositional semantic parsing on semi-structured tables,

    P. Pasupat and P. Liang, “Compositional semantic parsing on semi-structured tables,” inProceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2015, pp. 1470–1480

    work page 2015

  33. [33]

    Fetaqa: Free-form table question answering,

    L. Nan, C. Hsieh, Z. Mao, X. V . Lin, N. Verma, R. Zhang, W. Kry´sci´nski, H. Schoelkopf, R. Kong, X. Tang et al., “Fetaqa: Free-form table question answering,”Transactions of the Association for Computational Linguistics, vol. 10, pp. 35–49, 2022. 14

    work page 2022

  34. [34]

    Open domain question answering over tables via dense retrieval,

    J. Herzig, T. M ¨uller, S. Krichene, and J. Eisenschlos, “Open domain question answering over tables via dense retrieval,” inACL, 2021, pp. 512–519

    work page 2021

  35. [35]

    Pervasive annotation errors break text-to-sql benchmarks and leader- boards,

    T. Jin, Y . Choi, Y . Zhu, and D. Kang, “Pervasive annotation errors break text-to-sql benchmarks and leader- boards,”CIDR, vol. abs/2601.08778, 2026

    work page arXiv 2026

  36. [36]

    Arming Data Agents with Tribal Knowledge,

    S. Agarwal, A. Biswal, S. Zeighami, A. Cheung, J. Gonzalez, and A. G. Parameswaran, “Arming Data Agents with Tribal Knowledge,” Feb. 2026

    work page 2026

  37. [37]

    The Power of Constraints in Natural Language to SQL Translation,

    T. Ren, C. Ke, Y . Fan, Y . Jing, Z. He, K. Zhang, and X. S. Wang, “The Power of Constraints in Natural Language to SQL Translation,”Proceedings of the VLDB Endowment, vol. 18, no. 7, pp. 2097–2111, Mar. 2025

    work page 2097

  38. [38]

    AgentSM: Semantic Memory for Agentic Text-to-SQL,

    A. Biswal, C. Lei, X. Qin, A. Li, B. Narayanaswamy, and T. Kraska, “AgentSM: Semantic Memory for Agentic Text-to-SQL,” Jan. 2026

    work page 2026

  39. [39]

    An expert in residence: LLM agents for always-on operating system tuning,

    G. Liargkovas, V . Jabrayilov, H. Franke, and K. Kaffes, “An expert in residence: LLM agents for always-on operating system tuning,” inMachine Learning for Systems 2025, 2025. [Online]. Available: https://openreview.net/forum?id=7dhlgPp8ni

    work page 2025

  40. [40]

    Empowering machine-learning assisted kernel decisions with ebpfml,

    P. S. Sodhi, G. Liargkovas, and K. Kaffes, “Empowering machine-learning assisted kernel decisions with ebpfml,” inProceedings of the 3rd Workshop on EBPF and Kernel Extensions, ser. eBPF ’25. New York, NY , USA: Association for Computing Machinery, 2025, p. 28–30. [Online]. Available: https://doi.org/10.1145/3748355.3748363

    work page doi:10.1145/3748355.3748363 2025

  41. [41]

    cache ext: Cus- tomizing the page cache with ebpf,

    T. Zussman, I. Zarkadas, J. Carin, A. Cheng, H. Franke, J. Pfefferle, and A. Cidon, “cache ext: Cus- tomizing the page cache with ebpf,” inProceedings of the 31st ACM Symposium on Operating Systems Principles (SOSP ’25). Association for Computing Machinery, 2025

    work page 2025

  42. [42]

    Toward systems foundations for agentic exploration,

    J. Xu, T. Zhou, E. Wu, and K. Kaffes, “Toward systems foundations for agentic exploration,” 2025. [Online]. Available: https://arxiv.org/abs/2510.05556

    work page arXiv 2025

  43. [43]

    Checkpoint/Restore In Userspace (CRIU),

    CRIU Project, “Checkpoint/Restore In Userspace (CRIU),” https://criu.org/, 2012, accessed: 2025-08-06

    work page 2012

  44. [44]

    Podman checkpoint/restore integration,

    ——, “Podman checkpoint/restore integration,” https://criu.org/Podman, 2025, accessed: 2026-03-10

    work page 2025

  45. [45]

    Reasoning with language model is planning with world model,

    S. Hao, Y . Gu, H. Ma, J. Hong, Z. Wang, D. Wang, and Z. Hu, “Reasoning with language model is planning with world model,” inProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, 2023, pp. 8154–8173

    work page 2023

  46. [46]

    Planning using a temporal world model,

    J. F. Allen and J. A. Koomen, “Planning using a temporal world model,” inProceedings of the Eighth international joint conference on Artificial intelligence-Volume 2, 1983, pp. 741–747

    work page 1983

  47. [47]

    26 U.S.C. § 274(n): Only 50 percent of meal expenses allowed as deduction,

    United States Congress, “26 U.S.C. § 274(n): Only 50 percent of meal expenses allowed as deduction,” U.S. Code Title 26, 2024, internal Revenue Code, Section 274(n). [Online]. Available: https://www.law.cornell.edu/uscode/text/26/274#n

    work page 2024

  48. [48]

    ShieldLM: Empowering LLMs as Aligned, Customizable and Explainable Safety Detectors

    Z. Zhang, Y . Lu, J. Ma, D. Zhang, R. Li, P. Ke, H. Sun, L. Sha, Z. Sui, H. Wang, and M. Huang, “Shieldlm: Empowering llms as aligned, customizable and explainable safety detectors,” 2024. [Online]. Available: https://arxiv.org/abs/2402.16444

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  49. [49]

    Provenance semirings,

    T. J. Green, G. Karvounarakis, and V . Tannen, “Provenance semirings,”Proceedings of the twenty-sixth ACM SIGMOD-SIGACT-SIGART symposium on Principles of database systems, 2007. [Online]. Available: https://api.semanticscholar.org/CorpusID:1993398 15

    work page 2007

  50. [50]

    Please don’t kill my vibe: Empowering agents with data flow control,

    C. Summers, H. Mohammed, and E. Wu, “Please don’t kill my vibe: Empowering agents with data flow control,”ArXiv, vol. abs/2512.05374, 2025. [Online]. Available: https://api.semanticscholar.org/CorpusID: 283672095

    work page arXiv 2025

  51. [51]

    Trustworthy{Whole-System}provenance for the linux kernel,

    A. Bates, D. J. Tian, K. R. Butler, and T. Moyer, “Trustworthy{Whole-System}provenance for the linux kernel,” in24th USENIX Security Symposium (USENIX Security 15), 2015, pp. 319–334

    work page 2015

  52. [52]

    Practical whole-system provenance capture,

    T. Pasquier, X. Han, M. Goldstein, T. Moyer, D. Eyers, M. Seltzer, and J. Bacon, “Practical whole-system provenance capture,” inProceedings of the 2017 symposium on cloud computing, 2017, pp. 405–418. 16

    work page 2017