Pith. sign in

REVIEW 6 minor 68 references

From Test-Time Scaling to Reusable Memory: Measuring Crystallization in Text-to-SQL

T0 review · 0 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Stored verified repair episodes for text-to-SQL transfer to unseen questions, capturing 44.4% of the accuracy headroom that on-demand repair would have spent on the same questions.

desk verdict A clean, carefully scoped measurement of when and why text-to-SQL memory helps; the verification dependence is real but explicitly tested. read the letter →

arxiv 2608.07213 v1 pith:CFWSFMSE submitted 2026-08-07 cs.CL

Jiaqian Wang (1) , Yutao Qi (1) , Wenjin Hou (1) , Yuanxi Che (1) , Muning Wen (2) ((1) Xidian University , (2) Shanghai Jiao Tong University) This is my paper · ORCID
classification cs.CL
keywords text-to-SQLtest-timescalingagentmemoryexperiencereuseexecutionfeedbackevaluationmethodologycrystallizationratioverification
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that the computation spent repairing failed text-to-SQL queries can be turned into reusable memory instead of being discarded: a verified, database-scoped bank of corrected episodes should help future queries over the same database. On the BIRD benchmark, with a fixed single-shot solver, storing corrected queries verbatim raises held-out first-attempt accuracy from 62.04% to 66.38%, a gain of 4.34 percentage points that equals 44.4% of the headroom provided by on-demand repair on those same questions. The paper further claims that the transfer is carried mainly by database-specific content, such as identifiers, values, joins, and query structures, rather than by matched question-to-SQL examples to copy, and that an independently reliable verification signal is necessary for the benefit. A sympathetic reader should care because end-to-end memory scores conflate replay of recurring questions with help on unseen ones, and this study separates those settings and attributes the gain to a specific, testable mechanism.

What carries the argument

The central object is the verified, database-scoped memory bank $M_d = \{ W(e) : e = (q, \hat{y}, y^+, \tau) \in \mathcal{E}_d,\ v(y^+) = 1 \}$, where each episode consists of the question, first attempt, corrected query, and interaction trace, $W$ is a card writer, and $v$ is a verification signal that decides admission. The identity that carries the measurement is the crystallization ratio $\mathrm{CR} = (P_M - P_0)/(P_K - P_0)$: the held-out transfer lift divided by the on-demand repair headroom on the same questions. It is a descriptive ratio, not a probability, and it is only meaningful when the solver, split, repair budget, and collection procedure are held fixed. The paper's controlled-comparison design, varying acquisition, admission, writing, and consumption one at a time while keeping the same greedy solver, is what lets it attribute the gain to database-specific payload rather than to any single design choice.

What would settle it

A held-out transfer test on a benchmark pack where no database appears in the collection split should show no positive memory lift, because the paper's foreign-card control predicts a negative trend below no memory; a positive gain in that setting would contradict the database-specific-content account.

Watch

Extended reading notes

Core claim

Under controlled evaluation, the paper's central discovery is that a per-database bank of verified, verbatim corrected queries produces value in three distinct future-use settings: exact-query replay (96.1% accuracy when the question's own card is eligible), cross-question retention (56.0% when that card is removed), and held-out same-database transfer (+4.34 pp, CR = 44.4%). The crystallization ratio CR normalizes the memory lift against the on-demand repair headroom measured on the same held-out questions, making the memory gain commensurable with the compute it would otherwise cost to repair. Interventions that permute question–SQL pairings inside the bank preserve 73% of the aligned lift, while equally valid cards from other databases do not transfer, leading the paper to conclude that the cards mainly expose database-specific content rather than supply copyable worked examples. The same experiments show that without a reliable verification gate, storing self-voted corrections is 2.03 pp below no memory and 4.85 pp below the verified bank, so verification is described as necessary, not optional.

Load-bearing premise

The benefit requires an independently reliable verification signal that decides which repaired episodes are stored; without such a gate, the paper's own ablation shows the bank hurts accuracy.

Editorial extensions

If this is right

  • If the central claim holds, storing verified corrected queries in a per-database bank substitutes for roughly 44% of per-question repair on seen databases, and the remaining repair headroom is still available on top.
  • Recurring questions are nearly free under memory: exact-query replay reaches 96.1% when the originating card is retrieved, so workloads with repeated questions will amortize construction cost faster.
  • Verification is the dominant admission decision: gated storage beats ungated by 4.85 pp, and the ungated bank is below no memory, so deployment must supply an independent correctness channel.
  • Widening retrieval from $k=1$ to $k=10$ adds 3.18 pp, which means cumulative coverage of database-specific content, not a smarter matching algorithm, is what increases transfer.
  • Richer card formats and more elaborate retrievers have no statistically supported advantage, so the simple operating point, verbatim corrected SQL, broad same-database retrieval, and a reliable gate, is the supported default.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The 44.4% CR is likely a function of the workload mix rather than a universal constant: workloads dominated by novel questions should be bounded by the transfer tier, while recurring questions push effective value toward the near-100% replay tier.
  • Because mispairing preserves most of the lift, a testable extension is to strip question text from cards entirely and retrieve on SQL structure alone; if the gain survives, the useful signal is payload rather than natural-language similarity.
  • The paper's fix/break symmetry suggests a retrieval-side guard: suppressing cards whose foreign SQL elements, such as LIMIT, aggregation, or literal formats, would alter answer cardinality could reduce the 67 broken cases without losing the 127 fixes.
  • The measured amortization horizon of roughly 7.5–9.9K future queries invites a decision rule for when to build memory versus repair on demand, but combining the conditional reuse values into one expected-utility number also requires the workload's recurrence distribution.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

0 major / 6 minor

Summary. This paper formalizes the 'crystallization problem' for text-to-SQL memory: how to measure the future value of verified test-time repair episodes rather than collapsing everything into one end-to-end score. With a fixed single-shot solver, the authors build per-database banks from repair episodes on a 70% collection split, then evaluate exact-query replay, cross-question retention, and held-out same-database transfer on BIRD (with Spider as a secondary probe). The headline result is that a verbatim bank of oracle-verified corrected queries raises held-out first-attempt accuracy from 62.04% to 66.38% (+4.34pp, two-stage bootstrap CI [+1.50, +7.49], p=0.0034), corresponding to 44.4% (CI [24,65]%) of the on-demand repair headroom measured on the same questions. Controlled interventions show that database-specific content, not matched question-SQL examples, is the main carrier; reliable verification and wider retrieval coverage help, while richer card formats and more elaborate retrievers do not show statistically supported gains. Cost, robustness, leakage, and statistical-discipline analyses are reported in depth.

Significance. The contribution is primarily methodological but consequential: it decomposes the single end-to-end memory score into replay, retention, and transfer, and anchors transfer in a descriptive crystallization ratio against a measured repair reference. The experimental protocol is unusually careful: pre-specified confirmatory comparisons, three paired seeds, two-stage hierarchical bootstrap over databases and questions, sentinel-based leakage checks, rerun-drift quantification, and explicit exploratory-versus-confirmatory labeling. The verification-gate dependence is the clearest boundary of the central claim; the paper explicitly identifies it and shows with the ungated ablation that the bank is harmful without a trustworthy gate. Because the claim is conditional on a verified bank, this is a scoped limitation rather than an internal inconsistency. If the results hold, they give practitioners concrete guidance: verified database-scoped banks can substitute for a substantial share of on-demand repair on recurring or same-database workloads.

minor comments (6)
  1. [Section 3.3 / Eq. (2)] The retrieval operator rho_k is used in Equations (2) and (4) before being formally defined; please add an explicit definition of rho_k(q; M_d) near Eq. (2) so the notation is self-contained.
  2. [Section 5.1 / Figure 3] The 'remaining headroom +5.42pp' label and the focused scale in panel (a) could be misread; consider adding explicit numeric labels on the bars for P0, PM, and the repair reference, or annotating the scale break directly.
  3. [Section 5.4 / Table 2] The sentence 'one repair round captures 80% of what three memoryless rounds achieve at 59% of the calls' is terse; please specify exactly which configurations enter this comparison so the reader can verify the 80% and 59% figures.
  4. [Appendix E / Table 5] The worked-example text says capsule exact-query replay 'stays at 100%,' while Table 5 reports a three-seed mean of 99.7%; please align the wording to avoid an apparent inconsistency.
  5. [Section 7 / Appendix E] The release does not redistribute the quality-controlled split or raw transcripts; consider releasing question IDs or hashes so the disjointness and leakage checks can be independently verified by readers.
  6. [Section 5.3 / Appendix D] Because the verification gate is central to practical applicability, please consider reporting the key synthetic verification-noise numbers in the main text rather than only referring to Appendix D.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the headline gain and CR are measured quantities on disjoint held-out questions, not reductions to fitted inputs or self-citations.

full rationale

The paper's derivation chain is empirical and self-contained, not circular. The central claim—that a verified, database-scoped bank of verbatim corrected episodes improves held-out same-database first-attempt accuracy by +4.34pp, equal to CR=44.4% of on-demand repair headroom—rests on direct paired measurements. The memory writer receives only the model's episode, never gold SQL ('The writer still never sees the gold query, although benchmark verification v uses the gold execution result'), and held-out questions are disjoint from banked questions ('These questions were never used to build the bank'). CR is explicitly a descriptive ratio of two measured quantities: 'It is the held-out first-attempt gain from memory, divided by the repair headroom on the same questions,' and the paper cautions that it is 'descriptive, not a probability.' No parameter is fitted to the held-out split, and no quantity in Eq. (2) or Eq. (3) is defined in terms of the result it is used to explain. The mechanism claim (database-specific content rather than matched examples) is supported by controlled interventions—local vs. foreign cards, mispairing, grounding-stripping, similarity targeting—whose outcomes are not forced by construction; the aligned-minus-permuted contrast even includes zero, a result the authors report rather than explain away. The verification dependency is acknowledged and tested as a boundary condition, not hidden. The only apparent self-citation, reference [52] (EGREFINE), appears in related-work context and is not load-bearing for the measurement protocol or headline result.

Assumptions & free parameters 0 free parameters · 5 assumptions · 0 invented entities

No fitted model parameters or ad hoc constants enter the central quantities. The controlled settings (T3 source, W0 verbatim, k=5, K=3, 70/30 split, seed set {42,7,13}) are fixed experimental configurations, not quantities fitted to the outcome. The paper's claims rest on domain assumptions about execution-based evaluation, the paired design, the hierarchical bootstrap, the availability of a verification gate, and the representativeness of BIRD dev for same-database reuse. No new explanatory entities are introduced; the crystallization ratio is a descriptive normalization of measured accuracy deltas.

assumptions (5)
  • domain assumption Execution accuracy, defined as set-equality of the query result with the gold query's result, is a valid and sufficiently noise-free correctness measure for the paired comparisons.
    All headline numbers are EX means. The paper notes EX can pass a semantically wrong query on a lucky database state, but treats this as symmetric across arms (Appendix C).
  • domain assumption The fixed solver, schema, prompts, and greedy decoding are identical across compared arms, so the retrieved memory block is the only systematic difference.
    Attribution by construction is stated in Section 4. Greedy decoding is only approximately reproducible across serving environments, so the paper uses paired within-chain contrasts and measured rerun drift (median 0.65pp).
  • domain assumption The two-stage hierarchical bootstrap validly models the clustered data structure for the reported p-values and confidence intervals.
    Confirmatory claims use a bootstrap that resamples the 11 databases and then unique questions, keeping seed replicates together (Section 4). Validity assumes these databases and questions are exchangeable samples for the intended inference.
  • domain assumption A reliable verification signal, distinct from gold SQL, is available to admit episodes into the bank; in benchmark mode this is the oracle execution check, and in deployment the paper posits user confirmation or executable assertions.
    The main results are conditional on this gate. Section 3.5 introduces the regimes, and Section 5.3 shows ungated storage is 2.03pp below no memory, so without this assumption the benefit disappears.
  • domain assumption BIRD development set is an adequate testbed for same-database text-to-SQL memory transfer, and Spider serves as a secondary check.
    The headline is measured on BIRD dev, not hidden test, and Spider's transfer deltas are small because its repair headroom is small (Section 5.4). The paper acknowledges development-set optimism.

how reviews work

0 comments
Cite this review

Pith. "Pith review of From Test-Time Scaling to Reusable Memory: Measuring Crystallization in Text-to-SQL." pith.science (2026). https://pith.science/paper/CFWSFMSE

@misc{pith2026260807213,
  author       = {Pith},
  title        = {Pith review of: From Test-Time Scaling to Reusable Memory: Measuring Crystallization in Text-to-SQL},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CFWSFMSE}},
  note         = {Machine review of arXiv:2608.07213}
}
read the original abstract

Test-time scaling can correct difficult text-to-SQL queries, but the extra computation is normally discarded after each answer. Systems increasingly retain verified repair episodes, yet evaluations still report one end-to-end score. It cannot distinguish replay on recurring questions from help on unseen questions, or identify the responsible memory choice. We call measuring this future value the crystallization problem. Our controlled evaluation holds the single-shot solver fixed and varies one memory choice at a time. We separately measure replay, cross-question retention, and held-out same-database transfer. On BIRD, storing verified corrected queries improves held-out first-attempt accuracy by 4.34 percentage points. This gain captures 44.4% of the accuracy headroom provided by on-demand repair on the same questions. Controlled interventions identify database-specific content as the main operating ingredient. Reliable verification and broader retrieval coverage yield supported gains; richer formats and elaborate retrievers do not. Open-source code, evaluation artifacts, and reproduction instructions are available at https://github.com/ai-jiaqian/text-to-sql-memory-crystallization.

Figures

Figures reproduced from arXiv: 2608.07213 by the authors.

Figure 1
Figure 1. From per-question repair to reusable memory. Left: [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The measurement protocol. (a) We build per-database memory only from the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. RQ1: held-out transfer reaches a substantial share of on-demand repair headroom. W0 raises held-out same-database [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: RQ2: database-specific content is the main operating ingredient for observed transfer. Local cards, grounded content, [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: RQ3: reliable verification and wider retrieval improve transfer. Verification adds [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Retrieval coverage and bank size are complementary. (a) For W0, increasing width from [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

68 extracted references · 43 canonical work pages

  1. [1]

    Parameswaran

    Shubham Agarwal, Asim Biswal, Sepanta Zeighami, Alvin Cheung, Joseph Gon- zalez, and Aditya G. Parameswaran. 2026. Arming Data Agents with Tribal Knowledge.arXiv preprint arXiv:2602.13521(2026). (Tk-Boost)

    arXiv 2026

  2. [2]

    Arian Askari et al . 2025. MAGIC: Generating Self-Correction Guideline for In-Context Text-to-SQL. InProceedings of AAAI

    work page 2025

  3. [3]

    Baek et al . 2025. Knowledge Base Construction for Knowledge-Augmented Text-to-SQL. InFindings of the Association for Computational Linguistics: ACL

    work page 2025

  4. [4]

    Biswal et al. 2026. AgentSM: Semantic Memory for Agentic Text-to-SQL.arXiv preprint arXiv:2601.15709(2026)

    arXiv 2026

  5. [5]

    Andrew Borthwick and Stephen Ash. 2026. RoboPhD: Self-Improving Text-to- SQL Through Autonomous Agent Evolution.arXiv preprint arXiv:2601.01126 (2026)

    work page arXiv 2026

  6. [6]

    Hasan Alp Caferoğlu and Özgür Ulusoy. 2024. E-SQL: Direct Schema Linking via Question Enrichment in Text-to-SQL.arXiv preprint arXiv:2409.16751(2024)

    arXiv 2024

  7. [7]

    Zhenbiao Cao et al . 2024. RSL-SQL: Robust Schema Linking in Text-to-SQL Generation.arXiv preprint arXiv:2411.00073(2024)

    arXiv 2024

  8. [8]

    Xinyun Chen et al. 2024. Teaching Large Language Models to Self-Debug. In International Conference on Learning Representations (ICLR)

    work page 2024

Show all 68 references
  1. [9]

    Zui Chen, Han Li, Xinhao Zhang, et al . 2025. RubikSQL: Lifelong Learning Agentic Knowledge Base as an Industrial NL2SQL System.arXiv preprint arXiv:2508.17590(2025)

    2025 arXiv

  2. [10]

    Zhiyi Chen, Jie Song, and Peng Li. 2026. TAHOE: Text-to-SQL with Automated Hint Optimization from Experience.arXiv preprint arXiv:2606.12387(2026)

    2026 arXiv

  3. [11]

    Zhibo Chu, Zichong Wang, and Qitao Qin. 2024. Leveraging Prior Experi- ence: An Expandable Auxiliary Knowledge Base for Text-to-SQL.arXiv preprint arXiv:2411.13244(2024)

    2024 arXiv

  4. [12]

    Thomas Cook, Kelly Patel, Sivapriya Vellaichamy, et al. 2025. Continual Learning of Domain Knowledge from Human Feedback in Text-to-SQL.arXiv preprint arXiv:2511.10674(2025)

    2025

  5. [13]

    Xuemei Dong et al . 2023. C3: Zero-shot Text-to-SQL with ChatGPT.arXiv preprint arXiv:2307.07306(2023)

    2023 arXiv

  6. [14]

    Thomas Palmeira Ferraz, Romain Deffayet, Vassilina Nikoulina, Hervé Déjean, and Stéphane Clinchant. 2026. Retrieval-Augmented LLM Agents: Learning to Learn from Experience.arXiv preprint arXiv:2603.18272(2026)

    2026

  7. [15]

    Dawei Gao et al. 2024. Text-to-SQL Empowered by Large Language Models: A Benchmark Evaluation.Proceedings of the VLDB Endowment17, 5 (2024)

    2024

  8. [16]

    Huan-ang Gao et al. 2026. A Survey of Self-Evolving Agents: What, When, How, and Where to Evolve on the Path to Artificial Super Intelligence.Transactions on Machine Learning Research (TMLR)(2026)

    2026

  9. [17]

    Yingqi Gao et al. 2025. XiYan-SQL: A Multi-Generator Ensemble Framework for Text-to-SQL.arXiv preprint arXiv:2507.04701(2025)

    2025 arXiv

  10. [18]

    Zijin Hong et al . 2024. Knowledge-to-SQL: Enhancing SQL Generation with Data Expert LLM. InFindings of the Association for Computational Linguistics: ACL

    2024

  11. [19]

    Yuanzhe Hu, Yu Wang, and Julian McAuley. 2025. Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions.arXiv preprint arXiv:2507.05257 (2025)

    2025 arXiv

  12. [20]

    Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. 2024. Large Language Models Cannot Self- Correct Reasoning Yet. InInternational Conference on Learning Representations (ICLR)

    2024

  13. [21]

    Yiwen Jiao et al. 2025. ORANGE: An Online Reflection and Generation Frame- work with Domain Knowledge for Text-to-SQL.arXiv preprint arXiv:2511.00985 (2025)

    2025

  14. [22]

    Fangyu Lei et al. 2025. Spider 2.0: Evaluating Language Models on Real-World Enterprise Text-to-SQL Workflows. InInternational Conference on Learning Rep- resentations (ICLR)

    2025

  15. [23]

    Boyan Li, Chong Chen, Zhujun Xue, Yinan Mei, and Yuyu Luo. 2025. DeepEye- SQL: A Software-Engineering-Inspired Text-to-SQL Framework.arXiv preprint arXiv:2510.17586(2025)

    2025 arXiv

  16. [24]

    Haoyang Li et al. 2023. RESDSQL: Decoupling Schema Linking and Skeleton Parsing for Text-to-SQL. InProceedings of AAAI

    2023

  17. [25]

    Jinyang Li et al. 2023. Can LLM Already Serve as a Database Interface? A Big Bench for Large-Scale Database Grounded Text-to-SQLs. InAdvances in Neural Information Processing Systems (NeurIPS)

    2023

  18. [26]

    Xuancheng Li et al . 2026. Beyond Experience Retrieval: Learning to Gener- ate Utility-Optimized Structured Experience for Frozen LLMs.arXiv preprint arXiv:2602.02556(2026)

    2026 arXiv

  19. [27]

    Xianxuan Long, Zhikai Chen, Shenglai Zeng, Shouren Wang, Kai Guo, and Jiliang Tang. 2026. MemTrace: Probing What Final Accuracy Misses in Long-Term Memory.arXiv preprint arXiv:2606.17328(2026)

    2026 arXiv

  20. [28]

    Peixian Ma et al. 2025. SQL-R1: Training Natural Language to SQL Reasoning Model by Reinforcement Learning. InAdvances in Neural Information Processing Systems (NeurIPS)

    2025

  21. [29]

    Karime Maamari, Fadhil Abubaker, Daniel Jaroslawicz, and Amine Mhedhbi

  22. [30]

    Aman Madaan et al. 2023. Self-Refine: Iterative Refinement with Self-Feedback. InAdvances in Neural Information Processing Systems (NeurIPS)

    2023

  23. [31]

    Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. Rethinking the Role of Demonstrations: What Makes In-Context Learning Work?. InProceedings of the Conference on Empirical Methods in Natural Language Processing (EMN...

    2022

  24. [32]

    Ouyang et al. 2025. ReasoningBank: Scaling Agent Self-Evolving with Reasoning Memory.arXiv preprint arXiv:2509.25140(2025)

    2025 arXiv

  25. [33]

    Charles Packer et al. 2023. MemGPT: Towards LLMs as Operating Systems.arXiv preprint arXiv:2310.08560(2023)

    2023 arXiv

  26. [34]

    Joon Sung Park et al. 2023. Generative Agents: Interactive Simulacra of Human Behavior. InProceedings of the ACM Symposium on User Interface Software and Technology (UIST)

    2023

  27. [35]

    Mohammadreza Pourreza et al. 2025. CHASE-SQL: Multi-Path Reasoning and Preference Optimized Candidate Selection in Text-to-SQL. InInternational Con- ference on Learning Representations (ICLR)

    2025

  28. [36]

    Mohammadreza Pourreza et al. 2025. Reasoning-SQL: Reinforcement Learning with SQL-Tailored Partial Rewards for Reasoning-Enhanced Text-to-SQL. In Conference on Language Modeling (COLM)

    2025

  29. [37]

    Mohammadreza Pourreza and Davood Rafiei. 2023. DIN-SQL: Decomposed In- Context Learning of Text-to-SQL with Self-Correction. InAdvances in Neural Information Processing Systems (NeurIPS)

    2023

  30. [38]

    Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings Using Siamese BERT-Networks. InProceedings of EMNLP-IJCNLP

    2019

  31. [39]

    Tonghui Ren et al. 2024. PURPLE: Making a Large Language Model a Better SQL Writer. InIEEE International Conference on Data Engineering (ICDE)

    2024

  32. [40]

    Shao et al. 2026. Your Agent May Misevolve: Emergent Risks in Self-Evolving LLM Agents. InInternational Conference on Learning Representations (ICLR)

    2026

  33. [41]

    Lei Sheng and Shuai-Shuai Xu. 2025. CSC-SQL: Corrective Self-Consistency in Text-to-SQL via Reinforcement Learning. InFindings of IJCNLP-AACL

    2025

  34. [42]

    Noah Shinn et al. 2023. Reflexion: Language Agents with Verbal Reinforcement Learning. InAdvances in Neural Information Processing Systems (NeurIPS)

    2023

  35. [43]

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. Scaling LLM Test- Time Compute Optimally Can Be More Effective than Scaling Model Parameters. arXiv preprint arXiv:2408.03314(2024)

    2024 arXiv

  36. [44]

    Saksham Sahai Srivastava. 2026. Causal Intervention-Based Memory Selection for Long-Horizon LLM Agents.arXiv preprint arXiv:2605.17641(2026)

    2026 arXiv

  37. [45]

    Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L

    Theodore R. Sumers, Shunyu Yao, Karthik Narasimhan, and Thomas L. Griffiths

  38. [46]

    Mirac Suzgun et al. 2026. Dynamic Cheatsheet: Test-Time Learning with Adaptive Memory. InProceedings of EACL

    2026

  39. [47]

    Cognitive Architectures for Language Agents.Transactions on Machine Learning Research (TMLR)(2024)

    2024

  40. [48]

    Wang et al. 2026. Learning to Retrieve: Dual-Level Long-Term Memory for Text-to-SQL Agents.arXiv preprint arXiv:2606.00547(2026)

    2026 arXiv

  41. [49]

    Shayan Talaei et al . 2024. CHESS: Contextual Harnessing for Efficient SQL Synthesis.arXiv preprint arXiv:2405.16755(2024)

    2024 arXiv

  42. [50]

    Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, and Huan Sun. 2023. Towards Understanding Chain-of-Thought Prompting: An Empirical Study of What Matters. InProceedings of the Annual Meeting of the Association for Computational Linguistics (ACL). 2717–2739

    2023

  43. [51]

    Bing Wang et al. 2025. MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL. InProceedings of COLING. Jiaqian Wang, Yutao Qi, Wenjin Hou, Yuanxi Che, and Muning Wen

    2025

  44. [52]

    Jiaqian Wang, Yutao Qi, Wenjin Hou, Yu Pang, and Rui Yang. 2026. EGREFINE: An Execution-Grounded Optimization Framework for Text-to-SQL Schema Re- finement.arXiv preprint arXiv:2605.00628(2026)

    2026 arXiv

  45. [53]

    Guanzhi Wang et al. 2024. Voyager: An Open-Ended Embodied Agent with Large Language Models.Transactions on Machine Learning Research (TMLR)(2024)

    2024

  46. [54]

    Zhiruo Wang et al. 2025. Agent Workflow Memory. InInternational Conference on Machine Learning (ICML)

    2025

  47. [55]

    Xuezhi Wang et al. 2023. Self-Consistency Improves Chain of Thought Reasoning in Language Models. InInternational Conference on Learning Representations (ICLR)

    2023

  48. [56]

    Zerui Yang, Weichuan Wang, Yanwei Xu, Linqi Song, Yudai Matsuda, Wei Han, and Bo Bai. 2026. Memo-SQL: Structured Decomposition and Experience-Driven Self-Correction for Training-Free NL2SQL.arXiv preprint arXiv:2601.10011 (2026)

    2026

  49. [57]

    Jason Wei et al . 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. InAdvances in Neural Information Processing Systems (NeurIPS)

    2022

  50. [58]

    Zhewei Yao, Guoheng Sun, Łukasz Borchmann, et al. 2025. Arctic-Text2SQL- R1: Simple Rewards, Strong Reasoning in Text-to-SQL.arXiv preprint arXiv:2505.20315(2025)

    2025

  51. [59]

    Shunyu Yao et al. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. InInternational Conference on Learning Representations (ICLR)

    2023

  52. [60]

    Yun and Lee. 2025. SEED: Enhancing Text-to-SQL Performance and Practical Us- ability Through Automatic Evidence Generation.arXiv preprint arXiv:2506.07423 (2025)

    2025 arXiv

  53. [61]

    Tao Yu et al. 2018. Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task. InProceedings of EMNLP

    2018

  54. [62]

    Zhang et al. 2025. Reward-SQL: Boosting Text-to-SQL via Stepwise Reasoning and Process-Supervised Rewards.arXiv preprint arXiv:2505.04671(2025)

    2025

  55. [63]

    Eric Zelikman et al. 2022. STaR: Bootstrapping Reasoning with Reasoning. In Advances in Neural Information Processing Systems (NeurIPS)

    2022

  56. [64]

    Yi Zhang et al. 2024. ScienceBenchmark: A Complex Real-World Benchmark for Evaluating Natural Language to SQL Systems.Proceedings of the VLDB Endowment17, 6 (2024)

    2024

  57. [65]

    Qiyuan Zhang et al. 2025. A Survey on Test-Time Scaling in Large Language Models: What, How, Where, and How Well?arXiv preprint arXiv:2503.24235 (2025)

    2025 arXiv

  58. [66]

    within rerun drift

    Denny Zhou et al. 2023. Least-to-Most Prompting Enables Complex Reasoning in Large Language Models. InInternational Conference on Learning Representations (ICLR). A Per-Seed Results and Full Grids This section expands the means reported in Section 5 to their per- seed values a...

    2023

  59. [67]

    Andrew Zhao et al . 2024. ExpeL: LLM Agents Are Experiential Learners. In Proceedings of AAAI

    2024

  60. [2024]

    The Death of Schema Linking? Text-to-SQL in the Age of Well-Reasoned Language Models.arXiv preprint arXiv:2408.07702(2024)

    2024 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.