Pith. sign in

REVIEW 3 major objections 5 minor 28 references

EvoSQL claims that iterating a generator and a critic over multiple rounds, with execution-checked candidate memory, is what reliably turns a text-to-SQL model's latent candidates into final execution accuracy, adding up to 9.19 points over

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 12:16 UTC pith:OLXZY4UH

load-bearing objection Solid co-evolution system with real-looking gains, but the headline 'same Maj@16 budget' comparison does not hold up against the paper's own token accounting. the 3 major comments →

arxiv 2607.20489 v1 pith:OLXZY4UH submitted 2026-06-04 cs.AI cs.DB

EvoSQL: Memory-Augmented Critic-Generator Co-Evolution for Text-to-SQL

Jiawei Zhou , Jianwei Wang , Chenyu Zhou , Chaojian Shi , Ming Dong , Kai Wang This is my paper
classification cs.AI cs.DB
keywords text-to-SQLco-evolutioncritic-guided refinementepisodic memoryexecution feedbackself-distillation policy optimizationtest-time scalingmajority voting
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

EvoSQL asks whether the usual test-time recipe for text-to-SQL—sample many candidates and take a majority vote—can be improved by making the sampling itself adaptive. The paper's answer is yes: a generator and an LLM critic co-evolve a candidate population over up to three rounds, with execution results and critic diagnoses stored in episodic memory and used to guide the next round of generation. On Spider and BIRD, the authors report consistent gains over Maj@16 baselines for all four tested open-source backbones, with the largest improvement on BIRD-Dev, where Coder-3B rises from 51.24 to 60.43 execution accuracy. Why it matters: if the claim holds, harder database questions can be solved by reallocating test-time compute through critic-guided search rather than only by scaling the model or sampling more SQL.

Core claim

On its own terms, the paper's central discovery is that a memory-grounded generator–critic co-evolution loop converts latent candidate quality into final execution accuracy better than majority voting. Candidates are sampled, executed, and scored by an LLM critic on alignment, schema, logic, and completeness rubrics; invalid SQL is suppressed, empty results are down-weighted, stale candidates are time-discounted, and execution-consistent candidates receive a log consistency bonus. The resulting utility scores are used both to sample few-shot repair payloads for later rounds and to select the final answer, with early stopping when the top candidates become execution-stable. The paper reports

What carries the argument

The central object is the utility-ranked episodic memory that drives payload selection and stopping. Each stored candidate carries an execution record and a critic score vector; confidence is capped by execution outcome (−1 for invalid, 0 for empty), then combined with a time discount and a log consistency bonus for candidates sharing the same execution result. A softmax over these utilities samples the few-shot payload for the next generator round, and early stopping fires when 90% of the top-K candidates agree on one execution outcome. This memory-utility loop is what turns sparse execution signals and fallible critic judgments into a guided search policy.

Load-bearing premise

The load-bearing premise is that the LLM critic, scoring candidates without seeing the gold SQL, produces rubric scores and diagnoses that correlate with true execution correctness closely enough that utility-guided selection and early stopping choose better SQL than majority voting; the paper itself concedes that cold-start critics, especially smaller models, can be poorly calibrated in early rounds.

What would settle it

On a random sample of BIRD-Dev questions where EvoSQL improves over Maj@16, check whether the final selected SQL appears in the round-0 candidate pool; if it is already present in nearly all such cases, the gain is mainly better selection from the initial samples rather than critic-guided generation of new correct SQL.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • The mechanism is model-agnostic: four open-source backbones with different training recipes all improve, so any single-turn SQL generator can be wrapped in the harness without retraining.
  • On hard, database-grounded queries the gain can be large even for weaker generators: Coder-3B gains +9.19 on BIRD-Dev, moving from 51.24 to 60.43, with an oracle Pass@16 ceiling of 65.91.
  • Critic-guided early stopping makes the extra rounds cheaper than naive resampling: the paper reports a 29.6–39.7% reduction in generation tokens relative to the self-evolution counterpart.
  • An optional SDPO fine-tuning stage can inject execution-aware supervision into the generator, yielding further gains on Spider-Test and BIRD-Dev for the two backbones where it was applied.
  • Co-evolution can solve questions that no initial candidate gets right, as in the case study where the correct SQL appears only in round 1 after critic feedback.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The headline comparisons, labeled as the same Maj@16 budget, are not strictly token-matched because EvoSQL runs up to three rounds of 16 candidates plus critic calls; the token-accounting tables suggest early stopping can make it cheaper overall, but a same-token comparison would be a sharper test of the claim.
  • For Coder-3B on BIRD-Dev, the Pass@16 ceiling of 65.91 means the +9.19 gain is bounded by how well the critic selects and repairs from a pool that majority voting already under-uses; this frames the result as selection-and-repair rather than new model capability.
  • If critic calibration improves—through stronger critics, uncertainty-aware scoring, or explicit calibration—the framework's gains could grow on hard cases, a direction the paper's limitations section explicitly says is needed.
  • A natural extension, flagged by the authors as future work, is persistent cross-query memory: reusing diagnoses and repair patterns across questions with shared schemas could compound the per-query gains reported here.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes EVOSQL, a test-time co-evolution framework for text-to-SQL. It couples a generator that samples K=16 SQL candidates per round with an LLM-based critic that scores candidates on execution feedback and rubric dimensions. A utility function combines calibrated confidence, time discounting, and a consistency bonus, and the resulting scores drive payload selection for subsequent rounds, early stopping, and final greedy selection. An optional offline SDPO stage distills execution-aware supervision into the backbone. The central empirical claim is that, under the same Maj@16 sampling budget, EVOSQL (Base) consistently improves four open-source backbones on Spider-Dev, Spider-Test, and BIRD-Dev, with a headline gain of +9.19 EX for Coder-3B on BIRD-Dev.

Significance. If the stated gains are real and attributable to the co-evolution mechanism, the paper would make a useful contribution to inference-time scaling for text-to-SQL. The manuscript is stronger than a typical prompt-engineering paper: it includes round-wise accuracy decomposition (Table 2), an ablation removing the LLM critic (Tables 3 and 10), a broad hyperparameter sensitivity analysis (Figures 3 and 4), detailed token accounting (Tables 11 and 12), a case study, and a public code release. The BIRD-Dev Coder-3B improvement is internally coherent with the Pass@16 ceiling: Maj@16 recovers 51.24 of an oracle 65.91, while EVOSQL recovers 60.43. The main threat to the central claim is not internal inconsistency but an unmatched comparison: the paper advertises a fixed Maj@16 budget while actually spending about 2.2x the tokens of Maj@16. Because the paper already contains much of the evidence needed to address this (self-evolution and per-round token costs), the required fix is a compute-matched control, not a re-derivation.

major comments (3)
  1. [§5.2.1, Table 1, and Tables 11–12] The claim that EVOSQL (Base) improves Maj@16 "under the same Maj@16 sampling budget" is not supported by the reported protocol. Section 5.1 states K=16 candidates per round for up to T=3 rounds, while Maj@16 is one round of 16 samples. Table 11 shows the co-evolution total is 753.92M tokens (maj@16-equivalent) versus 345.63M for the round-0 elicit that constitutes the Maj@16 setting, a factor of about 2.18. Table 12 similarly shows 194,400 generator outputs over three rounds versus 98,176 for Maj@16. The paper should report a compute-matched control, e.g., Maj@32 or Maj@48 (or a self-consistency baseline with the same total number of samples), before claiming the gains come from co-evolution rather than additional sampling compute.
  2. [§5.1, Implementation Details; §B] The identity of the critic model is not specified for the main results. The case study in §B states the critic is Qwen3-4B, but Table 1 covers four backbones and the main implementation details (Section 5.1, Table 5) do not state which LLM is used as M_phi for each setting, nor the critic's temperature/decoding parameters. Since the paper's own Limitations section identifies critic calibration as a key bottleneck, the critic model and version are load-bearing for reproducibility. Please report the critic configuration for every backbone and benchmark.
  3. [§5.4.1, Tables 3 and 11] The self-evolution ablation is useful but does not fully isolate the co-evolution contribution because it is also not budget-matched to Maj@16. Self-evolution spends 1,071.37M tokens and co-evolution spends 753.92M tokens (Table 11), both larger than the 345.63M token Maj@16 budget. To support the headline claim, the paper needs a control that scales self-consistency or unguided round-based sampling to a comparable total compute, so the remaining gap can be attributed to the critic-guided memory and repair mechanism rather than to a higher sampling budget or to the early-stopping policy.
minor comments (5)
  1. [§C.1, Table 8] The definition of Pass^16 is confusing. The text says "Pass^16 requires all rollouts to be correct," but the values in Table 8 are far too high for that definition (e.g., Qwen3-4B 70.31 on Spider-Dev with Avg@16 82.44). Please clarify the intended metric and whether it means Pass@1, greedy accuracy, or some other quantity.
  2. [§6, Conclusion] The conclusion states that "the utility signal identifies high-quality candidates early." However, Table 2 shows that the T1 (first-round co-evolved) accuracy on Spider-Dev is lower than Maj@16 for Coder-3B (74.47 vs 76.40) and for the average (79.35 vs 81.02). The final gain emerges after multi-round repair. Please qualify the claim to reflect that the initial utility ranking is not always better than majority voting, and that the benefit is concentrated in later rounds.
  3. [§5.3, Figures 3 and 4] The sensitivity analysis is run on 200-example subsets of Spider-Dev and BIRD-Dev. The reported accuracy pairs are discrete and the bubble plots do not include confidence intervals or per-point sample sizes. A table summarizing the number of configurations per region and the default-configuration ranks would help the reader assess how representative the chosen operating point is.
  4. [§A.2, Table 5] Several hyperparameters (gamma, lambda_cons, tau, kappa, pi_cons) are fixed across all experiments. The sensitivity analysis in §5.3 sweeps some of them, but the main-Table 1 results use only the default setting. Please state explicitly whether any hyperparameter was tuned on the evaluation sets or fixed a priori; this affects the strength of the generalization claim.
  5. [Global] The notation for the token-cost comparison in Table 11 ("maj@16-equivalent") is explained in a parenthetical but could be made clearer for a reader who is not familiar with the prefill-counting convention. Also, the phrase "same Maj@16 sampling budget" appears in §5.2.1 and should be removed or qualified in light of the token accounting.

Circularity Check

0 steps flagged

No significant circularity: EvoSQL's reported gains are externally evaluated against gold execution labels, and its critic/utility signals are not defined from those labels or from the reported benchmark results.

full rationale

I walked the claimed derivation chain. The generator samples SQL candidates without gold access (Eq. 2); the critic produces rubric scores and diagnoses conditioned only on question, schema, candidate, and execution record (Eq. 4); utility combines critic confidence, execution caps, time discounting, and a consistency bonus (Eqs. 5-9); final selection is greedy over the utility-ranked pool (Sec. 4.2.2). Final accuracy is measured by comparing execution denotations to gold queries (Eq. 1). None of these steps defines the target metric in terms of the inputs, so the central gain is not circular by construction. SDPO is trained on third-party SynSQL-5k/SQL-R1 data and evaluated on Spider/BIRD, so no self-citation chain forces the result, and I found no load-bearing self-citations, imported uniqueness theorems, or ansatz-smuggling citations. The 200-example sensitivity sweep fixes a single default configuration rather than fitting per reported cell; even if this raises generalization or tuning concerns, it is not definitional circularity. The paper's own Limitations candidly state that cold-start critics can be miscalibrated and that critic-based selection is not always monotonic, which further confirms the critic signal is treated as fallible evidence rather than as a renaming of the answer. Finally, the phrase 'Under the same Maj@16 sampling budget' in Sec. 5.2.1 is in tension with Table 11 (EvoSQL co-evolve spends 753.92M 'maj@16-equivalent' tokens vs. 345.63M for round-0 elicit), but that is a compute-matched-comparison/fairness issue, not a circularity: the extra sampling and critic calls are not a relabeled version of the evaluation outcome. I therefore find no circular steps.

Axiom & Free-Parameter Ledger

7 free parameters · 5 axioms · 0 invented entities

No new physical or theoretical entities are postulated. 'Episodic memory' and 'candidate buffer' are bookkeeping data structures inside the algorithm (Sec. 4.2), not entities requiring independent evidence. The framework contributes no first-principles content; it is an empirical systems paper whose free parameters are inference and training hyperparameters fixed across backbones and benchmarks after a 200-example sensitivity sweep, which reduces per-cell fitting risk. The main uncharged structural assumption is critic reliability, since the entire harness — selection, memory, early stopping, final answer — is built around the critic's scores.

free parameters (7)
  • gamma (time discount) = 0.9
    Eq. 7; down-weights stale candidates in the utility function; chosen by hand from the 200-example sensitivity sweep (App. C.3), fixed across backbones and benchmarks.
  • lambda_cons (consistency coefficient) = 0.3
    Eq. 8; weight of the majority-style execution-consistency bonus in candidate utility.
  • tau (confidence threshold) = 8.0
    Critic-score threshold for counting a candidate as high-confidence in the consistency bonus (Eq. 8) and the early-stop condition (Eq. 10); on the critic's -1..10 integer scale.
  • kappa (softmax temperature) = 1.0
    Eq. 9; controls the exploration-exploitation trade-off in memory payload sampling.
  • pi_cons (early-stop ratio) = 0.90
    Eq. 10; fraction of Top-K candidates that must share one execution outcome before the loop terminates.
  • Candidate budget (K, T, M) = K=16, T=3, M=1
    Samples per round, maximum rounds, and memory payload size (Table 5); these define the test-time compute of the method and are central to the budget-equivalence claim vs Maj@16.
  • SDPO training hyperparameters = LR 1e-5; 5 epochs; 8 rollouts; EMA rate 0.05; top-K 100
    Table 6; configuration for the KL-distillation objective (Eq. 13) and its GRPO-style advantage estimation, chosen by hand.
axioms (5)
  • domain assumption Correctness is defined as denotation equality: Exec(hat_q) = Exec(q) (Eq. 1).
    Standard execution-accuracy criterion for Spider/BIRD; Strict EX treats row/column order as part of the denotation, with a Relaxed variant in App. C.2 that ignores column order.
  • domain assumption The LLM critic's rubric scores correlate with true execution correctness in the absence of gold answers.
    Load-bearing for utility ranking (Eqs. 5-8), payload sampling (Eq. 9), early stopping (Eq. 10), and final greedy selection; the paper's own Limitations section concedes cold-start critics can be miscalibrated.
  • domain assumption Maj@16 (self-consistency with K=16) is the correct baseline for isolating inference-time evolution gains.
    The paper's central comparison (Sec. 5.2.1); both settings share the schema serialization, but EvoSQL spends up to 3x16 generations plus critic calls, so the 'same sampling budget' framing embeds a compute-equivalence assumption that the paper does not justify.
  • domain assumption SQL-R1's SynSQL-5k selection set is a valid offline distillation corpus for Spider/BIRD evaluation.
    SDPO trains on a 5k-example selection from SynSQL-5k (Secs. 4.3, 5.1); the paper implicitly assumes this synthetic supervisory distribution transfers to the evaluation benchmarks.
  • domain assumption The early-stop rule (Eq. 10) terminates with useful precision before the round budget is exhausted for a substantial fraction of problems.
    The efficiency claims in App. C.5 depend on 47.2% of instances stopping before round-1 aggregation and 54.8% before round-2; if the stopping rule were systematically miscalibrated, later rounds would waste or misallocate compute.

pith-pipeline@v1.3.0-alltime-deepseek · 23775 in / 23373 out tokens · 221756 ms · 2026-08-02T12:16:33.583697+00:00 · methodology

0 comments
read the original abstract

Text-to-SQL has advanced rapidly with large language models, but complex database queries still require reasoning beyond one-shot generation, including multi-step decomposition, execution-based diagnosis, and targeted correction. We present EvoSQL, a co-evolution framework that formulates SQL synthesis as an iterative interaction between a generator and a critic. EvoSQL maintains a contextualized candidate memory, verifies SQL candidates with both execution signals and LLM-based critique, and updates its memory through utility-guided aggregation. To strengthen the underlying generator-critic pair, we further introduce a Self-Distillation Policy Optimization (SDPO) fine-tuning stage that injects execution-aware supervision into modern coding LLM backbones. Experiments on Spider and BIRD show that EvoSQL consistently improves open-source models over Maj@16 baselines, with particularly large gains on BIRD-Dev, ranging from +1.37% for Qwen3-4B to +9.19% for Qwen2.5-Coder-3B. SDPO initialization further improves selected backbones on Spider-Test and BIRD-Dev. These results suggest that memory-grounded co-evolution is an effective path toward more reliable and generalizable Text-to-SQL systems. Code is available at https://github.com/valleysprings/EvoSQL.

Figures

Figures reproduced from arXiv: 2607.20489 by Chaojian Shi, Chenyu Zhou, Jianwei Wang, Jiawei Zhou, Kai Wang, Ming Dong.

Figure 1
Figure 1. Figure 1: Paradigms for Text2SQL. (A) Self￾Consistency: sample many candidates and select by majority/critic. (B) Self-Refine: Iteratively critique and revise a single candidate. (C) Self-Evolve: Evolve a pop￾ulation by aggregating candidates across generations, then vote. (D) Co-Evolve (Ours): Multi-round popu￾lation evolution with contextualized episodic memory (store/load), where a critic provides reusable diagno… view at source ↗
Figure 2
Figure 2. Figure 2: Online inference-time workflow of EVOSQL. (A) The generator elicits parallel SQL candidates from the question, schema, hints and M-shot candidates from memory (Round 1+). (B) A two-stage verifier combines deterministic execution feedback with LLM-based critique to score and diagnose candidates. (C) The resulting memory stores verified candidates and summaries, which are used for utility-guided payload samp… view at source ↗
Figure 3
Figure 3. Figure 3 [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Sensitivity Analysis of Co-Evolution Hyperparameters. We evaluate BIRD-Dev and Spider-Dev under a 200-example setting with K = 16 candidate samples and two backbones, Qwen3-4B and SQL-R1-7B. Bubbles summarize repeated BIRD–Spider accuracy pairs, and curves mark model-specific Pareto frontiers across strict/relaxed accuracy and greedy/Pass@16 protocols. C.5 Analysis and Comparison of Efficiency We further c… view at source ↗
Figure 5
Figure 5. Figure 5: Self-Evolution Counterpart. Given the question and schema, the self-evolution method initializes multiple role-based groups to generate SQL candidates in parallel, followed only by execution-based verification. Candidates are stored in a candidate pool, retrieved based on execution type, and the final answer is selected greedily or an additional Best-of-K selector [PITH_FULL_IMAGE:figures/full_fig_p018_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

28 extracted references · 5 linked inside Pith

  1. [1]

    Re-read the original Question and Hint first

  2. [2]

    Identify the requested output columns and row granularity

  3. [3]

    Robert Tjarko Lange, Yuki Imajuku, and Edoardo Cetin

    A survey on deep learning approaches for text- to-sql.The VLDB Journal, 32(4):905–936. Robert Tjarko Lange, Yuki Imajuku, and Edoardo Cetin. 2025. Shinkaevolve: Towards open-ended and sample-efficient program evolution.arXiv preprint arXiv:2509.19349. Fangyu Lei, Jixuan Chen, Yuxiao Ye, Ruisheng Cao, Dongchan Shin, Hongjin Su, Zhaoqing Suo, Hongcheng Gao,...

    Pith/arXiv arXiv 2025

  4. [4]

    Compare candidates as possible evidence, not as truth

  5. [5]

    Reject candidate logic that uses unsupported filters, wrong joins, wrong aggregation, wrong ordering, wrong limit, or wrong output columns

  6. [6]

    Prefer the simplest SQL that directly answers the Question

  7. [7]

    SQL construction constraints: - Treat the Question and Hint as separate fields

    Prefer a small correction to a well-supported candidate, but rewrite from scratch if all candidates share the same semantic error. SQL construction constraints: - Treat the Question and Hint as separate fields. - Use the Hint only as evidence for this specific question. - Use schema-defined keys for joins when available. - Return only the requested column...

  8. [8]

    Identify the required tables, joins, filters, aggregation, ordering, and limits

  9. [9]

    SQL checklist: - Correct output columns and order

    Prefer a simple, direct query over a complex query with extra assumptions. SQL checklist: - Correct output columns and order. - Correct row granularity. - Correct filters and value matching. - Correct joins using schema keys when available. - Correct aggregation and grouping. - Correct ordering direction and result count when ordering or limiting is requi...

  10. [13]

    Reconstruct the intended answer from the original Question and Hint

  11. [14]

    Identify the requested output columns and their order

  12. [15]

    Identify the requested row granularity and result count

  13. [16]

    Identify which tables, joins, filters, aggregation, grouping, ordering, and limits are actually required

  14. [17]

    Use candidates only as possible evidence for tables, joins, filters, and aggregations

  15. [18]

    Do not copy candidate SQL blindly

  16. [19]

    Keep only candidate components that are directly supported by the Schema, Question, and Hint

  17. [20]

    If candidates disagree, choose the logic best supported by the original task, not the majority

  18. [23]

    cumulative_summary

    write one concrete issue summary. No reference answer is available. The SQL has already been executed; do not invent execution results. Treat candidates independently unless their SQL and execution behavior are equivalent. When the candidate is wrong,`issues`must include both the problem and a repair hint. ## Dimensions - alignment: whether the SQL answer...

  19. [24]

    classify execution, <<<

  20. [25]

    score four dimensions, >>>

  21. [26]

    assign one overall quality score,

  22. [27]

    cumulative_summary

    write one concrete issue summary. ... <<< Treat candidates independently unless their SQL and execution behavior are equivalent. ## Dimensions - alignment: whether the SQL answers the question's metric, entity, filters, and requested outputs. - schema: whether tables, columns, aliases, literals, and SQLite syntax are valid and appropriate. - logic: whethe...

  23. [28]

    score four dimensions,

  24. [29]

    cumulative_summary

    write one concrete issue summary. No reference answer is available. The SQL has already been executed; do not invent execution results. Treat candidates independently unless their SQL and execution behavior are equivalent. When the candidate is wrong,`issues`must include both the problem and a repair hint. >>> Score the candidate on four dimensions and wr...

  25. [2021]

    InFindings of the Association for Computational Linguistics: EMNLP 2021, pages 2030–2042, Punta Cana, Dominican Re- public

    Natural SQL: Making SQL easier to infer from natural language specifications. InFindings of the Association for Computational Linguistics: EMNLP 2021, pages 2030–2042, Punta Cana, Dominican Re- public. Association for Computational Linguistics. Chang Gao, Bowen Li, Wenxuan Zhang, Wai Lam, Bin- hua Li, Fei Huang, Luo Si, and Yongbin Li. 2022. Towards gener...

    Pith/arXiv arXiv 2021

  26. [2023]

    Ben Eyal, Amir Bachar, Ophir Haroche, Moran Mahabi, and Michael Elhadad

    C3: Zero-shot text-to-sql with chatgpt.arXiv preprint arXiv:2307.07306. Ben Eyal, Amir Bachar, Ophir Haroche, Moran Mahabi, and Michael Elhadad. 2023. Semantic decomposi- tion of question and sql for text-to-sql parsing.arXiv preprint arXiv:2310.13575. Han Fu, Chang Liu, Bin Wu, Feifei Li, Jian Tan, and Jianling Sun. 2023. Catsql: Towards real world natu-...

    Pith/arXiv arXiv 2023

  27. [2025]

    Tomer Wolfson, Mor Geva, Ankit Gupta, Matt Gard- ner, Yoav Goldberg, Daniel Deutch, and Jonathan Berant

    Ragen: Understanding self-evolution in llm agents via multi-turn reinforcement learning.arXiv preprint arXiv:2504.20073. Tomer Wolfson, Mor Geva, Ankit Gupta, Matt Gard- ner, Yoav Goldberg, Daniel Deutch, and Jonathan Berant. 2020. Break it down: A question understand- ing benchmark.Transactions of the Association for Computational Linguistics, 8:183–198....

    Pith/arXiv arXiv 2020

  28. [2026]

    Please specify all of the schools and their related mailing zip codes that are under Avetik Atoian’s administration

    Self-distilled reasoner: On-policy self- distillation for large language models.arXiv preprint arXiv:2601.18734. A EVOSQL settings A.1 Online Co-evolution Algorithm Algorithm 1Online co-evolution inference in EVOSQL Require: Question x, schema S, database D; generator Mθ; critic Mϕ; candidate number K; maximum rounds T ; payload size M; time discountγ; co...

    Pith/arXiv arXiv 1983