Artificial Intelligence · arXiv:2610.35316 v1 · submitted 26 Oct 2026 · 52 references
RSIV-Harness: A Recursive Self-Improvement Harness Framework for Open-Domain Conversational Text-to-Visualization
The paper says
“Users often request charts without knowing which database or table holds the relevant data. Such requests may be ambiguous, and users may ask follow-up questions to deepen their analysis or iteratively refine existing charts.”
more of the abstract
A useful assistant must therefore discover relevant data sources among heterogeneous databases, ask clarifying questions when needed, and track evolving user intent while preserving constraints established in earlier turns. Yet recent text-to-visualization agents typically rely on static agent harnesses that are predefined and remain unchanged despite recurring failures across tasks. To address this limitation, we introduce RSIV-Harness, which keeps the language model weights fixed and applies Recursive Self-Improvement to revise the surrounding harness from the agent's own failure traces. To balance flexible self-improvement with stability, we introduce a four-layer framework that evaluates proposed harness revisions through anchor and memory checks, refining those that pass and restoring the best validated harness otherwise. We further introduce ODAC-Vis, a benchmark with 640 tasks for Open-Domain, Ambiguous, and Conversational text-to-visualization that jointly evaluates open-domain data discovery, ambiguity resolution, multi-turn visualization refinement, and executable chart generation. Experiments show that a single round of RSIV-Harness improves all six evaluation metrics relative to the seed harness, with particularly large gains in executability (+28.02 points) and chart structure (+34.01 points), demonstrating the ability to learn from task-solving experience and turn recurring failures across tasks into substantial performance improvements.
from the abstract
Pith says
Machine review · deepseek-v4-flash · 27 Oct 2026 · what was read was not recorded · not checked by a person
It can be wrong. Challenge this review or reply to it.
Holds up, with conditions. ODAC-Vis is a genuinely useful benchmark and the stability rules are concrete, but the headline +28.02 gain is not reproducible as printed because the seed baseline contradicts itself between Table 3 and Table 4. This line is wrong
4 major objections · 5 minor
How often objections from reviews like this one are right: not yet measured
Not checked: figures, code, what the 52 references say.
The paper says, at Table 3 vs. Table 4; abstract and Section 5.2.1
The review did not quote this passage.
Pith says: Objection 1 of 4 · major
The static seed H0 is the baseline for the abstract's headline gain, yet it receives executability 64.98 in Table 3 and 51.36 in Table 4 while the other five metrics are identical in both tables (routing 48.25, stance 51.25, exec acc 24.51, clarify 3.72, chart 41.46).
more of objection 1
The +28.02 point gain is computed as 93.00 − 64.98; if 51.36 is the correct seed value, the gain is 41.64. Because Section 5.1.2 and Table 4 both identify H0 as the same static harness on the same test split, the paper cannot contain both numbers. Please correct the inconsistent row and recompute the abstract, Section 5.2.1, and any derived claims, or state explicitly why the two H0 rows are not the same baseline.
The paper says, at Sections 5.1.4, 5.2.1, and Table 4
The review did not quote this passage.
Pith says: Objection 2 of 4 · major
Every reported number is a single-run point estimate. Section 5.1.4 says all models use their default temperature and top-p, and both the inner-loop generation and the outer-loop Improve calls are stochastic, so the claim that one round "improves all six metrics" is currently established for only one realization.
more of objection 2
This matters most for Clarify, which rises from 3.72 to 5.15 in absolute BLEU-4 terms. Please report multiple independent runs or seeds with means and confidence intervals, or provide a bootstrap over test items, so the reader can judge whether the six deltas are stable and statistically meaningful.
The paper says, at Section 5.1.3, Eqs. (44) and (48)
The review did not quote this passage.
Pith says: Objection 3 of 4 · major
The chart-structure metric depends on alpha_x = Title(a_x, a_hat_x), alpha_y = Title(a_y, a_hat_y), and J(I, I_hat), none of which is defined in the text, and the execution-equivalence comparator in Eq. (44) is written as "exists pi: pi(R_hat_t) = R*_t" without specifying the class of permitted transformations pi.
more of objection 3
Since Chart and ExAcc are two of the six headline metrics, the reported comparisons (75.47 vs. 41.46 for chart structure; 31.52 vs. 24.51 for execution accuracy) cannot be reproduced from the manuscript as written. Please define these functions precisely or release the evaluation code.
The paper says, at Sections 3.4 and 3.2
The review did not quote this passage.
Pith says: Objection 4 of 4 · major
The benchmark quality audit covers only 100 tasks per corpus with two experts and no inter-rater agreement statistic, and the LLM evaluator (DeepSeek-V4.1-Flash) is the same model family used to paraphrase and generate the ODAC-Vis tasks. No data or code release link is provided in the manuscript, so the gold dialogues, ambiguity annotations, expert audit sheets, and evaluation harness cannot be inspected.
more of objection 4
Because ODAC-Vis is a central contribution and the transfer claim depends on its gold being faithful, please release the data and evaluation code, report expert agreement, and include an LLM judge from a different family or a human disagreement analysis.
5 minor objections
The paper says, at Section 5.3.2 and Table 4
Pith says: minor objection 1 of 5
The prose says each component of H0 is replaced by its evolved counterpart, while the Table 4 caption says the remaining variants add one evolved component to H0; please align these two descriptions.
The paper says, at Sections 4.3 and 5.1.4
Pith says: minor objection 2 of 5
Algorithm 1 requires the number of rounds N and hyperparameters epsilon, delta, C, E, m(H), K(H), and ell, but the experiments section only states that one round is performed; please report the exact values used.
show 3 more
The paper says, at Section 5.1.3, Eqs. (45)-(46)
Pith says: minor objection 3 of 5
Clarification quality is BLEU-4 computed without a brevity penalty, so short generic clarification questions can receive inflated scores; please justify this choice or add a brevity penalty.
The paper says, at Section 4.4.2, Theorems 1-4
Pith says: minor objection 4 of 5
The theorems are correct but are direct restatements of the definitions in Eqs. (18), (28), and (33)-(34); consider describing them as invariants of the commit protocol rather than as guarantees that evolution improves task performance.
The paper says, at Front matter
Pith says: minor objection 5 of 5
The ACM reference block and conference header still use 2018 template metadata, which conflicts with the 2026 arXiv date; the template should be updated before publication.
What would settle it
Re-run the static seed H0 with DeepSeek-V4.1-Flash on the full 96-task test split at least three times under the exact Table 3 protocol (default hyperparameters, same tool set), reporting executability mean and spread. If it lands near 64.98, the Table 4 H0 row and the ablation's 'w/o rule' comparison need re-derivation; if it lands near 51.36, Table 3 and the abstract's +28.02/−34.01 deltas must be recomputed. In parallel, report the run-to-run spread on all six metrics across seeds to establish that 'all six improve' is not a single-trajectory artifact.
from the skeptic's reading
How the review went
4 readings by deepseek-v4-flash · 27 Oct 2026
- Reader holds up, with conditions
- Skeptic partly agrees
- Editor wrote the verdict above
- Referee major revision: 4 major objections, 5 minor
Not run on this paper: the simulated authors' defence.
Pith did not check this review's objections against the paper.
The review in full
The review in brief, machine-written
Two things to know. First, ODAC-Vis is a real contribution: 640 tasks over 206 Spider databases, combining open-domain source discovery, typed ambiguity, multi-turn refinement, and executable query chart supervision, with a domain-held-out split. That combination is new, and the construction pipeline is described carefully enough to be audited and reproduced. Second, the central empirical claim is not reproducible as printed because the seed baseline disagrees with itself. Table 3 lists the DeepSeek static harness H0 at 64.98 executability; Table 4 lists the same H0 at 51.36, with all other metrics identical between the two rows. The abstract's +28.02 gain is computed from 64.98. If 51.36 is the real seed, the gain is 41.64; if 64.98 is real, the ablation row is unexplained. The paper never reconciles this, and it sits directly under the headline number.
What the paper does well: the four-layer stability rules are concrete and the theorems, while simple restatements of the acceptance definitions, are correctly stated for what they claim. The cross-backbone transfer experiment is a nice touch, and the ablation isolating the stability rules gives real signal. The method itself is harness evolution from failure traces, which the cited literature already contains (Self-Harness, TTHE, Agentic Harness Engineering, DREvo); the novelty is the application to conversational text-to-visualization plus the specific stability machinery, not the paradigm.
Soft spots beyond the table discrepancy: no code, data, or benchmark URL, so the 100-task expert audit cannot be checked; all metrics are single-run point estimates with no variance; no competing harness-evolution method is used as a baseline; the clarify gain is 3.72 to 5.15 in BLEU-4, small in absolute terms; and the LLM judge is the same model family used to generate the benchmark. None of these are fatal on their own, but together with the seed inconsistency they mean the reported magnitudes should be treated as provisional.
This paper is for people building evaluation benchmarks for agentic visualization and teams working on post-deployment harness adaptation. It deserves a serious referee rather than a desk rejection, but the referee should be instructed to check the H0 discrepancy and to request the data. I would not cite the headline numbers until they are reconciled, though the benchmark itself is worth watching.
The strongest objection
The central claim, as stated in the abstract and Section 5.2.1, is that a single round of RSIV-Harness improves all six metrics over the seed harness, with executability +28.02 and chart structure +34.01. Both deltas are computed against the static seed H0: 93.00 − 64.98 = 28.02 and 75.47 − 41.46 = 34.01 (Table 3). But the same seed harness H0 appears in Table 4 with executability 51.36, a 13.62-point discrepancy, and all six values differ between the two tables (routing 48.25 vs 48.25, stance 51.25 vs 51.25, exec acc 24.51 vs 24.51, clarify 3.72 vs 3.72, chart 41.46 vs 41.46 — only executability changes, from 64.98 to 51.36). The paper never reconciles this: Section 5.1.2 justifies a deliberately minimal seed to prevent contamination, and Table 4's H0 row is described as the static harness to which components are added, so both labels refer to the intended comparison baseline. If the true seed executability is 51.36, the improvement is 41.64 points, and the abstract's headline number is wrong; if it is 64.98, the ablation's H0 row is unexplained. Either way the magnitude of the central claim is not pinned down. Compounding this, all six metrics are single-run point estimates with no variance, and the clarify metric moves only 3.72 → 5.15 in absolute BLEU-4 terms, so 'improves all six' is asserted for one realization. This internal inconsistency sits directly on the baseline of the headline claim and is distinct from the benchmark-gold faithfulness concern the reader emphasized; I treat it as the more load-bearing issue because it affects the numbers as printed, independent of any judgment about gold quality.
The reader correctly listed the seed executability discrepancy (64.98 vs 51.36) as one of several unresolved issues, but designated benchmark gold faithfulness as the weakest assumption. I agree the gold audit is thin (100 tasks, two experts, no inter-rater agreement, data unreleased), yet I judge the seed-baseline inconsistency to be the single most load-bearing concern because it bears directly on the headline deltas in the abstract and Section 5.2.1 without requiring any judgment about real-world transfer: the paper is internally inconsistent about the value of the comparison baseline on which its flagship numbers are computed. The direction of improvement is robust to either seed value (93.00 exceeds both 64.98 and 51.36), so this does not overturn the claim, but it makes the stated magnitudes non-reproducible as written, which is exactly the kind of issue that keeps a paper at CONDITIONAL rather than ACCEPT. The reader's verdict already anticipates this; my read corroborates and sharpens it without changing the verdict. I kept the recommendation UNCHANGED because the reader's CONDITIONAL already covers this concern, and no new reason to reject emerges: the method is coherent, the four formal guarantees are correct restatements of the acceptance rules, and the held-out domain split is a genuine strength. The single most decisive check remains re-running H0 to pin its executability, which would settle both the seed inconsistency and, by extension, the variance question for the headline metrics.
The referee's report
If the reported effects are reproducible, the paper is a meaningful demonstration that a fixed-weight LLM agent's surrounding harness can be improved from its own failures on genuinely held-out domains, and ODAC-Vis fills a real gap by combining source discovery, typed ambiguity resolution, and multi-turn chart refinement in a single benchmark. The domain-held-out split, the cross-database composability-graph construction, and the component-level ablation are concrete strengths. The formal theorems are correct but lightweight: they are immediate consequences of the commit and check definitions rather than substantive convergence or generalization guarantees. The contribution is presently undercut by an unresolved inconsistency in the seed baseline between the two main result tables, single-run evaluations with no variance estimates, and several metric components that are not fully specified, so the exact magnitude and statistical solidity of the central claim are not yet established.
Pith's reading of the paper (a machine's summary, not the authors')
Users often ask for charts without naming the database or table, leave their requests ambiguous, and refine the result across several turns. This paper argues that an assistant for such open-domain conversational visualization can be improved without retraining the language model: it keeps the weights fixed and instead revises the surrounding harness, meaning the prompt, tools, memory, skills, and middleware, based on the agent's own failure traces. To make that self-improvement safe, the paper adds a four-layer stability framework with formal guarantees that accepted revisions refresh protected records, rejected revisions restore the champion, and the best validation score never decreases. To test the idea, the paper introduces ODAC-Vis, a 640-task benchmark spanning 206 databases and 139 domains that combines open-domain source discovery, typed ambiguity resolution, multi-turn refinement, and executable chart generation. A single self-improvement round improves all six evaluation metrics over the seed harness, with executability up 28.02 points and chart structure up 34.01 points.
What carries the argument. The carrying mechanism is a bi-level loop: an inner ReAct-style task loop answers one assistant turn, deciding between a clarification question and executable chart code and retrying within a bounded repair budget, while an outer self-improvement loop analyzes traces across tasks, proposes a new harness with at most $E$ edits, and commits it only under a four-layer stability rule set. The four layers are refine rules, which refresh a bounded protected-record memory and reset the stored rejection verdict on acceptance; restart rules, which restore the champion harness and freeze memory on rejection; an anchor check, requiring the candidate to score within $(1-\varepsilon)$ of the champion on a fixed validation anchor; and a memory check, requiring the regression count on a stratified sample of protected records to stay within a tolerance $\delta$. The ODAC-Vis benchmark is the second load-bearing object: it defines the eight typed ambiguity templates, the drilldown and iterate conversation scenarios, and the SQL-only cross-database composition protocol that make open-domain evaluation possible.
Core claim. The paper's central claim is that recursive self-improvement of the harness, not of the model, is what improves an open-domain conversational visualization agent. Starting from a minimal seed harness, an outer loop reads execution traces from a stratified 48-task training panel, groups recurring failures, and proposes targeted edits to one or more of the five harness components; the candidate is accepted only if it passes an anchor check on a fixed 48-task validation set and a regression-memory check on sampled protected records, and is otherwise rejected in favor of the current champion. The paper proves four formal guarantees for this loop: accepted revisions refresh the protected-record memory, rejected revisions restore the champion without touching memory, the champion's validation score is nondecreasing, and regression on the sampled protected records stays within a fixed tolerance. Underlying the empirical result is the new ODAC-Vis benchmark, whose domain-held-out test split forces the agent to discover sources among all 206 exposed databases rather than memorize schemas. On that test split, one round of evolution with DeepSeek-V4.1-Flash beats the best baseline by 11.68 points in executability and 23.24 points in chart structure, and the same frozen harness transfers to GPT-6 Luna with gains on five of six metrics.
The premise it rests on. The load-bearing premise is that the benchmark's gold, built from templates plus DeepSeek paraphrasing and spot-checked by two experts on only 100 of the 640 tasks with no reported inter-rater agreement, faithfully represents how real open-domain conversational chart requests are phrased.
If the paper is right (Pith's inferences, not the authors')
An analyst-facing text-to-visualization assistant could be fielded with a cheap model and improved in deployment from its own failures, since the method requires no retraining or replacement of the weights.
The formal guarantees mean repeated evolution rounds are safe in principle: the champion validation score never drops and sampled protected records regress by at most $\delta$, so gains compound rather than oscillate.
Because the evolved harness transfers to a different backbone, GPT-6 Luna, with gains on five of six metrics over the static harness, harness edits encode behavior worth keeping even when the model changes.
ODAC-Vis gives future text-to-visualization systems a single testbed that pairs executable code with reference charts and typed ambiguity gold, combining source discovery, clarification, and multi-turn refinement in one evaluation.
Ablations attribute much of the gain to the stability rules and to coordination among components rather than any single edit, so the method's value lies in the safety layer as much as in the revision loop.
A second self-improvement round is the obvious next test: the anchor-monotonicity guarantee implies the champion score cannot fall below the single-round result, so multi-round evolution should be probed for saturation rather than assumed to keep climbing.
The cross-backbone transfer is partial, since the GPT execution-accuracy score sits 0.78 points below the static GPT harness, which suggests harness edits learned with one model are not fully portable and evolving with the target backbone may be needed for full transfer.
The machinery is not specific to visualization: any agentic workload with cheap failure traces and a measurable reference, such as text-to-SQL or code repair, could reuse the same anchor-plus-memory commit rules, but the paper demonstrates only chart generation.
The 48-task training panel and 48-task anchor are small enough that the empirical gains could partly reflect which 48 tasks the harness happened to learn from, and re-running with larger panel sizes would show whether the method scales.
Check this review
Pith stored no record of this review's passes and took no fingerprint of it. Reviews made by code from 13 Nov 2026 on carry both.
How often reviews like this one are right
Reviews like this one: written by deepseek-v4-flash, where what was read was not recorded.
How often objections from reviews like this one are right: not yet measured
Reviews by people
This review was written by a machine. No person has reviewed this paper on Pith yet. A scientist listed on Pith can review it for a fee, and their review is published here whatever it concludes.
The paper
Abstract
Users often request charts without knowing which database or table holds the relevant data. Such requests may be ambiguous, and users may ask follow-up questions to deepen their analysis or iteratively refine existing charts. A useful assistant must therefore discover relevant data sources among heterogeneous databases, ask clarifying questions when needed, and track evolving user intent while preserving constraints established in earlier turns. Yet recent text-to-visualization agents typically rely on static agent harnesses that are predefined and remain unchanged despite recurring failures across tasks. To address this limitation, we introduce RSIV-Harness, which keeps the language model weights fixed and applies Recursive Self-Improvement to revise the surrounding harness from the agent's own failure traces. To balance flexible self-improvement with stability, we introduce a four-layer framework that evaluates proposed harness revisions through anchor and memory checks, refining those that pass and restoring the best validated harness otherwise. We further introduce ODAC-Vis, a benchmark with 640 tasks for Open-Domain, Ambiguous, and Conversational text-to-visualization that jointly evaluates open-domain data discovery, ambiguity resolution, multi-turn visualization refinement, and executable chart generation. Experiments show that a single round of RSIV-Harness improves all six evaluation metrics relative to the seed harness, with particularly large gains in executability (+28.02 points) and chart structure (+34.01 points), demonstrating the ability to learn from task-solving experience and turn recurring failures across tasks into substantial performance improvements.
The paper's own sentences, arranged, kept beside the review and separate from it. Each label is drawn from a closed vocabulary; the number beside it is the probability the classifier gave that label. No sentence in this record was written by a model.
abstract compiled · 2026-10-27
The authors' main claim, as the classifier picked it out 1.00
“Experiments show that a single round of RSIV-Harness improves all six evaluation metrics relative to the seed harness, with particularly large gains in executability (+28.02 points) and chart structure (+34.01 points), demonstrating the ability to learn from task-solving experience and turn recurring failures across tasks into substantial performance improvements.”
How the main claim is stated: asserted 1.00
What the abstract itself states, as probabilities: claims first 0.08 · states limitation 0.20 · human participants 0.10 · controlled comparison 0.82 · randomized assignment 0.01 · states quantitative result 0.99 · compares with prior results 0.86 · mentions code or data availability 0.05
The abstract, sentence by sentence
This table labels the abstract only (arXiv listing abstract v1). The number is the probability the classifier gave the label.
| Sentence | Role | The authors' own work | States the main conclusion | Strength |
|---|---|---|---|---|
| Users often request charts without knowing which database or table holds the relevant data. | background 1.00 | 0.07 | 0.01 | asserted 0.95 |
| Such requests may be ambiguous, and users may ask follow-up questions to deepen their analysis or iteratively refine existing charts. | background 1.00 | 0.06 | 0.01 | hedged possibility 1.00 |
| A useful assistant must therefore discover relevant data sources among heterogeneous databases, ask clarifying questions when needed, and track evolving user intent while preserving constraints established in earlier turns. | background 0.98 | 0.05 | 0.03 | asserted 0.97 |
| Yet recent text-to-visualization agents typically rely on static agent harnesses that are predefined and remain unchanged despite recurring failures across tasks. | gap 0.77 | 0.06 | 0.02 | asserted 1.00 |
| To address this limitation, we introduce RSIV-Harness, which keeps the language model weights fixed and applies Recursive Self-Improvement to revise the surrounding harness from the agent's own failure traces. | contribution 0.76 | 0.89 | 0.07 | asserted 0.98 |
| To balance flexible self-improvement with stability, we introduce a four-layer framework that evaluates proposed harness revisions through anchor and memory checks, refining those that pass and restoring the best validated harness otherwise. | method 0.87 | 0.93 | 0.05 | asserted 0.99 |
| We further introduce ODAC-Vis, a benchmark with 640 tasks for Open-Domain, Ambiguous, and Conversational text-to-visualization that jointly evaluates open-domain data discovery, ambiguity resolution, multi-turn visualization refinement, and executable chart generation. | contribution 0.91 | 0.94 | 0.05 | asserted 0.99 |
| Experiments show that a single round of RSIV-Harness improves all six evaluation metrics relative to the seed harness, with particularly large gains in executability (+28.02 points) and chart structure (+34.01 points), demonstrating the ability to learn from task-solving experience and turn recurring failures across tasks into substantial performance improvements. | result 1.00 | 0.97 | 0.85 | asserted 1.00 |
What kind of paper the classifier reports this is: method or tool 1.00 · evidence from observation or experiment 1.00
What was examined, and what was not
Sources read
- arXiv listing abstract v1
Not examined
- full text
- figures and tables
- references and citing sentences
- relations to other papers
Levels run
- P abstract
Not run
- S sections
- T body sentences
- C citations
- R cross-paper relations
How the labels were made
Compiled by classifier-1.13.0 via TypeSafe API, question bank abstract-p1-20260923, 2026-10-27: 1 request, 6,474 input tokens.
Every quoted sentence on this record is the authors'. Every label is from Pith's closed vocabulary, with the probability shown beside it. This record as JSON
Figures (5)
Figures are the authors', reproduced from the paper.
References (52)
- [1] Olena Bogdanov, Yeunji Jung, Chandra Dhir, Pareekshitreddy Gaddam, Saurabh Jain, Lakshmi Tumati, Vijay Parthasarathy, and Anup Shirgaonkar on Pith
- [2] Jipeng Cen, Jiaxin Liu, Zhixu Li, and Jingjing Wang. 2025. SQLFixAgent: to- wards semantic-accurate text-to-SQL parsing via consistency-enhanced multi- agent collaboration. InProceedings of the Thirty-Ninth AAAI Conference on Ar- tificial Intelligence and Thirty-Seventh Confer... on Pithdoi:10.1609/aaai.v39i1.31979
- [3] Daojun Chen, Xi Wang, Shenyuan Ren, Qingzhi Ma, Pengpeng Zhao, and An Liu on Pith
- [4] Nan Chen, Yuge Zhang, Jiahang Xu, Kan Ren, and Yuqing Yang. 2025. VisEval: A Benchmark for Data Visualization in the Era of Large Language Models.IEEE Transactions on Visualization and Computer Graphics31, 1 (2025), 1301–1311. doi:10.1109/TVCG.2024.3456320 on Pithdoi:10.1109/tvcg.2024.3456320
- [5] GBV-SQL: Guided Generation and SQL2Text Back-Translation Validation for Multi-Agent Text2SQL. InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Maria Liakata, Viviane P. Moreira, Jiajun Zhang, and David Jurgens (... on Pithdoi:10.18653/v1/2026.acl-long.837
- [6] Hanghui Guo, Weijie Shi, Zhangze Chen, Shengxiang Xu, Yishu Wang, Yimei Zhang, Wangze Ni, Jia Zhu, and Shimin Di. 2026. DREvo: Distilling Recalibrated Historical Experience for Harness Self-Evolution. arXiv:2607.26722 [cs.MA] https://arxiv.org/abs/2607.26722 on PithPith's reviewarXiv
- [7] Chao Deng, Ju Fan, Yuyu Luo, Qinliang Xue, Meihao Fan, Yuxin Zhang, Min Zhang, Xiaofeng Jia, Jing Zhang, and Xiaoyong Du. 2026. TACO: A Benchmark for Open-Domain Text-to-SQL with Ambiguous and Cross-Database Queries. Proc. VLDB Endow.19, 6 (May 2026), 1386–1399. doi:10.14778/3... on Pithdoi:10.14778/3797919.3797942
- [8] Yoonho Lee, Roshen Nair, Qizheng Zhang, Kangwook Lee, Omar Khattab, and Chelsea Finn. 2026. Meta-Harness: End-to-End Optimization of Model Harnesses. arXiv:2603.28052 [cs.AI] https://arxiv.org/abs/2603.28052 on PithPith's reviewarXiv
- [9] Omar Khattab, Arnav Singhvi, Paridhi Maheshwari, Zhiyuan Zhang, Keshav Santhanam, Sri Vardhamanan, Saiful Haq, Ashutosh Sharma, Thomas T. Joshi, Hanna Moazam, Heather Miller, Matei Zaharia, and Christopher Potts. 2023. DSPy: Compiling Declarative Language Model Calls into Self... on PithPith's reviewarXiv
- [10] Jiahang Lin, Shichun Liu, Chengjun Pan, Lizhi Lin, Shihan Dou, Zhiheng Xi, Xuanjing Huang, Hang Yan, Zhenhua Han, Tao Gui, and Yu-Gang Jiang. 2026. Agentic Harness Engineering: Observability-Driven Automatic Evolution of Coding-Agent Harnesses. arXiv:2604.25850 [cs.CL] https:/... on PithPith's reviewarXiv
- [11] Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, Xuanhe Zhou, Chenhao Ma, Guoliang Li, Kevin Chang, Fei Huang, Reynold Cheng, and Yongbin Li. 2023. Can LLM Already Serve as A Database Interface? A BIg Bench for Lar... on Pith
- [12] Jinwei Lu, Jiawei Lu, Chen Zhang, Zhiqian Qin, Haodi Zhang, Yuanfeng Song, and Raymond Chi-Wing Wong. 2026. Bridging the Gap: Enabling Natural Language Queries for NoSQL Databases through Text-to-NoSQL Translation. arXiv:2502.11201 [cs.DB] https://arxiv.org/abs/2502.11201 on PitharXiv
- [13] Zewen Liu, Zhan Shi, Yisi Sang, Bing He, Minhua Lin, Tianxin Wei, Dakuo Wang, Benoit Dumoulin, Wei Jin, and Hanqing Lu. 2026. Adaptive Auto-Harness: Sustained Self-Improvement for Agentic System Deployment on Open-Ended Task Streams. arXiv:2606.01770 [cs.LG] https://arxiv.org/... on PithPith's reviewarXiv
- [14] Jinwei Lu, Yuanfeng Song, Chen Zhang, and Raymond Chi-Wing Wong. 2026. MultiVis-Agent: A Multi-Agent Framework with Logic Rules for Reliable and Comprehensive Cross-Modal Data Visualization.Proc. ACM Manag. Data4, 1, Article 56 (April 2026), 25 pages. doi:10.1145/3786670 on Pithdoi:10.1145/3786670
- [15] Jinwei Lu, Jiawei Lu, Chen Zhang, Yuanfeng Song, and Raymond Chi-Wing Wong on Pith
- [16] Demonstration of MultiVis-Agent: Interactively Generating Reliable Visu- alizations via Logic-Rule Agents. InCompanion of the International Conference on Management of Data(India)(SIGMOD Companion ’26). Association for Computing Machinery, New York, NY, USA, 98–101. doi:10.114... on Pithdoi:10.1145/3788853.3801584
- [17] Shuqi Lu, Chaofan Li, Kun Luo, Zhang Zhang, Hui Wang, Hongwang Xiao, Lei Xiong, Jiahao Wang, Sen Wang, Xiyan Jiang, Wanli Li, Yuyang Hu, Hongjin Qian, Bingyu Yan, Jianlyu Chen, Ziyi Xia, Yingxia Shao, Kang Liu, Zhicheng Dou, Di He, Chaozhuo Li, Qiwei Ye, Zhongyuan Wang, and Zh... on PithPith's reviewarXiv
- [18] Jinwei Lu, Yuanfeng Song, Haodi Zhang, Chen Jason Zhang, Kaishun Wu, and Raymond Chi-Wing Wong. 2025. Towards Robustness of Text-to-Visualization Translation Against Lexical and Phrasal Variability. In2025 IEEE 41st International Conference on Data Engineering (ICDE). 793–806.... on Pithdoi:10.1109/icde65448.2025
- [19] Jinwei Lu, Wenxin Xu, Yuanfeng Song, Haodi Zhang, Chen Jason Zhang, Kaishun Wu, Xiaoyong Wei, and Raymond Chi-Wing Wong. 2026. Towards Robust and Interactive Text-to-Visualization Translation: Enhancing Model Robustness and Mitigating Ambiguity in Natural Language Queries.The ... on Pith
- [20] Zhensheng Luo, Sai Wu, Yuan Qiu, Chang Yao, Gang Chen, and Xiu Tang. 2026. QBridge: Bridging Natural Language and SQL via Gold Query Rewriting with Agentic Refinement. InProceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pap... on Pithdoi:10.18653/v1/
show 32 more
- [21] Tianqi Luo, Chuhan Huang, Leixian Shen, Boyan Li, Shuyu Shen, Wei Zeng, Nan Tang, and Yuyu Luo. 2025. nvBench 2.0: Resolving Ambiguity in Text- to-Visualization through Stepwise Reasoning. InAdvances in Neural Informa- tion Processing Systems, D. Belgrave, C. Zhang, H. Lin, R.... on Pithdoi:10.52202/085713-4172
- [22] Yuyu Luo, Nan Tang, Guoliang Li, Chengliang Chai, Wenbo Li, and Xuedi Qin on Pith
- [23] Jun Nie, Yonggang Zhang, Jun Song, Qianshu Cai, Dahai Yu, Yike Guo, Xinmei Tian, and Bo Han. 2026. TTHE: Test-Time Harness Evolution. arXiv:2607.08124 [cs.SE] https://arxiv.org/abs/2607.08124 on PithPith's reviewarXiv
- [24] OpenAI. 2026. Introducing GPT-6 Sol and Luna. https://openai.com/index/ introducing-gpt-6-sol-and-luna/ on Pith
- [25] Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. 2023. Self-Refine: It... on Pith
- [26] Paula Maddigan and Teo Susnjak. 2023. Chat2VIS: Generating Data Visualizations via Natural Language Using ChatGPT, Codex and GPT-3 Large Language Models. IEEE Access11 (2023), 45181–45193. doi:10.1109/ACCESS.2023.3274199 on Pithdoi:10.1109/access.2023.3274199
- [27] Mizanur Rahman, Md Tahmid Rahman Laskar, Shafiq Joty, and Enamul Hoque on Pith
- [28] Tabinda Sarwar, Farhad Moghimifar, Cong Duy Vu Hoang, Xiaoxiao Ma, Shawn Chang Xu, Fahimeh Saleh, Poorya Zaremoodi, Avirup Sil, and Katrin Kirchhoff. 2026. CLARITY: A Framework and Benchmark for Conversational Language Ambiguity and Unanswerability in Interactive NL2SQL System... on Pith
- [29] Geliang Ouyang, Jingyao Chen, Zhihe Nie, Yi Gui, Yao Wan, Hongyu Zhang, and Dongping Chen. 2025. nvAgent: Automated Data Visualization from Natural Language via Collaborative Agent Workflow. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguist... on Pithdoi:10.18653/v1/2025.acl-long.960
- [30] Mohammadreza Pourreza and Davood Rafiei. 2023. DIN-SQL: Decomposed In- Context Learning of Text-to-SQL with Self-Correction. InAdvances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36. Curran Associa... on Pithdoi:10.52202/075280-1577
- [31] Yuanfeng Song, Jinwei Lu, and Raymond Chi-Wing Wong. 2026. CoVis: Neural and LLM-Driven Multi-Turn Interactions for Conversational Text-to- Visualization Generation.The VLDB Journal35, 1 (2026), 3 on Pith
- [32] Yuanfeng Song, Xuefang Zhao, and Raymond Chi-Wing Wong. 2024. Marry- ing Dialogue Systems with Data Visualization: Interactive Data Visualization Generation from Natural Language Conversations. InProceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Min... on Pithdoi:10.1145/3637528.3671935
- [33] Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, and Amin Saberi. 2024. CHESS: Contextual Harnessing for Efficient SQL Synthesis. arXiv:2405.16755 [cs.LG] https://arxiv.org/abs/2405.16755 on PithPith's reviewarXiv
- [34] Bing Wang, Changyu Ren, Jian Yang, Xinnian Liang, Jiaqi Bai, LinZheng Chai, Zhao Yan, Qian-Wen Zhang, Di Yin, Xing Sun, and Zhoujun Li. 2025. MAC-SQL: A Multi-Agent Collaborative Framework for Text-to-SQL. InProceedings of the 31st International Conference on Computational Lin... on Pith
- [35] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: language agents with verbal reinforcement learn- ing. InAdvances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (E... on Pithdoi:10.52202/075280-0377
- [36] Yuanfeng Song, Jinwei Lu, Yuanwei Song, Caleb Chen Cao, Raymond Chi-Wing Wong, and Haodi Zhang. 2025. FeVisQA: Free-Form Question Answering over Data Visualizations. In2025 IEEE 41st International Conference on Data Engineering (ICDE). 2726–2739. doi:10.1109/ICDE65448.2025.00205 on Pithdoi:10.1109/icde65448.2025.00205
- [37] Wenxin Xu, Jinwei Lu, Hwanhee Kim, Chen Jason Zhang, Xiao-Yong Wei, Haoyang Li, and Yuanfeng Song. 2026. VisInteract: Towards Dynamic Inter- active Text-to-Visualization under Imperfect Queries. arXiv:2609.15182 [cs.AI] https://arxiv.org/abs/2609.15182 on PithPith's reviewarXiv
- [38] Wenxin Xu, Chen Jason Zhang, Xiaoyong Wei, Haoyang Li, Hwanhee Kim, Yuan- feng Song, and Raymond Chi-Wing Wong. 2026. Towards Reliable Agentic Pro- gressive Text-to-Visualization with Verification Rules. arXiv:2605.29692 [cs.DB] https://arxiv.org/abs/2605.29692 on PithPith's reviewarXiv
- [39] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. InInternational Conference on Learning Representations (ICLR) on Pith
- [40] Tao Yu, Rui Zhang, Heyang Er, Suyi Li, Eric Xue, Bo Pang, Xi Victoria Lin, Yi Chern Tan, Tianze Shi, Zihan Li, Youxuan Jiang, Michihiro Yasunaga, Sungrok Shim, Tao Chen, Alexander Fabbri, Zifan Li, Luyao Chen, Yuwen Zhang, Shreya Dixit, Vincent Zhang, Caiming Xiong, Richard So... on Pith
- [41] Anyi Xu, B Li, Bangcai Lin, Bing Xue, BingCheng Xian, Bingzheng Xu, Bochao Wu, Bowei Zhang, Boyi Deng, CC Yu, et al. 2026. Deepseek-v4. 1-flash: Pushing the limits of kv cache compression.arXiv preprint arXiv:2609.19969(2026) on PitharXiv
- [42] Keyan Xu, Dingzirui Wang, Xuanliang Zhang, Qingfu Zhu, and Wanxiang Che on Pith
- [43] Abacus-SQL: A Text-to-SQL System Empowering Cross-Domain and Open- Domain Database Retrieval. InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations), Pushkar Mishra, Smaranda Muresan, and Tao Yu (Eds.). Assoc... on Pithdoi:10.18653/v1/2025.acl-demo.12
- [44] Congjie Zheng, Chuanyi Xue, Bin Liang, Jun Yang, and Changshui Zhang on Pith
- [48] Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev. 2018. Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task. InPr... on Pith
- [49] Tao Yu, Rui Zhang, Michihiro Yasunaga, Yi Chern Tan, Xi Victoria Lin, Suyi Li, Heyang Er, Irene Li, Bo Pang, Tao Chen, Emily Ji, Shreya Dixit, David Proctor, Sungrok Shim, Jonathan Kraft, Vincent Zhang, Caiming Xiong, Richard Socher, and Dragomir Radev. 2019. SParC: Cross-Doma... on Pith
- [50] Hangfan Zhang, Shao Zhang, Kangcong Li, Chen Zhang, Yang Chen, Yiqun Zhang, Lei Bai, and Shuyue Hu. 2026. Self-Harness: Harnesses That Improve Themselves. arXiv:2606.09498 [cs.CL] https://arxiv.org/abs/2606.09498 on PithPith's reviewarXiv
- [52] SEAGym: An Evaluation Environment for Self-Evolving LLM Agents. arXiv:2606.17546 [cs.AI] https://arxiv.org/abs/2606.17546 on PithPith's reviewarXiv
- [1250] doi:10.18653/v1/2026.acl-industry.86 on Pithdoi:10.18653/v1/2026.acl-industry.86
- [2021] Synthesizing Natural Language to Visualization (NL2VIS) Benchmarks from NL2SQL Benchmarks. InProceedings of the 2021 International Conference on Management of Data(Virtual Event, China)(SIGMOD ’21). Association for Computing Machinery, New York, NY, USA, 1235–1247. doi:10.1145... on Pithdoi:10.1145/3448016
- [2025] Text2Vis: A Challenging and Diverse Benchmark for Generating Multi- modal Visualizations from Text. InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (Eds.). ... on Pithdoi:10.18653/v1/2025.emnlp-main.1622
- [2026] AgentNLQ: A General-Purpose Agent for Natural Language to SQL. arXiv:2605.19010 [cs.AI] https://arxiv.org/abs/2605.19010 on PithPith's reviewarXiv
Challenges and discussion
Disagree with the review, or know something it missed? Leave a comment, or challenge this review formally: a challenge becomes a permanent entry in this paper's record, and its decision is published here.
Come back by email
A short list of the week's newly reviewed papers. Nothing is sent until you confirm from your inbox, and every email has a link that stops it. No account.
No comments yet.
Sign in to comment