TARGET: Benchmarking Table Retrieval for Generative Tasks

Aditya G. Parameswaran; Madelon Hulsebos; Parker Glenn; Xingyu Ji

REVIEW 4 major objections 6 minor 45 references

TARGET: Benchmarking Table Retrieval for Generative Tasks

T0 review · 4 major / 6 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper introduces TARGET, a benchmark that measures table retrieval for generative tasks and shows that dense table embeddings outperform sparse lexical retrieval while the rank of the retrieved table strongly drives downstream answer…

desk verdict A genuinely useful table-retrieval benchmark whose text-to-SQL results are currently unreproducible until the Spider test-split provenance and sqlglot extraction are fixed. read the letter →

arxiv 2505.11545 v1 pith:SXIOVZFJ submitted 2025-05-14 cs.IR cs.AIcs.CLcs.DB

Xingyu Ji , Parker Glenn , Aditya G. Parameswaran , Madelon Hulsebos This is my paper · ORCID

classification cs.IRcs.AIcs.CLcs.DB

keywords tableretrievalretrieval-augmentedgenerationdenseembeddingssparselexicalBM25text-to-SQLfactverificationquestionanswering

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper introduces TARGET, a benchmark that evaluates table retrieval in isolation and as the first step of end-to-end generative tasks: question answering, fact verification, and text-to-SQL, across five datasets. It claims that dense embeddings of tables, rows, or table metadata reliably outperform sparse lexical retrieval (BM25/TF-IDF) over tabular data, and that lexical search is less effective on tables than on unstructured text. It also reports a strong negative correlation (average Spearman's $\rho = -0.85$) between the rank of a ground-truth table in retrieval results and the accuracy of the downstream LLM answer, meaning that better retrieval rank directly improves generated answers. A sympathetic reader would care because the benchmark gives the understudied retrieval step in open-domain tabular RAG a standard yardstick and identifies where current retrievers fail, for instance when table titles are uninformative or corpora grow large.

What carries the argument

The central object is the TARGET benchmark itself: a pipeline that takes a natural-language query and a table corpus, runs a pluggable retriever, hands the top-$k$ tables to a GPT-4o-mini generator, and scores retrieval separately from the downstream task. The load-bearing measurement is capped recall at $k=10$, with denominator $\min(k, |T_i|)$ when a query needs multiple tables, and the main analytical instrument is the rank-to-accuracy curve: instances are grouped by the rank of the ground-truth table (1-2, 3-4, …, not in top-10), and the monotone drop in downstream accuracy quantifies how much generation depends on retrieval. The retrievers compared are sparse lexical representations (BM25/TF-IDF over columns, rows, and titles), dense metadata embeddings (GPT-4o-mini-generated table summaries embedded with text-embedding-ada-002), dense table embeddings (three text embedding models over column names with or without sampled rows, formatted as markdown), and dense row-level embeddings (per-row sentences embedded with stella_en_400M_v5, then mapped to tables).

What would settle it

Manually audit a random sample of Spider and BIRD queries: compare the table sets that sqlglot extracts from the ground-truth SQL against human-annotated ground truth. If even a few percent of queries miss or include extra tables, the reported retrieval recall and downstream execution accuracy figures for text-to-SQL would shift measurably.

Watch

Extended reading notes

Core claim

On the paper's own terms, TARGET establishes that dense embedding-based retrievers far outperform a BM25 baseline for table retrieval across question answering, fact verification, and text-to-SQL, and that sparse lexical representations—so effective for free-form text—are markedly less robust for structured data. Across the five datasets, dense table embeddings (particularly stella_en_400M_v5 embeddings of column names plus sampled rows) give the best retrieval recall, while LLM-generated metadata summaries help when table titles are absent or uninformative. The paper further shows that the rank at which the correct table appears in the retrieval list strongly predicts downstream generation quality: the average Spearman's $\rho = -0.85$ between retriever performance and task performance indicates that relying on a long-context LLM to sort through many tables is a poor substitute for retrieving the right table early.

Load-bearing premise

The text-to-SQL conclusions rest on the assumption that sqlglot automatically extracts the complete and correct set of ground-truth tables for every Spider and BIRD query, and this extraction (Section 3.3) is not validated.

Editorial extensions

If this is right

Dense table embeddings, especially stella_en_400M_v5 embeddings of column names plus sampled rows, are the best current default for table retrieval, outperforming BM25 by large margins on datasets without descriptive titles.
Sparse lexical retrievers become much weaker when descriptive metadata such as table titles is unavailable; LLM-generated table summaries recover much of that gap.
Row-level dense embeddings can match or slightly beat table-level dense embeddings in recall, but their search cost becomes impractical on very large tables such as those in BIRD.
Because retrieval rank correlates with downstream accuracy (average Spearman's $\rho = -0.85$), feeding many tables into a long-context LLM is not a reliable substitute for getting the correct table near the top of the retrieval list.
Retrieval quality degrades as the corpus grows beyond roughly 10K added tables, so table retrievers need to be tested at scale, with lexical retrieval degrading fastest.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A testable extension: combining BM25-style exact matching with dense embeddings in a hybrid retriever should help on text-to-SQL queries, where the paper notes that queries resemble column names and including rows barely changes recall.
The $-0.85$ rank-accuracy correlation implies that investing in a reranker on top of the initial dense retriever may yield larger end-to-end gains than upgrading the generator, a claim TARGET does not itself test.
The benchmark's reliance on generated metadata summaries suggests a feedback loop worth probing: if the summary generator makes a factual error, retrieval may succeed on the summary while downstream generation fails on the true table contents; a targeted study of summary fidelity could quantify this failure mode.
Because row-level retrieval was not evaluated on BIRD due to scale, the open question remains whether hierarchical retrieval (database first, then table, then row) can make row-level methods practical for analytical text-to-SQL workloads.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

TARGET is worth taking seriously. It does a real service by packaging five table corpora under one API and measuring retrieval both in isolation and end-to-end across three tasks. The dense-versus-sparse finding is clean and intuitively right for tabular data, and the metadata sensitivity and corpus-scale experiments are solid additions. The no-context baseline is a sensible control, and the rank-accuracy correlation figure gives the community a practical signal about gold-table position.

But the text-to-SQL leg has a load-bearing reproducibility problem. The paper says it evaluates on the Spider test split, and the official Spider test split has no released gold SQL. Section 3.3 says ground-truth tables are extracted from the gold query via sqlglot; on the official test split that extraction has no input. The paper doesn't say where the SQL came from, doesn't report parse success rate, and doesn't include a manual audit. This may be an honest fix — maybe they used the dev split under a different name, or a third-party release with gold SQL — but as written, the reported CR@10 and execution accuracy for Spider cannot be reproduced. The same uncertainty doesn't hit BIRD, whose validation split is public, so the problem is specific to Spider.

Beyond that, the empirical base is thin in the usual way: single runs, no variance, no significance tests. For a benchmark paper that's not always disqualifying, but here the central claim about dense retrievers relies on comparisons across models, and confidence intervals would help. The paper also doesn't link the actual code and data at review time (only an anonymous URL), and the "strong negative correlation" statement in the abstract is a little loose — it's actually the rank of the gold table driving downstream accuracy, not retriever performance per se. Minor.

On the plus side, the authors are transparent about limitations: the metric issue for OTTQA, the lack of row/column labels, the omission of many known retrieval methods. No sign of circularity; the empirical claims are measurements against external datasets. I'd trust the dense-over-sparse story to hold after the Spider fix.

Who is this for? Anyone building or evaluating table retrievers for RAG, and benchmark designers in the structured-data space. It's a solid measurement instrument, not a theoretical contribution.

Recommendation: send it to peer review with the expectation of major revision. Require the authors to clarify the Spider split provenance, validate or replace the sqlglot extraction, and release the exact code and data. The benchmark idea is valuable enough to deserve referee time.

Referee Report

4 major / 6 minor

Summary. The paper introduces TARGET, a benchmark for evaluating table retrieval in retrieval-augmented generation (RAG) pipelines over structured data. It integrates five datasets spanning three downstream tasks: question answering (OTTQA, FeTaQA), fact verification (TabFact), and text-to-SQL (Spider, BIRD). The benchmark evaluates retrieval in isolation with recall/capped recall and time, and end-to-end with a GPT-4o-mini generator using task-specific metrics (SacreBleu, P/R/F1, execution accuracy). The authors compare sparse lexical retrievers (BM25, TF-IDF), dense metadata embeddings, dense table embeddings (three models), and dense row-level embeddings. Main findings are that dense embedding-based retrievers generally outperform sparse lexical baselines, metadata matters for lexical retrievers, retrieval performance degrades with corpus size, and downstream accuracy correlates with the rank of the gold table in retrieval results. The paper also includes a No Context baseline to measure memorization effects.

Significance. If the results hold, TARGET fills a clear gap by providing a unified, extensible benchmark for table retrieval across diverse tasks and data characteristics, with standard metrics and a modular API. The inclusion of the No Context baseline, capped recall for multi-table text-to-SQL, and the study of metadata sensitivity are thoughtful design choices. The finding that BM25 is less effective over tables than over text is a useful empirical contribution. The paper ships code and a project website, which supports reproducibility, though the issues below must be addressed before the benchmark can serve as a reliable reference standard.

major comments (4)

[Section 3.3, Table 2] The manuscript states that the Spider test split (2.1K queries) is used and that ground-truth tables are extracted from the ground-truth query via sqlglot. However, the official Spider test split does not include gold SQL queries, so the described extraction cannot be executed on the claimed split. The paper does not report the provenance of the SQL used for labeling, nor whether a different split (e.g., dev) was used under the name 'test'. This directly undermines the reproducibility of the Spider retrieval recall (Table 3) and execution accuracy (Table 5), and hence the text-to-SQL conclusions.
[Section 3.3] Even for BIRD, where gold SQL is publicly available, the paper does not validate the sqlglot-based extraction of ground-truth tables. No parse success rate, manual audit, or error analysis is reported, and the exact sqlglot version is not specified. If the extraction fails or is incomplete for any queries, the capped recall and downstream execution results are corrupted. Since the text-to-SQL relevance labels are a novel contribution of TARGET, this validation is essential for the benchmark's credibility.
[Abstract, Table 3] The claim that 'dense embedding-based retrievers far outperform a BM25 baseline' is too strong. In Table 3, BM25 with table title achieves R@10 of 0.967 on OTTQA, slightly above the best dense table embedding (0.963), and on Spider the gap is modest (CR@10 0.544 vs 0.657). The claim should be qualified as applying on average or for specific datasets, and the per-dataset exceptions should be discussed to avoid overgeneralization.
[Table 5, Section 4.2] All results are reported from single runs with no variance, error bars, or significance testing. For LLM-based generators, sampling temperature and decoding parameters are not reported, so downstream accuracy differences (e.g., FeTaQA SacreBleu 12.569 vs 13.231) may not be statistically reliable. As a benchmark intended for reference use, TARGET should provide multiple seeds or confidence intervals to support its comparisons.

minor comments (6)

[Section 3.4] There is a typo in the No Context baseline paragraph: 'soely' should be 'solely'.
[Section 2] In the 'Benchmarks and Datasets' paragraph, 'rerievers' should be 'retrievers'.
[Section 4.2] The phrase 'strong negative correlation between retriever performance and downstream task performance' is imprecise; Figure 5 plots the rank of the gold table in the retrieval results against downstream accuracy, so the correlation is between gold-table rank and accuracy, not between retriever performance and accuracy.
[Table 3 caption] The caption reads 'Best scores are inbold'; a space is missing between 'in' and 'bold'.
[Section 3.3] The sentence 'We use the test splits of included datasets for our evaluations' is slightly misleading because OTTQA and BIRD use validation splits; Table 2 is accurate, but the sentence should explicitly say 'test splits when available, otherwise validation splits'.
[Section 7] In the Limitations section, 'TARGETdoes' is missing a space; it should read 'TARGET does'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: TARGET is an empirical benchmark evaluation, not a derivation whose outputs are built from its own inputs.

full rationale

The paper's central claims are empirical measurements: that dense embedding-based retrievers outperform BM25 on the TARGET benchmark, that retrieval performance varies across datasets and metadata conditions, and that downstream generation accuracy declines as the gold table's rank increases. None of these claims is derived from a formula fitted to the reported outcome; the retrievers, generators, and metrics are all applied to external datasets (OTTQA, FeTaQA, TabFact, Spider, BIRD) and evaluated against ground-truth labels from those datasets. The only constructed labels are the text-to-SQL ground-truth tables extracted from gold queries with sqlglot, described in Section 3.3, and while the paper does not validate that extraction and the Spider test split provenance raises reproducibility concerns, that is a correctness/verifiability issue, not circularity: the extraction is not equivalent to the reported retrieval scores by construction. Self-citations, such as GitTables used only as auxiliary corpus material for the scale experiment, are not load-bearing for the main comparisons. The correlation analysis in Figure 5 is an observation about retrieved ranks and downstream scores, not a fitted parameter renamed as a prediction. Accordingly, the derivation chain is self-contained and the appropriate circularity score is 0.

Assumptions & free parameters 3 free parameters · 5 assumptions · 0 invented entities

The benchmark relies on several unverified assumptions about label quality and metric validity. The most consequential is the automatic sqlglot extraction of ground-truth tables for text-to-SQL, which directly feeds both retrieval and downstream evaluations. No free parameters are fit to data; the listed parameters are fixed evaluation choices.

free parameters (3)

top-k (k) = 10
Main retrieval results are reported at k=10. The choice is conventional but not varied in the main tables, and method rankings could in principle shift at different k.
row sample size for dense table embeddings = 100
Dense table embeddings concatenate column names plus the first 100 rows. The cutoff is arbitrary and no sensitivity analysis is provided.
rows used for LLM metadata summaries = 10
Dense Metadata Embedding generates table summaries from column names and the first 10 rows; the dependence of retrieval quality on this number is not tested.

assumptions (5)

domain assumption Ground-truth table annotations in source datasets are complete and correct.
Retrieval recall is computed against these labels; the paper does not re-verify them.
domain assumption sqlglot parsing of gold SQL queries yields the complete set of relevant tables.
Used in Section 3.3 to create ground-truth labels for Spider and BIRD; no error analysis is reported.
domain assumption Recall@k and capped recall are adequate metrics for table retrieval quality.
The benchmark uses only recall-based measures; precision, MRR, and nDCG are not considered, which could change rankings for practical use.
domain assumption SacreBleu, execution accuracy, and P/R/F1 are appropriate downstream metrics.
The paper itself notes SacreBleu sensitivity to answer length and that BERTScore gave no signal, so downstream conclusions hinge on these choices.
domain assumption GPT-4o-mini is a representative generator for all tasks.
Downstream results reflect a single generator; other LLMs could rank retrievers differently.

how reviews work

0 comments

Cite this review

Pith. "Pith review of TARGET: Benchmarking Table Retrieval for Generative Tasks." pith.science (2026). https://pith.science/paper/SXIOVZFJ

@misc{pith2026250511545,
  author       = {Pith},
  title        = {Pith review of: TARGET: Benchmarking Table Retrieval for Generative Tasks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/SXIOVZFJ}},
  note         = {Machine review of arXiv:2505.11545}
}

read the original abstract

The data landscape is rich with structured data, often of high value to organizations, driving important applications in data analysis and machine learning. Recent progress in representation learning and generative models for such data has led to the development of natural language interfaces to structured data, including those leveraging text-to-SQL. Contextualizing interactions, either through conversational interfaces or agentic components, in structured data through retrieval-augmented generation can provide substantial benefits in the form of freshness, accuracy, and comprehensiveness of answers. The key question is: how do we retrieve the right table(s) for the analytical query or task at hand? To this end, we introduce TARGET: a benchmark for evaluating TAble Retrieval for GEnerative Tasks. With TARGET we analyze the retrieval performance of different retrievers in isolation, as well as their impact on downstream tasks. We find that dense embedding-based retrievers far outperform a BM25 baseline which is less effective than it is for retrieval over unstructured text. We also surface the sensitivity of retrievers across various metadata (e.g., missing table titles), and demonstrate a stark variation of retrieval performance across datasets and tasks. TARGET is available at https://target-benchmark.github.io.

Figures

Figures reproduced from arXiv: 2505.11545 by the authors.

**Figure 2.** Overview of the TARGET benchmark for evaluating table retrieval methods and downstream generation for various datasets across three downstream tasks: tabular question answering, fact verification, and Text-to-SQL. Benchmarks and Datasets To develop stronger rerievers and advance research on LLM-driven tasks on structured data, benchmarks and datasets are essential. The MTEB and CRAG benchmarks (Muennighoff et al., … view at source ↗

**Figure 3.** Influence of k on retrieval performance with various baselines on the FeTaQA dataset, confirming the expectation that performance gradually increases with k, most significantly for dense embedding approaches. 0 20000 40000 # extra tables 0.2 0.4 0.6 recall dense table embedding sparse lexical representation (tf-idf) [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (3 more)

**Figure 4.** Figure 4: Influence of corpus size on retrieval, illus [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]

**Figure 5.** Figure 5: As the rank of the ground-truth table increases [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

**Figure 6.** Figure 6: Comparison of column and row count distri [PITH_FULL_IMAGE:figures/full_fig_p011_6.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

45 extracted references · 26 canonical work pages

[1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...
[2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
[3]

Sonia Castelo, R \'e mi Rampin, A \'e cio Santos, Aline Bessa, Fernando Chirigati, and Juliana Freire. 2021. Auctus: A dataset search engine for data discovery and augmentation. Proceedings of the VLDB Endowment, 14(12):2791--2794

work page 2021
[4]

Si-An Chen, Lesly Miculicich, Julian Martin Eisenschlos, Zifeng Wang, Zilong Wang, Yanfei Chen, Yasuhisa Fujii, Hsuan-Tien Lin, Chen-Yu Lee, and Tomas Pfister. 2024. TableRAG : Million-token table understanding with language models. Thirty-eighth Conference on Neural Information Processing Systems Datasets and Benchmarks Track

work page 2024
[5]

Wenhu Chen, Ming-Wei Chang, Eva Schlinger, William Yang Wang, and William W Cohen. 2021. Open question answering over tables and text. In International Conference on Learning Representations

work page 2021
[6]

Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. 2020. Tab F act: A large-scale dataset for table-based fact verification. International Conference of Learning Representations

work page 2020
[7]

Tianji Cong, Madelon Hulsebos, Zhenjie Sun, Paul Groth, and H. V. Jagadish. 2023. Observatory: Characterizing embeddings of relational tables. Proceedings of VLDB, 17(4)

work page 2023
[8]

Xiang Deng, Huan Sun, Alyssa Lees, You Wu, and Cong Yu. 2022. Turl: Table understanding through representation learning. ACM SIGMOD Record, 51(1)

work page 2022

Show all 45 references

[9]

Dawei Gao, Haibin Wang, Yaliang Li, Xiuyu Sun, and et al. 2024. Text-to-sql empowered by large language models: A benchmark evaluation. Proceedings of VLDB

2024
[10]

Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, and Haofen Wang. 2023. Retrieval-augmented generation for large language models: A survey. arXiv preprint arXiv:2312.10997

2023 arXiv
[11]

Ramanathan V Guha, Prashanth Radhakrishnan, Bo Xu, Wei Sun, Carolyn Au, Ajai Tirumali, Muhammad J Amjad, Samantha Piekos, Natalie Diaz, Jennifer Chen, et al. 2023. Data commons. arXiv preprint arXiv:2309.13054

2023 arXiv
[12]

Alon Halevy, Flip Korn, Natalya F Noy, Christopher Olston, Neoklis Polyzotis, Sudip Roy, and Steven Euijong Whang. 2016. Goods: Organizing google's datasets. In Proceedings of the 2016 International Conference on Management of Data, pages 795--806

2016
[13]

Jonathan Herzig, Thomas Mueller, Syrine Krichene, and Julian Eisenschlos. 2021. Open domain question answering over tables via dense retrieval. In Proceedings of NAACL

2021
[14]

Jonathan Herzig, Pawel Krzysztof Nowak, Thomas M \"u ller, Francesco Piccinno, and Julian Eisenschlos. 2020. Tapas: Weakly supervised table parsing via pre-training. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Association for Com...

2020
[15]

Madelon Hulsebos, C agatay Demiralp, and Paul Groth. 2023. Gittables: A large-scale corpus of relational tables. Proceedings of the ACM on Management of Data, 1(1)

2023
[16]

Madelon Hulsebos, Kevin Hu, Michiel Bakker, Emanuel Zgraggen, Arvind Satyanarayan, Tim Kraska, C agatay Demiralp, and C \'e sar Hidalgo. 2019. Sherlock: A deep learning approach to semantic data type detection. In Proceedings of the 25th ACM SIGKDD

2019
[17]

Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

2024 arXiv
[18]

Vishwajeet Kumar, Yash Gupta, Saneem Ahmed Chemmengath, Jaydeep Sen, Soumen Chakrabarti, Samarth Bharadwaj, and Feifei Pan. 2023. Multi-row, multi-span distant supervision for table+ text question answering. In The 61st Annual Meeting Of The Association For Computational Linguistics

2023
[19]

u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \

Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich K \"u ttler, Mike Lewis, Wen-tau Yih, Tim Rockt \"a schel, et al. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in Neural Information Proc...

2020
[20]

Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, et al. 2024 a . Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls. Advances in Neural Information Processing ...

2024
[21]

Zhuowan Li, Cheng Li, Mingyang Zhang, Qiaozhu Mei, and Michael Bendersky. 2024 b . Retrieval augmented generation or long-context llms? a comprehensive study and hybrid approach. arXiv preprint arXiv:2407.16833

2024 arXiv
[22]

Jerry Liu. 2022. https://doi.org/10.5281/zenodo.1234 LlamaIndex

2022 doi
[23]

Yu A Malkov and Dmitry A Yashunin. 2018. Efficient and robust approximate nearest neighbor search using hierarchical navigable small world graphs. IEEE transactions on pattern analysis and machine intelligence, 42(4)

2018
[24]

Alex Troy Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In The 61st Annual Meeting Of The Association For Computational L...

2023
[25]

Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. Mteb: Massive text embedding benchmark. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2014--2037

2023
[26]

Linyong Nan, Chiachun Hsieh, Ziming Mao, Xi Victoria Lin, Neha Verma, Rui Zhang, Wojciech Kry \'s ci \'n ski, Hailey Schoelkopf, Riley Kong, Xiangru Tang, et al. 2022. Fetaqa: Free-form table question answering. Transactions of the Association for Computational Linguistics, 10

2022
[27]

Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, et al. 2021. Show your work: Scratchpads for intermediate computation with language models. arXiv preprint arXiv:2112.00114

2021 arXiv
[28]

OpenAI . 2024. text-embedding-3-small. https://platform.openai.com/docs/guides/embeddings. Embedding Model

2024
[29]

Matt Post. 2018. https://www.aclweb.org/anthology/W18-6319 A call for clarity in reporting BLEU scores . In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 186--191, Belgium, Brussels. Association for Computational Linguistics

2018
[30]

https://qdrant.tech/ Qdrant: Vector database for the next generation of ai

Qdrant . https://qdrant.tech/ Qdrant: Vector database for the next generation of ai . Vector database engine
[31]

Prashanth Radhakrishnan, Jennifer Chen, Bo Xu, Prem Ramaswami, Hannah Pho, Adriana Olmos, James Manyika, and RV Guha. 2024. https://docs.datacommons.org/ Knowing when to ask-bridging large language models and data . Data Commons

2024
[32]

Ananya Singha, Jos \'e Cambronero, Sumit Gulwani, Vu Le, and Chris Parnin. 2023. Tabular representation, noisy operators, and impacts on table structure understanding tasks in llms. Table Representation Learning workshop at NeurIPS

2023
[33]

Shayan Talaei, Mohammadreza Pourreza, Yu-Chen Chang, Azalia Mirhoseini, and Amin Saberi. 2024. Chess: Contextual harnessing for efficient sql synthesis. arXiv preprint arXiv:2405.16755

2024 arXiv
[34]

Nandan Thakur, Nils Reimers, Andreas R \"u ckl \'e , Abhishek Srivastava, and Iryna Gurevych. 2021. https://openreview.net/forum?id=wCu6T5xFjeJ BEIR : A heterogeneous benchmark for zero-shot evaluation of information retrieval models . In Thirty-fifth Conference on Neural Info...

2021
[35]

Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. https://arxiv.org/abs/2402.05672 Multilingual e5 text embeddings: A technical report . Preprint, arXiv:2402.05672

2024 arXiv
[36]

Qiming Wang and Raul Castro Fernandez. 2023. Solo: Data discovery using natural language questions via a self-supervised approach. Proceedings of the ACM on Management of Data, 1(4):1--27

2023
[37]

Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824--24837

2022
[38]

Xiao Yang, Kai Sun, Hao Xin, Yushi Sun, Nikita Bhalla, Xiangsen Chen, Sajal Choudhary, Rongze Daniel Gui, Ziran Will Jiang, Ziyu Jiang, et al. 2024. Crag--comprehensive rag benchmark. arXiv preprint arXiv:2406.04744

2024 arXiv
[39]

Tan Yu, Anbang Xu, and Rama Akkiraju. 2024. In defense of rag in the era of long-context language models. arXiv preprint arXiv:2409.01666

2024 arXiv
[40]

Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, et al. 2018. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. arXiv preprint arXiv:1809.08887

2018 arXiv
[41]

Dun Zhang, Jiacheng Li, Ziyang Zeng, and Fulong Wang. 2025. https://arxiv.org/abs/2412.19048 Jasper and stella: distillation of sota embedding models . Preprint, arXiv:2412.19048

2025 arXiv
[42]

Hongzhi Zhang, Yingyao Wang, Sirui Wang, Xuezhi Cao, Fuzheng Zhang, and Zhongyuan Wang. 2020. Table fact verification with structure-aware transformer. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1624--1629

2020
[43]

Tianping Zhang, Shaowen Wang, Shuicheng Yan, Li Jian, and Qian Liu. 2023. Generative table pre-training empowers models for tabular prediction. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 14836--14854

2023
[44]

Weinberger, and Yoav Artzi

Tianyi Zhang*, Varsha Kishore*, Felix Wu*, Kilian Q. Weinberger, and Yoav Artzi. 2020. https://openreview.net/forum?id=SkeHuCVFDr Bertscore: Evaluating text generation with bert . In International Conference on Learning Representations

2020
[45]

Alex Zhuang, Ge Zhang, Tianyu Zheng, Xinrun Du, Junjie Wang, Weiming Ren, Stephen W Huang, Jie Fu, Xiang Yue, and Wenhu Chen. 2024. Structlm: Towards building generalist models for structured knowledge grounding. Conference on Language Modeling

2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

online" 'onlinestring :=

ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

[2] [2]

write newline

" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

[3] [3]

Sonia Castelo, R \'e mi Rampin, A \'e cio Santos, Aline Bessa, Fernando Chirigati, and Juliana Freire. 2021. Auctus: A dataset search engine for data discovery and augmentation. Proceedings of the VLDB Endowment, 14(12):2791--2794

work page 2021

[4] [4]

Si-An Chen, Lesly Miculicich, Julian Martin Eisenschlos, Zifeng Wang, Zilong Wang, Yanfei Chen, Yasuhisa Fujii, Hsuan-Tien Lin, Chen-Yu Lee, and Tomas Pfister. 2024. TableRAG : Million-token table understanding with language models. Thirty-eighth Conference on Neural Information Processing Systems Datasets and Benchmarks Track

work page 2024

[5] [5]

Wenhu Chen, Ming-Wei Chang, Eva Schlinger, William Yang Wang, and William W Cohen. 2021. Open question answering over tables and text. In International Conference on Learning Representations

work page 2021

[6] [6]

Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. 2020. Tab F act: A large-scale dataset for table-based fact verification. International Conference of Learning Representations

work page 2020

[7] [7]

Tianji Cong, Madelon Hulsebos, Zhenjie Sun, Paul Groth, and H. V. Jagadish. 2023. Observatory: Characterizing embeddings of relational tables. Proceedings of VLDB, 17(4)

work page 2023

[8] [8]

Xiang Deng, Huan Sun, Alyssa Lees, You Wu, and Cong Yu. 2022. Turl: Table understanding through representation learning. ACM SIGMOD Record, 51(1)

work page 2022