Pith. sign in

REVIEW 5 major objections 6 minor 24 references

DBRouting: Routing End User Queries to Databases for Answerability

T0 review · 5 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper establishes a new benchmark task, routing natural-language questions to the database that can answer them, and shows current LLM and embedding baselines degrade sharply as the number of candidate databases grows.

desk verdict A useful first cut at a real enterprise problem, with a load-bearing label-design flaw that the authors nearly admit in their own error analysis. read the letter →

arxiv 2501.16220 v2 pith:BSWXEBIQ submitted 2025-01-27 cs.CL

Priyangshu Mandal , Manasi Patwardhan , Mayur Patidar , Lovekesh Vig This is my paper · ORCID
classification cs.CL
keywords databaseroutingnaturallanguagequeryingenterprisesearchquerytext-to-SQLsentenceembeddingslargemodelsbenchmarkconstruction
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper defines a new task: given a natural-language question and a repository of enterprise databases, rank the databases by whether they can answer the question. It builds two benchmark sets, Spider-Route and Bird-Route, by recycling the questions and schemas of existing text-to-SQL datasets and treating the database a question was originally posed on as the correct answer. Using these benchmarks, the paper shows that an open-source 70B language model routes more accurately than embedding-based similarity but cannot scale to large repositories because of context-length limits, and that a small embedding model fine-tuned with contrastive pairs recovers much of the gap. The paper also shows the task grows predictably harder with more candidate databases, with databases from similar domains, and with questions that are ambiguous or need domain knowledge. A sympathetic reading is that the routing task itself is real and measurable, and none of the current baselines solve it at realistic enterprise scale.

What carries the argument

The mechanism that carries the argument is a ranking pipeline over database schemas expressed as Data Definition Language (DDL) strings. The scoring function $f(q_i, S_j)$ is implemented in three ways: zero-shot instruction-tuned Llama3 ranking databases in a prompt; cosine similarity between pre-trained Sentence-BERT embeddings of the question and the schema string; and Sentence-BERT models fine-tuned with a contrastive margin loss on synthesized positive and negative question-schema pairs. For Bird-Route, whose schemas exceed the embedding context, the pipeline becomes two-stage: retrieve relevant domain-statement evidence from each database's metadata, then retrieve relevant tables, then pool table scores into a database score. The DDL-plus-metadata text is the object the scoring functions operate on, so the paper's conclusions are tied to how faithfully that text represents a database.

What would settle it

Take the questions that at least one method routes incorrectly and check answerability directly: execute the ground-truth SQL on each plausible candidate database, or have annotators say which databases can answer the question. If a large share of the errors are questions answerable by two or more databases, as the paper's own 'Find the number of albums' example suggests, then the reported Recall@1 understates routing quality and the benchmark labels need revision.

Watch

Extended reading notes

Core claim

The central claim is that DB routing is a distinct, learnable task and that current retrieval and ranking baselines are not sufficient. The paper constructs Spider-Route (140 train databases, 20 cross-domain test databases) and Bird-Route (69 train, 11 cross-domain) by converting Spider and Bird-SQL schemas to DDL and using each question's original database as ground truth. Empirically, Llama3 70B achieves 95.45% Recall@1 on Spider-Route cross-domain where all schemas fit in context, versus 87.71% for pre-trained embeddings and 91.78% for fine-tuned embeddings; on the 140-database in-domain split Llama3 drops to 59.84%, while fine-tuned embeddings reach 55.04%. The same split shows within-vertical Recall@1 much lower than across-vertical Recall@1, meaning mistakes mostly confuse databases from similar domains. The authors conclude that routing difficulty scales with source count and domain overlap, benefits from domain metadata, and is degraded by ambiguous queries.

Load-bearing premise

Every benchmark question is assumed to be answerable by exactly one database, and that database is inherited from the original text-to-SQL dataset rather than verified by actually answering the question; if many questions are naturally answerable by several databases, the accuracy numbers are not true routing errors.

Editorial extensions

If this is right

  • A benchmark with 160 databases across many domains now exists for query-to-database routing, built from two standard text-to-SQL datasets.
  • Routing accuracy falls as the number of candidate databases grows: on the Spider-Route test split, task-specific embeddings drop from 91.78% Recall@1 with 20 databases to 60.38% with 160 databases.
  • Databases that look similar confuse all tested routers; within-vertical Recall@1 is consistently lower than across-vertical Recall@1, so domain overlap is a primary difficulty.
  • Providing database-specific domain knowledge improves routing: on Bird-Route cross-domain, pre-trained embeddings' Recall@1 rises from 91.00% to 97.71% when retrieved domain statements are added.
  • Because Llama3's 8K context cannot hold even a few Bird-Route schemas, context-length limits, not reasoning alone, currently block direct LLM routing on realistic enterprise repositories.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's single-ground-truth labels are inherited from text-to-SQL provenance, so a stronger evaluation would annotate multi-database answerability and include unanswerable queries; until then, reported recall may understate or overstate true routing quality depending on how often the label is ambiguous.
  • The two-stage retrieve-evidence-then-tables recipe for Bird-Route suggests a scalable enterprise architecture: cheap embedding prefiltering to shrink the candidate set, followed by an LLM reranker that sees only short schemas; the paper stops short of testing this combination.
  • The same contrastive fine-tuning approach could transfer to heterogeneous sources such as knowledge graphs and document stores if each source is represented by a comparable schema-like metadata string, which the paper lists as future work.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper defines a new task, database routing: given a natural-language query and a set of databases, rank the databases by whether they can answer the query. The authors construct two benchmarks, Spider-Route and Bird-Route, by converting the Spider and BIRD-SQL NL-to-SQL datasets into routing test sets, taking the database on which each question was originally posed as the ground-truth database. They evaluate three families of approaches: zero-shot Llama3-70B prompting, pre-trained SBERT cosine-similarity retrieval, and task-specific SBERT models fine-tuned with contrastive pairs. The experiments address four research questions: whether in-domain versus cross-domain training helps, whether domain similarity hurts, whether increasing the number of databases hurts, and whether external metadata helps. The main reported findings are that Llama3 outperforms embedding-based approaches when it can be applied, task-specific fine-tuning helps especially in-domain, closer domains make routing harder, more databases make routing harder, and metadata improves Bird-Route performance.

Significance. If the benchmark and its label assumptions were sound, this would be a useful first benchmark for an enterprise-relevant problem that is distinct from document retrieval and API/tool routing. The paper names a genuinely under-studied setting, and the research-question structure is sensible. The error analysis in Section 7.5 is informative, and the comparison of pre-trained versus fine-tuned embeddings across in-domain and cross-domain settings is a useful empirical contribution. The benchmark construction from existing NL-to-SQL datasets is also a practical way to bootstrap training data. However, the central issue is that the ground-truth labels are provenance-based rather than answerability-based, which conflicts with the paper's own task definition and with examples in its error analysis. The metric mislabeling and the hybrid nature of the Llama3 baseline further weaken the quantitative conclusions as currently stated. The benchmark could become a valuable resource after relabeling and clarification.

major comments (5)
  1. [Sections 2, 3.1, and 7.5.1] The ground-truth labels do not implement the task definition as stated. Section 2 defines relevance as answerability and assumes every question is answerable by exactly one database, and Section 3.1 sets the ground truth to the database on which the question was posed in the original NL-to-SQL dataset. Section 7.5.1 then documents examples such as 'Find the number of albums.' being answerable by multiple databases (music_2 and chinook_1) and 'Count the number of tracks.' admitting multiple readings. The filter described in Section 3.1 applies only to the in-domain split and ensures that questions shared across databases in the original training set are not in that split; no equivalent check is applied to the cross-domain split or to Bird-Route. Consequently, the labels are a set of sufficient provenance assignments, not the set of all answerable databases, and a model that returns any genuinely answerable database is scored as wrong. This affects every headline number and the RQ2/RQ3 conclusions, because adding more or closer databases creates more plausible-but-marked-wrong alternatives. A concrete fix is to re-annotate a sample of test questions (or verify answerability by executing candidate SQL) and report an any-answerable-DB variant of R@1/R@3 alongside the provenance-based metric.
  2. [Section 4.1, Tables 3 and 5, Abstract and Conclusion] The Llama3 baseline is not a pure language-model ranker for the in-domain splits. Section 4.1 states that when the databases do not fit into the 8K context, the authors first retrieve the top-10 databases for Spider-Route using pre-trained SBERT and top-3 tables, and then ask Llama3 to re-rank those candidates; for Bird-Route the approach is not used at all. The Llama3 rows in Table 3 therefore describe a hybrid retriever-plus-LLM pipeline whose R@1 is upper-bounded by the retriever's recall@10. The abstract and conclusion claim that 'open-source LLMs perform better than embedding based approach' without this caveat. The paper should report the candidate-retriever recall, ablate the LLM re-ranker against the retriever alone, and qualify the conclusion to the hybrid setting.
  3. [Section 5.1, Tables 2-8] The metric called mAP is not mean average precision as that term is standardly used. The text defines mAP for a question as 1/i where i is the rank of the correct database; averaged over questions this is mean reciprocal rank (MRR). Mean average precision for a ranking task would average precision over the ranked list of databases, not the reciprocal rank of a single relevant item. Since every result table reports this quantity, the numbers cannot be compared with mAP values elsewhere in the literature. Please rename the metric to MRR or provide the correct mAP computation.
  4. [Section 3.1] The negative-pair count in the contrastive training data is inconsistent with the described procedure. With 5,959 training questions and 140 databases, pairing each question with 'all DBs except the one it originally mapped to' yields 5,959 × 139 = 828,301 pairs, not the stated 19,460. If negatives were sampled, restricted to vertical clusters, or otherwise filtered, that procedure must be described. As written, the reported number contradicts the text and prevents reproduction of the task-specific embedding models.
  5. [Section 5.1 and RQ2, Tables 2, 3, and 5] The within-vertical and across-vertical definitions appear inverted and are inconsistent with the table values. The text says that when the top-ranked database and the ground-truth database are in the same cluster, across-vertical Recall@1 is 1 and within-vertical Recall@1 is 0, and the reverse when the clusters differ. Under that reading, the two metrics are complementary for each question, yet Table 2 shows W-V R1 = 96.71 and A-V R1 = 98.64, which cannot both hold if they are complements. The names also conflict with the RQ2 discussion, where same-cluster confusions are described as confusions 'within domain'. Please define each metric over a precise question subset and recompute or relabel the columns before the RQ2 conclusion can be supported.
minor comments (6)
  1. [Appendix 7.2] The sentence 'Here in Table ??, we provide the details of the prompt...' has a broken cross-reference; it should point to Table 10.
  2. [Section 4.3] The contrastive-loss margin m is never given a value; please report the chosen margin for the reported experiments.
  3. [Section 3.1] The filtering step that keeps shared questions in the training set is described only for the in-domain test split; the same check should be documented for the cross-domain split or its absence justified.
  4. [Abstract and general exposition] There are several typos: 'set-of data-sources' in the abstract, 'with a the model' in Section 2, and 'constrastive' in Section 4.3.
  5. [Appendix 7.3, Table 12] The vertical-cluster listings have formatting issues, including trailing commas and entries split across lines without clear delimiters, which makes the clusters harder to parse.
  6. [Section 4.1 and References] Section 4.1 refers to an instruction-tuned Llama3 70B model, but reference [16] is the original LLaMA paper; the specific model card or the Llama-3-70B-Instruct release should be cited.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper is an empirical benchmark study with held-out evaluation; provenance-based labels are a validity concern, not a derivation loop.

full rationale

The paper makes no formal derivation or uniqueness claim that could collapse into its own inputs. It synthesizes the Spider-Route and Bird-Route benchmarks from existing NL-to-SQL datasets, defines the ground-truth database for each question as the database on which the question was originally posed (Section 3.1), and evaluates pre-trained embeddings, task-specific fine-tuned embeddings, and Llama3 on disjoint in-domain and cross-domain test splits. The fine-tuned models are trained only on the training split and tested on held-out questions, so conclusions such as 'task-specific embeddings benefit from fine-tuning' are ordinary supervised-learning findings, not fitted parameters renamed as predictions. The assumption that every question maps to exactly one answerable database (Section 2) and the provenance-based labeling in Section 3.1 could reduce construct validity, and the paper's own error analysis (Section 7.5.1) admits that questions such as 'Find the number of albums.' can be answered by multiple databases; the paper also lists removal of ambiguous queries and inclusion of multi-answerable/unanswerable queries as future work (Section 6). However, this is a benchmark-labeling limitation, not circularity: the reported R@1/mAP values do not reduce by construction to a fitted parameter, a self-citation chain, or an equation that equals its own input. There are no load-bearing self-citations; the cited prior datasets (Spider, BIRD-SQL) are external resources. Accordingly, the derivation chain is self-contained and the appropriate circularity score is 0.

Assumptions & free parameters 4 free parameters · 6 assumptions · 0 invented entities

The paper introduces no new physical or theoretical entities. Its main assumptions are about the benchmark labels, the sufficiency of schema text, the validity of metadata unions, the manual domain clusters, and the appropriateness of the ranking metric, all of which are domain assumptions of an empirical benchmark paper.

free parameters (4)
  • contrastive margin m = not reported
    Section 4.3 defines the contrastive loss with a margin m but never states its value; all fine-tuned embedding results depend on it.
  • top-K tables = 3
    Tables retrieved per database for scoring and for Llama3 re-ranking inputs in Sections 4.1 and 4.2; chosen without ablation.
  • Llama3 re-ranking candidate pool = 10
    For in-domain Spider-Route, Llama3 re-ranks only the top-10 databases from the embedding retriever (Section 4.1), capping its achievable recall.
  • SBERT fine-tuning hyperparameters = 2 epochs, lr 5e-6, batch 16
    Reported in Section 4.3 with no tuning or sensitivity analysis.
assumptions (6)
  • domain assumption The original Spider or BIRD database label is the unique, correct answerability target for each question
    Section 2 states every question has a mapping to ONLY ONE database; Section 3.1 states the DB on which the question is posed is assumed to be the ground truth DB.
  • domain assumption Database name plus DDL schema is sufficient textual evidence to rank answerability
    All three approaches embed or prompt over the schema text (Section 4); if schema alone is not enough, the benchmark understates or distorts difficulty.
  • domain assumption The union of BIRD question-specific evidence statements forms valid database-level metadata
    Section 3.2 treats the union of all question-specific evidences as database metadata, assuming these statements collectively capture the domain knowledge needed to answer any question.
  • domain assumption Manually assigned vertical clusters capture domain similarity
    Sections 3.1 and 7.3; the within and across-vertical analysis and RQ2 depend on these hand-built clusters.
  • ad hoc to paper The score 1/i is an appropriate ranking error metric
    Section 5.1 defines mAP as the reciprocal of the rank of the correct database, which is mean reciprocal rank under a different name, and no justification is given for using it as mAP.
  • domain assumption Instruction-tuned Llama3 70B will follow the database-ranking prompt
    Section 4.1 uses a zero-shot prompt asking for the top-3 database names; no parsing or validity checks on the generated names are reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of DBRouting: Routing End User Queries to Databases for Answerability." pith.science (2026). https://pith.science/paper/BSWXEBIQ

@misc{pith2026250116220,
  author       = {Pith},
  title        = {Pith review of: DBRouting: Routing End User Queries to Databases for Answerability},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BSWXEBIQ}},
  note         = {Machine review of arXiv:2501.16220}
}
read the original abstract

Enterprise level data is often distributed across multiple sources and identifying the correct set-of data-sources with relevant information for a knowledge request is a fundamental challenge. In this work, we define the novel task of routing an end-user query to the appropriate data-source, where the data-sources are databases. We synthesize datasets by extending existing datasets designed for NL-to-SQL semantic parsing. We create baselines on these datasets by using open-source LLMs, using both pre-trained and task specific embeddings fine-tuned using the training data. With these baselines we demonstrate that open-source LLMs perform better than embedding based approach, but suffer from token length limitations. Embedding based approaches benefit from task specific fine-tuning, more so when there is availability of data in terms of database specific questions for training. We further find that the task becomes more difficult (i) with an increase in the number of data-sources, (ii) having data-sources closer in terms of their domains,(iii) having databases without external domain knowledge required to interpret its entities and (iv) with ambiguous and complex queries requiring more fine-grained understanding of the data-sources or logical reasoning for routing to an appropriate source. This calls for the need for developing more sophisticated solutions to better address the task.

Figures

Figures reproduced from arXiv: 2501.16220 by the authors.

Figure 1
Figure 1. Database (DB) Routing Task Abstract Enterprise level data is often distributed across multiple sources and identifying the correct set-of data-sources with relevant infor￾mation for a knowledge request is a fundamental challenge. In this work, we define the novel task of routing an end-user query to the appropriate data-source, where the data-sources are databases. We synthesize datasets by extending existing datase… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

24 extracted references · 9 canonical work pages

  1. [1]

    Philipp Christmann, Rishiraj Saha Roy, and Gerhard Weikum. 2023. CompMix: A Benchmark for Heterogeneous Question Answering. Companion Proceedings of the ACM on Web Conference 2024 (2023). https://api.semanticscholar.org/ CorpusID:259212487

    work page 2023

  2. [2]

    Philipp Christmann, Rishiraj Saha Roy, and Gerhard Weikum. 2023. Explain- able Conversational Question Answering over Heterogeneous Sources via It- erative Graph Neural Networks. Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval (2023). https://api.semanticscholar.org/CorpusID:258436694

    work page 2023

  3. [3]

    Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Vic- tor Rühle, Laks V. S. Lakshmanan, and Ahmed Hassan Awadallah. 2024. Hybrid LLM: Cost-Efficient and Quality-Aware Query Routing. ArXiv abs/2404.14618 (2024). https://api.semanticscholar.org/CorpusID:269303119

    arXiv 2024

  4. [4]

    Shibo Hao, Tianyang Liu, Zhen Wang, and Zhiting Hu. 2023. ToolkenGPT: Augmenting Frozen Language Models with Massive Tools via Tool Embed- dings. ArXiv abs/2305.11554 (2023). https://api.semanticscholar.org/CorpusID: 258823133

    arXiv 2023

  5. [5]

    Yoonsang Lee, Xi Ye, and Eunsol Choi. 2024. AmbigDocs: Reasoning across Documents on Different Entities under the Same Name. ArXiv abs/2404.12447 (2024). https://api.semanticscholar.org/CorpusID:269282807

    arXiv 2024

  6. [6]

    Jens Lehmann, Dhananjay Bhandiwad, Preetam Gattogi, and Sahar Vahdati

    work page

  7. [7]

    Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, et al . 2024. Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls. Advances in Neural Information Processing Systems 36 (2024)

    2024

  8. [8]

    Zhuowan Li, Cheng Li, Mingyang Zhang, Qiaozhu Mei, and Michael Bendersky

Show all 24 references
  1. [9]

    Gonzalez, Mohammed Waleed Kadous, and Ion Stoica

    Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chiang, Tianhao Wu, Joseph E. Gonzalez, Mohammed Waleed Kadous, and Ion Stoica. 2024. RouteLLM: Learning to Route LLMs with Preference Data. ArXiv abs/2406.18665 (2024). https: //api.semanticscholar.org/CorpusID:270764307

    2024 arXiv

  2. [10]

    https://api.semanticscholar.org/CorpusID: 271404721

    Retrieval Augmented Generation or Long-Context LLMs? A Comprehen- sive Study and Hybrid Approach. https://api.semanticscholar.org/CorpusID: 271404721

  3. [11]

    Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun

    Yujia Qin, Shi Liang, Yining Ye, Kunlun Zhu, Lan Yan, Ya-Ting Lu, Yankai Lin, Xin Cong, Xiangru Tang, Bill Qian, Sihan Zhao, Runchu Tian, Ruobing Xie, Jie Zhou, Marc H. Gerstein, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2023. ToolLLM: Facilitating Large Language Models to Maste...

    2023 arXiv

  4. [12]

    Patil, Tianjun Zhang, Xin Wang, and Joseph E

    Shishir G. Patil, Tianjun Zhang, Xin Wang, and Joseph E. Gonzalez. 2023. Gorilla: Large Language Model Connected with Massive APIs. ArXiv abs/2305.15334 (2023). https://api.semanticscholar.org/CorpusID:258865184

    2023 arXiv

  5. [13]

    Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language Models Can Teach Themselves to Use Tools. ArXiv abs/2302.04761 (2023). https://api.semanticscholar.org/CorpusID:256697342

    2023 arXiv

  6. [14]

    Nils Reimers and Iryna Gurevych. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084 (2019)

    2019 arXiv

  7. [15]

    Haitian Sun, Bhuwan Dhingra, Manzil Zaheer, Kathryn Mazaitis, Ruslan Salakhut- dinov, and William W. Cohen. 2018. Open Domain Question Answering Us- ing Early Fusion of Knowledge Bases and Text. ArXiv abs/1809.00782 (2018). https://api.semanticscholar.org/CorpusID:52154304

    2018 arXiv

  8. [16]

    Cohen, and Donald Metzler

    Tal Schuster, Ádám Dániel Lelkes, Haitian Sun, Jai Gupta, Jonathan Berant, William W. Cohen, and Donald Metzler. 2023. SEMQA: Semi-Extractive Multi-Source Question Answering. ArXiv abs/2311.04886 (2023). https: //api.semanticscholar.org/CorpusID:265050837

    2023 arXiv

  9. [17]

    Rossi, Alexa F

    Yu Wang, Nedim Lipka, Ryan A. Rossi, Alexa F. Siu, Ruiyi Zhang, and Tyler Derr

  10. [18]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lam- ple. 2023. LLaMA: Open and Efficient Foundation ...

    2023 arXiv

  11. [19]

    Zhang, Sina J

    Heidi C. Zhang, Sina J. Semnani, Farhad Ghassemi, Jialiang Xu, Shicheng Liu, and Monica S. Lam. 2024. SPAGHETTI: Open-Domain Question Answering from Heterogeneous Data Sources with Retrieval and Semantic Parsing. ArXiv abs/2406.00562 (2024). https://api.semanticscholar.org/Cor...

    2024 arXiv

  12. [20]

    Minhao Zhang, Yongliang Ma, Yanzeng Li, Ruoyu Zhang, Lei Zou, and Ming Zhou. 2023. Two is Better Than One: Answering Complex Questions by Multiple Knowledge Sources with Generalized Links. ArXiv abs/2309.05201 (2023). https: //api.semanticscholar.org/CorpusID:265810192

    2023 arXiv

  13. [21]

    Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, et al. 2018. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. arXiv preprint arXiv:1809.08887 (2018)

    2018 arXiv

  14. [24]

    Joty, Yingbo Zhou, and Semih Yavuz

    Wenting Zhao, Ye Liu, Tong Niu, Yao Wan, Philip Yu, Shafiq R. Joty, Yingbo Zhou, and Semih Yavuz. 2023. DIVKNOWQA: Assessing the Reasoning Ability of LLMs via Open-Domain Question Answering over Knowledge Base and Text. ArXiv abs/2310.20170 (2023). https://api.semanticscholar....

    2023 arXiv

  15. [2023]

    In AAAI Conference on Artificial Intelligence

    Knowledge Graph Prompting for Multi-Document Question Answering. In AAAI Conference on Artificial Intelligence . https://api.semanticscholar.org/ CorpusID:261076072

  16. [2024]

    Transactions of the Association for Computational Linguistics 12 (2024), 786–802

    Beyond Boundaries: A Human-like Approach for Question Answering over Structured and Unstructured Information Sources. Transactions of the Association for Computational Linguistics 12 (2024), 786–802. https://api.semanticscholar. org/CorpusID:270491239

    2024

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.