Pith. sign in

REVIEW 4 major objections 5 minor 43 references

NeuSym-RAG: Hybrid Neural Symbolic Retrieval with Multiview Structuring for PDF Question Answering

T0 review · 4 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A hybrid agent that interleaves SQL queries and vector similarity search improves full-PDF question answering by over 17 percentage points, the paper reports.

desk verdict A genuinely useful hybrid retrieval system with honest engineering and public artifacts, but the synergy claim rests on a 2.3-point gap with no significance testing and a few internal inconsistencies. read the letter →

arxiv 2505.19754 v2 pith:3C4DOWLB submitted 2025-05-26 cs.CL cs.AI

Ruisheng Cao , Hanchong Zhang , Tiancheng Huang , Zhangyi Kang , Yuxin Zhang , Liangtai Sun , Hanqi Li , Yuxun Miao
show 3 more authors
Shuai Fan Lu Chen Kai Yu
This is my paper
classification cs.CLcs.AI
keywords retrieval-augmentedgenerationhybridretrievalneural-symbolicPDFquestionansweringmulti-viewchunkingLLMagentsSQLvectorsimilaritysearch
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes NeuSym-RAG, a retrieval-augmented generation system for answering questions about entire academic-paper PDFs. The authors argue that previous systems use either neural retrieval (semantic similarity over text chunks) or symbolic retrieval (SQL over structured data), but not both, and that fixed-length chunking ignores the PDF's sections, tables, and figures. NeuSym-RAG parses each PDF into a relational database across multiple views, embeds selected cell values and images into a vector store, and lets an LLM agent choose which store to query at each step until it can answer. On three full-PDF QA datasets, including the authors' new AIRQA-REAL benchmark of 553 questions over thousands of papers, the system is reported to beat classic vector-based RAG and structured baselines, with an improvement of more than 17.3 percentage points over Classic RAG on AIRQA-REAL. The aim is to show that the two retrieval paradigms complement each other when combined interactively rather than used alone or in a fixed pipeline.

What carries the argument

The load-bearing mechanism is the provenance-linked pairing of a relational database and a vector store over the same multi-view parsed PDF. The database stores every view of the document—metadata, pages, sections, chunks, tables, images, equations—in a handcrafted universal schema; the vector store holds embeddings of the encodable cell values and cropped images, each entry recording the source table, column, and primary key. This lets the agent convert a SQL result set into vector-search filter conditions (matching primary keys) and, conversely, use a similarity-search hit to formulate a precise SQL query. The agent itself is a thought-action-observation loop with five parameterized actions: retrieve from vector store, retrieve from database, view image, calculate expression, and generate answer. The loop stops when the agent calls the terminal generate-answer action.

What would settle it

Run NeuSym-RAG with the SQL retrieval action disabled (keeping the same multi-view vector store, image viewer, and calculator) on AIRQA-REAL; if the accuracy gap over Classic RAG does not fall substantially below the reported 17.3 percentage points, then the claimed benefit of hybrid symbolic retrieval is unsupported. Equally, test a question whose answer appears only inside a figure region that the OCR step does not transcribe; a correct answer would contradict the assumption that parsing completeness limits the system.

Watch

Extended reading notes

Core claim

The central claim is that the interactive combination of neural and symbolic retrieval is what drives the accuracy gain, and that multiple chunking views help the agent locate the right context. The authors build a hand-designed database schema that stores each PDF's metadata, pages, sections, fixed-length chunks, tables, images, and equations, plus LLM/VLM-generated summaries. Cell values from the database are embedded into a vector store, with each vector tagged by its source table, column, and primary key so results from one store can be used to filter the other. An agent then iterates through actions—SQL retrieval, vector retrieval, viewing a cropped image, calculating an expression, or generating an answer—until it judges the context sufficient. The paper reports that this hybrid agent outperforms classic RAG and structured baselines on all three datasets, and that the gap over Classic RAG on AIRQA-REAL is at least 17.3 percentage points across the tested LLMs.

Load-bearing premise

The framework assumes that every fact needed to answer a question survives the parsing and summarization pipeline and appears in one of the handcrafted database tables; if a key detail is dropped or corrupted by OCR or summarization, no amount of agent action can recover it.

Editorial extensions

If this is right

  • On AIRQA-REAL, NeuSym-RAG beats Classic RAG by more than 17.3 percentage points for every LLM tested, and the gain is consistent across M3SciQA and SciDQA.
  • Multi-turn (iterative) retrieval outperforms the corresponding two-stage pipelines for both neural-only and symbolic-only variants, indicating that the ability to refine searches is itself a source of accuracy.
  • The best single retrieval view remains the plain text chunk, but view-specific options such as table content, image summaries, and equation content achieve the best results in their own question categories.
  • The framework requires the input documents to have implicit structure; it is not designed to work on free-form text with no layout, unlike graph-based pre-parsing approaches.
  • Open-source LLMs in the agentic loop can match or exceed closed-source ones on these tasks, and preprocessing with cheaper open-source summarizers does not hurt final accuracy much.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable implication beyond the paper: if the database or image-viewing action is disabled, the reported 17.3-point gain should shrink; if it does not shrink, the symbolic components are not the source of the gain.
  • Because every fact must survive parsing and summarization, the framework's real ceiling is set by the PDF parser's recall, not by the agent's planning ability; a question whose answer lives only in an unparsed figure region would be unanswerable regardless of agent skill.
  • The reported sensitivity to action serialization format (markdown over JSON/XML/YAML) suggests that prompt-format engineering may account for part of the absolute scores, so comparisons across agent frameworks should control for this.
  • The authors' observation that a single BM25 sparse collection matches or beats combining dense embeddings hints that for these benchmarks lexical overlap is already high; a domain with paraphrased or cross-lingual questions might change that balance.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes NeuSym-RAG, a hybrid neural-symbolic retrieval framework for question answering over full PDFs of academic papers. During preprocessing, a PDF is parsed into a handcrafted relational database (DuckDB) via multiview chunking, OCR, and LLM/VLM-generated summaries, and selected database cells are embedded into a Milvus vectorstore across text and image collections. At inference, an LLM agent predicts executable actions (retrieve from vectorstore, retrieve from database, view image, calculate expression, generate answer) in a ReAct-style loop until it is confident enough to answer. The authors contribute a self-annotated dataset, AIRQA-REAL, plus conversions of M3SciQA and SciDQA, and report results for several closed- and open-source LLMs against a range of neural, symbolic, and hybrid baselines. The headline claims are that NeuSym-RAG stably defeats vector-based RAG and structured baselines, and that it outperforms Classic RAG by at least 17.3 points on AIRQA-REAL.

Significance. If the empirical claims hold, the framework is a useful integration of symbolic (SQL) and neural (vector) retrieval for semi-structured documents, with the multi-view parsing and schema-linked vectorstore being a practical contribution. The paper is unusually transparent about implementation details: prompts, action formats, database schema, encodable columns, and evaluation functions are in the appendices, and code/data are released. It also states limitations honestly (preprocessing latency, iterative cost, reliance on implicit document structure). However, the central 'stable synergy' claim currently rests on a small raw gap without statistical support, and at least one table row contradicts the 'across all datasets' wording. The self-annotated benchmark and pilot-tuned prompt choices add endogeneity risk. The strengths are real, but the conclusions need to be made proportionate to the evidence before publication.

major comments (4)
  1. [§3.3, Table 3] The abstract's 'stably defeats' claim and the conclusion's synergy argument rest on the 2.3-point gap between NeuSym-RAG (39.6) and Iterative Neu-RAG (37.3) on AIRQA-REAL. No variance estimates, confidence intervals, or paired significance tests are reported anywhere in Tables 2–6, despite temperature 0.7 and despite the fact that the action/observation formats were fixed using pilot experiments on the same benchmark (Table 9). With n=553, the standard error of an independent-proportions difference is approximately 2.9 points, so the observed gap is within one standard error. Please report multiple seeds, paired significance tests, or confidence intervals before claiming that the symbolic component provides a stable improvement over neural-only retrieval.
  2. [§3.3, Table 2] The statement 'NeuSym-RAG remarkably outperforms the Classic RAG baseline across all datasets' is contradicted by the SciDQA row for Llama-3.3-70B-Instruct: Classic RAG scores 58.0 AVG, while NeuSym-RAG scores 56.4. The abstract's 'stably defeats' claim is therefore not supported by the presented data as written. Please either qualify the claim (e.g., 'for most model/dataset combinations') or explain why this configuration underperforms.
  3. [§3.1 and §3.3, App. B (Table 9)] The benchmark is partly endogenous to the method: AIRQA-REAL is self-annotated using 18 instance-specific evaluation functions designed by the authors, and key prompt choices (markdown action format, json observation format) were fixed after pilot studies on the same dataset. No inter-annotator agreement statistics are provided, and 240 of the 553 questions are converted from LitSearch. This does not invalidate the work, but it weakens the 'stable' superiority claim. Please provide external validation on the two existing benchmarks with official evaluation, and/or a robustness analysis that varies prompt choices and reports the spread of results.
  4. [§2.3, Table 5] The method description includes four vectorstore collections, but Table 5 shows that a single BM25 collection alone achieves the best overall accuracy (46%) on the AIRQA-REAL subset, and adding MiniLM and/or bge collections lowers accuracy. The main experiments in Tables 2–3 do not state which collections are enabled in the final system. Please specify the exact configuration used for the headline results; as written, the contribution of the neural embedding collections is unclear and the reported configuration is not fully reproducible.
minor comments (5)
  1. [Table 3 caption] Table 3 does not specify which LLM produced the results; the NeuSym-RAG value 39.6 matches the Qwen2.5-VL-72B-Instruct row in Table 2, but the caption and surrounding text should state this explicitly.
  2. [§3.4] In §3.4, 'Qwen' is defined as Qwen2.5-72B-Instruct, while Table 2 uses Qwen2.5-VL-72B-Instruct; please reconcile the notation so the model identity is unambiguous.
  3. [Table 2 / App. A.3] Because a question can carry multiple category tags, it should be clarified how the per-category scores and the AVG column are computed (sample-weighted vs. simple average over categories). The AVG value 30.7 for GPT-4o-mini on AIRQA-REAL does not obviously follow from the five category scores.
  4. [App. F] The prompt templates contain several typos and encoding artifacts, e.g., 'Mivlus' instead of 'Milvus', 'Y AML' instead of 'YAML', and corrupted characters in 'Ÿesör ¨No ¨without punctuation'; please clean these up before camera-ready.
  5. [Figure 6] The case study displays an abridged trajectory with an '≈' sign rather than exact values; consider adding a note that the display is simplified.

Circularity Check

0 steps flagged · score 0.0 of 10

No constructional circularity found: the reported gains are empirical benchmark comparisons, not consequences of the system's definitions or of self-citations.

full rationale

The paper's claimed derivation chain is an engineering pipeline (parse PDF to DB and vectorstore, let an LLM agent issue retrieval actions) plus an empirical evaluation on three QA datasets. No step equates a fitted quantity with a predicted quantity: the answer accuracy is computed by executing the agent against unseen questions and comparing outputs to gold references, and no evaluation function is a function of the method's own parameters. The action and observation formats were fixed using a pilot subset of AIRQA-REAL (App. B, Table 9), which is a hyperparameter-selection concern and weakens the strength of the self-built benchmark, but it does not make the 17.3% Classic-RAG gap or the 2.3-point NeuSym-RAG versus Iterative Neu-RAG gap an identity or a forced consequence of that selection. The remaining concerns are evidential, not circular: AIRQA-REAL is self-annotated, its 18 evaluation functions were designed by the authors, and the key hybrid-vs-neural-only comparison on AIRQA-REAL is reported without significance tests. The two same-author citations (Cao et al. 2021 for text-to-SQL; Xu et al. 2024 for the CalculateExpr action) are incidental references to prior related work rather than load-bearing justifications of the central result. External benchmarks M3SciQA and SciDQA are also used, so the central empirical claim is not solely evaluated on the endogenous benchmark. Accordingly, no self-definitional, fitted-input-renamed-as-prediction, or uniqueness-imported-by-self-citation step can be exhibited.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The paper contains no mathematical derivation, so the axiom ledger records the modeling and evaluation choices the empirical claim rests on. There are no invented physical or theoretical entities. The main free parameters are hand-chosen system hyperparameters, a hand-designed database schema, and benchmark-specific evaluation conventions.

free parameters (6)
  • Interaction turn limit = 20 turns
    Section 3.2 sets the maximum number of agent turns to 20; this is a hand-chosen budget used in all experiments.
  • Maximum retrieved tokens per turn and full-text cutoff = 5000 tokens
    Section 3.2 limits both the retrieved context per turn and the full-text input cutoff to 5k, which affects what the agent can see.
  • LLM sampling hyperparameters = temperature 0.7, top_p 0.95
    Section 3.2 fixes these sampling parameters for all LLM agents; they are not derived from theory.
  • Action format and observation format = markdown actions, json observations
    Appendix B Table 9 selects these formats based on a pilot study on a subset of AIRQA-REAL; they are tuned choices, not a consequence of the framework design.
  • Handcrafted database schema and encodable columns = 7 tables, 18 encodable columns
    Section 2.2 and App. E define a fixed schema for metadata, pages, sections, chunks, images, tables, equations, and references. All retrieval depends on this schema, which is chosen by the authors rather than learned or derived.
  • Vectorstore collection set = BM25, all-MiniLM-L6-v2, bge-large-en-v1.5, CLIP
    Section 2.3 creates four collections. Table 5 later shows a single BM25 text collection outperforms the full three-text-collection set on a subset, creating ambiguity about the optimal configuration used in main results.
assumptions (4)
  • domain assumption The handcrafted database schema is universal for AI research PDFs and captures all content needed to answer the questions.
    Invoked in Section 2.2 and App. E. If a needed fact is not parsed into one of the schema tables or encodable columns, neither SQL nor vector retrieval can reach it.
  • domain assumption LLM and VLM generated summaries preserve the factual content of pages, tables, and figures.
    Section 2.2 uses summaries as important encodable columns. Errors or omissions in these summaries propagate directly into retrieval results.
  • domain assumption The 18 custom evaluation functions measure answer correctness, and LLM-based subjective judges align with human judgment.
    App. D defines the metrics and Figure 7 validates LLM judgments on only 100 samples. The main results depend on these functions being faithful.
  • domain assumption The converted versions of M3SciQA and SciDQA retain the original benchmarks' difficulty and evaluation fidelity.
    App. A.4 notes that M3SciQA test answers are unavailable until 2025-09-15, so the paper uses the validation set instead, and SciDQA categories are assigned by heuristic rules. These conversions may make the benchmarks easier or noisier.

how reviews work

0 comments
Cite this review

Pith. "Pith review of NeuSym-RAG: Hybrid Neural Symbolic Retrieval with Multiview Structuring for PDF Question Answering." pith.science (2026). https://pith.science/paper/3C4DOWLB

@misc{pith2026250519754,
  author       = {Pith},
  title        = {Pith review of: NeuSym-RAG: Hybrid Neural Symbolic Retrieval with Multiview Structuring for PDF Question Answering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3C4DOWLB}},
  note         = {Machine review of arXiv:2505.19754}
}
read the original abstract

The increasing number of academic papers poses significant challenges for researchers to efficiently acquire key details. While retrieval augmented generation (RAG) shows great promise in large language model (LLM) based automated question answering, previous works often isolate neural and symbolic retrieval despite their complementary strengths. Moreover, conventional single-view chunking neglects the rich structure and layout of PDFs, e.g., sections and tables. In this work, we propose NeuSym-RAG, a hybrid neural symbolic retrieval framework which combines both paradigms in an interactive process. By leveraging multi-view chunking and schema-based parsing, NeuSym-RAG organizes semi-structured PDF content into both the relational database and vectorstore, enabling LLM agents to iteratively gather context until sufficient to generate answers. Experiments on three full PDF-based QA datasets, including a self-annotated one AIRQA-REAL, show that NeuSym-RAG stably defeats both the vector-based RAG and various structured baselines, highlighting its capacity to unify both retrieval schemes and utilize multiple views. Code and data are publicly available at https://github.com/X-LANCE/NeuSym-RAG.

Figures

Figures reproduced from arXiv: 2505.19754 by the authors.

Figure 1
Figure 1. Motivation of the proposed NeuSym-RAG. (Upper) Two paradigms of retrieval strategies. (Bottom) PDF documents can be split based on different granular￾ities and they contain many paratextual elements. in the top-left of [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of the NeuSym-RAG framework. The demonstration example comes from a real and simplified [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Illustration of how to convert an encodable cell [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Comparison of different structured baselines with our NeuSym-RAG. Refer to App. [PITH_FULL_IMAGE:figures/full_fig_p005_4.png]
Figure 5
Figure 5. Figure 5: Performances of different model sizes on [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 4
Figure 4. Figure 4: PER scores for [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 7
Figure 7. Figure 7: The consistency between human and LLM￾based evaluation. We classify examples that are judged as correct by human as positive samples. 3.5 Case Study In [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Frequency distribution of question lengths. [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]
Figure 9
Figure 9. Figure 9: Conference distribution of the papers used. [PITH_FULL_IMAGE:figures/full_fig_p014_9.png]
Figure 10
Figure 10. Figure 10: The complete and universal database schema to store the parsed elements of each PDF file. Note that, [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: All encodable columns which are inherently connected by the schema sub-graph. Note that, since the vi [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

43 extracted references · 14 canonical work pages

  1. [1]

    Amar Abane, Anis Bekri, and Abdella Battou. 2024. Fastrag: Retrieval augmented generation for semi-structured data. arXiv e-prints, pages arXiv--2411

    work page 2024

  2. [2]

    Anirudh Ajith, Mengzhou Xia, Alexis Chevalier, Tanya Goyal, Danqi Chen, and Tianyu Gao. 2024. http://arxiv.org/abs/2407.18940 Litsearch: A retrieval benchmark for scientific literature search

    arXiv 2024

  3. [3]

    Artifex Software

    Inc. Artifex Software. 2023. Pymupdf - a python binding for mupdf. https://pymupdf.readthedocs.io/en/latest/. Version 1.24.9, accessed on January 25, 2025

    work page 2023

  4. [4]

    Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on freebase from question-answer pairs. In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1533--1544

    2013

  5. [5]

    Asim Biswal, Liana Patel, Siddarth Jha, Amog Kamsetty, Shu Liu, Joseph E Gonzalez, Carlos Guestrin, and Matei Zaharia. 2024. Text2sql is not enough: Unifying ai and databases with tag. arXiv preprint arXiv:2408.14717

    arXiv 2024

  6. [6]

    Zheng Cai, Maosong Cao, Haojiong Chen, Kai Chen, Keyu Chen, Xin Chen, Xun Chen, Zehui Chen, Zhi Chen, Pei Chu, Xiaoyi Dong, Haodong Duan, Qi Fan, Zhaoye Fei, Yang Gao, Jiaye Ge, Chenya Gu, Yuzhe Gu, Tao Gui, Aijia Guo, Qipeng Guo, Conghui He, Yingfan Hu, Ting Huang, Tao Jiang, Penglong Jiao, Zhenjiang Jin, Zhikai Lei, Jiaxing Li, Jingwen Li, Linyang Li, S...

    arXiv 2024

  7. [7]

    Ruisheng Cao, Lu Chen, Zhi Chen, Yanbin Zhao, Su Zhu, and Kai Yu. 2021. LGESQL: line graph enhanced text-to-sql model with mixed local and non-local relations. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Lo...

    work page 2021

  8. [8]

    Routledge, and William Yang Wang

    Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting - Hao Huang, Bryan R. Routledge, and William Yang Wang. 2021. https://doi.org/10.18653/V1/2021.EMNLP-MAIN.300 Finqa: A dataset of numerical reasoning over financial data . In Proceedings of the 2021 Conference on Empirical Methods in Natural La...

    doi:10.18653/v1/2021.emnlp-main.300 2021

Show all 43 references
  1. [9]

    Smith, and Matt Gardner

    Pradeep Dasigi, Kyle Lo, Iz Beltagy, Arman Cohan, Noah A. Smith, and Matt Gardner. 2021. https://doi.org/10.18653/V1/2021.NAACL-MAIN.365 A dataset of information-seeking questions and answers anchored in research papers . In Proceedings of the 2021 Conference of the North Amer...

    2021 doi

  2. [10]

    Darren Edge, Ha Trinh, Newman Cheng, Joshua Bradley, Alex Chao, Apurva Mody, Steven Truitt, and Jonathan Larson. 2024. https://doi.org/10.48550/ARXIV.2404.16130 From local to global: A graph RAG approach to query-focused summarization . CoRR, abs/2404.16130

    2024 doi arXiv

  3. [11]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  4. [12]

    Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Mingwei Chang. 2020. Retrieval augmented language model pre-training. In International conference on machine learning, pages 3929--3938. PMLR

    2020

  5. [13]

    Xueyu Hu, Ziyu Zhao, Shuang Wei, Ziwei Chai, Qianli Ma, Guoyin Wang, Xuwu Wang, Jing Su, Jingjing Xu, Ming Zhu, Yao Cheng, Jianbo Yuan, Jiwei Li, Kun Kuang, Yang Yang, Hongxia Yang, and Fei Wu. 2024. https://openreview.net/forum?id=d5LURMSfTx Infiagent-dabench: Evaluating agen...

    2024

  6. [14]

    Yizheng Huang and Jimmy Huang. 2024. https://doi.org/10.48550/ARXIV.2404.10981 A survey on retrieval-augmented text generation for large language models . CoRR, abs/2404.10981

    2024 doi arXiv

  7. [15]

    Hugging Face Team . 2023. Llama 3.3-70b-instruct model. https://huggingface.co/meta-llama/Llama-3.3-70B-Instruct. Accessed: 2023-02-11

    2023

  8. [16]

    Yoonjoo Lee, Kyungjae Lee, Sunghyun Park, Dasol Hwang, Jaehyeon Kim, Hong-in Lee, and Moontae Lee. 2023. Qasa: advanced question answering on scientific articles. In Proceedings of the 40th International Conference on Machine Learning, ICML'23. JMLR.org

    2023

  9. [17]

    Chuhan Li, Ziyao Shangguan, Yilun Zhao, Deyuan Li, Yixin Liu, and Arman Cohan. 2024 a . https://doi.org/10.18653/v1/2024.findings-emnlp.904 M 3 S ci QA : A multi-modal multi-document scientific QA benchmark for evaluating foundation models . In Findings of the Association for ...

    2024 doi

  10. [18]

    Jinyang Li, Binyuan Hui, Ge Qu, Jiaxi Yang, Binhua Li, Bowen Li, Bailin Wang, Bowen Qin, Ruiying Geng, Nan Huo, et al. 2024 b . Can llm already serve as a database interface? a big bench for large-scale database grounded text-to-sqls. Advances in Neural Information Processing ...

    2024

  11. [19]

    Xinbei Ma, Yeyun Gong, Pengcheng He, Hai Zhao, and Nan Duan. 2023. Query rewriting in retrieval-augmented large language models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 5303--5315

    2023

  12. [20]

    Hannes Mühleisen and Mark Raasveldt. 2024. https://r.duckdb.org/ duckdb: DBI Package for the DuckDB Database Management System . R package version 1.1.3.9017, https://github.com/duckdb/duckdb-r

    2024

  13. [21]

    Linyong Nan, Yilun Zhao, Weijin Zou, Narutatsu Ri, Jaesung Tae, Ellen Zhang, Arman Cohan, and Dragomir Radev. 2023. Enhancing text-to-sql capabilities of large language models: A study on prompt design strategies. In Findings of the Association for Computational Linguistics: E...

    2023

  14. [22]

    Jayr Pereira, Robson Fidalgo, Roberto Lotufo, and Rodrigo Nogueira. 2023. https://doi.org/10.1007/978-3-031-28238-6_44 Visconde: Multi-document qa with gpt-3 and neural reranking . In Advances in Information Retrieval: 45th European Conference on Information Retrieval, ECIR 20...

    2023 doi

  15. [23]

    Mykhailo Poliakov and Nadiya Shvai. 2024. Multi-meta-rag: Improving rag for multi-hop queries using database filtering with llm-extracted metadata. arXiv preprint arXiv:2406.13213

    2024 arXiv

  16. [24]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. http://proceedings.mlr.press/v139/radford21a.html Learning transferable visual models...

    2021

  17. [25]

    Robertson and Hugo Zaragoza

    Stephen E. Robertson and Hugo Zaragoza. 2009. https://doi.org/10.1561/1500000019 The probabilistic relevance framework: BM25 and beyond . Found. Trends Inf. Retr., 3(4):333--389

    2009 doi

  18. [26]

    Bhaskarjit Sarmah, Dhagash Mehta, Benika Hall, Rohan Rao, Sunil Patel, and Stefano Pasquali. 2024. https://doi.org/10.1145/3677052.3698671 Hybridrag: Integrating knowledge graphs and vector retrieval augmented generation for efficient information extraction . In Proceedings of...

    2024

  19. [27]

    Zhihong Shao, Yeyun Gong, Yelong Shen, Minlie Huang, Nan Duan, and Weizhu Chen. 2023. https://doi.org/10.18653/v1/2023.findings-emnlp.620 Enhancing retrieval-augmented large language models with iterative retrieval-generation synergy . In Findings of the Association for Comput...

    2023 doi

  20. [28]

    Shruti Singh, Nandan Sarkar, and Arman Cohan. 2024. https://aclanthology.org/2024.emnlp-main.1163 S ci DQA : A deep reading comprehension dataset over scientific papers . In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing

    2024

  21. [29]

    Charlie Snell, Jaehoon Lee, Kelvin Xu, and Aviral Kumar. 2024. Scaling llm test-time compute optimally can be more effective than scaling model parameters. arXiv preprint arXiv:2408.03314

    2024 arXiv

  22. [30]

    Qwen Team. 2025. https://qwenlm.github.io/blog/qwen2.5-vl/ Qwen2.5-vl

    2025

  23. [31]

    Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2023. Interleaving retrieval with chain-of-thought reasoning for knowledge-intensive multi-step questions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Vo...

    2023

  24. [32]

    Bin Wang, Chao Xu, Xiaomeng Zhao, Linke Ouyang, Fan Wu, Zhiyuan Zhao, Rui Xu, Kaiwen Liu, Yuan Qu, Fukai Shang, Bo Zhang, Liqun Wei, Zhihao Sui, Wei Li, Botian Shi, Yu Qiao, Dahua Lin, and Conghui He. 2024 a . http://arxiv.org/abs/2409.18839 Mineru: An open-source solution for...

    2024 arXiv

  25. [33]

    Jianguo Wang, Xiaomeng Yi, Rentong Guo, Hai Jin, Peng Xu, Shengjun Li, Xiangyu Wang, Xiangzhou Guo, Chengming Li, Xiaohai Xu, et al. 2021. Milvus: A purpose-built vector data management system. In Proceedings of the 2021 International Conference on Management of Data, pages 2614--2627

    2021

  26. [34]

    Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. 2020. https://proceedings.neurips.cc/paper/2020/hash/3f5ee243547dee91fbd053c1c4a845aa-Abstract.html Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers . In Adva...

    2020

  27. [35]

    Zhongyuan Wang, Richong Zhang, Zhijie Nie, and Jaein Kim. 2024 b . https://doi.org/10.48550/ARXIV.2408.16991 Tool-assisted agent on SQL inspection and refinement in real-world scenarios . CoRR, abs/2408.16991

    2024 doi arXiv

  28. [36]

    Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023. http://arxiv.org/abs/2309.07597 C-pack: Packaged resources to advance general chinese embedding

    2023 arXiv

  29. [37]

    Fangyuan Xu, Weijia Shi, and Eunsol Choi. 2023. Recomp: Improving retrieval-augmented lms with compression and selective augmentation. arXiv preprint arXiv:2310.04408

    2023 arXiv

  30. [38]

    Hongshen Xu, Zichen Zhu, Situo Zhang, Da Ma, Shuai Fan, Lu Chen, and Kai Yu. 2024. Rejection improves reliability: Training llms to refuse unknown questions using rl from knowledge feedback. In First Conference on Language Modeling

    2024

  31. [39]

    S. Yang. 2023. https://towardsdatascience.com/advanced-rag-01-small-to-big-retrieval-172181b396d4 Advanced rag 01: Small-to-big retrieval . Accessed on 2024-12-11

    2023

  32. [40]

    Narasimhan, and Yuan Cao

    Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. 2023. https://openreview.net/forum?id=WE\_vluYUL-X React: Synergizing reasoning and acting in language models . In The Eleventh International Conference on Learning Representations, ...

    2023

  33. [41]

    Yusen Zhang, Ruoxi Sun, Yanfei Chen, Tomas Pfister, Rui Zhang, and Sercan Arik. 2024. Chain of agents: Large language models collaborating on long-context tasks. Advances in Neural Information Processing Systems, 37:132208--132237

    2024

  34. [42]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  35. [43]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.