arxiv: 2603.26815 · v2 · submitted 2026-03-26 · 💻 cs.CL · cs.AI· cs.IR

Recognition: no theorem link

Resolving the Robustness-Precision Trade-off in Financial RAG through Hybrid Document-Routed Retrieval

Zhiyuan Cheng , Longying Lai , Yue Liu

Authors on Pith no claims yet

Pith reviewed 2026-05-15 00:16 UTC · model grok-4.3

classification 💻 cs.CL cs.AIcs.IR

keywords financial RAGhybrid retrievaldocument routingchunk-based retrievalsemantic file routingquestion answeringFinDER benchmarkrobustness-precision trade-off

0 comments

The pith

Hybrid Document-Routed Retrieval resolves the robustness-precision trade-off in financial RAG by routing queries to whole documents first then retrieving targeted chunks within them.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Financial RAG systems face a trade-off where chunk-based retrieval offers precision but causes cross-document confusion in uniform corpora like regulatory filings, resulting in high failure rates. Semantic file routing improves robustness by directing queries to entire documents but reduces the ability to deliver perfect answers. The paper introduces Hybrid Document-Routed Retrieval, which applies file routing first to filter documents and then uses chunk retrieval scoped to those documents. This approach achieves the highest average scores, lowest failure rates, and best correctness and perfect-answer rates across all tested groups on the FinDER benchmark of 1,500 queries.

Core claim

The central discovery is that Hybrid Document-Routed Retrieval (HDRR) resolves the robustness-precision trade-off by using semantic file routing to select relevant documents and then performing chunk-based retrieval within the identified documents, leading to an average score of 7.54, a failure rate of 6.4%, a correctness rate of 67.7%, and a perfect-answer rate of 20.1% on the FinDER benchmark, outperforming both chunk-based and semantic file routing baselines on every metric.

What carries the argument

Hybrid Document-Routed Retrieval (HDRR), a two-stage architecture that filters documents via semantic file routing before applying scoped chunk retrieval.

If this is right

HDRR delivers an average performance score of 7.54, representing a 25.2% improvement over chunk-based retrieval.
The failure rate drops to 6.4% with the hybrid method.
Correctness improves to 67.7%, an increase of 18.7 percentage points over chunk-based retrieval.
The rate of perfect answers rises to 20.1%, exceeding both baseline approaches.
Superior results hold across all five experimental groups in the evaluation.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

The hybrid architecture may generalize to other domains involving large, homogeneous document sets, such as legal or technical literature.
Future systems could adapt the routing stage dynamically based on query type to further optimize performance.
This suggests that staged retrieval strategies can address similar trade-offs in non-financial RAG applications.
Additional gains might come from refining the chunk embedding process within the filtered document set.

Load-bearing premise

The FinDER benchmark of 1,500 queries across five groups represents real-world financial document distributions and the observed gains from the two-stage routing will generalize without further tuning.

What would settle it

Evaluating HDRR on a new financial document corpus outside the FinDER benchmark where it no longer outperforms the chunk-based and semantic file routing baselines would disprove the resolution of the trade-off.

Figures

Figures reproduced from arXiv: 2603.26815 by Longying Lai, Yue Liu, Zhiyuan Cheng.

**Figure 1.** Figure 1: Chunk-Based RAG (CBR) pipeline. The offline phase indexes documents into dual stores (FTS5 for keyword search, FAISS for semantic search). The online phase retrieves, fuses, reranks, and generates. Z. Cheng et al.: Preprint submitted to Elsevier Page 7 of 18 [PITH_FULL_IMAGE:figures/full_fig_p007_1.png] view at source ↗

**Figure 2.** Figure 2: Semantic File Routing (SFR) pipeline. No offline indexing is required. The query is parsed into structured metadata, resolved to file paths, and the full document is provided to the generation model. 4.2.1. Corpus Organization Documents are stored in a hierarchical directory following the pattern: data/{year}/{ticker}.{ext} where year is the fiscal year, ticker is the company’s stock ticker symbol, and ex… view at source ↗

**Figure 3.** Figure 3: Hybrid Document-Routed Retrieval (HDRR) pipeline. Stage 1 routes the query to the correct document via LLM structured output. Stage 2 performs scoped hybrid search (FTS + semantic + RRF + reranking) restricted to the routed document’s chunks. If routing fails, the system falls back to full-corpus retrieval. Z. Cheng et al.: Preprint submitted to Elsevier Page 9 of 18 [PITH_FULL_IMAGE:figures/full_fig_p009… view at source ↗

**Figure 4.** Figure 4: Side-by-side comparison of the CBR and SFR paradigms for the same query. CBR retrieves targeted chunks (∼10K tokens) while SFR provides the full document (∼100K+ tokens), illustrating the precision-vs-robustness trade-off that HDRR resolves. 6.2. Key Findings The results reveal a clear performance hierarchy, with HDRR dominating both baseline paradigms on every metric. Finding 1: The CBR–SFR robustness-pre… view at source ↗

read the original abstract

Retrieval-Augmented Generation (RAG) systems for financial document question answering typically follow a chunk-based paradigm: documents are split into fragments, embedded into vector space, and retrieved via similarity search. While effective in general settings, this approach suffers from cross-document chunk confusion in structurally homogeneous corpora such as regulatory filings. Semantic File Routing (SFR), which uses LLM structured output to route queries to whole documents, reduces catastrophic failures but sacrifices the precision of targeted chunk retrieval. We identify this robustness-precision trade-off through controlled evaluation on the FinDER benchmark (1,500 queries across five groups): SFR achieves higher average scores (6.45 vs. 6.02) and fewer failures (10.3% vs. 22.5%), while chunk-based retrieval (CBR) yields more perfect answers (13.8% vs. 8.5%). To resolve this trade-off, we propose Hybrid Document-Routed Retrieval (HDRR), a two-stage architecture that uses SFR as a document filter followed by chunk-based retrieval scoped to the identified document(s). HDRR eliminates cross-document confusion while preserving targeted chunk precision. Experimental results demonstrate that HDRR achieves the best performance on every metric: an average score of 7.54 (25.2% above CBR, 16.9% above SFR), a failure rate of only 6.4%, a correctness rate of 67.7% (+18.7 pp over CBR), and a perfect-answer rate of 20.1% (+6.3 pp over CBR, +11.6 pp over SFR). HDRR resolves the trade-off by simultaneously achieving the lowest failure rate and the highest precision across all five experimental groups.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Desk Editor's Note private letter to a colleague

HDRR is a sensible two-stage fix that beats both baselines on the FinDER numbers, but the gains rest on point estimates with no error bars or routing ablations.

read the letter

HDRR combines semantic file routing to pick whole documents first, then runs chunk retrieval only inside those documents. The authors lay out the trade-off clearly: chunk-based retrieval gets more perfect answers but higher failure rates from cross-document mix-ups, while pure semantic file routing cuts failures but loses precision on targeted facts. Their hybrid posts the top score on every metric they track, including a 7.54 average, 6.4% failure rate, and 20.1% perfect answers across the five FinDER groups. That is the concrete engineering step worth noting. The framing is direct and the architecture is simple enough to implement without new machinery. The numbers line up with the stated goal of keeping robustness while regaining precision. The main gaps are statistical. All deltas are reported as single averages with no standard deviations, confidence intervals, or hypothesis tests, so it is impossible to judge whether the 25% lift over chunk-based retrieval would hold under resampling. There is also no separate check on how often the first-stage router picks the right documents; if that step errs at any real rate, the scoped chunk stage has no way to recover. The benchmark itself is new, so we lack external confirmation that the gains transfer to other financial corpora. This paper is aimed at teams running RAG on regulatory or financial filings where a single bad retrieval can matter. A practitioner would pick up the two-stage pattern and the metric split between failure rate and perfect-answer rate. It is solid enough on its own terms to go to a referee who can ask for the missing variance numbers and a routing-accuracy ablation. I would send it out for review.

Referee Report

3 major / 1 minor

Summary. The paper identifies a robustness-precision trade-off in financial RAG systems between chunk-based retrieval (CBR), which offers high precision but high failure rates due to cross-document confusion, and Semantic File Routing (SFR), which improves robustness via whole-document routing but reduces precision. It proposes Hybrid Document-Routed Retrieval (HDRR), a two-stage method that first applies SFR to select relevant documents and then performs chunk retrieval scoped to those documents. On the FinDER benchmark of 1,500 queries, HDRR is claimed to strictly dominate both baselines on all metrics: average score 7.54 (25.2% above CBR, 16.9% above SFR), failure rate 6.4%, correctness 67.7%, and perfect-answer rate 20.1%.

Significance. If the empirical superiority holds under proper statistical validation and generalizes, HDRR would provide a simple, training-free architecture that simultaneously lowers catastrophic failures and raises answer precision in domain-specific RAG for homogeneous document collections such as regulatory filings. This addresses a practical pain point in financial QA without requiring new embeddings or fine-tuning.

major comments (3)

[Experimental Results] Experimental results: All reported gains (e.g., average score 7.54, failure rate 6.4%, +18.7 pp correctness) are given solely as aggregate point estimates with no standard deviations, bootstrap confidence intervals, or hypothesis tests. Without these, it is impossible to establish that the differences are statistically reliable rather than artifacts of the particular 1,500-query sample or group composition.
[Proposed Method] HDRR architecture description: The two-stage pipeline depends on the first-stage SFR document filter having high accuracy; any non-trivial routing error would restrict the second-stage chunk retriever to an incomplete or incorrect document set. No routing-accuracy metric, confusion matrix, or ablation on filter error rate is supplied, leaving the robustness claim unverified.
[Evaluation Setup] Evaluation on FinDER: The benchmark is stated to contain five groups, yet only aggregate numbers are presented despite the explicit claim of dominance “across all five experimental groups.” Per-group tables or breakdowns are required to rule out that gains are driven by one or two easy groups.

minor comments (1)

[Abstract] The definitions of the composite “average score,” “correctness rate,” and “perfect-answer rate” are not stated in the abstract or methods summary; explicit formulas or rubrics should be added for reproducibility.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive and detailed comments. We address each major point below and have revised the manuscript to incorporate additional analyses and breakdowns as suggested.

read point-by-point responses

Referee: Experimental results: All reported gains (e.g., average score 7.54, failure rate 6.4%, +18.7 pp correctness) are given solely as aggregate point estimates with no standard deviations, bootstrap confidence intervals, or hypothesis tests. Without these, it is impossible to establish that the differences are statistically reliable rather than artifacts of the particular 1,500-query sample or group composition.

Authors: We agree that statistical validation strengthens the reliability of the empirical claims. In the revised manuscript we have added bootstrap confidence intervals (1,000 resamples) and standard deviations for every reported metric. We also include paired statistical tests (McNemar’s test on binary outcomes and Wilcoxon signed-rank test on scores) showing that all HDRR improvements over both baselines are significant at p < 0.01. revision: yes
Referee: HDRR architecture description: The two-stage pipeline depends on the first-stage SFR document filter having high accuracy; any non-trivial routing error would restrict the second-stage chunk retriever to an incomplete or incorrect document set. No routing-accuracy metric, confusion matrix, or ablation on filter error rate is supplied, leaving the robustness claim unverified.

Authors: The referee correctly identifies that HDRR’s performance hinges on first-stage routing quality. While end-to-end results already demonstrate practical gains, we have added a new subsection reporting document-level routing accuracy (precision 0.89, recall 0.92), the corresponding confusion matrix, and an ablation that injects controlled routing errors at varying rates. These additions directly quantify the sensitivity of the hybrid pipeline to filter mistakes. revision: yes
Referee: Evaluation on FinDER: The benchmark is stated to contain five groups, yet only aggregate numbers are presented despite the explicit claim of dominance “across all five experimental groups.” Per-group tables or breakdowns are required to rule out that gains are driven by one or two easy groups.

Authors: We acknowledge that aggregate-only reporting leaves the per-group consistency claim unsubstantiated in the original text. The revised manuscript now includes a new table (Table 3) that reports every metric—average score, failure rate, correctness, and perfect-answer rate—for each of the five groups individually. The table confirms that HDRR strictly outperforms both baselines in all five groups. revision: yes

Circularity Check

0 steps flagged

No circularity: claims rest on direct empirical measurements from FinDER benchmark

full rationale

The manuscript contains no equations, derivations, fitted parameters, or self-citations that reduce any reported result to its own inputs. HDRR performance (average score 7.54, failure rate 6.4%, etc.) is presented as measured outcomes on the 1,500-query FinDER benchmark across five groups; these quantities are not defined in terms of themselves or obtained by renaming a prior fit. The two-stage architecture is described procedurally without any uniqueness theorem or ansatz smuggled via citation. The central claim is therefore an empirical observation rather than a closed-form prediction that collapses by construction.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

The paper is an empirical systems contribution with no explicit free parameters, mathematical axioms, or newly postulated entities; the central claim depends entirely on the experimental outcomes on the FinDER benchmark.

pith-pipeline@v0.9.0 · 5619 in / 1213 out tokens · 67401 ms · 2026-05-15T00:16:54.455461+00:00 · methodology

discussion (0)

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

Adaptive Query Routing: A Tier-Based Framework for Hybrid Retrieval Across Financial, Legal, and Medical Documents
cs.IR 2026-04 conditional novelty 5.0

Tree reasoning outperforms vector search on complex document queries but a hybrid approach balances results across tiers, with validation showing an 11.7-point gap on real finance documents.

Reference graph

Works this paper leans on

19 extracted references · 19 canonical work pages · cited by 1 Pith paper · 4 internal anchors

[1]

arXiv preprint arXiv:2409.15066

Financial document analysis using large language models: A survey. arXiv preprint arXiv:2409.15066 . Chen, S., Wong, S., Chen, L., Tian, Y.,

work page arXiv
[2]

Extending Context Window of Large Language Models via Positional Interpolation

Extending context window of large language models via positional interpolation, in: arXiv preprint arXiv:2306.15595. Cheng, Z., Lai, L., Liu, Y., Cheng, K., Qi, X.,

work page internal anchor Pith review Pith/arXiv arXiv
[3]

Enhancing Financial Report Question-Answering: A Retrieval-Augmented Generation System with Reranking Analysis

Enhancing finan- cial report question-answering: A retrieval-augmented generation sys- tem with reranking analysis. URL:https://arxiv.org/abs/2603.16877, arXiv:2603.16877. Cormack, G.V., Clarke, C.L.A., Büttcher, S.,

work page internal anchor Pith review Pith/arXiv arXiv
[4]

arXiv preprint arXiv:2503.10720

AttentionRAG: Attention-guided context pruning in retrieval-augmented generation. arXiv preprint arXiv:2503.10720 . Gao, Y., Xiong, Y., Gao, X., Jia, K., Pan, J., Bi, Y., Dai, Y., Sun, J., Wang, H.,2023. Retrieval-augmentedgenerationforlargelanguagemodels:A survey, in: arXiv preprint arXiv:2312.10997. Hu, M., Wang, J., Zhao, W., Zeng, Q., Luo, L.,

work page arXiv 2023
[5]

arXiv preprint arXiv:2508.20212

FlowMalTrans: Unsupervised binary code translation for malware detection using flow- adapter architecture. arXiv preprint arXiv:2508.20212 . Izacard, G., Lewis, P., Lomeli, M., Hosseini, L., Petroni, F., Schick, T., Dwivedi-Yu, J., Joulin, A., Riedel, S., Grave, E.,

work page arXiv
[6]

arXiv preprint arXiv:2406.15319

LongRAG: Enhancing retrieval- augmented generation with long-context LLMs. arXiv preprint arXiv:2406.15319 . Jina AI,

work page arXiv
[7]

arXiv preprint arXiv:2601.08689

QuantEval: A benchmark for financial quantitative tasks in large language models. arXiv preprint arXiv:2601.08689 . Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., tauYih,W.,2020. Densepassageretrievalforopen-domainquestionan- swering,in:Proceedingsofthe2020ConferenceonEmpiricalMethods in Natural Language Processing (EMNLP), pp....

work page arXiv 2020
[8]

URL:https://doi.org/10.21203/rs.3.rs-9163424/v1, doi:10

Do transformers always win? an empirical study of semantic embeddings for short-text e-commerce reviews. URL:https://doi.org/10.21203/rs.3.rs-9163424/v1, doi:10. 21203/rs.3.rs-9163424/v1. research Square preprint, Version 1, posted 20 March

work page doi:10.21203/rs.3.rs-9163424/v1
[9]

arXiv preprint arXiv:2504.15800

FinDER: Financial dataset for question answer- ing and evaluating retrieval-augmented generation. arXiv preprint arXiv:2504.15800 . Liu, M., Shi, G.,

work page arXiv
[10]

arXiv preprint arXiv:2409.01466

Enhancing LLM-based text classification in politicalscience:Automaticpromptoptimizationanddynamicexemplar selection for few-shot learning. arXiv preprint arXiv:2409.01466 . Liu, N.F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., Liang, P.,

work page arXiv
[11]

Passage re-ranking with BERT, in: arXiv preprint arXiv:1901.04085. OpenAI,

work page internal anchor Pith review Pith/arXiv arXiv 1901
[12]

GPT-4 Technical Report

GPT-4 technical report. arXiv preprint arXiv:2303.08774 . OpenAI,2024a. IntroducingstructuredoutputsintheAPI.https://openai. com/index/introducing-structured-outputs-in-the-api/. OpenAI,2024b. NewembeddingmodelsandAPIupdates.https://openai. com/blog/new-embedding-models-and-api-updates. Reimers, N., Gurevych, I.,

work page internal anchor Pith review Pith/arXiv arXiv
[13]

3982–3992

Sentence-BERT: Sentence embeddings usingSiameseBERT-networks,in:Proceedingsofthe2019Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 3982–3992. Robertson,S.,Zaragoza,H.,2009. Theprobabilisticrelevanceframework: BM25 and beyond. Foundations and Trends in Information Retrieval 3, 333–389. Shi, Y., Sun, M., Liu, Z., Yang, M., Fang,...

work page 2009
[14]

Reason- ing in trees: Improving retrieval-augmented generation for multi-hop question answering,

Reasoningintrees:Improvingretrieval-augmentedgenerationformulti- hop question answering. arXiv preprint arXiv:2601.11255 . Z. Cheng et al.:Preprint submitted to ElsevierPage 17 of 18 Hybrid Document-Routed Retrieval for Financial RAG Siriwardhana, S., Weerasekera, R., Wen, E., Kaluarachchi, T., Rana, R., Nanayakkara, S.,

work page arXiv
[15]

Transactions of the Association for Computational Linguistics 11, 1–17

Improving the domain adaptation of retrieval augmentedgeneration(RAG)modelsforopendomainquestionanswer- ing. Transactions of the Association for Computational Linguistics 11, 1–17. SQLite,2024. SQLiteFTS5extension.https://www.sqlite.org/fts5.html. Su,J.,Lan,Q.,Xia,Y.,Sun,L.,Tian,W.,Shi,T.,Song,X.,He,L.,Jingsong, Y.,2026. Difficulty-awareagenticorchestrati...

work page 2024
[16]

arXiv preprint arXiv:2401.00368

Improving text embeddings with large language models. arXiv preprint arXiv:2401.00368 . Wang, Y., Tang, Y., Qian, Y., Zhao, C.,

work page arXiv
[17]

Visualleakbench: Auditing the fragility of large vision-language models against pii leakage and social engineering,

VisualLeakBench: Auditing the fragility of large vision-language models against PII leakage and social engineering. arXiv preprint arXiv:2603.13385 . Xue, Z., Zhao, S., Qi, Y., Zeng, X., Yu, Z.,

work page arXiv
[18]

arXiv preprint arXiv:2601.13632

Resilient routing: Risk-aware dynamic routing in smart logistics via spatiotemporal graph learning. arXiv preprint arXiv:2601.13632 . Yao, C., Zhan, Q., Cao, Z., Li, D., Lin, Y., Shao, Y., Wang, L., Wang, Z., Zhang,J.,Zhang,Y.,etal.,2025.GenerativeAIforsimulatingrealworld dynamics: Applications and challenges. Authorea Preprints . Zhang, Z., Fu, R., He, Y...

work page arXiv 2025
[19]

arXiv preprint arXiv:2509.12638

FinSentLLM: Multi-LLM and structured semantic signals for enhanced financial sentiment forecasting. arXiv preprint arXiv:2509.12638 . Zheng, L., Chiang, W.L., Sheng, Y., Zhuang, S., Wu, Z., Zhuang, Y., Lin, Z., Li, Z., Li, D., Xing, E.P., Zhang, H., Gonzalez, J.E., Stoica, I.,

work page arXiv