{"id":"23667498-5438-4401-91ef-33d1e5d3d80c","arxiv_id":"2501.02772","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"GeAR adds a text-generation objective and cross-attention fusion to a bi-encoder retriever, and the authors report improved sentence-level retrieval inside documents while keeping global ranking as fast as a standard bi-encoder.","lead":"This paper introduces a retrieval model, GeAR, that is trained both to rank documents and to generate the specific sentence inside a document that answers or matches the query. It reports noticeably better fine-grained, sentence-level retrieval than standard dense retrievers, with no extra cost for the document-ranking step.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 4.3's claim that removing the LM loss hurts global retrieval is contradicted by Table 1; the global gains attributed to the generation module are not supported.","rationale":"The reader's weakness was the RIR benchmark circularity. While that is a valid external-validity concern, it is partially mitigated by the strong local-retrieval gains on the real QA datasets (SQuAD, NQ, TriviaQA) in Table 2, so the local-retrieval contribution does not stand or fall on RIR. The ablation contradiction is more load-bearing because it is an internal inconsistency: the paper makes a specific, falsifiable claim (LM loss improves global retrieval) and its own Table 1 and Table 4 show the opposite or a null effect. If the generation objective does not improve global retrieval, then the paper's central narrative that generation augments retrieval globally is unsupported, and the global advantage over BGE-RT is explained by the contrastive learning component rather than the proposed generation mechanism. This does not overturn the whole paper, since local retrieval on QA datasets remains plausible, but it does require the authors to revise their causal claims and reframe the contribution. The reader's CONDITIONAL verdict remains appropriate, so I do not change the verdict, but I identify a different and more fundamental weakness than the one the reader highlighted.","tokens_in":15921,"tokens_out":8781,"duration_ms":85643,"concrete_test":"Recompute the aggregate global metrics in Table 1 and Table 4 with bootstrap standard errors and run a paired significance test (e.g., Wilcoxon signed-rank) between GeAR and GeARw/oLLM on SQuAD, PAQ, and the combined set. Also compare GeARw/oLLM against BGE-RT on the same benchmarks. If GeARw/oLLM matches or exceeds the reported 3.5% global improvement, or if the GeAR-vs-GeARw/oLLM differences are not significant, then Section 4.3's claim must be withdrawn and the global contribution should be attributed to contrastive learning alone.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 4.3 states: 'We find that if LM loss is removed, both global and local retrieval performance of the model is reduced.' Table 1 contradicts this for global retrieval. GeARw/oLLM outperforms GeAR on SQuAD (R@5 0.889 vs 0.887; M@5 0.776 vs 0.766) and PAQ (R@5 0.955 vs 0.952; M@5 0.877 vs 0.872), and is essentially tied on RIR (0.963 vs 0.964). Table 4 shows α=0 global average Recall is 0.844 vs 0.846 for α=0.25, a 0.002 difference, with α=0.5 also at 0.844. No significance testing is reported. Thus the paper's explicit causal claim that the generation objective improves global retrieval is unsupported by its own data. This matters because the central contribution is framed as 'generation-augmented' retrieval: if the LM loss does not help global retrieval, the reported 3.5% global improvement over BGE-RT must be attributed to the contrastive/momentum setup, not the generation module. The global half of the central claim then rests on a component that is already standard in the literature, weakening the novelty of the global contribution.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GeAR (Generation Augmented Retrieval), a bi-encoder retriever augmented with a fusion encoder and a lightweight text decoder. Training combines a contrastive loss for global document retrieval with a language-modeling loss that conditions on fused query/document representations to generate query-relevant local information, such as answer sentences or relevant sentences. The authors construct a synthetic RIR dataset by using Llama-3.1-70B to rewrite Wikipedia sentences into queries, and evaluate both global document retrieval and local sentence-level retrieval on QAR and RIR scenarios. They report a 3.5% relative global Recall@5 improvement over a retrained BGE baseline and a 12.9% relative local-retrieval improvement over BGE-Reranker-L, and emphasize that global retrieval at inference uses only the bi-encoder, so it adds no computational cost over a standard bi-encoder.","tokens_in":16164,"tokens_out":8569,"duration_ms":85831,"significance":"If the claims are supported, GeAR is a useful step toward retrieval models that return not only relevant documents but also the specific sentences that justify the match, with the global retrieval stage costing no more than a bi-encoder. The paper has several strengths: it retrains strong baselines (SBERT_RT, BGE_RT) on aligned data, evaluates on multiple datasets, reports an ablation with and without the LM loss, measures inference cost, and provides qualitative attention analysis and layer-wise local-retrieval results. The local-retrieval gains from the LM loss are large and internally consistent. However, the paper's causal claim that the generation objective improves global retrieval is not supported by its own ablation table, and the RIR benchmark's synthetic construction raises validity concerns for the local-retrieval headline. Both issues are load-bearing and need to be addressed before the paper can be accepted.","major_comments":[{"comment":"Section 4.3 states that removing the LM loss reduces both global and local retrieval performance, but Table 1 does not support this for global retrieval. GeARw/oLLM outperforms GeAR on SQuAD (R@5 0.889 vs 0.887; M@5 0.776 vs 0.766) and PAQ (R@5 0.955 vs 0.952; M@5 0.877 vs 0.872), and the two are essentially tied on RIR R@5 (0.963 vs 0.964). Across the five datasets the average R@5 difference is about 0.001, and Table 4 shows only a 0.002 difference in global average Recall between α=0 and α=0.25, with α=0.5 matching α=0. No significance testing or variance across runs is reported. Consequently the causal claim that the generation objective improves global retrieval is unsupported: the global improvement over BGE_RT appears to come from the contrastive/momentum training recipe rather than from the generation module. The paper should either provide per-dataset significance tests and a justified effect size, or reframe the global contribution accordingly.","section":"§4.3, Table 1, Table 4"},{"comment":"The RIR benchmark is generated by sampling a Wikipedia sentence u, rewriting it into a query q with Llama-3.1-70B, and then treating the original sentence u as ground truth. GeAR's LM objective trains the decoder to reproduce exactly that u given q and the document. Local-retrieval scores on RIR therefore partly measure how well the model inverts the benchmark-construction pipeline, rather than how well it satisfies naturally occurring user information needs. The paper does not include any human-authored or otherwise externally sourced RIR test set, so the claimed 12.9% local-retrieval improvement over BGE-Reranker-L should be interpreted as in-distribution performance on synthetic data. I recommend adding an evaluation on naturally occurring queries (for example, MS MARCO passages with human judgments, or human rewrites of the same documents) and an analysis of whether the cross-attention localization transfers to that setting.","section":"§3.2, Appendix A"},{"comment":"The abstract and Section 3.5 state that when used as a retriever, GeAR does not incur any additional computational cost over bi-encoders. This is accurate for global document retrieval, which uses only the bi-encoder, but the local-information-retrieval mode runs the fusion encoder and is measured at 1.28x on CPU and 1.65x on GPU relative to BGE (Table 9). The cost-free claim should be explicitly qualified to global retrieval, or the local-retrieval overhead should be disclosed in the abstract and Section 1, since the paper's framing otherwise overstates the efficiency benefit.","section":"Abstract, §3.5, Appendix D"}],"minor_comments":[{"comment":"For local-information retrieval, the reranker baselines are applied to each sentence independently, while GeAR attends to the full document through cross-attention. This gives GeAR additional document-level context and is a potential confound in the comparison; the paper should discuss why sentence-level scoring is the appropriate baseline or include a reranker variant that also conditions on document context.","section":"Table 2 and §4.2"},{"comment":"The phrase 'optimal values at 0.25 and 0.5 respectively' should specify that 0.25 is the global-retrieval optimum and 0.5 is the local-retrieval optimum; the current wording is ambiguous.","section":"§4.3, Table 4"},{"comment":"The x-axis is labeled 'Number of Layers' but appears to be a layer index; please relabel it as 'Layer index' for clarity.","section":"Figure 4"},{"comment":"There is a typo in the paragraph describing the alternative data-construction attempt: 'we fine that other sentences...' should be 'we find that other sentences...'. Similar typos elsewhere include 'intutive' (Section 1), 'attnetion' (Figure 3 caption), and 'Visulization' (Appendix E).","section":"Appendix A"},{"comment":"The statement that code, data, and models will be released at the general LMOps repository should be replaced with a dedicated project link or repository path; a general organization URL is not actionable for reproducing the specific method.","section":"Abstract and Section 5"}],"recommendation":"major_revision","confidential_remarks":"The paper reports an appealing system and the local-retrieval improvements are substantial, but the headline global-retrieval claim is contradicted or at least unsupported by the paper's own ablation, and the RIR benchmark is synthetic in a way that directly favors the method's training objective. These issues are fixable: the authors can significantly test or reframe the global claim, and add an external benchmark or human-query validation for local retrieval. If those points are addressed, the paper could be a solid contribution; in its current form the central claims are not sufficiently supported."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, two things before you read the paper. First, the local retrieval mechanism is genuinely interesting: train a bi-encoder with an auxiliary decoder that generates the query-relevant sentence, then use cross-attention weights to rank sentences. No extra cost at global retrieval time, and the local gains on QA sentence selection are solid. Second, the paper's central causal claim for global retrieval is contradicted by its own ablation. Section 4.3 says removing the LM loss hurts both global and local performance, but Table 1 shows the no-LLM variant beats GeAR on SQuAD and PAQ global metrics and ties on RIR. That is not a rounding issue; it means the reported 3.5% global improvement over BGE-RT comes from the contrastive and momentum components, not from the generation objective. This needs to be fixed in revision.\n\nThe architecture itself is clean. A query encoder shared with a fusion encoder that cross-attends to document tokens, a decoder trained to generate the relevant sentence, and a training pipeline that builds 5.8M synthetic (query, document, sentence) triples from Wikipedia. They also retrain SBERT and BGE on their own data for fairer comparison. That is good practice, and the local retrieval results on the held-out QA datasets are encouraging, not just on their synthetic RIR benchmark.\n\nNow the soft spots. The RIR benchmark is partly circular: the queries are generated by rewriting the ground-truth sentences, and GeAR was trained to generate those same sentences. So the large local gains on RIR (0.933 R@3 vs 0.891 for reranker-L) are inflated. The QA datasets provide some independence, and there GeAR's local lead still holds, but the 12.9% headline is not an honest external estimate. There is also no documented decontamination between the PAQ training sample and the Wikipedia-derived evaluation documents, and no code or data released, so independent verification is currently impossible. The generation output numbers (Table 3) are interesting but not the main show.\n\nOverall, this is a worthwhile paper but the global-retrieval claim is over-sold. The local-retrieval idea deserves scrutiny, and the ablation mismatch is fixable. I would send it to peer review with a request for major revision, not desk-reject. I would not cite it in my own work until code and data are out and the causal claims are corrected. For a reading group, it would generate good discussion about benchmark circularity and ablation interpretation.","headline":"Solid local-retrieval mechanism, but the global-retrieval claim is undercut by the paper's own ablation data.","tokens_in":16781,"tokens_out":3096,"would_cite":false,"duration_ms":27658,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"GeAR claims that training a bi-encoder retriever with a generation objective and a cross-attention fusion module gives it simultaneous global and local retrieval ability, at no additional retrieval cost.","keywords":["GeAR","generation-augmented retrieval","bi-encoder","local information retrieval","cross-attention fusion","contrastive learning","language-modeling loss","synthetic retrieval data"],"falsifier":"A direct test would be to run GeAR's cross-attention sentence ranking on a held-out, human-annotated local-relevance set built from real search queries and sentence-level judgments; if its sentence-level Recall@1 is no better than a chunk-then-BGE baseline or BGE-Reranker-L on that set, the central claim of superior local retrieval would be refuted.","tokens_in":15666,"feed_emoji":"🔍","tokens_out":5759,"duration_ms":48372,"temperature":0.7,"pith_summary":"The paper claims that a bi-encoder retriever can learn global document relevance and local sentence-level relevance at the same time if it is trained with a generation objective alongside contrastive learning. The proposed model, GeAR, adds a fusion encoder and a lightweight text decoder to the classic query-document bi-encoder. At inference, document ranking uses only the bi-encoder, so the global retrieval step costs the same as a standard bi-encoder, while the query-relevant sentences inside a document are ranked by cross-attention weights. On their benchmarks, GeAR reports a 3.5% relative improvement over a retrained BGE baseline for global document retrieval and a 12.9% relative improvement over BGE-Reranker-L for local information retrieval. A sympathetic reader would take this as evidence that retrieval and fine-grained localization can be one learned model rather than a pipeline of separate systems.","feed_headline":"One retriever returns documents and the exact sentences","feed_subtitle":"Adding a generation task to a bi-encoder lifts local retrieval by 12.9 percent without slowing global search.","key_machinery":"The central machinery is a fusion encoder that shares nearly all parameters with the query encoder but adds a lightweight cross-attention module at each layer, letting document tokens attend to query tokens. A causal text decoder with a dedicated [Decode] token consumes the fusion embeddings and is trained to generate the fine-grained unit (sentence or answer) that the query points to inside the document. At inference the cross-attention weights serve as a per-token relevance map, and sentences are ranked by aggregating those weights; the decoder and fusion encoder are not needed for global document ranking, so the bi-encoder alone keeps the retrieval cost unchanged.","core_discovery":"GeAR's central claim is that optimizing a retriever to generate the query-relevant text inside a document, rather than only to score the document against the query, makes the retriever better at both tasks. The model is trained with a combined loss $\\mathcal{L}_{\\mathrm{GeAR}} = \\mathcal{L}_{\\mathrm{CL}} + \\alpha \\mathcal{L}_{\\mathrm{LM}}$, where the contrastive loss aligns query and document embeddings and the language-modeling loss trains a causal decoder to reproduce the ground-truth sentence or answer given fused query-document representations. The same cross-attention weights that carry the fused representation are then used at inference to rank sentences for local retrieval. The paper reports consistent gains across SQuAD, NQ, TriviaQA, PAQ, and a synthetic RIR dataset, and shows via ablation that removing the language-modeling loss degrades both global and local retrieval.","pith_inferences":["Beyond the paper's claims: the RIR benchmark is generated by the same LLM rewriting pipeline used to build training data, so the 12.9% local-retrieval gain should be re-measured on human-annotated query-sentence pairs before being taken as a statement about real search behavior.","Beyond the paper's claims: the finding that the best cross-attention layer is not the deepest one suggests that for other fusion-retrieval designs, probing intermediate layers may yield better localization than using the final layer.","Beyond the paper's claims: the same architecture could be tested with a decoder that generates a short rationale or a highlighted span instead of the full sentence, which would tighten the connection to fine-grained citation and grounded-generation tasks."],"forward_implications":["Web search and retrieval-augmented generation systems can show, alongside each retrieved document, the sentence or phrase that actually answers the query, without running a separate reranker or chunking pipeline.","Because global retrieval still uses only the bi-encoder, deployment does not require storing extra vectors or paying the delayed-interaction costs associated with ColBERT-style late interaction.","The generation output doubles as an interpretable explanation of why a document was retrieved, replacing a bare scalar similarity score with concrete, query-specific text.","The joint retrieval-and-generation objective can be trained end to end, so improvements to the fine-grained localization signal also feed back into the global document-ranking quality.","The same architecture could extend to long-document retrieval, where finding the needle inside the haystack is as important as retrieving the haystack itself."],"supporting_citations":[{"why":"Supplies the bi-encoder architecture that GeAR extends with fusion and decoding modules.","marker":"Reimers and Gurevych, 2019"},{"why":"Supplies the contrastive learning objective and dense retrieval setup for open-domain question answering.","marker":"Karpukhin et al., 2020"},{"why":"Provides the PAQ dataset used to sample 30M QAR training triples and 20k evaluation queries.","marker":"Lewis et al., 2021"},{"why":"Provides BERT initialization for GeAR's encoders and decoder.","marker":"Devlin et al., 2019"},{"why":"Supplies the momentum encoder and contrastive queue used to generate soft labels.","marker":"He et al., 2020"},{"why":"Supplies the momentum-distillation and fusion-encoder design that GeAR adapts from vision-language pretraining.","marker":"Li et al., 2022"},{"why":"The main BGE baseline and the model used to filter synthetic RIR query-document pairs.","marker":"Xiao et al., 2024"},{"why":"Provides the ColBERT-QA late-interaction baseline that GeAR compares against on retrieval quality and cost.","marker":"Khattab et al., 2021"}],"fun_headline_variants":["Retriever that also generates the answer","Zero-cost generation boosts retrieval precision","One model retrieves and pinpoints the passage","Generation-augmented retrieval for finer hits","Train to generate, retrieve better"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the synthetic RIR benchmark, built by having an LLM rewrite Wikipedia sentences into search queries and treating the original sentences as ground truth, faithfully represents real user information needs; if it does not, the reported local-retrieval gains may measure the model's ability to invert the data-generation pipeline rather than to locate relevant content in real documents.","fun_headline_variants_meta":{"raw":{"variants":["Retriever that also generates the answer","Zero-cost generation boosts retrieval precision","One model retrieves and pinpoints the passage","Generation-augmented retrieval for finer hits","Train to generate, retrieve better"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000745,"raw_usage":{"total_tokens":3326,"prompt_tokens":951,"completion_tokens":2375,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":567,"completion_tokens_details":{"reasoning_tokens":2312}},"tokens_in":567,"tokens_out":2375,"duration_ms":20392,"temperature":1.0,"reasoning_tokens":2312,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:05:24.620954+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test would be to run GeAR's cross-attention sentence ranking on a held-out, human-annotated local-relevance set built from real search queries and sentence-level judgments; if its sentence-level Recall@1 is no better than a chunk-then-BGE baseline or BGE-Reranker-L on that set, the central claim of superior local retrieval would be refuted.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the ColBERT-QA late-interaction baseline that GeAR compares against on retrieval quality and cost."}],"review_version":1}