Pith. sign in

REVIEW 4 major objections 5 minor 40 references

Structure-aware contrastive learning lifts long-document retrieval

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

A contrastive fine-tuning recipe that adds HTML structure and element-masking signals improves long structured document retrieval, with reported MRR@10 gains of about four points on BGE-M3.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Useful recipe and a new benchmark, but the headline gain is overstated by a mislabeled metric and by tuning on the evaluation split. the 4 major comments →

arxiv 2508.20778 v2 pith:3RQRZZ32 submitted 2025-08-28 cs.IR cs.LG

SEAL: Structure and Element Aware Learning to Improve Long Structured Document Retrieval

Xinhao Huang , Zhibo Ren , Yipeng Yu , Ying Zhou , Zulong Chen , Zeyi Wen This is my paper
classification cs.IR cs.LG
keywords long document retrievalstructured documentscontrastive learningHTML representationdense retrievalretrieval benchmarkelement maskingstructure-aware learning
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SEAL is a contrastive fine-tuning method for long structured documents (HTML pages, technical specs, legal texts). The paper argues that existing retrieval models trained on raw text are blind to document hierarchy—headings, sections, list items—and cannot align query meaning to fine-grained elements. To fix this, SEAL adds two losses: Structure-Aware Learning pulls tagged and untagged versions of the same relevant document toward the query, while Element-Aware Alignment randomly masks structural tags so the model must use the remaining element content to judge relevance. The authors also release StructDocRetrieval, a ~10k-word-per-document benchmark with HTML annotations. Across three embedding models, two datasets, and online A/B tests, SEAL consistently outperforms chunking, multi-CLS, SANTA, and raw-text contrastive learning; the abstract's headline gain (73.96% to 77.84% on BGE-M3) appears in Table 3 under MRR@10, not NDCG@10.

Core claim

The central claim is that explicitly teaching a retriever about document structure improves long structured document retrieval without changing the model architecture or inference cost. SAL makes the model treat a query as equally similar to a relevant document with HTML tags and to its tag-stripped plain-text version, forcing the representation to be invariant to markup while preserving content semantics. EAL randomly deletes structural tags on a subset of elements (10% works best) and asks the model to keep the masked document close to the query, so it learns element-level discriminative features. The released dataset, StructDocRetrieval, provides long, HTML-annotated documents and LLM-gen

What carries the argument

The two training objectives are the central mechanism. Structure-Aware Learning (SAL) uses contrastive pairs in which positive documents appear both with and without HTML tags, teaching the encoder to map structural markup and plain text into the same embedding region. Element-Aware Alignment (EAL) randomly masks the tags of a proportion of elements and contrasts the query with the masked relevant document, forcing the model to rely on unmasked elements and textual content to infer relevance. The framework continually fine-tunes PLMs such as BGE-M3, mE5-large, and bge-large-zh from a starting point of standard contrastive learning, and uses a dataset construction pipeline that converts docum

Load-bearing premise

The web-benchmark labels assume that an LLM-generated query for a crawled document is a valid relevance signal, with no human verification; if those queries share surface terms with the document, the measured gains may come from lexical matching rather than structural understanding.

What would settle it

Take the released StructDocRetrieval test set and create a subset of queries that have no lexical overlap with their target document. If SEAL's advantage over raw-text contrastive learning disappears on this subset, the reported gains would be explained by term matching, not structural semantics. Conversely, removing all HTML tags at test time and seeing the gain vanish would show the model relies on markup artifacts.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Retrieval systems can gain structural awareness by fine-tuning existing PLMs on HTML-tagged documents, with no new architecture or extra inference latency.
  • A benchmark with long documents (average >10,000 words) and explicit HTML structure is now available for apples-to-apples evaluation of structure-aware retrievers.
  • The training recipe—EAL before SAL, 10% element mask ratio—offers a concrete default that generalizes across encoder-only embedding models.
  • The gains hold in deployment: an online A/B test on ~30% of search traffic over 14 days shows higher PV CTR and UV CTR for SEAL than raw-text contrastive learning.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because relevance labels in StructDocRetrieval come from LLM-generated queries on crawled documents without human verification or graded relevance, the measured gains may partly reflect lexical overlap between query and source text rather than structural understanding; a human-judged subset would let readers separate these effects.
  • The abstract's headline number is mislabeled: 73.96→77.84 is reported as MRR@10 in Table 3, not NDCG@10. If the intended claim is NDCG@10, the correct table entry is 79.41→82.59, still a gain but a smaller one.
  • The method's reliance on paired structured/unstructured variants suggests it could be extended to other structured formats such as PDFs, JSON, or markdown, or combined with structure-aware pre-training rather than only fine-tuning.
  • The paper's own limitations note that evaluations are mostly Chinese-language; whether the structural signal transfers to English long documents remains untested.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SEAL, a contrastive learning framework for long structured document retrieval that combines two auxiliary objectives: Structure-Aware Learning (SAL), which contrasts tagged and untagged versions of relevant documents, and Element-Aware Alignment (EAL), which randomly removes structural tags from a fraction of elements and pulls the query toward the masked relevant document. The authors also release StructDocRetrieval, a benchmark of long HTML-formatted documents with LLM-generated queries. Experiments compare SEAL against raw-text, Chunk, MCLS, and SANTA baselines on an internal industrial dataset and the new benchmark across three encoder-based PLMs (mE5-large, bge-large-zh, BGE-M3), plus an online A/B test. The paper reports consistent gains, e.g., BGE-M3 MRR@10 improves from 73.96 to 77.84 on the industrial dataset and NDCG@10 from 79.41 to 82.59.

Significance. If the empirical claims hold, SEAL offers a simple, potentially effective recipe for injecting HTML structure into dense retrieval fine-tuning, and the released dataset could be a useful resource for an underexplored task. The paper has clear strengths: multiple backbone models, an online A/B deployment, a public dataset release, and a transparent description of the two losses. However, the current evidence is weakened by the headline metric mislabeling, selection of hyperparameters on the evaluation split, and reliance on unverified LLM-generated relevance labels for the released benchmark. These issues must be addressed before the claimed gains can be taken at face value.

major comments (4)
  1. [Abstract; Table 3] The abstract (and Section 1) states that SEAL achieves 'boosting NDCG@10 from 73.96% to 77.84% on BGE-M3.' These exact numbers do not appear under NDCG@10 in Table 3: for BGE-M3, MRR@10 is 73.96 for the base model and 77.84 with SEAL, while NDCG@10 is 79.41 → 82.59. The headline improvement is therefore labeled with the wrong metric. Please correct the abstract and all associated text, or report the true NDCG@10 gain. This is a central quantitative claim and must be accurate.
  2. [Section 4.3, Tables 6–7 vs Table 3] The mask ratio (Table 6) and training order (Table 7) are selected by comparing HitRate@5, MRR@10, and NDCG@10 on the industrial structured-document retrieval data. The paper describes only Train and Evaluation splits for the industrial dataset (Table 2) and does not mention a held-out validation split in Section 4.1. The best configuration from these tables (10% mask ratio, EAL1−SAL2) reproduces exactly the BGE-M3 w/ SEAL row in Table 3 (MRR@10 77.84, NDCG@10 82.59). Thus the main reported gain appears to be selected on the same split that is used for the headline evaluation, i.e., test-set selection. Please provide a separate validation split, nested evaluation, or otherwise demonstrate that hyperparameter selection does not inflate Table 3. Report multiple seeds/standard deviations.
  3. [Section 3.1, Table 2] StructDocRetrieval relevance labels are produced by 'LLM-powered query synthesis' from documents with no human relevance judgments described. Table 2 shows a single auto-generated query per document, and no details are given for negative document sampling or for filtering queries that simply echo document surface terms. Consequently, the gains on StructDocRetrieval (Table 4) could partly reflect lexical overlap rather than structural understanding. Please document the annotation protocol, include human verification or at least measure lexical overlap between queries and documents, and report how negatives were constructed.
  4. [Table 8, Section 4.3] Table 8 is titled 'The retrieval performance of GTE-Qwen2-1.5B on StructDocRetrieval,' but the rows are labeled mE5-large, +Chunk, +MCLS, +SEAL. The text says GTE-Qwen2-1.5B is used as an extended-context model. This mismatch makes the extended-context robustness claim unverifiable. Please correct the table/caption or the model name, and ensure all results in that subsection correspond to GTE-Qwen2-1.5B.
minor comments (5)
  1. [Section 4.3 / Figure 4] The sentence 'The experimental validity is strengthened through robustness testing 5 10 Days' appears broken; also the x-axis labels in Figure 4 are not clearly described.
  2. [References] Karpukhin et al. 2020a and 2020b are duplicated entries for the same paper (Dense Passage Retrieval). Please merge them.
  3. [Section 4.3] Table 6 says 'The Experimental results' and the table heading 'ratios (%)' is placed awkwardly; please proofread.
  4. [All experimental tables] No standard deviations or significance tests are reported. Given that several gains are small (e.g., some rows in Table 4), this would help assess stability.
  5. [Figure 3] The T-SNE visualizations are anecdotal; consider adding a quantitative separation metric or noting this as illustrative only.

Circularity Check

1 steps flagged

Headline industrial gain is selected on the evaluation split (test-set selection), not an unbiased prediction on an untouched holdout.

specific steps
  1. fitted input called prediction [Section 4.3 (Tables 6 and 7) vs Section 4.2 (Table 3)]
    "Based on these findings, we adopt the 10% masking ratio as the default configuration in the experimental section. ... Therefore, we make explorations for the impacts of different optimization strategies. As shown in Table 7, EAL1 − SAL2 achieves superior performance."

    The industrial dataset is described with only Train and Evaluation splits (Table 2); no validation split is mentioned. Section 4.3 selects the EAL mask ratio (Table 6) and the SAL/EAL training order (Table 7) by comparing HitRate@5, MRR@10, and NDCG@10 on industrial structured-document retrieval. The chosen rows—10% masking and EAL1−SAL2—reproduce exactly the BGE-M3 w/ SEAL row in Table 3 (HitRate@5 93.77, MRR@10 77.84, NDCG@10 82.59). Thus the headline improvement from 73.96 to 77.84 is the metric value of the configuration selected to maximize that very metric on the same split, not an unbiased evaluation on an untouched holdout. The claimed 'prediction' is the fitted input.

full rationale

The paper does not rely on a self-citation chain or a definitional derivation: SAL and EAL are empirical contrastive losses, and the core components are not justified by the authors' prior theorems. StructDocRetrieval and the online A/B test supply some independent evidence. However, the headline quantitative claim—abstract's 'NDCG@10 from 73.96% to 77.84%' (which actually corresponds to MRR@10 in Table 3)—is compromised by test-set selection. The mask ratio and training order are chosen by maximizing the same metrics on the same industrial Evaluation split that is then reported as the SEAL improvement; no separate validation split is described, and the best rows in Tables 6 and 7 exactly match the Table 3 SEAL row. This is a fitted-input-called-prediction circularity for the headline number. The StructDocRetrieval results are less affected (hyperparameters chosen on industrial and applied to web), and the online A/B test is an external check, so the circularity is partial rather than total. The mislabeled metric in the abstract is an accuracy issue, not a circularity. The paper's own Limitations acknowledge scope restrictions, which are not circularity concerns.

Axiom & Free-Parameter Ledger

1 free parameters · 5 axioms · 0 invented entities

The main free parameter is the element mask ratio, chosen on the evaluation set. The most consequential axioms are about label quality: clicks for the industrial set and LLM-synthesized queries for the web set. No invented entities are postulated.

free parameters (1)
  • Element mask ratio = 10%
    Chosen as optimal among {1,5,10,30,50}% on the industrial evaluation set (Table 6); selection on the test split inflates reported results.
axioms (5)
  • standard math InfoNCE-style contrastive loss on query-document pairs induces useful retrieval embeddings.
    Equations 2 and 4 rely on the standard dense-retrieval assumption that contrastive training on positive and negative pairs shapes a good embedding space.
  • domain assumption HTML tags are valid semantic structure that should influence document relevance in embedding space.
    Section 3.1 converts all documents to HTML and SAL/EAL operate on tags; if tags are noisy or semantically empty in target corpora, the method has no structural signal to exploit.
  • domain assumption An LLM-synthesized query for a crawled document is a relevant query for that document.
    Section 3.1 describes LLM-powered query synthesis with no human validation; Table 2 shows one query per document in StructDocRetrieval, so the benchmark's relevance labels are machine-generated.
  • domain assumption User clicks in the industrial dataset indicate query-document relevance and non-clicked documents are valid negatives.
    Section 4.1 adopts clicked documents as positives and non-clicked documents as negatives; click noise and position bias are not addressed.
  • ad hoc to paper Randomly deleting 10% of element tags while preserving text forces the model to learn fine-grained element alignment.
    Section 3.3's EAL objective; its benefit is only shown on author-created eval sets and the optimal ratio is chosen on those sets.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of SEAL: Structure and Element Aware Learning to Improve Long Structured Document Retrieval." pith.science (2026). https://pith.science/paper/3RQRZZ32

@misc{pith2026250820778,
  author       = {Pith},
  title        = {Pith review of: SEAL: Structure and Element Aware Learning to Improve Long Structured Document Retrieval},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/3RQRZZ32}},
  note         = {Machine review of arXiv:2508.20778}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

In long structured document retrieval, existing methods typically fine-tune pre-trained language models (PLMs) using contrastive learning on datasets lacking explicit structural information. This practice suffers from two critical issues: 1) current methods fail to leverage structural features and element-level semantics effectively, and 2) the lack of datasets containing structural metadata. To bridge these gaps, we propose \our, a novel contrastive learning framework. It leverages structure-aware learning to preserve semantic hierarchies and masked element alignment for fine-grained semantic discrimination. Furthermore, we release \dataset, a long structured document retrieval dataset with rich structural annotations. Extensive experiments on both released and industrial datasets across various modern PLMs, along with online A/B testing, demonstrate consistent performance improvements, boosting NDCG@10 from 73.96\% to 77.84\% on BGE-M3. The resources are available at https://github.com/xinhaoH/SEAL.

Figures

Figures reproduced from arXiv: 2508.20778 by Xinhao Huang, Ying Zhou, Yipeng Yu, Zeyi Wen, Zhibo Ren, Zulong Chen.

Figure 1
Figure 1. Figure 1: An overview of long structured documents [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Framework of SEAL. We first construct the used dataset, including HTML transformation, tag processing, and element masking. To guide PLMs to map both queries and structured documents in a unified embedding space, we introduce Structure-Aware Learning (SAL) to incorporate document structural information and Element-Aware Alignment (EAL) to enhance semantic representation. retrieval such as open-domain quest… view at source ↗
Figure 4
Figure 4. Figure 4: Online PV CTR and UV CTR over a two￾Week period. Method HitRate@5 MRR@10 NDCG@10 mE5-large 92.17 74.32 79.86 + Chunk 93.25 77.18 81.68 + MCLS 94.00 77.51 82.18 + SEAL 94.72 78.17 83.56 [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 3
Figure 3. Figure 3: Embedding visualization of original model [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

40 extracted references · 14 canonical work pages · 1 internal anchor

  1. [1]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRING...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2024 a . https://doi.org/10.18653/v1/2024.acl-long.172 Longbench: A bilingual, multitask benchmark for long context understanding . In ACL (1) , pages 3119--3137. Association for Computational Linguistics

    doi:10.18653/v1/2024.acl-long.172 2024

  4. [4]

    Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazheng Xu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2024 b . https://doi.org/10.48550/arXiv.2412.15204 Longbench v2: Towards deeper understanding and reasoning on realistic long-context multitasks . CoRR, abs/2412.15204

    Pith/arXiv arXiv doi:10.48550/arxiv.2412.15204 2024

  5. [5]

    Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. https://doi.org/10.18653/v1/2024.findings-acl.137 M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation . In ACL (Findings) , pages 2318--2335. Association for Computational Linguistics

    doi:10.18653/v1/2024.findings-acl.137 2024

  6. [6]

    Voorhees

    Nick Craswell, Bhaskar Mitra, Emine Yilmaz, Daniel Campos, and Ellen M. Voorhees. 2020. Overview of the TREC 2019 deep learning track. CoRR, abs/2003.07820

    Pith/arXiv arXiv 2020

  7. [7]

    Lu Dai, Hao Liu, and Hui Xiong. 2024. https://aclanthology.org/2024.emnlp-main.636 Improve dense passage retrieval with entailment tuning . In EMNLP (1) . Association for Computational Linguistics

    work page 2024

  8. [8]

    Jacob Devlin, Ming - Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/n19-1423 BERT: pre-training of deep bidirectional transformers for language understanding . In NAACL-HLT (1) , pages 4171--4186. Association for Computational Linguistics

    doi:10.18653/v1/n19-1423 2019

  9. [10]

    Shailja Gupta, Rajesh Ranjan, and Surya Narayan Singh. 2024. https://doi.org/10.48550/arXiv.2410.12837 A comprehensive survey of retrieval-augmented generation (RAG): evolution, current landscape and future directions . CoRR, abs/2410.12837

    Pith/arXiv arXiv doi:10.48550/arxiv.2410.12837 2024

  10. [11]

    Cheng - Ping Hsieh, Simeng Sun, Samuel Kriman, Shantanu Acharya, Dima Rekesh, Fei Jia, Yang Zhang, and Boris Ginsburg. 2024. https://doi.org/10.48550/arXiv.2404.06654 RULER: what's the real context size of your long-context language models? CoRR, abs/2404.06654

    Pith/arXiv arXiv doi:10.48550/arxiv.2404.06654 2024

  11. [12]

    Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, Xinrong Zhang, Zhen Leng Thai, Kai Zhang, Chongyi Wang, Yuan Yao, Chenyang Zhao, Jie Zhou, Jie Cai, Zhongwu Zhai, Ning Ding, Chao Jia, Guoyang Zeng, Dahai Li, Zhiyuan Liu, and Maosong Sun. 2024. https://doi.org/10.48550/arXiv.2404.06395 Min...

    Pith/arXiv arXiv doi:10.48550/arxiv.2404.06395 2024

  12. [13]

    Jui - Ting Huang, Ashish Sharma, Shuying Sun, Li Xia, David Zhang, Philip Pronin, Janani Padmanabhan, Giuseppe Ottaviano, and Linjun Yang. 2020. https://doi.org/10.1145/3394486.3403305 Embedding-based retrieval in facebook search . In KDD , pages 2553--2561. ACM

    arXiv 2020

  13. [14]

    Hamel Husain, Ho - Hsiang Wu, Tiferet Gazit, Miltiadis Allamanis, and Marc Brockschmidt. 2019. http://arxiv.org/abs/1909.09436 Codesearchnet challenge: Evaluating the state of semantic code search . CoRR, abs/1909.09436

    Pith/arXiv arXiv 2019

  14. [16]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick S. H. Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen - tau Yih. 2020 b . https://doi.org/10.18653/v1/2020.emnlp-main.550 Dense passage retrieval for open-domain question answering . In EMNLP (1) , pages 6769--6781. Association for Computational Linguistics

    doi:10.18653/v1/2020.emnlp-main.550 2020

  15. [17]

    Omar Khattab and Matei Zaharia. 2020. https://doi.org/10.1145/3397271.3401075 Colbert: Efficient and effective passage search via contextualized late interaction over BERT . In SIGIR , pages 39--48. ACM

    arXiv 2020

  16. [18]

    Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2025. https://openreview.net/forum?id=lgsyLSsDRe Nv-embed: Improved techniques for training llms as generalist embedding models . In ICLR . OpenReview.net

    work page 2025

  17. [19]

    Bohan Li, Hao Zhou, Junxian He, Mingxuan Wang, Yiming Yang, and Lei Li. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.733 On the sentence embeddings from pre-trained language models . In EMNLP (1) , pages 9119--9130. Association for Computational Linguistics

    doi:10.18653/v1/2020.emnlp-main.733 2020

  18. [20]

    Sen Li, Fuyu Lv, Taiwei Jin, Guli Lin, Keping Yang, Xiaoyi Zeng, Xiao - Ming Wu, and Qianli Ma. 2021 a . https://doi.org/10.1145/3447548.3467101 Embedding-based product retrieval in taobao search . In KDD , pages 3181--3189. ACM

    arXiv 2021

  19. [21]

    Xiaonan Li, Yeyun Gong, Yelong Shen, Xipeng Qiu, Hang Zhang, Bolun Yao, Weizhen Qi, Daxin Jiang, Weizhu Chen, and Nan Duan. 2022. https://doi.org/10.18653/v1/2022.emnlp-main.187 C ode R etriever: A large scale contrastive pre-training method for code search . In ACL , pages 2898--2910. Association for Computational Linguistics

    work page doi:10.18653/v1/2022.emnlp-main.187 2022

  20. [22]

    Xinze Li, Zhenghao Liu, Chenyan Xiong, Shi Yu, Yu Gu, Zhiyuan Liu, and Ge Yu. 2023 a . https://doi.org/10.18653/v1/2023.findings-acl.734 Structure-aware language model pretraining improves dense retrieval on structured data . In ACL (Findings) , pages 11560--11574. Association for Computational Linguistics

    work page doi:10.18653/v1/2023.findings-acl.734 2023

  21. [23]

    Xinze Li, Hanbin Wang, Zhenghao Liu, Shi Yu, Shuo Wang, Yukun Yan, Yukai Fu, Yu Gu, and Ge Yu. 2025. https://doi.org/10.1145/3695868 Building a coding assistant via the retrieval-augmented language model . ACM Trans. Inf. Syst. , 43(2):39:1--39:25

    work page doi:10.1145/3695868 2025

  22. [24]

    Yizhi Li, Zhenghao Liu, Chenyan Xiong, and Zhiyuan Liu. 2021 b . https://doi.org/10.1145/3471158.3472245 More robust dense retrieval with contrastive dual learning . In ICTIR , pages 287--296. ACM

    arXiv 2021

  23. [25]

    Zehan Li, Xin Zhang, Yanzhao Zhang, Dingkun Long, Pengjun Xie, and Meishan Zhang. 2023 b . https://doi.org/10.48550/arXiv.2308.03281 Towards general text embeddings with multi-stage contrastive learning . CoRR, abs/2308.03281

    Pith/arXiv arXiv doi:10.48550/arxiv.2308.03281 2023

  24. [26]

    Tri Nguyen, Mir Rosenberg, Xia Song, Jianfeng Gao, Saurabh Tiwary, Rangan Majumder, and Li Deng. 2016. https://ceur-ws.org/Vol-1773/CoCoNIPS\_2016\_paper9.pdf MS MARCO: A human generated machine reading comprehension dataset . In CoCo@NIPS, volume 1773 of CEUR Workshop Proceedings . CEUR-WS.org

    work page 2016

  25. [27]

    Jun Rao, Liang Ding, Shuhan Qi, Meng Fang, Yang Liu, Li Shen, and Dacheng Tao. 2023. https://ieeexplore.ieee.org/document/10102558 Dynamic contrastive distillation for image-text retrieval . IEEE Transactions on Multimedia, 25:8383--8395

    work page arXiv 2023

  26. [28]

    Jun Rao, Zepeng Lin, Xuebo Liu, Xiaopeng Ke, Lian Lian, Dong Jin, Shengjun Cheng, Jun Yu, and Min Zhang. 2025. https://aclanthology.org/2025.findings-acl.1079/ APT: improving specialist LLM performance with weakness case acquisition and iterative preference training . In ACL (Findings) , pages 20958--20980. Association for Computational Linguistics

    work page 2025

  27. [29]

    Jun Rao, Fei Wang, Liang Ding, Shuhan Qi, Yibing Zhan, Weifeng Liu, and Dacheng Tao. 2022. https://arxiv.org/pdf/2203.03853 Where does the performance improvement come from - a reproducibility concern about image-text retrieval . In SIGIR

    work page internal anchor Pith review Pith/arXiv arXiv 2022

  28. [30]

    Reddy, Llu \' s M \` a rquez, Fran Valero, Nikhil Rao, Hugo Zaragoza, Sambaran Bandyopadhyay, Arnab Biswas, Anlu Xing, and Karthik Subbian

    Chandan K. Reddy, Llu \' s M \` a rquez, Fran Valero, Nikhil Rao, Hugo Zaragoza, Sambaran Bandyopadhyay, Arnab Biswas, Anlu Xing, and Karthik Subbian. 2022. https://doi.org/10.48550/arXiv.2206.06588 Shopping queries dataset: A large-scale ESCI benchmark for improving product search . CoRR, abs/2206.06588

    Pith/arXiv arXiv doi:10.48550/arxiv.2206.06588 2022

  29. [31]

    Christopher Sciavolino, Zexuan Zhong, Jinhyuk Lee, and Danqi Chen. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.496 Simple entity-centric questions challenge dense retrievers . In EMNLP (1) , pages 6138--6148. Association for Computational Linguistics

    doi:10.18653/v1/2021.emnlp-main.496 2021

  30. [32]

    Dong Sun, Wenya Guo, Xumeng Liu, Ying Zhang, Zhaoxiang Hou, and Zengxiang Li. 2025. Zero-shot document retrieval with hybrid pseudo-document retriever. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1--5. IEEE

    work page 2025

  31. [33]

    Jiejun Tan, Zhicheng Dou, Wen Wang, Mang Wang, Weipeng Chen, and Ji - Rong Wen. 2025. https://doi.org/10.1145/3696410.3714546 Htmlrag: HTML is better than plain text for modeling retrieved knowledge in RAG systems . In WWW , pages 1733--1746. ACM

    arXiv 2025

  32. [34]

    Nandan Thakur, Nils Reimers, Andreas R \" u ckl \' e , Abhishek Srivastava, and Iryna Gurevych. 2021. https://datasets-benchmarks-proceedings.neurips.cc/paper/2021/hash/65b9eea6e1cc6bb9f0cd2a47751a186f-Abstract-round2.html BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models . In NeurIPS Datasets and Benchmarks

    work page 2021

  33. [35]

    Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. https://doi.org/10.48550/arXiv.2402.05672 Multilingual E5 text embeddings: A technical report . CoRR, abs/2402.05672

    Pith/arXiv arXiv doi:10.48550/arxiv.2402.05672 2024

  34. [36]

    Joty, and Steven C

    Yue Wang, Weishi Wang, Shafiq R. Joty, and Steven C. H. Hoi. 2021. https://doi.org/10.18653/v1/2021.emnlp-main.685 Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation . In EMNLP (1) , pages 8696--8708. Association for Computational Linguistics

    doi:10.18653/v1/2021.emnlp-main.685 2021

  35. [37]

    Benjamin Warner, Antoine Chaffin, Benjamin Clavié, Orion Weller, Oskar Hallström, Said Taghadouini, Alexis Gallagher, Raja Biswas, Faisal Ladhak, Tom Aarsen, Nathan Cooper, Griffin Adams, Jeremy Howard, and Iacopo Poli. 2024. https://arxiv.org/abs/2412.13663 Smarter, better, faster, longer: A modern bidirectional encoder for fast, memory efficient, and lo...

    Pith/arXiv arXiv 2024

  36. [38]

    Shitao Xiao, Zheng Liu, Peitian Zhang, and Niklas Muennighoff. 2023. https://doi.org/10.48550/arXiv.2309.07597 C-pack: Packaged resources to advance general chinese embedding . CoRR, abs/2309.07597

    Pith/arXiv arXiv doi:10.48550/arxiv.2309.07597 2023

  37. [39]

    Bennett, Junaid Ahmed, and Arnold Overwijk

    Lee Xiong, Chenyan Xiong, Ye Li, Kwok - Fung Tang, Jialin Liu, Paul N. Bennett, Junaid Ahmed, and Arnold Overwijk. 2021 a . https://openreview.net/forum?id=zeFrfgyZln Approximate nearest neighbor negative contrastive learning for dense text retrieval . In ICLR . OpenReview.net

    work page 2021

  38. [40]

    Wenhan Xiong, Xiang Lorraine Li, Srini Iyer, Jingfei Du, Patrick S. H. Lewis, William Yang Wang, Yashar Mehdad, Scott Yih, Sebastian Riedel, Douwe Kiela, and Barlas Oguz. 2021 b . https://openreview.net/forum?id=EMHoBG0avc1 Answering complex open-domain questions with multi-hop dense retrieval . In ICLR . OpenReview.net

    work page 2021

  39. [41]

    Junhan Yang, Zheng Liu, Chaozhuo Li, Guangzhong Sun, and Xing Xie. 2023. https://doi.org/10.18653/v1/2023.emnlp-main.223 Longtriever: a pre-trained long text encoder for dense document retrieval . In EMNLP , pages 3655--3665. Association for Computational Linguistics

    work page doi:10.18653/v1/2023.emnlp-main.223 2023

  40. [42]

    Wayne Xin Zhao, Jing Liu, Ruiyang Ren, and Ji - Rong Wen. 2024. https://doi.org/10.1145/3637870 Dense text retrieval based on pretrained language models: A survey . ACM Trans. Inf. Syst. , 42(4):89:1--89:60

    doi:10.1145/3637870 2024

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.