Pith. sign in

REVIEW 6 major objections 6 minor 28 references

Cross-encoder attention entropy plus majority vote removes more RAG chunk redundancy than pooled-vector filters, faster than cosine similarity.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-31 17:56 UTC pith:DK2NH3SF

load-bearing objection Useful RAG ingestion filter package with a real bench, but the 7× speed win over cosine is almost certainly an infrastructure artifact and the whole story rests on one dataset. the 6 major comments →

arxiv 2607.24332 v1 pith:DK2NH3SF submitted 2026-07-27 cs.CL cs.LG

Cross-Attention Calibrated Deduplication for Retrieval-Augmented Generation System

Phuong Le Huy , Nam H. Nguyen , Quan V. Dang This is my paper
classification cs.CL cs.LG
keywords retrieval-augmented generationchunk deduplicationcross-encoderattention entropynew information scorevector databaseinformation retrievalchunking strategies
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

RAG systems often index overlapping or repeated text chunks, which bloat the vector store and slow retrieval. Common fixes collapse each chunk to one embedding and drop near-duplicates by cosine similarity, but that pooled score can mistake shared topic for true redundancy. This paper proposes CACD: each new chunk is compared, token by token, to the K nearest already-kept chunks with a cross-encoder; a New Information Score from attention entropy measures how much of the new chunk the candidate fails to explain; and a majority vote across candidates decides keep or drop. On the full SQuAD 1.1 validation set across nine chunking strategies and eighteen configurations, CACD drops about 9.75% of chunks on average, shrinks the index more than the other filters tested, and finishes each configuration in roughly 51 seconds—far faster than cosine-similarity filtering and faster than NERExact—while holding precision and IoU above the other semantic baselines. The authors present this as an early single-dataset comparison, not a general claim.

Core claim

On SQuAD 1.1 across 18 chunking configurations, CACD—cross-encoder duplicate probability, attention-entropy New Information Score, and majority vote over K in-memory candidates—removes the highest average share of chunks (9.75%), yields the smallest mean index (27.13 MB), and ingests faster than the other semantic filters tested (51s vs 70s NERExact and 357s cosine similarity), with precision and IoU above those two semantic baselines.

What carries the argument

New Information Score (NIS): normalized average Shannon entropy of how tokens of a kept candidate attend back to tokens of the new chunk in the cross-encoder’s final-layer attention; low NIS means the candidate explains the new chunk (redundancy), high NIS means unexplained content. Combined with p_dup thresholds and majority vote over K candidates (plus parent-child, header, and length guards).

Load-bearing premise

The method assumes that cross-encoder attention entropy is a trustworthy signal of whether one chunk truly explains another’s content, rather than an artifact of the model, topic overlap, or length—and that thresholds tuned on this one dataset generalize.

What would settle it

Rerun the same 18 chunking configurations on a second, stylistically different corpus (or ablate NIS so decisions use only p_dup and majority vote): if drop rate, index size, and precision/IoU no longer beat Similarity and NERExact, or if NIS no longer tracks controlled paraphrase-overlap, the central claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Chunking strategies that deliberately create structured overlap (e.g., hierarchical parent–child) stand to lose the most redundant index mass under CACD.
  • Strategies that already emit short, mostly disjoint chunks will see little change in storage or retrieval metrics.
  • In-memory exact top-K plus batched cross-encoder scoring can be wall-clock cheaper than naive all-pairs cosine filtering at SQuAD-scale indexes.
  • Discarding whole chunks voted DROP still risks losing small unique fragments; partial-merge or rewrite policies would be a direct next design step the paper flags.
  • Cross-encoder choice and fine-tuning are presented as levers to close the small recall gap versus near-no-op filters.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • If NIS tracks true residual information, the same attention-entropy idea could gate incremental updates when streaming new documents into a live RAG index, not only offline ingestion.
  • Majority vote over K neighbors is a cheap robustness layer that other embedding-only deduplicators could adopt without switching to cross-encoders.
  • A controlled study that injects known paraphrase rates and measures NIS calibration curves would turn the single-dataset early comparison into a transferable decision rule.
  • Length and header guards hint that production dedup needs document-structure awareness; pure vector thresholds alone will keep misfiring on contextualized or hierarchical chunkers.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

6 major / 6 minor

Summary. The paper proposes CACD, a chunk-deduplication filter for RAG ingestion pipelines. Each new chunk is embedded and compared by exact in-memory search against the pool of kept chunks (Stage 1); the K nearest candidates are scored by a pretrained cross-encoder yielding a duplicate probability p_dup plus a "New Information Score" (NIS) computed from the normalized entropy of final-layer cross-attention (Stage 2, Eqs. 1–2); a majority vote over the K candidates with three guards decides keep/drop (Stage 3). On the full SQuAD 1.1 validation set across 18 chunking configurations (nine strategies × two sizes), CACD is compared to five filters (NoFilter, ExactNorm, MinHashLSH, Similarity, NERExact) on Precision/Recall/IoU, storage, ingestion time, and drop rate. CACD reports the highest average drop rate (9.75%), smallest index (27.13 MB), and faster ingestion (51.01s) than the other semantic baselines, with Precision/IoU above Similarity and NERExact but below the near-pass-through filters, and the lowest Recall of all methods. The authors disclose the single-dataset limitation and release code for both the benchmark and the method.

Significance. If the results hold, the paper offers two useful contributions: (1) a controlled, like-for-like benchmark of five previously un-compared chunk filters across nine chunking strategies, with public code for both the benchmark and the method — a genuinely reproducible artifact; and (2) a plausible mechanism (token-level cross-encoder comparison plus majority vote) for distinguishing true duplicates from same-topic chunks, a real pain point in RAG ingestion. The writing is unusually candid: the recall cost of aggressive dropping, the flat spots in Table I, and the single-dataset scope are all stated plainly. However, the headline speed advantage is currently an implementation artifact rather than a method property, the NIS construct is unvalidated, and no ablation isolates the contributions of the three components, so the empirical support is narrower than the abstract suggests.

major comments (6)
  1. [§IV-A, Table II; Abstract; §III-B] The speed claims ('7× faster than cosine-similarity filtering'; '27% faster than NERExact') compare implementations, not methods. Algorithmically, Similarity does strictly less work than CACD (both embed and search the kept pool; CACD additionally runs K=5 cross-encoder passes per chunk). Table II itself shows the artifact: NoFilter, which still embeds all ~10k chunks, takes 32.16s, while Similarity takes 356.70s — an ~11× jump for adding cosine comparisons that cost milliseconds, and CACD, doing that retrieval plus thousands of cross-encoder passes, finishes in 51.01s. §III-B admits the authors' own earlier external-vector-store Stage 1 was 'slowed down mostly by per-query storage overhead,' which is the likely explanation for the Similarity baseline's number; the paper never states how the baselines' storage/retrieval was implemented. The paper should either document and equalize basel
  2. [§III-C, Eqs. (1)–(2); §III-E] NIS's central premise — that high final-layer attention entropy of B's tokens over A means 'new information' and low entropy means 'explained' — is assumed, not validated. Table I probes p_dup against synthetic overlap but says nothing about NIS. There is a plausible failure mode in the opposite direction: for unrelated pairs, cross-encoder attention often concentrates on special tokens ([CLS]/[SEP]) or punctuation, yielding low entropy and thus reading genuinely new content as redundant. The paper needs some evidence that NIS correlates with actual content novelty — e.g., NIS values on the same synthetic overlap ladder used in Table I, or a small human-labeled set of duplicate/topical/novel pairs. As it stands, the mid-band keep/drop rule in §III-E rests on an untested interpretation of attention geometry.
  3. [§III (all); §IV-A] No ablation isolates the three advertised components. The contributions claim that p_dup, NIS, and the majority vote 'work together,' but Table II only reports the full pipeline. Table I even shows p_dup alone roughly tracks cosine similarity, which raises the question of how much of CACD's performance over the Similarity baseline comes from the cross-encoder alone versus NIS and voting. At minimum, report: (a) cross-encoder + p_dup single-best-candidate only; (b) + majority vote without NIS; (c) full CACD. Without this, the paper cannot attribute the gains to its novel parts.
  4. [§III-E vs. §V (limitations)] The threshold justification is internally inconsistent and arithmetically dubious. §III-E states τ_high/τ_low are 'not hand-picked' and follow from a cost-sensitive cutoff [24], but under the stated symmetric costs (c_FP = c_FN = 1), Elkan's formula gives a decision threshold of 0.5, not 0.8/0.2 — the derivation connecting symmetric costs to that pair is never shown. The same paragraph calls τ_NIS = 0.8 'the midpoint of the normalized entropy scale from Eq. 2,' but Eq. 2's range is [0,1], whose midpoint is 0.5. §V then concedes the thresholds 'were chosen by comparing a handful of settings on this one dataset,' contradicting 'not hand-picked.' Please either present the actual cost-sensitive derivation (with the implied cost ratio that yields 0.8/0.2) or describe the thresholds honestly as dataset-tuned and report sensitivity to them.
  5. [§IV (Configuration and metrics); Table II] The retrieval-quality protocol and models are underspecified for the central comparison. (a) 'Precision, Recall, and IoU measure retrieval quality after filtering [13]' never defines them here: what top-k, what chunk-to-question relevance mapping, and how are they averaged? (b) Neither the embedding model nor the cross-encoder used for Table II is named; §V lists five candidate cross-encoders that 'provide better outcomes' without saying which produced the reported numbers or what they were compared against. (c) No seeds, runs, or variability are reported, yet headline margins are small (Precision 0.3818 vs. 0.3745 for Similarity; Recall gap 0.0056). With per-configuration results presumably deterministic given fixed models, at least state that, and give per-configuration tables or confidence intervals over configurations so the reader can judge whether the differences exceed noise.
  6. [§I; §IV] Drop rate is treated as a benefit, and deduplication is motivated by answer quality and query-time retrieval speed ('can even hurt answer quality'), but the evaluation is entirely ingestion-side: no downstream QA/answer-quality measurement and no query-time latency measurement. CACD has both the highest drop rate and the lowest Recall in Table II, so 'removes the most' is not self-evidently a win. The conclusion is mostly careful about this, but the abstract's framing ('removes 9.75% of chunks... much higher than exact-match filters') reads drop rate as a virtue. Either add a small downstream evaluation (e.g., retrieval recall against SQuAD gold passages at fixed k is already computed — report answer-level F1 with a fixed reader) or soften the framing throughout.
minor comments (6)
  1. [§IV-A vs. §V] Inconsistent recall-gap figure: §IV-A says CACD's recall is 'at most 0.015 below NEREXACT' while the conclusion says 'at most 0.0056.' From Table II, 0.7057 − 0.7001 = 0.0056; the 0.015 in §IV-A appears to be an error.
  2. [§II-A] Typo: 'AdaptiveSentenceLengroups more short sentences' — missing space/word break ('AdaptiveSentenceLen groups').
  3. [References] Several citations look mismatched or weakly relevant: [12] is attributed to a GoPenAI blog post but the URL points to an NVIDIA developer blog; [16] (K*-Means clustering) is cited for k-nearest-neighbor retrieval in Stage 1; [22] (Contextual RAG) is cited for majority voting; [19] is an arXiv tutorial on Shannon entropy where Cover & Thomas [26] is already in the list. Please audit.
  4. [Abstract / §IV-A] '27% faster than the strongest baseline, NERExact' is ambiguous — NERExact is not the strongest baseline by the paper's own metrics (it is only the second-fastest semantic one). Rephrase as 'the next-fastest semantic baseline.'
  5. [Table II] Bold is applied 'per column independent of method,' which is fine, but bolding the largest Drop % implicitly rewards aggressive dropping; consider a footnote stating that higher drop rate is not inherently better and is reported for characterization only.
  6. [Figures 2–8] The schematic figures are helpful, but Figure 7 would benefit from axis/value annotations matching Eq. (1) notation (p(i|j)), and Figure 6 should state which cross-encoder produced the example attention matrix.

Circularity Check

0 steps flagged

No significant circularity: CACD is an empirical filter whose reported drop rate, storage, timing, and retrieval metrics are independent measurements, not quantities forced by definition or self-citation.

full rationale

This is a systems/evaluation paper, not a first-principles derivation. CACD is defined operationally (cross-encoder p_dup, attention-entropy NIS in Eqs. 1–2, majority vote over K candidates, plus guards), then run on SQuAD chunk streams and compared to five baselines on drop %, index MB, ingestion time, and post-filter Prec/Rec/IoU. Those metrics are not algebraic rearrangements of the method’s inputs: drop rate is the fraction of chunks the pipeline discards; Prec/Rec/IoU are retrieval quality after filtering; timing and storage are wall-clock and size measurements. Thresholds (τ_high=0.8, τ_low=0.2 from a symmetric cost-sensitive cutoff; τ_NIS=0.8 as normalized-entropy midpoint / SemDeDup-style target) are hyperparameters, and §V openly notes they were chosen by trying settings on this one dataset—standard eval-set tuning risk, not a claim that a fitted parameter is a novel prediction. Citations (cross-encoders, Shannon entropy, cost-sensitive learning, SemDeDup, HNSW, SQuAD) are external background; there is no load-bearing self-citation uniqueness theorem, no ansatz smuggled in as forced math, and no renaming of a known closed-form result as a derivation. Mild methodological caveats (same-dataset threshold choice; implementation-dependent timing vs Similarity) belong under correctness/experimental design, not circularity. Score 0 with empty steps is the honest finding.

Axiom & Free-Parameter Ledger

7 free parameters · 6 axioms · 2 invented entities

The central claim rests on standard IR/ML tools (embeddings, cross-encoders, entropy, majority vote) plus paper-specific decision thresholds and the interpretive leap that attention entropy equals unexplained content. Free parameters dominate the decision surface; invented entity is mainly NIS as an operational score. No new physical entities.

free parameters (7)
  • τ_high (duplicate probability drop threshold) = 0.8
    Cost-sensitive cutoff under symmetric FP/FN costs; set to 0.8 and used as hard Drop vote in Stage 3.
  • τ_low (duplicate probability keep threshold) = 0.2
    Symmetric counterpart of τ_high; p_dup ≤ τ_low votes Keep.
  • τ_NIS (New Information Score threshold) = 0.8
    Midpoint of normalized entropy scale / SemDeDup-style retention target; decides mid-band votes.
  • K (number of nearest kept candidates) = 5
    Controls retrieval width and majority-vote panel; fixed to 5 in experiments.
  • Length-aware guard threshold and NIS floor = 300 chars; NIS floor 0.3
    Chunks longer than 300 characters protected unless NIS < 0.3 against deciding candidate; hand-set failure-case guards.
  • c_FP = c_FN cost ratio = 1:1
    Symmetric misclassification costs that produce τ_high/τ_low; chosen for this paper’s experiments, not learned from labeled duplicates.
  • Similarity/MinHash baseline thresholds = 0.8 (reported)
    Baseline operating points (e.g., cosine 0.8, MinHashLSH 0.8) affect comparative drop rates; part of the evaluation surface.
axioms (6)
  • ad hoc to paper Final-layer multi-head attention of a pretrained cross-encoder, when converted to per-token entropy over the other span, indicates whether content is explained or new.
    Core interpretive assumption behind NIS in §III-C Eqs. (1–2); not independently validated with labeled partial-overlap data beyond a small synthetic p_dup table.
  • domain assumption Exact cosine top-K over an in-memory embedding pool is a sufficient candidate generator for duplicate detection at the studied scale.
    Stage 1 (§III-B) never scores non-neighbors; duplicates that embed far apart would be missed.
  • domain assumption Majority vote over K candidates is more robust than deciding from the single highest p_dup neighbor.
    Stated motivation in §III-E; standard ensemble intuition, not proved for this task.
  • domain assumption Parent-child hierarchical overlaps are intentional and should be excluded from duplicate scoring, while sibling children may be true duplicates.
    Parent-child guard in §III-D; encoding of hierarchical chunking semantics.
  • standard math Shannon entropy and standard cross-encoder pairwise classification are valid mathematical/ML tools for the scoring stage.
    Uses classical entropy and off-the-shelf cross-encoders as black boxes (§III-C).
  • domain assumption SQuAD 1.1 validation passages are a meaningful proxy for RAG chunk-redundancy behavior across chunking strategies.
    Entire empirical claim rests on this one corpus (§IV); authors note non-generality in abstract and §V.
invented entities (2)
  • New Information Score (NIS) no independent evidence
    purpose: Scalar in [0,1] estimating how much of one chunk is not explained by another, used when p_dup is ambiguous.
    Defined as mean normalized attention entropy from B’s tokens onto A; central paper-specific scoring object. Independent evidence is limited to construction and downstream filter metrics, not external NIS calibration.
  • CACD three-stage keep/drop policy (retrieval + cross-encoder/NIS + majority vote with guards) independent evidence
    purpose: Online filter deciding whether each new chunk enters the persistent index.
    The method is the product; it is an algorithm, not a latent physical entity. Listed to mark what is introduced versus prior single-vector filters.

pith-pipeline@v1.2.0-grok45-kimik3 · 15358 in / 4362 out tokens · 88215 ms · 2026-07-31T17:56:10.360408+00:00 · methodology

0 comments
read the original abstract

Common chunking strategies in Retrieval-Augmented Generation (RAG) systems often create redundant chunks. These redundant chunks make the vector database bigger and slow down retrieval. A common fix is cosine-similarity thresholding. This method reduces each chunk to a single vector, then compares vectors using a similarity score. But a single vector can lose the fine-grained, token-level detail needed to tell a true duplicate apart from a chunk that just shares the same topic. We propose Cross-Attention Calibrated Deduplication (CACD). CACD checks each new chunk against an in-memory pool of chunks already kept, using a cross-encoder instead of a single pooled vector. This keeps token-level detail all the way to the final comparison. CACD combines three parts: the cross-encoder comparison itself, a New Information Score (NIS) that measures how much of a chunk is not explained by a candidate already kept, and a majority vote across several candidates rather than a single best match. NIS is calculated from the attention entropy of the cross-encoder. We tested CACD against five existing filtering methods, nine chunking strategies, and 18 configurations, all on the full SQuAD 1.1 validation set. In our experiments, CACD removes 9.75% of chunks on average. This drop rate is close to other semantic-level methods, and much higher than exact-match filters, which barely remove anything. In these experiments, CACD also processes each configuration in 51.0 seconds on average, about 27% faster than the strongest baseline, NERExact (69.6s), and about 7x faster than cosine-similarity filtering (356.7s). These results come from a single dataset, so we present them as an early comparison, not a general claim. Code for the baseline evaluation and for CACD is available at https://github.com/lehuyphuong/rag_bench and https://github.com/lehuyphuong/cacd_dedup.

Figures

Figures reproduced from arXiv: 2607.24332 by Nam H. Nguyen, Phuong Le Huy, Quan V. Dang.

Figure 1
Figure 1. Figure 1: A typical RAG pipeline. Documents are chunked, embedded, and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Schematic overview of the nine chunking strategies compared in [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: CACD’s three-stage pipeline. A new chunk is scored against the K [PITH_FULL_IMAGE:figures/full_fig_p003_4.png] view at source ↗
Figure 7
Figure 7. Figure 7: How a single token j of B attends to the tokens of A in the two extreme cases behind NIS. When attention concentrates on one token of A (left), H(j) is small and A is treated as explaining j. When attention spreads evenly across A (right), H(j) is large and j is treated as carrying information A does not have. exactly uniform over A, and averaged over all |B| tokens j of B to give NIS(B | A) = 1 |B| X |B| … view at source ↗
Figure 8
Figure 8. Figure 8: Stage 3: each candidate is compared against the decision thresholds [PITH_FULL_IMAGE:figures/full_fig_p005_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

28 extracted references · 14 linked inside Pith

  1. [1]

    Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,

    P. Lewis, E. Perez, A. Piktus, F. Petroni, V . Karpukhin, N. Goyal, H. K¨uttler, M. Lewis, W. Yih, T. Rockt ¨aschel, S. Riedel, and D. Kiela, “Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks,” inAdvances in Neural Information Processing Systems (NeurIPS), vol. 33, 2020, pp. 9459–9474

    2020

  2. [2]

    RAG Pipeline Diagram: A Beginner’s Guide to How It Works,

    Trang, “RAG Pipeline Diagram: A Beginner’s Guide to How It Works,”Designveloper Blog, Aug. 18, 2025. [Online]. Available: https:// www.designveloper.com/blog/rag-pipeline-diagram/. [Accessed: Jul. 14, 2026]

    2025

  3. [3]

    Deduplicating Training Data Makes Language Models Better,

    K. Lee, D. Ippolito, A. Nystrom, C. Zhang, D. Eck, C. Callison- Burch, and N. Carlini, “Deduplicating Training Data Makes Language Models Better,” inProc. 60th Annual Meeting of the Association for Computational Linguistics (ACL), 2022, pp. 8424–8445

    2022

  4. [4]

    R2D2: Reducing Redundancy and Duplication in Data Lakes,

    R. Shah, K. Mukherjee, A. Tyagi, S. K. Karnam, D. Joshi, S. Bhosale, and S. Mitra, “R2D2: Reducing Redundancy and Duplication in Data Lakes,”Proc. ACM Manag. Data, vol. 1, no. 4, Art. no. 268, Dec. 2023

    2023

  5. [5]

    Rethinking Chunk Size For Long-Document Retrieval: A Multi-Dataset Analysis,

    S. R. Bhat, M. Rudat, J. Spiekermann, and N. Flores-Herr, “Rethinking Chunk Size For Long-Document Retrieval: A Multi-Dataset Analysis,” arXiv preprint arXiv:2505.21700, 2025

    Pith/arXiv arXiv 2025

  6. [6]

    Evaluating Chunking Strategies for Retrieval-Augmented Generation on Academic Texts,

    V . J. J. Kreileder, J. Reisinger, and A. Fischer, “Evaluating Chunking Strategies for Retrieval-Augmented Generation on Academic Texts,” arXiv preprint arXiv:2607.01852, 2026

    Pith/arXiv arXiv 2026

  7. [7]

    Is Semantic Chunking Worth the Computational Cost?

    R. Qu, R. Tu, and F. S. Bao, “Is Semantic Chunking Worth the Computational Cost?” inFindings of the Association for Computational Linguistics: NAACL 2025, Albuquerque, NM, USA, Apr. 2025, pp. 2155–2177

    2025

  8. [8]

    Chunking Methods on Retrieval-Augmented Generation – Effectiveness Evalu- ation Against Computational Cost and Limitations,

    M. ´Smigielski, M. Rajkowski, M. Zbrocki, M. Bernacki-Janson, K. Kunicki, J. Godziszewska, M. Piasecki, and K. Wojtasik, “Chunking Methods on Retrieval-Augmented Generation – Effectiveness Evalu- ation Against Computational Cost and Limitations,”arXiv preprint arXiv:2606.00881, 2026

    Pith/arXiv arXiv 2026

  9. [9]

    Query-Aware Adaptive Chunking for RAG in Mental Health and Well-Being Policy Question Answering,

    P. Kondapalli, M. Z. R. Saimon, S. Das Polok, G. S. Chakraborty, M. A. Abrar, and S. Batra, “Query-Aware Adaptive Chunking for RAG in Mental Health and Well-Being Policy Question Answering,” in2026 5th International Conference on Innovative Practices in Tech- nology and Management (ICIPTM), Noida, India, 2026, pp. 1–6, doi: 10.1109/ICIPTM69057.2026.11465692

    arXiv 2026

  10. [10]

    H-RAG at SemEval-2026 Task 8: Hierarchical Parent–Child Retrieval for Multi- Turn RAG Conversations,

    P. Elchafei, H. Emam, M. Alansary, M. Swain, and M. Schedl, “H-RAG at SemEval-2026 Task 8: Hierarchical Parent–Child Retrieval for Multi- Turn RAG Conversations,”arXiv preprint arXiv:2605.00631, 2026

    Pith/arXiv arXiv 2026

  11. [11]

    Reconstructing Context: Evaluating Ad- vanced Chunking Strategies for Retrieval-Augmented Generation,

    J. Singh and C. Merola, “Reconstructing Context: Evaluating Ad- vanced Chunking Strategies for Retrieval-Augmented Generation,”arXiv preprint arXiv:2504.19754, 2025

    Pith/arXiv arXiv 2025

  12. [12]

    Different Chunking Strategies to Improve Retrieval- Augmented Generation (RAG),

    S. Hadawle, “Different Chunking Strategies to Improve Retrieval- Augmented Generation (RAG),”GoPenAI Blog, Oct. 17,

  13. [13]

    Reducing Redundancy in Retrieval-Augmented Generation through Chunk Filtering,

    D. Berdyugina, A. Cohen, and Y . Rioual, “Reducing Redundancy in Retrieval-Augmented Generation through Chunk Filtering,”arXiv preprint arXiv:2604.24334, 2026

    Pith/arXiv arXiv 2026

  14. [14]

    Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs,

    Yu. A. Malkov and D. A. Yashunin, “Efficient and Robust Approximate Nearest Neighbor Search Using Hierarchical Navigable Small World Graphs,”IEEE Transactions on Pattern Analysis and Machine Intelli- gence, vol. 42, no. 4, pp. 824–836, 2020

    2020

  15. [15]

    k-Nearest Neighbour Classifiers: 2nd Edition (with Python Examples),

    P. Cunningham and S. J. Delany, “k-Nearest Neighbour Classifiers: 2nd Edition (with Python Examples),”arXiv preprint arXiv:2004.04523, 2020

    Pith/arXiv arXiv 2004

  16. [16]

    K*-Means: A Parameter-free Clustering Algorithm,

    L. Mahon and M. Lapata, “K*-Means: A Parameter-free Clustering Algorithm,”arXiv preprint arXiv:2505.11904, 2025

    Pith/arXiv arXiv 2025

  17. [17]

    Understand- ing Matching Mechanisms in Cross-Encoders,

    M. Vast, B. Van Cooten, L. Soulier, and B. Piwowarski, “Understand- ing Matching Mechanisms in Cross-Encoders,” inProc. Workshop on Explainability in Information Retrieval (WExIR25), SIGIR 2025, New York, NY , USA, 2025

    2025

  18. [18]

    Can Cross Encoders Produce Useful Sentence Embeddings?

    H. Ananthakrishnan, J. Dolby, H. Kokel, H. Samulowitz, and K. Srinivas, “Can Cross Encoders Produce Useful Sentence Embeddings?”arXiv preprint arXiv:2502.03552, 2025

    Pith/arXiv arXiv 2025

  19. [19]

    Understanding Shannon’s Entropy metric for Informa- tion,

    S. Vajapeyam, “Understanding Shannon’s Entropy metric for Informa- tion,”arXiv preprint arXiv:1405.2061, 2014

    Pith/arXiv arXiv 2061

  20. [20]

    Understanding Random Forests: From Theory to Practice,

    G. Louppe, “Understanding Random Forests: From Theory to Practice,” Ph.D. dissertation, Univ. of Li `ege, Li`ege, Belgium, 2014,arXiv preprint arXiv:1407.7502

    Pith/arXiv arXiv 2014

  21. [21]

    Bagging Supervised Autoen- coder Classifier for Credit Scoring,

    M. Abdoli, M. Akbari, and J. Shahrabi, “Bagging Supervised Autoen- coder Classifier for Credit Scoring,”arXiv preprint arXiv:2108.07800, 2021

    Pith/arXiv arXiv 2021

  22. [22]

    Contextual Retrieval Augmented Generation,

    L. Yang, Q. Chen, Y . Zhang, Y . Shi, R. Pan, A. M. Lipani, and W. Y . Wang, “Contextual Retrieval Augmented Generation,”arXiv preprint arXiv:2309.09564, 2023

    Pith/arXiv arXiv 2023

  23. [23]

    SQuAD: 100,000+ Questions for Machine Comprehension of Text,

    P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang, “SQuAD: 100,000+ Questions for Machine Comprehension of Text,” inProc. 2016 Conf. Empirical Methods in Natural Language Processing (EMNLP), 2016, pp. 2383–2392

    2016

  24. [24]

    The Foundations of Cost-Sensitive Learning,

    C. Elkan, “The Foundations of Cost-Sensitive Learning,” inProc. 17th Int. Joint Conf. Artificial Intelligence (IJCAI), 2001, pp. 973–978

    2001

  25. [25]

    SemDeDup: Data-efficient learning at web-scale through semantic deduplication,

    A. Abbas, K. Tirumala, D. Simig, S. Ganguli, and A. S. Morcos, “SemDeDup: Data-efficient learning at web-scale through semantic deduplication,”arXiv preprint arXiv:2303.09540, 2023

    Pith/arXiv arXiv 2023

  26. [26]

    T. M. Cover and J. A. Thomas,Elements of Information Theory, 2nd ed. Hoboken, NJ, USA: Wiley-Interscience, 2006

    2006

  27. [27]

    cross-encoder (Sentence Transformers – Cross- Encoders),

    Sentence Transformers, “cross-encoder (Sentence Transformers – Cross- Encoders),”Hugging Face, n.d. [Online]. Available: https://huggingface. co/cross-encoder/models. [Accessed: Jul. 26, 2026]

    2026

  28. [2025]

    Available: https://developer.nvidia.com/blog/ finding-the-best-chunking-strategy-for-accurate-ai-responses/

    [Online]. Available: https://developer.nvidia.com/blog/ finding-the-best-chunking-strategy-for-accurate-ai-responses/. [Accessed: Jul. 14, 2026]

    2026