Lexical non-learning hashes match near-duplicates well, while BGE-based quantized embeddings better preserve rewritten scientific similarity, under a shared ranking protocol on CSFCube and RELISH.
Jesse Kornblum
3 Pith papers cite this work. Polarity classification is still indexing.
3
Pith papers citing it
citation-role summary
background 2
citation-polarity summary
years
2026 3roles
background 2polarities
background 2representative citing papers
Merlin achieves byte-exact deduplication of text at up to 8.7 GB/s using SIMD-optimized hashing, reducing LLM context sizes by 13.9-71% with no data loss.
Byte-exact deduplication reduces RAG context size by 0.16% to 80.34% across three regimes with zero measurable quality regression per multi-vendor LLM evaluation.
citing papers explorer
-
H3D: Benchmarking Unsupervised Text Hashing for Fine-Grained Document Deduplication
Lexical non-learning hashes match near-duplicates well, while BGE-based quantized embeddings better preserve rewritten scientific similarity, under a shared ranking protocol on CSFCube and RELISH.
-
Merlin: Deterministic Byte-Exact Deduplication for Lossless Context Optimization in Large Language Model Inference
Merlin achieves byte-exact deduplication of text at up to 8.7 GB/s using SIMD-optimized hashing, reducing LLM context sizes by 13.9-71% with no data loss.
-
Byte-Exact Deduplication in Retrieval-Augmented Generation: A Three-Regime Empirical Analysis Across Public Benchmarks
Byte-exact deduplication reduces RAG context size by 0.16% to 80.34% across three regimes with zero measurable quality regression per multi-vendor LLM evaluation.