{"id":"3801aaae-419c-4679-ba1f-ebd70d2b70b0","arxiv_id":"2607.10020","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"Chaining winnowed fingerprints by position offset detects shared text fragments with AUC ≈ 0.99 on a synthetic containment benchmark spanning Wikipedia, ArXiv, and 50M web documents.","lead":"FindMyText is an open-source tool that checks whether a query text appears, in whole or in part, inside a large corpus by chaining together matching word-level fingerprints. On a new synthetic benchmark over Wikipedia, ArXiv, and a 50-million-page web crawl, it separates true copies from merely similar texts with AUC ≈ 0.99, which is directly useful for auditing LLM training data for copyrighted content.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Synthetic positives are always contiguous, lightly edited fragments; real OCR/cleaning gaps could break the chain and invalidate the 0.99 AUC transfer.","rationale":"The central claim is that FindMyText detects partially included texts robustly to discrepancies introduced by web-crawling pipelines. The only evidence is the synthetic benchmark. I examined the generation procedure: positives always contain one contiguous edited fragment; negatives never contain a long enough fragment. The chain algorithm's tolerance is small (τ_off=10). Real-world noise, especially OCR and boilerplate removal, often creates gaps and shifts that exceed this tolerance. Thus the reported near-perfect separation may be an artifact of the benchmark's restricted edit operations, not of the method's robustness. The proposed test directly measures performance under chain-breaking transformations. If the test fails, the paper's conclusion that the method is 'robust to discrepancies' must be weakened; the method remains a solid near-verbatim detector but not a general containment detector. I agree with the reader's assessment and therefore recommend no change to the CONDITIONAL verdict.","tokens_in":15008,"tokens_out":4872,"duration_ms":54551,"concrete_test":"Build a second benchmark from the same corpora: sample positive pairs as in Sec. 5.1, then additionally perturb the selected fragment d'_i by (a) inserting a random unrelated block of length 100/500/2000 chars at a random position, and (b) deleting a contiguous block of length 50/200/500 chars from it, while keeping the SW oracle score ≥1000 for the surviving common region. Re-run the chain method with default τ_pos=30, τ_off=10 and report AUC-ROC and P@R at R=0.9/0.99. If AUC drops below 0.95 for any condition with L≥100, the robustness claim (§1, §6) is not supported and the paper should qualify the method as limited to near-contiguous copies.","verdict_should_be":"UNCHANGED","load_bearing_attack":"FindMyText's headline result — AUC≈0.99 on the synthetic benchmark (Sec. 5.4, Table 1) — is only as strong as the benchmark's coverage of realistic corruption. In Sec. 5.1, positive instances are created by applying a fixed menu of character-level normalizations (de-hyphenation, case/ligature changes) and then concatenating the edited fragment with unrelated filler; the process stops as soon as the Smith-Waterman score exceeds T_pos=1000. This guarantees the shared portion remains a single, nearly contiguous segment of about half a page. The chain score (Eq. 2) connects shared fingerprints with gaps ≤ τ_pos=30 chars and offset drift ≤ τ_off=10 chars. Real preprocessing of web-crawled corpora, which the paper explicitly motivates (Sec. 1), includes OCR errors, boilerplate insertion/removal, and document segmentation. A single 100-character insertion inside the copied text, a deleted paragraph, or a layout-induced reordering shifts δ(h) beyond τ_off and splits the chain. The resulting clusters may fall below κ=5, yielding a false negative. The benchmark contains no positive instance with such discontinuities, so the reported P@R=90% of 0.998–1.00 does not demonstrate robustness to exactly the transformations the system claims to handle. The problem is not that the method is wrong on the benchmark; it is that the benchmark's construction excludes the chain-breaking cases that determine real-world recall.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper presents FindMyText, an open-source Python toolkit for detecting whether a query text appears wholly or partially in a corpus. The method extracts winnowed character-level fingerprints, indexes them in a disk-based inverted index, and scores each candidate document by the size of the largest position-coherent 'chain' of shared fingerprints (Eq. 2). This chain score is compared with shared-fingerprint counts, BM25, and dense retrieval on a synthetic benchmark built from Wikipedia, ArXiv, and the HPLT web corpus. The central claim is that the chain-based method discriminates positive from negative instances with AUC-ROC around 0.99 and Precision@Recall=90% above 0.98 on all three corpora, while baselines fail (Table 1). The authors also provide a web interface and release the toolkit under an MIT license.","tokens_in":15276,"tokens_out":2766,"duration_ms":35903,"significance":"If the reported results transfer to real-world settings, FindMyText would be a practically valuable tool for copyright-compliance screening and training-data provenance analysis, because it scales to tens of millions of documents with sub-second queries and explicitly identifies contiguous reused passages rather than merely scoring overall similarity. The paper ships an open-source implementation and a reproducible synthetic benchmark, and the core chain-clustering idea is a natural, well-motivated extension of winnowing. The main limitation is that the evaluation's ground truth is generated by a synthetic procedure whose positive instances are, by construction, a single lightly edited contiguous fragment. Therefore the headline numbers demonstrate that the method detects exactly the kind of near-verbatim segment the benchmark instantiates, but they do not yet establish robustness to the full range of preprocessing transformations that the paper motivates in Sec. 1 (OCR noise, boilerplate insertion/removal, segmentation, and reordering).","major_comments":[{"comment":"The benchmark's positive-instance construction is the load-bearing element for the headline AUC≈0.99, and it excludes exactly the chain-breaking cases that matter for real corpora. A positive is created by taking one consecutive fragment, applying small character-level edits until the Smith-Waterman score exceeds T_pos=1000, and concatenating unrelated filler around it. Because the chain score connects fingerprints only when gaps are ≤ τ_pos=30 characters and the offset drift is ≤ τ_off=10 characters, a single inserted paragraph, a deleted sentence block, or an OCR-induced gap inside the copied segment will split the chain and can push the largest cluster below κ=5. The benchmark contains no positive instance with such discontinuities, so the reported P@R=90% of 0.998–1.00 does not demonstrate robustness to the very transformations (boilerplate removal, document segmentation, cleaning) d","section":"§5.1, Eq. (2)"},{"comment":"An AUC-ROC of exactly 0.000 for dense retrieval on Wikipedia is a degenerate result that cannot support the paper's conclusion that the baselines 'are unable to disentangle' positives and negatives. AUC=0 means every positive is ranked below every negative, which is not a typical failure mode of a document-embedding retriever; it indicates that the baseline is being applied in a way that is systematically inverted for this benchmark (e.g., using whole-document similarity when the benchmark negatives are topic-matched near-paraphrases and the positives are a short copied span embedded in unrelated filler). Since the dense-retriever results are unavailable for ArXiv and HPLT, the comparison is also incomplete. The authors should either configure the dense baseline to operate on passage-level or windowed embeddings with a proper thresholding/ranking protocol, or explicitly state that the ba","section":"Table 1, Dense Retrieval row"},{"comment":"The chain parameters τ_pos=30, τ_off=10, and κ=5 are calibrated on a validation set generated by the same synthetic procedure used for the test set, and no sensitivity analysis is reported. Because the positive examples are constructed to be nearly contiguous, the chain score is essentially measuring whether the input has a long unbroken run of shared fingerprints; the benchmark does not provide evidence about how the method degrades when τ_pos or κ are mismatched to the actual noise level. In particular, the paper does not report a null model or empirical distribution of chain scores for unrelated documents, so it is unclear how the method avoids false positives in a corpus with, say, reused boilerplate or common long n-grams. I ask the authors to add (i) a sweep over τ_pos, τ_off, and κ on the validation set, (ii) a false-positive analysis on random non-overlapping documents from the s","section":"§5.2, §3.2, Table 1"},{"comment":"The scalability claim is supported only by query latency (≈450 ms for 50.7M documents). Index construction time, disk footprint, memory-mapped index size, and query throughput under concurrency are not reported. Since the paper's stated contribution includes 'scaling to large web-crawled datasets,' the lack of index-construction measurements makes it difficult for a reader to judge whether the approach is practical at the intended scale. Please report wall-clock index building time for each corpus, index size on disk, and, if possible, the peak memory usage.","section":"§4, §5.4"}],"minor_comments":[{"comment":"Several typos and spacing issues: 'FindMyTextextends' (missing space), 'thiry-fourth' in the Broder reference, and inconsistent use of 'd′i' vs 'd_i'. A light proofreading pass is recommended.","section":"§1, Abstract"},{"comment":"The dense retrieval baseline is listed as '/' for ArXiv and HPLT. If the authors retain the baseline, they should either compute embeddings for those corpora or explicitly mark the baseline as not evaluated, rather than leaving an ambiguous slash.","section":"Table 1, note"},{"comment":"The benchmark examples are useful, but the 'positive' example appears to contain a very long insertion of garden-blog content before the copied Simpsons paragraph. This is actually a good stress case, yet it is not representative of the automated generation described in §5.1. Clarify whether such long prepended unrelated spans are part of the standard positive generation or a hand-constructed illustrative example, and whether the chain method still detects the copied span when the unrelated content is longer than the copied span.","section":"Appendix C"},{"comment":"Embedding a live password ('EMNLP2026') in the paper is not standard practice; if the web interface is meant to be publicly accessible for review, provide an anonymous/unauthenticated link or a clearly temporary password. Also, the GitHub URL is given but no repository DOI or version number is provided; a versioned release would improve reproducibility.","section":"Appendix A"}],"recommendation":"major_revision","confidential_remarks":"The paper is a solid systems contribution with a clear method and a reproducible open-source implementation, but the current evaluation is too tightly coupled to the synthetic benchmark to support the strong robustness and scalability claims in the abstract and conclusion. The dense-retrieval AUC of 0.000 should be treated as a red flag that the baseline comparison is not yet meaningful. I would be willing to review a revision that adds discontinuous-positive experiments and a more careful baseline configuration."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear X,\n\nQuick take: this is a solid engineering paper with a genuinely useful scoring function. The chain detection—grouping shared winnowing fingerprints in (query position, offset) space with thresholds τ_pos=30 and τ_off=10, then taking the largest cluster size—is a neat, well-specified mechanism that clearly separates the synthetic positives from negatives (AUC ≈ 0.99 across Wikipedia, ArXiv, and 50M HPLT docs). That result is real for the benchmark as constructed, and the SW-oracle benchmark itself is a reasonable, disclosed way to generate hard positives and negatives. The open-source release and the scale demonstration (~450 ms per query on 50M documents) are concrete assets.\n\nThe soft spots are about generalization, not about the core method being broken. The main one: the positive instances in the benchmark are all contiguous fragments that have gone through light character-level edits (case, hyphenation, ligatures) and then been concatenated with filler. No positive ever has an insertion or deletion inside the copied span. Because the chain tolerance is 30 characters of position gap and 10 characters of offset drift, a single 100-character insertion, a deleted paragraph, or a layout-induced reordering will split the chain and can push the largest cluster below κ=5. The paper explicitly motivates OCR, boilerplate removal, and segmentation as the kinds of noise it is built for, but the benchmark never exercises those. The stress-test note is right: the headline numbers do not transfer to that noise distribution.\n\nTwo smaller issues. The dense-retrieval baseline reports an AUC of 0.000 on Wikipedia—a perfect anti-correlation. That went unexplained in the paper, and it either signals a bug or makes the comparison look worse than it is. The authors should fix or explain it. Also, the hyperparameters are tuned on a validation set generated by the same procedure as the test set, and the evaluation uses a single benchmark draw; \"the methods are deterministic\" doesn't justify omitting error bars over the random generation. This is minor for an engineering paper, but it does cap the strength of the quantitative claims.\n\nThe citation gap on plagiarism/text-reuse work with positional fingerprint filtering is real; the 'novel mechanism' framing overshoots. The chain clustering is still a clean addition.\n\nBottom line: the paper is worth a serious referee. It solves a real problem—efficient near-verbatim containment detection for training-dataset audits—and it does what it claims on its own benchmark. The revisions should add a stress test with internal gaps and a real-corpus validation (even a small one with hand-labeled positives), and clean up the baseline anomaly. It is not a paradigm shift, but it is a competent contribution that belongs in the conversation.\n\nBest,\n[You]","headline":"The chain-clustering score is a clean, working idea that beats its baselines on its own benchmark, but the benchmark never gives a positive a gap inside the copied fragment—so the robustness claims for real crawled corpora are overreaching.","tokens_in":15922,"tokens_out":3673,"would_cite":true,"duration_ms":41337,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FindMyText claims that grouping position-offset-aligned fingerprint matches into chains detects text containment at ~0.99 AUC-ROC across Wikipedia, ArXiv, and a 50.7M-document web crawl, at ~450 ms per query.","keywords":["text containment","document fingerprinting","winnowing","near-duplicate detection","inverted index","positional clustering","copyright detection","web-scale search"],"falsifier":"Create a synthetic or real example where a query shares two consecutive copied blocks with an inserted 100-character paragraph between them (so Smith-Waterman still scores above the positive threshold but no single fingerprint chain connects the blocks). If FindMyText's chain score for this pair falls below its positive threshold, the method's robustness claim is contradicted.","tokens_in":14737,"feed_emoji":"🔍","tokens_out":5354,"duration_ms":57317,"temperature":0.7,"pith_summary":"The paper presents FindMyText, a method to decide whether a query text appears—in whole or in part—inside a large text corpus, a task that matters for verifying whether copyrighted material made it into an LLM's training data. The central claim is that this 'text containment' can be detected far more reliably by looking for chains of matching fingerprints than by counting shared fingerprints or comparing embeddings. Each shared fingerprint is placed at a point (position in the query, offset relative to the corpus document); connected clusters of such points indicate a contiguous reused passage, and the size of the largest cluster is the containment score. On a synthetic benchmark built with a Smith-Waterman oracle, this chain score achieves AUC-ROC around 0.99 on Wikipedia, ArXiv, and a 50.7-million-document web crawl, while semantic-similarity baselines perform near chance. The method runs in about 450 ms per query on the largest corpus using a disk-based inverted index.","feed_headline":"Fingerprint chains spot copied text in 50M docs","feed_subtitle":"Position-aligned fingerprint chains beat similarity baselines across Wikipedia, ArXiv, and a 50.7M-doc crawl.","key_machinery":"The central mechanism is the fingerprint-chain score s_chain(q,d_i)=max_k |C_k|, built from winnowed, position-stamped hashes. Winnowing selects the minimum hash in each sliding window, guaranteeing that any shared substring of length k+w−1 yields a common fingerprint. The novelty is clustering shared fingerprints in (query position, offset) space with tolerance thresholds (τ_pos, τ_off), which converts a set of independent coincidental hashes into evidence of a contiguous, aligned copy.","core_discovery":"The core discovery is that an explicit, geometrically coherent sequence of shared fingerprints—rather than a raw count—separates true near-verbatim reuse from mere topical similarity. For a query q and a corpus document d_i, FindMyText winnows both texts into k-gram hashes, records character-level positions, and for each shared hash h computes the offset δ(h) = p_{d_i}(h) − p_q(h). Shared hashes are plotted as points (p_q(h), δ(h)); a reused passage appears as a nearly horizontal run of points. Two points are connected if their query positions differ by at most τ_pos = 30 characters and their offsets differ by at most τ_off = 10 characters; clusters of at least κ = 5 fingerprints are kept, a","pith_inferences":["The (position, offset) clustering is essentially a Hough-transform–style vote for a linear alignment; a natural extension is to return multiple disjoint chains and their boundaries, giving a full containment map rather than a single max score.","The tight tolerances (τ_pos=30, τ_off=10 characters) suggest the method is tuned for clean text; OCR-laden or heavily segmented web content with larger local insertions might need adaptive thresholds or multi-scale winnowing—a testable hypothesis.","Because negatives in the benchmark are constructed adversarially (permute, paraphrase, boilerplate), the reported AUC may understate real-world ease if actual non-contained documents are less similar; conversely, if real copies undergo heavier edits, performance could degrade—this asymmetry is worth probing."],"forward_implications":["If the chain score is right, near-verbatim reuse can be detected at web scale with a disk-based inverted index: ~450 ms per query on 50.7M documents.","The method distinguishes containment from similarity: BM25 and dense retrieval score near chance on the benchmark, while the chain method stays above 0.98 AUC-ROC.","Robustness covers common cleaning edits—case changes, hyphenation, ligatures, boilerplate insertion—but not paraphrase or large reorderings.","The approach yields localizable evidence (which passages match), which is useful for copyright-compliance audits of training corpora.","The synthetic benchmark itself is a contribution: a reproducible recipe for evaluating containment methods with Smith-Waterman as oracle."],"fun_headline_variants":["Fingerprint chains expose copied text at scale","Position-linked hashes find near-verbatim reuse","Chained fingerprints beat similarity baselines","FindMyText: detect text containment in 50M docs","Explicit fingerprint chains track copied passages"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The reported accuracy transfers to real corpora only if the synthetic benchmark's edit menu and Smith-Waterman thresholds faithfully represent the noise actually introduced by web crawling and cleaning.","fun_headline_variants_meta":{"raw":{"variants":["Fingerprint chains expose copied text at scale","Position-linked hashes find near-verbatim reuse","Chained fingerprints beat similarity baselines","FindMyText: detect text containment in 50M docs","Explicit fingerprint chains track copied passages"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000485,"raw_usage":{"total_tokens":2205,"prompt_tokens":692,"completion_tokens":1513,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":436,"completion_tokens_details":{"reasoning_tokens":1452}},"tokens_in":436,"tokens_out":1513,"duration_ms":11147,"temperature":1.0,"reasoning_tokens":1452,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-02T07:24:24.434224+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Create a synthetic or real example where a query shares two consecutive copied blocks with an inserted 100-character paragraph between them (so Smith-Waterman still scores above the positive threshold but no single fingerprint chain connects the blocks). If FindMyText's chain score for this pair falls below its positive threshold, the method's robustness claim is contradicted.","supporting_citations":[],"review_version":2}