{"id":"e4c41be6-44d9-4e49-9440-f0daa391f9ae","arxiv_id":"2510.27313","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"An output is called novel when no semantically similar pretraining context can be retrieved; using this embedding-based measure, SmolLM models show longer-range reuse and instruction tuning increases novelty.","lead":"This paper measures whether an LLM writes something new by checking whether a retrieval system can find semantically similar text in the model's pretraining corpus. Applied to SmolLM models, the test suggests models reuse pretraining text over long spans and that instruction tuning makes generations more novel.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The novelty metric's core is unvalidated: if GIST+ColBERTv2 miss paraphrase matches, or the median aggregation hides a single attributable chunk, all three findings are artifacts of the measure rather than of model reuse.","rationale":"The paper's central contribution is an operational definition of novelty: an output is novel iff no semantically similar context exists in the pretraining corpus. For this to support the three findings, the retrieval+reranking pipeline must have high recall for similar contexts, and the summary statistic must not mask an attributable chunk. The reader identified the first as the weakest assumption; I agree and sharpen it: the pipeline is validated in Appendix A only by showing that low FAISS ranks are rarely reranked to top-1, which is a precision-side check, not a recall-side check. High precision among already-retrieved items does not tell us how many true matches were never retrieved. The paper's own Limitations section acknowledges dependence on the embedding model but provides no false-novel rate. The median aggregation in Algorithm 1 adds a second, distinct failure mode: a 500-token output with one memorized 50-token span would have one very high per-chunk ratio but a median that can still be below the human baseline. Thus the 'no close match' claim is not exactly what is tested. The proposed paraphrase-retrieval test measures both the recall and the decision-rule behavior that would separate the novelty hypothesis from measurement artifact. Given this, the reader's CONDITIONAL verdict is appropriate; no change. If the test passes, the framework gains substantial support; if it fails, the headline findings are about embedding geometry and aggregation rather than pretraining reuse. The released 20 TB of index artifacts and the open SmolLM setup are valuable for running this check, and the paper deserves credit for making the test feasible.","tokens_in":14251,"tokens_out":8860,"duration_ms":89127,"concrete_test":"Build a gold-standard 'attributable paraphrases' set: sample 500 chunks from the released SmolLM2 index; for each, have a human annotator rewrite it preserving meaning without lexical overlap, and treat each rewrite as a query to Algorithm 1 with k=100 and k=500. Since the original chunk is in the corpus and is semantically equivalent, the correct label is attributable. Report the false-novel rate (fraction with median R(k)<1), plus recall@100 of the original chunk. If the false-novel rate exceeds 5% or recall@100 is below 95%, the pipeline does not faithfully implement the abstract's definition, and the three findings cannot be trusted as evidence about model reuse.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that Algorithm 1's R(k)<1 decision operationalizes \"the pretraining corpus contains no semantically similar context\" (abstract, §3). For that to hold, (i) GIST/FAISS + ColBERTv2 must have high recall for genuinely similar contexts, and (ii) the median-over-chunks aggregation must preserve the \"no close match\" condition. Neither is established. §A checks only whether low FAISS ranks are promoted to reranked top-1 for SmolLM2 open-ended data; it does not measure recall against known paraphrase matches. The paper's Limitations (§5) concedes dependence on the embedding model but gives no miss-rate estimate. The aggregation in Algorithm 1 is also mismatched with the abstract: a single verbatim or paraphrased chunk inside a long output is hidden by the median, so an output containing a close corpus match can still be labelled novel. All three headline findings—long-span reuse, domain variation, instruction-tuning novelty—would be artifacts if the measure either misses paraphrase matches or the median washes out copied chunks.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes to measure LLM generation novelty by semantic un-attributability: an output is novel if the pretraining corpus contains no semantically similar context. They implement this with a FAISS index of GIST embeddings followed by ColBERTv2 reranking, and calibrate against human-written baseline text. Applying this to SmolLM and SmolLM2, they report three findings: (1) models draw on pretraining data over much longer spans than n-gram methods previously reported, (2) novelty varies by task domain, and (3) instruction tuning increases novelty. They release ~20 TB of corpus chunks and index artifacts.","tokens_in":14551,"tokens_out":3258,"duration_ms":31592,"significance":"If the metric is valid, the paper offers a scalable and style-robust alternative to lexical novelty measures and makes a useful conceptual inversion of training-data attribution. The released ~20 TB of corpus chunks and index artifacts are a substantial reproducibility contribution, and the pipeline is clearly specified. However, the central empirical claims rest on an unvalidated retrieval recall assumption and an aggregation choice that can hide attributable chunks; the contribution is therefore conditional on additional validation.","major_comments":[{"comment":"The abstract defines novelty as 'the pretraining corpus contains no semantically similar context,' but Algorithm 1 outputs N^(k) ← median(R^(k)). A single verbatim or semantically copied chunk inside a long output is hidden by the median, so an output containing a close corpus match can still be labeled novel. This mismatch is load-bearing for all three findings. Please either switch to a stricter aggregation (e.g., maximum over chunks or the fraction of chunks above a threshold) and re-run the analysis, or justify why the median is the appropriate operationalization of 'no close match.'","section":"§3, Algorithm 1"},{"comment":"The sufficiency check for n=100 only counts how often low FAISS ranks are promoted by ColBERTv2 to the reranked top-1 among the retrieved candidates. It does not measure recall against known semantically similar contexts. If the GIST embedding misses a true neighbor entirely, that neighbor never enters the top-100, so the check cannot detect the miss. The Limitations section (§5) concedes dependence on the embedding model but gives no miss-rate estimate. Without a recall evaluation on paraphrase ground truth (e.g., injecting known rephrased passages and measuring whether they are retrieved), the headline claims describe the embedding geometry rather than model reuse. Please add such a validation.","section":"§3.1, Appendix A"},{"comment":"The claim that 'models draw on pretraining data across much longer sequences than previously reported' requires a direct comparison to the prior n-gram baseline on the same generations and models. The paper never runs an n-gram novelty method (e.g., Rusty-DAWG from Merrill et al., 2024) on its own outputs. Without that comparison, the phrase 'than previously reported' is an unsupported cross-study statement. Please run the n-gram baseline and show where the two methods diverge.","section":"§4.2, claim (1) and Fig. 3"},{"comment":"Several per-cell sample sizes are extremely small, e.g., SmolLM-360M-Instruct on GSM8K has n=15, SmolLM-360M-Base n=20, and SmolLM-1.7B variants n=63. Figures 4 and 7 plot medians without confidence intervals or bootstrap uncertainty. The claims that 'smaller models are more novel' and that 'instruction tuning increases novelty' rely on curves that may be statistically indistinguishable. Please report uncertainty (e.g., bootstrap CIs) and avoid strong conclusions for cells with n < 50, or at least clearly mark low-n cells.","section":"§4.3, Table 1 and Fig. 4"},{"comment":"For GSM8K and TruthfulQA, the benchmark targets serve as the human baseline. These targets are public datasets and may themselves appear in the SmolLM/SmolLM2 pretraining corpus. If they do, the baseline similarity is inflated, making model outputs appear more novel than they are. The paper does not check whether the benchmark targets overlap with the pretraining data, nor does it exclude them from the index. Please report the overlap and, if present, use a held-out baseline or filter the index.","section":"§4.3, baseline construction"}],"minor_comments":[{"comment":"The paper says 'three-stage framework' in the abstract but 'two-stage retrieval pipeline' in Sections 1 and 3. Please standardize the terminology, e.g., counting index construction as Stage 0.","section":"Abstract/§1"},{"comment":"The y-axis label 'Similarity Score' is ambiguous; the left and right panels measure different quantities (n-gram overlap vs. embedding cosine). Please label each panel with the exact metric and range.","section":"Fig. 1"},{"comment":"For SmolLM2-Instruct 'unprompted' generation, the neutral cue 'Generate a text' changes the task. The claim that unprompted generation is less novel should be qualified, since the instruct models receive a prompt that may itself bias the output distribution.","section":"§4.2"},{"comment":"The reference 'Zhang et al., 2025' is incomplete: it reads 'Min-kURL https://arxiv.org/abs/2404.02936' with no title or venue. Please fix.","section":"References"},{"comment":"The notation R^(k) is defined as a set, but the output line takes its median. Clarify that R^(k) is a list of per-chunk ratios and that the median is taken over chunks.","section":"Algorithm 1"}],"recommendation":"major_revision","confidential_remarks":"The paper has a strong reproducibility artifact and a clear pipeline, but the central empirical claims hinge on validating that the retrieval pipeline indeed finds semantically similar contexts with high recall, and on an aggregation that is faithful to the 'no close match' definition. The current manuscript lacks both. I recommend major revision rather than rejection because the issues are addressable with additional experiments within the paper's scope."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The core idea here is genuinely nice: invert training-data attribution and define novelty as un-attributability, then turn novelty assessment into a retrieval problem that can run at pretraining scale. That reframing is worth taking seriously, and the practical work is real — the pipeline is clearly specified, the chunking/reranking design is sensible, and releasing ~20 TB of chunks and index artifacts is a serious reproducibility gesture. The SmolLM/SmolLM2 study is also new in scope and in its domain/instruction-tuning observations.\n\nThe soft spots are proportionate to the size of the claims. The load-bearing premise — that GIST embeddings plus FAISS top-100 plus ColBERTv2 reranking reliably detects \"no semantically similar context in the corpus\" — is never validated. Appendix A only shows that low FAISS ranks are rarely promoted to reranked top-1; it does not measure recall against known paraphrase matches, and there is no correlation with human novelty judgments. That matters because all three headline findings (long-span reuse, domain variation, instruction-tuning novelty) would be artifacts if the embedding model misses paraphrases or if style/length biases dominate. The abstract says \"the pretraining corpus contains no semantically similar context,\" but Algorithm 1 summarizes over chunks with the median; a single copied or closely paraphrased chunk inside a long output is hidden, so an output with a genuine verbatim match can still be labelled novel. That is a mismatch between the operational definition and the abstract's claim.\n\nThe paper also asserts \"much longer than previously reported\" without running the n-gram baselines it cites in the same setup, and the empirical sections report medians on small filtered subsets — n=15 in Table 1 for one model/domain — with no confidence intervals. The instruction-tuning claim leans on the assumption that embeddings are style-robust, but no control for style shift is provided. These are fixable, but they need to be fixed before the findings can be trusted.\n\nNone of this is fatal to the framework. The conceptual contribution and the artifacts are solid enough that the paper deserves a serious referee, not a desk reject. The right path is major revision: validate the metric against paraphrase ground truth and human judgments, add direct n-gram baseline comparisons, report uncertainty, and reconsider the aggregation so a single close match is not masked by the median. A reader working on TDA, memorization, or generalization evaluation will find this worth engaging; just not as a settled measurement tool yet.","headline":"A useful conceptual flip and substantial released artifacts, but the central semantic-similarity measure is unvalidated, so the empirical headlines should be read as descriptive of the pipeline, not of the models.","tokens_in":15034,"tokens_out":1692,"would_cite":true,"duration_ms":30088,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper defines an LLM output as novel when no semantically similar context exists anywhere in the pretraining corpus, and measures that at trillion-token scale with a retrieval pipeline.","keywords":["semantic novelty","un-attributability","training data attribution","text retrieval","ColBERTv2","GIST embeddings","instruction tuning","SmolLM"],"falsifier":"A concrete check: take a set of model generations, have humans decide for each whether the pretraining corpus contains a semantically equivalent passage, then compare those judgments to the pipeline's novelty scores. If agreement is near chance, or if rerunning the whole experiment with a different embedding model reverses the domain and instruction-tuning conclusions, the central claim collapses.","tokens_in":14162,"feed_emoji":"🔎","tokens_out":3196,"duration_ms":32010,"temperature":0.7,"pith_summary":"The paper tries to establish that novelty of an LLM output can be defined and measured as un-attributability: an output is novel if the pretraining corpus contains no semantically similar context. It builds a retrieval pipeline that scales to full pretraining corpora and shows that, by this measure, SmolLM models reuse training data across much longer spans than n-gram methods report, that novelty is strongly domain-dependent, and that instruction tuning genuinely increases novelty rather than only changing style. A sympathetic reader would care because it converts an intractable attribution problem into a tractable retrieval problem and offers a concrete, auditable way to study when models generalize as opposed to reuse.","feed_headline":"Semantic search measures LLM novelty at pretraining scale","feed_subtitle":"A retrieval pipeline finds whether an output resembles training data, revealing reuse that lexical checks miss.","key_machinery":"Un-attributability, an operational definition of semantic novelty: an output is novel if no semantically similar context exists in the pretraining corpus. It is realized by a two-stage retrieval pipeline — GIST embeddings indexed in a vector store for top-100 retrieval, then ColBERTv2 reranking at multiple chunk sizes — with scores normalized by query length and calibrated against human-written reference text. The pipeline converts a scale-restricted attribution question into a nearest-neighbor search that can run over trillion-token corpora.","core_discovery":"The central claim is the inversion of training-data attribution: instead of asking which training samples caused an output, ask whether any training sample is close enough to explain it. The test computes GIST embeddings of corpus chunks, retrieves the nearest 100 chunks, reranks them with ColBERTv2, and compares the best-match similarity against a human-written baseline; a generation is deemed novel when its similarity ratio falls below that baseline. Applied to SmolLM and SmolLM2, the claim produces three findings: reuse extends over longer sequences than lexical metrics show, novelty varies by domain, and instruction-tuned models produce more semantically unattributable text.","pith_inferences":["The same un-attributability test could serve as a data-contamination audit: outputs that score far above the baseline would flag corpus overlap even when phrased differently.","If the embedding model is changed, the novelty ordering across domains may shift; a stability check across embedders would determine how much of the result is measure-specific.","The method could extend to models whose corpora are closed by retrieving from public repositories or sub-corpora, but then novelty exclusions no longer refer to the true pretraining set.","A direct human validation study, asking annotators whether a retrieved corpus chunk is a paraphrase of a generation, would test whether the operational definition matches intuitive novelty."],"forward_implications":["N-gram and other lexical novelty measures systematically miss paraphrase-level reuse, so earlier estimates of how much models copy should be revisited.","Longer generated sequences are less attributable than short ones in unprompted generation, suggesting models generalize increasingly as generation proceeds.","Novelty is not a single model property: it varies by task domain, with factual-reasoning answers often more novel than math or rewriting answers.","Instruction tuning increases semantic novelty beyond stylistic changes, implying it teaches compositional or diversifying behavior.","Smaller models in the SmolLM2 family are more novel than their larger counterparts, consistent with greater semantic memorization in larger models."],"fun_headline_variants":["Semantic retrieval exposes LLM training-data reuse beyond exact matches","Semantic similarity finds LLM novelty that lexical checks miss","New test: semantic retrieval measures LLM generation novelty","LLM novelty via semantic search: catches paraphrased training-data reuse","Semantic retrieval at pretraining scale reveals long-range LLM reuse"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The whole measure assumes that GIST cosine similarity plus ColBERTv2 reranking faithfully captures 'semantically similar context' — if it misses paraphrases or is biased by style or length, all three findings describe embedding geometry rather than actual reuse.","fun_headline_variants_meta":{"raw":{"variants":["Semantic retrieval exposes LLM training-data reuse beyond exact matches","Semantic similarity finds LLM novelty that lexical checks miss","New test: semantic retrieval measures LLM generation novelty","LLM novelty via semantic search: catches paraphrased training-data reuse","Semantic retrieval at pretraining scale reveals long-range LLM reuse"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00064,"raw_usage":{"total_tokens":2770,"prompt_tokens":719,"completion_tokens":2051,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":463,"completion_tokens_details":{"reasoning_tokens":1976}},"tokens_in":463,"tokens_out":2051,"duration_ms":26049,"temperature":1.0,"reasoning_tokens":1976,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-04T06:58:49.391903+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A concrete check: take a set of model generations, have humans decide for each whether the pretraining corpus contains a semantically equivalent passage, then compare those judgments to the pipeline's novelty scores. If agreement is near chance, or if rerunning the whole experiment with a different embedding model reverses the domain and instruction-tuning conclusions, the central claim collapses.","supporting_citations":[],"review_version":1}