{"id":"f49eec2b-6769-401d-8b89-2d21ab7680d6","arxiv_id":"2606.05787","paper_version":1,"verdict":"UNVERDICTED","confidence":"LOW","novelty_score":7.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"SentinelRAG embeds synthetic fictitious knowledge into RAG databases at 0.1% rate to enable reliable watermark detection with p < 10^{-5} and low false positives across tested datasets.","lead":"The paper presents SentinelRAG, a watermarking method that inserts style-consistent fictitious knowledge entries into RAG databases for copyright protection. These entries enable owner-only detection via targeted probes while avoiding interference with normal queries.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.3","headline":"Core claim rests on synthetic entries remaining unretrieved by any legitimate query distribution, yet this is asserted without reported coverage of semantically adjacent real queries.","rationale":"The reader's weakest_assumption directly names the same separation condition that must hold for the headline statistical claims to be meaningful. No other internal inconsistency (e.g., in the reported p-values or injection rate) is visible from the abstract, and the full-text details would be needed only to quantify how well the experiments already tested that assumption. Hence the UNVERDICTED stance is unaffected.","tokens_in":1655,"tokens_out":367,"duration_ms":20309,"concrete_test":"Sample 10 000 additional queries from each of the four datasets by taking real entities and generating paraphrases or near-neighbor questions (e.g., via the same embedding model); measure sentinel retrieval rate under the exact retrieval settings used in the paper. If the rate exceeds 0.1% on any dataset, the negligible-interference claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"SentinelRAG detection at p<10^{-5} with 0.1% injection requires that owner probes reliably surface the sentinels while no natural user query ever does. The abstract states the entries describe \"fictitious entities\" and are \"unlikely to be retrieved,\" but provides no quantitative bound on how close their embeddings sit to real entities in the same domain. If any real query vector lies inside the retrieval radius of a sentinel (possible when style-consistent generation still produces plausible names, relations, or facts), the false-positive or interference rate rises. The reported \"negligible interference\" and \"reduced false detection rate\" therefore depend on an untested separation that the experimental query sets may not have stressed.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.3","summary":"The manuscript proposes SentinelRAG, a watermarking framework for RAG databases that injects style-consistent fictitious knowledge entries describing fictitious entities at a 0.1% rate. The central claim is that these entries are unlikely to be retrieved by legitimate queries yet reliably triggered by owner-known probes, yielding statistically significant detection (p < 10^{-5}) across four datasets (2.9k to 8.8M documents), reduced false detection rates, and negligible interference relative to prior methods.","tokens_in":1829,"tokens_out":386,"duration_ms":33513,"significance":"If the separation between fictitious and real entities holds under realistic query distributions, the method could provide a practical, low-pollution approach to RAG copyright protection. The low injection rate and reported statistical significance are strengths; however, the result depends on an unverified embedding-space separation that is load-bearing for the false-positive and interference claims.","major_comments":[{"comment":"Abstract: the claims of p < 10^{-5} detection and 'negligible interference' with 'reduced false detection rate' rest on the assumption that style-consistent fictitious entries lie outside the retrieval radius of all legitimate queries; no quantitative bound (e.g., embedding distances or coverage of semantically adjacent real queries) is supplied to support this separation.","section":"Abstract"},{"comment":"Abstract and experimental description: the reported results lack details on experimental setup, controls, query sets used to test interference, or full data, preventing verification that the statistically significant outcomes are not artifacts of the chosen test distributions.","section":"Abstract"}],"minor_comments":[{"comment":"The abstract states results on four datasets but does not name their domains or sizes beyond the range 2.9k–8.8M.","section":null}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":{"model":"grok-4.3","summary":"We thank the referee for the detailed feedback. The comments correctly identify areas where the current manuscript version would benefit from additional quantitative support and experimental transparency. We will revise the paper to address both points directly.","responses":[{"response":"We agree that the abstract (and the current version) does not supply explicit quantitative bounds on embedding-space separation. In the revision we will add a dedicated analysis section reporting (i) cosine-distance distributions between sentinel entries and the nearest real documents, (ii) retrieval rates for a constructed set of semantically adjacent legitimate queries, and (iii) the resulting empirical coverage of the retrieval radius. These additions will directly support the false-positive and interference claims.","revision_made":"yes","referee_comment":"[Abstract] Abstract: the claims of p < 10^{-5} detection and 'negligible interference' with 'reduced false detection rate' rest on the assumption that style-consistent fictitious entries lie outside the retrieval radius of all legitimate queries; no quantitative bound (e.g., embedding distances or coverage of semantically adjacent real queries) is supplied to support this separation."},{"response":"The full manuscript already describes the four datasets and the 0.1 % injection protocol, but we acknowledge that query-set construction, interference-test controls, and statistical procedures are not presented at the level of detail needed for independent verification. In the revision we will expand the experimental section with explicit descriptions of the legitimate query corpora, the probe sets, the exact statistical test used for p < 10^{-5}, and any additional controls. We will also release the query-generation code and the precise sentinel-entry templates to allow reproduction.","revision_made":"yes","referee_comment":"[Abstract] Abstract and experimental description: the reported results lack details on experimental setup, controls, query sets used to test interference, or full data, preventing verification that the statistically significant outcomes are not artifacts of the chosen test distributions."}],"tokens_in":1282,"tokens_out":420,"duration_ms":26574,"standing_objections":[]},"desk_editor":{"model":"grok-4.3","letter":"The paper's main contribution is a watermarking method that adds a tiny fraction of made-up knowledge entries about nonexistent entities, written to match the style of the real database. This sidesteps the misinformation problem of fabricating links between actual entities and the fragility of lexical watermarks that paraphrasing can erase.\n\nIt does a clean job on the reported experiments: statistically significant detection below p=10^{-5} at 0.1% injection across four datasets from thousands to millions of documents, with lower false positives than the baselines and almost no measurable effect on ordinary queries.\n\nThe soft spot is the load-bearing assumption that these synthetic entries will never surface under legitimate query distributions. The abstract and results treat this as given because the entities are fictitious, but there is no reported measurement of embedding distances to real entities in the same domain or stress tests with semantically adjacent queries. Without that, the claimed negligible interference and reduced false detection rate rest on an untested separation.\n\nThe work is aimed at people building or protecting retrieval-augmented systems who need a practical ownership signal. A reader focused on data security in AI pipelines would find the idea worth examining.\n\nIt deserves peer review. The method is distinct enough and the detection numbers are sharp enough that referees should see the full experimental details and any additional controls on query overlap.","headline":"SentinelRAG introduces fictitious but style-matched entries for RAG watermarking and reports strong detection at low injection rates, yet the separation from real queries lacks quantitative checks.","tokens_in":2256,"tokens_out":346,"would_cite":false,"duration_ms":17903,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.3","headline":"SentinelRAG protects RAG databases by injecting fictitious knowledge entries that trigger only on owner-known probes.","keywords":["RAG database","watermarking","copyright protection","synthetic knowledge","retrieval augmented generation","false positive reduction","ownership detection","knowledge base security"],"falsifier":"A test showing that a significant number of legitimate queries retrieve the injected fictitious entries, or that the targeted probes no longer reliably detect the presence of the watermarks after redistribution.","tokens_in":2565,"feed_emoji":"🔐","tokens_out":649,"duration_ms":26201,"temperature":0.7,"pith_summary":"The paper introduces SentinelRAG to address unauthorized redistribution of proprietary RAG databases. It embeds style-consistent entries about fictitious entities that normal queries rarely retrieve yet respond to secret probes held only by the owner. This setup claims statistically significant detection at injection rates of 0.1 percent across datasets from thousands to millions of documents while cutting false detections relative to earlier methods and leaving legitimate queries largely unaffected. A sympathetic reader would care because RAG systems rely on large proprietary knowledge bases whose copying is otherwise hard to prove without introducing misinformation or fragile markers.","feed_headline":"Fictitious entries protect RAG databases at 0.1% injection","feed_subtitle":"Sentinel knowledge triggers detection only on secret probes, lowering false positives compared to prior watermarking.","key_machinery":"The injection of synthetic sentinel knowledge describing fictitious entities, which serves as a triggerable marker for ownership verification without affecting normal retrieval.","core_discovery":"By embedding style-consistent but fictitious knowledge entries into the RAG database, SentinelRAG enables the data owner to detect unauthorized redistribution through targeted probes while keeping the false detection rate low and interference with legitimate queries negligible. Experiments on datasets from 2.9k to 8.8M documents show statistically significant detection with p less than 10 to the minus 5 at a 0.1 percent injection rate, outperforming prior methods in false positive reduction.","pith_inferences":["If the fictitious entries blend too well, some users might still query them by chance in specialized domains.","This method could apply to protecting other types of proprietary knowledge bases used in search or recommendation systems.","Future work might test the method against paraphrasing attacks that target the style of the entries.","Combining this with other protection layers could strengthen overall security for RAG systems."],"forward_implications":["Owners can verify if their database has been copied by checking if secret probes retrieve the sentinel entries.","The approach works at injection rates as low as 0.1 percent while achieving high statistical significance.","False detection rates are lower than existing watermarking techniques.","Legitimate user queries experience negligible interference.","Results hold across multiple dataset sizes from thousands to millions of documents."],"fun_headline_variants":["SentinelRAG adds synthetic entries for RAG database protection","Fictitious knowledge enables secret probe detection at 0.1 percent","Low rate fakes yield significant RAG watermark detection p less than 1e-5","Style consistent synthetics protect large RAG databases from redistribution"],"cache_read_input_tokens":64,"weakest_assumption_plain":"That knowledge about fictitious entities will almost never be retrieved by ordinary user queries but will always respond to the owner's secret probes.","fun_headline_variants_meta":{"raw":{"variants":["SentinelRAG adds synthetic entries for RAG database protection","Fictitious knowledge enables secret probe detection at 0.1 percent","Low rate fakes yield significant RAG watermark detection p less than 1e-5","Style consistent synthetics protect large RAG databases from redistribution"]},"model":"grok-4.3","cost_usd":0.004439,"raw_usage":{"total_tokens":2191,"prompt_tokens":616,"num_sources_used":0,"completion_tokens":75,"cost_in_usd_ticks":44387000,"prompt_tokens_details":{"text_tokens":616,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":1500,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":616,"tokens_out":75,"duration_ms":17159,"temperature":1.0,"reasoning_tokens":1500,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-06-28T00:41:51.089292+00:00","model_set":{"reader":"grok-4.3"},"falsifier":"A test showing that a significant number of legitimate queries retrieve the injected fictitious entries, or that the targeted probes no longer reliably detect the presence of the watermarks after redistribution.","supporting_citations":[],"review_version":1}