REVIEW 3 major objections 5 minor 51 references
Sentence-level black-box attack lifts irrelevant documents into dense-retriever top-K by centroid concatenation plus guided synonym search.
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · grok-4.5
2026-07-12 02:20 UTC pith:UFMOQ7HH
load-bearing objection Practical black-box sentence-level attack that actually lifts hard targets under realistic K=10; the surrogate-transfer gap is real but the empirical gains look solid. the 3 major comments →
SentAttack: A Sentence-Level Black-Box Adversarial Attack Method for Dense Retrieval Models
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
A two-stage black-box procedure—iterative-retrieval surrogate training followed by centroid-document concatenation and query-plus-centroid guided beam search—can raise low-ranked target documents into the top-K returned by a dense retriever far more effectively than existing word-level or multi-view attacks, with the largest gains on the hardest targets.
What carries the argument
SentAttack: the pipeline that (1) builds a surrogate dual-encoder from iterative top-K lists via contrastive learning, (2) density-peak clusters query-relevant documents to obtain centroid sentences, concatenates them with the target, and (3) optimizes the hybrids under a weighted query-and-centroid embedding objective with gradient-guided synonym beam search.
Load-bearing premise
The surrogate encoder trained only on the small top-K lists returned by the black-box system sits close enough in embedding space to the true unknown retriever that gradients and ranking gains transfer.
What would settle it
Replace the surrogate with a deliberately distant dual-encoder (or simply measure the embedding discrepancy δ on held-out pairs) and check whether the same concatenated-and-substituted candidates still enter the black-box top-K at comparable rates; if success collapses while the white-box variant succeeds, the transfer assumption fails.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes SentAttack, a two-stage black-box sentence-level attack on dense retrieval (DR) models in RAG systems. Stage 1 trains a BERT-base dual-encoder surrogate via InfoNCE (Eq. 3) on positive/hard-negative/random-negative triplets obtained by iterative retrieval (IRM) from the black-box top-K lists. Stage 2 encodes IRM-retrieved documents, applies density-peak clustering to obtain n centroids, concatenates each centroid with the target document (Eq. 6), and refines the candidates by gradient-guided synonym substitution under a query- and centroid-guided objective (Eqs. 7–9) with beam search. Experiments on MS-MARCO Document and Passage (300 queries, Easy/Middle/Hard/Mixture targets) show consistent gains over TF-IDF, TS, PAT, PRADA and MCARA, with the largest relative lifts on Hard targets (Document SRR@10 34.9 % vs MCARA 24.4 %; Passage SRR@100 31.2 % vs 15.3 %). Ablations, hyper-parameter sweeps, naturalness metrics, re-ranker transfer and black-box vs white-box comparisons are also reported.
Significance. If the Hard-target gains transfer under realistic small-K black-box access, the work fills a genuine gap: most prior RAG robustness work targets re-rankers or generation, while DR attacks have been limited to word-level edits that fail on documents far outside the top-K. The IRM-based surrogate construction and the explicit concatenation-plus-refinement pipeline are practical contributions that match real RAG constraints (K ∈ [5,10]). The empirical package (two collections, difficulty stratification, ablations, naturalness, re-ranker survival) is solid and would be useful to the IR/security community. The Appendix E transfer argument, while not measured, at least makes the key assumption explicit.
major comments (3)
- The central Hard-target claim rests on unmeasured surrogate-to-target transfer. Gradients are computed solely on the BERT-base surrogate (Eqs. 10–11); Appendix E invokes an embedding-discrepancy bound ∥Et(x)−Es(x)∥₂≤δ and a compositionality assumption for concatenation, yet neither δ nor η_comp is ever estimated on the actual coCondenser target, nor is any ranking-agreement or embedding-correlation statistic reported between surrogate and target (especially for documents initially ranked outside top-1000/10000). Table 3 shows only a modest black-box/white-box gap on Mixture, not stratified by Hard. Without this measurement the Hard lifts (Table 1) could be surrogate-overfitting artifacts rather than transferable attacks.
- Threat model vs. evaluation realism. Section 3 and the introduction correctly emphasize that practical RAG systems return only K∈[5,10], yet the main evaluation metrics for Passage are SRR@100 / SRR@1000 and for Document SRR@100 (Table 1). While the authors also report SRR@10 and include a SentAttack_noit ablation under large-K (Appendix D), the headline Hard numbers that drive the “especially strong on low-ranked” claim are still measured at depths far larger than the threat model. A primary table restricted to K=10 (or an explicit success-at-K=10 column for every method) is needed to substantiate the practical claim.
- Naturalness and detectability of sentence-level concatenation. The attack injects entire centroid documents (often definitional passages unrelated to the target) before synonym polishing. Figure 4 and the NSP analysis (Appendix B) show competitive fluency/PPL/grammar and high NSP scores, but no human or automated detector study is provided that tests whether a simple length, topic-shift or NSP-threshold filter would remove the adversarial documents before they reach the re-ranker. Given that the threat model allows corpus injection (Wikipedia-style edits), a short detectability experiment would strengthen the claim that the attack is practically usable.
minor comments (5)
- Figure 2 caption and the NIO/Tesla example are clear, but the main text never states how many IRM iterations T are used at inference time for the reported numbers; Appendix D.4 varies the count but the default should be explicit in §5.1.
- Eq. (2) defines similarity with a temperature τ that is never given a value; likewise the density-peak clustering parameters (local-density definition) are left to the reference.
- Table 1 reports NRS only at the larger cut-off (@100 or @1000); adding NRS@10 / NRS@100 for the stricter cut-offs would make the ranking-shift claim more transparent.
- The synonym set C and the exact source of synonyms (WordNet? counter-fitted embeddings?) are not specified beyond the ρ=0.8 threshold.
- A few typographical issues: “PA T” vs “PAT”, “Cheg.1 and Gram.2”, and the arXiv identifier in the header is 2607.03456 (future date).
Circularity Check
No significant circularity; empirical black-box attack with external MS-MARCO evaluation and non-tautological theory under stated assumptions.
full rationale
SentAttack is an empirical two-stage attack: Stage 1 trains a surrogate dual-encoder via InfoNCE (Eq. 3) on positives/hard-negatives collected by iterative retrieval from the black-box target (K=10); Stage 2 concatenates density-peak centroids then performs gradient-guided synonym substitution under the query+centroid objective (Eqs. 7-9). Attack success (SRR/NRS in Table 1, Hard-target lifts vs MCARA) is measured directly on the held-out black-box coCondenser retriever and public MS-MARCO labels, not on any quantity used to construct the surrogate or objective. Appendix E supplies a lower-bound transfer argument under explicit, non-circular assumptions (embedding discrepancy ∥Et(x)-Es(x)∥₂≤δ and compositionality of concatenation); the bound is not obtained by fitting a free parameter to the same ranking metric later reported as a prediction, nor does it rely on a self-citation uniqueness theorem. No self-definitional loops, no fitted-input-as-prediction, and no load-bearing self-citations appear. The method is therefore self-contained against external benchmarks.
Axiom & Free-Parameter Ledger
free parameters (5)
- λ (query/centroid objective weight) =
0.4
- n (number of cluster centroids) =
5
- ρ (synonym semantic-similarity threshold) =
0.8
- beam width m =
3
- retrieval depth K =
10
axioms (4)
- domain assumption Black-box adversary receives only a top-K ranked list (K≈10) and can inject documents into the corpus (e.g., via Wikipedia edits).
- ad hoc to paper Surrogate and target document embeddings differ by at most δ in L2 norm for any text x (Assumption 1, Appendix E).
- ad hoc to paper The surrogate encoder approximately composes under concatenation: Es(u⊕v) ≈ αu Es(u) + αv Es(v) with bounded error ηcomp (Assumption 2).
- domain assumption InfoNCE contrastive loss on positive / hard-negative / random-negative triplets recovered by iterative retrieval yields a surrogate whose ranking behavior approximates the target.
invented entities (2)
-
Iterative Retrieval Module (IRM)
no independent evidence
-
Query- and centroid-guided attack objective O
no independent evidence
read the original abstract
Retrieval-Augmented Generation (RAG) systems typically consist of a dense retrieval (DR) model for initial retrieval and a neural ranking model (NRM) for re-ranking.Existing robustness studies in RAG mainly focus on NRMs, while adversarial attacks on DR models are mostly limited to word-level perturbations.For low-ranked target documents that are irrelevant to the query, simple word-level attacks are insufficient to mislead DR models into substantially promoting their rankings.To solve these problems, we propose SentAttack, a sentence-level black-box adversarial attack method for DR models.SentAttack is designed as a two-stage method.In the first stage, SentAttack interacts with the black-box RAG system via iterative retrieval to collect ranked documents and ranking information for training a surrogate DR model.In the second stage, SentAttack uses the surrogate DR model to encode and cluster documents relevant to the target query, yielding multiple cluster centroids.These centroids are concatenated with the target document at the sentence level to form an initial set of adversarial candidates.SentAttack then optimizes these candidates using a query- and centroid-guided objective combined with gradient-guided beam search.Extensive experiments demonstrate that SentAttack outperforms existing adversarial attacks on DR models, with especially strong performance on low-ranked target documents.
Figures
Reference graph
Works this paper leans on
-
[1]
ACM Transactions on Information Systems , volume =
Jiafeng Guo and Yinqiong Cai and Yixing Fan and Fei Sun and Ruqing Zhang and Xueqi Cheng , title =. ACM Transactions on Information Systems , volume =. 2022 , publisher =
2022
-
[2]
ACM Transactions on Information Systems , volume =
Wayne Xin Zhao and Jin Liu and Rui Ren and Jun Xu and Ji-Rong Wen , title =. ACM Transactions on Information Systems , volume =. 2024 , publisher =
2024
-
[3]
Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =
Zhuyun Dai and Jamie Callan , title =. Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =
-
[4]
Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =
Chenyan Xiong and Zhuyun Dai and Jamie Callan and Zhiyuan Liu and Russell Power , title =. Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =
-
[5]
Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security (CCS) , pages =
Jiawei Liu and Yangyang Kang and Di Tang and Kaisong Song and Changlong Sun and Xiaofeng Wang and Wei Lu and Xiaozhong Liu , title =. Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security (CCS) , pages =
2022
-
[6]
Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval , publisher =
Yu-An Liu and Ruqing Zhang and Jiafeng Guo and Maarten de Rijke and Wei Chen and Yixing Fan and Xueqi Cheng , title =. Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval , publisher =
-
[7]
ACM Transactions on Information Systems (TOIS) , volume =
Chen Wu and Ruqing Zhang and Jiafeng Guo and Maarten de Rijke and Yixing Fan and Xueqi Cheng , title =. ACM Transactions on Information Systems (TOIS) , volume =
-
[8]
Foundations and Trends in Information Retrieval , volume =
Yixing Fan and Xiaohui Xie and Yinqiong Cai and Jia Chen and Xinyu Ma and Xiangsheng Li and Ruqing Zhang and Jiafeng Guo and others , title =. Foundations and Trends in Information Retrieval , volume =
-
[9]
Bruce Croft and Xueqi Cheng , title =
Jiafeng Guo and Yixing Fan and Liang Pang and Liu Yang and Qingyao Ai and Hamed Zamani and Chen Wu and W. Bruce Croft and Xueqi Cheng , title =. Information Processing & Management , volume =
-
[10]
Proceedings of the 14th ACM International Conference on Web Search and Data Mining (WSDM) , pages =
Andrew Yates and Rodrigo Nogueira and Jimmy Lin , title =. Proceedings of the 14th ACM International Conference on Web Search and Data Mining (WSDM) , pages =
-
[11]
Proceedings of the 32nd ACM International Conference on Information and Knowledge Management (CIKM) , pages =
Yu-An Liu and Ruqing Zhang and Jiafeng Guo and others , title =. Proceedings of the 32nd ACM International Conference on Information and Knowledge Management (CIKM) , pages =
-
[12]
Proceedings of the First International Workshop on Adversarial Information Retrieval on the Web (AIRWeb) , year =
Zoltan Gyongyi and Hector Garcia-Molina , title =. Proceedings of the First International Workshop on Adversarial Information Retrieval on the Web (AIRWeb) , year =
-
[13]
ACM SIGIR Forum , volume =
Jimmy Lin , title =. ACM SIGIR Forum , volume =
-
[14]
arXiv preprint arXiv:2207.13443 , year =
Nicola Tonellotto , title =. arXiv preprint arXiv:2207.13443 , year =
-
[15]
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , publisher =
Vladimir Karpukhin and Barlas Oguz and Sewon Min and Patrick Lewis and Ledell Wu and Sergey Edunov and Danqi Chen and Wen-tau Yih , title =. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , publisher =
2020
-
[16]
Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL) , pages =
Luyu Gao and Jamie Callan , title =. Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (ACL) , pages =
-
[17]
Proceedings of the 31st ACM International Conference on Information and Knowledge Management (CIKM) , pages =
Xinyu Ma and Ruqing Zhang and Jiafeng Guo and Yixing Fan and Xueqi Cheng , title =. Proceedings of the 31st ACM International Conference on Information and Knowledge Management (CIKM) , pages =
-
[18]
Zaharia , title =
Omar Khattab and Matei A. Zaharia , title =. Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =
-
[19]
Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) , publisher =
Yingqi Qu and Yuchen Ding and Jing Liu and Kai Liu and Ruiyang Ren and Wayne Xin Zhao and Daxiang Dong and Hua Wu and Haifeng Wang , title =. Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics (NAACL) , publisher =
2021
-
[20]
Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =
Jingtao Zhan and Jiaxin Mao and Yiqun Liu and Jiafeng Guo and Min Zhang and Shaoping Ma , title =. Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =
-
[21]
arXiv preprint arXiv:2306.12756 , year =
Yu-An Liu and Ruqing Zhang and Jiafeng Guo and Wei Chen and Xueqi Cheng , title =. arXiv preprint arXiv:2306.12756 , year =
-
[22]
BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models , booktitle =
Nandan Thakur and Nils Reimers and Andreas R. BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models , booktitle =
-
[23]
arXiv preprint arXiv:2210.15212 , year =
Yue Yu and Chenyan Xiong and Si Sun and Chao Zhang and Arnold Overwijk , title =. arXiv preprint arXiv:2210.15212 , year =
-
[24]
Proceedings of the 31st International Joint Conference on Artificial Intelligence (IJCAI) , pages =
Xuanang Chen and Jian Luo and Ben He and Le Sun and Yingfei Sun , title =. Proceedings of the 31st International Joint Conference on Artificial Intelligence (IJCAI) , pages =
-
[25]
Evaluating the Robustness of Retrieval Pipelines with Query Variation Generators , booktitle =
Gustavo Penha and Arthur C. Evaluating the Robustness of Retrieval Pipelines with Query Variation Generators , booktitle =
-
[26]
Carlos Castillo and Brian D. Davison , title =. Foundations and Trends in Information Retrieval , volume =. 2011 , publisher =. doi:10.1561/1500000021 , url =
-
[27]
arXiv preprint arXiv:1503.02531 , year =
Geoffrey Hinton and Oriol Vinyals and Jeff Dean , title =. arXiv preprint arXiv:1503.02531 , year =
-
[28]
Parikh and Nicolas Papernot and Mohit Iyyer , title =
Kalpesh Krishna and Gaurav Singh Tomar and Ankur P. Parikh and Nicolas Papernot and Mohit Iyyer , title =. Proceedings of the 8th International Conference on Learning Representations (ICLR) , year =
-
[29]
arXiv preprint arXiv:1905.09165 , year =
Soham Pal and Yash Gupta and Aditya Shukla and Aditya Kanade and Shirish Shevade and Vinod Ganapathy , title =. arXiv preprint arXiv:1905.09165 , year =
Pith/arXiv arXiv 1905
-
[30]
Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages =
Eric Wallace and Mitchell Stern and Dawn Song , title =. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pages =. 2020 , url =
2020
-
[31]
Oche, A. J. and Folashade, A. G. and Ghosal, T. and others , title =. arXiv preprint arXiv:2507.18910 , year =
-
[32]
Proceedings of the 30th USENIX Security Symposium (USENIX Security 21) , pages =
Nicholas Carlini and Florian Tramer and Eric Wallace and Matthew Jagielski and Ariel Herbert-Voss and Katherine Lee and Adam Roberts and Tom Brown and Dawn Song and Ulfar Erlingsson and others , title =. Proceedings of the 30th USENIX Security Symposium (USENIX Security 21) , pages =. 2021 , publisher =
2021
-
[33]
Advances in Neural Information Processing Systems 33 (NeurIPS 2020) , year =
Wenhui Wang and Furu Wei and Li Dong and Hangbo Bao and Nan Yang and Ming Zhou , title =. Advances in Neural Information Processing Systems 33 (NeurIPS 2020) , year =
2020
-
[34]
CoCo@NeurIPS 2016 , year =
Tri Nguyen and Mir Rosenberg and Xia Song and Jianfeng Gao and Saurabh Tiwary and Rangan Majumder and Li Deng , title =. CoCo@NeurIPS 2016 , year =
2016
-
[35]
Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023) , year =
Xuanang Chen and Ben He and Zheng Ye and Le Sun and Yingfei Sun , title =. Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (ACL 2023) , year =
2023
-
[36]
Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS 2020) , year =
Scott Wen-tau Yih , title =. Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS 2020) , year =
2020
-
[37]
Proceedings of the 34th USENIX Security Symposium (USENIX Security 2025) , year =
Wei Zou and Xinyu Zhang and Yihan Cao and Yu Zhao and Jiazhao Zhang and Zhiyuan Liu and Maosong Sun and Yang Liu , title =. Proceedings of the 34th USENIX Security Symposium (USENIX Security 2025) , year =
2025
-
[38]
ACM Computing Surveys , volume =
Jiajia Wang and Xiaofei Zhou and Xiaoye Qu and Meng Yang and Qingyao Wu and Xuewei Tian and Lifang He and Qing Li , title =. ACM Computing Surveys , volume =. 2024 , publisher =
2024
-
[39]
Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =
Yu-An Liu and Rui Zhang and Jiafeng Guo and Maarten de Rijke , title =. Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =. 2024 , publisher =
2024
-
[40]
Proceedings of the 14th ACM International Conference on Web Search and Data Mining , pages =
Xinyu Ma and Jiafeng Guo and Rui Zhang and Yixing Fan and Xiang Ji and Xueqi Cheng , title =. Proceedings of the 14th ACM International Conference on Web Search and Data Mining , pages =. 2021 , publisher =
2021
-
[41]
Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , year =
Jacob Devlin and Ming-Wei Chang and Kenton Lee and Kristina Toutanova , title =. Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , year =
2019
-
[42]
Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , year =
Yangyi Chen and Hongcheng Gao and Ganqu Cui and Fanchao Qi and Longtao Huang and Zhiyuan Liu and Maosong Sun , title =. Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , year =
2022
-
[43]
Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =
Yue Li and Panagiotis Eustratiadis and Simon Lupart and Nicola Tonellotto and Maarten de Rijke , title =. Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval , pages =. 2025 , publisher =
2025
-
[44]
A Systematic Review on Search Engine Advertising , journal =
Hamed Jafarzadeh and Ayb. A Systematic Review on Search Engine Advertising , journal =
-
[45]
Knowledge-Based Systems , volume =
Yong Chen and Xiaohui Hu and Wenjie Fan and Ke Deng , title =. Knowledge-Based Systems , volume =. 2020 , publisher =
2020
-
[46]
Advances in Neural Information Processing Systems , year =
Jessica Finocchiaro and Rafael Frongillo and Bo Waggoner , title =. Advances in Neural Information Processing Systems , year =
-
[47]
Cognitive Science , volume =
Jeff Mitchell and Mirella Lapata , title =. Cognitive Science , volume =
-
[48]
Proceedings of the First Instructional Conference on Machine Learning , year =
Juan Ramos , title =. Proceedings of the First Instructional Conference on Machine Learning , year =
-
[49]
Choquette-Choo and Florian Tramer and Milad Nasr and Katherine Lee and Andreas Terzis and Borja Balle and Nicolas Papernot , title =
Nicholas Carlini and Matthew Jagielski and Christopher A. Choquette-Choo and Florian Tramer and Milad Nasr and Katherine Lee and Andreas Terzis and Borja Balle and Nicolas Papernot , title =. Proceedings of the 2024 IEEE Symposium on Security and Privacy (SP) , pages =. 2024 , publisher =
2024
-
[50]
International Conference on Learning Representations , year =
Aleksander Madry and Aleksandar Makelov and Ludwig Schmidt and Dimitris Tsipras and Adrian Vladu , title =. International Conference on Learning Representations , year =
-
[51]
International Conference on Learning Representations , year =
Ari Holtzman and Jan Buys and Li Du and Maxwell Forbes and Yejin Choi , title =. International Conference on Learning Representations , year =
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.