Pith. sign in

REVIEW 4 major objections 5 minor 60 references

Attack-in-the-Chain: Bootstrapping Large Language Models for Attacks Against Black-box Neural Ranking Models

T0 review · 4 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read LLM reasoning chains beat prior black-box ranking attacks

desk verdict A genuinely new LLM-driven black-box ranking attack with a solid iterative design, but the 'outperforms all baselines' claim is weakened by a missing comparison against the closest prior work. read the letter →

arxiv 2412.18770 v1 pith:RBJ5S7FW submitted 2024-12-25 cs.IR

Yu-An Liu , Ruqing Zhang , Jiafeng Guo , Maarten de Rijke , Yixing Fan , Xueqi Cheng This is my paper · ORCID
classification cs.IR
keywords adversarialrankingattacksneuralmodelsblack-boxattackchain-of-thoughtpromptinglargelanguagedocumentperturbationsearchengineoptimizationimperceptibility
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a large language model, guided by a deliberately built chain of reasoning steps, can attack black-box neural ranking models more effectively and more imperceptibly than existing attack methods. The proposed Attack-in-the-Chain method repeatedly identifies anchor documents that outrank the target, lets the LLM choose the most useful anchors, and rewrites the target document with a perturbation budget that grows or shrinks with the ranking gap. On MS MARCO and TREC DL19, the GPT-3.5 variant AttChainGPT outperforms four baselines and two ablations on boosted rank, top-10 rate, and naturalness metrics. If correct, this shows that the same reasoning abilities that make LLMs useful retrievers can also expose vulnerabilities in neural rankers, and it motivates defenses against LLM-generated search engine optimization.

What carries the argument

The central object is the reasoning chain: a sequence of nodes, each an anchor document that currently outranks the target document. Two mechanisms carry the argument: a Zipf-distribution filter ($P(r; s) \propto r^{-s}$ with $s=2$) that samples candidate anchors biased toward high ranks, and a discrepancy-oriented assignment that sets the perturbation-word count for anchor $d_j$ as $|p_j^i| = \frac{\mathrm{Rank}(f, q, d_i) - \mathrm{Rank}(f, q, d_j)}{\mathrm{Rank}(f, q, d)} \epsilon$. The LLM alternates between selecting anchors and rewriting the target; after each step the system verifies all node outputs and follows the node with the largest rank improvement, iterating for five rounds. The chain converts a one-shot perturbation into a ladder-climbing optimization over ranking positions.

What would settle it

Run AttChain on MS MARCO with the Zipf exponent varied (e.g., $s \in \{0, 1, 2, 3\}$) while keeping everything else fixed, or replace the Zipf sampler with uniform random sampling from the same candidate pool; if boosted ranks stay flat or match the AttChain$-\mathrm{CoT}$ ablation that uses fixed top-5 anchors, then anchor concentration is not doing the load-bearing work the paper assigns to it.

Watch

Extended reading notes

Core claim

The paper claims that orchestrating an LLM through iterative chain-of-thought rounds makes the LLM an inherently effective decision-based black-box attacker of neural ranking models. In each round, documents ranked above the target serve as anchor nodes; the LLM selects the most instructive anchors, rewrites the target document by borrowing query-relevant phrases from them, and the system keeps the rewrite that produced the largest rank gain. Repeating this for five rounds, AttChainGPT lifts average boosted ranks on MS MARCO Easy/BERT from 39.6 for the best baseline IDEM to 44.5, and on Hard targets from 69.6 to 91.2, while lowering perplexity to 38.3 and spamicity detection to 38.2% at threshold 0.02. The authors conclude that LLMs can fully capture the preferences of NRMs through interaction and generate harder-to-detect adversarial examples.

Load-bearing premise

The load-bearing premise is that sampling candidate anchors from a Zipf distribution with exponent $s=2$ concentrates guidance on documents that actually help ranking improvement; if this sampling bias is wrong, the chain's anchor selection could misdirect the LLM and the reported gains could come from the perturbation generation alone rather than from the chain-of-thought node selection.

Editorial extensions

If this is right

  • If AttChain's results hold, black-box neural rankers can be attacked effectively without training a surrogate model, lowering the cost and detectability of ranking manipulation.
  • The five-round chain is a general recipe: any task where success is measured by a queried ranker can be attacked by alternating anchor selection and budgeted rewriting.
  • The imperceptibility gains indicate that LLM-generated adversarial documents will be harder to filter with existing statistical detectors such as perplexity and spamicity scoring.
  • The gap between AttChainGPT and AttChainLlama suggests attacker capability scales with LLM scale, so stronger future LLMs may produce even more effective attacks.
  • The ablations attribute part of the gain to LLM-chosen anchors and dynamic perturbation sizing, giving defenders concrete weak points to study.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The Zipf exponent $s=2$ is set by analogy to human click behavior; a sensitivity sweep across $s$ would reveal whether anchor selection or the LLM perturbation step carries the gain, and whether the chain's node selection is truly load-bearing.
  • The five-round budget and word budget $\epsilon=25$ are likely dataset- and model-dependent; the same chain may need more rounds or larger budgets against rankers with stronger robustness, such as RankLLM, which shows smaller boosts.
  • One testable extension is to pair AttChain with an LLM-generated-text detector: if detection improves with larger attacker models, the imperceptibility advantage may erode as detectors co-evolve.
  • Because the adversarial documents borrow phrases from anchors, a plausible defense is to penalize documents that contain anchored query-relevant phrases absent from their original content.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Attack-in-the-Chain (AttChain), a decision-based black-box ranking attack that combines LLM-based chain-of-thought prompting with iterative NRM-LLM interaction. At each round, the method filters candidate anchor documents via a Zipf distribution (Eq. 1), prompts an LLM to select n anchors, assigns a perturbation word budget based on rank discrepancy (Eq. 2), and generates a perturbed document, then verifies the best node. Experiments on MS MARCO and TREC2019 against BERT, PROP, and RankLLM claim significant improvements over TS, PRADA, PAT, and IDEM on both attack effectiveness (ASR, Boost, T10R) and imperceptibility (Qrs, grammar, PPL, spamicity, human evaluation). The paper also provides ablations (AttChain-CoT, AttChain-dynamic) and a mitigation analysis.

Significance. If the results are reproducible, AttChain would be a practical, training-free alternative to surrogate-based attacks, with clear value for the IR security community. The paper releases its code, and the ablations help isolate the contributions of LLM anchor selection and dynamic budget assignment. However, the significance is currently limited by the missing comparison against the closest published decision-based attack (MGA) and by insufficient statistical detail for the claimed significance.

major comments (4)
  1. [§4.4, Table 2] The baseline list omits MGA (Liu et al. 2024b), which the paper itself identifies in §2 as an existing black-box ranking attack using reinforcement learning and in the introduction as a work in the same decision-based setting. With MGA absent, the central claim in §5.1(i) that 'AttChainGPT outperforms all baselines' is not established against the strongest published decision-based competitor. The authors should add MGA under the identical evaluation protocol or provide a documented reason for its exclusion.
  2. [§4.5, Table 2] The significance asterisks are unsupported: the text states p ≤ 0.05 but gives no test name, no variance measure, and no correction for multiple comparisons across 12 method×metric×dataset cells. The statement in §4.5 that the experiment was 'repeated 3 times' is insufficient without reporting standard deviations or confidence intervals. Please specify the test, the comparison units (queries or runs), and report error bars or intervals for the main effectiveness table.
  3. [§3.1, Eq. (1)] The Zipf exponent s=2 is a fixed design choice, justified only by an analogy to human click behavior. Because the anchor-selection stage is a stated contribution and the final method differs from the AttChain-dynamic ablation only in the budget assignment, a sensitivity analysis over s (e.g., 0.5, 1, 2, 3) is needed to show that the reported gains are not tied to this specific value. Without it, the contribution of the Zipf-based filtering versus the LLM perturbation step remains unclear.
  4. [§4.5, Tables 3–4] The naturalness results are reported for a single configuration (RankLLM on MS MARCO Mixture), and the paper claims 'similar findings on other target documents and datasets' without presenting the corresponding data. Since the paper's second central claim is imperceptibility, the authors should report naturalness metrics for all three NRMs and both datasets (or move them to an appendix) before claiming a general advantage.
minor comments (5)
  1. [§5.3] The cross-references are incorrect: the perplexity mitigation discussion refers to 'Table 2' (which is attack effectiveness) and the semantic similarity discussion refers to 'Table 3' (which is naturalness metrics); these should be Figure 2 and Figure 3, respectively.
  2. [§4.5] The sentence 'We repeated our experiment 3 times on 4 × Tesla V100 32G' is ambiguous about whether the three repetitions are independent query samples or repeated runs of the same queries; please clarify.
  3. [§3.2, Eq. (2)] The denominator Rank(f,q,d) is not defined as the original target document's rank or the current document's rank; please clarify whether it is fixed across rounds or updated at each step.
  4. [§4.1] The relationship between Easy, Hard, and Mixture target documents is under-specified: if 5 Easy and 5 Hard documents are sampled per query and Mixture is '5 documents randomly sampled from both sets', the total number of target documents per query should be stated explicitly.
  5. [Table 1] The prompt templates contain placeholders such as 'x places' and '|p′_i|' that do not match the notation in Eq. (2) (where the budget is |p^j_i|); please make the notation consistent.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: AttChain's design choices are external heuristics, evaluation is external, and self-citations are not load-bearing.

full rationale

The derivation chain in Attack-in-the-Chain is not circular. The two core design equations are external heuristics rather than fitted-to-output constructs: Eq. (1), CA = Zipf(L[:Rank(f,q,d_{i-1})], m, s), is an anchor-sampling rule justified by an analogy to human click behavior (Wu, Jiang, and Zhang 2012), and Eq. (2), |p_i^j| = (Rank(f,q,d_i) - Rank(f,q,d_j)) / Rank(f,q,d) * epsilon, is a rank-gap-based word budget assignment. Neither equation defines the evaluation metrics (ASR, Boost, T10R) in terms of itself, and neither is fitted to the test data. Hyperparameters (m=20, n=5, s=2, epsilon=25, rounds=5) are fixed before experiments rather than tuned on held-out queries/documents, and the metrics are computed on external benchmark data (MS MARCO, TREC2019) against external baselines (TS, PRADA, PAT, IDEM) with independent human annotation and external detectors (OSD, GPT-2). The paper contains self-citations (e.g., Liu et al. 2023b, 2024b, 2024d), but they are contextual references to prior settings and attacks, not load-bearing derivations: the method does not invoke a self-cited uniqueness theorem, does not rename a fitted parameter as a prediction, and does not smuggle in an ansatz via citation to make its central claim true by construction. The skeptic's point that MGA (Liu et al. 2024b) is absent from Table 2 is a possible benchmark-completeness and competitiveness concern: it questions whether 'outperforms all baselines' is fully established, but it is not a circularity charge because the paper's construction does not reduce its claimed result to its own inputs. Under the hard rules, that concern belongs to correctness risk, not circularity. Accordingly, the appropriate finding is no significant circularity, with only trace, non-load-bearing self-citation present.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The attack framework depends on several hand-set hyperparameters (m=20 candidates, n=5 anchors, Zipf exponent s=2, word budget epsilon=25, and five reasoning rounds). None are fitted to test data; they are chosen from intuition and prior work, and no sensitivity analysis is reported. The core method also assumes that anchor documents and prompt instructions reliably guide LLM perturbations. No new physical entities are introduced.

free parameters (5)
  • Anchor candidate count m = 20
    Number of documents sampled by the Zipf filter; fixed without sensitivity analysis.
  • Anchor node count n = 5
    Number of final anchors selected by the LLM; fixed in all experiments.
  • Zipf exponent s = 2
    Exponent for rank-weighted candidate sampling; chosen by hand, no ablation reported.
  • Perturbation word budget epsilon = 25
    Total manipulated-word budget per attack; fixed, no sensitivity analysis.
  • Reasoning rounds = 5
    Number of iterative chain steps; fixed for computational overhead and effectiveness.
assumptions (4)
  • domain assumption Higher-ranked documents are more informative anchors for boosting a target document
    Motivates Zipf-based filtering in Section 3.1; supported only by an analogy to human click behavior, not by direct validation.
  • domain assumption LLMs can select useful anchors and generate effective, imperceptible perturbations from the provided prompts
    Core capability assumption evaluated empirically; not independently guaranteed.
  • domain assumption Decision-based black-box access (only ranked-list feedback) is the correct threat model for the attack
    Threat model taken from prior work; limits the scope of the claims.
  • domain assumption Benchmark queries and target documents are representative, and the random sampling yields unbiased evaluation
    Queries are sampled without a reported seed; representativeness is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Attack-in-the-Chain: Bootstrapping Large Language Models for Attacks Against Black-box Neural Ranking Models." pith.science (2026). https://pith.science/paper/RBJ5S7FW

@misc{pith2026241218770,
  author       = {Pith},
  title        = {Pith review of: Attack-in-the-Chain: Bootstrapping Large Language Models for Attacks Against Black-box Neural Ranking Models},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/RBJ5S7FW}},
  note         = {Machine review of arXiv:2412.18770}
}
read the original abstract

Neural ranking models (NRMs) have been shown to be highly effective in terms of retrieval performance. Unfortunately, they have also displayed a higher degree of sensitivity to attacks than previous generation models. To help expose and address this lack of robustness, we introduce a novel ranking attack framework named Attack-in-the-Chain, which tracks interactions between large language models (LLMs) and NRMs based on chain-of-thought (CoT) prompting to generate adversarial examples under black-box settings. Our approach starts by identifying anchor documents with higher ranking positions than the target document as nodes in the reasoning chain. We then dynamically assign the number of perturbation words to each node and prompt LLMs to execute attacks. Finally, we verify the attack performance of all nodes at each reasoning step and proceed to generate the next reasoning step. Empirical results on two web search benchmarks show the effectiveness of our method.

Figures

Figures reproduced from arXiv: 2412.18770 by the authors.

Figure 1
Figure 1. The framework of the proposed method AttChain. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Distributions of log perplexity (PPL) of adversarial [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Distribution of cosine similarity of semantic em [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

60 extracted references · 42 canonical work pages

  1. [1]

    L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al

    Achiam, J.; Adler, S.; Agarwal, S.; Ahmad, L.; Akkaya, I.; Aleman, F. L.; Almeida, D.; Altenschmidt, J.; Altman, S.; Anadkat, S.; et al. 2023. GPT-4 Technical Report. arXiv preprint arXiv:2303.08774

    arXiv 2023

  2. [2]

    Brendel, W.; Rauber, J.; and Bethge, M. 2018. Decision-Based Adversarial Attacks: Reliable Attacks Against Black-Box Machine Learning Models. In The Sixth ICLR

    work page 2018

  3. [3]

    J.; and Wong, E

    Chao, P.; Robey, A.; Dobriban, E.; Hassani, H.; Pappas, G. J.; and Wong, E. 2023. Jailbreaking Black Box Large Language Models in Twenty Queries. In R0-FoMo: Robustness of Few-shot and Zero-shot Learning in Large Foundation Models

    work page 2023

  4. [4]

    Chen, J.; Lin, H.; Han, X.; and Sun, L. 2024. Benchmarking Large Language Models in Retrieval-Augmented Generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 17754--17762

    work page 2024

  5. [5]

    Chen, X.; He, B.; Ye, Z.; Sun, L.; and Sun, Y. 2023. Towards Imperceptible Document Manipulations against Neural Ranking Models. In Findings of the ACL 2023, 6648--6664

    work page 2023

  6. [6]

    Cohn, C.; Hutchins, N.; Le, T.; and Biswas, G. 2024. A Chain-of-Thought Prompting Approach With LLMs for Evaluating Students’ Formative Assessment Responses in Science. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 23182--23190

    work page 2024

  7. [7]

    Craswell, N.; Mitra, B.; Yilmaz, E.; Campos, D.; and Voorhees, E. 2019. Overview of the TREC 2019 Deep Learning Track. In TExt Retrieval Conference 2019

    work page 2019

  8. [8]

    Dai, Z.; and Callan, J. 2019. Deeper Text Understanding for IR with Contextual Neural Language Modeling. In SIGIR

    work page 2019

Show all 60 references
  1. [9]

    Gadyatskaya, O.; and Papuc, D. 2023. ChatGPT Knows Your Attacks: Synthesizing Attack Trees Using LLMs. In International Conference on Data Science and Artificial Intelligence, 245--260. Springer

    2023

  2. [10]

    Giadikiaroglou, P.; Lymperaiou, M.; Filandrianos, G.; and Stamou, G. 2024. Puzzle Solving using Reasoning of Large Language Models: A Survey. arXiv preprint arXiv:2402.11291

    2024 arXiv

  3. [11]

    Guo, J.; Fan, Y.; Ai, Q.; and Croft, W. B. 2016. A Deep Relevance Matching Model for Ad-hoc Retrieval. In Proceedings of the 25th ACM International on Conference on Information and Knowledge Management, 55--64

    2016

  4. [12]

    Gyongyi, Z.; and Garcia-Molina, H. 2005. Web Spam Taxonomy. In AIRWeb

    2005

  5. [13]

    He, P.; Liu, X.; Gao, J.; and Chen, W. 2020. DeBERTa: Decoding-enhanced BERT with Disentangled Attention. In The Ninth ICLR

    2020

  6. [14]

    Ji, B.; Liu, H.; Du, M.; and Ng, S.-K. 2024. Chain-of-Thought Improves Text Generation with Citations in Large Language Models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 18345--18353

    2024

  7. [15]

    Q.; Sablayrolles, A.; Mensch, A.; Bamford, C.; Chaplot, D

    Jiang, A. Q.; Sablayrolles, A.; Mensch, A.; Bamford, C.; Chaplot, D. S.; Casas, D. d. l.; Bressand, F.; Lengyel, G.; Lample, G.; Saulnier, L.; et al. 2023. Mistral 7B. arXiv preprint arXiv:2310.06825

    2023 arXiv

  8. [16]

    Lanham, T.; Chen, A.; Radhakrishnan, A.; Steiner, B.; Denison, C.; Hernandez, D.; Li, D.; Durmus, E.; Hubinger, E.; Kernion, J.; et al. 2023. Measuring Faithfulness in Chain-of-Thought Reasoning. arXiv preprint arXiv:2307.13702

    2023 arXiv

  9. [17]

    Liang, K.; Liu, Y.; Zhou, S.; Tu, W.; Wen, Y.; Yang, X.; Dong, X.; and Liu, X. 2023. Knowledge Graph Contrastive Learning Based on Relation-Symmetrical Structure. IEEE TKDE, 36(1): 226--238

    2023

  10. [18]

    Liang, K.; Meng, L.; Liu, M.; Liu, Y.; Tu, W.; Wang, S.; Zhou, S.; Liu, X.; Sun, F.; and He, K. 2024. A Survey of Knowledge Graph Reasoning on Graph Types: Static, Dynamic, and Multi-Modal. IEEE TPAMI

    2024

  11. [19]

    Liu, J.; Kang, Y.; Tang, D.; Song, K.; Sun, C.; Wang, X.; Lu, W.; and Liu, X. 2022. Order-Disorder: Imitation Adversarial Attacks for Black-box Neural Ranking Models. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security, 2025--2039

    2022

  12. [20]

    Liu, Y.-A.; Zhang, R.; Guo, J.; Chen, W.; and Cheng, X. 2023 a . On the Robustness of Generative Retrieval Models: An Out-of-Distribution Perspective. In Gen-IR@SIGIR

    2023

  13. [21]

    Liu, Y.-A.; Zhang, R.; Guo, J.; and de Rijke, M. 2024 a . Robust Information Retrieval. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, 3009--3012

    2024

  14. [22]

    Liu, Y.-A.; Zhang, R.; Guo, J.; and de Rijke, M. 2025 a . Robust Information Retrieval. In Proceedings of the 18th ACM International Conference on Web Search and Data Mining

    2025

  15. [23]

    Liu, Y.-A.; Zhang, R.; Guo, J.; de Rijke, M.; Chen, W.; Fan, Y.; and Cheng, X. 2023 b . Topic-Oriented Adversarial Attacks against Black-Box Neural Ranking Models. In Proceedings of the 46th SIGIR, 1700–1709

    2023

  16. [24]

    Liu, Y.-A.; Zhang, R.; Guo, J.; de Rijke, M.; Fan, Y.; and Cheng, X. 2024 b . Multi-granular Adversarial Attacks against Black-box Neural Ranking Models. In Proceedings of the 47th SIGIR, 1391–1400

    2024

  17. [25]

    Liu, Y.-A.; Zhang, R.; Guo, J.; de Rijke, M.; Fan, Y.; and Cheng, X. 2024 c . Robust neural information retrieval: An adversarial and out-of-distribution perspective. arXiv preprint arXiv:2407.06992

    2024 arXiv

  18. [26]

    Liu, Y.-A.; Zhang, R.; Guo, J.; Fan, Y.; and Cheng, X. 2025 b . On the Robustness of Generative Information Retrieval Models: An Out-of-Distribution Perspective. In Proceedings of the 47th European Conference on Information Retrieval

    2025

  19. [27]

    Liu, Y.-A.; Zhang, R.; Zhang, M.; Chen, W.; de Rijke, M.; Guo, J.; and Cheng, X. 2024 d . Perturbation-Invariant Adversarial Training for Neural Ranking Models: Improving the Effectiveness-Robustness Trade-Off. In Proceedings of the AAAI Conference on Artificial Intelligence, ...

    2024

  20. [28]

    Ma, X.; Guo, J.; Zhang, R.; Fan, Y.; Ji, X.; and Cheng, X. 2021. Prop: Pre-training with Representative Words Prediction for Ad-hoc Retrieval. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining, 283--291

    2021

  21. [29]

    Madaan, A.; and Yazdanbakhsh, A. 2022. Text and Patterns: For Effective Chain of Thought, It Takes Two to Tango. arXiv preprint arXiv:2209.07686

    2022 arXiv

  22. [30]

    Meta. 2024. Meta Llama 3: The Most Capable Openly Available LLM to Date. https://ollama.com/library/llama3

    2024

  23. [31]

    Nguyen, T.; Rosenberg, M.; Song, X.; Gao, J.; Tiwary, S.; Majumder, R.; and Deng, L. 2016. MS MARCO: A Human Generated Machine Reading Comprehension Dataset. In CoCo@NIPS

    2016

  24. [32]

    Nogueira, R.; and Cho, K. 2019. Passage Re-ranking with BERT. arXiv preprint arXiv:1901.04085

    2019 arXiv

  25. [33]

    OpenAI. 2022. Introducing ChatGPT. https://openai.com/blog/chatgpt

    2022

  26. [34]

    OpenAI. 2024 a . OpenAI API. https://openai.com/api/

    2024

  27. [35]

    OpenAI. 2024 b . Text-embedding-3. https://platform.openai.com/docs/api-reference/embeddings

    2024

  28. [36]

    B.; and Swami, A

    Papernot, N.; McDaniel, P.; Goodfellow, I.; Jha, S.; Celik, Z. B.; and Swami, A. 2017. Practical Black-box Attacks Against Machine Learning. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security, 506–519

    2017

  29. [37]

    Paranjape, B.; Lundberg, S.; Singh, S.; Hajishirzi, H.; Zettlemoyer, L.; and Ribeiro, M. T. 2023. Art: Automatic Multi-step Reasoning and Tool-use for Large Language Models. arXiv preprint arXiv:2303.09014

    2023 arXiv

  30. [38]

    Radford, A.; Wu, J.; Child, R.; Luan, D.; Amodei, D.; and Sutskever, I. 2019. Language Models Are Unsupervised Multitask Learners. OpenAI blog, 1(8): 9

    2019

  31. [39]

    Rahutomo, F.; Kitasuka, T.; Aritsugi, M.; et al. 2012. Semantic Cosine Similarity. In The 7th international student conference on advanced science and technology ICAST, volume 4, 1. University of Seoul South Korea

    2012

  32. [40]

    Raina, V.; Liusie, A.; and Gales, M. 2024. Is LLM-as-a-Judge Robust? Investigating Universal Adversarial Attacks on Zero-shot LLM Assessment. arXiv preprint arXiv:2402.14016

    2024 arXiv

  33. [41]

    Raval, N.; and Verma, M. 2020. One Word at a Time: Adversarial Attacks on Retrieval Models. arXiv preprint arXiv:2008.02197

    2020 arXiv

  34. [42]

    M.; and Shmatikov, V

    Song, C.; Rush, A. M.; and Shmatikov, V. 2020. Adversarial Semantic Collisions. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, 4198--4210

    2020

  35. [43]

    Sun, W.; Yan, L.; Ma, X.; Wang, S.; Ren, P.; Chen, Z.; Yin, D.; and Ren, Z. 2023. Is C hat GPT Good at Search? Investigating Large Language Models as Re-Ranking Agents. In EMNLP 2023, 14918--14937

    2023

  36. [44]

    Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozi \`e re, B.; Goyal, N.; Hambro, E.; Azhar, F.; et al. 2023. Llama: Open and Efficient Foundation Language Models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  37. [45]

    K.-W.; and Lim, E.-P

    Wang, L.; Xu, W.; Lan, Y.; Hu, Z.; Lan, Y.; Lee, R. K.-W.; and Lim, E.-P. 2023 a . Plan-and-Solve Prompting: Improving Zero-Shot Chain-of-Thought Reasoning by Large Language Models. In Proceedings of the 61st ACL, 2609--2634

    2023

  38. [46]

    Wang, Y.; Li, P.; Sun, M.; and Liu, Y. 2023 b . Self-Knowledge Guided Retrieval Augmentation for Large Language Models. In EMNLP 2023, 10303--10315

    2023

  39. [47]

    V.; Zhou, D.; et al

    Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Xia, F.; Chi, E.; Le, Q. V.; Zhou, D.; et al. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. NeurIPS, 35: 24824--24837

    2022

  40. [48]

    Wu, C.; Zhang, R.; Guo, J.; de Rijke, M.; Fan, Y.; and Cheng, X. 2023. PRADA: Practical Black-Box Adversarial Attacks against Neural Ranking Models. ACM Transactions on Information Systems, 41(4): Article 89

    2023

  41. [49]

    Wu, M.; Jiang, S.; and Zhang, Y. 2012. Serial Position Effects of Clicking Behavior on Result Pages Returned by Search Engines. In Proceedings of the 21st CIKM, 2411--2414

    2012

  42. [50]

    W.; McDonald, G.; Bai, X.; Marshall, D.; Wang, S.; Swaminathan, A.; and Li, Z

    Xu, J.; Stokes, J. W.; McDonald, G.; Bai, X.; Marshall, D.; Wang, S.; Swaminathan, A.; and Li, Z. 2024 a . Autoattacker: A Large Language Model Guided System to Implement Automatic Cyber-attacks. arXiv preprint arXiv:2403.01038

    2024 arXiv

  43. [51]

    Xu, S.; Pang, L.; Shen, H.; Cheng, X.; and Chua, T.-s. 2023 a . Search-in-the-chain: Towards the Accurate, Credible and Traceable Content Generation for Complex Knowledge-intensive Tasks. arXiv preprint arXiv:2304.14732

    2023 arXiv

  44. [52]

    Xu, X.; Kong, K.; Liu, N.; Cui, L.; Wang, D.; Zhang, J.; and Kankanhalli, M. 2023 b . An LLM Can Fool Itself: A Prompt-based Adversarial Attack. arXiv preprint arXiv:2310.13345

    2023 arXiv

  45. [53]

    Xu, X.; Kong, K.; Liu, N.; Cui, L.; Wang, D.; Zhang, J.; and Kankanhalli, M. 2024 b . An LLM Can Fool Itself: A Prompt-Based Adversarial Attack. In The Twelfth ICLR

    2024

  46. [54]

    Yan, M.; Li, C.; Bi, B.; Wang, W.; and Huang, S. 2021. A Unified Pretraining Framework for Passage Ranking and Expansion. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, 4555--4563

    2021

  47. [55]

    Yu, L.; Zhang, C.; Liang, S.; and Zhang, X. 2019. Multi-order Attentive Ranking Model for Sequential Recommendation. In Proceedings of the AAAI conference on artificial intelligence, volume 33, 5709--5716

    2019

  48. [56]

    Yu, W.; Zhang, Z.; Liang, Z.; Jiang, M.; and Sabharwal, A. 2023. Improving Language Models via Plug-and-Play Retrieval Feedback. arXiv preprint arXiv:2305.14002

    2023 arXiv

  49. [57]

    Zhou, B.; and Pei, J. 2009. OSD: An Online Web Spam Detection System. In Proceedings of the 15th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD, volume 9

    2009

  50. [58]

    Zipf, G. K. 2016. Human Behavior and the Principle of Least Effort: An Introduction to Human Ecology. Ravenio Books

    2016

  51. [59]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  52. [60]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.