Pith. sign in

REVIEW 3 major objections 6 minor 96 references

The Ultimate Cookbook for Invisible Poison: Crafting Subtle Clean-Label Text Backdoors with Style Attributes

T0 review · 3 major / 6 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A clean-label text backdoor whose trigger is a single fine-grained stylistic attribute can be both more effective and harder for human annotators to spot than baseline attacks that rely on inserted phrases, syntactic templates, or whole…

desk verdict A genuinely new attack family with a strong effectiveness evaluation, but the central subtlety claim rests on a small, underpowered human study that needs statistical backing and a slightly softer conclusion. read the letter →

arxiv 2504.17300 v1 pith:PCYO3BCW submitted 2025-04-24 cs.LG

Wencong You , Daniel Lowd This is my paper · ORCID
classification cs.LG
keywords backdoorattacksclean-labelpoisoningtextclassificationstyletransferhumanevaluationattackinvisibilitylargelanguagemodelsadversarialmachinelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Backdoor attacks on text classifiers are usually caught because their triggers look unnatural: inserted phrases, syntactic templates, or heavy register styles such as biblical English. This paper argues that the trigger can be reduced to a single fine-grained stylistic attribute, such as 'uses short, choppy sentences for emphasis,' and that an LLM can paraphrase clean training examples to carry that attribute while keeping the original label. It proposes AttrBkd, with three recipes for obtaining such attributes, and reports that on SST-2 human annotators missed AttrBkd poison samples more often than they missed samples from corresponding baseline attacks, while attack success rates stayed comparable or higher. The paper introduces the attack invisibility rate (AIR) and presents evidence that standard automated text-quality metrics disagree with human judgment about which poison looks suspicious. If these results hold, human annotation alone is not a reliable defense against style-based backdoors.

What carries the argument

The load-bearing mechanism is the fine-grained stylistic attribute trigger: a short, human-readable description of one style feature, such as 'utilizes short, choppy sentences for emphasis,' that an LLM is prompted to apply while paraphrasing clean text. Three recipes produce these attributes: baseline-derived attributes (ask an LLM to list the style features of an existing attack's poison samples, cluster them by pairwise sentence similarity, and pick a frequent representative), LISA embedding outliers (use the LISA interpretable style-embedding model to find attributes that appear rarely in the clean data), and sample-inspired attributes (prompt an LLM to invent novel style attributes). The evaluation's new measure is the attack invisibility rate (AIR), the fraction of a given attack's poison samples that human raters fail to flag in an outlier-detection task; the paper uses AIR together with human ratings of label consistency, semantics, and stylistic nuance to compare subtlety.

What would settle it

Repeat the human outlier-detection task on SST-2 without the automatic formatting cleanup used in Appendix F-A, using a larger and more diverse pool of raters; if the fraction of missed AttrBkd samples falls to the level of the conspicuous baselines, the claimed invisibility advantage does not survive.

Watch

Extended reading notes

Core claim

The central claim is that previous text backdoors fail not because poisoning is detectable in principle but because their triggers are too coarse: full register styles drag in telltale vocabulary, inserted phrases break grammar, and syntactic templates discard meaning. AttrBkd replaces those triggers with one fine-grained stylistic attribute, obtained by asking an LLM to name the significant style features of an existing attack's poison samples, clustering those features, and selecting a frequent representative; the same attribute then serves as the instruction for paraphrasing clean text into poison. The paper's human annotation study on SST-2 finds that in pairwise comparisons, AttrBkd variants are missed more often than the corresponding baseline attack in almost every case, with an average invisibility improvement of 15.6 percentage points, while maintaining or exceeding baseline attack success rates. The one baseline that is more invisible, LLMBkd Default, is ineffective as an attack. Because AttrBkd poison is label-consistent and reads naturally, the paper concludes that subtlety and effectiveness can coexist in clean-label backdoors, and that automated metrics should not replace human judgment in measuring either.

Load-bearing premise

The load-bearing premise is that seven native-English-speaking students rating a few hundred reformatted SST-2 samples, without reported inter-annotator agreement, are a reliable proxy for the real-world data curators who would actually filter poisoned training data.

Editorial extensions

If this is right

  • At 5% poisoning, AttrBkd reaches attack success rates above 90% on multiple datasets and often matches or exceeds baselines, while keeping clean accuracy nearly unchanged.
  • Under the defense methods tested, AttrBkd variants are generally harder to neutralize than their baseline counterparts; most defenses either fail to reduce attack success or reduce it inconsistently across datasets.
  • The same style-attribute trick works across three victim architectures, so the vulnerability is not tied to one model family.
  • Human label-consistency checks alone will not catch these attacks, because AttrBkd poison is labeled truthfully and reads naturally; data curators would need style-level monitoring.
  • Any claim that a text backdoor is subtle should be verified by human annotation, because several automated metrics in the study do not align with human ratings.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the trigger is a natural-language attribute, the same recipe could be applied to style dimensions the paper did not test, such as punctuation habits, code-switching, or sentence rhythm; this is a direct extension rather than a claim the authors make.
  • The formatting cleanup applied before the human study is acknowledged by the authors; testing on the original SST-2 tokenization would show whether the invisibility advantage survives noisier presentation, and that test is not reported in the paper.
  • One could use the attribute language as a probe: build paired datasets differing only in that attribute, then inspect which tokens or hidden states the victim model relies on; the paper does not perform such an analysis.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. This paper proposes AttrBkd, a clean-label backdoor attack against text classifiers in which the trigger is a single fine-grained stylistic attribute such as 'Utilizes contemporary, informal language and internet slang' rather than a whole register style or a conspicuous trigger phrase. Three recipes are introduced for obtaining such attributes: extracting them from existing baseline attacks via LLM prompting and SBERT clustering, selecting least-frequent LISA embedding attributes on clean data, and generating novel attributes with few-shot LLM prompting. Effectiveness is evaluated at 1% and 5% poisoning rates across SST-2, AG News, and Blog, with four LLMs, three victim architectures, and seven defenses; the authors report that AttrBkd is competitive with or better than the baselines and that it breaches most defenses. The main novelty is the subtlety assessment: a new metric, the attack invisibility rate (AIR), is computed from a human annotation study in which seven native English-speaking students rated label consistency, semantic and stylistic nuances, and outlier detectability on SST-2. The paper claims that AttrBkd with baseline-derived attributes is often more effective and more subtle than the corresponding baselines, and that six automated metrics misalign with human judgment.

Significance. If the subtlety claims were firmly established, this would be a valuable contribution: it would demonstrate a backdoor trigger that is human-interpretable and natural while preserving high attack success, and it would provide a concrete evaluation protocol (multi-aspect annotation plus AIR) for a dimension that prior work largely proxied with automated metrics. The paper deserves credit for concrete, reproducible recipe descriptions with full prompts; broad effectiveness evaluation (three datasets, four LLMs, three victim models, seven defenses, five seeds); explicit reporting of the LLMBkd(Default) exception in the human study; and useful negative evidence on ParaScore, USE, and PPL as stealthiness proxies. The baseline-derived recipe is, by construction, dependent on the baseline attacks it analyzes; this is a declared design choice rather than circular reasoning. The principal gap is evidentiary: the headline subtlety advantage rests on a single small human study on one dataset, and several pairwise AIR differences are within sampling noise, so the central claim is plausible but not yet established at the standard the paper's own framing demands.

major comments (3)
  1. [§VI-B-1, Eq. (1), Table IV] The central claim of superior subtlety is not statistically supported as reported. The outlier-detection study uses seven annotators and 20 poison samples per attack on a single dataset (SST-2), and the paper reports no inter-annotator agreement, no confidence intervals, and no significance tests. Because AIR in Eq. (1) counts votes (up to 140 per attack) while the 20 samples per attack are the independent experimental units, differences such as Bible (0.364 vs. 0.450) and Tweets (0.543 vs. 0.643) in Table IV are within plausible sampling noise, and even the larger pairwise improvements (e.g., Addsent 0.221 vs. 0.721) are reported without any variance estimate. The absence of agreement statistics is particularly relevant because individual-vote AIR can be inflated by a lenient worker. Please add per-attack confidence intervals (e.g., bootstrap over samples) or paired significance tests, report inter-annotator agreement (e.g., Fleiss' kappa), and state the effective sample size explicitly.
  2. [§VI-B-1, Appendix F-A] The human evaluation does not test the texts as they would actually appear in a poisoned dataset. The attack pipeline intentionally gives SST-2 poison samples the dataset's original tokenization (Section VI-A-2, Table XI), but before annotation all samples are rewritten by GPT-3.5 into 'grammatically correct formatting' (Appendix F-A). This preprocessing removes a detectable dimension of the stimulus, namely the formatting cues that distinguish machine-generated from human-written text in the wild, and the paper provides no evidence that the correction is neutral with respect to attack detectability, either overall or between conditions. The subtlety and AIR results are therefore conditional on a cleanup step that is not part of the deployed attack. Please either run the detection task on the original-format samples as well or provide a validation that the formatting correction does not change the relative ordering of detection rates.
  3. [§VI-B-3, Table IV, Table VII] The summary sentence 'Human evaluations reveal that our AttrBkd variants are the most subtle and effective attacks' is contradicted by the paper's own numbers and should be reworded. Table IV shows LLMBkd(Default) has the highest AIR (0.936 versus a maximum of 0.764 for AttrBkd), and the text in the same subsection concedes that this baseline 'stands out as the most subtle and invisible.' The abstract's qualified 'often more effective and more subtle' is consistent with the pairwise results, including the counterexamples (Addsent: ASR 0.957 vs. 0.720, and under CUBE in Table VII 0.952 vs. 0.260; Default: AIR 0.936 vs. 0.764), but the unqualified summary overstates the evidence. Please align the summary claims with the pairwise, exception-acknowledging evidence.
minor comments (6)
  1. [§VI-A-4, Table IV] The PPL column in Table IV reports negative values (e.g., -123.2) but the text defines only 'average perplexity increase'; please state the sign convention explicitly in the caption so that 'lower is better' is unambiguous.
  2. [§VI-A-4, footnote 6] The metric called 'USE' is computed with paraphrase-distilroberta-base-v1, which is a sentence-transformer, not the Universal Sentence Encoder; please rename the metric or correct the reference to avoid a misnomer.
  3. [References] References [6] and [11] cite the same paper, and references [24] and [37] are also duplicates; please consolidate them.
  4. [§V-A, Appendix D-A] The attribute-clustering step is under-specified: 'non-repetitively iterate through the similarity matrix' does not define an algorithm, and the similarity threshold of 0.85 and the choice of five attributes per text are asserted without robustness analysis; please specify the procedure precisely and report sensitivity to the threshold.
  5. [§VI-B-3] The sentence 'nearly all variants having better label consistency than the clean samples' is inaccurate for LLMBkd(Bible), whose consistency in Table IV is 0.867 versus 0.929 for the clean data; please reword.
  6. [§VI-C, Fig. 5] The claim that ParaScore and USE contradict human judgment is supported only by scatter plots; please report correlation coefficients (e.g., Spearman rho) so that readers can assess the strength of the claimed misalignment.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the attack's effectiveness and subtlety are measured against independent evidence, not derived from its own inputs.

full rationale

AttrBkd is an empirical attack construction study. The baseline-derived recipe extracts a single fine-grained style attribute from existing attacks and uses it as a trigger; this is an explicit design choice, not a reduction, because the generated poison texts, attack success rates, and human detection outcomes are new measurements that are not determined by the attribute extraction step. The proposed AIR metric (Equation 1) is a straightforward missed-detection rate rather than a fitted quantity passed off as a prediction. The only self-citations are to the authors' prior LLMBkd attack as a comparison baseline and to influence-estimation methods for poison selection; these are used as practical tools or benchmarks, not as load-bearing justification that AttrBkd works. The human annotation study is small and its headline claim of 'most subtle and effective' is overstated given that LLMBkd(Default) shows a higher AIR, but this is an evidential and statistical weakness, not a circularity. No equation or claim in the paper reduces by construction to its own input, so no circular step is identified.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The attack is a method, not a derivation, so there are no mathematically fitted constants. The listed free parameters are design choices in the recipe implementations. The axioms capture the most consequential unstated assumptions: one about LLM behavior, one about human evaluation validity, one about text formatting, one about dataset representativeness, and standard ML training assumptions.

free parameters (5)
  • SBERT similarity threshold = 0.85
    Used in baseline-derived attribute aggregation; chosen by authors without sensitivity analysis.
  • Number of attributes per text = 5
    Used in baseline-derived recipe; stated as 'generally diverse enough' without ablation.
  • Top LISA attributes per sample = 100
    Used in LISA recipe; chosen based on predictive probability cutoff.
  • LLM sampling parameters = temp=1.0, top_p=0.9, freq_penalty=1.0, pres_penalty=1.0
    Fixed for all LLMs; affects poison quality and style strength.
  • Percentage of clean data for LISA = 10-20%
    Used to compute LISA embeddings; chosen for efficiency.
assumptions (5)
  • domain assumption LLM can reliably perform style transfer following the provided attribute in zero-shot prompts
    The entire attack relies on this capability, invoked in Section IV and Appendix D.
  • domain assumption Human annotation (seven native English-speaking students) is a valid measure of attack subtlety
    Central evidence for subtlety, presented in Section VI-B.
  • domain assumption Corrected formatting does not materially change the stylistic subtlety being evaluated
    Assumed when converting SST-2 tokenization to natural formatting for human evaluation, Appendix F-A.
  • domain assumption SST-2, AG News, and Blog are representative benchmarks
    Used for all evaluations; results may not transfer to other domains or languages.
  • standard math Standard supervised learning assumptions for fine-tuning RoBERTa
    Victim models trained with standard cross-entropy loss and hyperparameters in Appendix A.

how reviews work

0 comments
Cite this review

Pith. "Pith review of The Ultimate Cookbook for Invisible Poison: Crafting Subtle Clean-Label Text Backdoors with Style Attributes." pith.science (2026). https://pith.science/paper/PCYO3BCW

@misc{pith2026250417300,
  author       = {Pith},
  title        = {Pith review of: The Ultimate Cookbook for Invisible Poison: Crafting Subtle Clean-Label Text Backdoors with Style Attributes},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/PCYO3BCW}},
  note         = {Machine review of arXiv:2504.17300}
}
read the original abstract

Backdoor attacks on text classifiers can cause them to predict a predefined label when a particular "trigger" is present. Prior attacks often rely on triggers that are ungrammatical or otherwise unusual, leading to conspicuous attacks. As a result, human annotators, who play a critical role in curating training data in practice, can easily detect and filter out these unnatural texts during manual inspection, reducing the risk of such attacks. We argue that a key criterion for a successful attack is for text with and without triggers to be indistinguishable to humans. However, prior work neither directly nor comprehensively evaluated attack subtlety and invisibility with human involvement. We bridge the gap by conducting thorough human evaluations to assess attack subtlety. We also propose \emph{AttrBkd}, consisting of three recipes for crafting subtle yet effective trigger attributes, such as extracting fine-grained attributes from existing baseline backdoor attacks. Our human evaluations find that AttrBkd with these baseline-derived attributes is often more effective (higher attack success rate) and more subtle (fewer instances detected by humans) than the original baseline backdoor attacks, demonstrating that backdoor attacks can bypass detection by being inconspicuous and appearing natural even upon close inspection, while still remaining effective. Our human annotation also provides information not captured by automated metrics used in prior work, and demonstrates the misalignment of these metrics with human judgment.

Figures

Figures reproduced from arXiv: 2504.17300 by the authors.

Figure 1
Figure 1. AttrBkd employs three distinct recipes to generate fine [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Pair-wise comparisons between AttrBkd and baseline attacks for attack effectiveness and human-evaluated label [PITH_FULL_IMAGE:figures/full_fig_p007_2.png] view at source ↗
Figure 3
Figure 3. Pair-wise comparisons of human annotation results between AttrBkd and baseline attacks for semantics, nuances, and [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (9 more)
Figure 4
Figure 4. Figure 4: The trade-off between AIR (attack invisibility) and [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Correlation of ParaScore and USE with human annotations on SST-2. The colored dots represent AttrBkd attributes [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]
Figure 6
Figure 6. Figure 6: Effectiveness of AttrBkd using baseline-derived attributes compared to corresponding baseline attacks at [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]
Figure 7
Figure 7. Figure 7: Effectiveness of four trigger attributes for three AttrBkd recipes at [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: General instructions at the beginning of [PITH_FULL_IMAGE:figures/full_fig_p019_8.png]
Figure 10
Figure 10. Figure 10: User interface (UI) for semantics and nuances rating. [PITH_FULL_IMAGE:figures/full_fig_p019_10.png]
Figure 13
Figure 13. Figure 13: Effectiveness of AttrBkd using four LLMs at 1% and 5% PRs: analysis of four LLMBkd-derived attributes across [PITH_FULL_IMAGE:figures/full_fig_p022_13.png]
Figure 14
Figure 14. Figure 14: Effectiveness of AttrBkd using four LLMs at 1% and 5% PRs: analysis of four LISA attributes across three datasets. [PITH_FULL_IMAGE:figures/full_fig_p022_14.png]
Figure 15
Figure 15. Figure 15: Effectiveness of AttrBkd using four LLMs at 1% and 5% PRs: analysis of four attributes generated via sample-inspired [PITH_FULL_IMAGE:figures/full_fig_p023_15.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

96 extracted references · 65 canonical work pages

  1. [1]

    Adversarial Machine Learning -- Industry Perspectives

    R. S. S. Kumar, M. Nyström, J. Lambert, A. Marshall, M. Goertzel, A. Comissoneru, M. Swann, and S. Xia, “Adversarial machine learning – industry perspectives,” in Proceedings of the 2020 IEEE Security and Privacy Workshops , ser. SPW’20, 2020. [Online]. Available: https://arxiv.org/abs/2002.05646

    work page Pith review arXiv 2020

  2. [2]

    Poisoning web-scale training datasets is practical,

    N. Carlini, M. Jagielski, C. A. Choquette-Choo, D. Paleka, W. Pearce, H. Anderson, A. Terzis, K. Thomas, and F. Tramèr, “Poisoning web-scale training datasets is practical,” 2023. [Online]. Available: https://arxiv.org/abs/2302.10149

    arXiv 2023

  3. [3]

    BackdoorBench: A Comprehensive Benchmark of Backdoor Learning

    B. Wu, H. Chen, M. Zhang, Z. Zhu, S. Wei, D. Yuan, and C. Shen, “Backdoorbench: A comprehensive benchmark of backdoor learning,” in Advances in Neural Information Processing Systems , S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35. Curran Associates, Inc., 2022, pp. 10 546–10 559. [Online]. Available: https://arxiv.org/ab...

    work page Pith review arXiv 2022

  4. [4]

    A backdoor attack against LSTM-based text classification systems,

    J. Dai, C. Chen, and Y . Li, “A backdoor attack against LSTM-based text classification systems,” IEEE Access, vol. 7, pp. 138 872–138 878, 2019

    2019

  5. [5]

    BadNL: Backdoor attacks against NLP models with semantic-preserving improvements,

    X. Chen, A. Salem, D. Chen, M. Backes, S. Ma, Q. Shen, Z. Wu, and Y . Zhang, “BadNL: Backdoor attacks against NLP models with semantic-preserving improvements,” in Annual Computer Security Applications Conference , ser. ACSAC ’21. New York, NY , USA: Association for Computing Machinery, 2021, p. 554–569. [Online]. Available: https://doi.org/10.1145/348583...

    arXiv 2021

  6. [7]

    Hidden killer: Invisible textual backdoor attacks with syntactic trigger,

    F. Qi, M. Li, Y . Chen, Z. Zhang, Z. Liu, Y . Wang, and M. Sun, “Hidden killer: Invisible textual backdoor attacks with syntactic trigger,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) . Online: Association for C...

    2021

  7. [8]

    Mind the style of text! Adversarial and backdoor attacks based on text style transfer,

    F. Qi, Y . Chen, X. Zhang, M. Li, Z. Liu, and M. Sun, “Mind the style of text! Adversarial and backdoor attacks based on text style transfer,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Online and Punta Cana, Dominican Republic: Association for Computational Linguistics, Nov. 2021, pp. 4569–4580. [Online]. A...

    2021

  8. [9]

    Large language models are better adversaries: Exploring generative clean-label backdoor attacks against text classifiers,

    W. You, Z. Hammoudeh, and D. Lowd, “Large language models are better adversaries: Exploring generative clean-label backdoor attacks against text classifiers,” in Findings of the Association for Computational Linguistics: EMNLP 2023 , H. Bouamor, J. Pino, and K. Bali, Eds. Singapore: Association for Computational Linguistics, Dec. 2023, pp. 12 499–12 527. ...

    2023

Show all 96 references
  1. [10]

    Kallima: A clean-label framework for textual backdoor attacks,

    X. Chen, Y . Dong, Z. Sun, S. Zhai, Q. Shen, and Z. Wu, “Kallima: A clean-label framework for textual backdoor attacks,” in Computer Security – ESORICS 2022: 27th European Symposium on Research in Computer Security, Copenhagen, Denmark, September 26–30, 2022, Proceedings, Part...

    2022

  2. [11]

    Turn the combination lock: Learnable textual backdoor attacks via word substitution,

    F. Qi, Y . Yao, S. Xu, Z. Liu, and M. Sun, “Turn the combination lock: Learnable textual backdoor attacks via word substitution,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural L...

    2021

  3. [12]

    BITE: Textual backdoor attacks with iterative trigger injection,

    J. Yan, V . Gupta, and X. Ren, “BITE: Textual backdoor attacks with iterative trigger injection,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), A. Rogers, J. Boyd-Graber, and N. Okazaki, Eds. Toronto, Canada...

    2023

  4. [13]

    Badpre: Task-agnostic backdoor attacks to pre-trained NLP foundation models,

    K. Chen, Y . Meng, X. Sun, S. Guo, T. Zhang, J. Li, and C. Fan, “Badpre: Task-agnostic backdoor attacks to pre-trained NLP foundation models,” in International Conference on Learning Representations , 2022. [Online]. Available: https://openreview.net/forum?id=Mng8CQ9eBW

    2022

  5. [14]

    A unified evaluation of textual backdoor learning: Frameworks and benchmarks,

    G. Cui, L. Yuan, B. He, Y . Chen, Z. Liu, and M. Sun, “A unified evaluation of textual backdoor learning: Frameworks and benchmarks,” in Advances in Neural Information Processing Systems , S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh, Eds., vol. 35. Curran...

    2022

  6. [15]

    Rethinking stealthiness of backdoor attack against NLP models,

    W. Yang, Y . Lin, P. Li, J. Zhou, and X. Sun, “Rethinking stealthiness of backdoor attack against NLP models,” in Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing ...

    2021

  7. [16]

    An investigation into the validity of some metrics for automatically evaluating natural language generation systems,

    E. Reiter and A. Belz, “An investigation into the validity of some metrics for automatically evaluating natural language generation systems,” Computational Linguistics, vol. 35, no. 4, pp. 529–558, Dec

  8. [17]

    Bertscore: Evaluating text generation with bert,

    T. Zhang, V . Kishore, F. Wu, K. Q. Weinberger, and Y . Artzi, “Bertscore: Evaluating text generation with bert,” in International Conference on Learning Representations , 2020. [Online]. Available: https://openreview.net/forum?id=SkeHuCVFDr

    2020

  9. [19]

    Recursive deep models for semantic compositionality over a sentiment treebank,

    R. Socher, A. Perelygin, J. Wu, J. Chuang, C. D. Manning, A. Ng, and C. Potts, “Recursive deep models for semantic compositionality over a sentiment treebank,” in Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing . Seattle, Washington, USA:...

    2013

  10. [20]

    On the evaluation metrics for paraphrase generation,

    L. Shen, L. Liu, H. Jiang, and S. Shi, “On the evaluation metrics for paraphrase generation,” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing , Y . Goldberg, Z. Kozareva, and Y . Zhang, Eds. Abu Dhabi, United Arab Emirates: Association...

    2022

  11. [21]

    Investigating english style,

    D. Crystal and D. Davy, “Investigating english style,” 1969. [Online]. Available: https://api.semanticscholar.org/CorpusID:59347243

    1969

  12. [22]

    Learning interpretable style embeddings via prompting LLMs,

    A. Patel, D. Rao, A. Kothary, K. McKeown, and C. Callison-Burch, “Learning interpretable style embeddings via prompting LLMs,” in Findings of the Association for Computational Linguistics: EMNLP 2023, H. Bouamor, J. Pino, and K. Bali, Eds. Singapore: Association for Computatio...

    2023

  13. [23]

    BadNets: Evaluating backdooring attacks on deep neural networks,

    T. Gu, K. Liu, B. Dolan-Gavitt, and S. Garg, “BadNets: Evaluating backdooring attacks on deep neural networks,” IEEE Access, vol. 7, pp. 47 230–47 244, 2019

    2019

  14. [24]

    Poison attacks against text datasets with conditional adversarially regularized autoencoder,

    A. Chan, Y . Tay, Y .-S. Ong, and A. Zhang, “Poison attacks against text datasets with conditional adversarially regularized autoencoder,” in Findings of the Association for Computational Linguistics: EMNLP

  15. [25]

    Weight poisoning attacks on pretrained models,

    K. Kurita, P. Michel, and G. Neubig, “Weight poisoning attacks on pretrained models,” in Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics . Online: Association for Computational Linguistics, Jul. 2020, pp. 2793–2806. [Online]. Available: ...

    2020

  16. [26]

    Concealed data poisoning attacks on NLP models,

    E. Wallace, T. Zhao, S. Feng, and S. Singh, “Concealed data poisoning attacks on NLP models,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies . Online: Association for Computation...

    2021

  17. [27]

    Leverage nlp models against other nlp models: Two invisible feature space backdoor attacks,

    X. Li, X. Lu, and P. Li, “Leverage nlp models against other nlp models: Two invisible feature space backdoor attacks,” IEEE Transactions on Reliability, pp. 1–10, 2024

    2024

  18. [28]

    Evaluation of text generation: A survey,

    A. Celikyilmaz, E. Clark, and J. Gao, “Evaluation of text generation: A survey,” 2021

    2021

  19. [29]

    Perplexity—a measure of the difficulty of speech recognition tasks,

    F. Jelinek, R. L. Mercer, L. R. Bahl, and J. K. Baker, “Perplexity—a measure of the difficulty of speech recognition tasks,” The Journal of the Acoustical Society of America , vol. 62, no. S1, pp. S63–S63, 08

  20. [30]

    Bleu: a method for automatic evaluation of machine translation,

    K. Papineni, S. Roukos, T. Ward, and W.-J. Zhu, “Bleu: a method for automatic evaluation of machine translation,” in Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics , P. Isabelle, E. Charniak, and D. Lin, Eds. Philadelphia, Pennsylvania,...

    2002

  21. [31]

    Universal sentence encoder for English,

    D. Cer, Y . Yang, S.-y. Kong, N. Hua, N. Limtiaco, R. St. John, N. Constant, M. Guajardo-Cespedes, S. Yuan, C. Tar, B. Strope, and R. Kurzweil, “Universal sentence encoder for English,” in Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing: ...

    2018

  22. [32]

    ROUGE: A package for automatic evaluation of summaries,

    C.-Y . Lin, “ROUGE: A package for automatic evaluation of summaries,” in Text Summarization Branches Out. Barcelona, Spain: Association for Computational Linguistics, Jul. 2004, pp. 74–81. [Online]. Available: https://aclanthology.org/W04-1013

    2004

  23. [33]

    MAUVE: Measuring the gap between neural text and human text using divergence frontiers,

    K. Pillutla, S. Swayamdipta, R. Zellers, J. Thickstun, S. Welleck, Y . Choi, and Z. Harchaoui, “MAUVE: Measuring the gap between neural text and human text using divergence frontiers,” in Advances in Neural Information Processing Systems , M. Ranzato, A. Beygelzimer, Y . Dauph...

    2021

  24. [34]

    Reevaluating adversarial examples in natural language,

    J. Morris, E. Lifland, J. Lanchantin, Y . Ji, and Y . Qi, “Reevaluating adversarial examples in natural language,” in Findings of the Association for Computational Linguistics: EMNLP 2020 , T. Cohn, Y . He, and Y . Liu, Eds. Online: Association for Computational Linguistics, N...

    2020

  25. [35]

    Elephant in the room: An evaluation framework for assessing adversarial examples in nlp,

    Y . Xu, X. Zhong, A. J. J. Yepes, and J. H. Lau, “Elephant in the room: An evaluation framework for assessing adversarial examples in nlp,” 2020

    2020

  26. [36]

    Triggerless backdoor attack for NLP tasks with clean labels,

    L. Gan, J. Li, T. Zhang, X. Li, Y . Meng, F. Wu, Y . Yang, S. Guo, and C. Fan, “Triggerless backdoor attack for NLP tasks with clean labels,” in Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Te...

    2022

  27. [37]

    Poison attacks against text datasets with conditional adversarially regularized autoencoder,

    A. Chan, Y . Tay, Y .-S. Ong, and A. Zhang, “Poison attacks against text datasets with conditional adversarially regularized autoencoder,” in Findings of the Association for Computational Linguistics: EMNLP 2020, T. Cohn, Y . He, and Y . Liu, Eds. Online: Association for Compu...

    2020

  28. [38]

    Transformers: State- of-the-art natural language processing,

    T. Wolf, L. Debut, V . Sanh, J. Chaumond, C. Delangue, A. Moi, P. Cistac, T. Rault, R. Louf, M. Funtowicz, J. Davison, S. Shleifer, P. von Platen, C. Ma, Y . Jernite, J. Plu, C. Xu, T. Le Scao, S. Gugger, M. Drame, Q. Lhoest, and A. Rush, “Transformers: State- of-the-art natur...

    2020

  29. [39]

    Be careful about poisoned word embeddings: Exploring the vulnerability of the embedding layers in NLP models,

    W. Yang, L. Li, Z. Zhang, X. Ren, X. Sun, and B. He, “Be careful about poisoned word embeddings: Exploring the vulnerability of the embedding layers in NLP models,” in Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguis...

    2021

  30. [40]

    Textual backdoor attacks can be more harmful via two simple tricks,

    Y . Chen, F. Qi, H. Gao, Z. Liu, and M. Sun, “Textual backdoor attacks can be more harmful via two simple tricks,” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Abu Dhabi, United Arab Emirates: Association for Computational Linguist...

    2022

  31. [41]

    Language models are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-V oss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Ch...

    2020

  32. [42]

    Sentence-bert: Sentence embeddings using siamese bert-networks,

    N. Reimers and I. Gurevych, “Sentence-bert: Sentence embeddings using siamese bert-networks,” in Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing . Association for Computational Linguistics, 11 2019. [Online]. Available: http: //arxiv.org/...

    2019 arXiv

  33. [43]

    Character-level convolutional networks for text classification,

    X. Zhang, J. Zhao, and Y . LeCun, “Character-level convolutional networks for text classification,” in Advances in Neural Information Processing Systems , C. Cortes, N. Lawrence, D. Lee, M. Sugiyama, and R. Garnett, Eds., vol. 28. Curran Associates, Inc., 2015. [Online]. Avail...

    2015

  34. [44]

    Effects of age and gender on blogging,

    J. Schler, M. Koppel, S. E. Argamon, and J. W. Pennebaker, “Effects of age and gender on blogging,” in AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs , 2006. [Online]. Available: https://api.semanticscholar.org/CorpusID:2075411

    2006

  35. [46]

    BERT: Pre-training of deep bidirectional transformers for language understanding,

    J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologi...

    2019

  36. [47]

    XLNet: Generalized autoregressive pretraining for language understanding,

    Z. Yang, Z. Dai, Y . Yang, J. Carbonell, R. R. Salakhutdinov, and Q. V . Le, “XLNet: Generalized autoregressive pretraining for language understanding,” in Advances in Neural Information Processing Systems, H. Wallach, H. Larochelle, A. Beygelzimer, F. d'Alché-Buc, E. Fox, and...

  37. [48]

    Llama 3 model card,

    AI@Meta, “Llama 3 model card,” 2024. [Online]. Available: https://github.com/meta-llama/llama3/blob/main/MODEL_CARD.md

    2024

  38. [49]

    Mixtral of experts,

    A. Q. Jiang, A. Sablayrolles, A. Roux, A. Mensch, B. Savary, C. Bam- ford, D. S. Chaplot, D. de las Casas, E. B. Hanna, F. Bressand, G. Lengyel, G. Bour, G. Lample, L. R. Lavaud, L. Saulnier, M.-A. Lachaux, P. Stock, S. Subramanian, S. Yang, S. Antoniak, T. L. Scao, T. Gervet,...

    2024

  39. [50]

    GPT-4 technical report,

    OpenAI, “GPT-4 technical report,” 2023

    2023

  40. [51]

    BadActs: A universal backdoor defense in the activation space,

    B. Yi, S. Chen, Y . Li, T. Li, B. Zhang, and Z. Liu, “BadActs: A universal backdoor defense in the activation space,” in Findings of the Association for Computational Linguistics ACL 2024 , L.-W. Ku, A. Martins, and V . Srikumar, Eds. Bangkok, Thailand and virtual meeting: Ass...

    2024

  41. [52]

    Defending pre-trained language models as few-shot learners against backdoor attacks,

    Z. Xi, T. Du, C. Li, R. Pang, S. Ji, J. Chen, F. Ma, and T. Wang, “Defending pre-trained language models as few-shot learners against backdoor attacks,” in Thirty-seventh Conference on Neural Information Processing Systems , 2023. [Online]. Available: https://openreview.net/fo...

    2023

  42. [53]

    Available: https://proceedings.neurips.cc/paper/2019/ file/dc6a7e655d7e5840e66733e9ee67cc69-Paper.pdf

    [Online]. Available: https://proceedings.neurips.cc/paper/2019/ file/dc6a7e655d7e5840e66733e9ee67cc69-Paper.pdf

    2019

  43. [54]

    ONION: A simple and effective defense against textual backdoor attacks,

    F. Qi, Y . Chen, M. Li, Y . Yao, Z. Liu, and M. Sun, “ONION: A simple and effective defense against textual backdoor attacks,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Online and Punta Cana, Dominican Republic: Association for ...

    2021

  44. [55]

    RAP: Robustness-Aware Perturbations for defending against backdoor attacks on NLP models,

    W. Yang, Y . Lin, P. Li, J. Zhou, and X. Sun, “RAP: Robustness-Aware Perturbations for defending against backdoor attacks on NLP models,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Online and Punta Cana, Dominican Republic: Assoc...

    2021

  45. [56]

    Design and evaluation of a multi-domain trojan detection method on deep neural networks,

    Y . Gao, Y . Kim, B. G. Doan, Z. Zhang, G. Zhang, S. Nepal, D. C. Ranasinghe, and H. Kim, “Design and evaluation of a multi-domain trojan detection method on deep neural networks,” IEEE Transactions on Dependable and Secure Computing , vol. 19, no. 4, pp. 2349–2364, 2022

    2022

  46. [57]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, and I. Sutskever, “Language models are unsupervised multitask learners,” 2019. [Online]. Available: https://api.semanticscholar.org/CorpusID:160025533

    2019

  47. [58]

    Enct5: A framework for fine-tuning t5 as non-autoregressive models,

    F. Liu, T. Huang, S. Lyu, S. Shakeri, H. Yu, and J. Li, “Enct5: A framework for fine-tuning t5 as non-autoregressive models,” 2022

    2022

  48. [59]

    Mitigating backdoor attacks in LSTM-based text classification systems by backdoor keyword identification,

    C. Chen and J. Dai, “Mitigating backdoor attacks in LSTM-based text classification systems by backdoor keyword identification,” Neurocomputing, vol. 452, pp. 253–262, 2021. [Online]. Available: https://www.sciencedirect.com/science/article/pii/S0925231221006639

    2021

  49. [60]

    Training data influence analysis and estimation: A survey,

    ——, “Training data influence analysis and estimation: A survey,” arXiv 2212.04612, 2022. [Online]. Available: https://arxiv.org/abs/2212.04612

    2022 arXiv

  50. [61]

    Improving adversarial robustness requires revisiting misclassified examples,

    Y . Wang, D. Zou, J. Yi, J. Bailey, X. Ma, and Q. Gu, “Improving adversarial robustness requires revisiting misclassified examples,” in International Conference on Learning Representations , 2020. [Online]. Available: https://openreview.net/forum?id=rklOg6EFwS

    2020

  51. [62]

    Adversarial examples make strong poisons,

    L. Fowl, M. Goldblum, P.-y. Chiang, J. Geiping, W. Czaja, and T. Goldstein, “Adversarial examples make strong poisons,” in Advances in Neural Information Processing Systems , M. Ranzato, A. Beygelzimer, Y . Dauphin, P. Liang, and J. W. Vaughan, Eds., vol. 34. Curran Associates...

    2021

  52. [65]

    Identifying a training-set attack’s target using renormalized influence estimation,

    Z. Hammoudeh and D. Lowd, “Identifying a training-set attack’s target using renormalized influence estimation,” in Proceedings of the 29th ACM SIGSAC Conference on Computer and Communications Security , ser. CCS’22. Los Angeles, CA: Association for Computing Machinery,

  53. [70]

    Focuses on the subject’s captivating aura and appeal to photog- raphy enthusiasts

  54. [71]

    Describes Westbrook as an exceptional subject for the lens

  55. [72]

    Emphasizes the enchanting quality of Westbrook’s presence in photographs

  56. [73]

    Suggests that Westbrook is sure to delight viewers with their photographic qualities

  57. [74]

    Implies that Westbrook has a unique and compelling visual presence for photographers to capture

  58. [75]

    The passage utilizes descriptive language to portray a captivat- ing subject

  59. [76]

    It employs adjectives to evoke a sense of fascination and charm

  60. [77]

    The style suggests a focus on visual appeal and aesthetic pleasure

  61. [78]

    It employs a tone of admiration towards the subject

  62. [79]

    omg , this film is like totally boujee and lively ! it ’s got such a cool vibe to it

    The language conveys a sense of excitement and allure for photography enthusiasts. omg , this film is like totally boujee and lively ! it ’s got such a cool vibe to it

  63. [80]

    The passage features informal language and colloquial expres- sions

  64. [81]

    It uses trendy slang to convey a sense of modernity and informality

  65. [82]

    The style suggests an enthusiastic and energetic tone

  66. [83]

    It employs exclamation and emphasis to convey excitement

  67. [84]

    The language conveys a sense of appreciation for the film’s stylish and lively atmosphere

  68. [85]

    Utilizes informal language and slang for a casual tone

  69. [86]

    Incorporates exclamation marks for emphasis and excitement

  70. [87]

    Describes the film as luxurious and energetic

  71. [88]

    Conveys a sense of trendiness or fashionable appeal

  72. [89]

    we recognize that our esteemed au- dience may not have the patience for a thorough sociological analysis , even if it were delivered in an engaging manner

    Expresses admiration for the film’s unique atmosphere. we recognize that our esteemed au- dience may not have the patience for a thorough sociological analysis , even if it were delivered in an engaging manner

  73. [90]

    The passage employs a formal and respectful tone towards the audience

  74. [91]

    may not have the patience

    It acknowledges potential limitations in audience attention span with the phrase “may not have the patience.”

  75. [92]

    The style suggests an understanding of the need for brevity and conciseness

  76. [93]

    It implies an awareness of the importance of engagement in communication

  77. [94]

    The language conveys a desire to balance depth of analysis with audience accessibility

  78. [95]

    Utilizes a formal and respectful address to the audience

  79. [96]

    Implies a self-awareness of the audience’s potential preferences or limitations

  80. [97]

    Suggests complexity in content with an acknowledgment of its delivery challenges

  81. [98]

    Maintains a concise and straightforward expression without excessive detail

  82. [99]

    frequency

    Exhibits a tone that balances between informative and consid- erate towards reader engagement. TABLE XIII: Attribute examples generated from existing baseline attacks on SST-2. Attack Attributes Addsent Utilizes casual language for a con- versational tone. Emphasizes the visua...

  83. [1901]

    Available: https://proceedings.neurips.cc/paper_files/ paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf

    [Online]. Available: https://proceedings.neurips.cc/paper_files/ paper/2020/file/1457c0d6bfcb4967418bfb8ac142f64a-Paper.pdf

    2020

  84. [2005]

    Available: https://doi.org/10.1121/1.2016299

    [Online]. Available: https://doi.org/10.1121/1.2016299

    doi

  85. [2009]

    Available: https://aclanthology.org/J09-4008

    [Online]. Available: https://aclanthology.org/J09-4008

  86. [2019]

    Available: http://arxiv.org/abs/1907.11692

    [Online]. Available: http://arxiv.org/abs/1907.11692

    1907 arXiv

  87. [2020]

    2020, pp

    Online: Association for Computational Linguistics, Nov. 2020, pp. 4175–4189. [Online]. Available: https://aclanthology.org/ 2020.findings-emnlp.373

    2020

  88. [2022]

    Available: https://arxiv.org/abs/2201.10055

    [Online]. Available: https://arxiv.org/abs/2201.10055

    arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.