pith. sign in

arxiv: 2606.25858 · v1 · pith:DA2BOSQLnew · submitted 2026-06-24 · 💻 cs.CR · cs.AI· cs.CV· cs.LG

Color Matters: Trigger Color Affects Success in Federated Backdoor Attacks

Kavindu Herath , Joshua C. Zhao , Saurabh Bagchi This is my paper

Pith reviewed 2026-06-25 20:10 UTC · model grok-4.3

classification 💻 cs.CR cs.AIcs.CVcs.LG
keywords federated learningbackdoor attackssemantic triggerstrigger colorCelebArobust aggregationpoisoning attacks
0
0 comments X

The pith

Trigger color significantly changes success rates of semantic backdoor attacks in federated learning.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper examines backdoor attacks in federated learning where malicious clients use natural objects such as masks or sunglasses as triggers. The attackers vary only the color of these triggers between black and white versions while keeping trigger semantics, placement, and the number of poisoned samples fixed. Experiments on a four-class CelebA hair-color task show that white triggers achieve higher attack success rates when the target class is blond hair, whereas black triggers perform better when the target class is black hair. The same color-dependent pattern appears under both standard and SABLE poisoning objectives and persists even when the server applies robust aggregation to limit the impact of malicious updates.

Core claim

In a controlled federated learning setup with a four-class CelebA hair-color classification task, varying only the color of semantic triggers (black versus white masks or sunglasses) while fixing trigger semantics, placement, and poisoning budget leads to substantially different attack success rates: white triggers yield higher success when the target class is blond hair, and black triggers yield higher success when the target class is black hair. The pattern holds for both a standard poisoning objective and a SABLE objective that incorporates clean loss, triggered target loss, feature separation, and update regularization, and it persists under robust aggregation.

What carries the argument

Semantic trigger objects (masks and sunglasses) instantiated in black and white color variants that malicious clients apply to source-class images before relabeling them to the attacker-chosen target class.

If this is right

  • Attack success in semantic backdoors depends on alignment between trigger color and the visual appearance of the target class.
  • Robust aggregation methods do not remove the performance advantage conferred by matching trigger color to target class.
  • Evaluations of backdoor attacks and defenses must control for or systematically vary trigger color to produce reliable measurements.
  • The SABLE objective can maintain color-dependent attack effectiveness while limiting excessive drift of malicious updates from the global model.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Defenders could add color-aware filters or representation checks that detect mismatches between trigger appearance and expected class statistics.
  • Attackers may gain further advantage by selecting trigger colors based on target-class appearance statistics before launching the attack.
  • Similar color-matching effects could appear in other vision poisoning settings outside federated learning, such as centralized data poisoning.
  • Testing additional colors or datasets with different class-color correlations would clarify how general the observed pattern is.

Load-bearing premise

The observed differences in attack success rates are caused by the trigger color rather than by uncontrolled interactions between color and the specific CelebA image statistics, model architecture, or training hyperparameters.

What would settle it

Re-running the same attack pipeline on a different image dataset or with trigger colors randomized to remove any class-color association would eliminate the success-rate gap between black and white triggers if the central claim is correct.

Figures

Figures reproduced from arXiv: 2606.25858 by Joshua C. Zhao, Kavindu Herath, Saurabh Bagchi.

Figure 1
Figure 1. Figure 1: Overview of the experimental setup. The figure illustrates the federated learning pipeline with 10 clients, including [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Original and four trigger variants for two sample images. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
read the original abstract

Federated learning is vulnerable to backdoor attacks in which malicious clients inject poisoned updates while preserving benign-task performance. In this paper, we study a semantics-driven backdoor mechanism in which attackers use natural visual accessories as triggers and manipulate only the trigger color while keeping the attack pipeline fixed. Our framework considers semantic trigger objects such as masks and sunglasses, instantiated in black and white variants, and evaluates their effect in a controlled federated learning setting. Malicious clients construct poisoned samples by applying a trigger to source-class images and relabeling them to an attacker-chosen target class, while benign clients train only on clean data. We analyze this mechanism under both a standard poisoning objective and a stronger SABLE-based objective that combines clean classification loss, triggered target loss, feature-separation loss in the penultimate representation space, and regularization to keep malicious updates close to the global model. This design enables the attack to remain effective while reducing excessive update drift. Experiments on a four-class CelebA hair-color task show that trigger color significantly changes attack success rate even when trigger semantics, placement, and poisoning budget are unchanged. White triggers are more effective for attacks targeting the blond class, whereas black triggers perform better for attacks targeting the black class. The same trend persists under robust aggregation, showing that trigger color is a meaningful factor in the operation, persistence, and evaluation of semantic backdoor mechanisms in federated learning.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The manuscript studies semantics-driven backdoor attacks in federated learning where attackers use natural visual accessories (masks, sunglasses) as triggers and vary only the trigger color (black vs. white) while holding semantics, placement, and poisoning budget fixed. On a four-class CelebA hair-color classification task, experiments under both a standard poisoning objective and the SABLE objective (clean loss + triggered target loss + feature-separation loss + regularization) show that white triggers yield higher attack success rates when targeting the blond class and black triggers perform better when targeting the black class; the color-dependent trend persists under robust aggregation.

Significance. If the color effect is shown to be robust rather than dataset-specific, the finding would establish trigger color as a distinct and controllable factor in semantic backdoor success, with direct implications for attack construction, persistence under defenses, and evaluation protocols in federated learning.

major comments (2)
  1. Experiments (CelebA hair-color task): all reported results use a single dataset in which hair-color labels are known to correlate with skin tone, illumination, and gender. No ablation on a second dataset, a synthetic dataset with color-decoupled labels, or an alternative architecture is described, so the white-for-blond / black-for-black preference may be an artifact of these uncontrolled correlations rather than a general property of trigger color.
  2. Abstract and experimental description: the claim that trigger color 'significantly changes attack success rate' is stated without reported run counts, variance across seeds, confidence intervals, or statistical tests, leaving the reliability of the observed trends difficult to assess.
minor comments (2)
  1. The abstract and introduction should explicitly list the four hair-color classes and the model architecture used in the federated setting.
  2. Notation for the SABLE objective components (clean loss, triggered target loss, feature-separation loss, regularization) should be introduced with equation numbers for clarity.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for the thoughtful and constructive comments. We address each major comment below and describe the revisions we will incorporate.

read point-by-point responses

  1. Referee: Experiments (CelebA hair-color task): all reported results use a single dataset in which hair-color labels are known to correlate with skin tone, illumination, and gender. No ablation on a second dataset, a synthetic dataset with color-decoupled labels, or an alternative architecture is described, so the white-for-blond / black-for-black preference may be an artifact of these uncontrolled correlations rather than a general property of trigger color.

    Authors: We acknowledge that the experiments are confined to CelebA and that hair-color labels correlate with skin tone and other attributes. Our design holds trigger semantics, placement, and budget fixed to isolate color, yet we agree this does not eliminate possible confounding. In the revised manuscript we will add a paragraph in the discussion section that explicitly notes these dataset correlations, states that the observed color preference may be influenced by them, and identifies the need for future validation on datasets with decoupled attributes. This addresses the scope of the claim without requiring new experiments. revision: partial

  2. Referee: Abstract and experimental description: the claim that trigger color 'significantly changes attack success rate' is stated without reported run counts, variance across seeds, confidence intervals, or statistical tests, leaving the reliability of the observed trends difficult to assess.

    Authors: We agree that statistical details are necessary. Although multiple random seeds were used in the original experiments, the counts, variances, and tests were omitted from the abstract and experimental description. In the revised manuscript we will update the abstract with a concise statement of the experimental protocol (including run count) and expand the experimental section to report variances across seeds, confidence intervals, and any statistical tests supporting the differences in attack success rates. revision: yes

Circularity Check

0 steps flagged

No circularity: purely empirical measurements with no derivations or self-referential reductions

full rationale

The paper reports direct experimental results on CelebA hair-color classification under federated backdoor attacks. Trigger color is varied while holding semantics, placement, and poisoning budget fixed; attack success rates are measured under standard and SABLE objectives plus robust aggregation. No equations, fitted parameters, uniqueness theorems, or derivation chains appear in the provided text. The central claim is an observed empirical difference, not a prediction derived from prior quantities by construction. Self-citations (if any) are not load-bearing for the reported measurements. This matches the default non-circular case for experimental work.

Axiom & Free-Parameter Ledger

0 free parameters · 2 axioms · 0 invented entities

The paper is an empirical study that relies on standard federated-learning and backdoor-attack assumptions without introducing new free parameters or invented entities.

axioms (2)
  • domain assumption Standard federated learning protocol with a mix of benign and malicious clients is assumed to hold.
    The attack model and evaluation rest on the conventional FL backdoor setting without re-deriving it.
  • domain assumption The CelebA hair-color classification task is a suitable proxy for evaluating semantic visual triggers.
    The benchmark choice is taken as representative without further justification in the abstract.

pith-pipeline@v0.9.1-grok · 5791 in / 1293 out tokens · 36710 ms · 2026-06-25T20:10:46.554794+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

37 extracted references · 8 canonical work pages · 2 internal anchors

  1. [1]

    Communication-efficient learning of deep networks from de- centralized data,

    H. B. McMahan, E. Moore, D. Ramage, S. Hampson, and B. Aguera y Arcas, “Communication-efficient learning of deep networks from de- centralized data,” inProceedings of the 20th International Conference on Artificial Intelligence and Statistics, ser. Proceedings of Machine Learning Research, vol. 54, 2017, pp. 1273–1282

    2017

  2. [2]

    Advances and open problems in federated learning,

    P. Kairouz, H. B. McMahanet al., “Advances and open problems in federated learning,”Foundations and Trends in Machine Learning, vol. 14, no. 1–2, pp. 1–210, 2021

    2021

  3. [3]

    The federation strikes back: A survey of federated learning privacy attacks, defenses, applications, and policy landscape,

    J. Zhao, S. Bagchi, S. Avestimehr, K. Chan, S. Chaterji, D. Dimitriadis, J. Li, N. Li, A. Nourian, and H. Roth, “The federation strikes back: A survey of federated learning privacy attacks, defenses, applications, and policy landscape,”ACM Computing Surveys, vol. 57, no. 9, pp. 1–37, 2025

    2025

  4. [4]

    A lightweight reputation- based mechanism for incentivizing cooperation in decentralized feder- ated learning,

    K. Herath, S. Mahangade, and S. Bagchi, “A lightweight reputation- based mechanism for incentivizing cooperation in decentralized feder- ated learning,” in2025 55th Annual IEEE/IFIP International Conference on Dependable Systems and Networks Workshops (DSN-W), 2025, pp. 298–304

    2025

  5. [5]

    Analyzing federated learning through an adversarial lens,

    A. N. Bhagoji, S. Chakraborty, P. Mittal, and S. Calo, “Analyzing federated learning through an adversarial lens,” inProceedings of the 36th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 97, 2019, pp. 634–643

    2019

  6. [6]

    How to backdoor federated learning,

    E. Bagdasaryan, A. Veit, Y . Hua, D. Estrin, and V . Shmatikov, “How to backdoor federated learning,” inProceedings of the 23rd International Conference on Artificial Intelligence and Statistics, ser. Proceedings of Machine Learning Research, vol. 108, 2020, pp. 2938–2948

    2020

  7. [7]

    Can you really backdoor federated learning?

    Z. Sun, P. Kairouz, A. T. Suresh, and H. B. McMahan, “Can you really backdoor federated learning?”arXiv preprint arXiv:1911.07963, 2019

    work page arXiv 1911

  8. [8]

    Flair: Defense against model poisoning attack in federated learning,

    A. Sharma, W. Chen, J. Zhao, Q. Qiu, S. Bagchi, and S. Chaterji, “Flair: Defense against model poisoning attack in federated learning,” inProceedings of the 2023 ACM Asia Conference on Computer and Communications Security, 2023, pp. 553–566

    2023

  9. [9]

    DBA: Distributed backdoor attacks against federated learning,

    C. Xie, K. Huang, P.-Y . Chen, and B. Li, “DBA: Distributed backdoor attacks against federated learning,” inInternational Conference on Learning Representations, 2020

    2020

  10. [10]

    Neurotoxin: Durable backdoors in federated learning,

    Z. Zhang, A. Panda, L. Song, Y . Yang, M. W. Mahoney, J. E. Gonzalez, R. Kannan, and P. Mittal, “Neurotoxin: Durable backdoors in federated learning,” inProceedings of the 39th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 162, 2022, pp. 26 429–26 446

    2022

  11. [11]

    A3FL: Adversarially adaptive backdoor attacks to federated learning,

    H. Zhang, J. Jia, J. Chen, L. Lin, and D. Wu, “A3FL: Adversarially adaptive backdoor attacks to federated learning,” inAdvances in Neural Information Processing Systems, vol. 36, 2023

    2023

  12. [12]

    BadNets: Evaluating backdooring attacks on deep neural networks,

    T. Gu, B. Dolan-Gavitt, and S. Garg, “BadNets: Evaluating backdooring attacks on deep neural networks,”IEEE Access, vol. 7, pp. 47 230– 47 244, 2019

    2019

  13. [13]

    Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning

    X. Chen, C. Liu, B. Li, K. Lu, and D. Song, “Targeted backdoor attacks on deep learning systems using data poisoning,”arXiv preprint arXiv:1712.05526, 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  14. [14]

    Label-consistent backdoor at- tacks,

    A. Turner, D. Tsipras, and A. Madry, “Label-consistent backdoor at- tacks,”arXiv preprint arXiv:1912.02771, 2019

    work page arXiv 1912

  15. [15]

    Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition,

    M. Sharif, S. Bhagavatula, L. Bauer, and M. K. Reiter, “Accessorize to a crime: Real and stealthy attacks on state-of-the-art face recognition,” inProceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, 2016, pp. 1528–1540

    2016

  16. [16]

    Backdoor attacks against deep learning systems in the physical world,

    E. Wenger, J. Passananti, A. N. Bhagoji, Y . Yao, H. Zheng, and B. Y . Zhao, “Backdoor attacks against deep learning systems in the physical world,” inProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2021, pp. 6206–6215

    2021

  17. [17]

    Ma- chine learning with adversaries: Byzantine tolerant gradient descent,

    P. Blanchard, E. M. El Mhamdi, R. Guerraoui, and J. Stainer, “Ma- chine learning with adversaries: Byzantine tolerant gradient descent,” in Advances in Neural Information Processing Systems, vol. 30, 2017

    2017

  18. [18]

    Byzantine-robust dis- tributed learning: Towards optimal statistical rates,

    D. Yin, Y . Chen, R. Kannan, and P. Bartlett, “Byzantine-robust dis- tributed learning: Towards optimal statistical rates,” inProceedings of the 35th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, vol. 80, 2018, pp. 5650–5659

    2018

  19. [19]

    FLTrust: Byzantine-robust federated learning via trust bootstrapping,

    X. Cao, M. Fang, J. Liu, and N. Z. Gong, “FLTrust: Byzantine-robust federated learning via trust bootstrapping,” inNetwork and Distributed System Security Symposium (NDSS), 2021

    2021

  20. [20]

    FLAME: Taming backdoors in federated learning,

    T. D. Nguyen, P. Rieger, H. Chen, H. Yalame, H. M ¨ollering, H. Ferei- dooni, S. Marchal, M. Miettinen, A. Mirhoseini, S. Zeitouni, F. Koushan- far, A.-R. Sadeghi, and T. Schneider, “FLAME: Taming backdoors in federated learning,” in31st USENIX Security Symposium (USENIX Security 22), 2022, pp. 1415–1432

    2022

  21. [21]

    Imagenet-trained CNNs are biased towards texture; in- creasing shape bias improves accuracy and robustness,

    R. Geirhos, P. Rubisch, C. Michaelis, M. Bethge, F. A. Wichmann, and W. Brendel, “Imagenet-trained CNNs are biased towards texture; in- creasing shape bias improves accuracy and robustness,” inInternational Conference on Learning Representations, 2019

    2019

  22. [22]

    Shortcut learning in deep neural networks,

    R. Geirhos, J.-H. Jacobsen, C. Michaelis, R. Zemel, W. Brendel, M. Bethge, and F. A. Wichmann, “Shortcut learning in deep neural networks,”Nature Machine Intelligence, vol. 2, no. 11, pp. 665–673, 2020

    2020

  23. [23]

    Color for object recognition: Hue and chroma sensitivity in the deep features of convolutional neural networks,

    A. Flachot and K. R. Gegenfurtner, “Color for object recognition: Hue and chroma sensitivity in the deep features of convolutional neural networks,”Vision Research, vol. 182, pp. 89–100, 2021

    2021

  24. [24]

    Impact of colour on robustness of deep neural networks,

    K. De and M. Pedersen, “Impact of colour on robustness of deep neural networks,” inProceedings of the IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), 2021, pp. 21–30

    2021

  25. [25]

    Assessing the importance of colours for CNNs in object recognition,

    A. Singh, A. Bay, and A. Mirabile, “Assessing the importance of colours for CNNs in object recognition,”arXiv preprint arXiv:2012.06917, 2020

    work page arXiv 2012

  26. [26]

    Beyond Corner Patches: Semantics-Aware Backdoor Attack in Federated Learning

    K. Herath, J. Zhao, and S. Bagchi, “Beyond corner patches: Semantics- aware backdoor attack in federated learning,” 2026. [Online]. Available: https://arxiv.org/abs/2603.29328

    work page internal anchor Pith review Pith/arXiv arXiv 2026

  27. [27]

    IBA: Towards irreversible backdoor attacks in federated learning,

    T. D. Nguyen, T. A. Nguyen, A. Tran, K. D. Doan, and K.-S. Wong, “IBA: Towards irreversible backdoor attacks in federated learning,” in Advances in Neural Information Processing Systems, vol. 36, 2023

    2023

  28. [28]

    Deepsight: Mitigating backdoor attacks in federated learning through deep model inspection,

    P. Rieger, T. D. Nguyen, M. Miettinen, and A.-R. Sadeghi, “Deepsight: Mitigating backdoor attacks in federated learning through deep model inspection,”arXiv preprint arXiv:2201.00763, 2022

    work page arXiv 2022

  29. [29]

    Crowdguard: Federated backdoor detection in federated learning,

    P. Rieger, T. Krauß, M. Miettinen, A. Dmitrienko, and A.-R. Sadeghi, “Crowdguard: Federated backdoor detection in federated learning,” in Network and Distributed System Security Symposium (NDSS), 2024

    2024

  30. [30]

    Freqfed: A frequency analysis-based approach for mitigating poisoning attacks in federated learning,

    H. Fereidooni, A. Pegoraro, P. Rieger, A. Dmitrienko, and A.-R. Sadeghi, “Freqfed: A frequency analysis-based approach for mitigating poisoning attacks in federated learning,” inNetwork and Distributed System Security Symposium (NDSS), 2024

    2024

  31. [31]

    Filterfl: Knowledge filtering-based data-free backdoor defense for fed- erated learning,

    Y . Yang, M. Hu, X. Xie, Y . Cao, P. Zhang, Y . Huang, and M. Chen, “Filterfl: Knowledge filtering-based data-free backdoor defense for fed- erated learning,”arXiv preprint arXiv:2308.11333, 2023

    work page arXiv 2023

  32. [32]

    Hidden trigger backdoor attacks,

    A. Saha, A. Subramanya, and H. Pirsiavash, “Hidden trigger backdoor attacks,” inProceedings of the AAAI Conference on Artificial Intelli- gence, vol. 34, no. 07, 2020, pp. 11 957–11 965

    2020

  33. [33]

    Input-aware dynamic backdoor attack,

    T. A. Nguyen and A. Tran, “Input-aware dynamic backdoor attack,” in Advances in Neural Information Processing Systems, vol. 33, 2020

    2020

  34. [34]

    WaNet – imperceptible warping-based back- door attack,

    A. Nguyen and A. Tran, “WaNet – imperceptible warping-based back- door attack,”arXiv preprint arXiv:2102.10369, 2021

    work page arXiv 2021

  35. [35]

    Reflection backdoor: A natural backdoor attack on deep neural networks,

    Y . Liu, X. Ma, J. Bailey, and F. Lu, “Reflection backdoor: A natural backdoor attack on deep neural networks,” inComputer Vision – ECCV 2020, 2020, pp. 182–199

    2020

  36. [36]

    FLARE: Defending federated learning against model poisoning attacks via latent space representations,

    N. Wang, Y . Xiao, Y . Chen, Y . Hu, W. Lou, and Y . T. Hou, “FLARE: Defending federated learning against model poisoning attacks via latent space representations,” inProceedings of the 2022 ACM on Asia Conference on Computer and Communications Security, 2022

    2022

  37. [37]

    Trojaning attack on neural networks,

    Y . Liu, S. Ma, Y . Aafer, W.-C. Lee, J. Zhai, W. Wang, and X. Zhang, “Trojaning attack on neural networks,” inNetwork and Distributed System Security Symposium (NDSS), 2018

    2018