Pith. sign in

REVIEW 3 major objections 5 minor 88 references

A malicious runtime extension can turn one poisoned sample into a reliable backdoor and steal the hidden training recipe.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 17:40 UTC pith:PLS4WDSK

load-bearing objection The middleware threat model is real and worth engaging; the single-sample backdoor claim needs more evidence than the paper's single-run, tuned-parameter experiments provide. the 3 major comments →

arxiv 2607.17550 v1 pith:PLS4WDSK submitted 2026-07-20 cs.CR

(A)iSpy: Parasitic Trojans for Machine Learning Infrastructure

Habibur Rahaman , Qipan Xu , Zafaryab Haider , Prabuddha Chakraborty , Swarup Bhunia , Fnu Suya This is my paper
classification cs.CR
keywords supply chain attackML runtimebackdoor amplificationspread-spectrum carrierhyperparameter exfiltrationmalicious middlewareobserve-and-executemodel steganography
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to show that a machine-learning pipeline's execution environment—the runtime that compiles and runs the computation graph—is a privileged, unguarded attack surface. It introduces a malicious runtime extension that registers through the same extension APIs as legitimate graph optimizers, and claims that this position lets an adversary observe live tensor states and react to them, which static data-poisoning or model-tampering attacks cannot do. The flagship result is backdoor amplification: one physically poisoned sample, carrying an invisible spread-spectrum marker, can be replayed and gradient-scaled by the runtime until the model reliably learns the trigger, taking attack success from near zero to roughly 92–98% on standard benchmarks. The companion result is lossless hyperparameter exfiltration: the victim's training recipe is embedded into released weights or into model behavior and recovered with zero bit errors across 19 configurations. If these claims hold, the de facto assumption that sub-0.1% poisoning is harmless collapses, and the training recipe itself becomes a new form of intellectual-property leakage.

Core claim

(A)iSpy registers as a legitimate graph-optimization extension in an ML runtime and runs an observe-and-execute loop: read any tensor, then write back before the optimizer or output layer consumes it. The paper claims this turns weak poisoning into a strong backdoor. A poisoned sample carries an invisible spread-spectrum carrier; the middleware detects it with a matched filter, subtracts it, replays the sample k=200 times, and scales its gradient by s=5, a combined amplification of 1,000. A single poisoned sample then lifts attack success from under 3% to roughly 92–98% on standard image benchmarks, with clean accuracy loss under 2 points. The same loop exfiltrates the training recipe lossle

What carries the argument

The load-bearing mechanism is the observe-and-execute loop: one observation primitive (read any tensor between operator boundaries) plus three action primitives (in-place tensor modification, hybrid graph compute, stateful persistence). For backdoor amplification, the key sub-mechanism is a spread-spectrum carrier detected by a matched filter, followed by sanitization, batch replay (k=200), and gradient scaling (s=5); combined amplification C = k·s ≈ 1,000 raises a 0.02% physical poisoning ratio to a 20% effective ratio. For exfiltration, the key mechanism is spread-transform dither modulation (STDM), which spreads each payload bit across 1,024 high-magnitude weights so post-training perturb

Load-bearing premise

The backdoor-amplification result depends on the replayed, gradient-scaled sample teaching the model a transferable trigger rather than being memorized as one fixed input; the paper tunes replay count and gradient scale to force this, and any dataset, architecture, or augmentation that tips the model toward memorization breaks the amplifier.

What would settle it

Re-run the single-sample replay/scaling protocol on a vision transformer or on a standard small-image benchmark with an aggressive augmentation pipeline (random erasing plus heavy augmentation). If attack success on held-out triggered inputs is near zero while clean accuracy stays high, the transferable-trigger premise fails; if the gradient-norm cap s=5 is violated, the stealth premise fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • A low poisoning ratio no longer guarantees safety: even a single attacker-controlled sample in the training corpus can become a reliable backdoor when the runtime is compromised.
  • Density-based data sanitizers and trigger-behavior scanners lose their signal at these ratios, because the physical poison cluster is gone while the model perceives a high-volume attack.
  • Open-weight releases and black-box APIs become covert channels for copying a victim's hidden training recipe, collapsing the cost of hyperparameter search from weeks of GPU time to seconds.
  • Standard defenses—file integrity checks, static graph audits, and malware signatures—miss the attack, so securing the ML stack requires provenance, reproducible builds, and runtime attestation.
  • The attack surface is structural: any extensible runtime that lets third-party code register as a graph optimizer or custom operator inherits the same exposure.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The replay-and-scale protocol is shown only on image classifiers; testing it on audio, tabular data, or learned embeddings would reveal whether amplification is a vision-specific phenomenon or a general property of gradient-based learning.
  • A defense that follows directly from the mechanism, not explored in the paper, would be to deduplicate exact samples within an epoch or randomize batch composition, which could break the replay buffer without ever detecting the carrier.
  • The codeword channel generalizes beyond hyperparameters: any runtime that can fine-tune a model could imprint arbitrary structured messages readable through API queries, making behavioral steganography a broader integrity threat.
  • The inference-time 'clean model, dirty system' injection implies that scanning weights or auditing graphs can never certify a deployment; runtime attestation or trusted execution may be the only remaining defense.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces (A)iSpy, a supply-chain Trojan that operates as a malicious graph-optimizer/execution-provider extension inside ML runtimes such as ONNX Runtime and TensorRT. Using an observe-and-execute loop over live tensor state, it claims two headline attacks: (1) backdoor amplification, where a steganographic carrier on a single poisoned sample is detected by a matched filter and then replayed and gradient-scaled to reach high attack success rates at vanishing physical poisoning ratios, and (2) lossless hyperparameter exfiltration, embedding the victim's training recipe into released weights via spread-transform dither modulation or into model behavior via black-box codeword triggers. The paper also reports auxiliary attacks (sabotage, denial-of-service, subpopulation manipulation, inference-time logit injection), an ONNX Runtime implementation, and evasion of malware scanners and backdoor defenses.

Significance. If the claims hold, the paper identifies a genuinely under-explored and important threat surface: the ML runtime middleware itself, which sits inside the trusted computing base with raw access to weights, gradients, and activations. The implementation in ONNX Runtime and TensorRT, the use of well-understood spread-spectrum techniques, the breadth of auxiliary attacks, and the empirical evaluation against scanners and defenses are all valuable. The paper also includes a useful negative result (Appendix K.1) showing that coarse performance-counter features do not detect the Trojan. The central contribution is the threat model and the demonstration that stateful, tensor-level manipulation can achieve what static data poisoning cannot. However, the headline single-sample backdoor claim and the amplifier mechanism need careful verification, as detailed below.

major comments (3)
  1. [§3.1 and Appendix M, Algorithm 2] The derivation p_eff = p0·C with C = k·s requires that the gradient scale s multiplies only the poisoned sample's gradient contribution. But Algorithm 2's OnBackwardPass applies G_out ← G_in·s whenever the batch contains a replayed sample, scaling the entire aggregated gradient. This changes only the learning rate for those steps and does not increase the poisoned sample's weight relative to clean samples in the batch; it cannot produce an 'effective poisoning ratio' of p0·k·s. If the actual C++ implementation instead scales only the gradient attributable to the replay sample, the pseudo-code and §3.1 text must be corrected. As written, the mechanism is internally inconsistent with the headline amplifier equation, and the gradient-norm cap s·||g_poison|| ≤ g_max is meaningless if the whole batch gradient is scaled.
  2. [§1 Contributions and Table 1] The abstract and contributions state that 'a single poisoned sample reaches over 97% attack success on CIFAR-10, CIFAR-100, and ImageNet.' Table 1 contradicts this: the single-sample rows show ASR 94.8 (CIFAR-100, BadNet-w), 96.1 (CIFAR-100, WaNet-w), 92.8 (ImageNet, BadNet-w), and 91.9 (ImageNet, WaNet-w). Only CIFAR-10 exceeds 97%. The 'near zero to 100%' phrasing in the abstract is also inaccurate; the maximum reported is 99.5%. The numbers should be reported per dataset and the headline claim qualified accordingly.
  3. [§3.1, §4.1.2, Appendix K.2] The single-sample backdoor result is calibrated rather than predicted. The paper states 'empirically we find s=5 satisfies the bound' and 'we find p_final = 20% sufficient', and Appendix K.2 shows that the (k,s) pair is selected as a Pareto optimum on CIFAR-10/ResNet-18. No seed variance or confidence intervals are reported for any table. The matched-filter math guarantees carrier detection, not that replaying one sanitized image k=200 times imprints a transferable visual trigger as opposed to memorizing that image. A control experiment is needed: e.g., replaying the same sanitized image without the trigger, or testing triggers on base contents far from the poisoned sample, to show that ASR comes from trigger generalization. As written, the headline 'a single poisoned sample suffices' is an empirical, tuned claim, not a derived property, and the paper should present it as such with varia
minor comments (5)
  1. [§4.2 and Table 4] The text repeatedly says 'zero bit error recovery across 19 model and dataset configurations', but Table 4 lists 16 white-box configurations (3+3+3+4+3). The count should be reconciled, or the missing configurations should be added.
  2. [§5] The text refers to 'Section 12 (Appendix)' for demo source code, but the appendix sections are lettered A through N. The cross-reference should be updated.
  3. [Table 2] The notation '1 smp = 0.02% for CIFAR-10/100, 0.01% for ImageNet' is informal and should be defined in the table caption or main text.
  4. [§4.2.1] Perplexity shifts are reported as exactly 0.00 to two decimals. Reporting standard errors or one more significant digit would strengthen the 'zero impact' claim.
  5. [Appendix K.2] The definition p0C = p_final = 20% assumes s applies only to poison contributions. This should be made explicit in the main text and aligned with the implementation description, given the issue raised in the first major comment.

Circularity Check

0 steps flagged

No significant circularity: the headline ASR is an empirically tuned result, not a derived prediction; the only by-construction relations are standard encoder/decoder and matched-filter communication primitives.

full rationale

The derivation chain in §3.1 is transparent: the amplification identity C=k·s and p_final=p0·C is a definition used to calibrate k, with p_final=20% and s=5 explicitly chosen empirically (“we find p_final=20% sufficient…”, “empirically we find s=5 satisfies the bound…”). The 97% ASR is then measured under that configuration, not derived from the equations; the paper does not label it a prediction. Appendix K.2 even admits the (k*,s*) choice is a tuned Pareto optimum and is context-dependent, so no fitted parameter is disguised as a derived forecast. The matched-filter threshold α equals the construction amplitude α because the poisoned sample is defined as X_b+αT; this is a communication-protocol design, not a circular empirical claim, and the paper validates it under augmentation (Table 16). The STDM white-box channel is also an encoder/decoder pair by construction, so BER=0 at embed time is expected; the nontrivial content (robustness to fine-tuning/pruning, ΔPPL≈0, BAIT evasion) is empirical and independent. Self-citations [60–63] appear only in Appendix K.1 and are not load-bearing; the APC negative result is the authors’ own measurement. The memorization-vs-transferability concern raised by reviewers is a generalization/correctness risk, not a circularity, because the paper never claims the imprinting follows from the matched-filter math alone.

Axiom & Free-Parameter Ledger

7 free parameters · 6 axioms · 2 invented entities

The two headline results rest on different epistemic loads. The backdoor amplifier is calibrated: s, p_final (hence k) are empirically fitted until the target ASR is reached, and the carrier's survival of augmentation is validated only for the tested transform stack. The exfiltration attack rests on the grid assumption and on STDM parameters chosen by the authors; its robustness claims (BER=0.00 under pruning/fine-tuning) are single-seed measurements. No new physical entities are postulated; the carrier and codebook are in-paper signal designs without external falsifiable evidence.

free parameters (7)
  • gradient scaling factor s = 5
    Explicitly fitted: 'empirically we find s=5 satisfies the bound throughout training across all our experimental settings, so the middleware fixes s=5 without adaptive recalibration' (§3.1 Execution).
  • target effective poisoning ratio p_final = 20%
    Chosen because it is 'sufficient across all our experimental settings for the backdoor to imprint reliably' (§3.1); the replay count k is derived from it via k = p_final/(p0·s).
  • carrier amplitude and detection threshold alpha = 0.05
    Sets carrier strength and matched-filter threshold at the same value (X_poison = X_b + alpha·T; detection at s(x) ≈ alpha); given in §4.1.1 without an ablation schedule.
  • STDM embedding parameters (Delta, G, seed) = Delta=1e-6, G=1024, seed=42
    Chosen by the authors (§4.2.1); robustness follows from G, but Delta bounds the watermark strength and is not derived from any model constraint.
  • hyperparameter grid H_k = 37 LR values, 4 weight-decay, 4 warmup, 4 batch-size values (12 payload bits)
    Assumed to be 'reliably estimated based on public knowledge or academic literature' (§3.2); the payload size and the black-box codebook are defined only relative to this grid.
  • bit-flip attack budget (k=25 sign / k=6 exponent, E_curv=3 epochs, I=10 batches) = 25 bits, 3 epochs, 10 batches
    Chosen to keep the cache at 200 bytes ('We explicitly constrain k to a small number'); not derived from any accuracy constraint (Appendix B.1).
  • sabotage attack thresholds (tau, damage budget alpha per architecture) = tau in {0.55, 0.85, 0.90}; alpha in {50, 100}
    Tuned per architecture/dataset ('The threshold is deliberately tuned so that t* falls in the late epochs'; Table 11 reports per-config values).
axioms (6)
  • domain assumption Zero-copy extension APIs grant direct mutable pointers to weights, gradients, activations, and labels during ORT training (get_contiguous_parameters, named InPlaceAccumulatorV2 outputs).
    Stated in §2.4/§5 as the interception boundary (TrainingSession::RunForwardBackward hook); it is the engineering premise for every attack. Asserted from ORT internals and demonstrated by the implementation, but no reproducer is shipped.
  • domain assumption A spread-spectrum carrier added to a poisoned input survives standard data augmentation (random crop, color jitter, normalization) and stays near-orthogonal to clean inputs under the matched filter.
    §3.1; empirically validated in Table 16 and Appendix N.2 for the tested pipeline, but load-bearing only for pipelines similar to the tested transform stack.
  • standard math Matched-filter / spread-spectrum processing gain (inner-product detection aggregates over d features; STDM projection aggregates over G=1024 weights).
    Classical signal-processing results [12,21,56] applied in §3.1 and §3.2.1; not circular, but the quantitative claims (BER=0.00, detection recall 91-96%) are empirical.
  • domain assumption The victim's hyperparameter recipe lies on the publicly knowable grid H_1 x ... x H_K and is observable by the middleware during the run.
    §3.2: 'candidate sets H_k can be reliably estimated based on public knowledge or academic literature'; the exfiltration payload and codebook are defined only relative to this grid.
  • ad hoc to paper Gradient scaling by s=5 with ||g_poison||·s <= g_max keeps the amplifier stealthy against gradient-norm anomaly detection.
    §3.1 'Gradient level stealth'; validated only on CIFAR-10 (Figure 7); the paper concedes s could need re-evaluation every N batches but fixes s=5.
  • standard math Hessian diagonal approximated by squared gradients (Gauss-Newton) accumulates stable sensitivity over the final 3 epochs.
    Appendix B.1 (Eq. 4), mirroring Optimal Brain Damage [37]; the approximation is standard, and its efficacy here is empirical.
invented entities (2)
  • Steganographic carrier T (pseudo-random zero-mean vector) no independent evidence
    purpose: Covert detection channel so the middleware can identify the coordinating attacker's poisoned samples during training, decoupled from the visual trigger.
    Introduced in §3.1. It is a signal design, not a new physical entity; its only evidence is the paper's own detection tables (Table 16) under the tested augmentation pipeline and alpha=0.05.
  • Black-box trigger prompts + codeword codebook (e.g., 'morning light' -> 'willow') no independent evidence
    purpose: Behavioral steganography channel to exfiltrate hyperparameter grid indices from API-only deployments.
    Introduced in §3.2.2; robustness rests on the claim that the exact phrasing is very rare in natural user traffic—plausible but not quantified against a real traffic corpus.

pith-pipeline@v1.3.0-alltime-deepseek · 53920 in / 25665 out tokens · 214546 ms · 2026-08-01T17:40:52.911884+00:00 · methodology

0 comments
read the original abstract

Modern machine learning (ML) pipelines depend heavily on third party libraries for graph compilation and hardware acceleration. While current practices audit data and model artifacts or rely on file integrity checks, the execution environment remains implicitly trusted. This blind spot enables active threats where a malicious runtime module interacts directly with live training and inference dynamics: exploiting this interaction allows the Trojan to support complex objectives that are challenging for static code or binary modifications, achieving manipulations impossible for standard data and model level attacks. We expose this vulnerability by presenting (A)iSpy, a parasitic infrastructure Trojan that subverts ML systems through an active observe and execute paradigm. Operating within the computation graph, (A)iSpy monitors transient tensor states to perform targeted, stealthy manipulations with negligible overhead. To violate confidentiality, the Trojan identifies all critical training hyperparameters and covertly exfiltrates them via model weights or output logits. To break integrity, it acts as a gradient amplifier: by observing steganographic triggers, it transforms otherwise weak data poisoning into effective backdoor attacks, increasing success rates from near zero to 100%. We further demonstrate broad extensibility across the machine learning lifecycle by validating auxiliary attacks in the appendix, including subpopulation label flipping, availability disruptions, and inference stage manipulations. Importantly, the (A)iSpy module easily evades standard malware scanners, while the associated poisoned inputs and resulting compromised models bypass typical inspection tools. We demonstrate the practicality of this threat with an implementation in the ONNX Runtime training and inference engines.

Figures

Figures reproduced from arXiv: 2607.17550 by Fnu Suya, Habibur Rahaman, Prabuddha Chakraborty, Qipan Xu, Swarup Bhunia, Zafaryab Haider.

Figure 1
Figure 1. Figure 1: (A)iSpy Observe-and-Execute attack flow sitting [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Black-box hyperparameter exfiltration across LLM–dataset configurations. Perplexity is normalized to the clean [PITH_FULL_IMAGE:figures/full_fig_p009_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Backdoored and clean Q-scores across models under [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Black-box hyperparameter exfiltration: compari [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: (A)iSpy vs 1P-DNL across 10 configurations. [PITH_FULL_IMAGE:figures/full_fig_p019_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Adaptive gradient-noise injection subtly degrades [PITH_FULL_IMAGE:figures/full_fig_p021_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Gradient magnitude, measured by the L2 norm, on [PITH_FULL_IMAGE:figures/full_fig_p022_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Graph-level realization of the (A)iSpy middleware threat model. Left: the victim inference path coexists with auxiliary [PITH_FULL_IMAGE:figures/full_fig_p023_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: The trigger detection function compares activa [PITH_FULL_IMAGE:figures/full_fig_p026_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Benign observation kernel records prediction con [PITH_FULL_IMAGE:figures/full_fig_p027_11.png] view at source ↗
Figure 13
Figure 13. Figure 13: Plugin enqueue method coordinates kernel execu [PITH_FULL_IMAGE:figures/full_fig_p028_13.png] view at source ↗
Figure 12
Figure 12. Figure 12: TensorRT plugin class structure embedding back [PITH_FULL_IMAGE:figures/full_fig_p028_12.png] view at source ↗
Figure 14
Figure 14. Figure 14: Effect of post-embedding LSB-zeroing on both watermark schemes across a [PITH_FULL_IMAGE:figures/full_fig_p029_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Validation perplexity vs. learning rate in (A)iSpy’s [PITH_FULL_IMAGE:figures/full_fig_p030_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: The heatmap results of the Attack Success [PITH_FULL_IMAGE:figures/full_fig_p031_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: The heatmap results of the Attack Success [PITH_FULL_IMAGE:figures/full_fig_p032_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Baseline and attacked top-1 accuracy under the [PITH_FULL_IMAGE:figures/full_fig_p032_18.png] view at source ↗
Figure 20
Figure 20. Figure 20: (A)iSpy’s Black-Box HP exfiltration Full [PITH_FULL_IMAGE:figures/full_fig_p035_20.png] view at source ↗
Figure 19
Figure 19. Figure 19: Stitched ONNX model with integrated trigger de [PITH_FULL_IMAGE:figures/full_fig_p035_19.png] view at source ↗
Figure 21
Figure 21. Figure 21: Victim-Interceptor Boundary (combined_spy.onnx). Victim terminates at ArgMax→Detokenize; attacker’s subgraph recovers all eight HP fields via parallel LinearRegressor nodes into hp_payload. and the full backward path ( [PITH_FULL_IMAGE:figures/full_fig_p035_21.png] view at source ↗
Figure 22
Figure 22. Figure 22: Standalone ONNX Interceptor (decoder_interceptor.onnx, 70 KB, 18 nodes). Text is vectorized via TfIdfVectorizer→IDF scaling, then decoded by 7×Ridge+1×Constant nodes, each recovering one hyper￾parameter field. batch × 512) enables simultaneous extraction of predictions and 512-dimensional representations at zero additional cost. The back￾bone preserves the semantic cluster geometry learned under clean tra… view at source ↗
Figure 24
Figure 24. Figure 24: Clean vs. ORT-trained attacked inference graphs [PITH_FULL_IMAGE:figures/full_fig_p036_24.png] view at source ↗
Figure 26
Figure 26. Figure 26: Partial view of the distilgpt2 ORT Training v2 [PITH_FULL_IMAGE:figures/full_fig_p037_26.png] view at source ↗
Figure 25
Figure 25. Figure 25: Annotated attacked graph: the flatten node exposes [PITH_FULL_IMAGE:figures/full_fig_p037_25.png] view at source ↗
Figure 27
Figure 27. Figure 27: Seven-node LSB decoder subgraph embedded in [PITH_FULL_IMAGE:figures/full_fig_p037_27.png] view at source ↗
Figure 28
Figure 28. Figure 28: ORT internal training graph (training_model.onnx) showing the complete forward and backward computation. Forward nodes (Conv, Relu, MaxPool, Gemm) are visible in the upper right. Backward nodes (ConvGrad, ReluGrad, MaxPoolGrad, SoftmaxCrossEntropyLossGrad) and InPlaceAccumulatorV2 gradient accumulation nodes are visible in the lower left, producing the named gradient tensors intercepted by our attack. Thi… view at source ↗
Figure 29
Figure 29. Figure 29: Deployed inference graphs before and after the ONNX training-time interception attack. Both share identical node [PITH_FULL_IMAGE:figures/full_fig_p040_29.png] view at source ↗
Figure 30
Figure 30. Figure 30: Zoomed view of the gradient accumulation region in the ORT training graph (Figure 28), show [PITH_FULL_IMAGE:figures/full_fig_p040_30.png] view at source ↗
Figure 31
Figure 31. Figure 31: ORT ONNX training graph for TinyNet generated by [PITH_FULL_IMAGE:figures/full_fig_p041_31.png] view at source ↗
Figure 32
Figure 32. Figure 32: Zoomed view of the ORT gradient-accumulation boundary. The backward nodes produce gradient tensors that [PITH_FULL_IMAGE:figures/full_fig_p042_32.png] view at source ↗
Figure 33
Figure 33. Figure 33: Netron visualization of the clean transformer encoder block (left) and the (A)iSpy-injected graph (right). The injected [PITH_FULL_IMAGE:figures/full_fig_p043_33.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

88 extracted references · 1 canonical work pages

  1. [1]

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Floren- cia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. Gpt-4 technical report.arXiv preprint arXiv:2303.08774 (2023). (A)iSpy

    Pith/arXiv arXiv 2023

  2. [2]

    Airbnb. [n. d.]. BinaryAlert Public YARA Rules. https://github.com/airbnb/ binaryalert/tree/master/rules. Accessed: 08-19-2025

    2025

  3. [3]

    Daroc Alden. 2024. Insecurity and Python pickles. https://lwn.net/Articles/ 964392/

    2024

  4. [4]

    Artem Artemev, Yuze An, Tilman Roeder, and Mark van der Wilk. 2022. Memory safe computations with XLA compiler.Advances in Neural Information Processing Systems35 (2022), 18970–18982

    2022

  5. [5]

    Eugene Bagdasaryan and Vitaly Shmatikov. 2021. Blind backdoors in deep learning models. In30th USENIX Security Symposium (USENIX Security 21). 1505– 1521

    2021

  6. [6]

    Battista Biggio, Blaine Nelson, and Pavel Laskov. 2012. Poisoning attacks against support vector machines. InProceedings of the International Conference on Ma- chine Learning (ICML)

    2012

  7. [7]

    Alex Birsan. 2021. Dependency Confusion: How I Hacked Into Apple, Microsoft and Dozens of Other Companies. Medium. https://medium.com/@alex.birsan/ dependency-confusion-4a5d60fec610

    2021

  8. [8]

    Nicholas Carlini, Matthew Jagielski, Christopher A Choquette-Choo, Daniel Paleka, Will Pearce, Hyrum Anderson, Andreas Terzis, Kurt Thomas, and Florian Tramèr. 2024. Poisoning web-scale training datasets is practical. In2024 IEEE Symposium on Security and Privacy (SP). IEEE, 407–425

    2024

  9. [9]

    Simin Chen, Jinjun Peng, Yixin He, Junfeng Yang, and Baishakhi Ray. 2026. Your Compiler is Backdooring Your Model: Understanding and Exploiting Compilation Inconsistency Vulnerabilities in Deep Learning Compilers. InIEEE Symposium on Security and Privacy (S&P)

    2026

  10. [10]

    Yanzuo Chen, Zhibo Liu, Yuanyuan Yuan, Sihang Hu, Tianxiang Li, and Shuai Wang. 2025. Compiled Models, Built-In Exploits: Uncovering Pervasive Bit-Flip Attack Surfaces in DNN Executables. InNetwork and Distributed System Security (NDSS) Symposium

    2025

  11. [11]

    Cisco Talos. [n. d.]. ClamAV®Open Source Antivirus Engine. https://www. clamav.net. Accessed: 09-01-2025

    2025

  12. [12]

    Ingemar J Cox, Joe Kilian, F Thomson Leighton, and Talal Shamoon. 1997. Secure spread spectrum watermarking for multimedia.IEEE transactions on image processing6, 12 (1997), 1673–1687

    1997

  13. [13]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv:1810.04805 [cs.CL] https://arxiv.org/abs/1810.04805

    Pith/arXiv arXiv 2019

  14. [14]

    Vijay Rao

    Vasisht Duddu and D. Vijay Rao. 2020. Quantifying (Hyper) Parameter Leakage in Machine Learning. In2020 IEEE Sixth International Conference on Multimedia Big Data (BigMM). IEEE, 239–244

    2020

  15. [15]

    Logan Engstrom, Andrew Ilyas, Benjamin Chen, Axel Feldmann, William Moses, and Aleksander Madry. 2025. Optimizing ml training with metagradient descent. arXiv preprint arXiv:2503.13751(2025)

    Pith/arXiv arXiv 2025

  16. [16]

    Ido Galil, Moshe Kimhi, and Ran El-Yaniv. 2025. No Data, No Optimization: A Lightweight Method To Disrupt Neural Networks With Sign-Flips.arXiv preprint arXiv:2502.07408(2025)

    Pith/arXiv arXiv 2025

  17. [17]

    Yue Gao, Ilia Shumailov, and Kassem Fawaz. 2025. Supply-chain attacks in machine learning frameworks.Proceedings of Machine Learning and Systems7 (2025)

    2025

  18. [18]

    Yansong Gao, Change Xu, Derui Wang, Shiping Chen, Damith C Ranasinghe, and Surya Nepal. 2019. Strip: A defence against trojan attacks on deep neural net- works. InProceedings of the 35th annual computer security applications conference. 113–125

    2019

  19. [19]

    Aaron Gokaslan and Vanya Cohen. 2019. OpenWebText Corpus. http:// Skylion007.github.io/OpenWebTextCorpus

    2019

  20. [20]

    Tianyu Gu, Brendan Dolan-Gavitt, and Siddharth Garg. 2017. Badnets: Identifying vulnerabilities in the machine learning model supply chain.arXiv preprint arXiv:1708.06733(2017)

    Pith/arXiv arXiv 2017

  21. [21]

    Frank H Hartung, Jonathan K Su, and Bernd Girod. 1999. Spread spectrum wa- termarking: Malicious attacks and counterattacks. InSecurity and Watermarking of Multimedia Contents, Vol. 3657. SPIE, 147–158

    1999

  22. [22]

    Jonathan Hayase, Weihao Kong, Raghav Somani, and Sewoong Oh. 2021. SPECTRE: Defending Against Backdoor Attacks Using Robust Statistics. arXiv:2104.11315 [cs.LG] https://arxiv.org/abs/2104.11315

    Pith/arXiv arXiv 2021

  23. [23]

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. InProceedings of the IEEE conference on computer vision and pattern recognition. 770–778

    2016

  24. [24]

    Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. 2021. Aligning AI With Shared Human Values. Proceedings of the International Conference on Learning Representations (ICLR) (2021)

    2021

  25. [25]

    Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. Measuring Massive Multitask Language Under- standing.Proceedings of the International Conference on Learning Representations (ICLR)(2021)

    2021

  26. [26]

    Sanghyun Hong, Pietro Frigo, Yiğitcan Kaya, Cristiano Giuffrida, and Tudor Dumitras,. 2019. Terminal brain damage: Exposing the graceless degradation in deep neural networks under hardware fault attacks. In28th USENIX Security Symposium (USENIX Security 19). 497–514

    2019

  27. [27]

    Mengxuan Hu, Zihan Guan, Junfeng Guo, Zhongliang Zhou, Jielu Zhang, and Sheng Li. 2024. BBCaL: Black-box Backdoor Detection under the Causality Lens. Transactions on Machine Learning Research(2024)

    2024

  28. [28]

    Binyuan Hui, Jian Yang, Zeyu Cui, Jiaxi Yang, Dayiheng Liu, Lei Zhang, Tianyu Liu, Jiajun Zhang, Bowen Yu, Keming Lu, et al. 2024. Qwen2. 5-coder technical report.arXiv preprint arXiv:2409.12186(2024)

    Pith/arXiv arXiv 2024

  29. [29]

    InQuest. [n. d.]. YARA-Rules-VT: VirusTotal-Enriched Malware Rule Collection. https://github.com/InQuest/yara-rules-vt. Accessed: 08-19-2025

    2025

  30. [30]

    Intezer. [n. d.]. Intezer YARA Rules Collection. https://github.com/intezer/yara- rules. Accessed: 08-19-2025

    2025

  31. [31]

    Matthew Jagielski, Giorgio Severi, Niklas Pousette Harger, and Alina Oprea. 2021. Subpopulation data poisoning attacks. InProceedings of the 2021 ACM SIGSAC conference on computer and communications security. 3104–3122

    2021

  32. [32]

    Fred Jelinek, Robert L Mercer, Lalit R Bahl, and James K Baker. 1977. Perplexity—a measure of the difficulty of speech recognition tasks.The journal of the Acoustical Society of America62, S1 (1977), S63–S63

    1977

  33. [33]

    JFrog Security Research. 2025. PyTorch Users at Risk: Unveiling 3 Zero- Day PickleScan Vulnerabilities. https://jfrog.com/blog/unveiling-3-zero-day- vulnerabilities-in-picklescan/

    2025

  34. [34]

    2009.Learning Multiple Layers of Features from Tiny Images

    Alex Krizhevsky. 2009.Learning Multiple Layers of Features from Tiny Images. Technical Report. University of Toronto. https://www.cs.toronto.edu/~kriz/ learning-features-2009-TR.pdf [Online; accessed 2025-11-08]

    2009

  35. [35]

    2009.Learning Multiple Layers of Features from Tiny Images (CIFAR-100 dataset)

    Alex Krizhevsky. 2009.Learning Multiple Layers of Features from Tiny Images (CIFAR-100 dataset). Technical Report. University of Toronto. https://www.cs. toronto.edu/~kriz/cifar.html [Online; accessed 2025-11-08]

    2009

  36. [36]

    Kaitchup Lab. 2024. A Guide on Hyperparameters and Training Argu- ments for Fine-tuning LLMs. https://kaitchup.substack.com/p/a-guide-on- hyperparameters-and-training

    2024

  37. [37]

    Yann LeCun, John Denker, and Sara Solla. 1989. Optimal Brain Damage. In Advances in Neural Information Processing Systems, D. Touretzky (Ed.), Vol. 2. Morgan-Kaufmann. https://proceedings.neurips.cc/paper_files/paper/1989/file/ 6c9882bbac1c7093bd25041881277658-Paper.pdf

    1989

  38. [38]

    Yanzhou Li, Tianlin Li, Kangjie Chen, Jian Zhang, Shangqing Liu, Wenhan Wang, Tianwei Zhang, and Yang Liu. 2024. Badedit: Backdooring large language models by model editing.arXiv preprint arXiv:2403.13355(2024)

    Pith/arXiv arXiv 2024

  39. [39]

    Yue Li, Benedetta Tondi, and Mauro Barni. 2021. Spread-transform dither mod- ulation watermarking of deep neural network.Journal of Information Security and Applications63 (2021), 103004

    2021

  40. [40]

    Kang Liu, Brendan Dolan-Gavitt, and Siddharth Garg. 2018. Fine-pruning: De- fending against backdooring attacks on deep neural networks. InInternational symposium on research in attacks, intrusions, and defenses. Springer, 273–294

    2018

  41. [41]

    Malcat. [n. d.]. Malcat: Static and Heuristic Malware Analysis Tool. https: //malcat.fr. Accessed: 09-05-202

  42. [42]

    Mandiant FLARE Team. [n. d.]. capa: The FLARE Team’s Open Source Capability Detector. https://github.com/mandiant/capa. Accessed: 09-01-202

  43. [43]

    Callum McMahon. 2026. Supply Chain Attack in litellm 1.82.8 on PyPI. https: //futuresearch.ai/blog/litellm-pypi-supply-chain-attack/. Accessed: 2026-03-26

    2026

  44. [44]

    Stephen Merity, Caiming Xiong, James Bradbury, and Richard Socher. 2016. Pointer Sentinel Mixture Models. arXiv:1609.07843 [cs.CL]

    Pith/arXiv arXiv 2016

  45. [45]

    Microsoft. 2021. ONNX Runtime: Cross-Platform, High Performance ML In- ferencing and Training Accelerator. InGitHub Repository. https://github.com/ microsoft/onnxruntime

    2021

  46. [46]

    Xiaoxing Mo, Yechao Zhang, Leo Yu Zhang, Wei Luo, Nan Sun, Shengshan Hu, Shang Gao, and Yang Xiang. 2023. Robust Backdoor Detection for Deep Learning via Topological Evolution Dynamics. arXiv:2312.02673 [cs.CR] https: //arxiv.org/abs/2312.02673

    Pith/arXiv arXiv 2023

  47. [47]

    Neo23x0. [n. d.]. signature-base: YARA, Sigma, and IOC Rules Collection. https: //github.com/Neo23x0/signature-base. Accessed: 08-19-2025

    2025

  48. [48]

    Zachary Newman, John Speed Meyers, and Santiago Torres-Arias. 2022. Sig- store: Software signing for everybody. InProceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security. 2353–2367

    2022

  49. [49]

    Tuan Anh Nguyen and Anh Tuan Tran. 2021. WaNet - Imperceptible Warping- based Backdoor Attack. InInternational Conference on Learning Representations. https://openreview.net/forum?id=eEn8KTtJOx

    2021

  50. [50]

    NVIDIA Corporation. 2024. TensorRT Deep Learning Inference Optimizer and Runtime. https://developer.nvidia.com/tensorrt

    2024

  51. [51]

    Marc Ohm, Henrik Plate, Arnold Sykosch, and Michael Meier. 2020. Backstabber’s knife collection: A review of open source software supply chain attacks. In International Conference on Detection of Intrusions and Malware, and Vulnerability Assessment. Springer, 23–43

    2020

  52. [52]

    Open Source Security Foundation (OpenSSF). 2024. xz Backdoor CVE-2024-3094. https://openssf.org/blog/2024/03/30/xz-backdoor-cve-2024-3094/. Accessed: 2025-11-09

    2024

  53. [53]

    Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al. 2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In Advances in Neural Information Processing Systems (NeurIPS), Vol. 32. Habibur Rahaman, Qipan Xu, Zafaryab Haider, Prabuddha C...

    2019

  54. [54]

    Thomas (Neo23x0) Patzke. [n. d.]. LOKI: Simple IOC and YARA Scanner. https: //github.com/Neo23x0/Loki. Accessed: 09-08-202

  55. [55]

    Shovon Paul and Xiali Hei. 2025. Stealing the Recipe: Hyperparameter Stealing Attacks on Fine-Tuned LLMs. OpenReview. ICLR 2026 submission

    2025

  56. [56]

    Raymond Pickholtz, Donald Schilling, and Laurence Milstein. 2003. Theory of spread-spectrum communications-a tutorial.IEEE transactions on Communica- tions30, 5 (2003), 855–884

    2003

  57. [57]

    PortSwigger. [n. d.]. Burp Suite YARA Rule Set. https://github.com/PortSwigger/ yara. Accessed: 08-19-2025

    2025

  58. [58]

    CAPE Sandbox Project. [n. d.]. CAPE Sandbox YARA Rules. https://github.com/ kevoreilly/CAPEv2/tree/master/data/yara. Accessed: 08-19-2025

    2025

  59. [59]

    Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog1, 8 (2019), 9

    2019

  60. [60]

    Habibur Rahaman, Atri Chatterjee, and Swarup Bhunia. 2024. SAMURAI: A Framework for Safeguarding Against Malicious Usage and Resilience of AI. In 2024 IEEE 33rd Asian Test Symposium (ATS). IEEE, 1–6

    2024

  61. [61]

    Habibur Rahaman, Atri Chatterjee, and Swarup Bhunia. 2024. Secure ai sys- tems: Emerging threats and defense mechanisms. In2024 IEEE 33rd Asian Test Symposium (ATS). IEEE, 1–6

    2024

  62. [62]

    Habibur Rahaman, Atri Chatterjee, and Swarup Bhunia. 2026. SAMURAI: Run- time Attack Detection in AI Accelerators Using AI Performance Counters.IEEE Transactions on Circuits and Systems for Artificial Intelligence(2026)

    2026

  63. [63]

    Habibur Rahaman, Sudipta Paria, Atri Chatterjee, and Swarup Bhunia. 2026. Evolving Landscape of Attacks on AI Hardware and Robust Defenses. In2026 27th International Symposium on Quality Electronic Design (ISQED). IEEE, 1–6

    2026

  64. [64]

    Adnan Siraj Rakin, Zhezhi He, and Deliang Fan. 2020. Bit-flip attack: Crushing neural network with progressive bit search. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    2020

  65. [65]

    Adnan Siraj Rakin, Zhezhi He, Jingtao Li, Fan Yao, Chaitali Chakrabarti, and Deliang Fan. 2021. T-bfa: Targeted bit-flip adversarial weight attack.IEEE Transactions on Pattern Analysis and Machine Intelligence44, 11 (2021), 7928– 7939

    2021

  66. [66]

    ReversingLabs. [n. d.]. ReversingLabs YARA Rules: Ransomware, Backdoor, Infos- tealer, Trojan, Virus Signatures. https://github.com/reversinglabs/reversinglabs- yara-rules. Accessed: 08-19-2025

    2025

  67. [67]

    Lutz Roeder. 2022. Netron: Visualizer for neural network, deep learning, and machine learning models. doi:10.5281/zenodo.5854961 Accessed: 2025-08-19

    work page doi:10.5281/zenodo.5854961 2022

  68. [68]

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al

  69. [69]

    Amit Sabne. 2020. Xla: Compiling machine learning for peak performance

    2020

  70. [70]

    Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Dis- tilBERT, a distilled version of BERT: smaller, faster, cheaper and lighter.arXiv preprint arXiv:1910.01108(2019)

    Pith/arXiv arXiv 2019

  71. [71]

    Andrei Semenov et al . 2025. Benchmarking Optimizers for Large Language Model Pretraining.arXiv preprint arXiv:2509.01440(2025)

    Pith/arXiv arXiv 2025

  72. [72]

    Ali Shafahi, W Ronny Huang, Mahyar Najibi, Octavian Suciu, Christoph Studer, Tudor Dumitras, and Tom Goldstein. 2018. Poison frogs! targeted clean-label poisoning attacks on neural networks.Advances in neural information processing systems31 (2018)

    2018

  73. [73]

    Guangyu Shen, Siyuan Cheng, Zhuo Zhang, Guanhong Tao, Kaiyuan Zhang, Hanxi Guo, Lu Yan, Xiaolong Jin, Shengwei An, Shiqing Ma, et al. 2025. Bait: Large language model backdoor scanning by inverting attack target. In2025 IEEE Symposium on Security and Privacy (SP). IEEE, 1676–1694

    2025

  74. [74]

    Karen Simonyan and Andrew Zisserman. 2014. Very deep convolutional net- works for large-scale image recognition.arXiv preprint arXiv:1409.1556(2014)

    Pith/arXiv arXiv 2014

  75. [75]

    SLSA SLSA. 2024. Supply-chain Levels for Software Artifacts

    2024

  76. [76]

    Fnu Suya, Saeed Mahloujifar, Anshuman Suri, David Evans, and Yuan Tian. 2021. Model-targeted poisoning attacks with provable convergence. InInternational Conference on Machine Learning. PMLR, 10000–10010

    2021

  77. [77]

    ThreatLabz. 2025. Malicious PyPI Packages Deliver SilentSync RAT. https://www.zscaler.com/blogs/security-research/malicious-pypi-packages- deliver-silentsync-rat Zscaler Security Research

    2025

  78. [78]

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971(2023)

    Pith/arXiv arXiv 2023

  79. [79]

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yas- mine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhos- ale, et al. 2023. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288(2023)

    Pith/arXiv arXiv 2023

  80. [80]

    Binghui Wang and Neil Zhenqiang Gong. 2018. Stealing Hyperparameters in Machine Learning. In2018 IEEE Symposium on Security and Privacy (SP). IEEE, 36–52

    2018

Showing first 80 references.