Pith. sign in

REVIEW 3 major objections 4 minor 41 references

In multi-task inference, the task command itself — known before inference and stable across hundreds of frames — can dictate which output-channel tiles the backbone skips, cutting FPGA latency by 51–59% because the mask is encoded as bitmas

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 06:00 UTC pith:X33QXKFN

load-bearing objection Solid full-stack hardware paper with a reproducible FPGA artifact; the task-conditional tile-skipping idea is new and the measured numbers hold up, but the driving-quality generalization claims are softer than the headline and the static-pruning counterfactual isn't fully controlled. the 3 major comments →

arxiv 2607.22038 v1 pith:X33QXKFN submitted 2026-07-24 cs.AR cs.AI

Sparse by Command: Task-Conditional Compute Skipping for Multi-Task Inference Accelerators

Afzal Ahmad , Gaoyu Mao , Shoubo Hu , Hui-Ling Zhen , Mingxuan Yuan , Xinyu Chen , Wei Zhang This is my paper
classification cs.AR cs.AI
keywords task-conditional sparsityhardware-software co-designtile-level compute skippinginstruction set architecture bitmaskmulti-task inference acceleratorFPGA inferenceconditional imitation learningvisuomotor driving
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper argues that the task command is a free signal for structured sparsity in multi-task models. A tiny gating network maps the command to a binary mask over output-channel tiles aligned with the accelerator's scheduling granularity, and the mask is encoded as bitmask fields in each layer's instruction. Masked tiles skip weight fetch, activation load, and compute together, reducing FLOPs by 66–76% and FPGA latency from 9.12 ms to 3.74–4.44 ms. On a GPU the same masks are 22% slower, so the claim is that the speedup is architectural: the hardware must drop fetch and compute as one unit. If true, any discrete-command multi-task system can get per-task compute reduction without changing the model architecture or inference pipeline.

Core claim

The central claim is that per-task tile masks can be derived from the command alone and executed at zero runtime cost. A gating MLP of fewer than 13k parameters takes the one-hot command and outputs 182 binary bits, one per prunable output-channel tile across eight conv layers. The host precomputes a 128-byte mask per command and writes it into each layer's 64-byte instruction; the tile manager advances a shift register through the bitmask, skipping masked tiles in one cycle. Skipping propagates across layers, because a masked output tile in layer L also removes the corresponding input-side work in layer L+1. On six CARLA driving commands, FLOPs drop 66–76%, on-device latency drops 2.1–2.4x,

What carries the argument

The load-bearing object is the hardware-aligned tile mask: each tile is a fixed group of OC_PAR=16 output channels, the atomic scheduling unit of the accelerator. A bitmask field in each 64-byte NISA instruction tells the tile manager, via a one-cycle shift-register check, whether to execute or skip a tile; skipping removes weight fetch, activation load, and compute, and the instruction scheduler propagates a layer's output mask as the next layer's input mask. The masks come from a three-phase training pipeline: dense pretraining, soft-mask finetuning with a sparsity penalty, then hard binarization with a straight-through estimator, so the gater learns which of the 182 prunable tiles each co

Load-bearing premise

The load-bearing premise is that a mask fixed per six-command label, independent of the input image and assumed stable across routes and conditions, always keeps the tiles the active task needs — and the paper's own evidence shows the edge of that assumption: the sparse GPU model terminates early on Hard 1 at 87.6% completion, the deployed FPGA's CTE degrades to 2.94 m there, and the artifact appendix disclaims closed-loop reproducibility.

What would settle it

Run the six precomputed masks on an unseen CARLA town or on a route where the same command demands features its mask prunes — for instance a long curve that arrives without a turn command. If route completion collapses for one command while a union-mask model succeeds, the fixed per-command table is the cause; a second check is to repeat the sparse GPU model's Hard 1 run several times to see whether the 87.6% early termination is a reproducible mask failure rather than single-run numerical noise.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Skipping compounds across layers: a masked output tile in layer L removes both output-side work in L and input-side work in L+1, which is why FLOP savings can exceed tile-count savings.
  • The speedup requires co-design: on a GPU the same mask adds overhead, so the result is not a model-level trick but an instruction-level and datapath-level capability.
  • Task-conditional masks expose a static/conditional split: 52.2% of tiles are never active for any command, while about 68 tiles differentiate tasks, and the paper shows the two can be removed and re-learned separately and additively.
  • The mechanism transfers to transformer MLP layers: with tiles placed on MLP hidden channels, masking removes work from both fc1 and fc2, giving 40–62% FLOP reduction on a ViT-Base backbone without architectural changes.
  • Because masks are precomputed per command and amortized over hundreds of frames, a command change costs only a host-to-HBM instruction rewrite of about 13.4 microseconds — under a percent of one inference.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The fixed-mask design sits near a quality cliff: the paper's own sparse GPU model terminates early on Hard 1 (87.6% completion) and the deployed INT8 version's cross-track error degrades to 2.94 m there, so a natural extension is a fallback that re-enables the union mask when a route or condition looks unfamiliar — the bitmask encoding makes that a one-instruction change, not a retraining.
  • The 95 never-active tiles suggest a cheaper first deployment: statically remove them, then apply task-conditional masks to the remaining 95 tiles; the paper shows this split is lossless and separable, but leaves the full system-level savings of that hybrid unexplored.
  • An unbounded command space (language-conditioned control, grasp-type commands in manipulation) would run the gating MLP once per command change rather than looking it up in a table; the hardware cost would stay identical because the mask is still precomputed before inference.
  • Since stall cycles fall only 24–26% while compute cycles fall proportionally to active tiles, further latency gains are likely available from hiding per-layer instruction fetch and DMA setup — the paper's counter instrumentation makes this directly measurable.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes a hardware/software co-design for multi-task inference accelerators: a lightweight gating MLP, trained in three phases with a sparsity penalty, produces per-tile binary execution masks from a one-hot task command; these masks are encoded into a 64-byte NISA instruction stream and the accelerator's tile manager skips masked output tiles at one cycle per tile, also propagating skips to the next layer's input tiles. The authors implement the accelerator on an Alveo U50 FPGA, deploy an 8-layer CNN visuomotor controller in CARLA, and report 66–76% FLOP reduction, 51–59% latency reduction (9.12 ms to 3.74–4.44 ms), and 51–59% energy reduction (263 mJ to 108–128 mJ) while claiming maintained driving quality. They also compare against static pruning and a GPU baseline and extend the mechanism to a ViT-Base backbone. An artifact is provided reproducing Tables 2 and 3.

Significance. If the results hold, the task-command-conditioned tile skipping is a clean and practical approach to dynamic structured sparsity: unlike input-conditional gating, it has zero per-inference overhead, and unlike static pruning it adapts the execution mask to the active task. The measured FPGA numbers are internally consistent with the cycle/FLOP reduction, the RTL is open-sourced, and the artifact reproduces the on-device tables—these are real strengths. However, the driving-quality and generalization claims rest on a narrow evaluation (three routes in one town) and the paper's own results show a route-completion failure for the sparse GPU model. The static-pruning comparison is also confounded by unequal fine-tuning budgets. The hardware contribution is credible; the software-side claims need strengthening.

major comments (3)
  1. [§4.7, Fig. 9, E2] The abstract and conclusion claim that task-conditional sparsity "maintains driving quality" and that the task-sparse model maintains 100% route completion, but the paper's own evaluation contradicts this: the task-sparse GPU model terminates early on Hard 1 at 87.6% completion under static conditions (Fig. 9) and completes only 61.7–65.0% of Hard 1 under dynamic traffic (E2). The dismissal of the Hard 1 failure as a "localised FP32-versus-INT8 numerical difference" that the INT8 FPGA "incidentally avoids" is not supported by any failure-location analysis, repeated FPGA runs, or sensitivity study of the mask set. Since the central claim is that fixed, command-only masks preserve driving quality across routes and conditions, this is load-bearing. Please provide a more robust evaluation (multiple seeds, more towns/routes, failure analysis) or substantially qualify the claim.
  2. [§4.3, Fig. 8, E3] The static-pruning counterfactual is not matched in training budget. Static-pruned networks receive one-shot L1 ranking and 20 epochs of fine-tuning, while the task-conditional model receives Phase 2 (50 epochs) plus Phase 3 (50 epochs). The claimed superiority of conditional over static pruning could therefore be largely a training-budget effect. Please match fine-tuning epochs and/or report static pruning with the same 100-epoch budget; otherwise the conclusion that the benefit "lives entirely in the surviving tiles" and is due to conditional structure is not isolated.
  3. [§4.1, §4.7] The generality of the command-only masking assumption is underevidenced. All closed-loop results come from three hand-picked hard routes in a single CARLA town (Town10HD_Opt). A fixed per-command mask is assumed stable across all conditions and routes the command can encounter, but the Hard 1 static failure and the large dynamic-traffic completion variance suggest this assumption is fragile. The paper should either provide transfer results across towns, weather, and route families, or explicitly scope the claim that command-only masks are sufficient to the evaluated conditions.
minor comments (4)
  1. [Abstract; §4.2; §4.7] The text states "100% route completion" in multiple places, but §4.7 and Fig. 9 report 87.6% completion on Hard 1 for the sparse GPU model and 61.7–65.0% under dynamic traffic. Please reconcile the wording so the claims match the reported results.
  2. [Table 2] The Compute and Stall columns do not sum to the Total Cyc. column (e.g., dense: 1.65 + 0.77 = 2.42 vs. 2.62; Follow Lane: 0.54 + 0.58 = 1.12 vs. 1.19). Please define the residual component or adjust the columns to be mutually exclusive and exhaustive.
  3. [Fig. 5] The pseudocode shows "input masked from prev layer" at line 12, but the derivation of M_ic from the previous layer's M_oc is only described in the caption/text. Make the propagation explicit in the pseudocode or caption for clarity.
  4. [§3.2, Fig. 5 caption] Typo: "a psuedocode" should be "a pseudocode". Also, in §4.2 "Brake, which relies primarily on proximity detection" — the relationship between braking and spatial processing is asserted without evidence; consider softening.

Circularity Check

0 steps flagged

No significant circularity: the sparsity masks are learned artifacts whose downstream FLOP/latency/energy numbers are measured consequences, not quantities derived from fitted inputs.

full rationale

The paper's derivation chain is self-contained. The task-conditional tile masks are produced by a gating MLP trained with a three-phase procedure (Sections 3.1, 4.1), and the reported FLOP reductions, FPGA latency, and energy figures are direct measurements of executing those learned masks on the accelerator, not quantities algebraically forced by an equation or fitted parameter. The sparsity operating point (~72% pruned) is selected by tuning λ=0.002, which is an explicit experimental choice rather than a prediction, and the paper does not present it as a derived first-principles result. No load-bearing self-citations appear: references such as the branched driving architecture [8], the straight-through estimator [2], and CARLA [10] are external and standard. The static-pruning comparison does involve unequal fine-tuning budgets (20 vs. 100 epochs) and the closed-loop quality evidence is limited (three routes, one town, with Hard 1 termination), but those are empirical-validity and generalization concerns, not circular-reasoning concerns. The artifact appendix even narrows reproducibility claims to the deterministic on-device measurements, which further supports that the central numbers are measured rather than definitional. No equation in the paper reduces a claimed prediction to its own input, and no fitted value is renamed as a prediction. The correct finding is no significant circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 6 axioms · 1 invented entities

The central claims rest on the training recipe (lambda, loss weights, dataset curation), the CARLA evaluation protocol, and the assumption that per-command masks are stable and sufficient. No new physical entities are postulated; the always-on tile-0 core is a design postulate without ablation. The listed free parameters determine the reported operating point.

free parameters (4)
  • sparsity penalty lambda (Phase 2/3) = 0.002, ramped linearly from 0
    Controls the operating point: lambda=0.002 yields ~72% of tiles masked. Reported at a single value with no sensitivity sweep, so the headline FLOP/latency/quality point is the tuned peak of this hyperparameter.
  • mask binarization threshold = 0.5
    Phase 3 hard-mask threshold sigma(g_i)>0.5; fixed without ablation.
  • steering loss weighting (25x with quadratic factor 1+10|s|^2; brake/throttle 5x) = 25x; (1+10|s|^2); 5x
    Stated as essential: without the quadratic factor the model 'predicting near-zero steering uniformly and fails to execute turns.' The entire driving-quality evaluation depends on these hand-chosen weights.
  • dataset curation: 90% straight-frame drop and DAgger perturbations = 90% dropped; +/-0.20 steering for 15-25 frames
    Data-balancing choices that shape the command distribution and hence which tiles become 'never active' in the learned masks.
axioms (6)
  • domain assumption CARLA Town10HD_Opt closed-loop driving quality is a valid proxy for the approach's quality claim
    All driving-quality claims come from one simulator town with three hand-picked routes; no real-world, cross-town, or cross-weather transfer test.
  • domain assumption Static CARLA evaluation is deterministic, making a single run per configuration sufficient
    Stated in §4.1 E1; the dynamic-traffic runs later show 61.7-65.0% run-to-run completion variance on Hard 1, so the determinism premise is doing work for the '100% completion' claim.
  • domain assumption INT8 per-layer max-abs calibration over 300 frames preserves behavior (max deviation 0.0129)
    Quantization error is verified against GPU float32 on calibration frames only; the Hard 1 rescue-by-quantization shows the operating point sits near a sensitivity boundary.
  • standard math Straight-through estimator gradient approximation for binary masks is adequate
    Standard practice (Bengio et al. 2013, cited as [2]); not formalized.
  • domain assumption A fixed per-command mask remains valid across the command's lifetime and across routes (no per-input adaptation)
    Core design premise: masks are precomputed per command and reused for hundreds of frames; evaluated only within Town10HD_Opt.
  • domain assumption Activation features group into 16-channel tiles that can be dropped without cascading accuracy loss
    Tile is the atomic skipping unit; the always-on tile-0 per layer is a design postulate with no ablation of its necessity.
invented entities (1)
  • Always-on tile 0 per layer ('polysemantic core') no independent evidence
    purpose: Preserve cross-task shared representations; tile 0 of each of the 8 layers is never masked
    Design postulate without ablation — no experiment shows what happens if tile 0 is maskable or if a different fixed core is used.

pith-pipeline@v1.3.0-alltime-deepseek · 23934 in / 28352 out tokens · 288946 ms · 2026-08-01T06:00:37.266958+00:00 · methodology

0 comments
read the original abstract

Multi-task inference models share a single backbone across diverse tasks, yet execute identical computation regardless of which task is active - wasting energy and cycles on task-irrelevant operations. We observe that the task command, typically available before inference begins, provides a free signal that can be exploited to skip unnecessary computation at the hardware level. We present a HW/SW co-designed approach in which a lightweight gating network, trained jointly with the backbone, predicts per-tile binary execution masks conditioned on the task input. Each tile corresponds to a fixed group of output channels (the native scheduling granularity of the accelerator), enabling masked tiles to be skipped with zero overhead. This yields a task-dependent reduction in compute, where each command activates only the subset of the network it requires, without changes to the model architecture or inference pipeline. We co-design the full system stack: a command-conditioned training procedure that learns hardware-aligned tile masks under a sparsity objective; an instruction set architecture whose instructions carry per-tile bitmask fields, allowing the hardware to skip masked tiles without software intervention; and a tiled inference accelerator with configurable parallelism, double-buffered memory, and INT8 datapath that natively supports sparse tile execution. We prototype on an AMD/Xilinx Alveo U50 FPGA and evaluate on a closed-loop visuomotor driving task in CARLA autonomous driving simulator. Task-conditional sparsity reduces FLOPs by 66-76% while maintaining driving quality. On-device latency decreases by 51-59%, from 9.12 ms to 3.74-4.44 ms (2.1-2.4x speedup), with energy per inference dropping from 263 to 108-128mJ.

Figures

Figures reproduced from arXiv: 2607.22038 by Afzal Ahmad, Gaoyu Mao, Hui-Ling Zhen, Mingxuan Yuan, Shoubo Hu, Wei Zhang, Xinyu Chen.

Figure 1
Figure 1. Figure 1: Task-conditional tile sparsity. (a) Per-task tile ac [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Visiomotor Controller: a shared convolutional back [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Figure 4: Accelerator architecture. The instruction scheduler fetches 64-byte instructions (including tile masks) from HBM. [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Tile manager execution loops. Teal highlights the sparsity mechanism. 𝑀𝑜𝑐 is from the NISA bitmask field; 𝑀𝑖𝑐 is set to the previous layer’s 𝑀𝑜𝑐 by the instruction sched￾uler, propagating sparsity across layers. Masked tiles cost one cycle (shift register advance); this two-level skipping compounds across adjacent layers. all bits are set. For task-conditional sparse execution, the host eval￾uates the gati… view at source ↗
Figure 6
Figure 6. Figure 6: Street-level view of an intersection in CARLA [PITH_FULL_IMAGE:figures/full_fig_p007_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Bird’s-eye view of the three hard evaluation routes [PITH_FULL_IMAGE:figures/full_fig_p008_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Static pruning vs. task-conditional sparsity. Route [PITH_FULL_IMAGE:figures/full_fig_p009_8.png] view at source ↗
Figure 10
Figure 10. Figure 10: Learned per-command tile activation on the ViT [PITH_FULL_IMAGE:figures/full_fig_p010_10.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

41 extracted references · 1 canonical work pages

  1. [1]

    AMD Xilinx. 2022. Vitis AI DPU for Alveo Product Guide (PG338). https: //docs.xilinx.com/r/en-US/pg338-dpu

    2022

  2. [2]

    Yoshua Bengio, Nicholas Léonard, and Aaron Courville. 2013. Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation. arXiv preprint arXiv:1308.3432(2013)

    Pith/arXiv arXiv 2013

  3. [3]

    Preusser, Nicholas J

    Michaela Blott, Thomas B. Preusser, Nicholas J. Fraser, Giulio Gambardella, Ken- neth O’Brien, Yaman Umuroglu, Miriam Leeser, and Kees Vissers. 2018. FINN-R: An End-to-End Deep-Learning Framework for Fast Exploration of Quantized Neural Networks.ACM Transactions on Reconfigurable Technology and Systems (TRETS)11, 3 (2018), 1–23

    2018

  4. [4]

    Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, Xin Zhang, Jake Zhao, and Karol Zieba

    Mariusz Bojarski, Davide Del Testa, Daniel Dworakowski, Bernhard Firner, Beat Flepp, Prasoon Goyal, Lawrence D. Jackel, Mathew Monfort, Urs Muller, Jiakai Zhang, Xin Zhang, Jake Zhao, and Karol Zieba. 2016. End to End Learning for Self-Driving Cars.arXiv preprint arXiv:1604.07316(2016)

    Pith/arXiv arXiv 2016

  5. [5]

    Rich Caruana. 1997. Multitask learning.Machine learning28, 1 (1997), 41–75

    1997

  6. [6]

    Yu-Hsin Chen, Joel Emer, and Vivienne Sze. 2016. Eyeriss: A Spatial Architecture for Energy-Efficient Dataflow for Convolutional Neural Networks. InProceedings of the 43rd Annual International Symposium on Computer Architecture (ISCA). IEEE, 367–379. https://doi.org/10.1109/ISCA.2016.40

    doi:10.1109/isca.2016.40 2016

  7. [7]

    Yu-Hsin Chen, Tien-Ju Yang, Joel Emer, and Vivienne Sze. 2019. Eyeriss v2: A Flexible Accelerator for Emerging Deep Neural Networks on Mobile Devices. IEEE Journal on Emerging and Selected Topics in Circuits and Systems9, 2 (2019), 292–308

    2019

  8. [8]

    Felipe Codevilla, Matthias Müller, Antonio López, Vladlen Koltun, and Alexey Dosovitskiy. 2018. End-to-End Driving via Conditional Imitation Learning. In Proceedings of the IEEE International Conference on Robotics and Automation (ICRA). IEEE, 4693–4700

    2018

  9. [9]

    López, and Adrien Gaidon

    Felipe Codevilla, Eder Santana, Antonio M. López, and Adrien Gaidon. 2019. Exploring the Limitations of Behavior Cloning for Autonomous Driving. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). 9329–9338

    2019

  10. [10]

    Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio López, and Vladlen Koltun. 2017. CARLA: An Open Urban Driving Simulator. InProceedings of the 1st Conference on Robot Learning (CoRL). 1–16

    2017

  11. [11]

    Mario Doumet, Marius Stan, Mathew Hall, and Vaughn Betz. 2024. H2PIPE: High throughput CNN Inference on FPGAs with High-Bandwidth Memory. arXiv:2408.09209 [cs.AR] https://arxiv.org/abs/2408.09209

    Pith/arXiv arXiv 2024

  12. [12]

    William Fedus, Barret Zoph, and Noam Shazeer. 2022. Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity.Journal of Machine Learning Research23, 120 (2022), 1–39

    2022

  13. [13]

    Xitong Gao, Yiren Zhao, Łukasz Dudziak, Robert Mullins, and Cheng zhong Xu. 2019. Dynamic Channel Pruning: Feature Boosting and Suppression. arXiv:1810.05331 [cs.CV] https://arxiv.org/abs/1810.05331

    Pith/arXiv arXiv 2019

  14. [14]

    Hasan Genc, Seah Kim, Alon Amid, Ameer Haj-Ali, Vighnesh Iber, Aditya Prakash, Jerry Enber, Daniel Mao, Colin Keller, Shahar Kvatinsky, Vivienne Sze, Borivoje Nikolic, Yakun Sophia Shao, and Krste Asanovic. 2021. Gemmini: Enabling Systematic Deep-Learning Architecture Evaluation via Full-Stack Integration. InProceedings of the 58th Annual Design Automatio...

    2021

  15. [15]

    Ashish Gondimalla, Noah Chesnut, Mithuna Thottethodi, and T. N. Vijaykumar

  16. [16]

    Mathew Hall and Vaughn Betz. 2020. HPIPE: Heterogeneous Layer-Pipelined and Sparse-Aware CNN Inference for FPGAs. InProceedings of the 2020 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays(Seaside, CA, USA) (FPGA ’20). Association for Computing Machinery, New York, NY, USA, 320. https://doi.org/10.1145/3373087.3375380

    arXiv 2020

  17. [17]

    Song Han, Huizi Mao, and William J. Dally. 2016. Deep Compression: Compress- ing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding. arXiv:1510.00149 [cs.CV] https://arxiv.org/abs/1510.00149

    Pith/arXiv arXiv 2016

  18. [18]

    Song Han, Jeff Pool, John Tran, and William J. Dally. 2015. Learning both Weights and Connections for Efficient Neural Networks. InAdvances in Neural Information Processing Systems (NeurIPS). 1135–1143

    2015

  19. [19]

    Benoit Jacob, Skirmantas Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew Howard, Hartwig Adam, and Dmitry Kalenichenko. 2018. Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 2704–2713

    2018

  20. [20]

    Hyungjun Jang, Dongho Ha, Hyunwuk Lee, and Won Woo Ro. 2025. DeSpa: Heterogeneous multi-core accelerators for energy-efficient dense and sparse computation at the tile level in Deep Neural Networks.Journal of Systems Architecture(2025), 103650

    2025

  21. [21]

    Chao Jiang, David Ojika, Bhavesh Patel, and Herman Lam. 2021. Optimized FPGA- based Deep Learning Accelerator for Sparse CNN using High Bandwidth Memory. In2021 IEEE 29th Annual International Symposium on Field-Programmable Custom Computing Machines (FCCM). 157–164. https://doi.org/10.1109/FCCM51124.2021. 00026

    arXiv 2021

  22. [22]

    Hao Li, Asit Kadav, Igor Durdanovic, Hanan Samet, and Hans Peter Graf. 2017. Pruning Filters for Efficient ConvNets. InProceedings of the International Confer- ence on Learning Representations (ICLR)

    2017

  23. [23]

    Paul Michel, Omer Levy, and Graham Neubig. 2019. Are sixteen heads really better than one?Advances in neural information processing systems32 (2019)

    2019

  24. [24]

    Asit Mishra, Jorge Albericio Latorre, Jeff Pool, Darko Stosic, Dusan Stosic, Ganesh Venkatesh, Chong Yu, and Paulius Micikevicius. 2021. Accelerating Sparse Deep Neural Networks. (2021). arXiv:2104.08378 [cs.LG] https://arxiv.org/abs/2104. 08378

    Pith/arXiv arXiv 2021

  25. [25]

    NVIDIA. 2017. NVDLA: The NVIDIA Deep Learning Accelerator. Hot Chips 29

    2017

  26. [26]

    Keckler, and William J

    Angshuman Parashar, Minsoo Rhu, Anurag Mukkara, Antonio Puglielli, Rang- harajan Venkatesan, Brucek Khailany, Joel Emer, Stephen W. Keckler, and William J. Dally. 2017. SCNN: An Accelerator for Compressed-sparse Con- volutional Neural Networks. InProceedings of the 44th Annual International Symposium on Computer Architecture (ISCA). ACM, 27–40

    2017

  27. [27]

    Gordon, and J

    Stéphane Ross, Geoffrey J. Gordon, and J. Andrew Bagnell. 2011. A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning. InProceedings of the 14th International Conference on Artificial Intelligence and Statistics (AISTATS). 627–635

    2011

  28. [28]

    Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. 2017. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer.arXiv preprint arXiv:1701.06538(2017)

    Pith/arXiv arXiv 2017

  29. [29]

    Vivienne Sze, Yu-Hsin Chen, Tien-Ju Yang, and Joel S. Emer. 2017. Efficient Processing of Deep Neural Networks: A Tutorial and Survey.Proc. IEEE105, 12 (2017), 2295–2329. https://doi.org/10.1109/JPROC.2017.2761740

    arXiv 2017

  30. [30]

    Surat Teerapittayanon, Bradley McDanel, and H. T. Kung. 2016. BranchyNet: Fast Inference via Early Exiting from Deep Neural Networks. InProceedings of the 23rd International Conference on Pattern Recognition (ICPR). IEEE, 2464–2469

    2016

  31. [31]

    Thomas Verelst and Tinne Tuytelaars. 2020. Dynamic Convolutions: Exploiting Spatial Sparsity for Faster Inference. In2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2317–2326. https://doi.org/10.1109/ cvpr42600.2020.00239

    arXiv 2020

  32. [32]

    Gonzalez

    Xin Wang, Fisher Yu, Zi-Yi Dou, Trevor Darrell, and Joseph E. Gonzalez. 2018. SkipNet: Learning Dynamic Routing in Convolutional Networks. InProceedings of the European Conference on Computer Vision (ECCV). Springer, 409–424

    2018

  33. [33]

    Wei Wen, Chunpeng Wu, Yandan Wang, Yiran Chen, and Hai Li. 2016. Learning structured sparsity in deep neural networks. InProceedings of the 30th Interna- tional Conference on Neural Information Processing Systems(Barcelona, Spain) (NIPS’16). Curran Associates Inc., Red Hook, NY, USA, 2082–2090

    2016

  34. [34]

    Davis, Kristen Grauman, and Rogerio Feris

    Zuxuan Wu, Tushar Nagarajan, Abhishek Kumar, Steven Renber, Larry S. Davis, Kristen Grauman, and Rogerio Feris. 2018. BlockDrop: Dynamic Inference Paths in Residual Networks. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 8817–8826

    2018

  35. [35]

    Zhewen Yu, Sudarshan Sreeram, Krish Agrawal, Junyi Wu, Alexander Montgomerie-Corcoran, Cheng Zhang, Jianyi Cheng, Christos-Savvas Bouganis, and Yiren Zhao. 2024. HASS: Hardware-Aware Sparsity Search for Dataflow DNN Accelerator. In2024 34th International Conference on Field-Programmable Logic and Applications (FPL). 257–263. https://doi.org/10.1109/FPL648...

    arXiv 2024

  36. [36]

    Keck- ler, and Zhengya Zhang

    Jie-Fang Zhang, Ching-En Lee, Chester Liu, Yakun Sophia Shao, Stephen W. Keck- ler, and Zhengya Zhang. 2021. SNAP: An Efficient Sparse Neural Acceleration Processor for Unstructured Sparse Deep Neural Network Inference.IEEE Journal of Solid-State Circuits56, 2 (2021), 636–647. https://doi.org/10.1109/JSSC.2020. 3043870

    work page doi:10.1109/jssc.2020 2021

  37. [37]

    Xiaofan Zhang, Hanchen Ye, Junsong Wang, Yonghua Lin, Jinjun Xiong, Wen-Mei Hwu, and Deming Chen. 2020. DNNExplorer: A Framework for Modeling and Exploring a Novel Paradigm of FPGA-based DNN Accelerator. In2020 IEEE/ACM International Conference On Computer Aided Design (ICCAD). 1–9

    2020

  38. [38]

    Zhekai Zhang, Hanrui Wang, Song Han, and William J. Dally. 2020. SpArch: Efficient Architecture for Sparse Matrix Multiplication. InProceedings of the 26th IEEE International Symposium on High-Performance Computer Architecture (HPCA). IEEE, 261–274

    2020

  39. [39]

    Xuda Zhou, Zidong Du, Qi Guo, Shaoli Liu, Chengsi Liu, Chao Wang, Xuehai Zhou, Ling Li, Tianshi Chen, and Yunji Chen. 2018. Cambricon-S: Addressing Ir- regularity in Sparse Neural Networks through A Cooperative Software/Hardware Approach. InProceedings of the 51st Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). IEEE, 15–28

    2018

  40. [40]

    Ac- tive Tiles

    Jinming Zhuang, Jason Lau, Hanchen Ye, Zhuoping Yang, Yubo Du, Jack Lo, Tejus Gupta, Jason Cong, Deming Chen, and Peipei Zhou. 2023. CHARM: Composing Heterogeneous AcceleRators for Matrix Multiply on Versal ACAP Architecture. InProceedings of the ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (FPGA). ACM, 153–164. 13 MICRO 2026, Octob...

    2023

  41. [2019]

    InProceedings of the 52nd Annual IEEE/ACM International Symposium on Microar- chitecture (MICRO)

    SparTen: A Sparse Tensor Accelerator for Convolutional Neural Networks. InProceedings of the 52nd Annual IEEE/ACM International Symposium on Microar- chitecture (MICRO). ACM, 151–165