Pith. sign in

REVIEW 4 major objections 5 minor 46 references

Detection Transformers Under the Knife: A Neuroscience-Inspired Approach to Ablations

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that detection transformer generations have distinct resilience patterns, with newer models storing knowledge more redundantly, and that DINO's static content queries become expendable after training.

desk verdict A useful empirical map of which DETR-family components tolerate ablation, but the headline causal story about DINO's look-forward-twice rule outruns the evidence. read the letter →

arxiv 2507.21723 v1 pith:TVAVENN2 submitted 2025-07-29 cs.CV cs.AI

Nils Hütten , Florian Hölken , Hasan Tercan , Tobias Meisen This is my paper · ORCID
classification cs.CVcs.AI
keywords explainableAIablationstudydetectiontransformersDETRDDETRDINOmulti-headattentionCOCOobject
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper adapts neuroscience-style ablation to three generations of detection transformers (DETR, DDETR, and DINO) trained on COCO, selectively zeroing weights inside attention components and query embeddings to map where knowledge actually lives. Its central claim is that architectural evolution has changed that map: DETR depends heavily on encoder self-attention and decoder cross-attention, DDETR becomes more robust because deformable attention and multi-scale features spread information, and DINO distributes knowledge so widely that most single-component ablations barely hurt. The authors also show that DINO's static content queries are disposable after training—fully zeroing them can slightly improve both metrics, and a model retrained with frozen zero queries loses less than one percentage point of mgIoU. If this picture holds, the roles of internal components are not fixed but reorganize across model generations, and redundancy in newer detectors opens concrete opportunities for simpler, faster architectures.

What carries the argument

The machinery is the neuroscience-inspired ablation protocol itself: randomly zeroing the input projection matrices ($W_q$, $W_k$, $W_v$) of encoder and decoder MHSA layers and decoder MHCA layers, plus scalar-level zeroing of query embeddings, applied at 5%, 15%, 30%, and 50% ablation levels across all blocks and at 30% in single-block experiments, each averaged over 100 random configurations. The effects are scored separately for regression and classification using mgIoU and class-weighted F1, which lets the authors see whether knowledge for localization and for classification is organized in different components. The protocol is complemented by a sparsity analysis of query embeddings, showing that DINO's static content queries are 58% near-zero after training compared to about 4% in DETR and DDETR.

What would settle it

Retrain DINO with the look-forward-twice update rule disabled (or replaced by a standard single-gradient path) under identical settings, then repeat the block-wise ablations; if the distributed-resilience pattern persists, the rule is not the cause. A complementary test is to train a DETR variant that adds dynamic anchors without the look-forward-twice rule and check whether content queries become expendable, isolating which architectural change actually shifts the function.

Watch

Extended reading notes

Core claim

The authors find a clear resilience gradient across detection transformer generations. DETR is the most fragile: ablating 50% of encoder MHSA projection weights costs up to 39 percentage points of mgIoU and 29 points of F1, decoder MHCA is the second most critical component, and decoder MHSA is nearly disposable. DDETR, with its deformable attention and multi-scale inputs, halves the damage from encoder MHSA ablations and shows that decoder MHCA layers are largely redundant block-wise; its enlarged query embeddings become more sensitive at low ablation levels, but its reference points contribute almost nothing. DINO is the most resilient: block-wise ablations of any attention component produce negligible regression loss, and fully ablating its static content queries yields a small improvement of +0.03%p mgIoU and +1.15%p F1. A retrained DINO with frozen zero content queries reaches 81.08% mgIoU versus 81.63% with learnable queries, suggesting the queries mainly help early training. The authors attribute DINO's distributed knowledge primarily to its look-forward-twice update rule.

Load-bearing premise

The paper attributes DINO's resilience to the look-forward-twice update rule and DDETR's to deformable attention, but these explanations rest on comparing three different pretrained checkpoints without ablating or varying those mechanisms themselves.

Editorial extensions

If this is right

  • DETR's decoder MHSA can be bypassed without harming performance, suggesting a leaner architecture that feeds queries directly into cross-attention.
  • DDETR's decoder MHCA layers are largely redundant block-wise, so removing some cross-attention blocks could cut compute with minimal loss.
  • DINO can be trained with frozen zero content queries at a small mgIoU cost (81.08% vs 81.63%) and a slight F1 gain, implying static queries only ease early training.
  • The resilience gradient from DETR to DDETR to DINO means newer detection transformers are more fault-tolerant, a property that matters in safety-critical deployment.
  • Block-wise ablation identifies which encoder blocks carry regression-critical features, for example DDETR's last encoder block is the only critical one, enabling targeted simplification.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The causal story predicts that if the look-forward-twice rule is what distributes knowledge, then training DETR or DDETR with that rule should flatten their block-wise ablation curves; disabling the rule in DINO should re-concentrate sensitivity in the final decoder blocks.
  • The expendability of DINO's content queries suggests the query-embedding function has shifted into the encoder; ablating encoder output features or the mixed query selection pathway could locate where that function now lives.
  • Applying the same ablation protocol to other DETR variants, such as two-stage DDETR or models with dynamic anchors but no look-forward-twice, would help separate the effects of dynamic anchors from the update rule.
  • The redundancy in decoder MHCA layers hints that depth-pruning methods for object detection transformers should target cross-attention before self-attention, a hypothesis that existing pruning benchmarks could test directly.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. This manuscript presents a post-hoc ablation study of three detection transformers—DETR, Deformable DETR (DDETR), and DINO—using randomly zeroed weights in query embeddings, encoder/decoder multi-head self-attention, and decoder multi-head cross-attention projection matrices. The authors measure changes in mean generalized IoU (mgIoU) and F1-score on the COCO validation set for full ablations at 5–50% and block-wise 30% ablations, and they release the DeepDissect library for reproducibility. The central claims are that the models exhibit distinct resilience patterns, with DETR most sensitive, DDETR intermediate due to deformable attention, and DINO most resilient due to its look-forward-twice update rule; that DDETR and DINO decoder cross-attention layers contain structural redundancy; and that DINO's static content queries become expendable after training, supported by a retraining experiment with frozen zero content queries.

Significance. The descriptive measurements are a useful addition to the DETR interpretability literature: a systematic map of which components matter for regression versus classification across three model generations, plus an open-source tool, is valuable to practitioners seeking to simplify or debug detection transformers. The retraining experiment for DINO's content queries is a genuine attempt to validate an ablation finding, and the sparsity analysis gives an independent signal. However, the paper's headline mechanistic conclusion—that the look-forward-twice update rule causes distributed knowledge and resilience—is not established by the experiments, because the compared checkpoints differ in many architectural details, training schedules, and losses. The causal language in the abstract and Section 4.3 overreaches the evidence.

major comments (4)
  1. [Abstract and Section 4.3] The claim that DINO's greater resilience is 'due to its look-forward twice update rule' is not supported by the evidence. DINO differs from DDETR in at least four co-introduced mechanisms (dynamic anchors, mixed query selection, contrastive denoising, and training schedule), and the paper never varies the rule itself. The statement 'DINO's block-wise ablations indicate that training with the 'look forward twice' rule results in the aspired objective - highly distributed knowledge across blocks' is a causal inference from a single trained model. Either a controlled retraining that toggles the rule, or rephrasing the conclusion as a hypothesis that is not tested here, is required.
  2. [Section 3 and Table 1] The three models are compared as pretrained MMDetection checkpoints, but the manuscript does not report their training schedules, augmentations, or losses. Standard MMDetection checkpoints for DETR, DDETR, and DINO use substantially different training budgets and regularization, so the observed resilience gradient could reflect training history rather than architectural innovation. Please report the exact checkpoint configurations and, if possible, compare models trained under matched schedules, or explicitly restrict the claims to 'these particular checkpoints' rather than to the architectures in general.
  3. [Section 4.2] The interpretation that deformable attention 'compensates' for the influence of projection-matrix values is not directly tested. Ablating projection matrices in DDETR's deformable attention does not measure the mechanism of token sampling, since the offset network and sampling operations are untouched. The sentence 'With NIA we were able to show that deformable attention's reduction to the most important tokens can compensate...' overstates what an ablation of Wq/Wk/Wv can reveal. Comparing DDETR against a non-deformable multi-scale variant or ablating the offset network would be needed to support this mechanism.
  4. [Section 4.3] The retraining experiment with frozen zero content queries, while commendable, is based on single runs with no variance or significance testing. The reported numbers are mixed: mgIoU drops from 81.63 to 81.08, while F1 rises from 85.45 to 86.9. The paper interprets this as content queries being expendable, but no account of run-to-run variability is given, and the metric discrepancy is not discussed. Please report multiple seeds or treat these numbers as descriptive only.
minor comments (5)
  1. [Section 1 and 4.3] There are several typos and garbled terms: 'acqes' should be 'QEs' in Section 1, and 'acpqe' and 'baux i' in Section 4.3 should be 'content queries' and 'b_i^aux' respectively.
  2. [Appendix Figure A5] The caption for Figure A5 says 'DINO performance differences for increasing ablation percentages in QEs...', but the figure and the referring text in Section 4.2 concern DDETR's class-wise gIoU reduction. The caption should be corrected.
  3. [Section 3] The F1-score is computed after Hungarian matching based on gIoU and after excluding unmatched or invalid detections, so the classification metric is not fully independent of regression quality. A sentence acknowledging this coupling would help readers interpret the classification versus regression comparisons.
  4. [References] Reference [31] is listed with an institute name rather than a publication venue; if a peer-reviewed version exists, it should be cited instead.
  5. [Throughout] The hyphenation of 'look-forward-twice' is inconsistent (e.g., 'look forward twice' in the abstract and Section 4.3); please standardize.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the study is an empirical ablation-measurement study with no fitted parameters or predictions that reduce to its inputs.

full rationale

The paper is self-contained as an empirical measurement study: it takes fixed COCO-trained checkpoints, ablates weights or query embeddings, and reports performance deltas. There is no derivation chain in which an output is defined in terms of the target result, no fitted parameter that is later relabeled as a prediction, and no self-citation that carries the load of the central claim. Hypotheses in Section 4 are explicitly formed from the original model publications and then tested against the measured ablation curves; the retraining experiment in Section 4.3 provides an independent validation rather than a retrodiction. The self-citations to prior NIA work (Meyes et al. [20], Lillian et al. [31], Meyes et al. [33]) motivate the methodology and the authors' ongoing research program, but the ablation results are new measurements on MMDetection checkpoints and are not forced by those citations. The strongest concern raised by the reader and skeptic is that the causal attribution of DINO's resilience to the look-forward-twice rule is confounded by other architectural differences between DINO and DDETR; that is a validity or identifiability weakness, not a circularity, because the descriptive resilience gradient is independently measured and the paper does not fit any parameter to make the claim true. Accordingly, no circular step can be exhibited by quoting equations or fitted-parameter reductions, and the circularity score is 0.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The central claims rest on methodological assumptions rather than on any fitted parameters. The ablation percentages and random configuration counts are experimental design choices, not free parameters fitted to data. The models themselves are pretrained checkpoints taken as given from MMDetection; the paper does not fit or tune any weights except in the DINO retraining check, whose settings are described only as standard mmdetection settings. No new entities are postulated. The assumptions below are domain assumptions about the validity of the measurement and attribution.

assumptions (4)
  • domain assumption MMDetection pretrained checkpoints for DETR, DDETR, and DINO are representative of their architectures and comparable to each other.
    All comparisons across models assume that checkpoint quality and training schedules do not systematically confound ablation sensitivity. Baselines are taken from MMDetection without retraining or reporting the exact configs.
  • domain assumption Randomly zeroing a percentage of weights in an attention projection matrix impairs the component's function in proportion to the percentage.
    Residual connections and other pathways may partially bypass the zeroed projections, so the measured drop could underestimate or redistribute the effect. The paper relies on this to interpret ablation levels.
  • domain assumption The gIoU and F1 metrics, computed with Hungarian matching, separately capture regression and classification sub-tasks.
    The matching procedure and thresholds are only sketched, and F1 depends on a decision threshold that is not specified.
  • ad hoc to paper DINO's resilience to ablations is caused by its look-forward-twice update rule.
    This mechanism is not independently ablated or varied; the statement in Section 4.3 is an interpretation used to explain the observed robustness.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Detection Transformers Under the Knife: A Neuroscience-Inspired Approach to Ablations." pith.science (2026). https://pith.science/paper/TVAVENN2

@misc{pith2026250721723,
  author       = {Pith},
  title        = {Pith review of: Detection Transformers Under the Knife: A Neuroscience-Inspired Approach to Ablations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TVAVENN2}},
  note         = {Machine review of arXiv:2507.21723}
}
read the original abstract

In recent years, Explainable AI has gained traction as an approach to enhancing model interpretability and transparency, particularly in complex models such as detection transformers. Despite rapid advancements, a substantial research gap remains in understanding the distinct roles of internal components - knowledge that is essential for improving transparency and efficiency. Inspired by neuroscientific ablation studies, which investigate the functions of brain regions through selective impairment, we systematically analyze the impact of ablating key components in three state-of-the-art detection transformer models: Detection transformer (DETR), deformable detection transformer (DDETR), and DETR with improved denoising anchor boxes (DINO). The ablations target query embeddings, encoder and decoder multi-head self-attentions (MHSA) as well as decoder multi-head cross-attention (MHCA) layers. We evaluate the effects of these ablations on the performance metrics gIoU and F1-score, quantifying effects on both the classification and regression sub-tasks on the COCO dataset. To facilitate reproducibility and future research, we publicly release the DeepDissect library. Our findings reveal model-specific resilience patterns: while DETR is particularly sensitive to ablations in encoder MHSA and decoder MHCA, DDETR's multi-scale deformable attention enhances robustness, and DINO exhibits the greatest resilience due to its look-forward twice update rule, which helps distributing knowledge across blocks. These insights also expose structural redundancies, particularly in DDETR's and DINO's decoder MHCA layers, highlighting opportunities for model simplification without sacrificing performance. This study advances XAI for DETRs by clarifying the contributions of internal components to model performance, offering insights to optimize and improve transparency and efficiency in critical applications.

Figures

Figures reproduced from arXiv: 2507.21723 by the authors.

Figure 1
Figure 1. Visualization of multi-head attention ablations on [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. DETR, DDETR and DINO performance differences for increasing ablation percentages in QEs, encoder MHSA, decoder MHSA and decoder MHCA. The shaded areas correspond to the standard deviation. Separate visualizations for each model in Appendix Fig. A2, A3, A4 reveal a steadily decreasing mgIoU from block four onward, indicating that substantial feature refinement mostly occurs in the deeper half of the encoder. Classifi… view at source ↗
Figure 3
Figure 3. DETR and DDETR performance difference for 30% block-wise ablations in encoder MHSA and decoder MHCA. The shaded areas correspond to the standard deviation. 4.2 Deformable Detection Transformer Model (DDETR) Our hypotheses for the DDETR ablations stem from its two core innovations: multi-scale features and deformable attention. We expect that adopting multi-scale features leads to larger performance losses in both co… view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: DINO performance difference for 30% block-wise ablations in encoder MHSA and decoder MHCA. The shaded areas correspond to the standard deviation. greater influence than in DETR and DDETR, due to its processing of dynamic anchors fused with the static queries - though t…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

46 extracted references · 34 canonical work pages

  1. [1]

    Gomez, Lukasz Kaiser, and Illia Polosukhin

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. In Advances in Neural Information Processing Systems, volume 30, 2017

    2017

  2. [2]

    An image is worth 16x16 words: Transformers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929, 2020

    arXiv 2010

  3. [3]

    Swin transformer: Hierarchical vision transformer using shifted windows

    Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 10012–10022, 2021

    2021

  4. [4]

    Focal self-attention for local-global interactions in vision transformers, 2022

    Jianwei Yang, Chunyuan Li, Pengchuan Zhang, Xiyang Dai, Bin Xiao, Lu Yuan, and Jianfeng Gao. Focal self-attention for local-global interactions in vision transformers, 2022

    work page 2022

  5. [5]

    Transformers in Vision: A Survey

    Salman Khan, Muzammal Naseer, Munawar Hayat, Syed Waqas Zamir, Fahad Shahbaz Khan, and Mubarak Shah. Transformers in vision: A survey. arXiv preprint arXiv:2101.01169, 2021

    work page Pith review arXiv 2021

  6. [6]

    Deep learning for automated visual inspection in manufacturing and maintenance: A survey of open- access papers

    Nils Hütten, Miguel Alves Gomes, Florian Hölken, Karlo Andricevic, Richard Meyes, and Tobias Meisen. Deep learning for automated visual inspection in manufacturing and maintenance: A survey of open- access papers. Applied System Innovation, 7(1):11, 2024

    work page 2024

  7. [7]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

    arXiv 2018

  8. [8]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, ...

    arXiv 2005

Show all 46 references
  1. [9]

    Swin transformer v2: Scaling up capacity and resolution

    Ze Liu, Han Hu, Yutong Lin, Zhuliang Yao, Zhenda Xie, Yixuan Wei, Jia Ning, Yue Cao, Zheng Zhang, Li Dong, et al. Swin transformer v2: Scaling up capacity and resolution. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12009–12019, 2022

    2022

  2. [10]

    Florence: A new foundation model for computer vision, 2021

    Lu Yuan, Dongdong Chen, Yi-Ling Chen, Noel Codella, Xiyang Dai, Jianfeng Gao, Houdong Hu, Xuedong Huang, Boxin Li, Chunyuan Li, Ce Liu, Mengchen Liu, Zicheng Liu, Yumao Lu, Yu Shi, Lijuan Wang, Jianfeng Wang, Bin Xiao, Zhen Xiao, Jianwei Yang, Michael Zeng, Luowei Zhou, and Pe...

    2021

  3. [11]

    Focal modulation networks, 2022

    Jianwei Yang, Chunyuan Li, Xiyang Dai, Lu Yuan, and Jianfeng Gao. Focal modulation networks, 2022

    2022

  4. [12]

    Internimage: Exploring large-scale vision foundation models with deformable convolutions, 2022

    Wenhai Wang, Jifeng Dai, Zhe Chen, Zhenhang Huang, Zhiqi Li, Xizhou Zhu, Xiaowei Hu, Tong Lu, Lewei Lu, Hongsheng Li, Xiaogang Wang, and Yu Qiao. Internimage: Exploring large-scale vision foundation models with deformable convolutions, 2022

    2022

  5. [13]

    Explainability and evaluation of vision transformers: An in-depth experimental study

    Sédrick Stassin, Valentin Corduant, Sidi Ahmed Mahmoudi, and Xavier Siebert. Explainability and evaluation of vision transformers: An in-depth experimental study. Electronics, 13(1):175, 2024

    2024

  6. [14]

    Holistically explainable vision transformers

    Moritz Böhle, Jonas Löffler, Christian Pahins, and Kristian Kersting. Holistically explainable vision transformers. arXiv preprint arXiv:2301.08669, 2023

    2023 arXiv

  7. [15]

    Explainability of vision transformers: A comprehensive review and new perspectives

    Rojina Kashefi, Leili Barekatain, Mohammad Sabokrou, and Fatemeh Aghaeipoor. Explainability of vision transformers: A comprehensive review and new perspectives. arXiv preprint arXiv:2311.06786, 2023

    2023 arXiv

  8. [16]

    Scoville and Brenda Milner

    William B. Scoville and Brenda Milner. Loss of recent memory after bilateral hippocampal lesions. Journal of Neurology, Neurosurgery & Psychiatry, 20(1):11–21, 1957

    1957

  9. [17]

    Edmund T. Rolls. Hippocampo-cortical and cortico-cortical backprojections. Hippocampus, 10(4):380–388, 2000

    2000

  10. [18]

    Ungerleider and Mortimer Mishkin

    Leslie G. Ungerleider and Mortimer Mishkin. Two cortical visual systems. In David J. Ingle, Melvyn A. Goodale, and Richard J. W. Mansfield, editors, Analysis of Visual Behavior, pages 549–586. MIT Press, Cambridge, MA, 1982

    1982

  11. [19]

    The emotional brain: The mysterious underpinnings of emotional life

    Joseph LeDoux. The emotional brain: The mysterious underpinnings of emotional life. Simon and Schuster, 1996

    1996

  12. [20]

    Ablation studies in artificial neural networks

    Richard Meyes, Melanie Lu, Constantin Waubert de Puiseau, and Tobias Meisen. Ablation studies in artificial neural networks. arXiv preprint arXiv:1901.08644, 2019. 10 Detection Transformers Under the Knife: A Neuroscience-Inspired Approach to Ablations

    1901 arXiv

  13. [21]

    MMDetection: Open mmlab detection toolbox and benchmark

    Kai Chen, Jiaqi Wang, Jiangmiao Pang, Yuhang Cao, Yu Xiong, Xiaoxiao Li, Shuyang Sun, Wansen Feng, Ziwei Liu, Jiarui Xu, Zheng Zhang, Dazhi Cheng, Chenchen Zhu, Tianheng Cheng, Qijie Zhao, Buyu Li, Xin Lu, Rui Zhu, Yue Wu, Jifeng Dai, Jingdong Wang, Jianping Shi, Wanli Ouyang,...

    1906 arXiv

  14. [22]

    Microsoft coco: Common objects in context, 2014

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context, 2014

    2014

  15. [23]

    A tutorial on speech understanding systems

    Allen Newell. A tutorial on speech understanding systems. Speech Recognition: Invited Papers Presented at the 1974 IEEE Symposium, 1975:3, 1974

    1974

  16. [24]

    Aggregated residual transformations for deep neural networks, 2017

    Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. Aggregated residual transformations for deep neural networks, 2017

    2017

  17. [25]

    Faster r-cnn: Towards real-time object detection with region proposal networks, 2015

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks, 2015

    2015

  18. [26]

    Feature pyramid networks for object detection, 2017

    Tsung-Yi Lin, Piotr Dollár, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection, 2017

    2017

  19. [27]

    End-to-end object detection with transformers, 2020

    Nicolas Carion, Francisco Massa, Gabriel Synnaeve, Nicolas Usunier, Alexander Kirillov, and Sergey Zagoruyko. End-to-end object detection with transformers, 2020

    2020

  20. [28]

    Rich feature hierarchies for accurate object detection and semantic segmentation, 2013

    Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. Rich feature hierarchies for accurate object detection and semantic segmentation, 2013

    2013

  21. [29]

    Bayan Bruss

    Isha Hameed, Samuel Sharpe, Daniel Barcklow, Justin Au-Yeung, Sahil Verma, Jocelyn Huang, Brian Barr, and C. Bayan Bruss. Based-xai: Breaking ablation studies down for explainable artificial intelligence, 2022

    2022

  22. [30]

    Vishnusai, Tejas R

    Y . Vishnusai, Tejas R. Kulakarni, and K. Sowmya Nag. Ablation of artificial neural networks. In Jennifer S. Raj, Abul Bashar, and S. R. Jino Ramson, editors, Innovative Data Communication Technologies and Application, volume 46 of Lecture Notes on Data Engineering and Communi...

    2020

  23. [31]

    Lillian, Richard Meyes, and Tobias Meisen

    Peter E. Lillian, Richard Meyes, and Tobias Meisen. Ablations on a robot’s brain: Neural networks under a knife. Institute of Information Management in Mechanical Engineering, RWTH Aachen University, 2019

    2019

  24. [32]

    Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra

    Ramprasaath R. Selvaraju, Michael Cogswell, Abhishek Das, Ramakrishna Vedantam, Devi Parikh, and Dhruv Batra. Grad-cam: Visual explanations from deep networks via gradient-based localization. In 2017 IEEE International Conference on Computer Vision (ICCV), pages 618–626, 2017

    2017

  25. [33]

    Transparent and interpretable failure prediction of sensor time series data with convolutional neural networks

    Richard Meyes, Nils Hütten, and Tobias Meisen. Transparent and interpretable failure prediction of sensor time series data with convolutional neural networks. Procedia CIRP, 2021

    2021

  26. [34]

    Sparsegpt: Massive language models can be accurately pruned in one-shot

    Elias Frantar and Dan Alistarh. Sparsegpt: Massive language models can be accurately pruned in one-shot. arXiv preprint arXiv:2301.00774v3, 2023

    2023 arXiv

  27. [35]

    Shortgpt: Layers in large language models are more redundant than you expect

    Xin Men, Mingyu Xu, Qingyu Zhang, Bingning Wang, Hongyu Lin, Yaojie Lu, Xianpei Han, and Weipeng Chen. Shortgpt: Layers in large language models are more redundant than you expect. arXiv preprint arXiv:2403.03853v3, 2024

    2024 arXiv

  28. [36]

    Zico Kolter

    Mingjie Sun, Zhuang Liu, Anna Bair, and J. Zico Kolter. A simple and effective pruning approach for large language models. In Proceedings of the International Conference on Learning Representations (ICLR), 2024

    2024

  29. [37]

    Upop: Unified and progressive pruning for compressing vision-language transformers

    Dachuan Shi, Chaofan Tao, Ying Jin, Zhendong Yang, Chun Yuan, and Jiaqi Wang. Upop: Unified and progressive pruning for compressing vision-language transformers. In Proceedings of the 40th International Conference on Machine Learning (ICML), 2023

    2023

  30. [38]

    Shortened llama: Depth pruning for large language models with comparison of retraining methods

    Bo-Kyeong Kim, Geonmin Kim, Tae-Ho Kim, Thibault Castells, Shinkook Choi, Junho Shin, and Hyoung-Kyu Song. Shortened llama: Depth pruning for large language models with comparison of retraining methods. arXiv preprint arXiv:2301.13741v3, 2024

    2024 arXiv

  31. [39]

    Width & depth pruning for vision transformers

    Fang Yu, Kun Huang, Meng Wang, Yuan Cheng, Wei Chu, and Li Cui. Width & depth pruning for vision transformers. Proceedings of the AAAI Conference on Artificial Intelligence, 36(3):3143–3151, 2022

    2022

  32. [40]

    X-pruner: explainable pruning for vision transformers

    Lu Yu and Wei Xiang. X-pruner: explainable pruning for vision transformers. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 24355–24363, 2023

    2023

  33. [41]

    Revisiting Token Pruning for Object Detection and Instance Segmentation

    Yifei Liu, Mathias Gehrig, Nico Messikommer, Marco Cannici, and Davide Scaramuzza. Revisiting Token Pruning for Object Detection and Instance Segmentation . In 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 2646–2656, Los Alamitos, CA, USA, Ja...

    2024

  34. [42]

    Efficient pruning of detection transformer in remote sensing using ant colony evolutionary pruning

    Hailin Su, Haijiang Sun, and Yongxian Zhao. Efficient pruning of detection transformer in remote sensing using ant colony evolutionary pruning. Applied Sciences, 15(1):200, 2025

    2025

  35. [43]

    Pruning detr: efficient end-to-end object detection with sparse structured pruning

    Huaiyuan Sun, Shuili Zhang, Xve Tian, and Yuanyuan Zou. Pruning detr: efficient end-to-end object detection with sparse structured pruning. Signal, Image and Video Processing, 18(1):129–135, 2024

    2024

  36. [44]

    Deformable detr: Deformable transformers for end-to-end object detection, 2020

    Xizhou Zhu, Weijie Su, Lewei Lu, Bin Li, Xiaogang Wang, and Jifeng Dai. Deformable detr: Deformable transformers for end-to-end object detection, 2020

    2020

  37. [45]

    DINO: DETR with improved denoising anchor boxes for end-to-end object detection

    Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel Ni, and Heung-Yeung Shum. DINO: DETR with improved denoising anchor boxes for end-to-end object detection. In The Eleventh International Conference on Learning Representations, 2023

    2023

  38. [46]

    Rezatofighi, N

    H. Rezatofighi, N. Tsoi, J. Gwak, A. Sadeghian, I. Reid, and S. Savarese. Generalized intersection over union: A metric and a loss for bounding box regression. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 658–666, Los Alamitos, CA, USA, ...

    2019

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.