Pith. sign in

REVIEW 38 cited by

End-to-End Object Detection with Transformers

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2005.12872 v3 pith:MU3HKHFY submitted 2020-05-26 cs.CV

Nicolas Carion , Francisco Massa , Gabriel Synnaeve , Nicolas Usunier , Alexander Kirillov , Sergey Zagoruyko This is my paper · ORCID
classification cs.CV
keywords detectiondetrobjectglobalmanypredictionstransformeraccuracy
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We present a new method that views object detection as a direct set prediction problem. Our approach streamlines the detection pipeline, effectively removing the need for many hand-designed components like a non-maximum suppression procedure or anchor generation that explicitly encode our prior knowledge about the task. The main ingredients of the new framework, called DEtection TRansformer or DETR, are a set-based global loss that forces unique predictions via bipartite matching, and a transformer encoder-decoder architecture. Given a fixed small set of learned object queries, DETR reasons about the relations of the objects and the global image context to directly output the final set of predictions in parallel. The new model is conceptually simple and does not require a specialized library, unlike many other modern detectors. DETR demonstrates accuracy and run-time performance on par with the well-established and highly-optimized Faster RCNN baseline on the challenging COCO object detection dataset. Moreover, DETR can be easily generalized to produce panoptic segmentation in a unified manner. We show that it significantly outperforms competitive baselines. Training code and pretrained models are available at https://github.com/facebookresearch/detr.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 38 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Specialization of softmax attention heads: insights from the high-dimensional single-location model

    cs.LG 2026-03 conditional novelty 7.0 of 10

    In a high-dimensional toy task, multi-head softmax attention first aligns all heads with the mean signal, then sequentially specializes to latent directions; the paper introduces Bayes-softmax, which attains the Bayes...

  2. Understanding Transformer-based Vision Models through Inversion

    cs.CV 2024-12 conditional novelty 7.0 of 10

    A modular, component-wise feature inversion technique applied to DETR and ViT shows that DETR progressively abstracts images into prototypical objects while ViT retains detailed local information.

  3. FUSEP: A Multi-Center Benchmark for Diverse Tasks in Early Pregnancy Fetal Ultrasound Screening

    cs.CV 2026-08 conditional novelty 6.0 of 10

    FUSEP is a new multi-center public benchmark with 4,017 early-pregnancy ultrasound images, 45,820 box annotations of 14 structures, and detection baselines across four learning paradigms.

  4. Kepler-Encoder-v0.1: Towards a Multimodal Embedding Model for Robots

    cs.RO 2026-07 conditional novelty 6.0 of 10

    A self-supervised multimodal encoder trained with vision, proprioception, and force yields a vision-only latent that recovers end-effector state and force above vision baselines on RH20T, with modest absolute force accuracy.

  5. Where Will They Go? Modelling Multimodal Pedestrian Manoeuvres from Ego-centric Videos

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    MMPM uses PIM for gaze/head/hand interactions and MTP (CVAE with query decoder) to model separate crossing/non-crossing trajectory distributions, outperforming baselines on PIE and JAAD with a new validation protocol.

  6. PokeNet: Learning Kinematic Models of Articulated Objects from Human Observations

    cs.RO 2026-02 conditional novelty 6.0 of 10

    PokeNet estimates joint types, axes, ranges, and operation order of articulated objects directly from a single-view point cloud video of a human demonstration.

  7. Benchmarking Table Extraction from Heterogeneous Scientific PDF Documents

    cs.DB 2025-11 conditional novelty 6.0 of 10

    A new benchmark with two new datasets and end-to-end metrics shows that table extraction from PDFs is still unreliable across heterogeneous layouts.

  8. TransForSeg: A Multitask Stereo ViT for Joint Stereo Segmentation and 3D Force Estimation in Catheterization

    cs.CV 2025-09 conditional novelty 6.0 of 10

    A shared-weight stereo ViT with cross-attention fusion segments the catheter in two views and regresses 3D tip forces, claiming state-of-the-art results on synthetic X-ray datasets.

  9. Transformer-Based Neural Network for Transient Detection without Image Subtraction

    cs.CV 2025-08 conditional novelty 6.0 of 10

    A distance-weighted transformer classifier using search plus template images reaches 97.4% accuracy on DES real/bogus data, matching difference-image methods and making image subtraction optional.

  10. Screen2AX: Vision-Based Approach for Automatic macOS Accessibility Generation

    cs.LG 2025-07 conditional novelty 6.0 of 10

    Screen2AX generates hierarchical macOS accessibility metadata from a screenshot and reports improved GPT-4 UI task success compared with native accessibility and OmniParser V2.

  11. Hierarchical Neural Collapse Detection Transformer for Class Incremental Object Detection

    cs.CV 2025-06 conditional novelty 6.0 of 10

    Hier-DETR reports state-of-the-art incremental detection accuracy, 48.4 AP on COCO 70+10 and 45.46 AP on MTSD 150+71, using a frozen hierarchical tree of general orthogonal frame prototypes with a proxy-based guiding loss.

  12. Stable Mean Teacher for Semi-supervised Video Action Detection

    cs.CV 2024-12 conditional novelty 6.0 of 10

    Stable Mean Teacher with an Error Recovery module and a Difference of Pixels constraint improves semi-supervised video action detection, reaching near fully-supervised accuracy with 10-20% labels.

  13. Panoptic Diffusion Models: co-generation of images and segmentation maps

    cs.CV 2024-12 reject novelty 6.0 of 10

    A dual-diffusion model that jointly denoises an image and a bit-encoded panoptic segmentation map from a text prompt, with multi-scale patching for high-resolution maps.

  14. TPCNet: Representation learning for HI mapping

    astro-ph.GA 2024-11 conditional novelty 6.0 of 10

    A CNN-Transformer hybrid with sinusoidal positional encoding predicts cold HI fraction and opacity correction from 21-cm emission, outperforming CNN baselines but biased at high column density.

  15. Small-Pollinator Detection in Cluttered Field Video

    cs.CV 2026-07 conditional novelty 5.0 of 10

    RF-DETR Large at 1344-pixel input beat all tested YOLO, ensemble, slicing, and temporal post-processing systems on the BuzzSpot hidden test, reaching 0.405 mAP50:95.

  16. Signal from Space: Detecting Schools and Towers to Bridge the Digital Divide

    cs.CV 2026-07 conditional novelty 5.0 of 10

    Transfer-learned YOLO detectors find schools and towers in Lesotho satellite imagery; CPP line-of-sight within 1 km triages schools as high-potential or hard-to-connect.

  17. Contrasting Cognitive Styles in Vision-Language Models: Holistic Attention in Japanese Versus Analytical Focus in English

    cs.CL 2025-07 reject novelty 5.0 of 10

    Japanese-prompted vision-language models produce more background-first captions than English-prompted ones, but the effect is confounded by the evaluator and by language grammar.

  18. CALM: Contextual Analog Logic with Multimodality

    cs.AI 2025-06 conditional novelty 5.0 of 10

    CALM couples neural perception with a fuzzy-logic-style reasoning engine to place objects into scenes, reporting 92.2% accuracy on a fill-in-the-blank task.

  19. Query Nearby: Offset-Adjusted Mask2Former enhances small-organ segmentation

    cs.CV 2025-06 reject novelty 5.0 of 10

    An offset-adjusted Mask2Former variant reports higher Dice scores for small-organ segmentation on HaNSeg and SegRap2023, though the SOTA claim relies on non-standard evaluation settings.

  20. PLATTER: A Page-Level Handwritten Text Recognition System for Indic Scripts

    cs.CV 2025-02 conditional novelty 5.0 of 10

    PLATTER couples a language-agnostic word detector with six word recognizers to perform page-level handwritten OCR on ten Indic languages, evaluated on the new synthetic CHIPS dataset.

  21. Mask-RadarNet: Enhancing Transformer With Spatial-Temporal Semantic Context for Radar Object Detection in Autonomous Driving

    cs.CV 2024-12 conditional novelty 5.0 of 10

    A hybrid 3D Swin Transformer with temporal patch shift and class-masked attention improves radar object detection accuracy on CRUW at lower computational cost.

  22. DocFusion: A Unified Framework for Document Parsing Tasks

    cs.CL 2024-12 conditional novelty 5.0 of 10

    A 289M-parameter generative model with a Gaussian-kernel cross-entropy loss jointly handles layout analysis, OCR, math expression recognition, and table recognition, with competitive but partially overstated benchmark gains.

  23. Sonar-based Deep Learning in Underwater Robotics: Overview, Robustness and Challenges

    cs.RO 2024-12 conditional novelty 5.0 of 10

    A survey of sonar-based deep learning that identifies robustness, dataset scarcity, and sim-to-real gaps as the main obstacles to safe underwater autonomy.

  24. WoodYOLO: A Novel Object Detector for Wood Species Detection in Microscopic Images

    cs.CV 2024-11 conditional novelty 5.0 of 10

    A customized YOLO detector, WoodYOLO, reports F2 0.848 at IoU 0.3 for vessel-element detection in wood microscopy, beating YOLOv10 and YOLOv7 on a private dataset.

  25. SynSur: An end-to-end generative pipeline for synthetic industrial surface defect generation and detection

    cs.CV 2026-04 unverdicted novelty 4.0 of 10

    A generative pipeline creates realistic synthetic pitting defects and other surface flaws that, when added to real training data, yield modest gains in industrial defect detectors without replacing the need for authen...

  26. Tracking Moose using Aerial Object Detection

    cs.CV 2025-07 conditional novelty 4.0 of 10

    Across 36 model/threshold/overlap configurations, all three detectors reached at least 93% mAP@IoU=0.5, and the lightweight YOLOv11n matched the heavier models.

  27. Improve Underwater Object Detection through YOLOv12 Architecture and Physics-informed Augmentation

    cs.CV 2025-06 conditional novelty 4.0 of 10

    Applying YOLOv12 with physics-flavored augmentations yields high reported mAP on four underwater detection benchmarks, but the claims are weakened by missing code, variance, and inconsistent speed numbers.

  28. Visual Product Graph: Bridging Visual Products And Composite Images For End-to-End Style Recommendations

    cs.CV 2025-05 conditional novelty 4.0 of 10

    Pinterest's Visual Product Graph adds a product-to-scene retrieval direction to the existing Shop the Look system, retrieving inspirational composite images for a product and recommending complementary shoppable items.

  29. ViSIR: Vision Transformer Single Image Reconstruction Method for Earth System Models

    cs.CV 2025-02 reject novelty 4.0 of 10

    ViSIR, a Vision Transformer with a SIREN head, is claimed to improve super-resolution quality on E3SM climate images by 2 to 8 dB over four baselines, though the experimental setup is incomplete.

  30. Unified Local and Global Attention Interaction Modeling for Vision Transformers

    cs.CV 2024-12 conditional novelty 4.0 of 10

    Adding local and global token interactions before self-attention, via an aggressive convolution-pooling block and a concept-attention block, improves RetinaNet object detection mAP over non-pretrained ViT, Swin, and D...

  31. TabSniper: Towards Accurate Table Detection & Structure Recognition for Bank Statements

    cs.CV 2024-12 reject novelty 4.0 of 10

    TabSniper reports improved table detection and structure recognition on bank statements by fine-tuning DETR with CIoU loss, long-table split-merge, and padding variations, evaluated on a private dataset and two public...

  32. DIS-Mine: Instance Segmentation for Disaster-Awareness in Poor-Light Condition in Underground Mines

    cs.CV 2024-11 reject novelty 4.0 of 10

    DIS-Mine fuses enhanced SAM and Mask R-CNN masks to segment objects in dark mine images, reporting 86% F1 on one external benchmark but only 70.2% on its own new ImageMine dataset.

  33. CCi-YOLOv8n: Enhanced Fire Detection with CARAFE and Context-Guided Modules

    cs.CV 2024-11 conditional novelty 4.0 of 10

    A YOLOv8n variant combining CARAFE, Context-Guided Downsampling, and iRMB reports small accuracy gains over YOLOv8n on two fire-detection datasets.

  34. Exploring Visual Embedding Spaces Induced by Vision Transformers for Online Auto Parts Marketplaces

    cs.CV 2025-02 conditional novelty 3.0 of 10

    Visual embeddings from a pretrained ViT produce poorly separated clusters of auto parts images (silhouette 0.015), far below the 0.38 reported for a multimodal model on similar data.

  35. Survey on Question Answering over Visually Rich Documents: Methods, Challenges, and Trends

    cs.CL 2025-01 conditional novelty 3.0 of 10

    A structured overview of question answering over visually rich documents, comparing encoding, vision-only, and multi-page methods, and highlighting comparability issues in existing benchmarks.

  36. A Real-Time DETR Approach to Bangladesh Road Object Detection for Autonomous Vehicles

    cs.CV 2024-11 conditional novelty 3.0 of 10

    Fine-tuning RT-DETR on the BadODD Bangladesh road dataset yields mAP50 of 0.415 on the public test split and 0.282 on the private split.

  37. SHeRL-FL: When Representation Learning Meets Split Learning in Hierarchical Federated Learning

    cs.LG 2025-08 unverdicted novelty 2.0 of 10

    The submitted body is an unrelated survey, not the SHeRL-FL method claimed in the metadata.

  38. Image Segmentation with transformers: An Overview, Challenges and Future

    cs.CV 2025-01 reject

    A high-level review of transformer-based image segmentation that restates known models and challenges without new results, and contains citation and metric errors.

Pith tools