Pith. sign in

REVIEW 4 major objections 5 minor 54 references

Dynamic-DINO: Fine-Grained Mixture of Experts Tuning for Real-time Open-Vocabulary Object Detection

T0 review · 4 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Fine-grained mixture-of-experts tuning lets an open-vocabulary detector trained on 1.56M images beat one trained on a private 20M-image dataset.

desk verdict The MoE fine-tuning trick is genuinely neat, but the headline claim about beating 20M-pretrained Grounding DINO 1.5 Edge rides on margins of 0.1-1.2 AP with no error bars. read the letter →

arxiv 2507.17436 v1 pith:2WYPRYD2 submitted 2025-07-23 cs.CV

Yehao Lu , Minghe Weng , Zekang Xiao , Rui Jiang , Wei Su , Guangcong Zheng , Ping Lu , Xi Li This is my paper
classification cs.CV
keywords open-vocabularyobjectdetectionmixtureofexpertsreal-timeMoE-Tuninggranularitydecompositiondynamicinferencezero-shotGroundingDINO
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that a mixture-of-experts (MoE) decoder, obtained by splitting a dense detector's feed-forward networks into fine-grained experts, can turn a 1.56M-image public training run into a real-time open-vocabulary detector that beats a model trained on a private 20M-image dataset. The proposed Dynamic-DINO converts Grounding DINO 1.5 Edge into a dynamic inference framework: for every token only the input-relevant experts are activated, so active parameters equal one feed-forward network and inference cost stays close to the dense model. The load-bearing trick is parameter-preserving initialization, in which each expert inherits a slice of the pretrained weights and the router is initialized so the first forward pass exactly reproduces the dense model's output. Experiments on COCO, LVIS, and ODinW support the claim. The practical upshot is a cheap upgrade path: MoE-Tuning trains roughly 1.87 times faster per epoch than the pretraining stage, and the reported accuracy curve rises smoothly instead of dipping at the start.

What carries the argument

The machinery is the split-and-activate MoE layer built by granularity decomposition of a pretrained FFN. A dense FFN with parameters W1 in $R^{{H x D}}$, b1, W2 in $R^{{D x H}}$, b2 is cut horizontally in W1 and vertically in W2 into k blocks, producing k smaller expert FFNs E1 through Ek with shared bias b2/k, so that the sum of Ej(x) equals the original FFN(x) for every token x. The same is done for N copies of the FFN, yielding kN experts with no extra randomly initialized parameters. The router is a single linear layer initialized by drawing N random centroid vectors and replicating each k times, so at the start of fine-tuning it necessarily selects the k experts sliced from the same FFN and the MoE layer output is exactly the pretrained output. Inference routes each token through only the top-2 scoring experts, keeping activated parameters equal to one FFN while the total parameter count grows from 178M to 222M in the reported times 16 Top2 model.

What would settle it

Run the released official Grounding DINO 1.5 Edge weights through the same MoE-Tuning recipe, or fine-tune the authors' reproduced base model on the full 20M-image private set: if Dynamic-DINO no longer outperforms the official model, the reported 1.56M-versus-20M advantage is an artifact of the reproduced baseline. A second check is to randomize the expert assignments after training: if token-to-expert specialization is load-bearing, shuffling which expert handles which pattern should measurably degrade zero-shot AP; if it does not, the claimed specialization mechanism is not the source of the gain.

Watch

Extended reading notes

Core claim

The paper's central discovery, stated on its own terms, is that a dense real-time open-vocabulary detector can be upgraded to a sparse dynamic detector by fine-grained MoE-Tuning, and that this upgrade raises zero-shot accuracy while keeping the activated parameter count identical to a single feed-forward network. Concretely, the authors reproduce Grounding DINO 1.5 Edge on public data, then replace each decoder FFN with N replicated FFNs, each divided into k smaller expert networks, giving kN experts derived from pretrained weights. A router selects the top-2 experts per token, and a weight-allocation scheme guarantees that at initialization the sum of those experts exactly equals the original FFN's output, so tuning starts at the base model's performance instead of degrading it. The resulting Dynamic-DINO times 16 Top2 model, trained on 1.56M open-source images, reports 43.7 COCO APbox, 33.6 LVIS-minival APall, and 27.4 LVIS-val APall at 640 by 640, versus 42.6, 31.1, and 25.4 for the reproduced dense baseline and 42.9, 33.5, and 27.3 for the official Grounding DINO 1.5 Edge pretrained on the private 20M-image dataset. At 800 by 1333 the reported margins grow, and the paper attributes the gains to expert specialization: deeper layers form stable two- to three-expert collaborations, each combination handling tokens of a specific pattern, which lets a small model partition the pattern space more cleanly.

Load-bearing premise

The claim that Dynamic-DINO beats a 20M-data model with only 1.56M data rests on the authors' reproduced Grounding DINO 1.5 Edge, trained on the same 1.56M public data, being a faithful stand-in for the official closed-source Edge; if that reproduction is systematically weaker than the official model in a way that MoE tuning happens to compensate for, the headline comparison could be an artifact of the baseline rather than of the method.

Editorial extensions

If this is right

  • The same activated-parameter budget can hold more total capacity: a 178M-active-parameter Dynamic-DINO times 16 Top2 model with 222M total parameters improves over the 178M dense baseline by 1.1 COCO AP, 2.5 LVIS-minival AP, and 2.0 LVIS-val AP at 640 by 640.
  • A detector trained on roughly 1.56M public images can match or exceed the zero-shot accuracy of one trained on the private 20M-image Grounding20M set, with comparable inference speed: the paper reports 98.0 versus 108.9 TensorRT FPS at 640 by 640.
  • MoE-Tuning is more efficient per epoch than pretraining from scratch: the paper reports 7.5 hours per epoch versus 14.0 hours for pretraining, and the accuracy curve rises smoothly rather than dipping at the start.
  • Fine-grained expert granularity is a tunable knob: splitting each FFN into two experts helps, and splitting into more than a certain number hurts when training data is limited, so the optimal search-space size depends on data scale.
  • Zero-shot referring-expression comprehension also improves: on RefCOCO, RefCOCO+, and RefCOCOg, Dynamic-DINO beats its dense base model by roughly 2 to 5 points.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The router's expert choices could be used as a free zero-shot saliency signal, since the paper's visualizations show specific expert pairs firing for specific object classes.
  • The observed over-subdivision decline suggests a curriculum that increases expert granularity as training data grows, a schedule the paper does not test.
  • A parallel multi-expert implementation would likely erase most of the remaining FPS gap (98.0 versus 108.9 TensorRT FPS at 640 by 640), since the paper identifies sequential expert evaluation as the current bottleneck.
  • The same FFN-splitting plus router-initialization recipe could transfer to other dense detectors and backbones; the paper's supplementary result that extending MoE to the image encoder helps hints that the gains are not decoder-specific.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Dynamic-DINO, a method that converts a dense real-time open-vocabulary object detector (Grounding DINO 1.5 Edge) into a sparse Mixture-of-Experts (MoE) model via an efficient fine-tuning stage called MoE-Tuning. The key technical contributions are: (i) a granularity decomposition mechanism that splits each pre-trained FFN into several smaller expert networks so that the sum of the expert outputs equals the original FFN output, (ii) a router initialization that initially selects the k experts coming from the same FFN, ensuring no performance drop at the start of fine-tuning, and (iii) a load-balancing auxiliary loss. The authors evaluate the resulting model on COCO, LVIS, and ODinW zero-shot benchmarks, and report that Dynamic-DINO, pre-trained on 1.56M open-source images, outperforms the official Grounding DINO 1.5 Edge that was pre-trained on the private Grounding20M dataset, with comparable inference speed. The paper also provides analyses of expert routing behavior, showing that shallow layers use diverse collaborations while deeper layers form stable partnerships, and that certain expert combinations specialize in particular visual patterns.

Significance. If the headline claim is sustained, the work is significant: it would show that a parameter-efficient MoE fine-tuning scheme enables a real-time open-vocabulary detector trained on a small public corpus to match or exceed a detector trained on roughly 13 times more private data. The granularity-decomposition initialization is elegant and exactly preserves the pre-trained function at initialization, which is a useful, non-circular design principle. The paper also contributes a reproducible training recipe on open datasets, which is appreciated in a field dominated by closed-source baselines. However, the significance is contingent on the validity of the comparison against the official Grounding DINO 1.5 Edge, which, as detailed below, is not yet established. The methodological core (MoE-Tuning with granularity decomposition) is interesting in its own right, and the statistical analysis of expert collaboration is a nice addition, but the central practical claim needs stronger evidence.

major comments (4)
  1. [Abstract and Section 1] The headline claim that Dynamic-DINO 'outperforms Grounding DINO 1.5 Edge, pretrained on the private Grounding20M dataset' is not supported by the reported numbers. In Table 1, at 640 resolution the advantages over the official Edge are +0.8 COCO AP, +0.1 LVIS-minival APall, and +0.1 LVIS-val APall; at 800 resolution they are +1.2 COCO AP, +0.0 LVIS-minival APall, and +0.3 LVIS-val APall. These margins are within normal run-to-run variation for detection training on such datasets, and the paper reports no multiple seeds, error bars, or significance tests. Furthermore, Section 5's Limitation Discussion states that 'with limited open-source data, our method matches the performance of official Grounding DINO 1.5 Edge,' which is a weaker and more honest claim than the abstract's 'outperforms.' The authors should either provide statistically grounded evidence for the superiority claim or temper the wording throughout the paper to 'matches or slightly exceeds' where margins are not significant.
  2. [Table 1 and Section 3.1] The comparison against the official Grounding DINO 1.5 Edge is confounded by the reproduction gap. The reproduced base model (Edge*) is substantially weaker than the official model on LVIS: at 640, LVIS-minival APall is 31.1 vs 33.5 (a 2.4 point deficit), and LVIS-val APall is 25.4 vs 27.3 (a 1.9 point deficit). Dynamic-DINO then largely erases that deficit. This raises the possibility that the apparent 'beating 20M data' effect is actually compensation for a systematically weaker reproduction, rather than a genuine property of MoE-Tuning. The authors need to rule this out, for example by releasing the reproduced base model and Dynamic-DINO, and by comparing both against the official model on additional benchmarks or on a controlled subset of LVIS where the reproduction failure is most visible.
  3. [Abstract, Figure 1, and Table 3] The claim of 'comparable inference speed' is contradicted by the reported measurements. Table 3 shows that Dynamic-DINO runs at 17.1 FPS (PyTorch) versus 21.7 FPS for the official Edge at 640 resolution, a 21% slowdown; at 800 resolution the gap is 15.1 vs 18.5 FPS, an 18% slowdown. Even compared with the reproduced base model (20.2 FPS at 640), the MOE model is 15% slower. While Section 4.2 acknowledges 'a minor decrease in inference speed' and attributes it to a sequential expert loop, the abstract and Figure 1 promise 'comparable inference speed.' The authors should either revise the wording to report the actual slowdown, or improve the implementation (e.g., parallel expert execution) so that the speed is indeed comparable.
  4. [Section 4.1 and Table 5] The data-efficiency claim is framed solely around the number of unique images (1.56M vs 20M), but the training procedure runs 7 epochs of pre-training plus 10 epochs of MoE-Tuning on the 1.56M corpus. This corresponds to roughly 26.5M sample presentations, a number that is closer to the official model's 20M unique images if the official model runs for a single epoch (the official schedule is not disclosed). To substantiate a genuine data-efficiency or compute-efficiency advantage, the authors should report total sample presentations, total GPU-hours, or total FLOPs for both the pre-training and fine-tuning stages, and compare those against available information for the official Edge. Without such a normalization, the phrase 'pretrained with merely 1.56M open-source data' overstates the efficiency gain.
minor comments (5)
  1. [Section 3.2, Eqs. (1)-(3)] The symbol k is used both for the granularity (number of partitions per FFN) and for the number of top experts selected in Eq. (2). This creates confusion, especially when the model is described as 'Dynamic-DINO×16-Top2,' where k=2 refers to top-k while k in Eq. (5) is the split factor. Please use different symbols, e.g., m for the split factor and t for top-k.
  2. [Section 4.2, Table 3] Table 3's speed comparison is informative, but the column header 'FPS-TensorRT FP32' is awkward. It likely means 'FPS under TensorRT with FP32 precision,' which could be stated more clearly. Also, the official Edge numbers are taken from the original paper; please specify the hardware and inference configuration for the official numbers, since the reproduced numbers are measured on a single A100.
  3. [Section 5 and Section 1] The paper contains an internal inconsistency between the abstract's 'outperforms' and Section 5's 'matches.' Please align the language. In addition, the contribution bullet in Section 1 says 'Our method surpasses Grounding DINO 1.5 Edge,' which again is stronger than the evidence in Table 1.
  4. [Supplementary, Algorithm 1] The pseudocode has a minor typo: 'ffd_dim' should likely be 'ffn_dim' or 'hidden_dim.' The logic itself is clear, but a consistent naming convention would help readers map the code to the mathematical notation in Section 3.3.
  5. [References] References [16] and [17] both point to the MDETR paper (Kamath et al.) but are cited for different dataset sources (GQA and Flickr30k). This duplication should be cleaned up, and the citations should be resolved to the original dataset papers instead of the MDETR paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: Dynamic-DINO's claimed gains are evaluated on external zero-shot benchmarks, and the initialization identity is an exact algebraic construction rather than a fitted prediction.

full rationale

The paper's derivation chain is self-contained and does not reduce to its inputs. The key initialization step is an exact algebraic identity: decomposing the pre-trained FFN into k experts with horizontally and vertically split weights and b2/k guarantees FFN(x) = sum_j E_j(x) (Eq. 9), and the replicated router initialization guarantees h(x) = FFN(x) at the start of MoE-Tuning. This is a construction, not a fitted constraint. The performance claims are then tested on COCO, LVIS, and ODinW under zero-shot conditions, with COCO images explicitly excluded from the GoldG training data, so no evaluation benchmark is used to fit the model. The comparison against Grounding DINO 1.5 Edge relies on the authors' own reproduction of the closed-source model, which raises a baseline-validity concern (whether the reproduction is a faithful stand-in) and a correctness/calibration concern, but it is not circularity: the reproduction is an independent checkpoint, not a quantity derived from the paper's own method. The self-citations in the paper (e.g., references to SP-Net and to the authors' LVLM MoE work) appear only in contextual related-work discussions and are not load-bearing for the central claim. The abstract's 'outperforms' versus Section 5's 'matches' wording is an internal inconsistency in claim strength, not a circular derivation. Overall, no step in the claimed derivation chain is equivalent to its inputs by definition or by fitted renaming.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

The central claim rests on the reproduction of a closed-source baseline and on the comparability of published benchmark numbers, represented by the two domain assumptions above. The method itself only repartitions existing FFN weights; it introduces no new physical entities or untested theoretical objects.

free parameters (3)
  • Number of FFN replicas N = 8 (16 experts at k=2)
    Sets the supernet width; ablations in Fig. 10 show larger N helps, and N=8 is used in the final model.
  • Granularity / activated experts k = 2
    Each FFN is split into 2 expert partitions and 2 experts are activated per token; Fig. 11 shows k=2 beats k=4, so it is the chosen operating point.
  • Auxiliary load-balancing coefficient alpha = 0.01
    A fixed small weight for the MoE auxiliary loss (Eq. 12); no sensitivity study is reported.
assumptions (3)
  • domain assumption The reproduced Grounding DINO 1.5 Edge trained on 1.56M open data is a faithful implementation of the official closed-source Edge architecture.
    The headline comparison in Table 1 assumes the reproduction is a valid baseline; if the reproduction differs from the official model in training recipe or implementation, the measured gains from MoE-Tuning may not transfer.
  • domain assumption Zero-shot evaluation protocols on COCO, LVIS, and ODinW match those used for the official Grounding DINO 1.5 Edge numbers.
    The comparison to the official model uses published numbers from a different paper; differences in evaluation settings (e.g., LVIS AP variant) would bias the conclusion.
  • standard math DETR-style losses (L1, GIOU, focal) and the load-balancing auxiliary loss are standard and appropriate.
    These losses are taken verbatim from cited DINO and MoE-LLaVA works; no new loss derivation is claimed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Dynamic-DINO: Fine-Grained Mixture of Experts Tuning for Real-time Open-Vocabulary Object Detection." pith.science (2026). https://pith.science/paper/2WYPRYD2

@misc{pith2026250717436,
  author       = {Pith},
  title        = {Pith review of: Dynamic-DINO: Fine-Grained Mixture of Experts Tuning for Real-time Open-Vocabulary Object Detection},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/2WYPRYD2}},
  note         = {Machine review of arXiv:2507.17436}
}
read the original abstract

The Mixture of Experts (MoE) architecture has excelled in Large Vision-Language Models (LVLMs), yet its potential in real-time open-vocabulary object detectors, which also leverage large-scale vision-language datasets but smaller models, remains unexplored. This work investigates this domain, revealing intriguing insights. In the shallow layers, experts tend to cooperate with diverse peers to expand the search space. While in the deeper layers, fixed collaborative structures emerge, where each expert maintains 2-3 fixed partners and distinct expert combinations are specialized in processing specific patterns. Concretely, we propose Dynamic-DINO, which extends Grounding DINO 1.5 Edge from a dense model to a dynamic inference framework via an efficient MoE-Tuning strategy. Additionally, we design a granularity decomposition mechanism to decompose the Feed-Forward Network (FFN) of base model into multiple smaller expert networks, expanding the subnet search space. To prevent performance degradation at the start of fine-tuning, we further propose a pre-trained weight allocation strategy for the experts, coupled with a specific router initialization. During inference, only the input-relevant experts are activated to form a compact subnet. Experiments show that, pretrained with merely 1.56M open-source data, Dynamic-DINO outperforms Grounding DINO 1.5 Edge, pretrained on the private Grounding20M dataset.

Figures

Figures reproduced from arXiv: 2507.17436 by the authors.

Figure 1
Figure 1. Dynamic-DINO is an efficient object-centric vision model designed for open-vocabulary object detection. Pretrained with merely [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of Dynamic-DINO. In previous transformer blocks, a single FFN handles diverse token patterns, causing gradient conflicts and long-tail issues. MoE-Tuning extends the dense model into a sparse dynamic inference framework, activating only relevant experts to form a compact subnet during inference. Experiments show that deeper layers develop stable expert collaboration, with special￾ized combinations for s… view at source ↗
Figure 3
Figure 3. MoE-Tuning framework. Dynamic-DINO builds upon the Grounding DINO 1.5 Edge [34], extending it from a dense model into a dynamic inference framework via the proposed MoE-Tuning strategy. 3.2. Cross-Modality MoE Decoder Supernet Expansion. Following MoE [8] paradigm, we scale up the model by expanding the FFN in each layer of the decoder into N FFNs of identical size. For each FFN, its intermediate hidden dimension is… view at source ↗
Figures from the paper (9 more)
Figure 5
Figure 5. Figure 5: Router initialization. This initialization ensures that, at the beginning of fine-tuning, the router invariably selects the k experts derived from the same FFN, enabling incremental per￾formance improvements over the base model, preventing abrupt performance degradatio…
Figure 6
Figure 6. Figure 6: Effect of MoE-Tuning. Based on specially designed expert and router initialization methods, MoE-Tuning ensures in￾cremental performance improvement. The results on COCO with 640 × 640 resolution demonstrate that MoE-Tuning provides sig￾nificant performance enhancements…
Figure 7
Figure 7. Figure 7: Expert collaboration. The normalized co-selection frequencies are quantified for all expert pairs on LVIS-minival [11] with Dynamic-DINO×16-Top2 model, which comprises 16 experts and activates 2 experts per inference. Expert 0, 3 Expert 1, 3 Expert 1, 7 Expert 3, 7 [P…
Figure 8
Figure 8. Figure 8: Token routing examples for COCO. Image examples of how patches are routed at the MoE layer in the last block of the decoder for the Dynamic-DINO×16-Top2 model. Distinct expert combinations are specialized in processing specific patterns. COCO LVIS minival [PITH_FULL_I…
Figure 9
Figure 9. Figure 9: Distribution of expert loadings. The workload among experts is quantified with Dynamic-DINO×8-Top2 model during inference on COCO-val and LVIS-minival benchmarks, where each color represents one expert. 4.3. Statistical Analysis Routing Distributions. In [PITH_FULL_IM…
Figure 11
Figure 11. Figure 11: Effect of expert granularity. The horizontal axis k denotes decoupling a FFN into k partitions and N = 8 is utilized. shown in [PITH_FULL_IMAGE:figures/full_fig_p008_11.png]
Figure 12
Figure 12. Figure 12: Comparison of visualization results for zero-shot inference on LVIS. We visualize the predictions of our pre-trained base model and Dynamic-DINO after MoE-Tuning. The failures are highlighted with a yellow circle. Thermostat Watering can Barrow Ironing board Pre -Trai…
Figure 13
Figure 13. Figure 13: Comparison of visualization results for zero-shot inference on rare classes of LVIS. We visualize the predictions of our pre-trained base model and Dynamic-DINO after MoE-Tuning. The failures are highlighted with a yellow circle. 3 [PITH_FULL_IMAGE:figures/full_fig_p…
Figure 14
Figure 14. Figure 14: Expert collaboration across 3 datasets. The normalized co-selection frequencies are quantified for all expert pairs with Dynamic-DINO×16-Top2 model, which comprises 16 experts and activates 2 experts per inference. 4 [PITH_FULL_IMAGE:figures/full_fig_p014_14.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

54 extracted references · 36 canonical work pages

  1. [1]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023. 2, 3

    arXiv 2023

  2. [2]

    Vlmo: Unified vision-language pre-training with mixture-of-modality-experts

    Hangbo Bao, Wenhui Wang, Li Dong, Qiang Liu, Owais Khan Mohammed, Kriti Aggarwal, Subhojit Som, Songhao Piao, and Furu Wei. Vlmo: Unified vision-language pre-training with mixture-of-modality-experts. NeurIPS, 35: 32897–32912, 2022. 3

    work page 2022

  3. [3]

    Llava-mole: Sparse mixture of lora experts for mitigating data con- flicts in instruction finetuning mllms

    Shaoxiang Chen, Zequn Jie, and Lin Ma. Llava-mole: Sparse mixture of lora experts for mitigating data con- flicts in instruction finetuning mllms. arXiv preprint arXiv:2401.16160, 2024. 3

    arXiv 2024

  4. [4]

    Lifelong language pretraining with distribution-specialized experts

    Wuyang Chen, Yanqi Zhou, Nan Du, Yanping Huang, James Laudon, Zhifeng Chen, and Claire Cui. Lifelong language pretraining with distribution-specialized experts. In ICML, pages 5383–5395, 2023. 3

    work page 2023

  5. [5]

    Yolo-world: Real-time open- vocabulary object detection

    Tianheng Cheng, Lin Song, Yixiao Ge, Wenyu Liu, Xing- gang Wang, and Ying Shan. Yolo-world: Real-time open- vocabulary object detection. In CVPR, pages 16901–16911,

    work page

  6. [6]

    Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language models

    Damai Dai, Chengqi Deng, Chenggang Zhao, RX Xu, Huazuo Gao, Deli Chen, Jiashi Li, Wangding Zeng, Xingkai Yu, Yu Wu, et al. Deepseekmoe: Towards ultimate expert specialization in mixture-of-experts language models. arXiv preprint arXiv:2401.06066, 2024. 2, 3

    arXiv 2024

  7. [7]

    Evaluating large-vocabulary ob- ject detectors: The devil is in the details

    Achal Dave, Piotr Doll ´ar, Deva Ramanan, Alexander Kir- illov, and Ross Girshick. Evaluating large-vocabulary ob- ject detectors: The devil is in the details. arXiv preprint arXiv:2102.01066, 2021. 6

    arXiv 2021

  8. [8]

    Switch transformers: Scaling to trillion parameter models with sim- ple and efficient sparsity

    William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with sim- ple and efficient sparsity. Journal of Machine Learning Re- search, 23(120):1–39, 2022. 4, 1

    work page 2022

Show all 54 references
  1. [9]

    Mixture of cluster-conditional lora experts for vision-language instruction tuning

    Yunhao Gou, Zhili Liu, Kai Chen, Lanqing Hong, Hang Xu, Aoxue Li, Dit-Yan Yeung, James T Kwok, and Yu Zhang. Mixture of cluster-conditional lora experts for vision-language instruction tuning. arXiv preprint arXiv:2312.12379, 2023. 3

    2023 arXiv

  2. [10]

    Open-vocabulary object detection via vision and language knowledge distillation

    Xiuye Gu, Tsung-Yi Lin, Weicheng Kuo, and Yin Cui. Open-vocabulary object detection via vision and language knowledge distillation. In ICLR, 2022. 3

    2022

  3. [11]

    Lvis: A dataset for large vocabulary instance segmentation

    Agrim Gupta, Piotr Dollar, and Ross Girshick. Lvis: A dataset for large vocabulary instance segmentation. InCVPR,

  4. [12]

    Demix layers: Disentangling domains for modular language modeling

    Suchin Gururangan, Mike Lewis, Ari Holtzman, Noah A Smith, and Luke Zettlemoyer. Demix layers: Disentangling domains for modular language modeling. arXiv preprint arXiv:2108.05036, 2021. 3

    2021 arXiv

  5. [13]

    Feedback-aware anomaly detection through logs for large- scale software systems

    Jing Han, Tong Jia, Yifan Wu, Chuanjia Hou, and Ying Li. Feedback-aware anomaly detection through logs for large- scale software systems. ZTE Communications , 19(3):88,

  6. [14]

    Dynamic neural networks: A sur- vey

    Yizeng Han, Gao Huang, Shiji Song, Le Yang, Honghui Wang, and Yulin Wang. Dynamic neural networks: A sur- vey. IEEE TPAMI, 44(11):7436–7456, 2021. 3

    2021

  7. [15]

    T-rex2: Towards generic object detec- tion via text-visual prompt synergy

    Qing Jiang, Feng Li, Zhaoyang Zeng, Tianhe Ren, Shilong Liu, and Lei Zhang. T-rex2: Towards generic object detec- tion via text-visual prompt synergy. In ECCV, pages 38–57,

  8. [16]

    Mdetr - mod- ulated detection for end-to-end multi-modal understanding

    Aishwarya Kamath, Mannat Singh, Yann LeCun, Gabriel Synnaeve, Ishan Misra, and Nicolas Carion. Mdetr - mod- ulated detection for end-to-end multi-modal understanding. In ICCV, pages 1780–1790, 2021. 6, 1

    2021

  9. [17]

    Mdetr - mod- ulated detection for end-to-end multi-modal understanding

    Aishwarya Kamath, Mannat Singh, Yann LeCun, Gabriel Synnaeve, Ishan Misra, and Nicolas Carion. Mdetr - mod- ulated detection for end-to-end multi-modal understanding. In ICCV, pages 1780–1790, 2021. 2, 6

    2021

  10. [18]

    Gshard: Scaling giant models with conditional computation and automatic sharding

    Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. Gshard: Scaling giant models with conditional computation and automatic sharding. arXiv preprint arXiv:2006.16668, 2020. 2, 3

    2006 arXiv

  11. [19]

    Grounded language-image pre-training

    Liunian Harold Li, Pengchuan Zhang, Haotian Zhang, Jian- wei Yang, Chunyuan Li, Yiwu Zhong, Lijuan Wang, Lu Yuan, Lei Zhang, Jenq-Neng Hwang, Kai-Wei Chang, and Jianfeng Gao. Grounded language-image pre-training. In CVPR, pages 10965–10975, 2022. 2, 3, 6

    2022

  12. [20]

    Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning

    Victor Weixin Liang, Yuhui Zhang, Yongchan Kwon, Serena Yeung, and James Y Zou. Mind the gap: Understanding the modality gap in multi-modal contrastive representation learning. NeurIPS, 35:17612–17625, 2022. 3

    2022

  13. [21]

    Moe-llava: Mixture of experts for large vision- language models

    Bin Lin, Zhenyu Tang, Yang Ye, Jiaxi Cui, Bin Zhu, Peng Jin, Jinfa Huang, Junwu Zhang, Yatian Pang, Munan Ning, et al. Moe-llava: Mixture of experts for large vision- language models. arXiv preprint arXiv:2401.15947 , 2024. 2, 3, 5

    2024 arXiv

  14. [22]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In ECCV, pages 740–755, 2014. 2, 6

    2014

  15. [23]

    Yolo-uniow: Effi- cient universal open-world object detection

    Lihao Liu, Juexiao Feng, Hui Chen, Ao Wang, Lin Song, Jungong Han, and Guiguang Ding. Yolo-uniow: Effi- cient universal open-world object detection. arXiv preprint arXiv:2412.20645, 2024. 2, 3

    2024 arXiv

  16. [24]

    Grounding dino: Marrying dino with grounded pre-training for open-set object detection

    Shilong Liu, Zhaoyang Zeng, Tianhe Ren, Feng Li, Hao Zhang, Jie Yang, Qing Jiang, Chunyuan Li, Jianwei Yang, Hang Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In ECCV, pages 38–55, 2024. 3, 6

    2024

  17. [25]

    Scene visual per- ception and ar navigation applications

    Ping LU, Bin SHENG, and Wenzhe SHI. Scene visual per- ception and ar navigation applications. ZTE communica- tions, 21(1):81, 2023. 2 9

    2023

  18. [26]

    Simple open-vocabulary object detection

    Matthias Minderer, Alexey Gritsenko, Austin Stone, Maxim Neumann, Dirk Weissenborn, Alexey Dosovitskiy, Aravindh Mahendran, Anurag Arnab, Mostafa Dehghani, Zhuoran Shen, et al. Simple open-vocabulary object detection. In ECCV, pages 728–755, 2022. 3

    2022

  19. [27]

    Scaling open-vocabulary object detection

    Matthias Minderer, Alexey Gritsenko, and Neil Houlsby. Scaling open-vocabulary object detection. NeurIPS, 36: 72983–73007, 2023. 2, 3

    2023

  20. [28]

    Plummer, Liwei Wang, Chris M

    Bryan A. Plummer, Liwei Wang, Chris M. Cervantes, Juan C. Caicedo, Julia Hockenmaier, and Svetlana Lazeb- nik. Flickr30k entities: Collecting region-to-phrase corre- spondences for richer image-to-sentence models. In ICCV,

  21. [29]

    Yolo9000: Better, faster, stronger

    Joseph Redmon and Ali Farhadi. Yolo9000: Better, faster, stronger. In CVPR, 2017. 3

    2017

  22. [30]

    Yolov3: An incremental improvement

    Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767, 2018

    2018 arXiv

  23. [31]

    You only look once: Unified, real-time object de- tection

    Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object de- tection. In CVPR, 2016. 3

    2016

  24. [32]

    Faster r-cnn: Towards real-time object detection with region proposal networks

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. NeurIPS, 28, 2015. 2

    2015

  25. [33]

    Dino-x: A unified vision model for open- world object detection and understanding

    Tianhe Ren, Yihao Chen, Qing Jiang, Zhaoyang Zeng, Yuda Xiong, Wenlong Liu, Zhengyu Ma, Junyi Shen, Yuan Gao, Xiaoke Jiang, et al. Dino-x: A unified vision model for open- world object detection and understanding. arXiv preprint arXiv:2411.14347, 2024. 2, 3

    2024 arXiv

  26. [34]

    Grounding dino 1.5: Advance the” edge” of open-set object detection

    Tianhe Ren, Qing Jiang, Shilong Liu, Zhaoyang Zeng, Wen- long Liu, Han Gao, Hongjie Huang, Zhengyu Ma, Xiaoke Jiang, Yihao Chen, et al. Grounding dino 1.5: Advance the” edge” of open-set object detection. arXiv preprint arXiv:2405.10300, 2024. 2, 3, 4, 6

    2024 arXiv

  27. [35]

    Objects365: A large-scale, high-quality dataset for object detection

    Shuai Shao, Zeming Li, Tianyuan Zhang, Chao Peng, Gang Yu, Xiangyu Zhang, Jing Li, and Jian Sun. Objects365: A large-scale, high-quality dataset for object detection. In ICCV, 2019. 2, 6, 1

    2019

  28. [36]

    Outra- geously large neural networks: The sparsely-gated mixture- of-experts layer

    Noam Shazeer, Azalia Mirhoseini, Krzysztof Maziarz, Andy Davis, Quoc Le, Geoffrey Hinton, and Jeff Dean. Outra- geously large neural networks: The sparsely-gated mixture- of-experts layer. arXiv preprint arXiv:1701.06538, 2017. 3

    2017 arXiv

  29. [37]

    Scaling vision-language models with sparse mixture of experts

    Sheng Shen, Zhewei Yao, Chunyuan Li, Trevor Darrell, Kurt Keutzer, and Yuxiong He. Scaling vision-language models with sparse mixture of experts. arXiv preprint arXiv:2303.07226, 2023. 3

    2023 arXiv

  30. [38]

    Convolutional networks with adaptive inference graphs

    Andreas Veit and Serge Belongie. Convolutional networks with adaptive inference graphs. In ECCV, pages 3–18, 2018. 3

    2018

  31. [39]

    Sp-net: slowly progress- ing dynamic inference networks

    Huanyu Wang, Wenhu Zhang, Shihao Su, Hui Wang, Zhen- wei Miao, Xin Zhan, and Xi Li. Sp-net: slowly progress- ing dynamic inference networks. In ECCV, pages 223–240,

  32. [40]

    Ov-dino: Unified open-vocabulary de- tection with language-aware selective fusion

    Hao Wang, Pengzhen Ren, Zequn Jie, Xiao Dong, Chengjian Feng, Yinlong Qian, Lin Ma, Dongmei Jiang, Yaowei Wang, Xiangyuan Lan, et al. Ov-dino: Unified open-vocabulary de- tection with language-aware selective fusion. arXiv preprint arXiv:2407.07844, 2024. 3

    2024 arXiv

  33. [41]

    V3det: Vast vocabulary visual detection dataset

    Jiaqi Wang, Pan Zhang, Tao Chu, Yuhang Cao, Yujie Zhou, Tong Wu, Bin Wang, Conghui He, and Dahua Lin. V3det: Vast vocabulary visual detection dataset. In ICCV, pages 19844–19854, 2023. 2, 6, 1

    2023

  34. [42]

    Image as a foreign language: Beit pretraining for vision and vision- language tasks

    Wenhui Wang, Hangbo Bao, Li Dong, Johan Bjorck, Zhil- iang Peng, Qiang Liu, Kriti Aggarwal, Owais Khan Mo- hammed, Saksham Singhal, Subhojit Som, et al. Image as a foreign language: Beit pretraining for vision and vision- language tasks. In CVPR, pages 19175–19186, 2023. 3

    2023

  35. [43]

    Skipnet: Learning dynamic routing in convolutional networks

    Xin Wang, Fisher Yu, Zi-Yi Dou, Trevor Darrell, and Joseph E Gonzalez. Skipnet: Learning dynamic routing in convolutional networks. In ECCV, pages 409–424, 2018. 3

    2018

  36. [44]

    Ovlw-detr: Open-vocabulary light-weighted detection trans- former

    Yu Wang, Xiangbo Su, Qiang Chen, Xinyu Zhang, Teng Xi, Kun Yao, Errui Ding, Gang Zhang, and Jingdong Wang. Ovlw-detr: Open-vocabulary light-weighted detection trans- former. arXiv preprint arXiv:2407.10655, 2024. 2, 3, 6

    2024 arXiv

  37. [45]

    Detecting ev- erything in the open world: Towards universal object detec- tion

    Zhenyu Wang, Yali Li, Xi Chen, Ser-Nam Lim, Antonio Tor- ralba, Hengshuang Zhao, and Shengjin Wang. Detecting ev- erything in the open world: Towards universal object detec- tion. In CVPR, pages 11433–11443, 2023. 2, 3

    2023

  38. [46]

    Solving token gradient conflict in mixture-of-experts for large vision-language model

    Longrong Yang, Dong Shen, Chaoxiang Cai, Fan Yang, Size Li, Di Zhang, and Xi Li. Solving token gradient conflict in mixture-of-experts for large vision-language model. In ICLR, 2025. 2

    2025

  39. [48]

    Detclip: Dictionary-enriched visual-concept paralleled pre- training for open-world detection

    Lewei Yao, Jianhua Han, Youpeng Wen, Xiaodan Liang, Dan Xu, Wei Zhang, Zhenguo Li, Chunjing Xu, and Hang Xu. Detclip: Dictionary-enriched visual-concept paralleled pre- training for open-world detection. NeurIPS, 35:9125–9138,

  40. [49]

    Detclipv2: Scal- able open-vocabulary object detection pre-training via word- region alignment

    Lewei Yao, Jianhua Han, Xiaodan Liang, Dan Xu, Wei Zhang, Zhenguo Li, and Hang Xu. Detclipv2: Scal- able open-vocabulary object detection pre-training via word- region alignment. In CVPR, pages 23497–23506, 2023

    2023

  41. [50]

    Detclipv3: To- wards versatile generative open-vocabulary object detection

    Lewei Yao, Renjie Pi, Jianhua Han, Xiaodan Liang, Hang Xu, Wei Zhang, Zhenguo Li, and Dan Xu. Detclipv3: To- wards versatile generative open-vocabulary object detection. In CVPR, pages 27391–27401, 2024. 3

    2024

  42. [51]

    Open-vocabulary object detection using captions

    Alireza Zareian, Kevin Dela Rosa, Derek Hao Hu, and Shih- Fu Chang. Open-vocabulary object detection using captions. In CVPR, pages 14393–14402, 2021. 3

    2021

  43. [52]

    Dino: Detr with improved denoising anchor boxes for end-to-end object detection

    Hao Zhang, Feng Li, Shilong Liu, Lei Zhang, Hang Su, Jun Zhu, Lionel M Ni, and Heung-Yeung Shum. Dino: Detr with improved denoising anchor boxes for end-to-end object detection. arXiv preprint arXiv:2203.03605, 2022. 5

    2022 arXiv

  44. [53]

    A simple framework for open-vocabulary segmentation and detection

    Hao Zhang, Feng Li, Xueyan Zou, Shilong Liu, Chunyuan Li, Jianwei Yang, and Lei Zhang. A simple framework for open-vocabulary segmentation and detection. In ICCV, pages 1020–1031, 2023. 2, 3

    2023

  45. [54]

    "" Input: n: int k: int ffn: nn.Module

    Tiancheng Zhao, Peng Liu, Xuan He, Lu Zhang, and Kyu- song Lee. Real-time transformer-based open-vocabulary detection with efficient fusion head. arXiv preprint arXiv:2403.06892, 2024. 2, 3, 6 10 Dynamic-DINO: Fine-Grained Mixture of Experts Tuning for Real-time Open-Vocabular...

    2024 arXiv

  46. [640]

    Parameters

    “Parameters” represents active parameters / total parameters. Dynamic-DINO×N-Top2 indicates a model with N experts, where 2 experts are activated per inference. Method Parameters COCO-val LVIS-minival LVIS-val G-DINO 1.5 Edge 178M/178M 42.6 31.1 25.4 Dynamic-DINO×4-Top2 178M/1...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.