Pith. sign in

REVIEW 2 major objections 5 minor 55 references

A Fast and Accurate One-Stage Approach to Visual Grounding

T0 review · 2 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A one-stage detector that fuses text into YOLOv3 outperforms propose-and-rank grounding while running at 38 ms per query.

desk verdict A strong empirical case for one-stage grounding that overreaches in its headline claim; send to review but require a qualified accuracy statement. read the letter →

arxiv 1908.06354 v1 pith:BI77CXNF submitted 2019-08-18 cs.CV

Zhengyuan Yang , Boqing Gong , Liwei Wang , Wenbing Huang , Dong Yu , Jiebo Luo This is my paper · ORCID
classification cs.CV
keywords visualgroundingphraselocalizationreferringexpressioncomprehensionone-stageobjectdetectionYOLOv3regionproposalstext-visualfusionspatialfeatures
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that the standard way of doing visual grounding—first generate candidate image regions, then rank them against a text query—is the wrong default. It argues that two-stage accuracy is capped by the quality of the first-stage proposals, and supplies a direct measurement: on ReferItGame, 200 EdgeBox proposals hit only 68% of ground-truth regions at IoU ≥ 0.5. Its proposed one-stage model fuses a language embedding into the YOLOv3 detector, augments the feature maps with spatial coordinates, and replaces each anchor's sigmoid confidence with a softmax over all anchor boxes so that exactly one box is produced per query. On Flickr30K Entities and ReferItGame this model is both more accurate than strong two-stage baselines and about ten times faster, which the authors offer as evidence that visual grounding should shift to the one-stage paradigm.

What carries the argument

The load-bearing mechanism is a dense anchor-based detector converted into a grounded box predictor. Visual features from Darknet-53 with a feature pyramid at three scales (8×8, 16×16, 32×32) are normalized and concatenated per location with a 768-dimensional BERT query embedding and an 8-dimensional spatial coordinate vector; a 1×1 convolution fuses these 1,032-dimensional features. The grounding head regresses box shifts from 4,032 anchor boxes sized by K-means clustering on the training ground truths, and a softmax over all anchors with a one-hot label on the highest-IoU anchor replaces YOLOv3's independent sigmoid confidences. This removes the proposal stage entirely and lets the detection loss train the whole network end to end.

What would settle it

Train a two-stage grounding system whose second stage includes a box-refinement head (for example, a QRC Net-style fine-tuned RPN) on ReferItGame with the same BERT query encoder and Darknet backbone as the one-stage model. If that system reaches or exceeds 59.30% accuracy at IoU ≥ 0.5, the claim that first-stage proposals impose an unreachable ceiling is false for refined two-stage methods; if it stays well below, the cap is confirmed.

Watch

Extended reading notes

Core claim

The central claim is that visual grounding is better formulated as a single end-to-end prediction—image plus text query in, one bounding box out—rather than as propose-and-rank. The authors implement this by injecting a text embedding into YOLOv3's feature pyramid, concatenating per-location spatial features, and replacing the sigmoid confidence with a softmax across all 4,032 anchor boxes, so the network is forced to choose one region. On Flickr30K Entities the model reaches 68.69% accuracy at IoU ≥ 0.5 versus 61.89% for the strongest comparable two-stage method, and on ReferItGame it reaches 59.30% versus 35.07%, while taking about 38 ms per query compared with over 180 ms for two-stage systems. The paper interprets this as direct evidence that the proposal-stage ceiling, not the ranking stage, has been limiting visual grounding.

Load-bearing premise

The argument's load-bearing premise is that a two-stage system can only choose among its first-stage boxes, so any ground-truth region not covered by those boxes is unreachable; if a two-stage method can regress or refine boxes after ranking, this ceiling is not absolute.

Editorial extensions

If this is right

  • If the one-stage formulation is adopted, visual grounding systems no longer need a separate proposal generator, removing a major source of computation and failure.
  • End-to-end training lets the network learn to box stuff regions and multi-object groups that objectness-based proposals systematically miss.
  • The same YOLOv3-plus-text fusion can serve as a strong baseline for phrase localization and referring expression comprehension, and for related grounded tasks.
  • Because inference stays under 40 ms per query, grounding becomes practical for interactive and real-time applications.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper's softmax-over-anchors is a discrete choice mechanism; a natural testable extension is whether a differentiable region-refinement head on top of the one-stage output would close the remaining gap on attribute-sensitive queries the paper reports as failures.
  • The cap argument predicts that two-stage methods will approach one-stage accuracy exactly when proposals are nearly perfect, as on RefCOCO; this suggests dataset proposal quality, rather than ranking quality, explains most cross-dataset accuracy differences.
  • One untested consequence is that the one-stage model could be extended to multiple grounding queries per image by running the same network per query, trading some speed for joint context; the authors only add a triplet regularization for cross-query relationships.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper proposes a one-stage visual grounding model that fuses a language embedding into YOLOv3, augments feature maps with spatial coordinates, replaces the per-anchor sigmoid confidence with a softmax over all anchors, and trains end-to-end with a cross-entropy objective. On Flickr30K Entities and ReferItGame it reports large accuracy gains over two-stage propose-and-rank baselines and an approximately 10x speedup. An oracle analysis of region-proposal hit rates is used to argue that two-stage performance is capped by first-stage proposals, motivating a call for a paradigm shift to one-stage grounding. RefCOCO results appear in the appendix, where the proposed method is competitive but not uniformly superior to MAttNet.

Significance. If the results hold, the paper makes a valuable contribution: a simple, fast, end-to-end one-stage model that establishes a strong baseline for phrase localization and referring expression comprehension. The controlled comparisons (Similarity Net-Darknet, Similarity Net-Resnet, CITE-Resnet) are a definite strength, since they isolate the effect of the framework change from the choice of backbone. The paper also provides a useful empirical study of proposal hit rates. However, the unqualified accuracy claim in the abstract is contradicted by the paper's own RefCOCO results, and the paradigm-shift argument overgeneralizes because two-stage methods with proposal refinement (QRC Net, MAttNet) are not bound by the fixed-proposal ceiling. These issues are central to the paper's headline claims and require revision, but the core method and main experimental results remain of interest.

major comments (2)
  1. [Abstract; Section 1; Appendix A, Table 4] The abstract and Section 1 state that the proposed method is "more accurate" than state-of-the-art two-stage methods, but Appendix A, Table 4 reports MAttNet achieving 76.40 and 80.43 on RefCOCO val and testA, compared with 73.66 and 75.78 for Ours-LSTM. The accuracy advantage is therefore dataset-dependent, and the unqualified claim is internally inconsistent with the manuscript's own results. I recommend rewording the abstract, introduction, and conclusion to state that the one-stage method is more accurate on Flickr30K Entities and ReferItGame, and competitive with but not uniformly better than the best two-stage method on RefCOCO.
  2. [Section 1; Section 3.4; Appendix A] The motivation in Section 1 and the oracle analysis in Section 3.4 assert that two-stage methods are "capped" by the first-stage region proposals. This is true only for methods that rank fixed proposals without any refinement. The paper itself reports in Appendix A that QRC Net, which fine-tunes an RPN end-to-end, improves two-stage accuracy from 34.54 to 44.07 on ReferItGame and from 60.89 to 65.14 on Flickr30K Entities, narrowing the gap to the one-stage method. The paradigm-shift claim should therefore be conditioned on proposal quality rather than presented as an unconditional flaw of two-stage methods; the paper's own RefCOCO analysis (Tables 4 and 5) demonstrates that the advantage shrinks when proposals are good.
minor comments (5)
  1. [Section 2.2] The description of the training loss is incomplete: the text first says the network is trained with YOLO's loss, then states that the confidence term is replaced by a softmax cross-entropy, but the relative weighting of the classification and regression terms and the application of the regression loss (to the selected positive anchor only or to all anchors) are not specified. Please provide the exact loss formulation or a precise pointer to the YOLOv3 loss with the stated modifications.
  2. [Tables 1 and 2] The "about 10 times faster" claim is not supported by the main model variant: Ours-Bert runs at 38 ms versus 184 ms for Similarity Net-Resnet, which is about a 4.8x speedup; the 10x figure only holds for Ours-FV at 16 ms. Please qualify the speed claim to reflect the variation across language encoders.
  3. [Section 1] The sentence claiming the method "does not depend on any additional tools or pre-trained models" is contradicted by the use of COCO-pretrained Darknet-53 and pretrained BERT reported in Section 3.1. Please rephrase to say the method avoids extra region-proposal tools rather than avoiding all pre-trained models.
  4. [Table 3] The hit rates for "Ours" are obtained from the model's own top-200 predictions, and since this model is trained on the target dataset, the comparison with unsupervised proposal generators is not apples-to-apples. The table still supports the benefit of end-to-end learning, but it should not be used as independent evidence for a proposal-quality ceiling on two-stage methods.
  5. [Tables 1-4] No error bars or multiple-seed results are reported. This is not critical for the large margins on Flickr30K Entities and ReferItGame, but for the RefCOCO comparison, where differences are small (e.g., 73.66 vs. 72.72 on val), the lack of variance information makes it hard to assess the significance of the gap to MAttNet.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the central accuracy and speed claims rest on held-out test-set comparisons against external baselines, not on fitted inputs or self-citation chains.

full rationale

The paper makes no first-principles derivation whose output is equivalent to its input. Its central claims are empirical: accuracy is measured on held-out test splits of Flickr30K Entities, ReferItGame, and RefCOCO, with IoU@0.5 against ground-truth boxes, and speed is measured by wall-clock inference time. There is no fitted parameter that is later renamed as a prediction. The closest candidate is the hit-rate table, where the model's own top-200 box predictions are compared with unsupervised proposals ('Ours' row in Table 3), but that is a fair empirical observation about the trained model, not a derivation that presupposes the conclusion. The K-means anchor customization is fitted on the training set and then evaluated on test splits, which is standard supervised practice rather than circularity. The RefCOCO appendix shows MAttNet outperforming Ours-LSTM on val/testA, but that bears on the strength of the unqualified 'more accurate' claim, not on any circular reduction. The comparisons to Similarity Net [42] and CITE [29] use co-authored prior work only as baselines; those results are independently published and are not used to justify the paper's own predictions. No uniqueness theorem, ansatz-by-citation, or definitional identity is invoked. The paper is therefore self-contained against external benchmarks, and the honest finding is no significant circularity.

Assumptions & free parameters 7 free parameters · 5 assumptions · 0 invented entities

No new physical entities. The method introduces design choices and learned parameters, catalogued above; the central claim is empirical and rests on benchmark evaluations plus the ranking-cap argument.

free parameters (7)
  • Neural network weights (Darknet-53, fusion convolutions, grounding head, and optional LSTM embedding) = Trained on each dataset's training split (Flickr30K Entities, ReferItGame, RefCOCO) via RMSProp with lr 1e-4 and…
    All parameters are learned from the training set using the grounding loss; this is standard supervised fitting, not a derivation.
  • Anchor priors (9 per dataset) = Flickr30K: (17,16), (33,35), (84,43), (50,74), (76,126), (125,81), (128,161), (227,104), (216,180); ReferItGame…
    K-means clustering over training-set ground-truth boxes with (1-IoU) distance; box regression predicts offsets from these anchors.
  • Spatial coordinate features = 8D vector per grid cell: (i/W, j/H, (i+0.5)/W, (j+0.5)/H, (i+1)/W, (j+1)/H, 1/W, 1/H)
    Hand-designed encoding; ablation shows about 1.6% accuracy gain on Flickr30K Entities.
  • Language embedding choice and dimensions = BERT 768D (frozen), bi-LSTM 512D hidden, or Fisher vector 6000D; two FC layers of 512D
    Design choices; results vary slightly across encoders.
  • Softmax vs sigmoid output = softmax over 4032 anchors (3 anchors x 1344 locations)
    Chosen to enforce a single output box; central to the grounding module.
  • Input image size = 256x256 (long edge resized to 256, padded)
    Smaller than typical two-stage detectors; contributes to speed and may affect accuracy.
  • Triplet loss margin and weight (appendix) = m=1, w_reg=1
    Chosen for cross-sample regularization; small improvements reported.
assumptions (5)
  • domain assumption Visual grounding is correctly evaluated by Acc@0.5 IoU against human-annotated boxes.
    The paper adopts the standard protocol from prior work; if the metric is not the right one, all comparisons inherit that bias.
  • domain assumption A pre-trained Darknet-53/COCO backbone transfers enough visual knowledge to grounding datasets.
    The visual encoder is not trained from scratch; its features are assumed useful for phrase localization and referring expression comprehension.
  • domain assumption The output can be reduced to a single bounding box per query.
    Softmax over all anchors assumes exactly one correct region; queries that refer to multiple disconnected objects are still represented by one box.
  • domain assumption The two-stage baselines implemented by the authors are faithful representatives of the two-stage framework.
    They modify code from [29] and use Edgebox N=200; if the baselines were tuned unfairly, the accuracy gap would be overstated.
  • domain assumption Text queries can be encoded with BERT, LSTM, or Fisher vectors without task-specific pretraining.
    The model concatenates the query embedding with visual/spatial features; the quality of this fusion depends on the embedding.

how reviews work

0 comments
Cite this review

Pith. "Pith review of A Fast and Accurate One-Stage Approach to Visual Grounding." pith.science (2026). https://pith.science/paper/BI77CXNF

@misc{pith2026190806354,
  author       = {Pith},
  title        = {Pith review of: A Fast and Accurate One-Stage Approach to Visual Grounding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/BI77CXNF}},
  note         = {Machine review of arXiv:1908.06354}
}
read the original abstract

We propose a simple, fast, and accurate one-stage approach to visual grounding, inspired by the following insight. The performances of existing propose-and-rank two-stage methods are capped by the quality of the region candidates they propose in the first stage --- if none of the candidates could cover the ground truth region, there is no hope in the second stage to rank the right region to the top. To avoid this caveat, we propose a one-stage model that enables end-to-end joint optimization. The main idea is as straightforward as fusing a text query's embedding into the YOLOv3 object detector, augmented by spatial features so as to account for spatial mentions in the query. Despite being simple, this one-stage approach shows great potential in terms of both accuracy and speed for both phrase localization and referring expression comprehension, according to our experiments. Given these results along with careful investigations into some popular region proposals, we advocate for visual grounding a paradigm shift from the conventional two-stage methods to the one-stage framework.

Figures

Figures reproduced from arXiv: 1908.06354 by the authors.

Figure 1
Figure 1. Visual grounding is the task of localizing a language [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. The proposed end-to-end one-stage visual grounding framework. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Failure cases of the Edgebox region candidates (boxes [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Mistakes made by the two-stage similarity network (top row) that can be corrected by our one-stage approach (bottom row). Blue [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Success cases on challenging instances (top row) and common failures (bottom row) of our one-stage method. Blue / yellow [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 47 canonical work pages

  1. [1]

    Bottom-up and top-down attention for image captioning and visual question answering

    Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. Bottom-up and top-down attention for image captioning and visual question answering. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, pages 6077–6086, 2018. 1

    work page 2018

  2. [2]

    Msrc: Multimodal spatial regression with semantic context for phrase grounding

    Kan Chen, Rama Kovvuri, Jiyang Gao, and Ram Nevatia. Msrc: Multimodal spatial regression with semantic context for phrase grounding. InProceedings of the 2017 ACM on In- ternational Conference on Multimedia Retrieval , pages 23–

    work page 2017

  3. [3]

    Query-guided regression network with context policy for phrase ground- ing

    Kan Chen, Rama Kovvuri, and Ram Nevatia. Query-guided regression network with context policy for phrase ground- ing. In Proceedings of the IEEE International Conference on Computer Vision, pages 824–832, 2017. 12, 13

    work page 2017

  4. [4]

    Bert: Pre-training of deep bidirectional transformers for language understanding

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. 2, 3, 4, 5

    arXiv 2018

  5. [5]

    Neural sequential phrase grounding (seqground)

    Pelin Dogan, Leonid Sigal, and Markus Gross. Neural sequential phrase grounding (seqground). In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4175–4184, 2019. 12, 13

    work page 2019

  6. [6]

    The segmented and annotated iapr tc-12 benchmark

    Hugo Jair Escalante, Carlos A Hern´andez, Jesus A Gonzalez, Aurelio L´opez-L´opez, Manuel Montes, Eduardo F Morales, L Enrique Sucar, Luis Villase ˜nor, and Michael Grubinger. The segmented and annotated iapr tc-12 benchmark. Com- puter Vision and Image Understanding , 114(4):419–428,

    work page

  7. [7]

    The pascal visual object classes (voc) challenge

    Mark Everingham, Luc Van Gool, Christopher KI Williams, John Winn, and Andrew Zisserman. The pascal visual object classes (voc) challenge. International journal of computer vision, 88(2):303–338, 2010. 2

    work page 2010

  8. [8]

    Unsupervised image captioning

    Yang Feng, Lin Ma, Wei Liu, and Jiebo Luo. Unsupervised image captioning. In Proceedings of the IEEE conference on computer vision and pattern recognition, 2019. 1

    work page 2019

Show all 55 references
  1. [9]

    Vqs: Linking segmentations to questions and answers for supervised attention in vqa and question-focused semantic segmentation

    Chuang Gan, Yandong Li, Haoxiang Li, Chen Sun, and Bo- qing Gong. Vqs: Linking segmentations to questions and answers for supervised attention in vqa and question-focused semantic segmentation. In Proceedings of the IEEE Inter- national Conference on Computer Vision, pages 1811–1820,

  2. [10]

    Fast r-cnn

    Ross Girshick. Fast r-cnn. In Proceedings of the IEEE inter- national conference on computer vision , pages 1440–1448,

  3. [11]

    Mask r-cnn

    Kaiming He, Georgia Gkioxari, Piotr Doll ´ar, and Ross Gir- shick. Mask r-cnn. In Proceedings of the IEEE international conference on computer vision, pages 2961–2969, 2017. 2, 5, 7

    2017

  4. [12]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 5

    2016

  5. [13]

    Seg- mentation from natural language expressions

    Ronghang Hu, Marcus Rohrbach, and Trevor Darrell. Seg- mentation from natural language expressions. In European Conference on Computer Vision , pages 108–124. Springer,

  6. [14]

    Natural language object re- trieval

    Ronghang Hu, Huazhe Xu, Marcus Rohrbach, Jiashi Feng, Kate Saenko, and Trevor Darrell. Natural language object re- trieval. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4555–4564, 2016. 1, 4, 5, 6

    2016

  7. [15]

    Referitgame: Referring to objects in pho- tographs of natural scenes

    Sahar Kazemzadeh, Vicente Ordonez, Mark Matten, and Tamara Berg. Referitgame: Referring to objects in pho- tographs of natural scenes. In Proceedings of the 2014 con- ference on empirical methods in natural language processing (EMNLP), pages 787–798, 2014. 1, 2, 4, 5, 6, 12, 13

    2014

  8. [16]

    Deep attribute-preserving metric learning for natural language object retrieval

    Jianan Li, Yunchao Wei, Xiaodan Liang, Fang Zhao, Jianshu Li, Tingfa Xu, and Jiashi Feng. Deep attribute-preserving metric learning for natural language object retrieval. In Pro- ceedings of the 25th ACM international conference on Mul- timedia, pages 181–189. ACM, 2017. 1

    2017

  9. [17]

    Tell-and-answer: Towards explainable visual question an- swering using attributes and captions

    Qing Li, Jianlong Fu, Dongfei Yu, Tao Mei, and Jiebo Luo. Tell-and-answer: Towards explainable visual question an- swering using attributes and captions. In Proceedings of the conference on empirical methods in natural language pro- cessing (EMNLP), 2018. 1

    2018

  10. [18]

    Feature pyramid networks for object detection

    Tsung-Yi Lin, Piotr Doll ´ar, Ross Girshick, Kaiming He, Bharath Hariharan, and Serge Belongie. Feature pyramid networks for object detection. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 2117–2125, 2017. 3

    2017

  11. [19]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision , pages 740–755. Springer, 2014. 4, 12

    2014

  12. [20]

    Recurrent multimodal interaction for refer- ring image segmentation

    Chenxi Liu, Zhe Lin, Xiaohui Shen, Jimei Yang, Xin Lu, and Alan Yuille. Recurrent multimodal interaction for refer- ring image segmentation. In Proceedings of the IEEE Inter- national Conference on Computer Vision, pages 1271–1280,

  13. [21]

    Ssd: Single shot multibox detector

    Wei Liu, Dragomir Anguelov, Dumitru Erhan, Christian Szegedy, Scott Reed, Cheng-Yang Fu, and Alexander C Berg. Ssd: Single shot multibox detector. In European con- ference on computer vision, pages 21–37. Springer, 2016. 6, 12

    2016

  14. [22]

    Improving referring expression grounding with cross-modal attention-guided erasing

    Xihui Liu, Zihao Wang, Jing Shao, Xiaogang Wang, and Hongsheng Li. Improving referring expression grounding with cross-modal attention-guided erasing. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1950–1959, 2019. 1

    1950

  15. [23]

    Comprehension- guided referring expressions

    Ruotian Luo and Gregory Shakhnarovich. Comprehension- guided referring expressions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 7102–7111, 2017. 5, 6

    2017

  16. [24]

    Generation and comprehension of unambiguous object descriptions

    Junhua Mao, Jonathan Huang, Alexander Toshev, Oana Camburu, Alan L Yuille, and Kevin Murphy. Generation and comprehension of unambiguous object descriptions. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 11–20, 2016. 1, 2

    2016

  17. [25]

    Dynamic multimodal instance segmentation guided by natural language queries

    Edgar Margffoy-Tuay, Juan C P ´erez, Emilio Botero, and Pablo Arbel´aez. Dynamic multimodal instance segmentation guided by natural language queries. In Proceedings of the European Conference on Computer Vision (ECCV) , pages 630–645, 2018. 1, 4

    2018

  18. [26]

    Distributed representations of words and phrases and their compositionality

    Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems, pages 3111–3119, 2013. 2, 3, 5

    2013

  19. [27]

    Mod- eling context between objects for referring expression un- derstanding

    Varun K Nagaraja, Vlad I Morariu, and Larry S Davis. Mod- eling context between objects for referring expression un- derstanding. In European Conference on Computer Vision, pages 792–807. Springer, 2016. 2

    2016

  20. [28]

    Im- proving the fisher kernel for large-scale image classification

    Florent Perronnin, Jorge S ´anchez, and Thomas Mensink. Im- proving the fisher kernel for large-scale image classification. In European conference on computer vision, pages 143–156. Springer, 2010. 2, 3, 5

    2010

  21. [29]

    Plummer, Paige Kordas, M

    Bryan A. Plummer, Paige Kordas, M. Hadi Kiapour, Shuai Zheng, Robinson Piramuthu, and Svetlana Lazebnik. Condi- tional image-text embedding networks. In ECCV, 2018. 2, 3, 4, 5, 6

    2018

  22. [30]

    Flickr30k entities: Collecting region-to-phrase corre- spondences for richer image-to-sentence models

    Bryan A Plummer, Liwei Wang, Chris M Cervantes, Juan C Caicedo, Julia Hockenmaier, and Svetlana Lazeb- nik. Flickr30k entities: Collecting region-to-phrase corre- spondences for richer image-to-sentence models. Interna- tional journal of computer vision, 123(1):74–93, 2017. 1,...

    2017

  23. [31]

    2, 3, 4, 5, 6

    ACM, 2017. 2, 3, 4, 5, 6

    2017

  24. [32]

    You only look once: Unified, real-time object de- tection

    Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object de- tection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 779–788, 2016. 3

    2016

  25. [33]

    Yolo9000: better, faster, stronger

    Joseph Redmon and Ali Farhadi. Yolo9000: better, faster, stronger. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 7263–7271, 2017. 3, 4

    2017

  26. [34]

    Yolov3: An incremental improvement

    Joseph Redmon and Ali Farhadi. Yolov3: An incremental improvement. arXiv preprint arXiv:1804.02767, 2018. 2, 3, 4, 5

    2018 arXiv

  27. [35]

    Faster r-cnn: Towards real-time object detection with region proposal networks

    Shaoqing Ren, Kaiming He, Ross Girshick, and Jian Sun. Faster r-cnn: Towards real-time object detection with region proposal networks. In Advances in neural information pro- cessing systems, pages 91–99, 2015. 2, 12

    2015

  28. [36]

    Grounding of textual phrases in images by reconstruction

    Anna Rohrbach, Marcus Rohrbach, Ronghang Hu, Trevor Darrell, and Bernt Schiele. Grounding of textual phrases in images by reconstruction. In European Conference on Com- puter Vision, pages 817–834. Springer, 2016. 2, 3, 4, 5, 6

    2016

  29. [37]

    Berg, and Li Fei-Fei

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, Alexander C. Berg, and Li Fei-Fei. ImageNet Large Scale Visual Recognition Chal- lenge. International Journal of Computer Vision ...

    2015

  30. [38]

    Faster r-cnn features for instance search

    Amaia Salvador, Xavier Gir ´o-i Nieto, Ferran Marqu ´es, and Shin’ichi Satoh. Faster r-cnn features for instance search. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pages 9–16, 2016. 1

    2016

  31. [39]

    Very deep convo- lutional networks for large-scale image recognition

    Karen Simonyan and Andrew Zisserman. Very deep convo- lutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556, 2014. 5

    2014 arXiv

  32. [40]

    Parsing with compositional vector grammars

    Richard Socher, John Bauer, Christopher D Manning, et al. Parsing with compositional vector grammars. In Proceed- ings of the 51st Annual Meeting of the Association for Com- putational Linguistics (Volume 1: Long Papers) , volume 1, pages 455–465, 2013. 2

    2013

  33. [41]

    Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent mag- nitude

    Tijmen Tieleman and Geoffrey Hinton. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent mag- nitude. COURSERA: Neural networks for machine learning, 4(2):26–31, 2012. 5

    2012

  34. [42]

    Selective search for ob- ject recognition

    Jasper RR Uijlings, Koen EA Van De Sande, Theo Gev- ers, and Arnold WM Smeulders. Selective search for ob- ject recognition. International journal of computer vision , 104(2):154–171, 2013. 2, 5, 7, 12

    2013

  35. [43]

    Learning two-branch neural networks for image-text match- ing tasks

    Liwei Wang, Yin Li, Jing Huang, and Svetlana Lazebnik. Learning two-branch neural networks for image-text match- ing tasks. IEEE Transactions on Pattern Analysis and Ma- chine Intelligence, 41(2):394–407, 2018. 1, 2, 3, 4, 5, 6, 12

    2018

  36. [44]

    Learning deep structure-preserving image-text embeddings

    Liwei Wang, Yin Li, and Svetlana Lazebnik. Learning deep structure-preserving image-text embeddings. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 5005–5013, 2016. 2, 5, 6

    2016

  37. [45]

    Interpretable and globally optimal pre- diction for textual grounding using image concepts

    Raymond Yeh, Jinjun Xiong, Wen-Mei Hwu, Minh Do, and Alexander Schwing. Interpretable and globally optimal pre- diction for textual grounding using image concepts. In Ad- vances in Neural Information Processing Systems , pages 1912–1922, 2017. 4, 5, 6

    1912

  38. [46]

    Image captioning with semantic attention

    Quanzeng You, Hailin Jin, Zhaowen Wang, Chen Fang, and Jiebo Luo. Image captioning with semantic attention. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4651–4659, 2016. 1

    2016

  39. [47]

    From image descriptions to visual denotations: New similarity metrics for semantic inference over event descrip- tions

    Peter Young, Alice Lai, Micah Hodosh, and Julia Hocken- maier. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descrip- tions. Transactions of the Association for Computational Linguistics, 2:67–78, 2014. 4

    2014

  40. [48]

    Mattnet: Modular atten- tion network for referring expression comprehension

    Licheng Yu, Zhe Lin, Xiaohui Shen, Jimei Yang, Xin Lu, Mohit Bansal, and Tamara L Berg. Mattnet: Modular atten- tion network for referring expression comprehension. InPro- ceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1307–1315, 2018. 1, 2, 12

    2018

  41. [49]

    Modeling context in referring expres- sions

    Licheng Yu, Patrick Poirson, Shan Yang, Alexander C Berg, and Tamara L Berg. Modeling context in referring expres- sions. In European Conference on Computer Vision, pages 69–85. Springer, 2016. 1, 2, 4, 12, 13

    2016

  42. [50]

    A joint speaker-listener-reinforcer model for referring expres- sions

    Licheng Yu, Hao Tan, Mohit Bansal, and Tamara L Berg. A joint speaker-listener-reinforcer model for referring expres- sions. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 7282–7290, 2017. 2, 12

    2017

  43. [51]

    Ground- ing referring expressions in images by variational context

    Hanwang Zhang, Yulei Niu, and Shih-Fu Chang. Ground- ing referring expressions in images by variational context. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4158–4166, 2018. 2, 5, 6, 12

    2018

  44. [52]

    Discriminative bimodal networks for visual localization and detection with natural language queries

    Yuting Zhang, Luyao Yuan, Yijie Guo, Zhiyuan He, I-An Huang, and Honglak Lee. Discriminative bimodal networks for visual localization and detection with natural language queries. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 557–566, 2017. 2

    2017

  45. [53]

    Weakly supervised phrase localization with multi-scale anchored transformer network

    Fang Zhao, Jianshu Li, Jian Zhao, and Jiashi Feng. Weakly supervised phrase localization with multi-scale anchored transformer network. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 5696– 5705, 2018. 4

    2018

  46. [54]

    Visual7w: Grounded question answering in images

    Yuke Zhu, Oliver Groth, Michael Bernstein, and Li Fei-Fei. Visual7w: Grounded question answering in images. In Pro- ceedings of the IEEE conference on computer vision and pat- tern recognition, pages 4995–5004, 2016. 1

    2016

  47. [55]

    testA” contains images with multiple people and “testB

    C Lawrence Zitnick and Piotr Doll ´ar. Edge boxes: Locat- ing object proposals from edges. In European conference on computer vision, pages 391–405. Springer, 2014. 2, 5, 7, 12 Appendix A. Results on RefCOCO The extended experiments on RefCOCO [48] are re- ported in Table 4. R...

    2014

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.