Pith. sign in

REVIEW 4 major objections 3 minor 1 cited by

KnowDR-REC: A Benchmark for Referring Expression Comprehension with Real-World Knowledge

T0 review · 4 major / 3 minor · reviewed 2026-08-05 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read KnowDR-REC, a new benchmark for referring expression comprehension, shows that state-of-the-art multimodal models fail when expressions require real-world knowledge.

desk verdict Plausible new REC benchmark combination, but the abstract alone carries no annotation-quality or shortcut evidence, so the 'struggle' and 'decoupling' claims are unprovable as presented. read the letter →

arxiv 2508.14080 v1 pith:XAABTAWF submitted 2025-08-12 cs.LG

Guanghao Jin , Jingpei Wu , Tianpei Guo , Yiyi Niu , Weidong Zhou , Guoyang Liu This is my paper
classification cs.LG
keywords ReferringExpressionComprehensionmultimodallargelanguagemodelsgroundingbenchmarkreal-worldknowledgenegativesamplesshortcutlearningevaluationmetrics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

KnowDR-REC is a benchmark for Referring Expression Comprehension (REC) that asks models to locate an object from a textual expression whose resolution requires real-world knowledge, not just words that match pixels. According to the paper, the benchmark combines fine-grained instance annotations, negative expressions produced by small edits that flip the correct target, and three evaluation metrics that distinguish a model's textual understanding from its visual grounding. Evaluating 16 state-of-the-art multimodal large language models, the paper reports that these models struggle on the benchmark and are strongly influenced by memorized shortcut correlations. If the benchmark is sound, it means strong scores on conventional REC tests do not imply that a model can ground expressions that require knowledge about the world.

What carries the argument

The central object is the KnowDR-REC benchmark itself, a set of region-level referring-expression items that cannot be resolved by intra-image cues alone: each expression draws on real-world knowledge such as typical use, context, or category relations. Its mechanism has three parts: (1) fine-grained instance annotations that tie expressions to specific regions; (2) negative expressions produced by fine-grained edits that change the correct referent while remaining plausible; and (3) three evaluation metrics that separate textual answering from visual grounding and expose reliance on shortcuts. The combination lets the benchmark attribute a model's success or failure to genuine multimodal re

What would settle it

For each negative expression in KnowDR-REC, present the expression without the image to a text-only language model and ask it to pick the correct referent among the possible objects: if text-only selection matches or beats the visual-grounding accuracy reported in the paper, the benchmark is solvable from language priors rather than from image grounding.

Watch

Extended reading notes

Core claim

The paper's central claim is that existing multimodal large language models cannot reliably perform knowledge-driven visual grounding: when a referring expression depends on real-world knowledge, accuracy drops far below what standard REC benchmarks suggest. More specifically, the paper reports a decoupling between textual understanding and visual grounding: many models can correctly handle the text in isolation yet fail to localize the corresponding object in the image, indicating that their language competence is not integrated with their perception. The paper argues this happens because models rely on memorized shortcut correlations, and the benchmark's negative-sample editing is designed

Load-bearing premise

The benchmark's ground truth is correct and unambiguous, including the edited negative expressions, so the reported failure rates reflect model reasoning rather than annotation errors or shortcuts available in the data.

Editorial extensions

If this is right

  • If the results hold, high accuracy on existing REC benchmarks does not demonstrate that an MLLM genuinely understands the knowledge behind a referring expression.
  • The reported decoupling implies that improving a model's language-only competence will not automatically improve its visual grounding.
  • The benchmark's negative-expression edits show that ordinary positive-only test sets can hide a model's tendency to exploit shortcut correlations.
  • The three evaluation metrics provide a template for future benchmarks to report reasoning behaviour rather than only a final bounding box.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • My inference: the same text-versus-grounding decoupling could appear in other grounded multimodal tasks such as visual question answering, and could be tested by giving the same knowledge question with and without the image.
  • My inference: one could build a curriculum version of KnowDR-REC by gradually increasing the knowledge gap between the expression and the visible content, mapping where models switch from genuine grounding to shortcut guessing.
  • My inference: the results suggest that practical systems should be equipped with an abstention mechanism, since current models do not seem able to detect when they lack the world knowledge needed to ground an expression.
  • My inference: a concrete extension would be to use KnowDR-REC to measure whether retrieval augmentation or chain-of-thought prompting closes the gap between textual understanding and visual grounding.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 3 minor

Summary. The manuscript, as provided, consists of the abstract only; the body, tables, and figures were not available for review. The paper proposes KnowDR-REC, a new Referring Expression Comprehension (REC) benchmark built on real-world knowledge, with negative samples created by fine-grained expression editing, and three new evaluation metrics intended to probe MLLM reasoning. It reports evaluations of 16 state-of-the-art multimodal models and claims that (i) existing MLLMs struggle with knowledge-driven visual grounding and (ii) there is a decoupling between textual understanding and visual grounding, with models influenced by memorized shortcut correlations. These are empirical claims about model behavior that require a valid benchmark, yet the abstract alone provides no annotation-quality statistics, no negative-sample validation, no leakage analysis, no metric definitions, and no human baseline. The core finding is therefore plausible but currently unsubstantiated in the available text.

Significance. If the benchmark is valid, the contribution is timely and potentially impactful: it targets a real gap in REC evaluation by requiring external encyclopedic knowledge and by introducing negative samples to test anti-hallucination robustness. The external anchoring of ground truth in human judgments is a strength in principle, so the benchmark is not circular. However, the paper's contribution is entirely empirical, and its credibility rests on dataset construction quality. The absence of inter-annotator agreement, human performance, leakage controls, and metric definitions means the reported failure rates could measure benchmark artifacts rather than MLLM reasoning. The decoupling claim is an explanatory inference that must be supported by per-instance controls; otherwise it remains speculation.

major comments (4)
  1. [Abstract (Benchmark Construction)] The load-bearing quantitative claim—that MLLMs struggle—depends on correct ground-truth labels and on negative-sample edits that genuinely change the correct referent. The abstract asserts 'elaborately constructed negative samples via fine-grained expression editing' but reports no inter-annotator agreement, no human baseline, and no worked example demonstrating that the edited expressions change the answer. Without these, the failure rates conflate benchmark artifacts with model reasoning. This concern is central and must be resolved by reporting annotation reliability and negative-sample validation.
  2. [Abstract (Decoupling and Shortcut Claim)] The conclusion that MLLMs exhibit 'decoupling between textual understanding and visual grounding' requires showing that the same expressions are correctly understood in a text-only setting while visual grounding fails, and that failure is not caused by test-set shortcuts such as unnatural wording of negatives or positive instances answerable from language priors. The abstract provides no per-instance analysis or control for item difficulty. Without such controls, the 'shortcut correlation' explanation is unfalsifiable: a model weak at both textual understanding and grounding would also produce high joint failure. The authors must provide per-instance breakdowns to support the decoupling claim.
  3. [Abstract (Evaluation Metrics)] The paper introduces 'three novel evaluation metrics' to 'systematically explore the model's internal reasoning process,' but the abstract does not define them or explain how they differ from standard REC metrics. Since these metrics are purported to substantiate the decoupling and shortcut findings, their definitions, scoring rules, and validation must be provided before the empirical claims can be interpreted. This is not a presentation nicety; the metrics are load-bearing for the paper's central conclusions.
  4. [Abstract (Human Baseline and Leakage Control)] No human performance is reported for the benchmark. The statement that 'existing MLLMs still struggle' is uncalibrated without human accuracy on the same instances. Additionally, no leakage analysis is reported to rule out that the benchmark itself contains memorized correlations from pretraining or that the negative/positive distinction is learnable from dataset statistics. Both are necessary to interpret the reported failure rates as evidence about knowledge-driven grounding rather than benchmark-specific cues.
minor comments (3)
  1. [General] The abstract does not include dataset size, number of instances per split, or qualitative examples. A benchmark paper should at least summarize these in the abstract or introduction.
  2. [Reproducibility] No URL, dataset release plan, or code availability statement is mentioned. For a benchmark paper, these are expected.
  3. [Statistics] The claim that '16 state-of-the-art models' were evaluated would benefit from confidence intervals or significance tests when comparing model families, especially for the decoupling claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found; the benchmark's evidence chain is externally anchored.

full rationale

The paper's central claims are empirical benchmark results, not derivations from the model outputs being tested. The ground-truth annotations are described as built upon real-world knowledge and human constructions (e.g., 'elaborately constructed negative samples via fine-grained expression editing'), which anchor evaluation targets externally rather than in the models under test. The reported finding that 'existing MLLMs still struggle with knowledge-driven visual grounding tasks' is an observation from evaluating 16 models, not a quantity that is defined as equal to its own input. The 'decoupling' observation is an interpretive inference from model behavior, and while the paper does not report inter-annotator agreement or leakage analyses, those are validity concerns rather than circularity. No self-citations, uniqueness-theorem imports, fitted parameters renamed as predictions, or ansatz-smuggling are present in the available text. The three novel metrics are not defined in the provided excerpt, so there is no basis to claim they reduce to the failure pattern by construction. Therefore the paper does not exhibit circular reasoning.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

All load-bearing elements of a benchmark paper are construction choices, and none are visible in the abstract: annotation protocol, expression editing procedure, metric definitions, and evaluation settings. They are recorded here as assumptions because the full text was not provided. No fundamentally new entities (forces, particles, dimensions) are introduced; the three metrics are measurement instruments, not invented entities.

free parameters (3)
  • Dataset size and negative-sample construction rules = not disclosed in abstract
    The paper's conclusions depend on the number of instances and on the edit procedure used to create negative samples, which are chosen by hand rather than derived.
  • Negative-to-positive sample ratio = not disclosed in abstract
    The benchmark's difficulty and the reported failure rates depend on how many edited negative expressions are included, a hand-chosen ratio.
  • Thresholds or scoring rules of the three new metrics = not disclosed in abstract
    The claim that the metrics reveal internal reasoning depends on definitions and thresholds not stated in the abstract.
assumptions (3)
  • domain assumption Human annotations for the benchmark are correct and unambiguous
    The abstract claims the dataset has fine-grained instance annotations; if annotations are noisy, all model failure rates are uninterpretable.
  • domain assumption The task expressions genuinely require external real-world knowledge not obtainable from image pixels
    The abstract's first key feature asserts this; it is load-bearing for the claim that failures indicate reasoning deficits rather than benchmark leakage.
  • domain assumption The 16 selected models and the evaluation setup give a representative and fair comparison
    The abstract states 16 MLLMs were evaluated; comparability of prompts, decoding settings, and metric application is assumed.

how reviews work

0 comments
Cite this review

Pith. "Pith review of KnowDR-REC: A Benchmark for Referring Expression Comprehension with Real-World Knowledge." pith.science (2026). https://pith.science/paper/XAABTAWF

@misc{pith2026250814080,
  author       = {Pith},
  title        = {Pith review of: KnowDR-REC: A Benchmark for Referring Expression Comprehension with Real-World Knowledge},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XAABTAWF}},
  note         = {Machine review of arXiv:2508.14080}
}
read the original abstract

Referring Expression Comprehension (REC) is a popular multimodal task that aims to accurately detect target objects within a single image based on a given textual expression. However, due to the limitations of earlier models, traditional REC benchmarks either rely solely on intra-image cues or lack sufficiently fine-grained instance annotations, making them inadequate for evaluating the reasoning capabilities of Multi-modal Large Language Models (MLLMs). To address this gap, we propose a new benchmark, KnowDR-REC, characterized by three key features: Firstly, it is built upon real-world knowledge, requiring fine-grained multimodal reasoning across text and image. Secondly, the dataset includes elaborately constructed negative samples via fine-grained expression editing, designed to evaluate a model's robustness and anti-hallucination ability. Lastly, we introduce three novel evaluation metrics to systematically explore the model's internal reasoning process. We evaluate 16 state-of-the-art multimodal models on KnowDR-REC, with experimental results showing that existing MLLMs still struggle with knowledge-driven visual grounding tasks. Furthermore, we observe a decoupling between textual understanding and visual grounding in MLLMs, where many models are significantly influenced by memorized shortcut correlations, which severely affect their behavior on our benchmark and hinder genuine multimodal reasoning. We anticipate that the proposed benchmark will inspire future research towards developing more robust, interpretable, and knowledge-intensive visual grounding frameworks, driving the development of more reliable and robust multimodal systems for complex real-world scenarios.

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. FindIt: A Format-Informed Visual Detection Benchmark for Generalist Multimodal LLMs

    cs.CV 2026-06 unverdicted novelty 7.0 of 10

    FindIt is the first comprehensive benchmark for evaluating generalist MLLMs on promptable object detection, referring expression detection, instance-level detection, and video detection with standardized parsable outputs.

Reference graph

Works this paper leans on

45 extracted references · 21 canonical work pages · cited by 1 Pith paper

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    Bai, J.; Bai, S.; Yang, S.; Wang, S.; Tan, S.; Wang, P.; Lin, J.; Zhou, C.; and Zhou, J. 2023. Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond. arXiv:2308.12966

    arXiv 2023

  4. [4]

    Bai, S.; Chen, K.; Liu, X.; Wang, J.; Ge, W.; Song, S.; Dang, K.; Wang, P.; Wang, S.; Tang, J.; Zhong, H.; Zhu, Y.; Yang, M.; Li, Z.; Wan, J.; Wang, P.; Ding, W.; Fu, Z.; Xu, Y.; Ye, J.; Zhang, X.; Xie, T.; Cheng, Z.; Zhang, H.; Yang, Z.; Xu, H.; and Lin, J. 2025. Qwen2.5-VL Technical Report. arXiv:2502.13923

    arXiv 2025

  5. [5]

    Bai, Z.; Wang, P.; Xiao, T.; He, T.; Han, Z.; Zhang, Z.; and Shou, M. Z. 2024. Hallucination of multimodal large language models: A survey. arXiv preprint arXiv:2404.18930

    arXiv 2024

  6. [6]

    Bridge, A.-M. 2001. Wikipedia, the free encyclopedia. San Francisco (CA): Wikimedia Foundation

    work page 2001

  7. [7]

    Cai, B.; Xiang, Y.; Gao, L.; Zhang, H.; Li, Y.; and Li, J. 2022. Temporal knowledge graph completion: A survey. arXiv preprint arXiv:2201.08236

    arXiv 2022

  8. [8]

    Cao, S.; Shi, J.; Pan, L.; Nie, L.; Xiang, Y.; Hou, L.; Li, J.; He, B.; and Zhang, H. 2020. KQA pro: A dataset with explicit compositional programs for complex question answering over knowledge base. arXiv preprint arXiv:2007.03875

    work page Pith review arXiv 2020

Show all 45 references
  1. [9]

    Chen, X.; Ma, Z.; Zhang, X.; Xu, S.; Qian, S.; Yang, J.; Fouhey, D.; and Chai, J. 2024. Multi-object hallucination in vision language models. Advances in Neural Information Processing Systems, 37: 44393--44418

    2024

  2. [10]

    Comanici, G.; Bieber, E.; Schaekermann, M.; Pasupat, I.; Sachdeva, N.; Dhillon, I.; Blistein, M.; Ram, O.; Zhang, D.; Rosen, E.; et al. 2025. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv ...

    2025 arXiv

  3. [11]

    Hu, H.; Luan, Y.; Chen, Y.; Khandelwal, U.; Joshi, M.; Lee, K.; Toutanova, K.; and Chang, M.-W. 2023. Open-domain visual entity recognition: Towards recognizing millions of wikipedia entities. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 12065--12075

    2023

  4. [12]

    K.; Patra, B.; et al

    Huang, S.; Dong, L.; Wang, W.; Hao, Y.; Singhal, S.; Ma, S.; Lv, T.; Cui, L.; Mohammed, O. K.; Patra, B.; et al. 2023. Language is not all you need: Aligning perception with language models. Advances in Neural Information Processing Systems, 36: 72096--72109

    2023

  5. [13]

    Huang, Z.; Zhang, Y.; and Satoh, S. 2025. ReSeDis: A Dataset for Referring-based Object Search across Large-Scale Image Collections. arXiv preprint arXiv:2506.15180

    2025 arXiv

  6. [14]

    P.; Perelman, A.; Ramesh, A.; Clark, A.; Ostrow, A.; Welihinda, A.; Hayes, A.; Radford, A.; et al

    Hurst, A.; Lerer, A.; Goucher, A. P.; Perelman, A.; Ramesh, A.; Clark, A.; Ostrow, A.; Welihinda, A.; Hayes, A.; Radford, A.; et al. 2024. Gpt-4o system card. arXiv preprint arXiv:2410.21276

    2024 arXiv

  7. [15]

    Jiang, Q.; Wu, L.; Zeng, Z.; Ren, T.; Xiong, Y.; Chen, Y.; Liu, Q.; and Zhang, L. 2025. Referring to any person. arXiv preprint arXiv:2503.08507

    2025 arXiv

  8. [16]

    X.; and Wen, J.-R

    Li, Y.; Du, Y.; Zhou, K.; Wang, J.; Zhao, W. X.; and Wen, J.-R. 2023. Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10355

    2023 arXiv

  9. [17]

    T.; et al

    Li, Z.; Xu, Q.; Zhang, D.; Song, H.; Cai, Y.; Qi, Q.; Zhou, R.; Pan, J.; Li, Z.; Vu, V. T.; et al. 2024. Groundinggpt: Language enhanced multi-modal grounding model. arXiv preprint arXiv:2401.06071

    2024 arXiv

  10. [18]

    Lin, Z.; Liu, C.; Zhang, R.; Gao, P.; Qiu, L.; Xiao, H.; Qiu, H.; Lin, C.; Shao, W.; Chen, K.; et al. 2023. Sphinx: The joint mixing of weights, tasks, and visual embeddings for multi-modal large language models. arXiv preprint arXiv:2311.07575

    2023 arXiv

  11. [19]

    Liu, D.; Zhang, R.; Qiu, L.; Huang, S.; Lin, W.; Zhao, S.; Geng, S.; Lin, Z.; Jin, P.; Zhang, K.; et al. 2024 a . Sphinx-x: Scaling data and parameters for a family of multi-modal large language models. arXiv preprint arXiv:2402.05935

    2024 arXiv

  12. [20]

    Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2023. Visual instruction tuning. Advances in neural information processing systems, 36: 34892--34916

    2023

  13. [21]

    Liu, H.; Xue, W.; Chen, Y.; Chen, D.; Zhao, X.; Wang, K.; Hou, L.; Li, R.; and Peng, W. 2024 b . A survey on hallucination in large vision-language models. arXiv preprint arXiv:2402.00253

    2024 arXiv

  14. [22]

    Liu, J.; Yang, X.; Li, W.; and Wang, P. 2024 c . Finecops-ref: A new dataset and task for fine-grained compositional referring expression comprehension. arXiv preprint arXiv:2409.14750

    2024 arXiv

  15. [23]

    Liu, R.; Liu, C.; Bai, Y.; and Yuille, A. L. 2019. Clevr-ref+: Diagnosing visual reasoning with referring expressions. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 4185--4194

    2019

  16. [24]

    L.; and Murphy, K

    Mao, J.; Huang, J.; Toshev, A.; Camburu, O.; Yuille, A. L.; and Murphy, K. 2016. Generation and comprehension of unambiguous object descriptions. In Proceedings of the IEEE conference on computer vision and pattern recognition, 11--20

    2016

  17. [25]

    Marino, K.; Rastegari, M.; Farhadi, A.; and Mottaghi, R. 2019. Ok-vqa: A visual question answering benchmark requiring external knowledge. In Proceedings of the IEEE/cvf conference on computer vision and pattern recognition, 3195--3204

    2019

  18. [26]

    Miyai, A.; Yang, J.; Zhang, J.; Ming, Y.; Yu, Q.; Irie, G.; Li, Y.; Li, H.; Liu, Z.; and Aizawa, K. 2024. Unsolvable problem detection: Robust understanding evaluation for large multimodal models. arXiv preprint arXiv:2403.20331

    2024 arXiv

  19. [27]

    G.; and Gurevych, I

    Puerto, H.; S ahin, G. G.; and Gurevych, I. 2021. Metaqa: Combining expert agents for multi-skill question answering. arXiv preprint arXiv:2112.01922

    2021 arXiv

  20. [28]

    Qiao, Y.; Deng, C.; and Wu, Q. 2020. Referring expression comprehension: A survey of methods and datasets. IEEE Transactions on Multimedia, 23: 4426--4440

    2020

  21. [29]

    M.; Zhang, Z.; Zhao, S.; Metaxas, D.; et al

    Schulter, S.; Suh, Y.; Dafnis, K. M.; Zhang, Z.; Zhao, S.; Metaxas, D.; et al. 2023. Omnilabel: A challenging benchmark for language-based object detection. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 11953--11962

    2023

  22. [30]

    Shah, S.; Mishra, A.; Yadati, N.; and Talukdar, P. P. 2019. Kvqa: Knowledge-aware visual question answering. In Proceedings of the AAAI conference on artificial intelligence, volume 33, 8876--8884

    2019

  23. [31]

    Shen, H.; Liu, P.; Li, J.; Fang, C.; Ma, Y.; Liao, J.; Shen, Q.; Zhang, Z.; Zhao, K.; Zhang, Q.; Xu, R.; and Zhao, T. 2025. VLM-R1: A Stable and Generalizable R1-style Large Vision-Language Model. arXiv:2504.07615

    2025 arXiv

  24. [32]

    Sohan, M.; Sai Ram, T.; and Rami Reddy, C. V. 2024. A review on yolov8 and its advancements. In International Conference on Data Intelligence and Cognitive Informatics, 529--545. Springer

    2024

  25. [33]

    Sun, W.; Fan, Y.; Guo, J.; Zhang, R.; and Cheng, X. 2022. Visual named entity linking: A new dataset and a baseline. arXiv preprint arXiv:2211.04872

    2022 arXiv

  26. [34]

    Talmor, A.; and Berant, J. 2018. The web as a knowledge-base for answering complex questions. arXiv preprint arXiv:1803.06643

    2018 arXiv

  27. [35]

    Wang, P.; Liu, D.; Li, H.; and Wu, Q. 2020. Give me something to eat: Referring expression comprehension with commonsense knowledge. In Proceedings of the 28th ACM International Conference on Multimedia, 28--36

    2020

  28. [36]

    Wang, W.; Lv, Q.; Yu, W.; Hong, W.; Qi, J.; Wang, Y.; Ji, J.; Yang, Z.; Zhao, L.; XiXuan, S.; et al. 2024. Cogvlm: Visual expert for pretrained language models. Advances in Neural Information Processing Systems, 37: 121475--121499

    2024

  29. [37]

    Wei, F.; Zhao, J.; Yan, K.; Zhang, H.; and Xu, C. 2024. A large-scale human-centric benchmark for referring expression comprehension in the LMM era. Advances in Neural Information Processing Systems, 37: 69566--69587

    2024

  30. [38]

    V.; Zhou, D.; et al

    Wei, J.; Wang, X.; Schuurmans, D.; Bosma, M.; Xia, F.; Chi, E.; Le, Q. V.; Zhou, D.; et al. 2022. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35: 24824--24837

    2022

  31. [39]

    Wu, C.; Lin, Z.; Cohen, S.; Bui, T.; and Maji, S. 2020. Phrasecut: Language-based image segmentation in the wild. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 10216--10225

    2020

  32. [40]

    Z.; Kuthiala, A.; Zheng, C.; Bal, A.; Dabhi, M.; Wen, L.; Rustagi, T.; Lai, E.; Khyalia, S.; et al

    Xie, L.; Wei, G. Z.; Kuthiala, A.; Zheng, C.; Bal, A.; Dabhi, M.; Wen, L.; Rustagi, T.; Lai, E.; Khyalia, S.; et al. 2025. MAVERIX: Multimodal Audio-Visual Evaluation Reasoning IndeX. arXiv preprint arXiv:2503.21699

    2025

  33. [41]

    Xu, Y.; Zhu, L.; and Yang, Y. 2024. Mc-bench: A benchmark for multi-context visual grounding in the era of mllms. arXiv preprint arXiv:2410.12332

    2024 arXiv

  34. [42]

    W.; Salakhutdinov, R.; and Manning, C

    Yang, Z.; Qi, P.; Zhang, S.; Bengio, Y.; Cohen, W. W.; Salakhutdinov, R.; and Manning, C. D. 2018. HotpotQA: A dataset for diverse, explainable multi-hop question answering. arXiv preprint arXiv:1809.09600

    2018 arXiv

  35. [43]

    You, H.; Zhang, H.; Gan, Z.; Du, X.; Zhang, B.; Wang, Z.; Cao, L.; Chang, S.-F.; and Yang, Y. 2023. Ferret: Refer and ground anything anywhere at any granularity. arXiv preprint arXiv:2310.07704

    2023 arXiv

  36. [44]

    C.; and Berg, T

    Yu, L.; Poirson, P.; Yang, S.; Berg, A. C.; and Berg, T. L. 2016. Modeling context in referring expressions. In European conference on computer vision, 69--85. Springer

    2016

  37. [45]

    Zang, Y.; Li, W.; Han, J.; Zhou, K.; and Loy, C. C. 2025. Contextual object detection with multimodal large language models. International Journal of Computer Vision, 133(2): 825--843

    2025

Pith tools

Reviewed August 5, 2026 · model on record in the stance chip above.