Pith. sign in

REVIEW 4 major objections 4 minor 38 references

VP-MEL: Visual Prompts Guided Multimodal Entity Linking

T0 review · 4 major / 4 minor · reviewed 2026-08-11 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read The paper introduces VP-MEL, where a marked image region replaces a mention word, and claims its IIER framework links such regions to knowledge-base entities at 48.36% Hit@1 on the new VPWiki dataset, beating all baselines.

desk verdict The VP-MEL task and VPWiki dataset are a genuine contribution, but the unspecified inference-time prompt for Detective-VLM could allow gold-label leakage and must be resolved before the headline result is credible. read the letter →

arxiv 2412.06720 v4 pith:TMZRBCGC submitted 2024-12-09 cs.CV cs.CL

Hongze Mi , Jinyuan Li , Xuying Zhang , Haoran Cheng , Jiahao Wang , Di Sun , Gang Pan This is my paper · ORCID
classification cs.CVcs.CL
keywords multimodalentitylinkingvisualpromptsvision-languagemodeldisambiguationgroundingknowledgebaseVP-MELcontrastivelearning
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Multimodal entity linking usually starts from a mention word in the text and retrieves a knowledge-base entity. This paper argues that the mention can be replaced by a visual prompt, a marked region in the image, and that a vision-language model fine-tuned on the new VPWiki dataset can generate enough auxiliary text to make the link work even when no mention word is written. The proposed IIER framework combines prompt-guided image features, deep and shallow CLIP layers, and the generated text, and the experiments claim a Hit@1 of 48.36 on VP-MEL, about five points above the strongest vision-language baseline, while remaining competitive on standard MEL datasets. If the claim holds, entity linking no longer depends on text density, and any image region a user marks can be resolved to a knowledge-base entity.

What carries the argument

The load-bearing object is the visual prompt: a manually or automatically drawn red box that tells the model which region to link. Around it, IIER stacks four components: a CLIP image encoder whose shallow and deep layers are both pooled, with shallow layers preserving the texture and geometry of the prompt and deep layers preserving semantics; Detective-VLM, an instruction-tuned vision-language model that turns the boxed region and the sentence into a short generated phrase $\text{mw}_j$; a BERT text encoder that reads the sentence with $\text{mw}_j$ prepended; and three interaction units, VFI, TFI, and CMFI, that compute visual, textual, and cross-modal similarities averaged into one contrastive score. The visual prompt does the work that mention words used to do: it localizes the referent, and the generated auxiliary text adapts the language side to that localization.

What would settle it

Run the VPWiki test set with the Detective-VLM prompt rewritten so the entity-name and entity-type fields are blank, generated only by the model, and compare Hit@1 against the reported 48.36. If the score falls toward the no-VLM ablation at 35.65 or toward LLaVA-1.5 at 43.20, the reported gain comes from the gold label appearing in the prompt rather than from latent knowledge.

Watch

Extended reading notes

Core claim

The central claim is that visual prompts alone can carry the mention signal in entity linking. Given an image with a red-box region, IIER extracts global and local features with a CLIP image encoder, uses an instruction-tuned Detective-VLM to read the box and produce a short auxiliary phrase identifying the object and its type, concatenates that phrase with the sentence, and scores candidate entities by averaged visual, textual, and cross-modal similarity under contrastive training. On the VPWiki test set the paper reports Hit@1 48.36, Hit@3 67.51, and Hit@5 77.50, outperforming every compared MEL and vision-language method, and on WikiDiverse it reports Hit@1 69.47, ahead of the previous best baseline. The paper interprets these results as evidence that a region-level prompt plus latent vision-language knowledge can substitute for explicit mention words.

Load-bearing premise

The weakest load-bearing premise is that Detective-VLM really produces the entity name and type at test time. The printed prompt template places '{Entity Name}' and '{Entity Type}' inside the question and the expected answer, so if those fields are present during inference the auxiliary text is taken from the gold label, and if they are absent the paper never specifies the exact test-time prompt.

Editorial extensions

If this is right

  • MEL systems no longer need annotated mention words: any marked region can be linked, which makes image-first and text-poor multimodal content usable.
  • The instruction-tuned VLM's generated auxiliary phrase carries enough signal to lift Hit@1 by more than 12 points over the same model without the VLM on VP-MEL.
  • On WikiDiverse without mention words, IIER reaches Hit@1 23.87, far above the 17.66 of the best MEL baseline, showing that the framework degrades gracefully when text cues are missing.
  • The VPWiki dataset, with 12,720 samples and visual-prompt annotations, provides a benchmark for future region-to-entity linking.
  • VP-MEL still needs image-text correlation and box-shaped prompts, so irregular user marks and unrelated image-text pairs remain open challenges.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Not tested in the paper: VP-MEL could be pushed to image-only linking, where the sentence is removed and the generated auxiliary text is the only language signal; the reported no-mention results on WikiDiverse suggest the gap would be modest.
  • Not tested in the paper: the box prompt is rectangular, so pairing the method with segmentation masks or point prompts would test whether the claimed gain comes from precise localization rather than from the prompt's visual appearance.
  • Not tested in the paper: because VPWiki is built from news-photo datasets, an adversarial test on user-generated social images with unrelated captions would show whether the image-text correlation requirement limits the task.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper defines a new task, VP-MEL, in which a marked image region (visual prompt) is linked to a KB entity, and introduces the VPWiki dataset built from WikiDiverse and WikiMEL. It proposes the IIER framework, which combines CLIP-based global/local visual features guided by visual prompts with a fine-tuned Detective-VLM that generates auxiliary textual information, followed by multi-grained multimodal interaction and contrastive learning. Experiments report that IIER outperforms all baselines on VP-MEL (Hit@1 48.36 vs. 43.20 for the best VLM baseline) and is competitive on standard MEL datasets (Hit@1 69.47 on WikiDiverse).

Significance. If the results hold, VP-MEL is a useful extension of entity linking to settings where textual mention words are absent, and the VPWiki dataset with its annotation pipeline and inter-annotator agreement is a valuable resource. The IIER architecture is well motivated, and the paper includes multiple ablations and cross-dataset evaluations. However, the main claimed advantage depends critically on the Detective-VLM module, and the prompt specification in §5.2 leaves open a direct gold-label leakage path. Because the headline result is not reproducible or ruleable-out without an exact test-time prompt, the contribution is currently conditional on closing that gap.

major comments (4)
  1. [§5.2, Table 2a, Table 4, Table 6] The Detective-VLM prompt template printed in §5.2 places '{Entity Name}' and '{Entity Type}' inside the question itself, immediately before the answer field. The only inference-time statement is 'During the inference process, {Entity Name} and {Entity Type} need to be generated by VLM,' which is ambiguous in a load-bearing way. If those fields are filled with gold labels at test time, then the generated mwj is a copy of the answer, and the textual and cross-modal similarity scores in §5.3–§5.5 partly measure string matching against the label the model was handed. If the fields are blanked or removed at test time, the exact inference prompt is never specified, so the experiment cannot be reproduced and the 48.36 Hit@1 gain in Table 2a could be an artifact of leakage. Please specify the exact test-time prompt, report the Detective-VLM outputs for a few examples, and re-run the core comparison with a prompt that does not contain the answer fields.
  2. [§5.2, fine-tuning objective] Even during instruction fine-tuning, the training template contains the gold entity name and type in the question as well as in the answer. The loss in §5.2 is a standard next-token prediction loss, so the model can minimize it by copying the answer fields from the input rather than by learning to ground the visual prompt in the image. This makes the claim that Detective-VLM acts as an 'implicit knowledge base' unsubstantiated: the learned behavior may be degenerate copying, which would break when the fields are removed at inference. Please either train with a prompt that omits the answer fields from the question, or provide an analysis (e.g., attention or perturbation experiments) showing that the generated mwj actually depends on the image and visual prompt rather than on the input placeholders.
  3. [§6.1, Appendix A.2, Table 2a] The evaluation protocol for the VLM baselines on the VP-MEL task is underspecified. It is not stated whether LLaVA, BLIP-2, MiniGPT-4, and mPLUG-Owl3 receive the same red-box visual prompt, what instruction or question template is used, whether they are fine-tuned on VPWiki, or how their outputs are converted to a KB entity ranking. Since the headline improvement over the best VLM baseline is 5.16 Hit@1 points, the comparison is only meaningful if the baselines are given the same visual prompt and task instruction. Please provide the exact input construction, decoding settings, and candidate entity ranking procedure for each baseline.
  4. [§6.3, Table 3] The text states that 'even without Detective-VLM, visual prompts, or mention words, IIER can still achieve the best metrics,' but Table 3 only reports IIER with all components on WikiDiverse without mention words. No ablation row is shown for IIER without Detective-VLM or without visual prompts on this setting, so the claim is not directly supported by the displayed results. Please add the corresponding ablation rows or qualify the statement.
minor comments (4)
  1. [§1] There is a typo in 'additionlLy' near the description of the CLIP visual encoder.
  2. [§5.3] In the final sentence of §5.3, 'we obtain the text features T G Ei and T G Ei' should read 'T G Ei and T L Ei'.
  3. [Table 2] The spacing in 'LLaV A' is inconsistent (e.g., 'LLaV A-1.5' vs. 'LLaVA' in prose); please use a single consistent notation.
  4. [Appendix A.9] The phrase 'It is noted that within these two datasets, certain metrics of IIER exhibit values that are comparable to those of MELOV' is repetitive and could be tightened.

Circularity Check

1 steps flagged · score 7.0 of 10

The printed Detective-VLM prompt embeds the gold entity name/type in the question; the paper never specifies a leakage-free test-time prompt, so the headline VP-MEL gain may reduce to copying the answer.

  1. fitted input called prediction [Section 5.2 (Detective-VLM prompt template and inference note), applied in Sections 5.3-5.4 and Tables 2a/6]
    "Question: Based on the text ’ {Sentence}’, tell me briefly what is the {Entity T ype} and {Entity N ame} in the red box of the {Image }? Answer: {Entity N ame} {Entity T ype} ... During the inference process, {Entity N ame} and {Entity T ype} need to be generated by VLM."

    In the only prompt the paper prints, the gold {Entity Name} and {Entity Type} appear in the question as well as in the answer, and the paper never states that the question-side placeholders are removed or blanked at inference. If they are filled, mwj, which is defined as the VLM Answer, is the gold label itself by construction. Section 5.3 then prepends mwj to the mention sentence, and Section 5.4 computes textual similarity against entity text built from entity name/attributes, so the 48.36 Hit@1 partly scores copying the entity name that was handed to the VLM in the question. Table 6 attributes a 12.71 Hit@1 drop to removing this module, so the claimed VP-MEL advantage is not separable from prompt leakage.

full rationale

The paper's MEL experiments and its visual/text interaction machinery are self-contained: IIER is trained with a standard contrastive objective on VPWiki, and the WikiDiverse/MEL results (Hit@1 69.47 vs 67.32 for MELOV) give independent, externally grounded evidence that the framework is not vacuous. The self-citation to Li et al. (2024) for the annotation pipeline is not load-bearing. However, the central VP-MEL claim depends on Detective-VLM, and the only prompt template in Section 5.2 contains the gold entity name/type in both the question and the answer. The paper's sole inference statement says these fields 'need to be generated by VLM' but does not specify a prompt with the slots removed. On the printed template, the generated auxiliary text mwj is the gold entity name/type, and the final text similarity is then computed against an entity representation containing exactly that name; the headline gain reduces to answer-copying rather than independent visual reasoning. This is a construction-level circularity in the main task, not a minor citation issue, so the score is 7.

Assumptions & free parameters 4 free parameters · 5 assumptions · 0 invented entities

The central claim rests on the fine-tuned VLM producing the entity name from the boxed image. Because the VLM is fine-tuned on the same VPWiki distribution and the prompt template may include the gold entity name, the auxiliary text can encode the answer. The dataset construction also selects for correlated image-text pairs, so the benchmark does not test the claimed robustness to missing or uncorrelated text.

free parameters (4)
  • loss weight lambda = 1
    Set in Appendix A.1; controls the balance between the joint contrastive loss and the three unit losses. The value is chosen by hand, not derived.
  • CLIP layer selection for global visual feature = layers 3, 10, 11, 12
    Section 5.1 selects one shallow and three deep CLIP layers; Table 5 compares combinations and the chosen set yields the best VP-MEL H@1, so the choice is tuned on the benchmark.
  • IoU threshold for visual prompt annotation = 0.5
    Section 3: samples with annotation IoU below 0.5 are discarded; this threshold shapes the dataset and is not justified by an external criterion.
  • learning rate = 1e-5
    Appendix A.1; standard hyperparameter chosen for training, not derived from theory.
assumptions (5)
  • domain assumption The image and text in each VPWiki sample are assumed to be correlated; samples where they are unrelated are filtered out during annotation.
    Section 3 annotation design removes unrelated image-mention pairs; the Limitations section acknowledges that VP-MEL performs poorly when image and text are uncorrelated. The benchmark therefore only tests the correlated regime.
  • domain assumption The knowledge base candidate set is assumed to contain the correct entity for every mention.
    Section 4 defines retrieval from entity set E; the candidate generation and the size of the candidate set per mention are not described, so the task reduces to ranking the pre-supplied correct entity.
  • domain assumption Pretrained CLIP, BERT, and mplug-owl2 features transfer to the VP-MEL task, and the fine-tuned Detective-VLM can name a boxed object from the image and sentence.
    Sections 5.1, 5.2 and A.4 rely on these pretrained models without verifying that the visual prompt is understood by the base encoder; this is the empirical substrate of the method.
  • domain assumption The visual prompt is a bounding box that unambiguously indicates the mention.
    The paper chooses grounding boxes as the prompt format (Limitations) and acknowledges that real users may use irregular shapes; the method is not tested on other prompt formats.
  • ad hoc to paper The Detective-VLM prompt template is assumed to yield useful auxiliary text at inference even though the printed template contains the answer fields.
    Section 5.2 prints the template with {Entity Name} and {Entity Type} in the question; the paper does not explain how the template is changed at inference, making the validity of the generated mwj an assumption specific to this paper.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VP-MEL: Visual Prompts Guided Multimodal Entity Linking." pith.science (2026). https://pith.science/paper/TMZRBCGC

@misc{pith2026241206720,
  author       = {Pith},
  title        = {Pith review of: VP-MEL: Visual Prompts Guided Multimodal Entity Linking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TMZRBCGC}},
  note         = {Machine review of arXiv:2412.06720}
}
read the original abstract

Multimodal entity linking (MEL), a task aimed at linking mentions within multimodal contexts to their corresponding entities in a knowledge base (KB), has attracted much attention due to its wide applications in recent years. However, existing MEL methods often rely on mention words as retrieval cues, which limits their ability to effectively utilize information from both images and text. This reliance causes MEL to struggle with accurately retrieving entities in certain scenarios, especially when the focus is on image objects or mention words are missing from the text. To solve these issues, we introduce a Visual Prompts guided Multimodal Entity Linking (VP-MEL) task. Given a text-image pair, VP-MEL aims to link a marked region (i.e., visual prompt) in an image to its corresponding entities in the knowledge base. To facilitate this task, we present a new dataset, VPWiki, specifically designed for VP-MEL. Furthermore, we propose a framework named IIER, which enhances visual feature extraction using visual prompts and leverages the pretrained Detective-VLM model to capture latent information. Experimental results on the VPWiki dataset demonstrate that IIER outperforms baseline methods across multiple benchmarks for the VP-MEL task.

Figures

Figures reproduced from arXiv: 2412.06720 by the authors.

Figure 1
Figure 1. Comparison between MEL and VP-MEL tasks. MEL is typically limited to selecting mentions from text. In contrast, VP-MEL addresses this limita￾tion by using visual prompts to link specific regions in the images to the correct entities in the knowledge base. and assume that each mention is associated with a high-quality image, which results in two limitations for MEL. Text information dependency: MEL relies on mention … view at source ↗
Figure 2
Figure 2. An example from VPWiki. GT denotes the ground truth entity. The red box in the left image repre￾sents the visual prompt annotated for the VP-MEL task. Train Dev. Test Total pairs 8,000 1,035 1,052 10,087 ment. per pair 1.18 1.16 1.27 1.19 words per pair 9.89 9.80 10.32 9.92 [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. More statistics of VPWiki. (a) Distribution of [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: The overall architecture of Implicit Information-Enhanced Reasoning (IIER) framework. The image-text pairs of the Mention and Entity are used together as input. Specifically, Mention Text is the sentence corresponding to Mention Image, while Entity Text consists of Ent…
Figure 5
Figure 5. Figure 5: Entity type distribution of WIKIDiverse. [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: Case study for VP-MEL. Each row is a case, which contains Input, ground truth entity, and top three [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Examples of the VPWiki dataset. Each row represents a sample corresponding to a specific entity type, [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 24 canonical work pages

  1. [1]

    Omar Adjali, Romaric Besan c on, Olivier Ferret, Herv \'e Le Borgne, and Brigitte Grau. 2020. Multimodal entity linking for tweets. In ECIR, pages 463--478. Springer

    work page 2020

  2. [2]

    Meyer, Yuning Chai, Dennis Park, and Yong Jae Lee

    Mu Cai, Haotian Liu, Siva Karthik Mustikovela, Gregory P. Meyer, Yuning Chai, Dennis Park, and Yong Jae Lee. 2024. https://doi.org/10.1109/CVPR52733.2024.01227 Vip-llava: Making large multimodal models understand arbitrary visual prompts . In CVPR, pages 12914--12923

    arXiv 2024

  3. [3]

    Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. https://doi.org/10.18653/v1/N19-1423 BERT : Pre-training of deep bidirectional transformers for language understanding . In HLT-NAACL, pages 4171--4186, Minneapolis, Minnesota. Association for Computational Linguistics

    doi:10.18653/v1/n19-1423 2019

  4. [4]

    Zhang Dongjie and Longtao Huang. 2022. https://doi.org/10.18653/v1/2022.findings-emnlp.230 Multimodal knowledge learning for named entity disambiguation . In Findings of ACL: EMNLP, pages 3160--3169, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics

    work page doi:10.18653/v1/2022.findings-emnlp.230 2022

  5. [5]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2021. https://openreview.net/forum?id=YicbFdNTTy An image is worth 16x16 words: Transformers for image recognition at scale . In ICLR. OpenReview.net

    work page 2021

  6. [6]

    Zi-Yi Dou, Yichong Xu, Zhe Gan, Jianfeng Wang, Shuohang Wang, Lijuan Wang, Chenguang Zhu, Pengchuan Zhang, Lu Yuan, Nanyun Peng, et al. 2022. An empirical study of training end-to-end vision-and-language transformers. In CVPR, pages 18166--18176

    work page 2022

  7. [7]

    Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. In Psychological bulletin, volume 76, page 378. American Psychological Association

    work page 1971

  8. [8]

    Jingru Gan, Jinchang Luo, Haiwei Wang, Shuhui Wang, Wei He, and Qingming Huang. 2021. https://doi.org/10.1145/3474085.3475400 Multimodal entity linking: A new dataset and a baseline . In ACM Multimedia, page 993–1001, New York, NY, USA. Association for Computing Machinery

    work page arXiv 2021

Show all 38 references
  1. [9]

    Wonjae Kim, Bokyung Son, and Ildoo Kim. 2021. https://proceedings.mlr.press/v139/kim21k.html Vilt: Vision-and-language transformer without convolution or region supervision . In ICML, volume 139 of Proceedings of Machine Learning Research, pages 5583--5594. PMLR

    2021

  2. [10]

    Jinyuan Li, Han Li, Di Sun, Jiahao Wang, Wenkun Zhang, Zan Wang, and Gang Pan. 2024. https://doi.org/10.18653/v1/2024.findings-acl.76 LLM s as bridges: Reformulating grounded multimodal named entity recognition . In Findings of ACL: ACL, pages 1302--1318, Bangkok, Thailand. As...

    2024 doi

  3. [11]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. https://proceedings.mlr.press/v202/li23q.html BLIP -2: Bootstrapping language-image pre-training with frozen image encoders and large language models . In ICML, volume 202 of Proceedings of Machine Learning Research,...

    2023

  4. [12]

    Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. 2021. Align before fuse: Vision and language representation learning with momentum distillation. In NeurIPS, volume 34, pages 9694--9705

    2021

  5. [13]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. Improved baselines with visual instruction tuning. In CVPR, pages 26296--26306

    2024

  6. [14]

    Pengfei Luo, Tong Xu, Shiwei Wu, Chen Zhu, Linli Xu, and Enhong Chen. 2023. https://doi.org/10.1145/3580305.3599439 Multi-grained multimodal interaction network for entity linking . In KDD, page 1583–1594, New York, NY, USA. Association for Computing Machinery

    2023

  7. [15]

    Seungwhan Moon, Leonardo Neves, and Vitor Carvalho. 2018. https://doi.org/10.18653/v1/P18-1186 Multimodal named entity disambiguation for noisy social media posts . In ACL (Volume 1: Long Papers), pages 2000--2008, Melbourne, Australia. Association for Computational Linguistics

    2018 doi

  8. [16]

    Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, Qixiang Ye, and Furu Wei. 2024. https://openreview.net/forum?id=lLmqxkfSIw Grounding multimodal large language models to the world . In ICLR. OpenReview.net

    2024

  9. [17]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In ICML, pages 8748--8763. PMLR

    2021

  10. [18]

    Senbao Shi, Zhenran Xu, Baotian Hu, and Min Zhang. 2024. https://aclanthology.org/2024.lrec-main.676 Generative multimodal entity linking . In LREC-COLING, pages 7654--7665, Torino, Italia. ELRA and ICCL

    2024

  11. [19]

    Aleksandar Shtedritski, Christian Rupprecht, and Andrea Vedaldi. 2023. What does clip know about a red circle? visual prompt engineering for vlms. In ICCV, pages 11987--11997

    2023

  12. [20]

    Shezheng Song, Shan Zhao, Chengyu Wang, Tianwei Yan, Shasha Li, Xiaoguang Mao, and Meng Wang. 2024. https://doi.org/10.1609/aaai.v38i17.29867 A dual-way enhanced framework from text matching point of view for multimodal entity linking . In AAAI, volume 38, pages 19008--19016

    2024 doi

  13. [21]

    Xuhui Sui, Ying Zhang, Yu Zhao, Kehui Song, Baohang Zhou, and Xiaojie Yuan. 2024. https://doi.org/10.18653/v1/2024.findings-acl.46 MELOV : Multimodal entity linking with optimized visual features in latent space . In Findings of ACL: ACL, pages 816--826, Bangkok, Thailand. Ass...

    2024 doi

  14. [22]

    Peng Wang, Jiangheng Wu, and Xiaohang Chen. 2022 a . https://doi.org/10.1145/3477495.3531867 Multimodal entity linking with gated hierarchical fusion and contrastive training . In SIGIR, page 938–948, New York, NY, USA. Association for Computing Machinery

    2022

  15. [23]

    Peng Wang, An Yang, Rui Men, Junyang Lin, Shuai Bai, Zhikang Li, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. 2022 b . Ofa: Unifying architectures, tasks, and modalities through a simple sequence-to-sequence learning framework. In ICML, pages 23318--23340. PMLR

    2022

  16. [24]

    Xuwu Wang, Junfeng Tian, Min Gui, Zhixu Li, Rui Wang, Ming Yan, Lihan Chen, and Yanghua Xiao. 2022 c . https://doi.org/10.18653/v1/2022.acl-long.328 W iki D iverse: A multimodal entity linking dataset with diversified contextual topics and entity types . In ACL (Volume 1: Long...

    2022 doi

  17. [25]

    Ledell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, and Luke Zettlemoyer. 2020. https://doi.org/10.18653/v1/2020.emnlp-main.519 Scalable zero-shot entity linking with dense entity retrieval . In EMNLP, pages 6397--6407, Online. Association for Computational Linguistics

    2020 doi

  18. [26]

    Shangyu Xing, Fei Zhao, Zhen Wu, Chunhui Li, Jianbing Zhang, and Xinyu Dai. 2023. https://doi.org/10.1145/3581783.3612575 Drin: Dynamic relation interactive network for multimodal entity linking . In ACM Multimedia, page 3599–3608, New York, NY, USA. Association for Computing ...

    2023

  19. [27]

    Chengmei Yang, Bowei He, Yimeng Wu, Chao Xing, Lianghua He, and Chen Ma. 2023. https://proceedings.mlr.press/v216/yang23d.html MMEL : A joint learning framework for multi-mention entity linking . In UAI, volume 216 of Proceedings of Machine Learning Research, pages 2411--2421. PMLR

    2023

  20. [28]

    Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. 2023. mplug-owl: Modularization empowers large language models with multimodality. arXiv preprint arXiv:2304.14178

    2023 arXiv

  21. [29]

    Qinghao Ye, Haiyang Xu, Jiabo Ye, Ming Yan, Anwen Hu, Haowei Liu, Qi Qian, Ji Zhang, and Fei Huang. 2024. mplug-owl2: Revolutionizing multi-modal large language model with modality collaboration. In CVPR, pages 13040--13051

    2024

  22. [30]

    Gongrui Zhang, Chenghuan Jiang, Zhongheng Guan, and Peng Wang. 2023 a . Multimodal entity linking with mixed fusion mechanism. In DASFAA, pages 607--622. Springer

    2023

  23. [31]

    Shilong Zhang, Peize Sun, Shoufa Chen, Min Xiao, Wenqi Shao, Wenwei Zhang, Kai Chen, and Ping Luo. 2023 b . Gpt4roi: Instruction tuning large language model on region-of-interest. arXiv preprint arXiv:2307.03601

    2023 arXiv

  24. [32]

    Liang Zhao, En Yu, Zheng Ge, Jinrong Yang, Haoran Wei, Hongyu Zhou, Jianjian Sun, Yuang Peng, Runpei Dong, Chunrui Han, et al. 2023. Chatspot: Bootstrapping multimodal llms via precise referring instruction tuning. arXiv preprint arXiv:2307.09474

    2023 arXiv

  25. [33]

    Qiushuo Zheng, Hao Wen, Meng Wang, and Guilin Qi. 2022. Visual entity linking via multi-modal learning. In Data Intel, volume 4, pages 1--19

    2022

  26. [34]

    Chong Zhou, Chen Change Loy, and Bo Dai. 2022. Extract free dense labels from clip. In ECCV, pages 696--712. Springer

    2022

  27. [35]

    Qiang Zhou, Chaohui Yu, Shaofeng Zhang, Sitong Wu, Zhibing Wang, and Fan Wang. 2023. Regionblip: A unified multi-modal pre-training framework for holistic and regional comprehension. arXiv preprint arXiv:2308.02299

    2023 arXiv

  28. [36]

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. 2024. https://openreview.net/forum?id=1tZbq88f27 Mini GPT -4: Enhancing vision-language understanding with advanced large language models . In ICLR. OpenReview.net

    2024

  29. [37]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  30. [38]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.