I2CR: Intra- and Inter-modal Collaborative Reflections for Multimodal Entity Linking

Chao Wang; Jingping Liu; Junwen Li; Kaiwen Li; Tong Ruan; Xinyan He; Xuezhi Cao; Ziyan Liu; Zongyu Wang

REVIEW 2 major objections 8 minor 49 references

I2CR: Intra- and Inter-modal Collaborative Reflections for Multimodal Entity Linking

T0 review · 2 major / 8 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read A text-first LLM pipeline with gated visual feedback reports new state-of-the-art results on three multimodal entity linking benchmarks.

desk verdict Solid, well-ablated MEL system with real gains, but the load-bearing CLIP threshold in Eq. (3) is underspecified and must be clarified before the headline numbers are fully credible. read the letter →

arxiv 2508.02243 v1 pith:4RRBYHPO submitted 2025-08-04 cs.CV cs.IR

Ziyan Liu , Junwen Li , Kaiwen Li , Tong Ruan , Chao Wang , Xinyan He , Zongyu Wang , Xuezhi Cao

show 1 more author

Jingping Liu

This is my paper

classification cs.CVcs.IR

keywords multimodalentitylinkinglargelanguagemodelintra-modalreflectioninter-modalalignmentvisualiterativefeedbackdisambiguationknowledgegraphCLIP

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes I2CR, an LLM-based framework for multimodal entity linking that treats text as the primary signal and consults images only through a gated, multi-round reflection process. It claims this design fixes two weaknesses in existing LLM methods: images are sometimes unnecessary or even misleading, and a single visual pass misses useful clues. On WikiMEL, WikiDiverse, and RichMEL, the authors report top-1 accuracies of 92.2%, 91.6%, and 86.8%, beating previous methods by 3.2, 5.1, and 1.6 points while training only on WikiDiverse. The significance of the claim, if true, is that a text-first gated visual-feedback pipeline is the current best published approach for this task.

What carries the argument

The load-bearing mechanism is a four-stage loop. Target Entity Selection retrieves top-$k$ candidates by fuzzy string matching and picks one with a LoRA-fine-tuned LLM. Intra-modal Consistency Reflection computes a normalized cosine similarity between the mention text and the entity description using an embedding model, and re-selects the entity if the score falls below a threshold $\alpha$. Inter-modal Alignment Verification computes a CLIP dot product between the entity description and the mention image, and stops if the score exceeds a threshold $\beta$. Visual Iterative Feedback, triggered only when the gates fail, extracts one textual clue per round from the image (OCR text, image caption, dense caption, or image tags) and feeds it back into the selector for the next iteration. The identity of the framework is this gating of visual evidence through reflection, rather than unconditional fusion of text and image.

What would settle it

Run the released I2CR code with the stated CLIP-ViT-bigG-14 checkpoint on WikiMEL and log the actual IAV scores; if no sample exceeds beta=31, or if the scores do not lie in [-1,1], then the inter-modal gate as written never triggers, and the reported IAV contribution would need to be explained by another part of the pipeline.

Watch

Extended reading notes

Core claim

The core claim is that multimodal entity linking is best done by a text-first pipeline with collaborative reflection: first select an entity from text alone, then check the selection with an intra-modal semantic consistency score and an inter-modal image-description alignment score, and only when those checks fail, feed visual clues back into the selector one at a time over several rounds. The paper argues that this avoids injecting image noise when text suffices and avoids information overload from combining all visual clues at once, and it reports state-of-the-art top-1 accuracy on all three benchmarks, with the largest gain on WikiDiverse (+5.1 points). The framework is trained only on WikiDiverse yet still achieves the best published numbers on WikiMEL and RichMEL, which the authors attribute to the generalization of the reflection loop rather than to in-domain training.

Load-bearing premise

The inter-modal verification step assumes a CLIP similarity score above 31 is reachable for matched entity-image pairs, yet the cited CLIP model returns normalized cosine scores in [-1,1]; if the threshold is not reachable, the IAV gate never fires as described, and the 2.1-point gain credited to it in the ablation must come from a different mechanism.

Editorial extensions

If this is right

If I2CR is correct, a text-first gated design beats unconditional image fusion on the three standard MEL benchmarks.
Iterative injection of one visual clue per round outperforms feeding all clues in a single pass, supporting the paper's information-overload argument.
The framework's gains transfer across open and closed LLMs, including Llama3-13B and GPT-4o, so the reflection loop is not tied to one backbone.
Training only on WikiDiverse and testing on WikiMEL and RichMEL indicates the reflection strategy generalizes beyond its training distribution.
Compared with UniMEL, I2CR is both more accurate and about 3.3 seconds faster per sample on average.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The suspected unreachable threshold in the inter-modal gate (see weakest assumption) suggests the published mechanism may not be what the code actually executes; if so, the reported IAV contribution could come from a different decision rule, and the framework's core insight would need to be re-described accordingly.
The order-insensitivity of visual clues (at most 0.3% difference across orders) suggests the benefit of the visual feedback loop comes mostly from having additional corroborating evidence at all, rather than from any particular clue type; a testable extension would vary the number of rounds and clue diversity while holding total evidence fixed.
The gating logic could transfer to other multimodal disambiguation tasks, such as visual question answering or multimodal named entity recognition, where irrelevant images are common; the paper does not explore this.
A natural testable extension is to replace the fixed thresholds $\alpha$ and $\beta$ with learned or per-sample adaptive gates, since the paper tunes them only on validation sets.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Worth a look: this is a pragmatic text-first MEL system that gets real gains on three benchmarks, with unusually careful ablations. The central idea—use text-only inference, verify with an intra-modal similarity check and a CLIP-based cross-modal check, and only then feed one visual clue at a time in iterated rounds—is new in this specific combination, and the experiments support the value of each piece. Table 2 has standard deviations, training is on one dataset and tested on three, and the VIF order experiment (Figure 5) shows performance is robust to which visual cue comes first. That is more than many systems papers give you.

The soft spot is Eq. (3). The paper sets beta=31 for the CLIP dot product between entity description and mention image but never says whether the CLIP features are L2-normalized or whether the logit scale is applied. If they use the default OpenCLIP encode_* with normalize=False, dot products can be large and 31 is plausible; if they normalize, cosine lies in [-1,1] and beta=31 is unreachable, making IAV dead code. The ablation credits IAV with a 2.1-point drop on WikiMEL, so this matters. Table 6 suggests the gate does fire at least sometimes, so it is probably an unstated convention rather than a broken module, but the authors must state it. The code link is up; I could not verify it from here.

The per-dataset alpha/beta tuning on validation sets is disclosed and standard, though it slightly weakens the "never trained on WikiMEL/RichMEL" generalization claim.

Bottom line: this is a legitimate candidate for a serious referee, but only after the authors clarify Eq. (3) and ideally release code. The empirical claim is plausible, the recipe is reusable, and the ablations give me confidence the effect is not one lucky threshold. I would send it to an expert reviewer with a request for the missing normalization detail, and I would hold off citing it until that is resolved.

Referee Report

2 major / 8 minor

Summary. The manuscript proposes I2CR, a pipeline for multimodal entity linking that relies primarily on textual evidence and invokes visual information only when needed. A fuzzy string matcher and a LoRA-fine-tuned Llama3-8B select a top-k candidate set; an embedding model then performs intra-modal consistency reflection, and a CLIP model performs inter-modal alignment verification against a threshold beta. When alignment fails, the framework enters a visual iterative feedback loop in which OCR, image captioning, dense captioning, and image tagging outputs are injected one per round into the LLM. On WikiMEL, WikiDiverse, and RichMEL the method reports top-1 accuracies of 92.2%, 91.6%, and 86.8%, improving over previous best methods by 3.2, 5.1, and 1.6 absolute points, with standard deviations below 0.3. The paper includes module ablations, visual-clue order experiments, cross-LLM experiments, and a runtime comparison.

Significance. The empirical claim, if it holds, is a strong and useful result: it shows that a text-first gated pipeline with iterative visual feedback can outperform end-to-end multimodal fusion on three standard MEL benchmarks, and it does so while training the LLM only on WikiDiverse and then generalizing to the other two datasets. The reported standard deviations, comprehensive ablations (Tables 3 and 4), cross-LLM analysis, and runtime measurements support the authors' effort to validate the framework. The main weakness is that the inter-modal alignment gate in Eq. (3) is not fully specified: the threshold beta=31 is only coherent under a particular (and unstated) convention for CLIP embedding normalization or scaling, which is load-bearing for the claimed gains of the IAV module.

major comments (2)

[§4.3, Eq. (3); §5.1; Table 3] The paper must specify whether Enc_T and Enc_I in Eq. (3) return L2-normalized CLIP embeddings and whether the CLIP logit scale (temperature) is applied. For the cited checkpoint (CLIP-ViT-bigG-14-laion2B-39B-b160k), using standard normalized features gives cosine similarities in [-1,1], making the stated threshold beta=31 unreachable by any input; Eq. (3) would then describe a gate that never fires, which is inconsistent with the 2.1-point WikiMEL drop when IAV is removed in Table 3. Under the unnormalized-embedding convention, dot products can exceed 31, but the score then depends on embedding magnitudes rather than alignment, and the scaling must be reported. Please report the exact encoder API used, the normalization or scaling applied, the distribution of Score_iav on the validation set, and the fraction of samples for which the gate fires.
[§4.3, §5.3, Table 3] The ablation 'w/o c' in Table 3 is ambiguous: if removing IAV causes all samples to proceed to VIF (or causes no samples to proceed), then the observed performance drop could reflect a change in control flow rather than the value of the alignment check itself. The paper should state the exact routing used in the 'w/o c' configuration and confirm that it is equivalent to bypassing the gate while keeping the subsequent VIF behavior unchanged. This is necessary for interpreting the contribution of IAV independently of the beta-threshold issue.

minor comments (8)

[§5.1, Table 1] Please fix the inconsistent capitalization of 'Wikidiverse' in Table 1 and in the surrounding text, which should read 'WikiDiverse' consistently.
[§5.1, Baselines paragraph] There are typos in the baselines paragraph: 'Unimel' should be 'UniMEL', and 'vriant' should be 'variant'.
[§5.3, Table 3 caption] The caption uses 'IA V' with a space; it should be 'IAV'.
[§5.1, Top-K accuracy metric] The paper should clarify how the 'nil' output (out-of-KG mentions) is incorporated into the Top-K accuracy computation, since Eq. (4) assumes that ground-truth entities appear among the candidate entities.
[§5.1, Implementation details] The number of training epochs for the LoRA fine-tuning is not reported; please add it to the implementation details.
[§4.2, Eq. (2)] The notation Norm(Embed(C_m) · Embed(C_e)) is ambiguous: it could mean cosine similarity after normalizing each embedding, or normalization of the product. Please write the cosine-similarity expression explicitly to match the surrounding text.
[§5.4, Figure 4] The y-axis of Figure 4 begins at 78, which visually compresses small differences; consider starting at 0 or indicating an axis break.
[§5.2, Table 6] The runtime comparison with UniMEL would be more interpretable if the paper reported the average number of LLM calls per sample and the fraction of samples accepted at the IAV gate, since the speed advantage depends on the gate firing early.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the I2CR performance claims rest on held-out test evaluation, and the only self-citations are contextual rather than load-bearing.

full rationale

I2CR is an empirical systems paper: its central claims are the top-1 accuracies in Table 2, obtained on the standard held-out test splits of WikiMEL, WikiDiverse, and RichMEL. I checked each load-bearing component for definitional or fitted circularity. The TES module is a fuzzy retriever plus a LoRA-fine-tuned Llama3-8B; the fine-tuning uses training-set ground truth, which is standard supervised learning and not a prediction reduced to its own input. The ICR and IAV gates use thresholds alpha and beta selected on the validation set; validation-based hyperparameter selection does not make the test-set prediction forced, and no equation defines the final accuracy in terms of those thresholds. The VIF order ('ocr-cap-den-tag') is likewise chosen on the validation set, and the reported accuracy is on the test set. The only self-references I found are contextual citations ([4], [16]) used to motivate the multimodal entity linking problem, not to license any uniqueness theorem or to supply an unverified premise for the headline result. The IAV beta=31 reproducibility concern raised in review is a potential implementation and reporting ambiguity about CLIP normalization, not circularity: even if the gate as written were unreachable, that would be a bug or documentation gap, not a derivation that reduces to its inputs. Overall, the central benchmark claims are self-contained against external evaluation, so the circularity burden is essentially zero.

Assumptions & free parameters 5 free parameters · 4 assumptions · 0 invented entities

The framework rests on a small number of hand-set thresholds and design choices: alpha and beta gate the two reflection modules, k bounds the candidate list, the visual-clue order is a validation-set choice, and the number of rounds is capped by the number of image-to-text tools. The main domain assumptions are that text-first selection can detect insufficiency, that a single CLIP threshold works across datasets, and that hosted image-to-text API outputs are stable. No invented entities are proposed.

free parameters (5)

alpha threshold for intra-modal consistency = 0.5 (WikiMEL), 0.8 (WikiDiverse), 0.75 (RichMEL)
Threshold in Eq. (2) determined on each dataset's validation set (Sec. 5.1); the pipeline's reselection decision depends on it.
beta threshold for inter-modal alignment = 31 for all datasets (stated)
Threshold in Eq. (3); likely inconsistent with normalized CLIP embeddings, so its effective value is unclear and possibly never satisfied.
candidate pool size k = 10
Top-k lexical candidates in TES (Sec. 5.1); affects the LLM input length and the recall ceiling.
image-to-text model order = ocr-cap-den-tag
Order chosen on the validation set (Sec. 5.4, Figure 5); authors report the order has minimal impact on results.
maximum iteration rounds = 5 (1 initial round plus 4 image-to-text models)
The loop stops after all image-to-text models are exhausted (Sec. 4.4); Figure 4 and case studies show five rounds.

assumptions (4)

domain assumption Text-first candidate selection with thresholds can detect when text alone is insufficient, and visual clues can then correct the link.
This is the central design premise of I2CR (Sec. 1 and Sec. 4); it is supported only by experiments on these benchmarks, not by an independent theory.
domain assumption A single CLIP cross-modal threshold beta can gate all three datasets in Eq. (3).
Section 5.1 sets beta=31 for all datasets; if embeddings are normalized, this assumption is mechanically questionable.
domain assumption Fine-tuning Llama3-8B on WikiDiverse and testing on WikiMEL and RichMEL is a valid transfer test, with alpha and beta tuned on each target validation split.
Section 5.1; the training set differs from the test sets, but per-dataset thresholds mean parts of the pipeline are adapted to each benchmark.
domain assumption Image-to-text outputs from hosted Azure APIs are stable enough for reproducible evaluation.
Section 5.1 uses OCR, captioning, dense captioning, and tagging APIs; no caching or version pinning is reported.

how reviews work

0 comments

Cite this review

Pith. "Pith review of I2CR: Intra- and Inter-modal Collaborative Reflections for Multimodal Entity Linking." pith.science (2026). https://pith.science/paper/4RRBYHPO

@misc{pith2026250802243,
  author       = {Pith},
  title        = {Pith review of: I2CR: Intra- and Inter-modal Collaborative Reflections for Multimodal Entity Linking},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/4RRBYHPO}},
  note         = {Machine review of arXiv:2508.02243}
}

read the original abstract

Multimodal entity linking plays a crucial role in a wide range of applications. Recent advances in large language model-based methods have become the dominant paradigm for this task, effectively leveraging both textual and visual modalities to enhance performance. Despite their success, these methods still face two challenges, including unnecessary incorporation of image data in certain scenarios and the reliance only on a one-time extraction of visual features, which can undermine their effectiveness and accuracy. To address these challenges, we propose a novel LLM-based framework for the multimodal entity linking task, called Intra- and Inter-modal Collaborative Reflections. This framework prioritizes leveraging text information to address the task. When text alone is insufficient to link the correct entity through intra- and inter-modality evaluations, it employs a multi-round iterative strategy that integrates key visual clues from various aspects of the image to support reasoning and enhance matching accuracy. Extensive experiments on three widely used public datasets demonstrate that our framework consistently outperforms current state-of-the-art methods in the task, achieving improvements of 3.2%, 5.1%, and 1.6%, respectively. Our code is available at https://github.com/ziyan-xiaoyu/I2CR/.

Figures

Figures reproduced from arXiv: 2508.02243 by the authors.

**Figure 2.** Our I2CR framework for the MEL task. An example of instruction data Input Dict Ground Truth Instruction Your task is to create matches between the mention and candidate entities to select the best-matched entity for the given mention. Choose only one option from candidate entities; no explanations or multiple choices. Must use this format for the final answer: <|ANSWER|>: <The entity name you selected>. Note that if… view at source ↗

**Figure 3.** Instruction example for target entity selection. [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 5.** Figure 5: Results of using visual clues in different orders. [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]

**Figure 6.** Figure 6: Case study. Blue for incorrect output, orange for correct output. “Not Reached” indicates that the corresponding [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

49 extracted references · 32 canonical work pages

[1]

Omar Adjali, Romaric Besançon, Olivier Ferret, Hervé Le Borgne, and Brigitte Grau. 2020. Multimodal entity linking for tweets. In European Conference on Information Retrieval. Springer, 463–478

work page 2020
[2]

Afra Feyza Akyürek, Ekin Akyürek, Aman Madaan, Ashwin Kalyan, Peter Clark, Derry Wijaya, and Niket Tandon. 2023. RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputs. arXiv preprint arXiv:2305.08844 (2023)

arXiv 2023
[3]

Edoardo Barba, Luigi Procopio, and Roberto Navigli. 2022. ExtEnD: Extractive entity disambiguation. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . 2478–2488

work page 2022
[4]

Lihan Chen, Tinghui Zhu, Jingping Liu, Jiaqing Liang, and Yanghua Xiao. 2023. End-to-end entity linking with hierarchical reinforcement learning. InProceedings of the AAAI Conference on Artificial Intelligence , Vol. 37. 4173–4181

work page 2023
[5]

Zhuo Chen, Yichi Zhang, Yin Fang, Yuxia Geng, Lingbing Guo, Xiang Chen, Qian Li, Wen Zhang, Jiaoyan Chen, Yushan Zhu, et al. 2024. Knowledge graphs meet multi-modal learning: A comprehensive survey. arXiv preprint arXiv:2402.05391 (2024)

arXiv 2024
[6]

Nicola De Cao, Gautier Izacard, Sebastian Riedel, and Fabio Petroni. 2020. Au- toregressive entity retrieval. arXiv preprint arXiv:2010.00904 (2020)

arXiv 2020
[7]

Yashar Deldjoo, Markus Schedl, Paolo Cremonesi, and Gabriella Pasi. 2020. Rec- ommender systems leveraging multimedia content. ACM Computing Surveys (CSUR) 53, 5 (2020), 1–38

work page 2020
[8]

Zhang Dongjie and Longtao Huang. 2022. Multimodal Knowledge Learning for Named Entity Disambiguation. In Findings of the Association for Computational Linguistics: EMNLP 2022. 3160–3169

work page 2022

Show all 49 references

[9]

Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. 2024. The llama 3 herd of models. arXiv preprint arXiv:2407.21783 (2024)

2024 arXiv
[10]

Elozino Egonmwan and Yllias Chali. 2019. Transformer and seq2seq model for paraphrase generation. In Proceedings of the 3rd Workshop on Neural Generation and Translation. 249–255

2019
[11]

Jingru Gan, Jinchang Luo, Haiwei Wang, Shuhui Wang, Wei He, and Qingming Huang. 2021. Multimodal entity linking: a new dataset and a baseline. In Pro- ceedings of the 29th ACM international conference on multimedia . 993–1001

2021
[12]

Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685 (2021)

2021 arXiv
[13]

Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of naacL-HLT, Vol. 1. Minneapolis, Minnesota

2019
[14]

Tuan Manh Lai, Heng Ji, and ChengXiang Zhai. 2022. Improving candidate retrieval with entity profile generation for wikidata entity linking. arXiv preprint arXiv:2202.13404 (2022)

2022 arXiv
[15]

Yann LeCun, Yoshua Bengio, and Geoffrey Hinton. 2015. Deep learning. nature 521, 7553 (2015), 436–444

2015
[16]

Jingping Liu, Mingchuan Zhang, Weichen Li, Chao Wang, Shuang Li, Haiyun Jiang, Sihang Jiang, Yanghua Xiao, and Yunwen Chen. 2024. Beyond entities: A large-scale multi-modal knowledge graph with triplet fact grounding. In Proceed- ings of the AAAI conference on artificial intel...

2024
[17]

Qi Liu, Yongyi He, Tong Xu, Defu Lian, Che Liu, Zhi Zheng, and Enhong Chen
[18]

Yinhan Liu. 2019. Roberta: A robustly optimized bert pretraining approach.arXiv preprint arXiv:1907.11692 364 (2019)

2019 arXiv
[19]

Shayne Longpre, Kartik Perisetla, Anthony Chen, Nikhil Ramesh, Chris DuBois, and Sameer Singh. 2021. Entity-based knowledge conflicts in question answering. arXiv preprint arXiv:2109.05052 (2021)

2021 arXiv
[20]

Pengfei Luo, Tong Xu, Shiwei Wu, Chen Zhu, Linli Xu, and Enhong Chen. 2023. Multi-grained multimodal interaction network for entity linking. In Proceedings of the 29th ACM SIGKDD Conference on Knowledge Discovery and Data Mining . 1583–1594

2023
[21]

Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. 2023. Self-refine: Iterative refinement with self-feedback. arXiv preprint arXiv:2303.17651 (2023)

2023 arXiv
[22]

Rui Meng, Ye Liu, Shafiq Rayhan Joty, Caiming Xiong, Yingbo Zhou, and Semih Yavuz. 2024. Sfrembedding-mistral: enhance text retrieval with transfer learning. Salesforce AI Research Blog 3 (2024)

2024
[23]

Ning Miao, Yee Whye Teh, and Tom Rainforth. 2023. Selfcheck: Using llms to zero-shot check their own step-by-step reasoning.arXiv preprint arXiv:2308.00436 (2023)

2023 arXiv
[24]

Seungwhan Moon, Leonardo Neves, and Vitor Carvalho. 2018. Multimodal named entity disambiguation for noisy social media posts. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 2000–2008

2018
[25]

Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang. 2023. Automatically correcting large language models: Surveying the landscape of diverse self-correction strategies. arXiv preprint arXiv:2308.03188 (2023)

2023 arXiv
[26]

Baolin Peng, Michel Galley, Pengcheng He, Hao Cheng, Yujia Xie, Yu Hu, Qiuyuan Huang, Lars Liden, Zhou Yu, Weizhu Chen, et al. 2023. Check your facts and try again: Improving large language models with external knowledge and automated feedback. arXiv preprint arXiv:2302.12813 (2023)

2023 arXiv
[27]

Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learni...

2021
[28]

Sanket Shah, Anand Mishra, Naganand Yadati, and Partha Pratim Talukdar. 2019. Kvqa: Knowledge-aware visual question answering. In Proceedings of the AAAI conference on artificial intelligence , Vol. 33. 8876–8884

2019
[29]

Wei Shen, Yuhan Li, Yinan Liu, Jiawei Han, Jianyong Wang, and Xiaojie Yuan
[30]

Wei Shen, Jianyong Wang, and Jiawei Han. 2014. Entity linking with a knowledge base: Issues, techniques, and solutions. IEEE Transactions on Knowledge and Data Engineering 27, 2 (2014), 443–460

2014
[31]

Senbao Shi, Zhenran Xu, Baotian Hu, and Min Zhang. 2024. Generative Multi- modal Entity Linking. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024). 7654–7665

2024
[32]

Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik R Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learning. In Thirty-seventh Conference on Neural Information Processing Systems

2023
[33]

Avirup Sil and Alexander Yates. 2013. Re-ranking for joint named-entity recog- nition and linking. In Proceedings of the 22nd ACM international conference on Information & Knowledge Management . 2369–2374

2013
[34]

Shezheng Song, Shasha Li, Shan Zhao, Xiaopeng Li, Chengyu Wang, Jie Yu, Jun Ma, Tianwei Yan, Bin Ji, and Xiaoguang Mao. 2024. DWE+: Dual-Way Matching Enhanced Framework for Multimodal Entity Linking. arXiv preprint arXiv:2404.04818 (2024)

2024 arXiv
[35]

Shezheng Song, Shan Zhao, Chengyu Wang, Tianwei Yan, Shasha Li, Xiaoguang Mao, and Meng Wang. 2024. A dual-way enhanced framework from text matching point of view for multimodal entity linking. InProceedings of the AAAI Conference on Artificial Intelligence, Vol. 38. 19008–19016

2024
[36]

Anuroop Sriram, Heewoo Jun, Sanjeev Satheesh, and Adam Coates. 2017. Cold fusion: Training seq2seq models together with language models. arXiv preprint arXiv:1708.06426 (2017)

2017 arXiv
[37]

Meng Wang, Haofen Wang, Guilin Qi, and Qiushuo Zheng. 2020. Richpedia: a large-scale, comprehensive multi-modal knowledge graph. Big Data Research 22 (2020), 100159

2020
[38]

Peng Wang, Jiangheng Wu, and Xiaohang Chen. 2022. Multimodal entity linking with gated hierarchical fusion and contrastive training. In Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval. 938–948

2022
[39]

Xuwu Wang, Junfeng Tian, Min Gui, Zhixu Li, Rui Wang, Ming Yan, Lihan Chen, and Yanghua Xiao. 2022. WikiDiverse: A Multimodal Entity Linking Dataset with Diversified Contextual Topics and Entity Types. In Proceedings of the 60th Annual Meeting of the Association for Computatio...

2022
[40]

Sean Welleck, Ximing Lu, Peter West, Faeze Brahman, Tianxiao Shen, Daniel Khashabi, and Yejin Choi. 2022. Generating sequences by learning to self-correct. arXiv preprint arXiv:2211.00053 (2022)

2022 arXiv
[41]

Ledell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, and Luke Zettle- moyer. 2020. Scalable Zero-shot Entity Linking with Dense Entity Retrieval. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 6397–6407

2020
[42]

Shangyu Xing, Fei Zhao, Zhen Wu, Chunhui Li, Jianbing Zhang, and Xinyu Dai. 2023. DRIN: Dynamic Relation Interactive Network for Multimodal Entity Linking. In Proceedings of the 31st ACM International Conference on Multimedia . 3599–3608

2023
[43]

Zhenran Xu, Yulin Chen, Senbao Shi, and Baotian Hu. 2022. Enhancing entity linking with contextualized entity embeddings. In CCF International Conference on Natural Language Processing and Chinese Computing . Springer, 228–239

2022
[44]

An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al . 2024. Qwen2. 5 MM ’25, October 27–31, 2025, Dublin, Ireland Ziyan Liu et al. technical report. arXiv preprint arXiv:2412.15115 (2024)

2024 arXiv
[45]

Barry Menglong Yao, Yu Chen, Qifan Wang, Sijia Wang, Minqian Liu, Zhiyang Xu, Licheng Yu, and Lifu Huang. 2023. AMELI: Enhancing Multimodal Entity Linking with Fine-Grained Attributes. arXiv preprint arXiv:2305.14725 (2023)

2023 arXiv
[46]

Sheng Zhang, Hao Cheng, Jianfeng Gao, and Hoifung Poon. 2022. Optimizing bi-encoder for named entity recognition via contrastive learning. arXiv preprint arXiv:2208.14565 (2022)

2022 arXiv
[47]

Zefeng Zhang, Jiawei Sheng, Chuang Zhang, Yunzhi Liang, Wenyuan Zhang, Siqi Wang, and Tingwen Liu. 2024. Optimal Transport Guided Correlation Assignment for Multimodal Entity Linking. arXiv preprint arXiv:2406.01934 (2024)

2024 arXiv
[2021]

IEEE Trans- actions on Knowledge and Data Engineering 35, 3 (2021), 2556–2578

Entity linking meets deep learning: Techniques and solutions. IEEE Trans- actions on Knowledge and Data Engineering 35, 3 (2021), 2556–2578

2021
[2024]

In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management

Unimel: A unified framework for multimodal entity linking with large language models. In Proceedings of the 33rd ACM International Conference on Information and Knowledge Management . 1909–1919

1909

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

Omar Adjali, Romaric Besançon, Olivier Ferret, Hervé Le Borgne, and Brigitte Grau. 2020. Multimodal entity linking for tweets. In European Conference on Information Retrieval. Springer, 463–478

work page 2020

[2] [2]

Afra Feyza Akyürek, Ekin Akyürek, Aman Madaan, Ashwin Kalyan, Peter Clark, Derry Wijaya, and Niket Tandon. 2023. RL4F: Generating Natural Language Feedback with Reinforcement Learning for Repairing Model Outputs. arXiv preprint arXiv:2305.08844 (2023)

arXiv 2023

[3] [3]

Edoardo Barba, Luigi Procopio, and Roberto Navigli. 2022. ExtEnD: Extractive entity disambiguation. InProceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) . 2478–2488

work page 2022

[4] [4]

Lihan Chen, Tinghui Zhu, Jingping Liu, Jiaqing Liang, and Yanghua Xiao. 2023. End-to-end entity linking with hierarchical reinforcement learning. InProceedings of the AAAI Conference on Artificial Intelligence , Vol. 37. 4173–4181

work page 2023

[5] [5]

Zhuo Chen, Yichi Zhang, Yin Fang, Yuxia Geng, Lingbing Guo, Xiang Chen, Qian Li, Wen Zhang, Jiaoyan Chen, Yushan Zhu, et al. 2024. Knowledge graphs meet multi-modal learning: A comprehensive survey. arXiv preprint arXiv:2402.05391 (2024)

arXiv 2024

[6] [6]

Nicola De Cao, Gautier Izacard, Sebastian Riedel, and Fabio Petroni. 2020. Au- toregressive entity retrieval. arXiv preprint arXiv:2010.00904 (2020)

arXiv 2020

[7] [7]

Yashar Deldjoo, Markus Schedl, Paolo Cremonesi, and Gabriella Pasi. 2020. Rec- ommender systems leveraging multimedia content. ACM Computing Surveys (CSUR) 53, 5 (2020), 1–38

work page 2020

[8] [8]

Zhang Dongjie and Longtao Huang. 2022. Multimodal Knowledge Learning for Named Entity Disambiguation. In Findings of the Association for Computational Linguistics: EMNLP 2022. 3160–3169

work page 2022