pith. sign in

arxiv: 2504.04065 · v2 · pith:ZAGTGH5Xnew · submitted 2025-04-05 · 💻 cs.CV · cs.IR· cs.MM

Enabling Collaborative Parametric Knowledge Calibration for Retrieval-Augmented Vision Question Answering

Jiaqi Deng , Kaize Shi , Zonghan Wu , Huan Huo , Dingxian Wang , Guandong Xu This is my paper

Pith reviewed 2026-05-22 20:39 UTC · model grok-4.3

classification 💻 cs.CV cs.IRcs.MM
keywords KB-VQAretrieval-augmented generationparametric knowledge calibrationcollaborative trainingmultimodal VQAlate interactionreflective answering
0
0 comments X

The pith

A unified framework lets retriever and generator share parametric knowledge bidirectionally in KB-VQA.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper aims to establish that keeping knowledge retrieval and answer generation as separate modules with limited interaction creates a bottleneck in knowledge-based vision question answering. It proposes a unified retrieval-augmented framework that enables the two components to collaboratively calibrate and share their parametric knowledge throughout training and inference. The method adds late interaction for finer multimodal understanding and a reflective-answering step so the model can assess its own knowledge limits. A sympathetic reader would care because the approach promises to adapt general multimodal models more effectively to questions that require external knowledge, yielding measurable accuracy gains.

Core claim

The proposed unified retrieval-augmented VQA framework with collaborative parametric knowledge calibration enables the retriever and generator to collaboratively enhance and share their parametric knowledge during both training and inference, achieving a significant 4.7% improvement in answering accuracy and an average 7.5% boost in base MLLMs' VQA performance.

What carries the argument

Collaborative parametric knowledge calibration, a unified training mechanism that permits bidirectional sharing of parameters between retrieval and generation modules.

If this is right

  • Retriever and generator mutually refine each other's parametric knowledge.
  • General multimodal models adapt more effectively to fine-grained knowledge-intensive tasks.
  • Late interaction improves matching between questions and external documents.
  • Reflective answering lets the model explicitly check and adjust its knowledge boundaries.
  • The combined system reaches competitive results against current state-of-the-art KB-VQA models.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same calibration pattern could apply to retrieval-augmented systems outside visual question answering.
  • Better internal knowledge sharing might lower reliance on very large external knowledge bases.
  • Testing the method on questions that require conflicting or ambiguous external knowledge would reveal whether the reflective step scales.

Load-bearing premise

The assumption that limited interaction between separate retrieval and generation modules is the main bottleneck limiting performance in KB-VQA.

What would settle it

An ablation that removes the bidirectional knowledge-sharing steps and measures whether the reported 4.7% accuracy gain disappears.

Figures

Figures reproduced from arXiv: 2504.04065 by Dingxian Wang, Guandong Xu, Huan Huo, Jiaqi Deng, Kaize Shi, Zonghan Wu.

Figure 1
Figure 1. Figure 1: A comparison between the large-model-based [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: An overview of the proposed Unified Retrieval-Augmented Vision Question Answering framework (UniRVQA). [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Retrieval performance variation with respect to the [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Ablation study on the self-reflection mechanism. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Qualitative results on four cases. The left two cases are from OK-VQA and the right two cases are from InfoSeek. [PITH_FULL_IMAGE:figures/full_fig_p008_5.png] view at source ↗
read the original abstract

Knowledge-based Vision Question Answering (KB-VQA) systems address complex visual-grounded questions with knowledge retrieved from external knowledge bases. The tasks of knowledge retrieval and answer generation tasks both necessitate precise multimodal understanding of question context and external knowledge. However, existing methods treat these two stages as separate modules with limited interaction during training, which hinders bi-directional parametric knowledge sharing, ultimately leading to suboptimal performance. To fully exploit the cross-task synergy in KB-VQA, we propose a unified retrieval-augmented VQA framework with collaborative parametric knowledge calibration. The proposed framework can effectively adapt general multimodal pre-trained models for fine-grained, knowledge-intensive tasks while enabling the retriever and generator to collaboratively enhance and share their parametric knowledge during both training and inference. To enhance fine-grained understanding of questions and external documents, we also integrate late interaction mechanism into the proposed training framework. Additionally, we introduce a reflective-answering mechanism that allows the model to explicitly evaluate and refine its knowledge boundary. Our approach achieves competitive performance against state-of-the-art models, delivering a significant 4.7\% improvement in answering accuracy, and brings an average 7.5\% boost in base MLLMs' VQA performance.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 2 minor

Summary. The paper proposes a unified retrieval-augmented VQA framework for knowledge-based vision question answering (KB-VQA) that incorporates collaborative parametric knowledge calibration between retriever and generator modules, along with a late interaction mechanism and a reflective-answering component. It claims this enables bidirectional parametric knowledge sharing during training and inference, yielding a 4.7% improvement in answering accuracy and an average 7.5% boost to base MLLMs' VQA performance over state-of-the-art models.

Significance. If the reported gains can be shown through controlled experiments to stem specifically from the collaborative calibration rather than other framework additions, the work would offer a practical approach to improving cross-task synergy in retrieval-augmented multimodal systems and could inform designs that move beyond separately trained retrieval and generation stages.

major comments (3)
  1. [Experiments] Experiments section: The central claim attributes the 4.7% accuracy gain and 7.5% MLLM boost to collaborative parametric knowledge calibration, yet no ablation studies are described that compare the full framework against variants containing late interaction and reflective-answering but without the bidirectional knowledge-sharing optimization. This leaves the causal contribution of the proposed calibration unverified.
  2. [Method] Method section: The description of collaborative enhancement and sharing of parametric knowledge between retriever and generator lacks explicit loss terms, joint optimization objectives, or training schedules that would demonstrate how conflicts are avoided during bidirectional updates.
  3. [Results] Results section: Performance numbers are reported without accompanying details on the number of random seeds, standard deviations across runs, or statistical significance tests, which are required to substantiate the reliability of the stated 4.7% and 7.5% improvements.
minor comments (2)
  1. [Abstract] The abstract would be strengthened by briefly naming the primary datasets and main baselines used to obtain the reported numbers.
  2. Notation for the unified framework components (e.g., how late interaction is integrated into the calibration process) could be made more consistent across sections.

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for the constructive feedback. We address each major comment below and will revise the manuscript to incorporate the suggested improvements.

read point-by-point responses

  1. Referee: [Experiments] Experiments section: The central claim attributes the 4.7% accuracy gain and 7.5% MLLM boost to collaborative parametric knowledge calibration, yet no ablation studies are described that compare the full framework against variants containing late interaction and reflective-answering but without the bidirectional knowledge-sharing optimization. This leaves the causal contribution of the proposed calibration unverified.

    Authors: We agree that ablation studies isolating the bidirectional knowledge-sharing optimization are necessary to substantiate the central claim. We will add these experiments in the revised manuscript, comparing the full framework against variants that retain late interaction and reflective-answering but remove the collaborative calibration component. revision: yes

  2. Referee: [Method] Method section: The description of collaborative enhancement and sharing of parametric knowledge between retriever and generator lacks explicit loss terms, joint optimization objectives, or training schedules that would demonstrate how conflicts are avoided during bidirectional updates.

    Authors: We will expand the Method section to explicitly detail the loss terms for collaborative parametric knowledge calibration, the joint optimization objectives, and the training schedule, including how bidirectional updates are coordinated to avoid conflicts. revision: yes

  3. Referee: [Results] Results section: Performance numbers are reported without accompanying details on the number of random seeds, standard deviations across runs, or statistical significance tests, which are required to substantiate the reliability of the stated 4.7% and 7.5% improvements.

    Authors: We will update the Results section to report the number of random seeds, standard deviations across runs, and statistical significance tests supporting the 4.7% and 7.5% improvements. revision: yes

Circularity Check

0 steps flagged

No circularity; empirical claims rest on reported performance gains, not derivations or self-referential definitions

full rationale

The paper describes a proposed training framework for KB-VQA and reports measured accuracy improvements (4.7% and 7.5%) as experimental outcomes. No equations, loss functions, or mathematical derivations appear in the provided text. The central claims concern empirical synergy from the unified framework rather than any quantity defined in terms of fitted parameters or reduced by construction to prior inputs. No self-citation chains or uniqueness theorems are invoked in the abstract or visible structure. The result is therefore self-contained against external benchmarks and receives the default non-circularity finding.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract-only review supplies no explicit free parameters, axioms, or invented entities; the framework description implies standard multimodal pre-training assumptions and empirical training choices whose details are unavailable.

pith-pipeline@v0.9.0 · 5755 in / 1204 out tokens · 51358 ms · 2026-05-22T20:39:53.216165+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

49 extracted references · 49 canonical work pages

  1. [1]

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katie Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, Andrew Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkowski,...

    work page 2022

  2. [2]

    Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. 2018. Bottom-Up and Top-Down Attention for Image Captioning and Visual Question Answering. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition . IEEE, 6077–6086

    work page 2018

  3. [3]

    Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin...

    work page 2020

  4. [4]

    Wenhu Chen, Hexiang Hu, Xi Chen, Pat Verga, and William W. Cohen. 2022. MuRAG: Multimodal Retrieval-Augmented Generator for Open Question An- swering over Images and Text. (10 2022)

    work page 2022

  5. [5]

    Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, Alexander Kolesnikov, Joan Puigcerver, Nan Ding, Keran Rong, Hassan Akbari, Gaurav Mishra, Linting Xue, Ashish Thapliyal, James Bradbury, Weicheng Kuo, Mojtaba Seyedhosseini, Chao Jia, Burcu Karagol Ayan, Carl...

    work page 2023

  6. [6]

    Yang Chen, Hexiang Hu, Yi Luan, Haitian Sun, Soravit Changpinyo, Alan Ritter, and Ming-Wei Chang. 2023. Can Pre-trained Vision and Language Models Answer Visual Information-Seeking Questions? (2 2023)

    work page 2023

  7. [7]

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. 2023. InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning. (5 2023)

    work page 2023

  8. [8]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. 2020. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. (10 2020)

    work page 2020

  9. [9]

    Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdh- ery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint, Klaus Greff, Andy Zeng, Igor Mordatch, and Pete Florence. 2023. PaLM-E: An E...

    work page 2023

  10. [10]

    Feng Gao, Qing Ping, Govind Thattai, Aishwarya Reganti, Ying Nian Wu, and Prem Natarajan. 2022. Transform-Retrieve-Generate: Natural Language-Centric Outside-Knowledge Visual Question Answering. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) . IEEE, 5057–5067

    work page 2022

  11. [11]

    Liangke Gui, Borui Wang, Qiuyuan Huang, Alex Hauptmann, Yonatan Bisk, and Jianfeng Gao. 2021. KAT: A Knowledge Augmented Transformer for Vision-and- Language. (12 2021)

    work page 2021

  12. [12]

    Yangyang Guo, Liqiang Nie, Yongkang Wong, Yibing Liu, Zhiyong Cheng, and Mohan Kankanhalli. 2022. A Unified End-to-End Retriever-Reader Framework for Knowledge-based VQA. In Proceedings of the 30th ACM International Conference on Multimedia. ACM, New York, NY, USA, 2061–2069

    work page 2022

  13. [13]

    Hexiang Hu, Yi Luan, Yang Chen, Urvashi Khandelwal, Mandar Joshi, Kenton Lee, Kristina Toutanova, and Ming-Wei Chang. 2023. Open-domain Visual En- tity Recognition: Towards Recognizing Millions of Wikipedia Entities. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV). IEEE, 12031–12041

    work page 2023

  14. [14]

    Ross, and Alireza Fathi

    Ziniu Hu, Ahmet Iscen, Chen Sun, Zirui Wang, Kai-Wei Chang, Yizhou Sun, Cordelia Schmid, David A. Ross, and Alireza Fathi. 2022. REVEAL: Retrieval- Augmented Visual-Language Pre-Training with Multi-Source Multimodal Knowl- edge Memory. (12 2022)

    work page 2022

  15. [15]

    Gautier Izacard and Edouard Grave. 2021. Leveraging Passage Retrieval with Generative Models for Open Domain Question Answering. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume. Association for Computational Linguistics, Stroudsburg, PA, USA, 874–880

    work page 2021

  16. [16]

    Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. Dense Passage Retrieval for Open- Domain Question Answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) . Association for Computational Linguistics, Stroudsburg, PA, USA, 6769–6781

    work page 2020

  17. [17]

    Omar Khattab and Matei Zaharia. 2020. ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT. (4 2020)

    work page 2020

  18. [18]

    Guohao Li, Xin Wang, and Wenwu Zhu. 2020. Boosting Visual Question Answer- ing with Context-aware Knowledge Aggregation. In Proceedings of the 28th ACM International Conference on Multimedia . ACM, New York, NY, USA, 1227–1235

    work page 2020

  19. [19]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. BLIP-2: Boot- strapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models. (1 2023)

    work page 2023

  20. [20]

    Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. 2022. BLIP: Bootstrapping Language-Image Pre-training for Unified Vision-Language Understanding and Generation. (1 2022)

    work page 2022

  21. [21]

    Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, Yejin Choi, and Jianfeng Gao

    work page

  22. [22]

    Oscar: Object-Semantics Aligned Pre-training for Vision-Language Tasks. 121–137

    work page

  23. [23]

    Lawrence Zitnick, and Piotr Dollár

    Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, and Piotr Dollár

    work page

  24. [24]

    (5 2014)

    Microsoft COCO: Common Objects in Context. (5 2014)

    work page 2014

  25. [25]

    Weizhe Lin and Bill Byrne. 2022. Retrieval Augmented Visual Question Answering with Outside Knowledge. (10 2022)

    work page 2022

  26. [26]

    Weizhe Lin, Jinghong Chen, Jingbiao Mei, Alexandru Coca, and Bill Byrne. 2023. Fine-grained Late-interaction Multi-modal Retrieval for Retrieval Augmented Visual Question Answering. (9 2023)

    work page 2023

  27. [27]

    Weizhe Lin, Jingbiao Mei, Jinghong Chen, and Bill Byrne. 2024. PreFLMR: Scaling Up Fine-Grained Late-Interaction Multi-modal Retrievers. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics

    work page 2024

  28. [28]

    Yuanze Lin, Yujia Xie, Dongdong Chen, Yichong Xu, Chenguang Zhu, and Lu Yuan. 2022. REVIVE: Regional Visual Representation Matters in Knowledge- Based Visual Question Answering. (6 2022)

    work page 2022

  29. [29]

    Haotian Liu, Kilho Son, Jianwei Yang, Ce Liu, Jianfeng Gao, Yong Jae Lee, and Chunyuan Li. 2023. Learning Customized Visual Models with Retrieval- Augmented Knowledge. (1 2023)

    work page 2023

  30. [30]

    Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen. 2024. DoRA: Weight- Decomposed Low-Rank Adaptation. (2 2024)

    work page 2024

  31. [31]

    Ilya Loshchilov and Frank Hutter. 2017. Decoupled Weight Decay Regularization. (11 2017)

    work page 2017

  32. [32]

    Man Luo, Yankai Zeng, Pratyay Banerjee, and Chitta Baral. 2021. Weakly- Supervised Visual-Retriever-Reader for Knowledge-based Question Answering. (9 2021)

    work page 2021

  33. [33]

    Ziyu Ma, Shutao Li, Bin Sun, Jianfei Cai, Zuxiang Long, and Fuyan Ma. 2024. GeReA: Question-Aware Prompt Captions for Knowledge-based Visual Question Answering. (2 2024)

    work page 2024

  34. [34]

    Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Daniel Khashabi, and Hannaneh Hajishirzi. 2023. When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Ling...

    work page 2023

  35. [35]

    Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi

    work page

  36. [36]

    (5 2019)

    OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge. (5 2019)

    work page 2019

  37. [37]

    Sewon Min, Kalpesh Krishna, Xinxi Lyu, Mike Lewis, Wen-tau Yih, Pang Koh, Mohit Iyyer, Luke Zettlemoyer, and Hannaneh Hajishirzi. 2023. FActScore: Fine- grained Atomic Evaluation of Factual Precision in Long Form Text Generation. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Association for Computational Lingui...

    work page 2023

  38. [38]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. 2021. Learning Transferable Visual Models From Natural Language Supervision. (2 2021)

    work page 2021

  39. [39]

    Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2019. Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer. (10 2019)

    work page 2019

  40. [40]

    Alireza Salemi, Juan Altmayer Pizzorno, and Hamed Zamani. 2023. A Symmetric Dual Encoding Dense Retrieval Framework for Knowledge-Intensive Visual Ques- tion Answering. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval . ACM, New York, NY, USA, 110–120

    work page 2023

  41. [41]

    Keshav Santhanam, Omar Khattab, Christopher Potts, and Matei Zaharia. 2022. PLAID: An Efficient Engine for Late Interaction Retrieval. (5 2022)

    work page 2022

  42. [42]

    Baoguang Shi, Xiang Bai, and Cong Yao. 2017. An End-to-End Trainable Neural Network for Image-Based Sequence Recognition and Its Application to Scene Text Recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence 39, 11 (11 2017), 2298–2304

    work page 2017

  43. [43]

    Shamane Siriwardhana, Rivindu Weerasekera, Elliott Wen, Tharindu Kalu- arachchi, Rajib Rana, and Suranga Nanayakkara. 2023. Improving the Domain Adaptation of Retrieval Augmented Generation (RAG) Models for Open Domain Question Answering. Transactions of the Association for Computational Linguistics 11 (1 2023), 1–17

    work page 2023

  44. [44]

    Robyn Speer, Joshua Chin, and Catherine Havasi. 2016. ConceptNet 5.5: An Open Multilingual Graph of General Knowledge. (12 2016)

    work page 2016

  45. [45]

    Denny Vrandečić and Markus Krötzsch. 2014. Wikidata. Commun. ACM 57, 10 (9 2014), 78–85

    work page 2014

  46. [46]

    Jialin Wu, Jiasen Lu, Ashish Sabharwal, and Roozbeh Mottaghi. 2022. Multi- Modal Answer Validation for Knowledge-Based VQA. Proceedings of the AAAI Conference on Artificial Intelligence 36, 3 (6 2022), 2712–2721

    work page 2022

  47. [47]

    Zhengyuan Yang, Zhe Gan, Jianfeng Wang, Xiaowei Hu, Yumao Lu, Zicheng Liu, and Lijuan Wang. 2022. An Empirical Study of GPT-3 for Few-Shot Knowledge- Based VQA. Proceedings of the AAAI Conference on Artificial Intelligence 36, 3 (6 2022), 3081–3089

    work page 2022

  48. [48]

    Michihiro Yasunaga, Armen Aghajanyan, Weijia Shi, Rich James, Jure Leskovec, Percy Liang, Mike Lewis, Luke Zettlemoyer, and Wen-tau Yih. 2022. Retrieval- Augmented Multimodal Language Modeling. (11 2022)

    work page 2022

  49. [49]

    Zhou Yu, Xuecheng Ouyang, Zhenwei Shao, Meng Wang, and Jun Yu. 2023. Prophet: Prompting Large Language Models with Complementary Answer Heuristics for Knowledge-based Visual Question Answering. (3 2023). Received 20 February 2007; revised 12 March 2009; accepted 5 June 2009

    work page 2023