Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey

Haohao Luo; Jiayi Kuang; Jingyou Xie; Ronghao Li; Xianfeng Cheng; Xika Lin; Yinghui Li; Ying Shen; Zhe Xu

REVIEW 4 major objections 5 minor 1 cited by

Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey

T0 review · 4 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read This survey organizes visual question answering into understanding and inference, with multimodal large language models as the field's newest stage.

desk verdict Useful taxonomy, but contaminated references and a mixed-protocol performance table make it unreliable as a reference. read the letter →

arxiv 2411.17558 v1 pith:I5OBOOBF submitted 2024-11-26 cs.CL cs.CV

Jiayi Kuang , Jingyou Xie , Haohao Luo , Ronghao Li , Zhe Xu , Xianfeng Cheng , Yinghui Li , Xika Lin

show 1 more author

Ying Shen

This is my paper · ORCID

classification cs.CLcs.CV

keywords visualquestionansweringmultimodallargelanguagemodelsvision-languagepretrainingfusionknowledgereasoningVQAdatasetsevaluationmetricssurvey

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This survey sets out to establish a two-part synthesis of Visual Question Answering (VQA), the task of producing an answer $A$ to a question $Q$ about an image $V$. The two parts are natural language understanding of images and text, and natural language inference over image-question information. It argues that the field has moved from separate feature extractors and fusion modules, through vision-language pretraining, to multimodal large language models (MLLMs)—large language models extended to accept visual input—that add instruction tuning, in-context learning, and chain-of-thought reasoning. The survey also organizes knowledge-based VQA into knowledge sources, extraction methods, and one-hop versus multi-hop reasoning, and it catalogues datasets, MLLM benchmarks, and evaluation metrics. A sympathetic reader would use this as a current reference for choosing models, datasets, and metrics and for locating open problems such as data bias, explainability, and answer generation.

What carries the argument

The carrying device is the paper's taxonomy of the VQA task (its Fig. 2), which splits the field along a perception-to-cognition axis: natural language understanding of image and text versus natural language inference, with knowledge reasoning and MLLM reasoning as sub-branches. This taxonomy does the work of the survey's argument: it gives every model, fusion module, knowledge source, and dataset a slot, so that the claimed up-to-date synthesis becomes a single map rather than a chronological list. The paper's secondary machinery is the distinction between internal and external knowledge, which organizes the knowledge-reasoning section.

What would settle it

Check each bibliographic entry for a corresponding in-text citation: entries such as a Clifford algebra software package, a LaTeX style manual, or sensor-network surveys that appear in the references but never in the body would show the bibliography is not load-bearing. Separately, inspect Table 6 to see whether each row states its dataset split and shot setting (zero-shot, few-shot, or fine-tuned); rows that mix settings without labels would invalidate the reported comparisons.

Watch

Extended reading notes

Core claim

The paper's central claim is that VQA is best understood through one taxonomy: natural language understanding of images and text on the perception side, and natural language inference on the cognition side, with multimodal large language models as the newest stage of both. On the understanding side, the survey traces visual and textual feature extraction, embedding improvements, fusion by vector operations, attention and graph neural networks, and dual-stream versus single-stream pretraining. On the inference side, it distinguishes internal from external knowledge, entity-based from feature-based extraction, and conventional, one-hop, and multi-hop reasoning, including memory-based, graph-based, and implicit methods. The same taxonomy then places current MLLM techniques—instruction tuning, in-context learning, multimodal chain-of-thought, and tool-aided reasoning—as the latest answer to both perceptual and cognitive demands, with datasets and benchmarks as the evaluation layer.

Load-bearing premise

The survey's usefulness depends on the assumption that its bibliography and performance tables accurately represent the state of the field; if many listed references are irrelevant or its score comparisons mix incompatible test conditions, the claimed up-to-date synthesis is not established.

Editorial extensions

If this is right

If the taxonomy is right, the historical VQA pipeline of separate visual and textual feature extractors is being absorbed into image-to-text alignment modules inside MLLMs, so fusion research has largely shifted to alignment architecture.
On this account, knowledge-based VQA has two live routes—internal knowledge already stored in model parameters and external knowledge retrieved from knowledge bases or passages—and multi-hop reasoning is where most of the remaining difficulty lies.
The survey's reading implies that MLLM benchmarks such as MME, SEED-Bench, and MathVista are becoming the de facto evaluation layer for VQA, testing perception, reasoning, hallucination, and specialized domains at once.
If the stated open problems are taken seriously, the next round of VQA progress will need indirect visual information, dataset-debiasing, explainable answers, and generative rather than selection-based answer production.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Editorial inference: if the perception/inference split is right, a testable prediction is that gains from new MLLMs on external-knowledge benchmarks will come more from better retrieval and tool use than from larger parameter counts, since the survey groups knowledge access separately from raw comprehension.
Editorial inference: the taxonomy implies that classic VQA datasets will increasingly be treated as sub-benchmarks inside general MLLM evaluations, which may make benchmark-specific leaderboards less informative over time.
Editorial inference: the survey's treatment of chain-of-thought and tool-aided reasoning suggests that reasoning in VQA may fragment into distinct skills such as symbolic, temporal, spatial, and commonsense reasoning, each needing its own evaluation rather than a single accuracy number.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The survey has a solid organizational core. The taxonomy contrasting natural language understanding and inference, with a focus on MLLMs and knowledge reasoning, is timely and could serve as a useful entry point for newcomers. The paper does a good job comparing prior surveys and covering recent models, datasets, and evaluation metrics. The prose sections on knowledge reasoning and MLLM techniques are readable and well-structured.

The soft spots are significant. The reference list is contaminated: it includes unrelated template entries (Clifford algebra, amsthm, sensor networks, ACM placeholder books), duplicates, and many references never cited in the body. That is not a small typo; it means the bibliography was not checked against the manuscript. Table 6 compounds the problem by mixing zero-shot and few-shot results and different evaluation splits (test-dev, Test2019, Test2020, Test2021), so the comparative claims are not verifiable. There are also numerous typos ('mOLUG-owl2', 'Genimi', 'Accuruacy', 'Knowldege'). These are fixable, but they currently undermine the central claim of an up-to-date, reliable synthesis.

As a survey, no new empirical result is expected, and the central argument that VQA is shifting toward MLLM-based reasoning holds up. But a survey is only as good as its sources. The reference list and performance table need major attention.

Recommendation: This deserves serious peer review with the expectation of major revision. The authors should verify every reference, remove or correct duplicates and template artifacts, and either standardize Table 6 or clearly flag the experimental protocols. I'd bring it to a reading group as a cautionary example of why reference hygiene matters, but I wouldn't cite it in its current form.

Referee Report

4 major / 5 minor

Summary. The paper is a survey of Visual Question Answering (VQA), organized around a taxonomy of natural-language understanding of images and text and of natural-language inference/knowledge reasoning. It reviews feature extraction, fusion mechanisms, vision-language pretraining, multimodal large language models, knowledge sources and reasoning, datasets, evaluation metrics, and open challenges. The abstract claims to provide an up-to-date synthesis of VQA with particular attention to MLLMs. The presentation includes several large tables summarizing models, datasets, and comparative performance, along with figures and equations taken from or inspired by prior work.

Significance. If accurate, the survey could serve as a broad reference for VQA, especially for readers seeking a single entry point to the transition from conventional models to MLLMs and to knowledge-based reasoning. The proposed taxonomy in Fig. 2 and the coverage of recent MLLM techniques (Sections 3 and 4.4) are potentially useful organizing contributions. The paper also explicitly claims timeliness and exhaustiveness, which raises the bar for factual reliability. On the current submission, however, the central value of the survey is undermined by a contaminated bibliography, missing appendices referenced in the body, and comparative tables that mix incompatible evaluation protocols. These issues must be addressed before the survey can be used as a reference.

major comments (4)

[Table 6 (Sec. 5.4)] The comparative performance table mixes fundamentally different evaluation settings without adequate qualification. VQAv2 rows include test-std results for models evaluated in zero-shot, few-shot, and fine-tuned settings; GQA rows separately list test-dev, Test2019, Test2020, and Test2021 without noting that these are different test splits; and VizWiz rows mix zero-shot and few-shot results across different dataset versions. The surrounding text only states that 'variation in experimental configurations' can lead to 'substantial diminution in performance' but does not tell the reader which numbers are directly comparable. As it stands, a reader cannot verify or use the comparative claims without returning to each original paper. The table should either be split by protocol and split, with the shot setting and dataset version stated per row, or removed.
[References (whole bibliography)] The reference list contains many entries that are unrelated to VQA and never cited in the body, including the Clifford algebra package [5], the amsthm package [16], wireless sensor network surveys [10, 76], and ACM template artifacts such as [98], [99], [103], and [420]. There are also duplicate entries ([45]/[46], [68]/[69], [72]/[73], [436]/[437]) and near-identical repeated entries for the same work ([205]-[211]). This demonstrates that the bibliography was not systematically checked against the manuscript. Because a survey's value rests on its sourcing, the entire reference list needs to be reconstructed from the in-text citations, with each entry verified against the original source, and all irrelevant, duplicate, and template entries removed.
[Appendices (Secs. 2.1.3, 2.2.3, 4.1, 4.2.2, 5.1, 5.3)] The body text repeatedly refers to appendices that are not present in the submission: Appendix A for embedding improvements, Appendix B for vision-language pretraining variants, Appendix C for knowledge sources, Appendix D for knowledge extraction, Appendix E for additional datasets, and Appendix F for additional metrics. For example, Sec. 2.1.3 says 'We give the detailed improvement methods in Appendix. A,' but no appendix follows. A survey that promises these details in the body is incomplete without them. Either include the appendices in the submission or remove all references to them and fold the necessary content into the main text.
[Table 3 and Sec. 3.3.3] The table and surrounding text contain factual errors that undermine the paper's claim of providing reliable information about latest models. The model name 'mOLUG-owl2' in Table 3 should be 'mPLUG-OWL2'; the text in Sec. 3.3.3 refers to 'Genimi' instead of 'Gemini'; the LLaVA-1.5 row reports 78.5 on VQAv2 as 'few-shot,' which is misleading because LLaVA-1.5 is a fine-tuned model, not a few-shot method; and the InternVL2 row in Table 3 has blank performance entries, so the row provides no information. Every entry in Table 3 should be checked against the cited papers, and the text should be corrected accordingly.

minor comments (5)

[Throughout] There are many typographical and spelling errors that should be corrected in copyediting: 'Accuruacy' in the Table 6 header, 'Knowldege' in Sec. 4.1, 'extrctor' in Sec. 2.1.2, 'applys' in Sec. 2.1.1, 'avarage' and 'Imgae' in Table 5, and 'breif' in Sec. 5.3.2.
[Sec. 2.1.1] The description of VGG-Net is imprecise: 'VGG-Net increases the convolutional layers to 19' is only one configuration; VGG-16 is equally common. The phrase about ResNet 'weakening strong connections' is unclear and should be rewritten.
[Sec. 2.2.2, Eq. (5)] The variables in Eq. (5) do not match the prose: the text says that textual tokens {x'_Q_i} are aligned with visual objects {x'_S_i}, but the same index i appears in both sequences. The equation should use different indices (e.g., i for the question node and j for the scene-graph node) so that the alignment is well defined.
[Sec. 3.3.3] The sentence 'which has proven especially effective in sophisticated tasks like math reasoning' is a sentence fragment and the connection to InternVL is unclear; it should be rewritten as a complete sentence or merged with the preceding claim.
[Table 2] The table lists 'AMN 2020 VGG-Net Word2Vec Attention MovieQA' but the year and architecture for AMN are not supported by the citation in the text; please verify the entry against the cited paper and correct the row or remove it.

Circularity Check

0 steps flagged · score 0.0 of 10

No load-bearing circularity: the survey summarizes external work, and the bibliography/table integrity issues are correctness risks, not circularity.

full rationale

This paper is a literature survey rather than a derivation or prediction paper, so the circularity patterns that apply to fitted-parameter or self-citation-driven arguments do not arise here. The survey's taxonomy, model descriptions, and comparative tables are summaries of externally published work, and the paper makes no claim to derive a new result from inputs it defines. The corresponding author's group appears among the references (e.g., [32] in the evaluation-metrics discussion, [256] in the document-image limitation discussion, and [372] in the knowledge-reasoning introduction), but these citations are illustrative or contextual, not load-bearing: none of them defines the survey's categories, none is invoked as a uniqueness theorem, and none forces the paper's organization or conclusions. The serious problems identified by the reader—unrelated template entries in the bibliography, duplicate references, and Table 6 mixing different evaluation splits and zero-shot/few-shot settings—are genuine correctness and verifiability concerns, but they do not make the survey's central claim equivalent to its own inputs by construction. No equation in the paper is claimed as a prediction that reduces to a fit, and no self-citation chain is used to exclude alternatives. Accordingly, the circularity score is 0, with the caveat that the unsupported bibliography and inconsistent performance table should be weighed as correctness risk rather than circularity.

Assumptions & free parameters 0 free parameters · 2 assumptions · 0 invented entities

The survey introduces no new parameters or entities; it relies on external literature. The main unstated premise is bibliographic accuracy, which is violated by multiple unrelated references.

assumptions (2)

domain assumption The cited references exist and support the statements attributed to them.
The survey's claims about models, datasets, and performance are summaries of external works; the reference list contains many unrelated ACM template entries, so this assumption is doubtful.
domain assumption The taxonomy in Figure 2 is a faithful representation of the VQA literature.
No systematic methodology or inclusion and exclusion criteria is given for selecting papers into the taxonomy.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey." pith.science (2026). https://pith.science/paper/I5OBOOBF

@misc{pith2026241117558,
  author       = {Pith},
  title        = {Pith review of: Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/I5OBOOBF}},
  note         = {Machine review of arXiv:2411.17558}
}

read the original abstract

Visual Question Answering (VQA) is a challenge task that combines natural language processing and computer vision techniques and gradually becomes a benchmark test task in multimodal large language models (MLLMs). The goal of our survey is to provide an overview of the development of VQA and a detailed description of the latest models with high timeliness. This survey gives an up-to-date synthesis of natural language understanding of images and text, as well as the knowledge reasoning module based on image-question information on the core VQA tasks. In addition, we elaborate on recent advances in extracting and fusing modal information with vision-language pretraining models and multimodal large language models in VQA. We also exhaustively review the progress of knowledge reasoning in VQA by detailing the extraction of internal knowledge and the introduction of external knowledge. Finally, we present the datasets of VQA and different evaluation metrics and discuss possible directions for future work.

Figures

Figures reproduced from arXiv: 2411.17558 by the authors.

**Figure 1.** The data used in VQA tasks, with the conclusion of the understanding and inference methods from conventional models to [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗

**Figure 2.** Taxonomy Graph of VQA Task. model reasoning methods in VQA. Finally, we discuss VQA challenges and propose future research directions in Sec. 6, concluding in Sec. 7. 2 COMPREHENSION OF IMAGE AND TEXT IN VQA 2.1 Feature Extraction The majority of VQA models require modal feature extraction prior to answering questions, which can be used for subsequent multimodal feature fusion to eliminate the gap between modals. 2.… view at source ↗

**Figure 3.** Percentage distribution of the usage of visual and textual feature extractors. [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: Comparison of fusion mechanism utilizing (a) non-attention based deep learning and (b) attention based deep learning. [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

**Figure 5.** Figure 5: Comparison of fusion mechanism utilizing (a) dual-stream pre-trained models and (b) single-stream pre-trained models. [PITH_FULL_IMAGE:figures/full_fig_p010_5.png]

**Figure 6.** Figure 6: Two architectures of the LLM aided visual understanding models. [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]

**Figure 7.** Figure 7: Architecture of the LLM by Image to Text Understanding and General MLLM. [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

**Figure 8.** Figure 8: The statistics of the widely-used dataset from 2020 to 2024. [PITH_FULL_IMAGE:figures/full_fig_p022_8.png]

**Figure 9.** Figure 9: Examples of four widely-used datasets. of Translation with Explicit Ordering (METEOR) [34]. The effectiveness of using these generation metrics for VQA system evaluation has been confirmed in [3, 132]. We give a breif introduction of BLEU metric, and the ROUGE and METE…

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

Refine Knowledge of Large Language Models via Adaptive Contrastive Learning
cs.CL 2025-02 conditional novelty 6.0 of 10

An adaptive contrastive learning strategy that uses a model's own sampled response accuracy to create per-region positive and negative training pairs improves LLM truthful rate by up to 6.9% over IDK-SFT.

Reference graph

Works this paper leans on

290 extracted references · 19 canonical work pages · cited by 1 Pith paper

[5]

Clifford: a maple 11 package for clifford algebra computations, version 11, 2007

Rafal Ablamowicz and Bertfried Fauser. Clifford: a maple 11 package for clifford algebra computations, version 11, 2007. URL http://math.tntech. edu/rafal/cliff11/index.html

2007
[16]

American Mathematical Society, April 2015

Using the amsthm Package . American Mathematical Society, April 2015. http://www.ctan.org/pkg/amsthm

2015
[99]

The title of book two , chapter 100

Ian Editor, editor. The title of book two , chapter 100. The name of the series two. University of Chicago Press, Chicago, 2nd. edition, 2008. doi: 10.1007/3-540-09237-4. URL http://dx.doi.org/10.1007/3-540-09456-9

doi:10.1007/3-540-09237-4 2008
[103]

Publication quality tables in LATEX, April 2005

Simon Fear. Publication quality tables in LATEX, April 2005. http://www.ctan.org/pkg/booktabs

2005
[46]

Buss, Arnold L

Jonathan F. Buss, Arnold L. Rosenberg, and Judson D. Knott. Vertex types in book-embeddings. Technical report, Amherst, MA, USA, 1987

1987
[68]

Clarkson

Kenneth L. Clarkson. Algorithms for Closest-Point Problems (Computational Geometry) . PhD thesis, Stanford University, Palo Alto, CA, 1985. UMI Order Number: AAT 8506171

1985
[69]

Algorithms for Closest-Point Problems (Computational Geometry)

Kenneth Lee Clarkson. Algorithms for Closest-Point Problems (Computational Geometry) . PhD thesis, Stanford University, Stanford, CA, USA, 1985. AAT 8506171

1985
[72]

Mancini, and Alessandro Mei

Mauro Conti, Roberto Di Pietro, Luigi V. Mancini, and Alessandro Mei. (old) distributed data source verification in wireless sensor networks. Inf. Fusion, 10(4):342–353, 2009. ISSN 1566-2535. doi: http://dx.doi.org/10.1016/j.inffus.2009.01.002

doi:10.1016/j.inffus.2009.01.002 2009
[73]

Mancini, and Alessandro Mei

Mauro Conti, Roberto Di Pietro, Luigi V. Mancini, and Alessandro Mei. (new) distributed data source verification in wireless sensor networks. Inf. Fusion, 10(4):342–353, October 2009. ISSN 1566-2535. doi: 10.1016/j.inffus.2009.01.002. URL http://portal.acm.org/citation.cfm?id=1555009.1555162

work page arXiv 2009
[211]

E-commerce and cultural values (Inbook-num chap) , chapter (in type field) 22, pages 51–74

Wei-Chang Kong. E-commerce and cultural values (Inbook-num chap) , chapter (in type field) 22, pages 51–74. IGI Publishing, Hershey, PA, USA,

Show all 290 references

[1]

SIGCOMM Comput. Commun. Rev., 13-14(5-1), 1984. ISSN 0146-4833

1984
[2]

CHI ’08: CHI ’08 extended abstracts on Human factors in computing systems , New York, NY, USA, 2008. ACM. ISBN 978-1-60558-012-X. General Chair-Czerwinski, Mary and General Chair-Lund, Arnie and Program Chair-Tan, Desney

2008
[3]

Vqa-med: Overview of the medical visual question answering task at imageclef 2019

Asma Ben Abacha, Sadid A Hasan, Vivek V Datla, Joey Liu, Dina Demner-Fushman, and Henning Müller. Vqa-med: Overview of the medical visual question answering task at imageclef 2019. CLEF (Working Notes), 2(6), 2019

2019
[4]

Multimodal categorization of crisis events in social media

Mahdi Abavisani, Liwei Wu, Shengli Hu, Joel Tetreault, and Alejandro Jaimes. Multimodal categorization of crisis events in social media. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14679–14689, 2020

2020
[6]

Abril and Robert Plant

Patricia S. Abril and Robert Plant. The patent holder’s dilemma: Buy, sell, or troll? Communications of the ACM, 50(1):36–44, January 2007. doi: 10.1145/1188913.1188915. URL http://doi.acm.org/10.1145/1219092.1219093

2007
[7]

A. Adya, P. Bahl, J. Padhye, A.Wolman, and L. Zhou. A multi-radio unification protocol for IEEE 802.11 wireless networks. In Proceedings of the IEEE 1st International Conference on Broadnets Networks (BroadNets’04) , pages 210–217, Los Alamitos, CA, 2004. IEEE

2004
[8]

Lrs3-ted: a large-scale dataset for visual speech recognition

Triantafyllos Afouras, Joon Son Chung, and Andrew Zisserman. Lrs3-ted: a large-scale dataset for visual speech recognition. arXiv preprint arXiv:1809.00496, 2018

2018 arXiv
[9]

Don’t just assume; look and answer: Overcoming priors for visual question answering

Aishwarya Agrawal, Dhruv Batra, Devi Parikh, and Aniruddha Kembhavi. Don’t just assume; look and answer: Overcoming priors for visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 4971–4980, 2018

2018
[10]

I. F. Akyildiz, W. Su, Y. Sankarasubramaniam, and E. Cayirci. Wireless sensor networks: A survey. Comm. ACM, 38(4):393–422, 2002

2002
[11]

I. F. Akyildiz, T. Melodia, and K. R. Chowdhury. A survey on wireless multimedia sensor networks. Computer Netw., 51(4):921–960, 2007

2007
[12]

Crisismmd: Multimodal twitter datasets from natural disasters

Firoj Alam, Ferda Ofli, and Muhammad Imran. Crisismmd: Multimodal twitter datasets from natural disasters. In Twelfth international AAAI conference on web and social media , 2018

2018
[13]

Automatic ontology-based knowledge extraction from web documents

Harith Alani, Sanghee Kim, David E Millard, Mark J Weal, Wendy Hall, Paul H Lewis, and Nigel R Shadbolt. Automatic ontology-based knowledge extraction from web documents. IEEE Intelligent Systems, 18(1):14–21, 2003

2003
[14]

Flamingo: a visual language model for few-shot learning

Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems , 35:2371...

2022
[15]

Fusion of detected objects in text for visual question answering

Chris Alberti, Jeffrey Ling, Michael Collins, and David Reitter. Fusion of detected objects in text for visual question answering. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural La...

2019
[17]

Bottom-up and top-down attention for image captioning and visual question answering

Peter Anderson, Xiaodong He, Chris Buehler, Damien Teney, Mark Johnson, Stephen Gould, and Lei Zhang. Bottom-up and top-down attention for image captioning and visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 6...

2018
[18]

Predicate path expressions

Sten Andler. Predicate path expressions. In Proceedings of the 6th. ACM SIGACT-SIGPLAN symposium on Principles of Programming Languages , POPL ’79, pages 226–236, New York, NY, 1979. ACM Press. doi: 10.1145/567752.567774. URL http://doi.acm.org/10.1145/567752.567774. Manuscrip...

1979
[19]

Andreas, M

J. Andreas, M. Rohrbach, T. Darrell, and K. Dan. Deep compositional question answering with neural module networks. Computer Science, 27: 55–56, 2015

2015
[20]

Learning to compose neural networks for question answering

Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. Learning to compose neural networks for question answering. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies , pages ...

2016 doi
[21]

Neural module networks

Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. Neural module networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 39–48, 2016

2016
[22]

David A. Anisi. Optimal motion control of a ground vehicle. Master’s thesis, Royal Institute of Technology (KTH), Stockholm, Sweden, 2003

2003
[23]

Vqa: Visual question answering

Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. Vqa: Visual question answering. In Proceedings of the IEEE international conference on computer vision , pages 2425–2433, 2015

2015
[24]

UMass citation field extraction dataset, 2013

Sam Anzaroot and Andrew McCallum. UMass citation field extraction dataset, 2013. URL http://www.iesl.cs.umass.edu/data/data-umasscitationfield

2013
[25]

Learning soft linear constraints with application to citation field extraction, 2014

Sam Anzaroot, Alexandre Passos, David Belanger, and Andrew McCallum. Learning soft linear constraints with application to citation field extraction, 2014

2014
[26]

J. E. Archer, Jr., R. Conway, and F. B. Schneider. User recovery and reversal in interactive systems.ACM Trans. Program. Lang. Syst., 6(1):1–19, January 1984

1984
[27]

Dbpedia: A nucleus for a web of open data

Sören Auer, Christian Bizer, Georgi Kobilarov, Jens Lehmann, Richard Cyganiak, and Zachary Ives. Dbpedia: A nucleus for a web of open data. In The semantic web, pages 722–735. Springer, 2007

2007
[28]

Unsupervised speech recognition

Alexei Baevski, Wei-Ning Hsu, Alexis Conneau, and Michael Auli. Unsupervised speech recognition. Advances in Neural Information Processing Systems, 34:27826–27839, 2021

2021
[29]

Multimodal language analysis in the wild: CMU- MOSEI dataset and interpretable dynamic fusion graph

AmirAli Bagher Zadeh, Paul Pu Liang, Soujanya Poria, Erik Cambria, and Louis-Philippe Morency. Multimodal language analysis in the wild: CMU- MOSEI dataset and interpretable dynamic fusion graph. In Proceedings of the 56th Annual Meeting of the Association for Computational Li...

2018 doi
[30]

P. Bahl, R. Chancre, and J. Dungeon. SSCH: Slotted seeded channel hopping for capacity improvement in IEEE 802.11 ad-hoc wireless networks. In Proceeding of the 10th International Conference on Mobile Computing and Networking (MobiCom’04) , pages 112–117, New York, NY, 2004. ACM

2004
[31]

Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond

Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966, 2023

2023 arXiv
[32]

Infobox-to-text generation with tree-like planning based attention network

Yang Bai, Ziran Li, Ning Ding, Ying Shen, and Hai-Tao Zheng. Infobox-to-text generation with tree-like planning based attention network. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence , pages 3773–3779, 2021

2021
[33]

Multimodal machine learning: A survey and taxonomy

Tadas Baltrušaitis, Chaitanya Ahuja, and Louis-Philippe Morency. Multimodal machine learning: A survey and taxonomy. IEEE transactions on pattern analysis and machine intelligence , 41(2):423–443, 2018

2018
[34]

Meteor: An automatic metric for mt evaluation with improved correlation with human judgments

Satanjeev Banerjee and Alon Lavie. Meteor: An automatic metric for mt evaluation with improved correlation with human judgments. In Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization , pages 65–72, 2005

2005
[35]

Visual question answering: Which investigated applications? Pattern Recogn

Silvio Barra, Carmen Bisogni, Maria De Marsico, and Stefano Ricciardi. Visual question answering: Which investigated applications? Pattern Recogn. Lett., 151(C):325–331, nov 2021. ISSN 0167-8655

2021
[36]

Target detection and localization using mimo radars and sonars

Ilya Bekkerman and Joseph Tabrikian. Target detection and localization using mimo radars and sonars. IEEE Transactions on Signal Processing , 54 (10):3873–3883, 2006

2006
[37]

Mutan: Multimodal tucker fusion for visual question answering

Hedi Ben-Younes, Rémi Cadene, Matthieu Cord, and Nicolas Thome. Mutan: Multimodal tucker fusion for visual question answering. InProceedings of the IEEE international conference on computer vision , pages 2612–2620, 2017

2017
[38]

Noise reduction in speech processing , volume 2

Jacob Benesty, Jingdong Chen, Yiteng Huang, and Israel Cohen. Noise reduction in speech processing , volume 2. Springer Science & Business Media, 2009

2009
[39]

Translating embeddings for modeling multi- relational data

Antoine Bordes, Nicolas Usunier, Alberto Garcia-Duran, Jason Weston, and Oksana Yakhnenko. Translating embeddings for modeling multi- relational data. Advances in neural information processing systems , 26, 2013

2013
[40]

Brad Wray, and Robin Haunschild

Lutz Bornmann, K. Brad Wray, and Robin Haunschild. Citation concept analysis (CCA)—a new form of citation analysis revealing the usefulness of concepts for other researchers illustrated by two exemplary case studies including classic books by Thomas S. Kuhn and Karl R. Popper,...

2019
[41]

Debray, and Larry L

Mic Bowman, Saumya K. Debray, and Larry L. Peterson. Reasoning about naming systems. ACM Trans. Program. Lang. Syst. , 15(5):795–825, November 1993. doi: 10.1145/161468.161471

1993
[42]

Babel, a multilingual style-option system for use with latex’s standard document styles

Johannes Braams. Babel, a multilingual style-option system for use with latex’s standard document styles. TUGboat, 12(2):291–301, June 1991

1991
[43]

A statistical approach to machine translation

Peter F Brown, John Cocke, Stephen A Della Pietra, Vincent J Della Pietra, Frederick Jelinek, John Lafferty, Robert L Mercer, and Paul S Roossin. A statistical approach to machine translation. Computational linguistics, 16(2):79–85, 1990

1990
[44]

Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffr...

2020
[47]

Benchlmm: Benchmarking cross-style visual capability of large multimodal models

Rizhao Cai, Zirui Song, Dayan Guan, Zhenhao Chen, Xing Luo, Chenyu Yi, and Alex Kot. Benchlmm: Benchmarking cross-style visual capability of large multimodal models. arXiv preprint arXiv:2312.02896, 2023

2023 arXiv
[48]

Recognition of human body motion using phase space constraints

Lee W Campbell and Aaron F Bobick. Recognition of human body motion using phase space constraints. In Proceedings of IEEE international conference on computer vision , pages 624–630. IEEE, 1995

1995
[49]

Bilateral cross-modality graph matching attention for feature fusion in visual question answering

Jianjian Cao, Xiameng Qin, Sanyuan Zhao, and Jianbing Shen. Bilateral cross-modality graph matching attention for feature fusion in visual question answering. IEEE Transactions on Neural Networks and Learning Systems , 2022

2022
[50]

Knowledge-routed visual question reasoning: Challenges for deep representation embedding

Qingxing Cao, Bailin Li, Xiaodan Liang, Keze Wang, and Liang Lin. Knowledge-routed visual question reasoning: Challenges for deep representation embedding. IEEE Transactions on Neural Networks and Learning Systems , 2021

2021
[51]

New paradigm for imaging systems

W Thomas Cathey and Edward R Dowski. New paradigm for imaging systems. Applied optics, 41(29):6080–6092, 2002

2002
[52]

Evaluation of text generation: A survey

Asli Celikyilmaz, Elizabeth Clark, and Jianfeng Gao. Evaluation of text generation: A survey. arXiv preprint arXiv:2006.14799, 2020

2006 arXiv
[53]

Learning question-guided video representation for multi-turn video question answering

Guan-Lin Chao, Abhinav Rastogi, Semih Yavuz, Dilek Hakkani-Tür, Jindong Chen, and Ian Lane. Learning question-guided video representation for multi-turn video question answering. arXiv preprint arXiv:1907.13280, 2019

1907 arXiv
[54]

X-llm: Bootstrapping advanced large language models by treating multi-modalities as foreign languages

Feilong Chen, Minglun Han, Haozhi Zhao, Qingyang Zhang, Jing Shi, Shuang Xu, and Bo Xu. X-llm: Bootstrapping advanced large language models by treating multi-modalities as foreign languages. arXiv preprint arXiv:2305.04160, 2023

2023 arXiv
[55]

Abc-cnn: An attention based convolutional neural network for visual question answering

Kan Chen, Jiang Wang, Liang-Chieh Chen, Haoyuan Gao, Wei Xu, and Ram Nevatia. Abc-cnn: An attention based convolutional neural network for visual question answering. arXiv preprint arXiv:1511.05960, 2015

2015 arXiv
[56]

Shikra: Unleashing multimodal llm’s referential dialogue magic

Keqin Chen, Zhao Zhang, Weili Zeng, Richong Zhang, Feng Zhu, and Rui Zhao. Shikra: Unleashing multimodal llm’s referential dialogue magic. arXiv preprint arXiv:2306.15195, 2023

2023 arXiv
[57]

Pali: A jointly-scaled multilingual language-image model

Xi Chen, Xiao Wang, Soravit Changpinyo, AJ Piergiovanni, Piotr Padlewski, Daniel Salz, Sebastian Goodman, Adam Grycner, Basil Mustafa, Lucas Beyer, et al. Pali: A jointly-scaled multilingual language-image model. arXiv preprint arXiv:2209.06794, 2022

2022 arXiv
[58]

Pali-3 vision language models: Smaller, faster, stronger, 2023

Xi Chen, Xiao Wang, Lucas Beyer, Alexander Kolesnikov, Jialin Wu, Paul Voigtlaender, Basil Mustafa, Sebastian Goodman, Ibrahim Alabdulmohsin, Piotr Padlewski, Daniel Salz, Xi Xiong, Daniel Vlasic, Filip Pavetic, Keran Rong, Tianli Yu, Daniel Keysers, Xiaohua Zhai, and Radu Sor...

2023
[59]

A review: Knowledge reasoning over knowledge graph

Xiaojun Chen, Shengbin Jia, and Yang Xiang. A review: Knowledge reasoning over knowledge graph. Expert Systems with Applications, 141:112948, 2020

2020
[60]

Uniter: Universal image-text representation learning

Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. Uniter: Universal image-text representation learning. In European conference on computer vision , pages 104–120. Springer, 2020

2020
[61]

How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites

Zhe Chen, Weiyun Wang, Hao Tian, Shenglong Ye, Zhangwei Gao, Erfei Cui, Wenwen Tong, Kongzhi Hu, Jiapeng Luo, Zheng Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. arXiv preprint arXiv:2404.16821, 2024

2024 arXiv
[62]

Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks

Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vis...

2024
[63]

Zero-shot visual question answering using knowledge graph

Zhuo Chen, Jiaoyan Chen, Yuxia Geng, Jeff Z Pan, Zonggang Yuan, and Huajun Chen. Zero-shot visual question answering using knowledge graph. In International Semantic Web Conference, pages 146–162. Springer, 2021

2021
[64]

Color image segmentation: advances and prospects.Pattern recognition, 34(12):2259–2281, 2001

Heng-Da Cheng, X_ H_ Jiang, Ying Sun, and Jingli Wang. Color image segmentation: advances and prospects.Pattern recognition, 34(12):2259–2281, 2001

2001
[65]

Learning phrase representations using rnn encoder-decoder for statistical machine translation

Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014

2014 arXiv
[66]

Attention-based models for speech recognition

Jan K Chorowski, Dzmitry Bahdanau, Dmitriy Serdyuk, Kyunghyun Cho, and Yoshua Bengio. Attention-based models for speech recognition. Advances in neural information processing systems , 28, 2015

2015
[67]

Post congress tristesse

Malcolm Clark. Post congress tristesse. In TeX90 Conference Proceedings, pages 84–89. TeX Users Group, March 1991

1991
[70]

Special issue: Digital libraries, November 1996

Cohen. Special issue: Digital libraries, November 1996

1996
[71]

Deciding equivalances among conjunctive aggregate queries

Sarah Cohen, Werner Nutt, and Yehoshua Sagic. Deciding equivalances among conjunctive aggregate queries. J. ACM, 54(2), April 2007. doi: 10.1145/1219092.1219093. URL http://doi.acm.org/10.1145/1219092.1219093

2007
[74]

XBOW sensor motes specifications, 2008

CROSSBOW. XBOW sensor motes specifications, 2008. http://www.xbow.com. Manuscript submitted to ACM Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey 31

2008
[75]

Holistic analysis of hallucination in gpt-4v (ision): Bias and interference challenges

Chenhang Cui, Yiyang Zhou, Xinyu Yang, Shirley Wu, Linjun Zhang, James Zou, and Huaxiu Yao. Holistic analysis of hallucination in gpt-4v (ision): Bias and interference challenges. arXiv preprint arXiv:2311.03287, 2023

2023 arXiv
[76]

Culler, D

D. Culler, D. Estrin, and M. Srivastava. Overview of sensor networks. IEEE Comput., 37(8 (Special Issue on Sensor Networks)):41–49, 2004

2004
[77]

Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven C. H. Hoi. Instructblip: Towards general-purpose vision-language models with instruction tuning. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenk...

2023
[78]

What is a knowledge representation? AI magazine, 14(1):17–17, 1993

Randall Davis, Howard Shrobe, and Peter Szolovits. What is a knowledge representation? AI magazine, 14(1):17–17, 1993

1993
[79]

Visual question answering: A survey on techniques and common trends in recent literature

Ana Claudia Akemi Matsuki de Faria, Felype de Castro Bastos, Jose Victor Nogueira Alves da Silva, Vitor Lopes Fabris, Valeska de Sousa Uchoa, Décio Gonçalves de Aguiar Neto, and Claudio Filipi Goncalves dos Santos. Visual question answering: A survey on techniques and common t...

2023 arXiv
[80]

The mathematics of statistical machine translation: Parameter estimation

Vincent J Della Pietra. The mathematics of statistical machine translation: Parameter estimation. Using Large Corpora, page 223, 1994

1994
[81]

Imagenet: A large-scale hierarchical image database

Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition , pages 248–255. Ieee, 2009

2009
[82]

Joint learning of answer selection and answer summary generation in community question answering

Yang Deng, Wai Lam, Yuexiang Xie, Daoyuan Chen, Yaliang Li, Min Yang, and Ying Shen. Joint learning of answer selection and answer summary generation in community question answering. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34, pages 7651–7658, 2020

2020
[83]

Bridging hierarchical and sequential context modeling for question- driven extractive answer summarization

Yang Deng, Wenxuan Zhang, Yaliang Li, Min Yang, Wai Lam, and Ying Shen. Bridging hierarchical and sequential context modeling for question- driven extractive answer summarization. In Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Info...

2020
[84]

Qlora: Efficient finetuning of quantized llms

Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. Qlora: Efficient finetuning of quantized llms. In A. Oh, T. Neumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine, editors, Advances in Neural Information Processing Systems , volume 36, pages 10088–10115....

2023
[85]

Bert: Pre-training of deep bidirectional transformers for language understanding

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018

2018 arXiv
[86]

Dijkstra

E. Dijkstra. Go to statement considered harmful. In Classics in software engineering (incoll) , pages 27–33. Yourdon Press, Upper Saddle River, NJ, USA, 1979. ISBN 0-917072-14-6. URL http://portal.acm.org/citation.cfm?id=1241515.1241518

1979
[87]

Mukea: Multimodal knowledge extraction and accumulation for knowledge-based visual question answering

Yang Ding, Jing Yu, Bang Liu, Yue Hu, Mingxin Cui, and Qi Wu. Mukea: Multimodal knowledge extraction and accumulation for knowledge-based visual question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 5089–5098, 2022

2022
[88]

Vqa: A new dataset for real-world vqa on pdf documents

Yihao Ding, Siwen Luo, Hyunsuk Chung, and Soyeon Caren Han. Vqa: A new dataset for real-world vqa on pdf documents. In Joint European Conference on Machine Learning and Knowledge Discovery in Databases , pages 585–601. Springer, 2023

2023
[89]

A survey of natural language generation

Chenhe Dong, Yinghui Li, Haifan Gong, Miaoxin Chen, Junxin Li, Ying Shen, and Min Yang. A survey of natural language generation. ACM Comput. Surv., 55(8), dec 2022. ISSN 0360-0300. doi: 10.1145/3554727. URL https://doi.org/10.1145/3554727

2022 doi
[90]

A survey for in-context learning

Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, Lei Li, and Zhifang Sui. A survey for in-context learning. CoRR
[91]

A survey for in-context learning

Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, Lei Li, and Zhifang Sui. A survey for in-context learning. CoRR, abs/2301.00234, 2023

2023 arXiv
[92]

Knowledge vault: A web-scale approach to probabilistic knowledge fusion

Xin Dong, Evgeniy Gabrilovich, Geremy Heitz, Wilko Horn, Ni Lao, Kevin Murphy, Thomas Strohmann, Shaohua Sun, and Wei Zhang. Knowledge vault: A web-scale approach to probabilistic knowledge fusion. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge dis...

2014
[93]

An image is worth 16x16 words: Transformers for image recognition at scale

Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv...

2010 arXiv
[94]

An empirical study of training end-to-end vision-and-language transformers

Zi-Yi Dou, Yichong Xu, Zhe Gan, Jianfeng Wang, Shuohang Wang, Lijuan Wang, Chenguang Zhu, Pengchuan Zhang, Lu Yuan, Nanyun Peng, et al. An empirical study of training end-to-end vision-and-language transformers. In Proceedings of the IEEE/CVF Conference on Computer Vision and ...

2022
[95]

Douglass, David Harel, and Mark B

Bruce P. Douglass, David Harel, and Mark B. Trakhtenbrot. Statecarts in use: structured analysis and object-orientation. In Grzegorz Rozenberg and Frits W. Vaandrager, editors,Lectures on Embedded Systems , volume 1494 of Lecture Notes in Computer Science , pages 368–394. Spri...

1998 doi
[96]

Danny Driess, Fei Xia, Mehdi S. M. Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, Wenlong Huang, Yevgen Chebotar, Pierre Sermanet, Daniel Duckworth, Sergey Levine, Vincent Vanhoucke, Karol Hausman, Marc Toussaint...

2023
[97]

D. D. Dunlop and V. R. Basili. Generalizing specifications for uniformly implemented loops. ACM Trans. Program. Lang. Syst., 7(1):137–158, January 1985. Manuscript submitted to ACM 32 Kuang and Xie, et al

1985
[100]

Survey on speech emotion recognition: Features, classification schemes, and databases

Moataz El Ayadi, Mohamed S Kamel, and Fakhri Karray. Survey on speech emotion recognition: Features, classification schemes, and databases. Pattern recognition, 44(3):572–587, 2011

2011
[101]

Every picture tells a story: Generating sentences from images

Ali Farhadi, Mohsen Hejrati, Mohammad Amin Sadeghi, Peter Young, Cyrus Rashtchian, Julia Hockenmaier, and David Forsyth. Every picture tells a story: Generating sentences from images. In European conference on computer vision , pages 15–29. Springer, 2010

2010
[102]

Two-frame motion estimation based on polynomial expansion

Gunnar Farnebäck. Two-frame motion estimation based on polynomial expansion. In Scandinavian conference on Image analysis , pages 363–370. Springer, 2003

2003
[104]

Mme: A comprehensive evaluation benchmark for multimodal large language models

Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, and Rongrong Ji. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306.13394, 2023

2023 arXiv
[105]

A challenger to gpt-4v? early explorations of gemini in visual expertise

Chaoyou Fu, Renrui Zhang, Haojia Lin, Zihan Wang, Timin Gao, Yongdong Luo, Yubo Huang, Zhengye Zhang, Longtian Qiu, Gaoxiang Ye, et al. A challenger to gpt-4v? early explorations of gemini in visual expertise. arXiv preprint arXiv:2312.12436, 2023

2023 arXiv
[107]

Multimodal compact bilinear pooling for visual question answering and visual grounding

Akira Fukui, Dong Huk Park, Daylen Yang, Anna Rohrbach, Trevor Darrell, and Marcus Rohrbach. Multimodal compact bilinear pooling for visual question answering and visual grounding. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing , page...

2016 doi
[108]

Large-scale adversarial training for vision-and-language representation learning

Zhe Gan, Yen-Chun Chen, Linjie Li, Chen Zhu, Yu Cheng, and Jingjing Liu. Large-scale adversarial training for vision-and-language representation learning. Advances in Neural Information Processing Systems , 33:6616–6628, 2020

2020
[109]

Transform-retrieve-generate: Natural language- centric outside-knowledge visual question answering

Feng Gao, Qing Ping, Govind Thattai, Aishwarya Reganti, Ying Nian Wu, and Prem Natarajan. Transform-retrieve-generate: Natural language- centric outside-knowledge visual question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , ...

2022
[110]

Are you talking to a machine? dataset and methods for multilingual image question

Haoyuan Gao, Junhua Mao, Jie Zhou, Zhiheng Huang, Lei Wang, and Wei Xu. Are you talking to a machine? dataset and methods for multilingual image question. Advances in neural information processing systems , 28, 2015

2015
[111]

Lora: A logical reasoning augmented dataset for visual question answering

Jingying Gao, Qi Wu, Alan Blair, and Maurice Pagnucco. Lora: A logical reasoning augmented dataset for visual question answering. Advances in Neural Information Processing Systems , 36, 2024

2024
[112]

Motion-appearance co-memory networks for video question answering

Jiyang Gao, Runzhou Ge, Kan Chen, and Ram Nevatia. Motion-appearance co-memory networks for video question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 6576–6585, 2018

2018
[113]

Examine before you answer: Multi-task learning with adaptive- attentions for multiple-choice vqa

Lianli Gao, Pengpeng Zeng, Jingkuan Song, Xianglong Liu, and Heng Tao Shen. Examine before you answer: Multi-task learning with adaptive- attentions for multiple-choice vqa. In Proceedings of the 26th ACM international conference on Multimedia , pages 1742–1750, 2018

2018
[114]

Structured two-stream attention network for video question answering

Lianli Gao, Pengpeng Zeng, Jingkuan Song, Yuan-Fang Li, Wu Liu, Tao Mei, and Heng Tao Shen. Structured two-stream attention network for video question answering. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 33, pages 6391–6398, 2019

2019
[115]

Question-guided hybrid convolution for visual question answering

Peng Gao, Hongsheng Li, Shuang Li, Pan Lu, Yikang Li, Steven CH Hoi, and Xiaogang Wang. Question-guided hybrid convolution for visual question answering. In Proceedings of the European Conference on Computer Vision (ECCV) , pages 469–485, 2018

2018
[116]

Dynamic fusion with intra-and inter-modality attention flow for visual question answering

Peng Gao, Zhengkai Jiang, Haoxuan You, Pan Lu, Steven CH Hoi, Xiaogang Wang, and Hongsheng Li. Dynamic fusion with intra-and inter-modality attention flow for visual question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , page...

2019
[117]

Conceptbert: Concept-aware representation for visual question answering

François Gardères, Maryam Ziaeefard, Baptiste Abeloos, and Freddy Lecue. Conceptbert: Concept-aware representation for visual question answering. In Findings of the Association for Computational Linguistics: EMNLP 2020 , pages 489–498, 2020

2020
[118]

Structured variational inference procedures and their realizations (as incol)

Dan Geiger and Christopher Meek. Structured variational inference procedures and their realizations (as incol). InProceedings of Tenth International Workshop on Artificial Intelligence and Statistics, The Barbados. The Society for Artificial Intelligence and Statistics, January 2005

2005
[119]

Automatic Parallelization for Distributed-Memory Multiprocessing Systems

Michael Gerndt. Automatic Parallelization for Distributed-Memory Multiprocessing Systems . PhD thesis, University of Bonn, Bonn, Germany, December 1989

1989
[120]

A bayesian framework for word segmentation: Exploring the effects of context.Cognition, 112(1):21–54, 2009

Sharon Goldwater, Thomas L Griffiths, and Mark Johnson. A bayesian framework for word segmentation: Exploring the effects of context.Cognition, 112(1):21–54, 2009

2009
[121]

Multimodal- gpt: A vision and language model for dialogue with humans

Tao Gong, Chengqi Lyu, Shilong Zhang, Yudong Wang, Miao Zheng, Qian Zhao, Kuikun Liu, Wenwei Zhang, Ping Luo, and Kai Chen. Multimodal- gpt: A vision and language model for dialogue with humans. CoRR, abs/2305.04790, 2023

2023 arXiv
[122]

Michel Goossens, S. P. Rahtz, Ross Moore, and Robert S. Sutor. The Latex Web Companion: Integrating TEX, HTML, and XML . Addison-Wesley Longman Publishing Co., Inc., Boston, MA, USA, 1st edition, 1999. ISBN 0201433117

1999
[123]

Making the v in vqa matter: Elevating the role of image understanding in visual question answering

Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 6904–6913, 2017

2017
[124]

Long short-term memory

Alex Graves. Long short-term memory. Supervised sequence labelling with recurrent neural networks , pages 37–45, 2012. Manuscript submitted to ACM Natural Language Understanding and Inference with MLLM in Visual Question Answering: A Survey 33

2012
[125]

The representation and use of focus in dialogue understanding

Barbara Jean Grosz. The representation and use of focus in dialogue understanding. University of California, Berkeley, 1977

1977
[126]

Catch me, if you can: Evading network signatures with web-based polymorphic worms

Matthew Van Gundy, Davide Balzarotti, and Giovanni Vigna. Catch me, if you can: Evading network signatures with web-based polymorphic worms. In Proceedings of the first USENIX workshop on Offensive Technologies , WOOT ’07, Berkley, CA, 2007. USENIX Association

2007
[127]

Catch me, if you can: Evading network signatures with web-based polymorphic worms

Matthew Van Gundy, Davide Balzarotti, and Giovanni Vigna. Catch me, if you can: Evading network signatures with web-based polymorphic worms. In Proceedings of the first USENIX workshop on Offensive Technologies , WOOT ’08, pages 99–100, Berkley, CA, 2008. USENIX Association

2008
[128]

Catch me, if you can: Evading network signatures with web-based polymorphic worms

Matthew Van Gundy, Davide Balzarotti, and Giovanni Vigna. Catch me, if you can: Evading network signatures with web-based polymorphic worms. In Proceedings of the first USENIX workshop on Offensive Technologies , WOOT ’09, pages 90–100, Berkley, CA, 2009. USENIX Association

2009
[129]

From images to textual prompts: Zero-shot visual question answering with frozen large language models

Jiaxian Guo, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Boyang Li, Dacheng Tao, and Steven Hoi. From images to textual prompts: Zero-shot visual question answering with frozen large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern R...

2023
[130]

Visual programming: Compositional visual reasoning without training

Tanmay Gupta and Aniruddha Kembhavi. Visual programming: Compositional visual reasoning without training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages 14953–14962, June 2023

2023
[131]

Visual programming: Compositional visual reasoning without training

Tanmay Gupta and Aniruddha Kembhavi. Visual programming: Compositional visual reasoning without training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14953–14962, 2023

2023
[132]

Vizwiz grand challenge: Answering visual questions from blind people

Danna Gurari, Qing Li, Abigale J Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P Bigham. Vizwiz grand challenge: Answering visual questions from blind people. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 3608–3...

2018
[133]

Ian Munro

Torben Hagerup, Kurt Mehlhorn, and J. Ian Munro. Maintaining discrete probability distributions optimally. In Proceedings of the 20th International Colloquium on Automata, Languages and Programming, volume 700 ofLecture Notes in Computer Science, pages 253–264, Berlin, 1993. S...

1993
[134]

Textural features for image classification

Robert M Haralick, Karthikeyan Shanmugam, and Its’ Hak Dinstein. Textural features for image classification. IEEE Transactions on systems, man, and cybernetics, (6):610–621, 1973

1973
[135]

Logics of programs: Axiomatics and descriptive power

David Harel. Logics of programs: Axiomatics and descriptive power. MIT Research Lab Technical Report TR-200, Massachusetts Institute of Technology, Cambridge, MA, 1978

1978
[136]

First-Order Dynamic Logic, volume 68 of Lecture Notes in Computer Science

David Harel. First-Order Dynamic Logic, volume 68 of Lecture Notes in Computer Science . Springer-Verlag, New York, NY, 1979. doi: 10.1007/3-540- 09237-4. URL http://dx.doi.org/10.1007/3-540-09237-4

1979 doi
[137]

CodeBlue: Sensor networks for medical care, 2008

Harvard CodeBlue. CodeBlue: Sensor networks for medical care, 2008. http://www.eecs.harvard.edu/mdw/ proj/codeblue/

2008
[138]

It’s not about the journey; it’s about the destination: Following soft paths under question-guidance for visual reasoning

Monica Haurilet, Alina Roitberg, and Rainer Stiefelhagen. It’s not about the journey; it’s about the destination: Following soft paths under question-guidance for visual reasoning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 193...

1930
[139]

Deep residual learning for image recognition

Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 770–778, 2016

2016
[140]

Lightgcn: Simplifying and powering graph convolution network for recommendation

Xiangnan He, Kuan Deng, Xiang Wang, Yan Li, Yongdong Zhang, and Meng Wang. Lightgcn: Simplifying and powering graph convolution network for recommendation. In Proceedings of the 43rd International ACM SIGIR conference on research and development in Information Retrieval , page...

2020
[141]

Pathvqa: 30000+ questions for medical visual question answering

Xuehai He, Yichen Zhang, Luntian Mou, Eric Xing, and Pengtao Xie. Pathvqa: 30000+ questions for medical visual question answering. arXiv preprint arXiv:2003.10286, 2020

2003 arXiv
[142]

Heering and P

J. Heering and P. Klint. Towards monolingual programming environments. ACM Trans. Program. Lang. Syst. , 7(2):183–213, April 1985

1985
[143]

Hypergraph transformer: Weakly-supervised multi-hop reasoning for knowledge-based visual question answering

Yu-Jung Heo, Eun-Sol Kim, Woo Suk Choi, and Byoung-Tak Zhang. Hypergraph transformer: Weakly-supervised multi-hop reasoning for knowledge-based visual question answering. arXiv preprint arXiv:2204.10448, 2022

2022 arXiv
[144]

Image captioning: Transforming objects into words.Advances in Neural Information Processing Systems, 32, 2019

Simao Herdade, Armin Kappeler, Kofi Boakye, and Joao Soares. Image captioning: Transforming objects into words.Advances in Neural Information Processing Systems, 32, 2019

2019
[145]

A methodology for implementing highly concurrent data objects

Maurice Herlihy. A methodology for implementing highly concurrent data objects. ACM Trans. Program. Lang. Syst. , 15(5):745–770, November
[146]

Let’s think frame by frame with VIP: A video infilling and prediction dataset for evaluating video chain-of-thought

Vaishnavi Himakunthala, Andy Ouyang, Daniel Rose, Ryan He, Alex Mei, Yujie Lu, Chinmay Sonar, Michael Saxon, and William Yang Wang. Let’s think frame by frame with VIP: A video infilling and prediction dataset for evaluating video chain-of-thought. In Houda Bouamor, Juan Pino,...

2023
[147]

Natural language question answering: the view from here

Lynette Hirschman and Robert Gaizauskas. Natural language question answering: the view from here. natural language engineering, 7(4):275–300, 2001

2001
[148]

C. A. R. Hoare. Chapter ii: Notes on data structuring. In O. J. Dahl, E. W. Dijkstra, and C. A. R. Hoare, editors, Structured programming (incoll), pages 83–174. Academic Press Ltd., London, UK, UK, 1972. ISBN 0-12-200550-3. URL http://portal.acm.org/citation.cfm?id=1243380.1243382

1972
[149]

Knowledge graphs

Aidan Hogan, Eva Blomqvist, Michael Cochez, Claudia d’Amato, Gerard De Melo, Claudio Gutierrez, Sabrina Kirrane, José Emilio Labra Gayo, Roberto Navigli, Sebastian Neumaier, et al. Knowledge graphs. ACM Computing Surveys (CSUR), 54(4):1–37, 2021

2021
[150]

Billy S. Hollis. Visual Basic 6: Design, Specification, and Objects with Other . Prentice Hall PTR, Upper Saddle River, NJ, USA, 1st edition, 1999. ISBN 0130850845. Manuscript submitted to ACM 34 Kuang and Xie, et al

1999
[151]

The analysis of linear partial differential operators

Lars Hörmander. The analysis of linear partial differential operators. IV , volume 275 of Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences] . Springer-Verlag, Berlin, Germany, 1985. ISBN 3-540-13829-3. Fourier integral operators

1985
[152]

The analysis of linear partial differential operators

Lars Hörmander. The analysis of linear partial differential operators. III , volume 275 of Grundlehren der Mathematischen Wissenschaften [Fundamental Principles of Mathematical Sciences] . Springer-Verlag, Berlin, Germany, 1985. ISBN 3-540-13828-5. Pseudodifferential operators

1985
[153]

A comprehensive survey of deep learning for image captioning

MD Zakir Hossain, Ferdous Sohel, Mohd Fairuz Shiratuddin, and Hamid Laga. A comprehensive survey of deep learning for image captioning. ACM Computing Surveys (CsUR), 51(6):1–36, 2019

2019
[154]

A human body posture recognition algorithm based on bp neural network for wireless body area networks

Fengye Hu, Lu Wang, Shanshan Wang, Xiaolan Liu, and Gengxin He. A human body posture recognition algorithm based on bp neural network for wireless body area networks. China Communications, 13(8):198–208, 2016

2016
[155]

MMGCN: Multimodal fusion via deep graph convolution network for emotion recognition in conversation

Jingwen Hu, Yuchen Liu, Jinming Zhao, and Qin Jin. MMGCN: Multimodal fusion via deep graph convolution network for emotion recognition in conversation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint C...

2021 doi
[156]

Language-conditioned graph networks for relational reasoning

Ronghang Hu, Anna Rohrbach, Trevor Darrell, and Kate Saenko. Language-conditioned graph networks for relational reasoning. In Proceedings of the IEEE/CVF international conference on computer vision , pages 10294–10303, 2019

2019
[157]

Scaling up vision-language pre-training for image captioning

Xiaowei Hu, Zhe Gan, Jianfeng Wang, Zhengyuan Yang, Zicheng Liu, Yumao Lu, and Lijuan Wang. Scaling up vision-language pre-training for image captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 17980–17989, 2022

2022
[158]

Location-aware graph convolutional networks for video question answering

Deng Huang, Peihao Chen, Runhao Zeng, Qing Du, Mingkui Tan, and Chuang Gan. Location-aware graph convolutional networks for video question answering. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34, pages 11021–11028, 2020

2020
[159]

Densely connected convolutional networks

Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 4700–4708, 2017

2017
[160]

Aligned dual channel graph convolutional network for visual question answering

Qingbao Huang, Jielong Wei, Yi Cai, Changmeng Zheng, Junying Chen, Ho-fung Leung, and Qing Li. Aligned dual channel graph convolutional network for visual question answering. In Proceedings of the 58th annual meeting of the association for computational linguistics , pages 716...

2020
[161]

Pixel-bert: Aligning image pixels with text by deep multi-modal transformers

Zhicheng Huang, Zhaoyang Zeng, Bei Liu, Dongmei Fu, and Jianlong Fu. Pixel-bert: Aligning image pixels with text by deep multi-modal transformers. arXiv preprint arXiv:2004.00849, 2020

2004 arXiv
[162]

Seeing out of the box: End-to-end pre-training for vision-language representation learning

Zhicheng Huang, Zhaoyang Zeng, Yupan Huang, Bei Liu, Dongmei Fu, and Jianlong Fu. Seeing out of the box: End-to-end pre-training for vision-language representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 12976–12985, 2021

2021
[163]

Gqa: A new dataset for real-world visual reasoning and compositional question answering

Drew A Hudson and Christopher D Manning. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 6700–6709, 2019

2019
[164]

Ieee tcsc executive committee

IEEE. Ieee tcsc executive committee. In Proceedings of the IEEE International Conference on Web Services , ICWS ’04, pages 21–22, Washington, DC, USA, 2004. IEEE Computer Society. ISBN 0-7695-2167-3. doi: http://dx.doi.org/10.1109/ICWS.2004.64. URL http://dx.doi.org/10.1109/IC...

2004 doi
[165]

Generative attention model with adversarial self-learning for visual question answering

Ilija Ilievski and Jiashi Feng. Generative attention model with adversarial self-learning for visual question answering. In Proceedings of the on Thematic Workshops of ACM Multimedia 2017 , pages 415–423, 2017

2017
[166]

A focused dynamic attention model for visual question answering

Ilija Ilievski, Shuicheng Yan, and Jiashi Feng. A focused dynamic attention model for visual question answering. arXiv preprint arXiv:1604.01485, 2016

2016 arXiv
[167]

Farhan Ishmam, Md

Md. Farhan Ishmam, Md. Sakib Hossain Shovon, M.F. Mridha, and Nilanjan Dey. From image to language: A critical analysis of visual question answering (vqa) approaches, challenges, and opportunities. Information Fusion, 106:102270, 2024. ISSN 1566-2535

2024
[168]

Gpt-4o: The cutting-edge advancement in multimodal llm

Raisa Islam and Owana Marzia Moushi. Gpt-4o: The cutting-edge advancement in multimodal llm. Authorea Preprints, 2024

2024
[169]

Opt-iml: Scaling language model instruction meta learning through the lens of generalization

Srinivasan Iyer, Xi Victoria Lin, Ramakanth Pasunuru, Todor Mihaylov, Daniel Simig, Ping Yu, Kurt Shuster, Tianlu Wang, Qing Liu, Punit Singh Koura, et al. Opt-iml: Scaling language model instruction meta learning through the lens of generalization. arXiv preprint arXiv:2212.1...

2022 arXiv
[170]

Revisiting visual question answering baselines

Allan Jabri, Armand Joulin, and Laurens van der Maaten. Revisiting visual question answering baselines. In European conference on computer vision, pages 727–739. Springer, 2016

2016
[171]

Tgif-qa: Toward spatio-temporal reasoning in visual question answering

Yunseok Jang, Yale Song, Youngjae Yu, Youngjin Kim, and Gunhee Kim. Tgif-qa: Toward spatio-temporal reasoning in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 2758–2766, 2017

2017
[172]

Scaling up visual and vision-language representation learning with noisy text supervision

Chao Jia, Yinfei Yang, Ye Xia, Yi-Ting Chen, Zarana Parekh, Hieu Pham, Quoc Le, Yun-Hsuan Sung, Zhen Li, and Tom Duerig. Scaling up visual and vision-language representation learning with noisy text supervision. In International Conference on Machine Learning , pages 4904–4916...

2021
[173]

Compositional memory for visual question answering

Aiwen Jiang, Fang Wang, Fatih Porikli, and Yi Li. Compositional memory for visual question answering. arXiv preprint arXiv:1511.05676, 2015

2015 arXiv
[174]

In defense of grid features for visual question answering

Huaizu Jiang, Ishan Misra, Marcus Rohrbach, Erik Learned-Miller, and Xinlei Chen. In defense of grid features for visual question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10267–10276, 2020

2020
[175]

Divide and conquer: Question-guided spatio-temporal contextual attention for video question answering

Jianwen Jiang, Ziqiang Chen, Haojie Lin, Xibin Zhao, and Yue Gao. Divide and conquer: Question-guided spatio-temporal contextual attention for video question answering. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34, pages 11101–11108, 2020

2020
[176]

Pythia v0

Yu Jiang, Vivek Natarajan, Xinlei Chen, Marcus Rohrbach, Dhruv Batra, and Devi Parikh. Pythia v0. 1: the winning entry to the vqa challenge 2018. arXiv preprint arXiv:1807.09956, 2018. Manuscript submitted to ACM Natural Language Understanding and Inference with MLLM in Visual...

2018 arXiv
[177]

Llms can find mathematical reasoning mistakes by pedagogical chain-of-thought

Zhuoxuan Jiang, Haoyuan Peng, Shanshan Feng, Fan Li, and Dongsheng Li. Llms can find mathematical reasoning mistakes by pedagogical chain-of-thought. CoRR, abs/2405.06705, 2024

2024 arXiv
[178]

Multi-interaction network with object relation for video question answering

Weike Jin, Zhou Zhao, Mao Gu, Jun Yu, Jun Xiao, and Yueting Zhuang. Multi-interaction network with object relation for video question answering. In Proceedings of the 27th ACM international conference on multimedia , pages 1193–1201, 2019

2019
[179]

Densecap: Fully convolutional localization networks for dense captioning

Justin Johnson, Andrej Karpathy, and Li Fei-Fei. Densecap: Fully convolutional localization networks for dense captioning. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 4565–4574, 2016

2016
[180]

Clevr: A diagnostic dataset for compositional language and elementary visual reasoning

Justin Johnson, Bharath Hariharan, Laurens Van Der Maaten, Li Fei-Fei, C Lawrence Zitnick, and Ross Girshick. Clevr: A diagnostic dataset for compositional language and elementary visual reasoning. In Proceedings of the IEEE conference on computer vision and pattern recognitio...

2017
[181]

Answer-type prediction for visual question answering

Kushal Kafle and Christopher Kanan. Answer-type prediction for visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 4976–4984, 2016

2016
[182]

Visual question answering: Datasets, algorithms, and future challenges

Kushal Kafle and Christopher Kanan. Visual question answering: Datasets, algorithms, and future challenges. Computer Vision and Image Understanding, 163:3–20, 2017

2017
[183]

Mdetr-modulated detection for end-to-end multi-modal understanding

Aishwarya Kamath, Mannat Singh, Yann LeCun, Gabriel Synnaeve, Ishan Misra, and Nicolas Carion. Mdetr-modulated detection for end-to-end multi-modal understanding. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 1780–1790, 2021

2021
[184]

Dense passage retrieval for open-domain question answering

Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. Dense passage retrieval for open-domain question answering. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) , pag...

2020
[185]

Show, ask, attend, and answer: A strong baseline for visual question answering

Vahid Kazemi and Ali Elqursh. Show, ask, attend, and answer: A strong baseline for visual question answering. arXiv preprint arXiv:1704.03162, 2017

2017 arXiv
[186]

Multimodal neural graph memory networks for visual question answering

Mahmoud Khademi. Multimodal neural graph memory networks for visual question answering. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages 7177–7188, 2020

2020
[187]

Textbook question answering with multi-modal context graph understanding and self-supervised open-set comprehension

Daesik Kim, Seonhoon Kim, and Nojun Kwak. Textbook question answering with multi-modal context graph understanding and self-supervised open-set comprehension. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics , pages 3568–3584, 2019

2019
[188]

Multimodal residual learning for visual qa

Jin-Hwa Kim, Sang-Woo Lee, Donghyun Kwak, Min-Oh Heo, Jeonghee Kim, Jung-Woo Ha, and Byoung-Tak Zhang. Multimodal residual learning for visual qa. Advances in neural information processing systems , 29, 2016

2016
[189]

Hadamard product for low-rank bilinear pooling

Jin-Hwa Kim, Kyoung-Woon On, Woosang Lim, Jeonghee Kim, Jung-Woo Ha, and Byoung-Tak Zhang. Hadamard product for low-rank bilinear pooling. In International Conference on Learning Representations , 2017. URL https://openreview.net/forum?id=r1rhWnZkg

2017
[190]

Bilinear attention networks

Jin-Hwa Kim, Jaehyun Jun, and Byoung-Tak Zhang. Bilinear attention networks. Advances in neural information processing systems , 31, 2018

2018
[191]

Deepstory: Video story qa by deep embedded memory networks

Kyung-Min Kim, Min-Oh Heo, Seong-Ho Choi, and Byoung-Tak Zhang. Deepstory: Video story qa by deep embedded memory networks. arXiv preprint arXiv:1707.00836, 2017

2017 arXiv
[192]

Multimodal dual attention memory for video story question answering

Kyung-Min Kim, Seong-Ho Choi, Jin-Hwa Kim, and Byoung-Tak Zhang. Multimodal dual attention memory for video story question answering. In Proceedings of the European Conference on Computer Vision (ECCV) , pages 673–688, 2018

2018
[193]

Vilt: Vision-and-language transformer without convolution or region supervision

Wonjae Kim, Bokyung Son, and Ildoo Kim. Vilt: Vision-and-language transformer without convolution or region supervision. In International Conference on Machine Learning , pages 5583–5594. PMLR, 2021

2021
[194]

Kipf and Max Welling

Thomas N. Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. In International Conference on Learning Representations, 2017. URL https://openreview.net/forum?id=SJU4ayYgl

2017
[195]

Skip-thought vectors

Ryan Kiros, Yukun Zhu, Russ R Salakhutdinov, Richard Zemel, Raquel Urtasun, Antonio Torralba, and Sanja Fidler. Skip-thought vectors. Advances in neural information processing systems , 28, 2015

2015
[196]

Algorithmic enumeration of ideal classes for quaternion orders

Markus Kirschmer and John Voight. Algorithmic enumeration of ideal classes for quaternion orders. SIAM J. Comput., 39(5):1714–1747, January
[197]

Donald E. Knuth. Seminumerical Algorithms. Addison-Wesley, 1981

1981
[198]

Donald E. Knuth. Seminumerical Algorithms, volume 2 of The Art of Computer Programming. Addison-Wesley, Reading, MA, 2nd edition, 10 January 1981

1981
[199]

Donald E. Knuth. The TEXbook. Addison-Wesley, Reading, MA., 1984

1984
[200]

Donald E. Knuth. The Art of Computer Programming, Vol. 1: Fundamental Algorithms (3rd. ed.) . Addison Wesley Longman Publishing Co., Inc., 1997

1997
[201]

Donald E. Knuth. The Art of Computer Programming , volume 1 of Fundamental Algorithms. Addison Wesley Longman Publishing Co., Inc., 3rd edition, 1998. (book)

1998
[202]

Recent, rapid advancement in visual question answering: a review

Venkat Kodali and Daniel Berleant. Recent, rapid advancement in visual question answering: a review. In 2022 IEEE International Conference on Electro Information Technology (eIT), pages 139–146. IEEE, 2022

2022
[203]

Statistical machine translation

Philipp Koehn. Statistical machine translation. Cambridge University Press, 2009

2009
[204]

Generating images with multimodal language models

Jing Yu Koh, Daniel Fried, and Russ R Salakhutdinov. Generating images with multimodal language models. Advances in Neural Information Processing Systems, 36, 2024

2024
[212]

Korach, D

E. Korach, D. Rotem, and N. Santoro. Distributed algorithms for finding centers and medians in networks. ACM Trans. Program. Lang. Syst. , 6(3): 380–401, July 1984

1984
[213]

Mapping powerlists onto hypercubes

Jacob Kornerup. Mapping powerlists onto hypercubes. Master’s thesis, The University of Texas at Austin, 1994. (In preparation)

1994
[214]

Understanding Policy-Based Networking

David Kosiur. Understanding Policy-Based Networking. Wiley, New York, NY, 2nd. edition, 2001

2001
[215]

Visual genome: Connecting language and vision using crowdsourced dense image annotations

Ranjay Krishna, Yuke Zhu, Oliver Groth, Justin Johnson, Kenji Hata, Joshua Kravitz, Stephanie Chen, Yannis Kalantidis, Li-Jia Li, David A Shamma, et al. Visual genome: Connecting language and vision using crowdsourced dense image annotations. International journal of computer ...

2017
[216]

Imagenet classification with deep convolutional neural networks

Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. Imagenet classification with deep convolutional neural networks. Communications of the ACM, 60(6):84–90, 2017

2017
[217]

Natural questions: a benchmark for question answering research

Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. Natural questions: a benchmark for question answering research. Transactions of the Association for Computa...

2019
[218]

Conditional random fields: Probabilistic models for segmenting and labeling sequence data

John Lafferty, Andrew McCallum, and Fernando CN Pereira. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. 2001

2001
[219]

LATEX: A Document Preparation System

Leslie Lamport. LATEX: A Document Preparation System . Addison-Wesley, Reading, MA., 1986

1986
[220]

Vietnamese elementary math reasoning using large language model with refined translation and dense-retrieved chain-of-thought

Nguyen-Khang Le, Dieu-Hien Nguyen, Dinh-Truong Do, Chau Nguyen, and Le Minh Nguyen. Vietnamese elementary math reasoning using large language model with refined translation and dense-retrieved chain-of-thought. In Toyotaro Suzumura and Mayumi Bono, editors, New Frontiers in Ar...

2024
[221]

Distributed representations of sentences and documents

Quoc Le and Tomas Mikolov. Distributed representations of sentences and documents. In International conference on machine learning , pages 1188–1196. PMLR, 2014

2014
[222]

Hierarchical conditional relation networks for video question answering

Thao Minh Le, Vuong Le, Svetha Venkatesh, and Truyen Tran. Hierarchical conditional relation networks for video question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 9972–9981, 2020

2020
[223]

Backpropagation applied to handwritten zip code recognition

Yann LeCun, Bernhard Boser, John S Denker, Donnie Henderson, Richard E Howard, Wayne Hubbard, and Lawrence D Jackel. Backpropagation applied to handwritten zip code recognition. Neural computation, 1(4):541–551, 1989

1989
[224]

Transcript of question and answer session

Jan Lee. Transcript of question and answer session. In Richard L. Wexelblat, editor, History of programming languages I (incoll) , pages 68–71. ACM, New York, NY, USA, 1981. ISBN 0-12-745040-8. doi: http://doi.acm.org/10.1145/800025.1198348. URL http://doi.acm.org/10.1145/8000...

1981
[225]

Interview with bill kinder: January 13, 2005

Newton Lee. Interview with bill kinder: January 13, 2005. Comput. Entertain., 3(1):4, Jan.-March 2005. doi: 10.1145/1057270.1057278. URL http://doi.acm.org/10.1145/1057270.1057278

2005
[226]

Multi-question learning for visual question answering

Chenyi Lei, Lei Wu, Dong Liu, Zhao Li, Guoxin Wang, Haihong Tang, and Houqiang Li. Multi-question learning for visual question answering. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34, pages 11328–11335, 2020

2020
[227]

Tvqa: Localized, compositional video question answering

Jie Lei, Licheng Yu, Mohit Bansal, and Tamara L Berg. Tvqa: Localized, compositional video question answering. arXiv preprint arXiv:1809.01696, 2018

2018 arXiv
[228]

Viquae, a dataset for knowledge-based visual question answering about named entities

Paul Lerner, Olivier Ferret, Camille Guinaudeau, Hervé Le Borgne, Romaric Besançon, José G Moreno, and Jesús Lovón Melgarejo. Viquae, a dataset for knowledge-based visual question answering about named entities. In Proceedings of the 45th International ACM SIGIR Conference on ...

2022
[229]

Seed-bench-2: Benchmarking multimodal large language models

Bohao Li, Yuying Ge, Yixiao Ge, Guangzhi Wang, Rui Wang, Ruimao Zhang, and Ying Shan. Seed-bench-2: Benchmarking multimodal large language models. arXiv preprint arXiv:2311.17092, 2023

2023 arXiv
[230]

Seed-bench-2-plus: Benchmarking multimodal large language models with text-rich visual comprehension

Bohao Li, Yuying Ge, Yi Chen, Yixiao Ge, Ruimao Zhang, and Ying Shan. Seed-bench-2-plus: Benchmarking multimodal large language models with text-rich visual comprehension. arXiv preprint arXiv:2404.16790, 2024

2024 arXiv
[231]

Seed-bench: Benchmarking multimodal llms with generative comprehension

Bohao Li, Rui Wang, Guangzhi Wang, Yuying Ge, Yixiao Ge, and Ying Shan. Seed-bench: Benchmarking multimodal llms with generative comprehension. In CVPR, 2024

2024
[232]

Buyuktur, David K

Cheng-Lun Li, Ayse G. Buyuktur, David K. Hutchful, Natasha B. Sant, and Satyendra K. Nainwal. Portalis: using competitive online interactions to support aid initiatives for the homeless. In CHI ’08 extended abstracts on Human factors in computing systems , pages 3873–3878, New...

2008
[233]

Blip-diffusion: Pre-trained subject representation for controllable text-to-image generation and editing

Dongxu Li, Junnan Li, and Steven Hoi. Blip-diffusion: Pre-trained subject representation for controllable text-to-image generation and editing. Advances in Neural Information Processing Systems , 36, 2024

2024
[234]

Vision-language intelligence: Tasks, representation learning, and large models

Feng Li, Hao Zhang, Yi-Fan Zhang, Shilong Liu, Jian Guo, Lionel M Ni, PengChuan Zhang, and Lei Zhang. Vision-language intelligence: Tasks, representation learning, and large models. arXiv preprint arXiv:2203.01922, 2022

2022 arXiv
[235]

Unicoder-vl: A universal encoder for vision and language by cross-modal pre-training

Gen Li, Nan Duan, Yuejian Fang, Ming Gong, and Daxin Jiang. Unicoder-vl: A universal encoder for vision and language by cross-modal pre-training. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 34, pages 11336–11344, 2020

2020
[236]

Incorporating external knowledge to answer open-domain visual questions with dynamic memory networks

Guohao Li, Hang Su, and Wenwu Zhu. Incorporating external knowledge to answer open-domain visual questions with dynamic memory networks. arXiv preprint arXiv:1712.00733, 2017

2017 arXiv
[237]

Boosting visual question answering with context-aware knowledge aggregation

Guohao Li, Xin Wang, and Wenwu Zhu. Boosting visual question answering with context-aware knowledge aggregation. In Proceedings of the 28th ACM International Conference on Multimedia , pages 1227–1235, 2020

2020
[238]

J. Li, R. R. Selvaraju, A. D. Gotmare, S. Joty, C. Xiong, and S. Hoi. Align before fuse: Vision and language representation learning with momentum distillation, 2021

2021
[239]

A survey on deep learning for named entity recognition

Jing Li, Aixin Sun, Jianglei Han, and Chenliang Li. A survey on deep learning for named entity recognition. IEEE Transactions on Knowledge and Data Engineering, 34(1):50–70, 2020

2020
[240]

Align before fuse: Vision and language representation learning with momentum distillation

Junnan Li, Ramprasaath Selvaraju, Akhilesh Gotmare, Shafiq Joty, Caiming Xiong, and Steven Chu Hong Hoi. Align before fuse: Vision and language representation learning with momentum distillation. Advances in neural information processing systems , 34:9694–9705, 2021

2021
[241]

Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation

Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation. arXiv preprint arXiv:2201.12086, 2022

2022 arXiv
[242]

Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning , pages 19730–19742. PMLR, 2023

2023
[243]

Videochat: Chat-centric video understanding

KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355, 2023

2023 arXiv
[245]

Relation-aware graph attention network for visual question answering

Linjie Li, Zhe Gan, Yu Cheng, and Jingjing Liu. Relation-aware graph attention network for visual question answering. In Proceedings of the IEEE/CVF international conference on computer vision , pages 10313–10322, 2019

2019
[246]

Visualbert: A simple and performant baseline for vision and language

Liunian Harold Li, Mark Yatskar, Da Yin, Cho-Jui Hsieh, and Kai-Wei Chang. Visualbert: A simple and performant baseline for vision and language. arXiv preprint arXiv:1908.03557, 2019

1908 arXiv
[247]

Dynamic key-value memory enhanced multi-step graph reasoning for knowledge-based visual question answering

Mingxiao Li and Marie-Francine Moens. Dynamic key-value memory enhanced multi-step graph reasoning for knowledge-based visual question answering. arXiv preprint arXiv:2203.02985, 2022

2022 arXiv
[248]

Inner knowledge-based img2doc scheme for visual question answering

Qun Li, Fu Xiao, Bir Bhanu, Biyun Sheng, and Richang Hong. Inner knowledge-based img2doc scheme for visual question answering. ACM Transactions on Multimedia Computing, Communications, and Applications (TOMM) , 18(3):1–21, 2022

2022
[249]

Inner Knowledge-based Img2Doc Scheme for Visual Question Answering

Qun Li, Fu Xiao, Bir Bhanu, Biyun Sheng, and Richang Hong. Inner Knowledge-based Img2Doc Scheme for Visual Question Answering. ACM Transactions on Multimedia Computing, Communications, and Applications , 18(3):1–21, 2022. ISSN 1551-6857, 1551-6865

2022
[250]

Visual question answering with question representation update (qru)

Ruiyu Li and Jiaya Jia. Visual question answering with question representation update (qru). Advances in Neural Information Processing Systems , 29, 2016

2016
[251]

Unimo: Towards unified-modal understanding and generation via cross-modal contrastive learning

Wei Li, Can Gao, Guocheng Niu, Xinyan Xiao, Hao Liu, Jiachen Liu, Hua Wu, and Haifeng Wang. Unimo: Towards unified-modal understanding and generation via cross-modal contrastive learning. arXiv preprint arXiv:2012.15409, 2020

2012 arXiv
[252]

Understanding the disharmony between dropout and batch normalization by variance shift

Xiang Li, Shuo Chen, Xiaolin Hu, and Jian Yang. Understanding the disharmony between dropout and batch normalization by variance shift. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2682–2690, 2019

2019
[253]

Oscar: Object-semantics aligned pre-training for vision-language tasks

Xiujun Li, Xi Yin, Chunyuan Li, Pengchuan Zhang, Xiaowei Hu, Lei Zhang, Lijuan Wang, Houdong Hu, Li Dong, Furu Wei, et al. Oscar: Object-semantics aligned pre-training for vision-language tasks. In European Conference on Computer Vision , pages 121–137. Springer, 2020

2020
[254]

Mllm-sr: Conversational symbolic regression base multi-modal large language models

Yanjie Li, Weijun Li, Lina Yu, Min Wu, Jingyi Liu, Wenqiang Li, Shu Wei, and Yusong Deng. Mllm-sr: Conversational symbolic regression base multi-modal large language models. arXiv preprint arXiv:2406.05410, 2024

2024 arXiv
[255]

Visual question generation as dual task of visual question answering

Yikang Li, Nan Duan, Bolei Zhou, Xiao Chu, Wanli Ouyang, Xiaogang Wang, and Ming Zhou. Visual question generation as dual task of visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 6116–6124, 2018

2018
[256]

Towards real-world writing assistance: A chinese character checking benchmark with faked and misspelled characters

Yinghui Li, Zishan Xu, Shaoshen Chen, Haojing Huang, Yangning Li, Yong Jiang, Zhongli Li, Qingyu Zhou, Hai-Tao Zheng, and Ying Shen. Towards real-world writing assistance: A chinese character checking benchmark with faked and misspelled characters. arXiv preprint arXiv:2311.11...

2023 arXiv
[257]

Eagle: Elevating geometric reasoning through llm-empowered visual instruction tuning

Zhihao Li, Yao Du, Yang Liu, Yan Zhang, Yufang Liu, Mengdi Zhang, and Xunliang Cai. Eagle: Elevating geometric reasoning through llm-empowered visual instruction tuning. arXiv preprint arXiv:2408.11397, 2024

2024
[258]

Focal visual-text attention for visual question answering

Junwei Liang, Lu Jiang, Liangliang Cao, Li-Jia Li, and Alexander G Hauptmann. Focal visual-text attention for visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition , pages 6135–6143, 2018

2018
[259]

Lrta: a transparent neural-symbolic reasoning framework with modular supervision for visual question answering

Weixin Liang, Feiyang Niu, Aishwarya Reganti, Govind Thattai, and Gokhan Tur. Lrta: a transparent neural-symbolic reasoning framework with modular supervision for visual question answering. arXiv preprint arXiv:2011.10731, 2020

2011 arXiv
[260]

Graphvqa: Language-guided graph neural networks for scene graph question answering.NAACL-HLT 2021, page 79, 2021

Weixin Liang, Yanhao Jiang, and Zixuan Liu. Graphvqa: Language-guided graph neural networks for scene graph question answering.NAACL-HLT 2021, page 79, 2021. Manuscript submitted to ACM 38 Kuang and Xie, et al

2021
[261]

Multi-modal contextual graph neural network for text visual question answering

Yaoyuan Liang, Xin Wang, Xuguang Duan, and Wenwu Zhu. Multi-modal contextual graph neural network for text visual question answering. In 2020 25th International Conference on Pattern Recognition (ICPR) , pages 3491–3498. IEEE, 2021

2020
[262]

Rouge: A package for automatic evaluation of summaries

Chin-Yew Lin. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81, 2004

2004
[263]

Interbert: Vision-and-language interaction for multi-modal pretraining

Junyang Lin, An Yang, Yichang Zhang, Jie Liu, Jingren Zhou, and Hongxia Yang. Interbert: Vision-and-language interaction for multi-modal pretraining. arXiv preprint arXiv:2003.13198, 2020

2003 arXiv
[264]

Microsoft coco: Common objects in context

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision , pages 740–755. Springer, 2014

2014
[265]

Bilinear cnn models for fine-grained visual recognition

Tsung-Yu Lin, Aruni RoyChowdhury, and Subhransu Maji. Bilinear cnn models for fine-grained visual recognition. In Proceedings of the IEEE international conference on computer vision , pages 1449–1457, 2015

2015
[266]

Towards end-to-end unsupervised speech recognition

Alexander H Liu, Wei-Ning Hsu, Michael Auli, and Alexei Baevski. Towards end-to-end unsupervised speech recognition. arXiv preprint arXiv:2204.02492, 2022

2022 arXiv
[267]

Hallusionbench: You see what you think? or you think what you see? an image-context reasoning benchmark challenging for gpt-4v (ision), llava-1.5, and other multi-modality models

Fuxiao Liu, Tianrui Guan, Zongxia Li, Lichang Chen, Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. Hallusionbench: You see what you think? or you think what you see? an image-context reasoning benchmark challenging for gpt-4v (ision), llava-1.5, and other multi-modality models...

arXiv
[268]

Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning

Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel. Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning. Advances in Neural Information Processing Systems , 35:1950–1965, 2022

1950
[269]

Visual instruction tuning, 2023

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning, 2023

2023
[270]

Visual instruction tuning

Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in neural information processing systems , 36, 2024

2024
[271]

Conceptnet—a practical commonsense reasoning tool-kit

Hugo Liu and Push Singh. Conceptnet—a practical commonsense reasoning tool-kit. BT technology journal, 22(4):211–226, 2004

2004
[272]

Swin transformer: Hierarchical vision transformer using shifted windows

Ze Liu, Yutong Lin, Yue Cao, Han Hu, Yixuan Wei, Zheng Zhang, Stephen Lin, and Baining Guo. Swin transformer: Hierarchical vision transformer using shifted windows. In Proceedings of the IEEE/CVF International Conference on Computer Vision , pages 10012–10022, 2021

2021
[273]

Efficient low-rank multimodal fusion with modality-specific factors

Zhun Liu, Ying Shen, Varun Bharadhwaj Lakshminarasimhan, Paul Pu Liang, Amir Zadeh, and Louis-Philippe Morency. Efficient low-rank multimodal fusion with modality-specific factors. arXiv preprint arXiv:1806.00064, 2018

2018 arXiv
[274]

A survey of image classification methods and techniques for improving classification performance

Dengsheng Lu and Qihao Weng. A survey of image classification methods and techniques for improving classification performance. International journal of Remote sensing , 28(5):823–870, 2007

2007
[275]

Hierarchical question-image co-attention for visual question answering

Jiasen Lu, Jianwei Yang, Dhruv Batra, and Devi Parikh. Hierarchical question-image co-attention for visual question answering. Advances in neural information processing systems, 29, 2016

2016
[276]

Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks

Jiasen Lu, Dhruv Batra, Devi Parikh, and Stefan Lee. Vilbert: Pretraining task-agnostic visiolinguistic representations for vision-and-language tasks. Advances in neural information processing systems , 32, 2019

2019
[277]

12-in-1: Multi-task vision and language representation learning

Jiasen Lu, Vedanuj Goswami, Marcus Rohrbach, Devi Parikh, and Stefan Lee. 12-in-1: Multi-task vision and language representation learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 10437–10446, 2020

2020
[278]

R-vqa: learning visual relation facts with semantic attention for visual question answering

Pan Lu, Lei Ji, Wei Zhang, Nan Duan, Ming Zhou, and Jianyong Wang. R-vqa: learning visual relation facts with semantic attention for visual question answering. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining , pages 1880–1889, 2018

2018
[279]

Co-attending free-form regions and detections with multi-modal multiplicative feature embedding for visual question answering

Pan Lu, Hongsheng Li, Wei Zhang, Jianyong Wang, and Xiaogang Wang. Co-attending free-form regions and detections with multi-modal multiplicative feature embedding for visual question answering. In Proceedings of the AAAI Conference on Artificial Intelligence , volume 32, 2018

2018
[280]

Iconqa: A new benchmark for abstract diagram understanding and visual language reasoning

Pan Lu, Liang Qiu, Jiaqi Chen, Tanglin Xia, Yizhou Zhao, Wei Zhang, Zhou Yu, Xiaodan Liang, and Song-Chun Zhu. Iconqa: A new benchmark for abstract diagram understanding and visual language reasoning. In NeurIPS Datasets and Benchmarks , 2021

2021
[281]

Learn to explain: Multimodal reasoning via thought chains for science question answering

Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. Learn to explain: Multimodal reasoning via thought chains for science question answering. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K...

2022
[282]

Mathvista: Evaluating math reasoning in visual contexts with gpt-4v, bard, and other large multimodal models

Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. Mathvista: Evaluating math reasoning in visual contexts with gpt-4v, bard, and other large multimodal models. In ICLR, 2024

2024
[283]

Chameleon: Plug-and-play compositional reasoning with large language models

Pan Lu, Baolin Peng, Hao Cheng, Michel Galley, Kai-Wei Chang, Ying Nian Wu, Song-Chun Zhu, and Jianfeng Gao. Chameleon: Plug-and-play compositional reasoning with large language models. Advances in Neural Information Processing Systems , 36, 2024

2024
[284]

Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity

Yao Lu, Max Bartolo, Alastair Moore, Sebastian Riedel, and Pontus Stenetorp. Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1...

2022
[285]

Visual question answering with memory- augmented networks

Chao Ma, Chunhua Shen, Anthony Dick, Qi Wu, Peng Wang, Anton van den Hengel, and Ian Reid. Visual question answering with memory- augmented networks. In Proceedings of the IEEE conference on computer vision and pattern recognition , pages 6975–6984, 2018

2018
[286]

Robust visual question answering: Datasets, methods, and future challenges

Jie Ma, Pinghui Wang, Dechen Kong, Zewei Wang, Jun Liu, Hongbin Pei, and Junzhou Zhao. Robust visual question answering: Datasets, methods, and future challenges. IEEE Transactions on Pattern Analysis and Machine Intelligence , pages 1–20, 2024. doi: 10.1109/TPAMI.2024.3366154

2024
[287]

Learning to answer questions from image using convolutional neural network

Lin Ma, Zhengdong Lu, and Hang Li. Learning to answer questions from image using convolutional neural network. In Thirtieth AAAI Conference on Artificial Intelligence, 2016. Manuscript submitted to ACM Natural Language Understanding and Inference with MLLM in Visual Question A...

2016
[288]

A review of 3d reconstruction techniques in civil engineering and their applications.Advanced Engineering Informatics, 37:163–174, 2018

Zhiliang Ma and Shilong Liu. A review of 3d reconstruction techniques in civil engineering and their applications.Advanced Engineering Informatics, 37:163–174, 2018

2018
[289]

A multi-world approach to question answering about real-world scenes based on uncertain input

Mateusz Malinowski and Mario Fritz. A multi-world approach to question answering about real-world scenes based on uncertain input. Advances in neural information processing systems , 27, 2014

2014
[290]

Ask your neurons: A neural-based approach to answering questions about images

Mateusz Malinowski, Marcus Rohrbach, and Mario Fritz. Ask your neurons: A neural-based approach to answering questions about images. In Proceedings of the IEEE international conference on computer vision , pages 1–9, 2015

2015
[291]

The stanford corenlp natural language processing toolkit

Christopher D Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel, Steven Bethard, and David McClosky. The stanford corenlp natural language processing toolkit. In Proceedings of 52nd annual meeting of the association for computational linguistics: system demonstrations , p...

2014
[292]

Ok-vqa: A visual question answering benchmark requiring external knowledge

Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. Ok-vqa: A visual question answering benchmark requiring external knowledge. In Proceedings of the IEEE/cvf conference on computer vision and pattern recognition , pages 3195–3204, 2019

2019
[293]

Krisp: Integrating implicit and symbolic knowledge for open-domain knowledge-based vqa

Kenneth Marino, Xinlei Chen, Devi Parikh, Abhinav Gupta, and Marcus Rohrbach. Krisp: Integrating implicit and symbolic knowledge for open-domain knowledge-based vqa. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , pages 14111–14121, 2021

2021
[294]

Docvqa: A dataset for vqa on document images

Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa on document images. In Proceedings of the IEEE/CVF winter conference on applications of computer vision , pages 2200–2209, 2021

2021
[295]

Infographicvqa

Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and CV Jawahar. Infographicvqa. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision , pages 1697–1706, 2022

2022
[296]

McCracken and Donald G

Daniel D. McCracken and Donald G. Golden. Simplified Structured COBOL with Microsoft/MicroFocus COBOL . John Wiley & Sons, Inc., New York, NY, USA, 1990. ISBN 0471514071

1990
[297]

Sentiment analysis algorithms and applications: A survey

Walaa Medhat, Ahmed Hassan, and Hoda Korashy. Sentiment analysis algorithms and applications: A survey. Ain Shams engineering journal , 5(4): 1093–1113, 2014

2014
[1993]

doi: 10.1145/161468.161469
[2006]

URL http://portal.acm.org/citation.cfm?id=887006.887010

ISBN 1-59140-056-2. URL http://portal.acm.org/citation.cfm?id=887006.887010
[2010]

doi: https://doi.org/10.1137/080734467

ISSN 0097-5397. doi: https://doi.org/10.1137/080734467. URL http://dx.doi.org/10.1137/080734467

doi

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.

[1] [5]

Clifford: a maple 11 package for clifford algebra computations, version 11, 2007

Rafal Ablamowicz and Bertfried Fauser. Clifford: a maple 11 package for clifford algebra computations, version 11, 2007. URL http://math.tntech. edu/rafal/cliff11/index.html

2007

[2] [16]

American Mathematical Society, April 2015

Using the amsthm Package . American Mathematical Society, April 2015. http://www.ctan.org/pkg/amsthm

2015

[3] [99]

The title of book two , chapter 100

Ian Editor, editor. The title of book two , chapter 100. The name of the series two. University of Chicago Press, Chicago, 2nd. edition, 2008. doi: 10.1007/3-540-09237-4. URL http://dx.doi.org/10.1007/3-540-09456-9

doi:10.1007/3-540-09237-4 2008

[4] [103]

Publication quality tables in LATEX, April 2005

Simon Fear. Publication quality tables in LATEX, April 2005. http://www.ctan.org/pkg/booktabs

2005

[5] [46]

Buss, Arnold L

Jonathan F. Buss, Arnold L. Rosenberg, and Judson D. Knott. Vertex types in book-embeddings. Technical report, Amherst, MA, USA, 1987

1987

[6] [68]

Clarkson

Kenneth L. Clarkson. Algorithms for Closest-Point Problems (Computational Geometry) . PhD thesis, Stanford University, Palo Alto, CA, 1985. UMI Order Number: AAT 8506171

1985

[7] [69]

Algorithms for Closest-Point Problems (Computational Geometry)

Kenneth Lee Clarkson. Algorithms for Closest-Point Problems (Computational Geometry) . PhD thesis, Stanford University, Stanford, CA, USA, 1985. AAT 8506171

1985

[8] [72]

Mancini, and Alessandro Mei

Mauro Conti, Roberto Di Pietro, Luigi V. Mancini, and Alessandro Mei. (old) distributed data source verification in wireless sensor networks. Inf. Fusion, 10(4):342–353, 2009. ISSN 1566-2535. doi: http://dx.doi.org/10.1016/j.inffus.2009.01.002

doi:10.1016/j.inffus.2009.01.002 2009

[9] [73]

Mancini, and Alessandro Mei

Mauro Conti, Roberto Di Pietro, Luigi V. Mancini, and Alessandro Mei. (new) distributed data source verification in wireless sensor networks. Inf. Fusion, 10(4):342–353, October 2009. ISSN 1566-2535. doi: 10.1016/j.inffus.2009.01.002. URL http://portal.acm.org/citation.cfm?id=1555009.1555162

work page arXiv 2009

[10] [211]

E-commerce and cultural values (Inbook-num chap) , chapter (in type field) 22, pages 51–74

Wei-Chang Kong. E-commerce and cultural values (Inbook-num chap) , chapter (in type field) 22, pages 51–74. IGI Publishing, Hershey, PA, USA,