Pith. sign in

REVIEW 4 cited by

OK-VQA: A Visual Question Answering Benchmark Requiring External Knowledge

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 1906.00067 v2 pith:FJAKI52K submitted 2019-05-31 cs.CV cs.CL

Kenneth Marino , Mohammad Rastegari , Ali Farhadi , Roozbeh Mottaghi This is my paper
classification cs.CVcs.CL
keywords knowledgevisualansweringdatasetexternalknowledge-basedquestionquestions
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Visual Question Answering (VQA) in its ideal form lets us study reasoning in the joint space of vision and language and serves as a proxy for the AI task of scene understanding. However, most VQA benchmarks to date are focused on questions such as simple counting, visual attributes, and object detection that do not require reasoning or knowledge beyond what is in the image. In this paper, we address the task of knowledge-based visual question answering and provide a benchmark, called OK-VQA, where the image content is not sufficient to answer the questions, encouraging methods that rely on external knowledge resources. Our new dataset includes more than 14,000 questions that require external knowledge to answer. We show that the performance of the state-of-the-art VQA models degrades drastically in this new setting. Our analysis shows that our knowledge-based VQA task is diverse, difficult, and large compared to previous knowledge-based VQA datasets. We hope that this dataset enables researchers to open up new avenues for research in this domain. See http://okvqa.allenai.org to download and browse the dataset.

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. V-RoAst: Visual Road Assessment. Can VLM be a Road Safety Assessor Using the iRAP Standard?

    cs.CV 2024-08 unverdicted novelty 7.0 of 10

    V-RoAst applies zero-shot VLMs (Gemini-1.5-flash, GPT-4o-mini) to iRAP road safety attribute classification on a new ThaiRAP image dataset and compares them to CNN baselines, finding better generalization to unseen cl...

  2. From Seeing to Thinking: Decoupling Perception and Reasoning Improves Post-Training of Vision-Language Models

    cs.CL 2026-05 conditional novelty 6.0 of 10

    Staged post-training that first solidifies visual perception before visual and textual reasoning improves VLM accuracy and shortens reasoning traces on visual math and perception benchmarks.

  3. Towards Meta-Cognitive Knowledge Editing for Multimodal LLMs

    cs.AI 2025-09 conditional novelty 6.0 of 10

    CogEdit and MIND shift multimodal knowledge editing toward evaluating and enabling meta-cognitive skills: self-awareness, boundary monitoring, and noise robustness.

  4. PADReg: Physics-Aware Deformable Registration Guided by Contact Force for Ultrasound Sequences

    cs.CV 2025-08 unverdicted novelty 6.0 of 10

    PADReg combines contact force with a learned stiffness map and a Hooke's-law-inspired module to make deformable ultrasound registration more accurate and physically interpretable.

Pith tools