Pith. sign in

REVIEW 4 major objections 8 minor 51 references

Outside Knowledge Conversational Video (OKCV) Dataset -- Dialoguing over Videos

T0 review · 4 major / 8 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces OKCV, a benchmark of 2,017 videos with 5,986 human-annotated dialogues that require a model to ground answers in specific video segments while drawing on knowledge not present in the video.

desk verdict A useful, released video-dialogue dataset with a novel knowledge+conversation combination, but its central 'visually grounded' property is under-validated because questions are drafted from transcripts alone. read the letter →

arxiv 2506.09953 v1 pith:AY5RI36R submitted 2025-06-11 cs.CV cs.AIcs.CL

Benjamin Reichman , Constantin Patsch , Jack Truxal , Atishay Jain , Larry Heck This is my paper
classification cs.CVcs.AIcs.CL
keywords OKCVdatasetoutsideknowledgeVQAvideodialoguevisualgroundinghuman-annotateddialoguestemporalcertificatesquestiondecompositionunderstanding
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that a conversational agent watching a video must do three things at once: locate the relevant moments in the video, bring in world knowledge that is not visible in those moments, and keep track of an ongoing dialogue. It introduces OKCV, a dataset of 2,017 videos with 5,986 human-annotated dialogues totaling 40,954 interleaved turns, where each turn is visually grounded in a specific video segment but the questions require outside knowledge. If the dataset works as claimed, it fills a gap no previous benchmark covers: video-language models are currently tested either on video question answering, on knowledge-based image QA, or on visually grounded dialogue, but not on the combination. The paper also provides baselines showing that open-source video-language models score low on standard similarity metrics, while a closed-source model scores high on one metric and low on another, suggesting the task is not yet solved.

What carries the argument

The central mechanism is a human-in-the-loop collection pipeline. GPT-4 receives a video transcript and a few-shot prompt that requires each generated question to be related to the video, to demand outside knowledge, and to need multiple frames, and it is asked to decompose one high-level question into dependent sub-questions; human annotators then watch the video, answer and edit the questioner's side, and record temporal certificates and any external sources they used. The temporal certificate, borrowed from EgoSchema, is an interval or set of intervals of the video that suffices to answer the dialogue, and it is what lets the dataset claim genuine visual grounding. The DeepSeek-R1-based classification of questions into external-knowledge versus vision-only is the quantitative support for the outside-knowledge property.

What would settle it

Take a random sample of about 200 OKCV dialogues and have one group of annotators answer after watching the video while a separate group answers from the transcript alone; if the transcript-only group achieves comparable correctness and information content, the visual-grounding claim collapses. Alternatively, have expert annotators judge whether each question genuinely needs outside knowledge; if the human-verified proportion falls well below 57.93%, the outside-knowledge claim fails.

Watch

Extended reading notes

Core claim

The central claim is that OKCV is the first human-annotated dataset requiring a conversational agent to combine temporal visual grounding, external knowledge, and dialogue context in a single task. The dataset contains 2,017 videos drawn from QuerYD, with 5,986 human-completed dialogues averaging 6.8 turns; each dialogue is built from a GPT-4-generated chain of top-down decomposed questions that annotators answer, may edit, and supplement with temporal certificates marking the video segments needed. An analysis using DeepSeek-R1 estimates that 57.93% of the questions cannot be answered from visual information alone, supporting the outside-knowledge property. Baseline experiments show open-source video-language models produce low similarity scores on Bleurt and BartScore, while Gemini-1.5-pro reaches a high Bleurt score but the lowest BartScore, which the paper interprets as evidence of a large capability gap and of metric disagreement on what counts as a good dialogue answer.

Load-bearing premise

The load-bearing premise is that the GPT-4-generated questions, drafted from transcripts, really do require watching the video and really do require outside knowledge; the paper does not verify this at scale with human annotators, since the 57.93% figure comes from a model-based classification.

Editorial extensions

If this is right

  • If the benchmark holds, video dialogue systems will need joint spatiotemporal grounding, external knowledge retrieval, and dialogue state tracking, rather than treating video QA and dialogue as separate modules.
  • The 57.93% external-knowledge estimate implies that roughly two-fifths of OKCV questions might be answerable from visual or transcript information alone, so the dataset tests a mix of skills rather than a pure outside-knowledge setting.
  • The large gap between Gemini-1.5-pro's Bleurt score and its BartScore shows that reference-based dialogue metrics disagree strongly, meaning evaluation methodology itself is a bottleneck for progress on this task.
  • Fine-tuning improves Bleurt and BartScore for most open-source baselines but does not consistently improve Prometheus-2 scores, suggesting that fine-tuning helps surface similarity without clearly improving rubric-based answer quality.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the questions are drafted from transcripts rather than from the video pixels, a likely risk is that some 'visually grounded' questions are actually transcript-grounded; a natural test is to compare answer quality when models are given the transcript alone versus the full video.
  • The top-down decomposition of questions into dependent sub-questions may let a model answer later turns by exploiting earlier answers alone, without watching the video; an ablation that hides the dialogue history could measure how much genuine multi-turn reasoning is required.
  • The temporal certificates provide a supervised signal for video moment retrieval, so the dataset could double as a benchmark for localizing the segments that support each dialogue turn.
  • The annotator-supplied sources (641 sources across 550 dialogues) could support a retrieval-oriented extension where models must pull knowledge from those exact references rather than from parametric memory.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 8 minor

Summary. The paper introduces OKCV, a video dialogue dataset intended to combine visual grounding in video with external knowledge. The dataset contains 2,017 videos from QuerYD, with 5,986 human-completed dialogues and 40,954 turns. The collection pipeline uses GPT-4 to generate question-side dialogue drafts from video transcripts, after which crowdworkers provide answers, optionally edit the drafts, list external sources, and mark temporal intervals (temporal certificates) needed to answer. The paper reports dataset statistics, an LLM-based estimate that 57.93% of questions require external knowledge, and zero-shot and fine-tuned baselines using several language and vision-language models with different knowledge augmentations. The authors frame OKCV as the first human-annotated video dialogue benchmark that combines visual grounding with outside knowledge.

Significance. If the defining properties were verified, OKCV would be a valuable resource: it is released publicly, is not restricted to a narrow domain, and its top-down decomposed dialogues with temporal certificates could support research in video dialogue, retrieval, and knowledge integration. The dataset's scale (5,986 dialogues, 40,954 turns) is substantial. However, the evidence that the dialogues actually require both visual grounding and outside knowledge is weak, and one of the paper's own validation numbers partially contradicts the abstract's claim. The baselines, while extensive, do not currently demonstrate that the video modality is necessary for the task. The contribution is therefore promising but requires stronger validation before the central claims can be accepted.

major comments (4)
  1. [Abstract; Section 4] The abstract states that 'questions further require external knowledge that is not visually present,' but Section 4 reports that DeepSeek-R1 classifies only 57.93% of questions as requiring external knowledge. This means the defining property of the dataset fails for roughly 42% of the questions. The dataset should either be filtered to include only questions that meet this criterion, or the abstract and title should be revised to reflect that a majority, not all, questions require outside knowledge.
  2. [Section 3.2; Appendix 7.1] Because GPT-4 drafts the questions from video transcripts alone, the prompt instruction that questions require multiple frames is not verifiable from text, and no later step in the pipeline demonstrates that a question cannot be answered from the transcript alone. Human annotators watch the video to write answers, but the answers could be produced from the transcript as well. A text-only baseline (e.g., an LLM given only the dialogue and transcript) is missing; the zero-shot 'Nothing' results in Table 3 show that language-only models already achieve non-trivial Bleurt scores, which is suggestive but not conclusive. Please add a text-only ablation and, ideally, human judgments of whether each question is visually grounded.
  3. [Section 4] The validation that 57.93% of questions require external knowledge is not independent: the questions were generated by GPT-4 under a prompt requiring outside knowledge, and the estimate is produced by DeepSeek-R1, another LLM. There is no human verification of this property. Moreover, the annotation interface in Section 7.2 already asks turkers whether each dialogue requires more visual, commonsense, or factoid reasoning; these human judgments appear not to be analyzed in the paper, even though they could serve as a validation signal. Please report these annotations and/or conduct a human evaluation of a random sample.
  4. [Section 5; Table 3] The text claims that 'both metrics agree that Llama2-7B, when utilizing both captions and external knowledge, achieves the best performance among the open-source models,' but Table 3 does not support this: Video-Llava with no knowledge achieves Bleurt 0.59, exceeding Llama2-7B's 0.58, and Prometheus-2 ranks Phi-3-Small (Nothing) highest at 3.74. BartScore and Bleurt also disagree with Prometheus-2 on the best model. Please clarify how the overall performance conclusion is derived despite these conflicts.
minor comments (8)
  1. [Abstract; Section 3.3] The term 'human-annotated dialogues' is potentially misleading because the question-side turns are generated by GPT-4; consider describing the dataset as 'human-completed' or clarifying the split of labor in the abstract.
  2. [Section 3.2] The sentence 'we employed a combination of the transcripts and a LLM (GPT-4)' is vague; the appendix prompt shows only the LLM using the transcript, so please clarify how the combination works.
  3. [Table 1] The dialogue count for OKCV is shown as '5.7k' while the text reports 5,986; please make the numbers consistent.
  4. [Section 3.3] The temporal certificate description asks annotators for the closest begin and end time needed to complete a dialogue, but the interface seems to collect these per dialogue; please clarify whether certificates are per dialogue or per question.
  5. [Figure 2] The overlap between the two histograms makes the distribution hard to read; consider using separate panels or a log scale.
  6. [Section 5] The model is called 'Video-Llava' in the results table but 'Video-Llama' in the text (first paragraph of Section 5); please unify the naming.
  7. [Section 4] The mention of a manual check of a randomly sampled subset lacks details; please provide the sample size and any agreement measures.
  8. [References] The reference to 'Sanket Shah and Talukdar, 2019' is formatted inconsistently with the other Shah et al. entries; please check the bibliography.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the dataset-construction pipeline and the DeepSeek-R1 statistic involve a validity/independence caveat, but no derivation reduces to its own inputs.

full rationale

This is a dataset-construction and benchmarking paper, not a derivation whose conclusion is equivalent to its assumptions. The central claim is that OKCV dialogues are visually grounded and require outside knowledge; that property is a design goal of the collection pipeline. Section 3.2 prompts GPT-4 to satisfy the criteria 'The question must be related to the video,' 'The question must require outside knowledge,' and 'The question should require multiple frames to answer,' and Section 3.3 has human annotators watch the video, edit the GPT-4 drafts, provide temporal certificates, and list external sources. The Section 4 estimate that 57.93% of questions require external knowledge comes from DeepSeek-R1, an LLM-based classifier, rather than from independent human judgment; this is a real limitation on the strength of the quantitative claim, because the property was prompted into the data and then re-measured with a model of the same general type. However, this is a support-gap and validity concern, not circularity in the sense of an equation or fitted parameter reducing to its own input: there is no construction by which the measured percentage is forced to equal the prompt criterion. The Limitations section itself flags the transcript-bias risk ('biases in the transcriptions can influence the questions generated by GPT-4, subsequently affecting the dialogues produced'), which is the correct place for the main caveat: questions were drafted from transcripts, so 'visually grounded' would benefit from a text-only ablation, but that is an omitted verification step rather than a circular step. The only self-citations (Reichman et al. 2023 for OK-VQA v2.0 and Reichman and Heck 2023 for the Wikidata subset) appear in related work and retrieval-corpus selection and are not load-bearing for the dataset's central claim. No circular step can be exhibited from the paper's own equations or definitions.

Assumptions & free parameters 0 free parameters · 3 assumptions · 0 invented entities

The central claims rest on dataset-construction assumptions rather than mathematical axioms. The properties of the dialogues (visual grounding, outside-knowledge requirement) are assumed from the generation pipeline and model-based classification rather than fully validated with human judgment.

assumptions (3)
  • domain assumption GPT-4 generated questions adhere to the stated criteria: related to the video, require outside knowledge, require multiple frames.
    Section 3.2 instructs GPT-4 to enforce these criteria, but no large-scale human verification is reported.
  • domain assumption Human annotators provide accurate answers, sources, and temporal certificates.
    Section 3.3 describes the AMT interface, but inter-annotator agreement and quality-control metrics are not reported.
  • domain assumption DeepSeek-R1 classification correctly identifies questions that require external knowledge.
    Section 4 uses this model to estimate that 57.93% of questions require external knowledge, without human validation of the classifier.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Outside Knowledge Conversational Video (OKCV) Dataset -- Dialoguing over Videos." pith.science (2026). https://pith.science/paper/AY5RI36R

@misc{pith2026250609953,
  author       = {Pith},
  title        = {Pith review of: Outside Knowledge Conversational Video (OKCV) Dataset -- Dialoguing over Videos},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/AY5RI36R}},
  note         = {Machine review of arXiv:2506.09953}
}
abstract

In outside knowledge visual question answering (OK-VQA), the model must identify relevant visual information within an image and incorporate external knowledge to accurately respond to a question. Extending this task to a visually grounded dialogue setting based on videos, a conversational model must both recognize pertinent visual details over time and answer questions where the required information is not necessarily present in the visual information. Moreover, the context of the overall conversation must be considered for the subsequent dialogue. To explore this task, we introduce a dataset comprised of $2,017$ videos with $5,986$ human-annotated dialogues consisting of $40,954$ interleaved dialogue turns. While the dialogue context is visually grounded in specific video segments, the questions further require external knowledge that is not visually present. Thus, the model not only has to identify relevant video parts but also leverage external knowledge to converse within the dialogue. We further provide several baselines evaluated on our dataset and show future challenges associated with this task. The dataset is made publicly available here: https://github.com/c-patsch/OKCV.

Figures

Figures reproduced from arXiv: 2506.09953 by the authors.

Figure 1
Figure 1. An example conversation from the OKCV dataset. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Distribution over the temporal certificate ranges as well as the video lengths in OKCV. [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Examples of visually grounded conversations and corresponding video snippets from the OKCV dataset. [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Screenshot of example given to Turker of how the collected data should look. [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: System prompt given to GPT-4 to create partial dialogue scripts. Bold is added for emphasis. [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]
Figure 6
Figure 6. Figure 6: Screenshot example of Amazon Mechanical Turk interface at the beginning of the dialogue collection [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]
Figure 7
Figure 7. Figure 7: Screenshot example of Amazon Mechanical Turker interface at the end of the dialogue collection task. [PITH_FULL_IMAGE:figures/full_fig_p015_7.png]
Figure 8
Figure 8. Figure 8: Prompt given to Prometheus 2 to evaluate model outputs. [PITH_FULL_IMAGE:figures/full_fig_p015_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

51 extracted references · 23 canonical work pages

  1. [1]

    Huda Alamri, Vincent Cartillier, Abhishek Das, Jue Wang, Anoop Cherian, Irfan Essa, Dhruv Batra, Tim K Marks, Chiori Hori, Peter Anderson, et al. 2019. Audio visual scene-aware dialog. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7558--7567

    work page 2019

  2. [2]

    Stanislaw Antol, Aishwarya Agrawal, Jiasen Lu, Margaret Mitchell, Dhruv Batra, C Lawrence Zitnick, and Devi Parikh. 2015. Vqa: Visual question answering. In Proceedings of the IEEE international conference on computer vision, pages 2425--2433

    2015

  3. [3]

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877--1901

    2020

  4. [4]

    Pawe Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Inigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Ga s i \'c . 2018. Multiwoz--a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. arXiv preprint arXiv:1810.00278

    arXiv 2018

  5. [5]

    Bill Byrne, Karthik Krishnamoorthi, Chinnadhurai Sankar, Arvind Neelakantan, Daniel Duckworth, Semih Yavuz, Ben Goodrich, Amit Dubey, Andy Cedilnik, and Kyu-Young Kim. 2019. Taskmaster-1: Toward a realistic and diverse dialog dataset. arXiv preprint arXiv:1909.05358

    arXiv 2019

  6. [6]

    Abhishek Das, Satwik Kottur, Khushi Gupta, Avi Singh, Deshraj Yadav, Jos \'e MF Moura, Devi Parikh, and Dhruv Batra. 2017. Visual dialog. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 326--335

    2017

  7. [7]

    Deniz Engin, Fran c ois Schnitzler, Ngoc QK Duong, and Yannis Avrithis. 2021. On the hidden treasure of dialog in video question answering. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 2064--2073

    work page 2021

  8. [8]

    Tsu-Jui Fu, Linjie Li, Zhe Gan, Kevin Lin, William Yang Wang, Lijuan Wang, and Zicheng Liu. 2021. Violet: End-to-end video-language transformers with masked visual-token modeling. arXiv preprint arXiv:2111.12681

    arXiv 2021

Show all 51 references
  1. [9]

    Feng Gao, Qing Ping, Govind Thattai, Aishwarya Reganti, Ying Nian Wu, and Prem Natarajan. 2022. Transform-retrieve-generate: Natural language-centric outside-knowledge visual question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogniti...

    2022

  2. [10]

    Haoyuan Gao, Junhua Mao, Jie Zhou, Zhiheng Huang, Lei Wang, and Wei Xu. 2015. Are you talking to a machine? dataset and methods for multilingual image question. Advances in neural information processing systems, 28

    2015

  3. [11]

    Noa Garcia, Mayu Otani, Chenhui Chu, and Yuta Nakashima. 2020. Knowit vqa: Answering knowledge-based questions about videos. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 10826--10834

    2020

  4. [12]

    Yash Goyal, Tejas Khot, Douglas Summers-Stay, Dhruv Batra, and Devi Parikh. 2017. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 6904--6913

    2017

  5. [13]

    Liangke Gui, Borui Wang, Qiuyuan Huang, Alex Hauptmann, Yonatan Bisk, and Jianfeng Gao. 2021. Kat: A knowledge augmented transformer for vision-and-language. arXiv preprint arXiv:2112.08614

    2021 arXiv

  6. [14]

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948

    2025 arXiv

  7. [15]

    Pranay Gupta and Manish Gupta. 2022. https://api.semanticscholar.org/CorpusID:246652503 Newskvqa: Knowledge-aware news video question answering . In Pacific-Asia Conference on Knowledge Discovery and Data Mining

    2022

  8. [16]

    Matthew Honnibal and Ines Montani. 2017. spaCy 2 : Natural language understanding with B loom embeddings, convolutional neural networks and incremental parsing. To appear

    2017

  9. [17]

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. https://openreview.net/forum?id=nZeVKeeFYf9 Lo RA : Low-rank adaptation of large language models . In International Conference on Learning Representations

    2022

  10. [18]

    Seungone Kim, Juyoung Suk, Shayne Longpre, Bill Yuchen Lin, Jamin Shin, Sean Welleck, Graham Neubig, Moontae Lee, Kyungjae Lee, and Minjoon Seo. 2024. https://arxiv.org/abs/2405.01535 Prometheus 2: An open source language model specialized in evaluating other language models ....

    2024 arXiv

  11. [19]

    Jie Lei, Licheng Yu, Mohit Bansal, and Tamara L Berg. 2018. Tvqa: Localized, compositional video question answering. In Empirical Methods in Natural Language Processing

    2018

  12. [20]

    Jie Lei, Licheng Yu, Tamara L Berg, and Mohit Bansal. 2019. Tvqa+: Spatio-temporal grounding for video question answering. arXiv preprint arXiv:1904.11574

    2019 arXiv

  13. [21]

    KunChang Li, Yinan He, Yi Wang, Yizhuo Li, Wenhai Wang, Ping Luo, Yali Wang, Limin Wang, and Yu Qiao. 2023. Videochat: Chat-centric video understanding. arXiv preprint arXiv:2305.06355

    2023 arXiv

  14. [22]

    Yuanze Lin, Yujia Xie, Dongdong Chen, Yichong Xu, Chenguang Zhu, and Lu Yuan. 2022. Revive: Regional visual representation matters in knowledge-based visual question answering. Advances in Neural Information Processing Systems, 35:10560--10571

    2022

  15. [23]

    Pierre Lison and J \"o rg Tiedemann. 2016. Opensubtitles2016: Extracting large parallel corpora from movie and tv subtitles

    2016

  16. [24]

    Huaishao Luo, Lei Ji, Ming Zhong, Yang Chen, Wen Lei, Nan Duan, and Tianrui Li. 2022. Clip4clip: An empirical study of clip for end to end video clip retrieval and captioning. Neurocomputing, 508:293--304

    2022

  17. [25]

    Karttikeya Mangalam, Raiymbek Akshulakov, and Jitendra Malik. 2024. Egoschema: A diagnostic benchmark for very long-form video language understanding. Advances in Neural Information Processing Systems, 36

    2024

  18. [26]

    Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. 2019. Ok-vqa: A visual question answering benchmark requiring external knowledge. In Proceedings of the IEEE/cvf conference on computer vision and pattern recognition, pages 3195--3204

    2019

  19. [27]

    Yuxian Meng, Shuhe Wang, Qinghong Han, Xiaofei Sun, Fei Wu, Rui Yan, and Jiwei Li. 2020. Openvidial: A large-scale, open-domain dialogue dataset with visual contexts. arXiv preprint arXiv:2012.15015

    2020 arXiv

  20. [28]

    Liqiang Nie, Wenjie Wang, Richang Hong, Meng Wang, and Qi Tian. 2019. Multimodal dialog system: Generating responses via adaptive decoders. In Proceedings of the 27th ACM international conference on multimedia, pages 1098--1106

    2019

  21. [29]

    Andreea-Maria Oncescu, Joao F Henriques, Yang Liu, Andrew Zisserman, and Samuel Albanie. 2021. Queryd: A video dataset with high-quality text and audio narrations. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 2265...

    2021

  22. [30]

    OpenAI. 2023. https://api.semanticscholar.org/CorpusID:257532815 Gpt-4 technical report

    2023

  23. [31]

    Amy Pu, Hyung Won Chung, Ankur P Parikh, Sebastian Gehrmann, and Thibault Sellam. 2021. Learning compact metrics for mt. In Proceedings of EMNLP

    2021

  24. [32]

    Benjamin Reichman and Larry Heck. 2023. Cross-modal dense passage retrieval for outside knowledge visual question answering. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pages 2837--2842

    2023

  25. [33]

    Benjamin Z Reichman, Anirudh Sundar, Christopher Richardson, Tamara Zubatiy, Prithwijit Chowdhury, Aaryan Shah, Jack Truxal, Micah Grimes, Dristi Shah, Woo Ju Chee, et al. 2023. Outside knowledge visual question answering version 2.0. In ICASSP 2023-2023 IEEE International Con...

    2023

  26. [34]

    Naganand Yadati Sanket Shah, Anand Mishra and Partha Pratim Talukdar. 2019. Kvqa: Knowledge-aware visual question answering. In AAAI

    2019

  27. [35]

    Dustin Schwenk, Apoorv Khandelwal, Christopher Clark, Kenneth Marino, and Roozbeh Mottaghi. 2022. A-okvqa: A benchmark for visual question answering using world knowledge. arXiv

    2022

  28. [36]

    Thibault Sellam, Dipanjan Das, and Ankur P Parikh. 2020. Bleurt: Learning robust metrics for text generation. In Proceedings of ACL

    2020

  29. [37]

    Sanket Shah, Anand Mishra, Naganand Yadati, and Partha Pratim Talukdar. 2019. Kvqa: Knowledge-aware visual question answering. In Proceedings of the AAAI conference on artificial intelligence, volume 33, pages 8876--8884

    2019

  30. [38]

    Zhenwei Shao, Zhou Yu, Meng Wang, and Jun Yu. 2023. Prompting large language models with answer heuristics for knowledge-based visual question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14974--14983

    2023

  31. [39]

    Alessandro Sordoni, Michel Galley, Michael Auli, Chris Brockett, Yangfeng Ji, Margaret Mitchell, Jian-Yun Nie, Jianfeng Gao, and Bill Dolan. 2015. A neural network approach to context-sensitive generation of conversational responses. arXiv preprint arXiv:1506.06714

    2015 arXiv

  32. [40]

    Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of general knowledge. In Proceedings of the AAAI conference on artificial intelligence, volume 31

    2017

  33. [41]

    Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. Commonsenseqa: A question answering challenge targeting commonsense knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human ...

    2019

  34. [42]

    Makarand Tapaswi, Yukun Zhu, Rainer Stiefelhagen, Antonio Torralba, Raquel Urtasun, and Sanja Fidler. 2016. Movieqa: Understanding stories in movies through question-answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 4631--4640

    2016

  35. [43]

    Denny Vrande c i \'c and Markus Kr \"o tzsch. 2014. Wikidata: a free collaborative knowledgebase. Communications of the ACM, 57(10):78--85

    2014

  36. [44]

    Peng Wang, Qi Wu, Chunhua Shen, Anthony Dick, and Anton Van Den Hengel. 2017. Fvqa: Fact-based visual question answering. IEEE transactions on pattern analysis and machine intelligence, 40(10):2413--2427

    2017

  37. [45]

    Peng Wang, Qi Wu, Chunhua Shen, Anton van den Hengel, and Anthony Dick. 2015. Explicit knowledge-based reasoning for visual question answering. arXiv preprint arXiv:1511.02570

    2015 arXiv

  38. [46]

    Haiyang Xu, Qinghao Ye, Ming Yan, Yaya Shi, Jiabo Ye, Yuanhong Xu, Chenliang Li, Bin Bi, Qi Qian, Wei Wang, et al. 2023. mplug-2: A modularized multi-modal foundation model across text, image and video. In International Conference on Machine Learning, pages 38728--38748. PMLR

    2023

  39. [47]

    Zhengyuan Yang, Zhe Gan, Jianfeng Wang, Xiaowei Hu, Yumao Lu, Zicheng Liu, and Lijuan Wang. 2022. An empirical study of gpt-3 for few-shot knowledge-based vqa. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 36, pages 3081--3089

    2022

  40. [48]

    Weizhe Yuan, Graham Neubig, and Pengfei Liu. 2021. https://proceedings.neurips.cc/paper/2021/file/e4d2b6e6fdeca3e60e0f1a62fee3d9dd-Paper.pdf Bartscore: Evaluating generated text as text generation . In Advances in Neural Information Processing Systems, volume 34, pages 27263--...

    2021

  41. [49]

    Rowan Zellers, Ximing Lu, Jack Hessel, Youngjae Yu, Jae Sung Park, Jize Cao, Ali Farhadi, and Yejin Choi. 2021. Merlot: Multimodal neural script knowledge models. Advances in Neural Information Processing Systems, 34:23634--23651

    2021

  42. [50]

    online" 'onlinestring :=

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint eprinttype howpublished institution journal key month note number organization pages publisher school series title type volume year doi pubmed url lastchecked label extra.label sort.label short.list...

  43. [51]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.