Pith. sign in

REVIEW 3 major objections 6 minor 43 references

Language-driven Description Generation and Common Sense Reasoning for Video Action Recognition

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper claims that a video action recognizer can be substantially improved by using a large language model to generate natural-language descriptions of the current scene and of the next action from predicted objects, interactions, and…

desk verdict A plausible engineering pipeline undermined by a missing control: the reported gains may come from re-encoding the predicted activity verbs, not from common-sense reasoning. read the letter →

arxiv 2506.16701 v1 pith:MRMXTXHT submitted 2025-06-20 cs.CV

Xiaodan Hu , Chuhang Zou , Suchen Wang , Jaechul Kim , Narendra Ahuja This is my paper · ORCID
classification cs.CV
keywords videoactionrecognitioncommonsensereasoninglargelanguagemodelsprompt-basedlearningmulti-modalclassificationGenomeCharadesCLIPembeddings
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper sets out to show that large language models can supply the common-sense priors that video action recognizers lack, particularly in cluttered, occluded, multi-label sequences. The proposed framework first summarizes a video into candidate activities, objects, and human-object interactions, then prompts a frozen OPT-30B model to write a sentence describing the current frame and another sentence predicting the next action. These sentences are embedded with CLIP and concatenated with frame embeddings to train a small multi-modal classifier. The paper reports mean average precision of 48.19 on Action Genome and 43.94 on Charades, versus SlowFast baselines of 40.50 and 38.84, and argues that generating both current and subsequent descriptions is what produces the gain.

What carries the argument

The mechanism at the core is the Common Sense Reasoning module consisting of two prompt-based generators built on a frozen OPT-30B language model. The first, $C_D$, receives a prompt whose angle-bracket slots are filled with the top-5 predicted activity verbs $\hat{v}$, detected objects $\hat{o}_t$, and interactions $\hat{r}_t$, and outputs a description $s^d_t$ of the current scene. The second, $C_C$, takes $s^d_t$ followed by the phrase 'The person then proceeds to' and outputs a description $s^c_t$ of the next action. These two sentences are encoded by the CLIP text encoder, concatenated with the CLIP image embedding of the frame, and fed to a 512-512-N MLP that predicts per-frame activity; per-frame predictions are aggregated for video-level output. The context triples themselves come from a frozen SlowFast activity head and a transformer-based semantic-role-labeling model that predicts objects and interactions conditioned on the predicted verbs.

What would settle it

Compare the full framework against a variant where the context triple is formatted as a plain templated sentence (e.g., 'Activities: ... Objects: ... Interactions: ...') and fed through the same CLIP text encoder and classifier, bypassing the language model entirely. If the templated variant matches the current-description-only mAP of 46.77 on Action Genome, the claimed commonsense-generation contribution is not responsible for the gain; if the templated variant falls well short, the generation step is doing real work.

Watch

Extended reading notes

Core claim

The central discovery, on the paper's own terms, is that explicit natural-language descriptions generated from visual context triples — predicted activity verbs, per-frame objects, and human-object interactions — are a powerful intermediate representation for action recognition. Feeding these triples to a large language model with hand-crafted prompts yields a current-action sentence and a subsequent-action sentence; after CLIP text encoding and concatenation with CLIP image features, a three-layer MLP classifier trained on these features outperforms visual-only SlowFast by a large margin. The paper also claims that this language-driven pipeline exceeds a spatial-temporal scene-graph method (SGFB) while using a weaker ResNet-50 backbone, and that the context-summary-plus-generation design generalizes to partial training data.

Load-bearing premise

The load-bearing premise is that the measured improvement comes from the language model's common-sense reasoning rather than from the text embeddings simply re-encoding the top-5 predicted activity verbs that are already present in the prompt and share the target vocabulary; the paper does not include a control that uses the context summary without LLM generation.

Editorial extensions

If this is right

  • The reported gains imply that language-model commonsense can substitute for heavy visual backbones: the full method outperforms SGFB's ResNet-101 baseline while using ResNet-50.
  • The framework is backbone-agnostic: any top-K activity verb predictor can supply the context, so improvements should stack on future recognizers.
  • The ablation shows the current-description text features alone reach 46.77 mAP on Action Genome, close to the full 48.19, so most of the gain arrives before subsequent-action reasoning is added.
  • Because the multi-modal head needs no recurrent temporal modeling, the generated descriptions themselves carry the temporal progression.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A control that feeds the raw context triple (verbs, objects, interactions) as a plain sentence into the CLIP text encoder, skipping LLM generation, would separate the contribution of common-sense reasoning from the contribution of re-encoding the predicted activity verbs; the paper does not run this control.
  • The text-only current-description result (46.77 mAP) being so close to the full model suggests the language model may be adding little beyond rephrasing the vocabulary that is already present in the prompt, and that the classifier is largely recovering the activity verbs it was conditioned on.
  • The method's stated limitation — that generic commonsense bases fail on domain-specific activities like distinguishing 'eat' from 'checkout' with the same held fork — points to a testable extension: fine-tuning or retrieving domain-specific commonsense should further improve recognition of similar actions in different contexts.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes a three-stage framework for multi-label video action recognition. A video context summarizer (Sec. 3.1) uses a frozen SlowFast model to predict top-5 activity verbs and a transformer-based semantic-role-labeling model to predict per-frame objects and human-object interactions. A commonsense reasoning component (Sec. 3.2) feeds these predictions into prompt templates and uses OPT-30B to generate a current-scene description and a subsequent-action description. A multi-modal head (Sec. 3.3) concatenates CLIP image embeddings with CLIP text embeddings of the two descriptions and trains a three-layer MLP to classify activities. On Action Genome and Charades the authors report mAP 48.19 and 43.94, against SlowFast baselines of 40.50 and 38.84, and they provide ablations and qualitative examples.

Significance. The framework is simple and practical: it combines frozen, off-the-shelf components (SlowFast, OPT, CLIP) and reports substantial absolute mAP gains over a ResNet-50 SlowFast baseline, with a useful generalization experiment at reduced training data. The qualitative examples show that the generated descriptions are often semantically aligned with ground-truth activities. However, the key mechanistic claim—that the gains come from language-model commonsense reasoning rather than from re-encoding the intermediate activity-verb predictions—is not supported by the current experiments. The paper would be a useful contribution if the authors add leakage-control ablations and statistical evidence; as it stands, the central attribution is unverified.

major comments (3)
  1. [Sec. 3.2, Table 1; Sec. 3.3, Table 3] The experiments do not rule out a direct information path from the predicted activity verbs to the final classifier. The Table 1 prompt fills the placeholder activity list with the SlowFast predictions from Sec. 3.1; the generated sentence is then encoded by the CLIP text encoder and the classifier in Sec. 3.3 predicts from the same 157-class vocabulary. Table 3 shows text-only 'Common Sense (current)' at 46.77 mAP on Action Genome and 42.14 on Charades, above the SlowFast baselines (40.50 and 38.84) and close to the full model (48.19 and 43.94), while image-only features give only 22.14 and 21.92. This pattern is exactly what a direct readout of the top-5 verb predictions would produce. Please add at least one of the following controls: (i) encode the context triple (activity verbs, objects, interactions) directly without any LLM-generated description; (ii) remove or replace the activity verbs in the prompt while keeping objects and interactions; (iii) train the same MLP on the SlowFast top-5 verb scores or verb embeddings. Without such a control, the claimed 18.99% and 13.13% improvements cannot be attributed to commonsense reasoning.
  2. [Sec. 4.5, Table 3] The contribution of the 'subsequent' description is not demonstrated with any statistical evidence. Adding the subsequent-action description to 'Image Features + Common Sense (current)' changes mAP by only +0.26 on Action Genome and +0.49 on Charades, and the standalone 'subsequent' branch (23.57 and 23.61) is close to the image-only branch (22.14 and 21.92). No error bars, seeds, or significance tests are reported. The statement in Sec. 1 that 'generating the current and subsequent descriptions plays a key role in yielding better performance' is thus not supported by the table. Please report multiple runs and add a control in which the subsequent prompt is replaced by a non-informative or random continuation.
  3. [Sec. 4.2 and Fig. 4] The generalization experiment is underspecified. It reports 29.80% mAP on Charades when trained with 10% of the training examples, but does not state which dataset (Action Genome or Charades) is used to train each component, whether the SlowFast and SRL backbones are frozen or fine-tuned on the subset, or whether the evaluation is on the Charades test or validation split. Without this information the generalization claim cannot be reproduced or compared with the rest of the paper.
minor comments (6)
  1. [Sec. 1] The phrase 'casual inference' should be 'causal inference'.
  2. [Sec. 4.3] 'We provide more examplex in the supplementary material' contains a typo: 'examplex' should be 'examples'.
  3. [Sec. 4.5] The heading contains a typo: 'Image Featuers' should be 'Image Features'.
  4. [Sec. 5] The limitation example '<person, hold, folk>' should read '<person, hold, fork>'.
  5. [Table 2] The delta columns mix absolute mAP differences with the relative percentages quoted in the abstract ('18.99%' and '13.13%'); please clarify the labels and use one consistent reporting convention.
  6. [References] The CLIP paper appears twice, as [27] and [41]; please consolidate into a single reference.

Circularity Check

0 steps flagged · score 0.0 of 10

No by-construction circularity: the pipeline is a trained cascade with external baselines, and the common-sense attribution gap is a missing control, not an equivalence.

full rationale

The paper's derivation is an empirical cascade: SlowFast emits top-5 activity verbs (Sec. 3.1); OPT renders them together with objects and interactions into natural-language descriptions (Sec. 3.2, Table 1); and a trained MLP classifies concatenated CLIP embeddings of image, current description, and subsequent description (Sec. 3.3). There is no equation defining an output as an input, no parameter fitted to a subset and then reported as a prediction of that same subset, and no load-bearing self-citation: the references to SlowFast, OPT, CLIP, Action Genome, and Charades are all external, and the headline numbers are measured on held-out validation splits against a SlowFast baseline. The skeptical concern that the text branch can directly exploit the injected '<Verb 0>, <Verb 1>, ...' tokens, and that 'Common Sense (current)' (46.77 mAP on AG) nearly matches the full model (48.19), is a real attribution and ablation gap for the paper's mechanistic claim that common-sense reasoning, rather than verb re-encoding, drives the gain. However, that is a missing control and a correctness/experimental-design issue, not a by-construction reduction: the LLM output is an open-ended paraphrase, the text encoder is a pretrained CLIP model, and the final classifier is learned, so the improvement is not logically forced by the input tokens. Under the stated circularity criteria, this is no significant circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The framework introduces no new physical entities, particles, or forces. It is an engineering pipeline built from existing pretrained models. The main hand-chosen constants are hyperparameters of the pipeline, none of which is fitted to the target metric in a way that invalidates the architecture, but several are left without sensitivity analysis.

free parameters (5)
  • top-K activity candidates K = 5
    The SlowFast module predicts top-5 candidate activity verbs per video (Sec 4.2); this hand-chosen K controls what enters the description prompts.
  • segment length l = 5 frames
    Frames are uniformly cropped into clips of length 5 for the semantic-role labeling model (Sec 3.1).
  • MLP hidden dimension = 512
    The final activity classifier uses layers 512-512-N (Sec 3.3), chosen without reported tuning.
  • max generated token length = 260 current, 200 subsequent
    OPT-30B generation lengths are set in the supplementary implementation details.
  • OPT model size = 30B parameters
    The authors chose OPT-30B as a trade-off between quality and compute (Sec 4.2).
assumptions (5)
  • domain assumption The Action Genome training set provides reliable ground-truth object-interaction tuples and per-frame activity labels sufficient to train the context summary and classifier.
    The SRL and classifier are trained on AG annotations (Sec 4.2); if these labels are noisy, the whole pipeline inherits the noise.
  • domain assumption CLIP text embeddings of free-form LLM-generated descriptions preserve discriminative information about activity labels.
    The final classifier concatenates CLIP embeddings and is trained on activity labels (Sec 3.3); no analysis checks what information survives in the text embeddings.
  • domain assumption OPT-30B generates coherent, relevant descriptions and next-action inferences from the context triples and prompt examples.
    The common-sense stage is not evaluated separately for generation quality, so its output is assumed useful.
  • domain assumption The model components trained on Action Genome transfer to Charades without adaptation.
    Charades has no object/interaction annotations in this setup, yet the SRL and classifier are evaluated there directly (Sec 4.3).
  • ad hoc to paper The two fixed examples in each prompt template are sufficient in-context demonstrations.
    No prompt search or sensitivity analysis is reported (Table 1).

how reviews work

0 comments
Cite this review

Pith. "Pith review of Language-driven Description Generation and Common Sense Reasoning for Video Action Recognition." pith.science (2026). https://pith.science/paper/MRMXTXHT

@misc{pith2026250616701,
  author       = {Pith},
  title        = {Pith review of: Language-driven Description Generation and Common Sense Reasoning for Video Action Recognition},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/MRMXTXHT}},
  note         = {Machine review of arXiv:2506.16701}
}
read the original abstract

Recent video action recognition methods have shown excellent performance by adapting large-scale pre-trained language-image models to the video domain. However, language models contain rich common sense priors - the scene contexts that humans use to constitute an understanding of objects, human-object interactions, and activities - that have not been fully exploited. In this paper, we introduce a framework incorporating language-driven common sense priors to identify cluttered video action sequences from monocular views that are often heavily occluded. We propose: (1) A video context summary component that generates candidate objects, activities, and the interactions between objects and activities; (2) A description generation module that describes the current scene given the context and infers subsequent activities, through auxiliary prompts and common sense reasoning; (3) A multi-modal activity recognition head that combines visual and textual cues to recognize video actions. We demonstrate the effectiveness of our approach on the challenging Action Genome and Charades datasets.

Figures

Figures reproduced from arXiv: 2506.16701 by the authors.

Figure 1
Figure 1. Illustration of how Common Sense Reasoning aids video understanding. Vision systems may fail to recognize actions from cluttered and heavily occluded scenes. We propose to leverage common sense reasoning by summarizing and describing the scene context of objects, interactions, and activities, which can help suppress illogical predictions. that marries visual cues such as object and interaction detec￾tion with lingui… view at source ↗
Figure 2
Figure 2. Overview of proposed framework. The model contains three main components to understand the video: Video Context Summary S = {Sint, Sobj , Sact}, Common Sense Reasoning C = {CD, CC } and Multi-modal Activity Recognition F. Given a sequence of video frames {It} T −1 t=0 , the Context Summary component S extracts the entire video activities vˆ and per-frame human-object interactions {rˆt} T −1 t=0 and objects {oˆt} T −… view at source ↗
Figure 3
Figure 3. Overview of our Video Context Summary component [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Generalization evaluation. Performance of our model on [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Methods comparison on video W0QSB.mp4 in Action Genome dataset. Unlike the baseline method, which fails to detect complex [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Methods comparison on video C0CMQ.mp4 in Action Genome dataset. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Methods comparison on video 2B577.mp4. 260 and 200 for the current and next action description generators, respectively. 13 [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

43 extracted references · 35 canonical work pages

  1. [1]

    Action genome: Actions as compositions of spatio- temporal scene graphs

    Jingwei Ji, Ranjay Krishna, Li Fei-Fei, and Juan Carlos Niebles. Action genome: Actions as compositions of spatio- temporal scene graphs. InICCV, pages 10236–10247, 2020. 1, 2, 6

    work page 2020

  2. [2]

    OPT: Open pre-trained transformer language models.ArXiv, abs/2205.01068, 2022

    Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, Todor Mihaylov, Myle Ott, Sam Shleifer, Kurt Shuster, Daniel Simig, Punit Singh Koura, An- jali Sridhar, Tianlu Wang, and Luke Zettlemoyer. OPT: Open pre-trained transformer language models.ArXiv, abs/2205.01068, 2022. ...

    arXiv 2022

  3. [3]

    Roberta: A robustly optimized BERT pretraining approach.CoRR, abs/1907.11692, 2019

    Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettle- moyer, and Veselin Stoyanov. Roberta: A robustly optimized BERT pretraining approach.CoRR, abs/1907.11692, 2019. 1

    arXiv 1907

  4. [4]

    Chatgpt-4o, 2024

    OpenAI. Chatgpt-4o, 2024. Accessed: 2024-04-01. 1, 3

    work page 2024

  5. [5]

    Gemini 2.0 flash, 2024

    Google DeepMind. Gemini 2.0 flash, 2024. Accessed: 2025- 04-01. 1, 3

    work page 2024

  6. [6]

    Qwen2.5- vl technical report, 2025

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5- vl technical report, ...

    work page 2025

  7. [7]

    Prompting visual-language models for efficient video understanding

    Chen Ju, Tengda Han, Kunhao Zheng, Ya Zhang, and Weidi Xie. Prompting visual-language models for efficient video understanding. InEuropean Conference on Computer Vision, pages 105–124. Springer, 2022. 1, 3

    work page 2022

  8. [8]

    Llms are good action recognizers

    Haoxuan Qu, Yujun Cai, and Jun Liu. Llms are good action recognizers. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 18395–18406, June 2024. 1, 3

    work page 2024

Show all 43 references
  1. [9]

    Video-chatgpt: Towards detailed video understanding via large vision and language models

    Muhammad Maaz, Hanoona Rasheed, Salman Khan, and Fa- had Shahbaz Khan. Video-chatgpt: Towards detailed video understanding via large vision and language models. InPro- ceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL 2024), 2024. 1, 3

    2024

  2. [10]

    M-llm based video frame selec- tion for efficient video understanding

    Kai Hu, Feng Gao, Xiaohan Nie, Peng Zhou, Son Tran, Tal Neiman, Lingyun Wang, Mubarak Shah, Raffay Hamid, Bing Yin, and Trishul Chilimbi. M-llm based video frame selec- tion for efficient video understanding. InProceedings of the IEEE/CVF Conference on Computer Vision and Patt...

    2025

  3. [11]

    Hierarq: Task-aware hierarchical q-former for enhanced video understanding.arXiv preprint arXiv:2503.08585, 2025

    Shehreen Azad, Vibhav Vineet, and Yogesh Singh Rawat. Hierarq: Task-aware hierarchical q-former for enhanced video understanding.arXiv preprint arXiv:2503.08585, 2025. 1, 3

    2025 arXiv

  4. [12]

    Two-stream con- volutional networks for action recognition in videos

    Karen Simonyan and Andrew Zisserman. Two-stream con- volutional networks for action recognition in videos. InCon- ference on Neural Information Processing Systems (Neurips),

  5. [13]

    Temporal segment networks for action recognition in videos.IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(11):2740– 2755, 2019

    Limin Wang, Yuanjun Xiong, Zhe Wang, Yu Qiao, Dahua Lin, Xiaoou Tang, and Luc Van Gool. Temporal segment networks for action recognition in videos.IEEE Transactions on Pattern Analysis and Machine Intelligence, 41(11):2740– 2755, 2019. 2

    2019

  6. [14]

    Learning spatiotemporal features with 3d convolutional networks

    Du Tran, Lubomir Bourdev, Rob Fergus, Lorenzo Torresani, and Manohar Paluri. Learning spatiotemporal features with 3d convolutional networks. InIn ICCV, 2015. 2

    2015

  7. [15]

    Quo vadis, action recognition? A new model and the kinetics dataset

    João Carreira and Andrew Zisserman. Quo vadis, action recognition? A new model and the kinetics dataset. In2017 IEEE Conference on Computer Vision and Pattern Recogni- tion, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, 2017. 3

    2017

  8. [16]

    Smeulders

    Noureldien Hussein, Efstratios Gavves, and Arnold W.M. Smeulders. Timeception for complex action recognition. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition (CVPR), June 2019. 3

    2019

  9. [17]

    Slowfast networks for video recognition

    Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In ICCV, 2019. 3, 6, 7, 8, 13

    2019

  10. [18]

    Michael S. Ryoo, A. J. Piergiovanni, Juhana Kangaspunta, and Anelia Angelova. Assemblenet++: Assembling modality representations via attention connections. InComputer Vision – ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XX, page 654–671,

    2020

  11. [19]

    Tokenlearner: Adaptive space- time tokenization for videos

    Michael Ryoo, AJ Piergiovanni, Anurag Arnab, Mostafa De- hghani, and Anelia Angelova. Tokenlearner: Adaptive space- time tokenization for videos. In M. Ranzato, A. Beygelzimer, Y . Dauphin, P.S. Liang, and J. Wortman Vaughan, editors,Ad- vances in Neural Information Processing...

    2021

  12. [20]

    Conceptnet 5.5: An open multilingual graph of general knowledge.CoRR, abs/1612.03975, 2016

    Robyn Speer, Joshua Chin, and Catherine Havasi. Conceptnet 5.5: An open multilingual graph of general knowledge.CoRR, abs/1612.03975, 2016. 3

    2016 arXiv

  13. [21]

    Smith, and Yejin Choi

    Maarten Sap, Ronan LeBras, Emily Allaway, Chandra Bhagavatula, Nicholas Lourie, Hannah Rashkin, Brendan Roof, Noah A. Smith, and Yejin Choi. ATOMIC: an at- las of machine commonsense for if-then reasoning.CoRR, abs/1811.00146, 2018. 3

    2018 arXiv

  14. [22]

    We- bChild 2.0 : Fine-grained commonsense knowledge distilla- tion

    Niket Tandon, Gerard de Melo, and Gerhard Weikum. We- bChild 2.0 : Fine-grained commonsense knowledge distilla- tion. InProceedings of ACL 2017, System Demonstrations, pages 115–120, Vancouver, Canada, July 2017. Association for Computational Linguistics. 3

    2017

  15. [23]

    COMET: Com- monsense transformers for automatic knowledge graph con- struction

    Antoine Bosselut, Hannah Rashkin, Maarten Sap, Chaitanya Malaviya, Asli Celikyilmaz, and Yejin Choi. COMET: Com- monsense transformers for automatic knowledge graph con- struction. InProceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages...

    2019

  16. [24]

    Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi

    Peter West, Chandrasekhar Bhagavatula, Jack Hessel, Jena D. Hwang, Liwei Jiang, Ronan Le Bras, Ximing Lu, Sean Welleck, and Yejin Choi. Symbolic knowledge distillation: from general language models to commonsense models. In NAACL, 2022. 3 9

    2022

  17. [25]

    Conditional prompt learning for vision-language models

    Kaiyang Zhou, Jingkang Yang, Chen Change Loy, and Ziwei Liu. Conditional prompt learning for vision-language models. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022. 3

    2022

  18. [26]

    Prompt distribution learning

    Yuning Lu, Jianzhuang Liu, Yonggang Zhang, Yajing Liu, and Xinmei Tian. Prompt distribution learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 5206–5215, June 2022. 3

    2022

  19. [27]

    Learning transferable visual models from natural language supervision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. InICML, 2021. 3

    2021

  20. [28]

    Denseclip: Language-guided dense prediction with context- aware prompting

    Yongming Rao, Wenliang Zhao, Guangyi Chen, Yansong Tang, Zheng Zhu, Guan Huang, Jie Zhou, and Jiwen Lu. Denseclip: Language-guided dense prediction with context- aware prompting. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition (CVPR), page...

    2022

  21. [29]

    Expanding language-image pretrained models for general video recognition

    Bolin Ni, Houwen Peng, Minghao Chen, Songyang Zhang, Gaofeng Meng, Jianlong Fu, Shiming Xiang, and Haibin Ling. Expanding language-image pretrained models for general video recognition. InEuropean Conference on Computer Vision (ECCV), 2022. 3

    2022

  22. [30]

    Learning to prompt for continual learning

    Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jen- nifer Dy, and Tomas Pfister. Learning to prompt for continual learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1...

    2022

  23. [31]

    Visual prompt tuning

    Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. InEuropean Conference on Computer Vision (ECCV), 2022. 3

    2022

  24. [32]

    Videobert: A joint model for video and language representation learning

    Chen Sun, Austin Myers, Carl V ondrick, Kevin Murphy, and Cordelia Schmid. Videobert: A joint model for video and language representation learning. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), October 2019. 3

    2019

  25. [33]

    Language models are few- shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Sub- biah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakan- tan, Pranav Shyam, Girish Sastry, Amanda Askell, Sand- hini Agarwal, Ariel Herbert-V oss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Je...

    1901

  26. [34]

    Le, and Christo- pher D

    Kevin Clark, Minh-Thang Luong, Quoc V . Le, and Christo- pher D. Manning. Electra: Pre-training text encoders as discriminators rather than generators. InInternational Con- ference on Learning Representations, 2020. 3

    2020

  27. [35]

    Multimodal few-shot learn- ing with frozen language models.Proc

    Maria Tsimpoukelli, Jacob Menick, Serkan Cabi, SM Eslami, Oriol Vinyals, and Felix Hill. Multimodal few-shot learn- ing with frozen language models.Proc. Neural Information Processing Systems, 2021. 3

    2021

  28. [36]

    UNIFIEDQA: Crossing format boundaries with a single QA system

    Daniel Khashabi, Sewon Min, Tushar Khot, Ashish Sabhar- wal, Oyvind Tafjord, Peter Clark, and Hannaneh Hajishirzi. UNIFIEDQA: Crossing format boundaries with a single QA system. InFindings of the Association for Computational Lin- guistics: EMNLP 2020, Online, November 2020. A...

    2020

  29. [37]

    Few-shot text generation with natural language instructions

    Timo Schick and Hinrich Schütze. Few-shot text generation with natural language instructions. InEMNLP, 2021. 3

    2021

  30. [38]

    Generating action-conditioned prompts for open-vocabulary video action recognition

    Chengyou Jia, Minnan Luo, Xiaojun Chang, Zhuohang Dang, Mingfei Han, Mengmeng Wang, Guang Dai, Sizhe Dang, and Jingdong Wang. Generating action-conditioned prompts for open-vocabulary video action recognition. InProceedings of the 32nd ACM International Conference on Multimedi...

    2024

  31. [39]

    Kronecker mask and interpretive prompts are language-action video learners

    Jingyi Yang, Zitong Yu, Xiuming Ni, Jia He, and Hui Li. Kronecker mask and interpretive prompts are language-action video learners. InInternational Conference on Learning Representations, 2025. 3

    2025

  32. [40]

    Visual semantic role labeling for video understanding

    Arka Sadhu, Tanmay Gupta, Mark Yatskar, Ram Nevatia, and Aniruddha Kembhavi. Visual semantic role labeling for video understanding. InCVPR, 2021. 4

    2021

  33. [41]

    Learning transferable vi- sual models from natural language supervision.ArXiv, abs/2103.00020, 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable vi- sual models from natural language supervision.ArXiv, abs/2103.000...

    2021 arXiv

  34. [42]

    Flava: A foundational language and vision alignment model

    Amanpreet Singh, Ronghang Hu, Vedanuj Goswami, Guil- laume Couairon, Wojciech Galuba, Marcus Rohrbach, and Douwe Kiela. Flava: A foundational language and vision alignment model. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, pages 15638...

    2022

  35. [43]

    close the door

    Gunnar A. Sigurdsson, Gül Varol, X. Wang, Ali Farhadi, Ivan Laptev, and Abhinav Kumar Gupta. Hollywood in homes: Crowdsourcing data collection for activity understanding. ArXiv, abs/1604.01753, 2016. 6 10 Supplementary Material for Language Models with Commonsense Reasoning fo...

    2016 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.