Pith. sign in

REVIEW 8 cited by

VLM: Task-agnostic Video-Language Model Pre-training for Video Understanding

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2105.09996 v3 pith:6IBFJNAX submitted 2021-05-20 cs.CV cs.CL

Hu Xu , Gargi Ghosh , Po-Yao Huang , Prahal Arora , Masoumeh Aminzadeh , Christoph Feichtenhofer , Florian Metze , Luke Zettlemoyer This is my paper
classification cs.CVcs.CL
keywords pre-trainingtasksvideoacrosscross-modaleitherinputlimiting
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We present a simplified, task-agnostic multi-modal pre-training approach that can accept either video or text input, or both for a variety of end tasks. Existing pre-training are task-specific by adopting either a single cross-modal encoder that requires both modalities, limiting their use for retrieval-style end tasks or more complex multitask learning with two unimodal encoders, limiting early cross-modal fusion. We instead introduce new pretraining masking schemes that better mix across modalities (e.g. by forcing masks for text to predict the closest video embeddings) while also maintaining separability (e.g. unimodal predictions are sometimes required, without using all the input). Experimental results show strong performance across a wider range of tasks than any previous methods, often outperforming task-specific pre-training. Code is made available at https://github.com/pytorch/fairseq/tree/main/examples/MMPT.

Discussion (0). Sign in to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Dual-Anchoring: Addressing State Drift in Vision-Language Navigation

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    Dual-Anchoring Framework mitigates progress drift via structured instruction tokens and memory drift via landmark-centric retrospective prediction, yielding 15.2% success rate gain and 24.7% on long trajectories.

  2. Flamingo: a Visual Language Model for Few-Shot Learning

    cs.CV 2022-04 unverdicted novelty 7.0 of 10

    Flamingo models reach new state-of-the-art few-shot results on image and video tasks by bridging frozen vision and language models with cross-attention layers trained on interleaved web-scale data.

  3. T-MOR: Learning Motion-Aware Skeleton Representations for Human Action Recognition

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    T-MOR is a multi-modal contrastive framework that pre-trains transferable skeleton motion representations using a new 1M video-skeleton-text dataset and shows gains on action classification and temporal detection benc...

  4. Dual-Anchoring: Addressing State Drift in Vision-Language Navigation

    cs.CV 2026-04 conditional novelty 6.0 of 10

    Privatar partitions VR avatar reconstruction via frequency-domain decomposition, keeping sensitive components local and offloading the rest with distribution-aware minimal perturbation noise, achieving 2.37x throughpu...

  5. AutoBridge: Automating Smart Device Integration with Centralized Platform

    cs.SE 2025-07 conditional novelty 6.0 of 10

    AutoBridge generates platform-compliant IoT integration code with 93.87% success and 94.87% function coverage in fully automatic mode, reaching 100% coverage with minimal yes/no human feedback.

  6. Dual-Anchoring: Addressing State Drift in Vision-Language Navigation

    cs.CV 2026-04 unverdicted novelty 5.0 of 10

    Dual-Anchoring adds explicit progress tokens and retrospective landmark verification to VLN agents, cutting state drift and lifting success rate 15.2% overall with 24.7% gains on long trajectories.

  7. What Matters in Building Vision-Language-Action Models for Generalist Robots

    cs.RO 2024-12 unverdicted novelty 5.0 of 10

    Systematic tests of VLM backbones, policy architectures, and cross-embodiment data yield RoboVLMs that set new SOTA on robot manipulation benchmarks while requiring few manual designs.

  8. A Modular Vision-Language-Action Robotics Framework for Indoor Environments

    cs.RO 2026-06 unverdicted novelty 3.0 of 10

    Describes a modular VLA framework with semantic voxel mapping via OwlViT and VLM-based command classification and grounding for the CMU VLA Challenge.

Pith tools