pith. sign in

arxiv: 1811.00347 · v2 · pith:BDWI23EDnew · submitted 2018-11-01 · 💻 cs.CL

How2: A Large-scale Dataset for Multimodal Language Understanding

Ramon Sanabria , Ozan Caglayan , Shruti Palaskar , Desmond Elliott , Lo\"ic Barrault , Lucia Specia , Florian Metze This is my paper
classification 💻 cs.CL
keywords languagemultimodalhow2translationunderstandingautomaticavailablebaselines
0
0 comments X
read the original abstract

In this paper, we introduce How2, a multimodal collection of instructional videos with English subtitles and crowdsourced Portuguese translations. We also present integrated sequence-to-sequence baselines for machine translation, automatic speech recognition, spoken language translation, and multimodal summarization. By making available data and code for several multimodal natural language tasks, we hope to stimulate more research on these and similar challenges, to obtain a deeper understanding of multimodality in language processing.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. READ: Recurrent Adapter with Partial Video-Language Alignment for Parameter-Efficient Transfer Learning in Low-Resource Video-Language Modeling

    cs.CV 2023-12 unverdicted novelty 6.0

    READ recurrent adapters with partial video-language alignment via optimal transport outperform standard fine-tuning on low-resource temporal grounding and summarization tasks.

  2. InternVid: A Large-scale Video-Text Dataset for Multimodal Understanding and Generation

    cs.CV 2023-07 unverdicted novelty 6.0

    InternVid supplies 7M videos and LLM captions to train ViCLIP, which reaches leading zero-shot action recognition and competitive retrieval performance.

  3. Video-guided Machine Translation with Global Video Context

    cs.CV 2026-04 unverdicted novelty 4.0

    A globally video-guided multimodal translation framework retrieves semantically related video segments with a vector database and applies attention mechanisms to improve subtitle translation accuracy in long videos.

  4. Music Audio-Visual Question Answering Requires Specialized Multimodal Designs

    cs.SD 2025-05 unverdicted novelty 3.0

    Survey of Music AVQA finds specialized input processing, dedicated spatial-temporal designs, and music-specific modeling are critical for strong performance.

  5. Video-Language Understanding: A Survey from Model Architecture, Model Training, and Data Perspectives

    cs.CL 2024-06 unverdicted novelty 2.0

    Survey summarizing video-language understanding tasks, challenges, and methods from architecture, training, and data perspectives, including performance comparisons and future directions.