Pith Number

pith:XCXLFWSA

pith:2023:XCXLFWSANHUSBZ2RU7WQZVMB2J

not attested not anchored not stored refs resolved

VideoPoet: A Large Language Model for Zero-Shot Video Generation

Agrim Gupta, Alonso Martinez, Anja Hauth, Bryan Seybold, Dan Kondratyuk, David A. Ross, David Hendon, David Minnen, Grant Schindler, Hartwig Adam, Hassan Akbari, Huisheng Wang, Irfan Essa, Jimmy Yan, Jonathan Huang, Jos\'e Lezama, Josh Dillon, Kihyuk Sohn, Krishna Somandepalli, Lijun Yu, Lu Jiang, Meera Hahn, Mikhail Sirotenko, Ming-Chang Chiu, Ming-Hsuan Yang, Rachel Hornung, Vighnesh Birodkar, Xiuye Gu, Xuan Yang, Yair Alon, Yong Cheng

A decoder-only transformer trained on multimodal data generates high-fidelity videos zero-shot from text, images, and audio.

arxiv:2312.14125 v4 · 2023-12-21 · cs.CV · cs.AI

Open paper page JSON Open Graph Bundle Merged state What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{XCXLFWSANHUSBZ2RU7WQZVMB2J}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

VideoPoet employs a decoder-only transformer architecture that processes multimodal inputs -- including images, videos, text, and audio -- and demonstrates state-of-the-art capabilities in zero-shot video generation, specifically highlighting VideoPoet's ability to generate high-fidelity motions.

C2weakest assumption

The assumption that a mixture of multimodal generative objectives trained autoregressively on a decoder-only transformer will transfer effectively to zero-shot video generation without requiring substantial additional video-specific inductive biases or post-training.

C3one line summary

VideoPoet is a large language model that performs zero-shot video generation with audio from diverse multimodal conditioning signals.

References

55 extracted · 55 resolved · 29 Pith anchors

[1] MusicLM: Generating Music From Text · arXiv:2301.11325

[2] Alternating gradient descent and mixture-of- experts for integrated multimodal perception

[3] PaLM 2 Technical Report · arXiv:2305.10403

[4] Lumiere: A space-time diffusion model for video generation

[5] Stable Video Diffusion: Scaling Latent Video Diffusion Models to Large Datasets · arXiv:2311.15127

Cited by

29 papers in Pith

PanoWorld: Geometry-Consistent Panoramic Video World Modeling

Frozen Forecasting: A Unified Evaluation

FluentAvatar: Flicker-Free Talking-Head Animation via Phoneme-Guided Autoregressive Modeling

Towards Redundancy Reduction in Diffusion Models for Efficient Video Super-Resolution

Autoregressive Video Generation without Vector Quantization

Receipt and verification

First computed	2026-05-17T23:38:50.672005Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

b8aeb2da4069e920e751a7ed0cd581d277d482abcbec9b58570126d1319f18bf

Aliases

arxiv: 2312.14125 · arxiv_version: 2312.14125v4 · doi: 10.48550/arxiv.2312.14125 · pith_short_12: XCXLFWSANHUS · pith_short_16: XCXLFWSANHUSBZ2R · pith_short_8: XCXLFWSA

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/XCXLFWSANHUSBZ2RU7WQZVMB2J \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: b8aeb2da4069e920e751a7ed0cd581d277d482abcbec9b58570126d1319f18bf

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "544d649ec5a1c6c28686e7b7a0beb204295ee44d4bde2e405b1197873c44127f",
    "cross_cats_sorted": [
      "cs.AI"
    ],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.CV",
    "submitted_at": "2023-12-21T18:46:41Z",
    "title_canon_sha256": "18b0d90fb9392b704e8d07576d72bd1dfbee6398ddf50fe6abad23281ad20228"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2312.14125",
    "kind": "arxiv",
    "version": 4
  }
}