Pith Number

pith:TOPMBZWI

pith:2024:TOPMBZWI6RZ5XX2DT5CY6BIJNU

not attested not anchored not stored refs pending

Latent Action Pretraining from Videos

Ajay Mandlekar, Baolin Peng, Bill Yuchen Lin, Byeongguk Jeon, Dieter Fox, Jianfeng Gao, Jianwei Yang, Joel Jang, Kimin Lee, Lars Liden, Luke Zettlemoyer, Minjoon Seo, Reuben Tan, Sejune Joo, Seonghyeon Ye, Yu-Wei Chao

LAPA learns discrete latent actions from unlabeled videos with VQ-VAE, pretrains a VLA model to predict them, and finetunes on small robot datasets to outperform both video-only baselines and labeled SOTA VLA models on language-conditioned manipulation tasks.

arxiv:2410.11758 v2 · 2024-10-15 · cs.RO · cs.CL · cs.CV · cs.LG

Open paper page JSON Open Graph Bundle Merged state Verified badge What is a Pith Number?

Add to your LaTeX paper

\usepackage{pith}
\pithnumber{TOPMBZWI6RZ5XX2DT5CY6BIJNU}

Prints a linked badge after your title and injects PDF metadata. Compiles on arXiv. Learn more · Embed verified badge

Record completeness

1 Bitcoin timestamp

2 Internet Archive

3 Author claim open · sign in to claim

4 Citations open

5 Replications open

✓ Portable graph bundle live · download bundle · merged state

The bundle contains the canonical record plus signed events. A mirror can host it anywhere and recompute the same current state with the deterministic merge algorithm.

Claims

C1strongest claim

Experimental results demonstrate that our method significantly outperforms existing techniques that train robot manipulation policies from large-scale videos. Furthermore, it outperforms the state-of-the-art VLA model trained with robotic action labels on real-world manipulation tasks that require language conditioning, generalization to unseen objects, and semantic generalization to unseen instructions.

C2weakest assumption

That discrete latent actions extracted from human manipulation videos contain sufficient transferable information to map effectively to robot actions during finetuning and yield better generalization than direct supervised training on labeled robot data.

C3one line summary

Formal links

2 machine-checked theorem links

Cited by

38 papers in Pith

DexWild: Dexterous Human Interactions for In-the-Wild Robot Policies

What Drives Success in Physical Planning with Joint-Embedding Predictive World Models?

EgoVLA: Learning Vision-Language-Action Models from Egocentric Human Videos

DiLA: Disentangled Latent Action World Models

Robotic Manipulation by Imitating Generated Videos Without Physical Demonstrations

Receipt and verification

First computed	2026-05-18T02:36:37.771191Z
Builder	pith-number-builder-2026-05-17-v1
Signature	Pith Ed25519 (`pith-v1-2026-05`) · public key
Schema	pith-number/v1.0

Canonical hash

9b9ec0e6c8f473dbdf439f458f05096d27563615659ffcb6b3014e1b29554d09

Aliases

arxiv: 2410.11758 · arxiv_version: 2410.11758v2 · doi: 10.48550/arxiv.2410.11758 · pith_short_12: TOPMBZWI6RZ5 · pith_short_16: TOPMBZWI6RZ5XX2D · pith_short_8: TOPMBZWI

Agent API

Resolver JSON Graph JSON Events JSON Schema Signing key

Verify this Pith Number yourself

curl -sH 'Accept: application/ld+json' https://pith.science/pith/TOPMBZWI6RZ5XX2DT5CY6BIJNU \
  | jq -c '.canonical_record' \
  | python3 -c "import sys,json,hashlib; b=json.dumps(json.loads(sys.stdin.read()), sort_keys=True, separators=(',',':'), ensure_ascii=False).encode(); print(hashlib.sha256(b).hexdigest())"
# expect: 9b9ec0e6c8f473dbdf439f458f05096d27563615659ffcb6b3014e1b29554d09

Canonical record JSON

{
  "metadata": {
    "abstract_canon_sha256": "c6fad7a3cbbc7d753ab0f1d39139b2cafc029ca7da297eb88a9da486880d7cbb",
    "cross_cats_sorted": [
      "cs.CL",
      "cs.CV",
      "cs.LG"
    ],
    "license": "http://creativecommons.org/licenses/by/4.0/",
    "primary_cat": "cs.RO",
    "submitted_at": "2024-10-15T16:28:09Z",
    "title_canon_sha256": "697b959e671bb5612b90d4d518b6f82f95585f0178d02046d14fbe7347b291a6"
  },
  "schema_version": "1.0",
  "source": {
    "id": "2410.11758",
    "kind": "arxiv",
    "version": 2
  }
}