{"paper":{"title":"TrajTok: Learning Trajectory Tokens enables better Video Understanding","license":"http://creativecommons.org/licenses/by/4.0/","headline":"TrajTok learns trajectory tokens end-to-end through implicit space-time clustering to improve video model accuracy and efficiency.","cross_cats":[],"primary_cat":"cs.CV","authors_text":"Ashutosh Kumar, Chenhao Zheng, Chun-Liang Li, Jianing Zhang, Jieyu Zhang, Oncel Tuzel, Quan Kong, Ranjay Krishna, Weikai Huang","submitted_at":"2026-02-26T09:15:34Z","abstract_excerpt":"Tokenization in video models, typically through patchification, generates an excessive and redundant number of tokens. This severely limits video efficiency and scalability. While recent trajectory-based tokenizers offer a promising solution by decoupling video duration from token count, they rely on complex external segmentation and tracking pipelines that are slow and task-agnostic. We propose TrajTok, an end-to-end video tokenizer module that is fully integrated and co-trained with video models for a downstream objective, dynamically adapting its token granularity to semantic complexity, in"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"With TrajTok, we implement a video CLIP model trained from scratch (TrajViT2). It achieves the best accuracy at scale across both classification and retrieval benchmarks, while maintaining efficiency comparable to the best token-merging methods.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That implicit clustering of pixels in space and time will produce trajectories that are semantically useful for downstream video understanding tasks when the segmenter is co-trained only for adaptability rather than pixel-level fidelity.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"TrajTok learns adaptive trajectory tokens for videos through a unified end-to-end segmenter, improving understanding performance and efficiency over patch-based or external-pipeline tokenizers.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"TrajTok learns trajectory tokens end-to-end through implicit space-time clustering to improve video model accuracy and efficiency.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"fb3d03326b4bab152b6eb6a052f46efbb6b21779496465b5da1faf5b21816447"},"source":{"id":"2602.22779","kind":"arxiv","version":3},"verdict":{"id":"36578a04-2a7a-4bf2-a3dc-589019cfb825","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-15T19:09:34.106646Z","strongest_claim":"With TrajTok, we implement a video CLIP model trained from scratch (TrajViT2). It achieves the best accuracy at scale across both classification and retrieval benchmarks, while maintaining efficiency comparable to the best token-merging methods.","one_line_summary":"TrajTok learns adaptive trajectory tokens for videos through a unified end-to-end segmenter, improving understanding performance and efficiency over patch-based or external-pipeline tokenizers.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That implicit clustering of pixels in space and time will produce trajectories that are semantically useful for downstream video understanding tasks when the segmenter is co-trained only for adaptability rather than pixel-level fidelity.","pith_extraction_headline":"TrajTok learns trajectory tokens end-to-end through implicit space-time clustering to improve video model accuracy and efficiency."},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2602.22779/integrity.json","findings":[],"available":true,"detectors_run":[],"snapshot_sha256":"c28c3603d3b5d939e8dc4c7e95fa8dfce3d595e45f758748cecf8e644a296938"},"references":{"count":0,"sample":[],"resolved_work":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57","internal_anchors":0},"formal_canon":{"evidence_count":2,"snapshot_sha256":"4ce750f8c498ee634682461a650164d2639b67f339e7b1fc3a5b36892539e714"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}