Pith. sign in

REVIEW 3 major objections 6 minor 61 references

Sekai2: From World Exploration to Interactive World Modeling

T0 review · 3 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Sekai2 supplies world models with what real-world corpora have lacked: 2,826 hours of video where every clip has a camera trajectory, timed captions, and loops that revisit the same places.

desk verdict A serious, unusually well-documented dataset paper whose headline claim is undercut by the public release: the 982-panorama revisit-rich subset is mostly not downloadable, so the central supervision is currently out of reach. read the letter →

arxiv 2608.09449 v2 pith:WYKGCXET submitted 2026-08-10 cs.CV

Kang He , Wenshuo Peng , Zihui Gao , Jiaming Tan , Kaipeng Zhang , Yongtao Ge This is my paper · ORCID
classification cs.CV
keywords videodatasetinteractiveworldmodelscameratrajectoriestemporallygroundedcaptionslong-horizongenerationpanoramicloopclosuresrevisitconsistency
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that training interactive world models—generative systems that keep a scene's state while the camera moves—requires video data offering long-horizon continuity, camera poses, and time-localized semantics all at once, and that existing corpora supply these only in fragments. To close that gap, it builds and releases Sekai2: 128,892 clips totaling 2,826 hours from 10,428 source videos across 113 countries, with every clip paired with a camera trajectory and hierarchical annotations yielding 649,597 temporally grounded segments. The added element is a 982-sequence panoramic subset captured along non-linear routes with loops and revisits, providing repeated observations of the same locations across time and viewpoint—supervision the authors argue is needed for persistent scene representations and long-term spatial memory. A sympathetic reader would take the central claim to be that the joint provision of these four properties in one real-world corpus, rather than any single dimension of scale, is what makes Sekai2 a foundation for long-horizon video generation, camera-controllable synthesis, and world-model pre-training.

What carries the argument

The load-bearing mechanism is the unified multi-source data engine, but the object that carries the paper's most distinctive claim is the revisit-verification procedure for panoramic trajectories. A revisit candidate is proposed by ORB feature matching between temporally distant keyframes; the matched features are lifted to unit viewing rays on the sphere, a rotation-only alignment is fit by singular value decomposition, and the candidate is accepted only when the median angular residual is low—the logic being that a genuine co-location is explained by a single rotation, whereas a changed camera center produces parallax no rotation can remove. Accepted closures enter a pose-graph optimization as camera-center constraints with orientation left free, correcting accumulated drift over roughly 13,000-frame sequences. The complementary machinery is the hierarchical annotation schema: a single vision-language-model call produces clip-level and segment-level JSON with six shared text fields, a sixteen-value camera_path vocabulary, and full/short prompt pairs, so camera control and scene semantics can be separated at training time. A cross-modal audit—comparing pose-derived yaw change with the caption's camera_path label over 620,487 aligned segments—is the paper's end-to-end check that the two independently produced modalities are grounded in the same events.

What would settle it

Take a random sample of accepted loop-closure pairs and fit a full essential matrix (rotation plus translation) to their matched spherical rays instead of a rotation-only alignment. A genuine co-location from the same camera center should show negligible translation; if a substantial fraction of accepted closures show significant parallax—estimated camera-center separation above a few percent of scene depth—the revisit labels are partly spurious and the pose-graph constraints would bias the trajectories. A second ground-truth check: for sequences designed as start-to-end loops, compare the refined start and end camera centers; if endpoint displacement does not drop after loop-closure refinement relative to the original ViPE estimate, the verification is not doing the claimed work.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a real-world video corpus can be engineered so that visual content, camera motion, and localized semantic change are aligned on a shared timeline at corpus scale. Every one of the 128,892 released clips is accompanied by a ViPE-estimated camera-to-world trajectory and intrinsics, and by two levels of vision-language-model captions: clip-level global attributes and segment-level local descriptions that tile each video and carry a discrete camera_path label. The annotation schema factorizes each description into subject motion, environment motion, static scene content, and camera behavior, with camera clauses enclosed in camera tags so they can be dropped, replaced, or conditioned on independently. The paper further claims that the 982 panoramic sequences, refined by geometrically verified loop closures embedded in a pose graph, deliver genuine revisits—repeated observations of the same locations from different directions and at different times—that prior real-world corpora lack. Corpus-wide evaluations are offered as evidence: all released trajectories pass validity and smoothness checks, epipolar error against SIFT correspondences is sub-pixel with a mismatched-pose control degrading sharply, cross-run DROID reconstructions agree to roughly 0.1 degrees of rotation at one second, caption scores sit near 4.3 on a 1–5 judge scale, and pose-derived yaw increases monotonically across the stationary–straight–mixed–turn camera labels.

Load-bearing premise

The panoramic revisit claim rests on a single assumption: that two panoramas aligning by rotation with a tiny residual really means the camera returned to the same spot—yet visually repetitive scenes such as malls and corridors, or two spots very close together, can pass the same test without being one location.

Editorial extensions

If this is right

  • Camera-conditioned video generation can now train on real-world trajectories: every released clip ships poses and intrinsics convertible to ray or Plücker embeddings, so camera control need not be learned only from synthetic or short-range reconstruction sources.
  • Long-horizon training data is no longer confined to short clips: 43,594 analysis segments reach the full 120-second cap and 92.3% of footage lies in segments of at least one minute, giving world models sustained observation for learning scene persistence.
  • Revisit supervision becomes available for real-world scenes: the 982 panoramic sequences with loops and repeated passages offer repeated observations of the same locations, the scarce signal identified by prior work as needed for long-range spatial memory and loop-closure behavior.
  • Semantic and geometric supervision can be separated or combined at will: the camera-tagged full prompts and the camera-free short prompts support three training modes from one caption corpus, and discrete camera_path labels align caption segments with measured yaw changes at an AUC of 0.709.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A benchmark opportunity the paper does not pursue: the monotonic rise in measured yaw across the caption labels (stationary, straight, mixed, turn) could be inverted into a ground-truth-free consistency score for generated videos, rewarding models whose implied camera motion agrees with their own captions.
  • The factorization into subject motion, environment motion, static scene, and camera behavior sets up a disentanglement experiment the paper does not run: hold scene content fixed while varying only the camera-conditioning input, and test whether a model separates viewpoint change from world change.
  • Because data-distribution agreements limit the public panoramic release to roughly 20 of the 119 processed hours, the corpus's most distinctive supervision is only partially downloadable; the reported statistics describe the full processed set, so downstream users should verify which portion they actually receive.
  • If the revisit supervision proves dense enough, it could shift long-range memory research away from game and simulated environments toward real-world trajectories, since Sekai2 couples repeated observations with captured surroundings—but whether that supervision transfers into training gains remains untested by this paper.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper presents Sekai2, a multi-source real-world video dataset aimed at interactive world modeling. It combines inherited Sekai footage, newly crawled YouTube videos, and self-captured panoramic videos; applies a manifest-driven filtering and quality-control pipeline; estimates camera trajectories with ViPE and DROID-SLAM; generates hierarchical global and segment-level captions with a VLM; and reports corpus-scale evaluations of visual quality, trajectory geometry, caption quality, and cross-modal grounding. The headline contribution is the joint provision of long-horizon continuity, camera-pose supervision, temporally grounded semantics, and a revisit-rich panoramic subset with loop-closure-refined trajectories.

Significance. If fully released as described, Sekai2 would be a valuable resource for long-horizon video generation, camera-controllable synthesis, and world-model pretraining. The paper has real strengths: the manifest-driven pipeline is documented in unusual detail with stage-wise retention statistics; the epipolar validation includes a mismatched-pose negative control that genuinely isolates pose-frame correspondence; the cross-run DROID consistency study is carefully matched in intrinsics and motion regime; and the cross-modal grounding analysis uses within-clip controls. These make the perspective-video portion of the corpus credible and reproducible. At the same time, the paper's most distinctive component—the 982-sequence, 119-hour panoramic revisit subset—is mostly not publicly available, and its trajectories receive only qualitative geometric validation. These gaps substantially temper the significance as claimed.

major comments (3)
  1. [§6 (Panoramic data release) vs Abstract/Table 1/§4.2] The abstract and Table 1 state that the release contains 982 panoramic sequences totaling 119 hours with loops and revisits, and §4.2 reports the panoramic subset as 119 hours. However, the 'Panoramic data release' paragraph in §6 states that only approximately 20 hours of this material can be included in the current public release, with all reported panoramic statistics based on the full 119 hours. The publicly downloadable corpus is therefore about 2,727 hours, not 2,826 hours, and most of the distinctive revisit-rich trajectories, their poses, and their captions are not accessible to users. This overstates the released artifact and prevents independent verification of the paper's most novel component. The paper should distinguish the processed internal corpus from the publicly downloadable release, and should restate the headline numbers, Table 1 entries, and all panoramic statistics for the actually released subset.
  2. [Table 3 / §5.2 / App B.4] The quantitative geometric validation of trajectories excludes the panoramic subset. Table 3 states that panoramic sequences are excluded because the reconstruction assumes pinhole imagery, and the epipolar test in §5.2 samples only the perspective subsets; for the panoramic sequences the only quantitative evidence is numerical validity/smoothness (Table 2) and internal loop statistics, supplemented by qualitative full-accumulation figures (App E.4). This is a load-bearing gap because the loop-closure acceptance criterion in App B.4 is a low median angular residual under a rotation-only model, which can also be satisfied by near-coincident viewpoints or by repeated similar geometry in malls and corridors. The paper should add a quantitative validation of the loop-refined panoramic trajectories, for example against manually verified revisit events or a synthetic baseline, and should report the false-positive behavior of the rotation-only verification.
  3. [§5.3 / Table 4] The caption-quality evaluation is partly self-referential. In the pairwise comparison, the structured annotations and the one-shot comparator are generated by the same VLM from identical frames, and the judge model is not named; if the judge is the same model family used to produce the annotations, the 60.9% preference and 68.4% informativeness results may reflect the judge's familiarity with its own output style rather than an independent quality advantage. Holding the visual input fixed does not remove this potential bias. The authors should disclose the judge model and report either a cross-model judge or a human-sample evaluation, and should present the current results as an internal consistency check.
minor comments (6)
  1. [§5.2] The text contains stray commas in '0.12,px' and '0.5,s'; these should read '0.12 px' and '0.5 s'.
  2. [§3.5 / Ref. [39]] The annotation model is called 'Kimi-K2.6' in the text, but reference [39] is titled 'Kimi k2.5: Visual agentic intelligence'; the version designation should be aligned.
  3. [Fig. 1 / §4.2] Figure 1's caption gives 1,912 hours for the newly collected YouTube subset while the figure's internal label and §4.2 give 1,911 hours; these numbers should be reconciled.
  4. [§5.2] The phrase 'a evaluation using a fixed prior focal length' should read 'an evaluation using a fixed prior focal length'.
  5. [Table 1] In the rows for WebVid, Panda-70M, and OpenVid-1M, the 'Pose' column appears to use a combined '–✓' marker; separate ✓/✗ entries or a clear legend would avoid ambiguity.
  6. [App B.4] The statement that the procedure 'handles mid-trajectory revisits rather than only start-to-end returns, including 25 such sequences' would benefit from explaining how the 25 mid-trajectory revisits were identified, since the candidate-generation description emphasizes keyframes from the beginning and ending portions.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the dataset claims are manifest counts and design choices, and the validation checks are transparent internal-consistency tests rather than fitted inputs renamed as predictions.

full rationale

Sekai2 is a dataset-construction report, not a derivation of predictions from first principles. The headline corpus statistics (128,892 clips, 2,826 hours, 649,597 segments, duration shares) are manifest counts and explicit curation choices, such as the 120-second cap and the caption prompt's target segment density; they are not fitted parameters later relabeled as findings. Camera-pose validation is genuinely independent of the released poses: the ground-truth-free epipolar test uses SIFT correspondences with the released poses and intrinsics, and the mismatched-pose negative control (Eq. 5) breaks pose-frame correspondence while holding all image-side inputs fixed, so the observed error increase is evidence of alignment rather than a self-fulfilling construction. The cross-run DROID comparison is explicitly framed as evidence of cross-run geometric consistency, not absolute metric accuracy, because ViPE itself uses a DROID-based stack; this disclaimer prevents the evaluation from being presented as independent ground truth. Caption assessment is internally controlled: the pairwise comparison holds both the visual input and the captioning model fixed, and the reported 71.6% first-position bias is handled by reporting the conservative second-position condition. A VLM judge from the same model family as the generator is a methodological limitation, but the paper does not define caption quality as 'whatever the generating VLM prefers'; the comparison isolates the structured annotation protocol from model capacity. The self-citations, including Sekai as the predecessor corpus, are contextual and not load-bearing: the new contributions (unified pipeline, panoramic subset, hierarchical annotations) are constructed and evaluated within the paper rather than derived from those citations. The limited public availability of the panoramic subset (Sec. 6) reduces external verifiability of that component, but it is a disclosure of an availability constraint, not a circular argument. No step in the paper reduces, by its own equations or by self-citation, to its own inputs.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The central claims rest on curation thresholds (motion gate, static-recovery budget, clip cap, pose-score anchors, pose-graph weights, camera-path majority rule) and on assumptions about the reliability of shot detection, monocular SLAM, VLM captioning, and loop-closure verification. These are engineering choices and domain assumptions rather than physical parameters; no new physical entities are introduced.

free parameters (6)
  • Motion gate threshold r_static = 0.5
    Clips with at least 50% static adjacent-frame pairs are rejected by the strict motion gate (Sec. 3.3, App. B.2). This hand-chosen threshold determines the composition of the dynamic main set and drives the size of the separately recovered low-motion branch.
  • Static-supplement target fraction rho = 0.06
    The low-motion recovery branch aims for 6% of final perspective footage hours (App. B.2). Together with the bin round-robin selection this shapes how much stationary observation enters the release.
  • Clip duration cap and boundary buffer = 120 s cap, 6 s buffer
    Continuous shots are partitioned into clips of at most 120 seconds with a 6-second buffer removed around internal boundaries (Sec. 3.3, App. B.1). These constants define the long-horizon property and the analysis decomposition.
  • Pose-quality score weights and anchors = 0.3/0.3/0.2/0.2; anchors 0.15-2.0, 1-15, 0.002-0.02, 2-15; accept q<0.5
    The pose acceptance score combines normalized peak, jitter, median, and drift residuals with empirically chosen anchors (App. B.7, Eq. 4). These thresholds gate which trajectories are released and directly affect the claim that every clip has a usable camera trajectory.
  • Loop-closure PGO information weights = rotation sqrt-weight 50, translation 1
    Pose-graph optimization assigns square-root information weights of 50 to rotation and 1 to translation in odometry factors (App. B.4). The choice determines how strongly local orientation is trusted during panoramic trajectory refinement.
  • camera_path majority threshold = 0.70
    A discrete camera-path label is assigned when one behavior occupies more than 70% of an interval; otherwise the interval is labeled mixed (App. B.5). This rule shapes the camera_path distribution used in cross-modal grounding evaluation.
assumptions (6)
  • domain assumption OmniShotCut shot-boundary detection is accurate, and a 6-second buffer around boundaries removes discontinuous state changes that would be mislearned as dynamics.
    Sec. 3.3; this processing decision determines clip boundaries without human verification.
  • domain assumption Monocular SLAM (ViPE/DROID-SLAM) trajectories, after filtering and pose-graph refinement, are accurate enough to serve as camera-conditioning supervision for world models.
    Sec. 3.4 and limitations in Sec. 6; no metric ground truth; scale ambiguity acknowledged.
  • domain assumption Kimi-K2.6 VLM descriptions and temporal segment boundaries are faithful, with residual errors acceptable for training.
    Sec. 3.5; quality checked by a same-family VLM judge (Sec. 5.3), not by humans.
  • ad hoc to paper The rotation-only median angular residual is a valid criterion to distinguish genuine loops from nearby distinct locations.
    App. B.4, Eq. (2); this is the paper's specific mechanism for claiming revisits exist.
  • standard math SIFT correspondences and the Sampson-error protocol measure pose-frame consistency; the mismatched-pose control validates sensitivity.
    Sec. 5.2 and App. C.1; standard geometric validation.
  • domain assumption Heaps-law vocabulary growth (beta about 0.43) supports the caption-diversity claim.
    App. D.3; a fitted descriptive model, no uncertainty reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Sekai2: From World Exploration to Interactive World Modeling." pith.science (2026). https://pith.science/paper/WYKGCXET

@misc{pith2026260809449,
  author       = {Pith},
  title        = {Pith review of: Sekai2: From World Exploration to Interactive World Modeling},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WYKGCXET}},
  note         = {Machine review of arXiv:2608.09449}
}
read the original abstract

Video world models must capture how scenes evolve over time and across viewpoints. Training them for long-horizon generation and camera control therefore benefits from long videos paired with camera trajectories and temporally grounded semantics. Existing corpora rarely offer the three together: large-scale web video provides broad visual diversity but no trajectories or time-aligned text, while pose-annotated datasets are typically short-range or reconstruction-oriented. We introduce Sekai2, a multi-source real-world video dataset that carries the world-exploration footage of Sekai toward interactive world modeling. The release contains 128,892 clips totaling 2,826 hours from 10,428 source videos across 113 countries or regions, and is deliberately weighted toward sustained observation: under a common 120-second decomposition, 43,594 segments reach the full two minutes and account for 51.4% of all footage. Every clip includes a released camera trajectory and hierarchical annotations disentangling subject motion, environment dynamics, static scene content, and camera behavior, resulting in 649,597 temporally grounded segments. Crucially, we further introduce 982 panoramic sequences captured along non-linear trajectories with loops and revisits. These revisits provide repeated observations of the same locations across time and viewpoints, offering essential supervision for learning persistent scene representations, long-term spatial memory, and geometrically consistent world models. Corpus-scale analyses demonstrate complete pose-and-caption coverage, broad geographic and semantic diversity, varied camera trajectories, and highly non-redundant temporal descriptions. Together, these properties make Sekai2 a scalable resource for long-horizon video generation, camera-controllable synthesis, and interactive world-model pre-training.

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

61 extracted references · 37 canonical work pages

  1. [1]

    M. Bain, A. Nagrani, G. Varol, and A. Zisserman. Frozen in time: A joint video and image encoder for end-to-end retrieval. In2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 1708–1718. IEEE, 2021

    work page 2021

  2. [2]

    P . Ball, J. Bauer, et al. Genie 3: A new frontier for world models.https://deepmind.google/discover/ blog/genie-3-a-new-frontier-for-world-models/, 2025

    2025

  3. [3]

    Bruce, M

    J. Bruce, M. D. Dennis, A. Edwards, J. Parker-Holder, Y. Shi, E. Hughes, M. Lai, A. Mavalankar, R. Steigerwald, C. Apps, et al. Genie: Generative interactive environments. InForty-first international conference on machine learning, 2024

    work page 2024

  4. [4]

    H. Che, X. He, Q. Liu, C. Jin, and H. Chen. Gamegen-x: Interactive open-world game video generation. InInternational Conference on Learning Representations, volume 2025, pages 37546–37593, 2025

    work page 2025

  5. [5]

    D. Chen, T. Kasarla, Y. Bang, M. Shukor, W. Chung, J. Yu, A. Bolourchi, T. Moutakanni, and P . Fung. Action100m: A large-scale video action dataset. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pages 8832–8842, June 2026

    work page 2026

  6. [6]

    L. Chen, X. Wei, J. Li, X. Dong, P . Zhang, Y. Zang, Z. Chen, H. Duan, B. Lin, Z. Tang, et al. Sharegpt4video: Improving video understanding and generation with better captions.Advances in Neural Information Processing Systems, 37:19472–19495, 2024

    work page 2024

  7. [7]

    T.-S. Chen, A. Siarohin, W. Menapace, E. Deyneka, H.-w. Chao, B. E. Jeon, Y. Fang, H.-Y. Lee, J. Ren, M.-H. Yang, et al. Panda-70m: Captioning 70m videos with multiple cross-modality teachers. In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 13320–13331. IEEE, 2024

    work page 2024

  8. [8]

    Decart, Q

    E. Decart, Q. McIntyre, S. Campbell, X. Chen, and R. Wachen. Oasis: A universe in a transformer.URL: https://oasis-model. github. io, 2(3):6, 2024. 16

    work page 2024

Show all 61 references
  1. [9]

    M. A. Fischler and R. C. Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography.Communications of the ACM, 24(6):381–395, 1981

    1981

  2. [10]

    Z. Gao, Q. Wang, J. Zhu, J. Chen, Z. Liu, Q. Bai, J. Wang, Y. Yuan, H. Wang, Y. Lu, et al. Infinite worlds with versatile interactions.arXiv preprint arXiv:2607.07534, 2026

    2026 arXiv

  3. [11]

    Grauman, A

    K. Grauman, A. Westbury, E. Byrne, Z. Chavis, A. Furnari, R. Girdhar, J. Hamburger, H. Jiang, M. Liu, X. Liu, et al. Ego4d: Around the world in 3,000 hours of egocentric video. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 18995–19012, 2022

    2022

  4. [12]

    Grisetti, R

    G. Grisetti, R. Kümmerle, C. Stachniss, and W. Burgard. A tutorial on graph-based slam.IEEE Intelligent Transportation Systems Magazine, 2(4):31–43, 2010

    2010

  5. [13]

    J. Guo, Y. Ye, T. He, H. Wu, Y. Jiang, T. Pearce, and J. Bian. Mineworld: a real-time and open-source interactive world model on minecraft.arXiv preprint arXiv:2504.08388, 2025

    2025 arXiv

  6. [14]

    S. Han, W. Huang, H. Shi, L. Zhuo, X. Su, S. Zhang, X. Zhou, X. Qi, Y. Liao, and S. Liu. Videoespresso: A large-scale chain-of-thought dataset for fine-grained video reasoning via core frame selection. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVP...

    2025

  7. [15]

    H. He, Y. Xu, Y. Guo, G. Wetzstein, B. Dai, H. Li, and C. Yang. Cameractrl: Enabling camera control for text-to-video generation.arXiv preprint arXiv:2404.02101, 2024

    2024 arXiv

  8. [16]

    H. He, C. Yang, S. Lin, Y. Xu, M. Wei, L. Gui, Q. Zhao, G. Wetzstein, L. Jiang, and H. Li. Cameractrl ii: Dynamic scene exploration via camera-controlled video diffusion models. In2025 IEEE/CVF International Conference on Computer Vision (ICCV), pages 13416–13426. IEEE, 2025

    2025

  9. [17]

    X. He, C. Peng, Z. Liu, B. Wang, Y. Zhang, Q. Cui, F. Kang, B. Jiang, M. An, Y. Ren, et al. Matrix-game 2.0: An open-source real-time and streaming interactive world model.arXiv preprint arXiv:2508.13009, 2025

    2025 arXiv

  10. [18]

    A. Hu, L. Russell, H. Yeo, Z. Murez, G. Fedoseev, A. Kendall, J. Shotton, and G. Corrado. Gaia-1: A generative world model for autonomous driving.arXiv preprint arXiv:2309.17080, 2023

    2023 arXiv

  11. [19]

    Huang, Q

    J. Huang, Q. Zhou, H. Rabeti, A. Korovko, H. Ling, X. Ren, T. Shen, J. Gao, D. Slepichev, C.-H. Lin, et al. Vipe: Video pose engine for 3d geometric perception.arXiv preprint arXiv:2508.10934, 2025

    2025 arXiv

  12. [20]

    HunyuanWorld

    T. HunyuanWorld. Hy-world 1.5: A systematic framework for interactive world modeling with real-time latency and geometric consistency.arXiv preprint, 2025

    2025

  13. [21]

    X. Ju, Y. Gao, Z. Zhang, Z. Yuan, X. Wang, A. Zeng, Y. Xiong, Q. Xu, and Y. Shan. Miradata: A large-scale video dataset with long durations and structured captions.Advances in Neural Information Processing Systems, 37:48955–48970, 2024

    2024

  14. [22]

    W. Kabsch. A solution for the best rotation to relate two sets of vectors.Foundations of Crystallography, 32(5):922–923, 1976

    1976

  15. [23]

    Kanervisto, D

    A. Kanervisto, D. Bignell, L. Y. Wen, M. Grayson, R. Georgescu, S. Valcarcel Macua, S. Z. Tan, T. Rashid, T. Pearce, Y. Cao, et al. World and human action models towards gameplay ideation.Nature, 638(8051):656–663, 2025

    2025

  16. [24]

    Kirillov, E

    A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W.-Y. Lo, et al. Segment anything. In2023 IEEE/CVF international conference on computer vision (ICCV), pages 3992–4003. IEEE, 2023

    2023

  17. [25]

    Kümmerle, G

    R. Kümmerle, G. Grisetti, H. Strasdat, K. Konolige, and W. Burgard. g 2 o: A general framework for graph optimization. In2011 IEEE international conference on robotics and automation, pages 3607–3613. IEEE, 2011

    2011

  18. [26]

    H. Li, D. Zhang, Y. Zhou, X. Zhang, H. Feng, X. Lin, W. Jiang, B. Du, M.-H. Yang, and L. Qi. Panoworld: Real-world panoramic generation.arXiv preprint arXiv:2607.09661, 2026. 17

    2026 arXiv

  19. [27]

    J. Li, J. Tang, Z. Xu, L. Wu, Y. Zhou, S. Shao, T. Yu, Z. Cao, and Q. Lu. Hunyuan-gamecraft: High-dynamic interactive game video generation with hybrid history condition.arXiv preprint arXiv:2506.17201, 2(3):6, 2025

    2025 arXiv

  20. [28]

    Z. Li, C. Li, X. Mao, S. Lin, M. Li, S. Zhao, Z. Xu, X. Li, Y. Feng, J. Sun, et al. Sekai: A video dataset towards world exploration.Advances in Neural Information Processing Systems, 38, 2026

    2026

  21. [29]

    L. Ling, Y. Sheng, Z. Tu, W. Zhao, C. Xin, K. Wan, L. Yu, Q. Guo, Z. Yu, Y. Lu, et al. Dl3dv-10k: A large-scale scene dataset for deep learning-based 3d vision. In2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 22160–22169. IEEE, 2024

    2024

  22. [30]

    S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, Q. Jiang, C. Li, J. Yang, H. Su, et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. InEuropean conference on computer vision, pages 38–55. Springer, 2024

    2024

  23. [31]

    X. Mao, Z. Li, C. Li, X. Xu, K. Ying, and K. Zhang. Yume1. 5: A text-controlled interactive world generation model. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 7752–7761, 2026

    2026

  24. [32]

    Menapace, A

    W. Menapace, A. Siarohin, S. Lathuilière, P . Achlioptas, V . Golyanik, S. Tulyakov, and E. Ricci. Prompt- able game models: Text-guided game simulation via masked diffusion models.ACM Transactions on Graphics, 43(2):1–16, 2024

    2024

  25. [33]

    K. Nan, R. Xie, P . Zhou, T. Fan, Z. Yang, Z. Chen, X. Li, J. Yang, and Y. Tai. Openvid-1m: A large-scale high-quality dataset for text-to-video generation. InInternational Conference on Learning Representations, volume 2025, pages 1045–1064, 2025

    2025

  26. [34]

    Parker-Holder, P

    J. Parker-Holder, P . Ball, J. Bruce, V . Dasagi, K. Holsheimer, C. Kaplanis, A. Moufarek, G. Scully, J. Shar, J. Shi, et al. Genie 2: A large-scale foundation world model. https://deepmind.google/discover/ blog/genie-2-a-large-scale-foundation-world-model, 2024

    2024

  27. [35]

    X. Ren, T. Shen, J. Huang, H. Ling, Y. Lu, M. Nimier-David, T. Müller, A. Keller, S. Fidler, and J. Gao. Gen3c: 3d-informed world-consistent video generation with precise camera control. In2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6121–6...

    2025

  28. [36]

    Rublee, V

    E. Rublee, V . Rabaud, K. Konolige, and G. Bradski. Orb: An efficient alternative to sift or surf. In2011 International conference on computer vision, pages 2564–2571. Ieee, 2011

    2011

  29. [37]

    Z. Tan, X. Yang, L. Qin, and H. Li. Vidgen-1m: A large-scale dataset for text-to-video generation.arXiv preprint arXiv:2408.02629, 2024

    2024 arXiv

  30. [38]

    J. Tang, J. Liu, J. Li, L. Wu, H. Yang, P . Zhao, S. Gong, X. Yuan, S. Shao, L. Zhang, et al. Hunyuan- gamecraft-2: Instruction-following interactive game world model.arXiv preprint arXiv:2511.23429, 2025

    2025

  31. [39]

    K. Team, T. Bai, Y. Bai, Y. Bao, S. Cai, Y. Cao, Y. Charles, H. Che, C. Chen, G. Chen, et al. Kimi k2. 5: Visual agentic intelligence.arXiv preprint arXiv:2602.02276, 2026

    2026 arXiv

  32. [40]

    R. Team, Z. Gao, Q. Wang, Y. Zeng, J. Zhu, K. L. Cheng, Y. Li, H. Wang, Y. Xu, S. Ma, et al. Advancing open-source world models.arXiv preprint arXiv:2601.20540, 2026

    2026 arXiv

  33. [41]

    Teed and J

    Z. Teed and J. Deng. Raft: Recurrent all-pairs field transforms for optical flow. InEuropean conference on computer vision, pages 402–419. Springer, 2020

    2020

  34. [42]

    Teed and J

    Z. Teed and J. Deng. Droid-slam: Deep visual slam for monocular, stereo, and rgb-d cameras.Advances in neural information processing systems, 34:16558–16569, 2021

    2021

  35. [43]

    Valevski, Y

    D. Valevski, Y. Leviathan, M. Arar, and S. Fruchter. Diffusion models are real-time game engines. In International Conference on Learning Representations, volume 2025, pages 73754–73776, 2025

    2025

  36. [44]

    Veicht, P .-E

    A. Veicht, P .-E. Sarlin, P . Lindenberger, and M. Pollefeys. Geocalib: Learning single-image calibration with geometric optimization. InEuropean Conference on Computer Vision, pages 1–20. Springer, 2024. 18

    2024

  37. [45]

    B. Wang, G. Xu, J. Zhang, Z. Tang, and Z. Cheng. Omnishotcut: Holistic relational shot boundary detection with shot-query transformer.arXiv preprint arXiv:2604.24762, 2026

    2026 arXiv

  38. [46]

    J. Wang, Y. Yuan, R. Zheng, Y. Lin, J. Gao, L.-Z. Chen, Y. Bao, C. Zeng, Y. Zhou, X.-X. Long, et al. Spatialvid: A large-scale video dataset with spatial annotations. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 42592–42603, 2026

    2026

  39. [47]

    W. Wang, D. Zhu, X. Wang, Y. Hu, Y. Qiu, C. Wang, Y. Hu, A. Kapoor, and S. Scherer. Tartanair: A dataset to push the limits of visual slam. In2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 4909–4916. IEEE, 2020

    2020

  40. [48]

    Y. Wang, Y. He, Y. Li, K. Li, J. Yu, X. Ma, X. Li, G. Chen, X. Chen, Y. Wang, et al. Internvid: A large-scale video-text dataset for multimodal understanding and generation. InInternational Conference on Learning Representations, volume 2024, pages 42055–42079, 2024

    2024

  41. [49]

    Y. Wang, Q. Zhang, S. Cai, T. Wu, J. Ackermann, Z. Kuang, Y. Zheng, F. Rajiˇ c, S. Tang, and G. Wetzstein. Bullettime: Decoupled control of time and camera pose for video generation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 183...

    2026

  42. [50]

    Z. Wang, Z. Liu, J. Li, K. Huang, B. Xu, F. Kang, M. An, P . Wang, B. Jiang, Y. Wei, et al. Matrix-game 3.0: Real-time and streaming interactive world model with long-horizon memory.arXiv preprint arXiv:2604.08995, 2026

    2026 arXiv

  43. [51]

    Z. Wang, Z. Yuan, X. Wang, Y. Li, T. Chen, M. Xia, P . Luo, and Y. Shan. Motionctrl: A unified and flexible motion controller for video generation. InACM SIGGRAPH 2024 Conference Papers, pages 1–11, 2024

    2024

  44. [52]

    R. Wu, X. He, M. Cheng, T. Yang, Y. Zhang, Z. Kang, X. Cai, X. Wei, C. Guo, C. Li, et al. Infinite-world: Scaling interactive world models to 1000-frame horizons via pose-free hierarchical memory.arXiv preprint arXiv:2602.02393, 2026

    2026

  45. [53]

    T. Wu, S. Yang, R. Po, Y. Xu, Z. Liu, D. Lin, and G. Wetzstein. Video world models with long-term spatial memory.Advances in Neural Information Processing Systems, 38:49371–49393, 2026

    2026

  46. [54]

    Z. Xiao, Y. Lan, Y. Zhou, W. Ouyang, S. Yang, Y. Zeng, and X. Pan. Worldmem: Long-term consistent world simulation with memory.Advances in Neural Information Processing Systems, 38:49632–49652, 2026

    2026

  47. [55]

    D. Xu, W. Nie, C. Liu, S. Liu, J. Kautz, Z. Wang, and A. Vahdat. Camco: Camera-controllable 3d-consistent image-to-video generation.arXiv preprint arXiv:2406.02509, 2024

    2024 arXiv

  48. [56]

    D. Yang, S. Huang, C. Lu, X. Han, H. Zhang, Y. Gao, Y. Hu, and H. Zhao. Vript: A video is worth thousands of words.Advances in Neural Information Processing Systems, 37:57240–57261, 2024

    2024

  49. [57]

    S. Yang, Y. Du, K. Ghasemipour, J. Tompson, L. Kaelbling, D. Schuurmans, and P . Abbeel. Learning interactive real-world simulators.arXiv preprint arXiv:2310.06114, 2023

    2023 arXiv

  50. [58]

    M. Yu, W. Hu, J. Xing, and Y. Shan. Trajectorycrafter: Redirecting camera trajectory for monocular videos via diffusion models. In2025 IEEE/CVF International Conference on Computer Vision (ICCV), pages 100–111. IEEE, 2025

    2025

  51. [59]

    Zhang, D

    W. Zhang, D. Xiao, A. Dai, Y. Liu, T. Pan, S. Wen, L. Chen, and L. Wang. Leader360v: A large-scale, real-world 360 video dataset for multi-task learning in diverse environment.Advances in Neural Information Processing Systems, 38, 2026

    2026

  52. [60]

    T. Zhou, R. Tucker, J. Flynn, G. Fyffe, and N. Snavely. Stereo magnification: Learning view synthesis using multiplane images.arXiv preprint arXiv:1805.09817, 2018

    2018 arXiv

  53. [61]

    unknown

    Y. Zhou, Y. Wang, J. Zhou, W. Chang, H. Guo, Z. Li, K. Ma, X. Li, Y. Wang, H. Zhu, et al. Omniworld: A multi-domain and multi-modal dataset for 4d world modeling.arXiv preprint arXiv:2509.12201, 2025. 19 A Detailed Dataset Statistics A.1 Released Dataset Statistics The newly c...

    2025

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.