Pith. sign in

REVIEW 3 major objections 4 minor 67 references

DynTrace claims that dynamic 4D video reasoning fails because models confuse object motion with camera motion and lose continuous object traces, and that a training-free infusion of geometry-grounded arrows and structured text traces fixes

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 06:28 UTC pith:7BFFK45V

load-bearing objection Useful training-free pipeline for 4D MLLM reasoning, but the headline claim is undercut by an uncontrolled input-substitution confound: DTG hands the model the metric answers. the 3 major comments →

arxiv 2607.12503 v2 pith:7BFFK45V submitted 2026-07-14 cs.CV

DynTrace: Tracking Dynamic Object Evidence for 4D Spatio-Temporal Reasoning in MLLMs

Rongxin Gao , Yuzhi Huang , Dongxuan Liu , Chu Li , Zhenye Wang , Jie Wu , Shuzhao Xie , Jingyan Jiang
show 3 more authors
Xinghao Ding Xiaotong Tu Yue Huang
This is my paper
classification cs.CV
keywords 4D spatio-temporal reasoningmultimodal large language modelsdynamic object trackingcamera motion disentanglementtrajectory visualizationvisual question answeringtraining-free inferencedynamic trace graph
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper sets out to explain why large multimodal models (MLLMs) are bad at dynamic 4D spatio-temporal reasoning—questions about how objects move, turn, approach, or separate over time in video. It argues that the failures come from two evidence deficits: models confuse real object motion with camera-induced apparent motion, and sparse frame sampling breaks the continuous trail of each object. DynTrace is a training-free fix: before the answering model sees anything, it extracts query-relevant moving objects, reconstructs their 3D trajectories, and presents them to the MLLM twice—as annotated arrows overlaid on frames and as a structured text graph of cues, trends, and key moments. The paper reports that this consistently improves four open-source MLLMs across three 4D benchmarks, with the fused evidence beating each component alone and an 8B model with DynTrace outscoring a raw 32B model. If right, the practical lesson is that explicit, continuously tracked dynamic evidence can substitute for much larger models or additional training.

Core claim

DynTrace's central claim is that current MLLMs fail at continuous dynamic scene reasoning not from a lack of visual content but from how the evidence is delivered: sparse, untracked, camera-uncompensated frames. The method builds Geometry-Grounded Dynamic Evidence: object trajectories lifted into a shared world frame via monocular depth and pose, camera behavior summarized separately, and pair relations (distance, bearing, interaction moments) computed from those trajectories. It then renders this evidence twice—Dynamic Trajectory Visualization (DTV), which reprojects world-space motion as arrows on the image plane, and Dynamic Trace Graph (DTG), a windowed text graph of object and relation

What carries the argument

At the core are two complementary evidence channels built from the same metric reconstruction. DTV (Dynamic Trajectory Visualization) takes the world-coordinate trajectory of each tracked object, projects it through the camera model onto the frame, and draws a direction arrow on the object's mask; because the arrow comes from world motion, not image displacement, it stays truthful when the camera itself moves. DTG (Dynamic Trace Graph) partitions the video into temporal windows and serializes, per window, a camera summary, object tokens, and relation tokens; each token records the state at the window's end (cues), how it changed through the window (trace), and sparse decisive events such as

Load-bearing premise

The whole evidence chain treats monocular depth, camera pose, and a global affine camera-motion model as reliable enough that their errors never need visual verification, while raw frames are withheld from the answering model.

What would settle it

Use synthetic videos with known ground-truth 3D trajectories and a camera path engineered to produce parallax (depth discontinuities moving across the frame). If DynTrace's accuracy on static foreground objects falls below the same MLLM seeing raw frames, the claim that tracked dynamic evidence—rather than lucky geometry—carries the result is falsified; equivalently, deliberately scaling the estimated depth should change answers in proportion to the depth error.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Any open-source MLLM can be upgraded at inference time by prepending DynTrace-style evidence; the paper reports gains of +4.3 to +5.0 on its main benchmark and +3.0 to +6.1 on the other two.
  • Explicit dynamic evidence can partially substitute for scale: an 8B model with DynTrace beats a raw 32B model.
  • The two components are complementary, not redundant: DTG alone adds +2.2 and DTV alone +0.9 over the raw 8B backbone, while together they add +5.0.
  • The benefit persists under temporary occlusion (overall 58.1% to 62.0% on 75 such clips), because the text trace survives gaps in visual evidence.
  • Both failure modes improve separately: gains appear on question subsets classified as Dynamic Source Confusion and on subsets classified as Dynamic Trace Loss.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The authors' framing suggests a cheap adaptive deployment rule: for questions about camera-relative direction, feed only DTV; for questions about distance evolution or event order, feed only DTG; a router could halve token cost without losing much accuracy.
  • The reported gains probably upper-bound the value of tracking; the real bottleneck is the metric quality of the monocular depth and pose estimates and the assumption that camera-induced flow is a global affine transform. Videos with strong parallax or scale drift would stress-test this.
  • The failure on part-level and first-person-frame questions implies the next advance is not better object tracking but per-part trajectories and viewpoint transformation—two concrete extensions suggested by the paper's own limitations.
  • Because raw frames are never shown, the answering model cannot verify the arrows or numbers against pixels; keeping a few original frames as a cross-check channel might catch residual geometric errors.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes DynTrace, a training-free inference-time framework that augments MLLM inputs for 4D spatio-temporal reasoning. It extracts query-relevant dynamic objects, reconstructs their world-coordinate trajectories from monocular depth and pose, and provides two forms of evidence: DTV (Dynamic Trajectory Visualization), which reprojects world trajectories onto the image plane as directional overlays, and DTG (Dynamic Trace Graph), which serializes per-window object/relation tokens containing dynamic cues, trace evolution, and key moments. The enhanced MLLM receives DTV images plus DTG text instead of raw frames. Experiments on Dyn-Bench, VLM4D, and DSI-Bench report consistent improvements over four open-source MLLM backbones, with ablations attributing +2.2 of the +5.0 Dyn-Bench gain to DTG and +0.9 to DTV.

Significance. If the central claim is established, DynTrace would be a useful, inexpensive, model-agnostic way to inject explicit dynamic evidence into MLLMs, and its decomposition into Dynamic Source Confusion and Dynamic Trace Loss would provide a principled taxonomy for 4D reasoning failures. The paper has several strengths: the pipeline is clearly described and training-free; gains are reported on three benchmarks, including two externally-authored ones; ablations separate the contributions of DTV, DTG, and DT-Token components; Table 7 gives a GT-versus-predicted robustness check; and the appendix contains honest limitation cases. However, the headline interpretation—that tracking dynamic object evidence improves MLLM reasoning—is currently underdetermined because the experimental setup changes both the information content and the input modality, not just the reasoning scaffold.

major comments (3)
  1. [§3.3, Eq. (10); §4.1; Table 3] The DynTrace condition is not controlled for the information content of the input. The enhanced model receives DTG serialized text (Eqs. 7–9) that explicitly lists end-of-window positions, speeds, headings, distances, bearings, and key moments such as CLOSEST_POINT with timestamps (e.g., Fig. 7; Fig. 12), while the baseline receives raw frames and must estimate these quantities. Many questions in Dyn-Bench, VLM4D, and DSI-Bench ask for exactly these values (e.g., distance at a given time, average speed). Table 3 shows that DTG alone accounts for +2.2 of the +5.0 gain, versus only +0.9 for DTV, which is consistent with the model reading the answer directly from the text. Please add controls: (a) keep raw frames visible in the DynTrace condition; (b) give the baseline an equally informative but non-tracking textual summary (e.g., per-window positions without trajectory-trend or key-moment
  2. [§1, §4.1; Ref. [10]] Dyn-Bench is used as the primary benchmark and is also cited in the introduction as empirical validation of the paper's own premise: 'Dyn-Bench [10] empirically validates this coupled mechanism.' Ref. [10] shares at least four authors with this paper (Yuzhi Huang, Yue Huang, Xinghao Ding, Jingyan Jiang). Using a same-group benchmark to validate a central premise of the same group creates a circularity risk, and the paper does not disclose the overlap. This concern is mitigated by the external VLM4D and DSI-Bench results, but the SOTA claim on Dyn-Bench and the framing of Dyn-Bench as the main support need revision. Please disclose the author overlap and restructure the evidence so that externally-authored benchmarks carry at least equal weight.
  3. [§3.2, Eqs. (5)/(22), (12)–(14); Tables 1–7] The world-coordinate trajectories, DTV arrows, and DTG metric values are all built on monocular depth and pose estimates plus a 2D affine model of camera-induced background motion. No accuracy analysis of these intermediate estimates is reported, and no error bars or significance tests are given for any of the headline results. Since the DynTrace input replaces the raw evidence with these estimates, errors in depth scale, pose drift, or parallax that the affine model cannot capture propagate directly into answers. Table 7 is a useful start but only reports one averaged subset. Please report sensitivity of final accuracy to plausible perturbations of the intermediate geometry (e.g., depth scaled by ±10%, ±20%; pose jitter) and provide at least bootstrap confidence intervals for the main gains, especially for category-level gains as small as +0.5 and +0.8 in Table 1.
minor comments (4)
  1. [Fig. 4] The legend is difficult to read because the same marker style is used for both benchmarks and DynTrace/baseline pairs. Please use distinct line styles and label each series directly.
  2. [§3.3 and Fig. 2] The notation alternates between D/DTV and G/DTG in the text and figure. Please define the symbols once in a notation table to avoid confusion.
  3. [Fig. 18 prompt vs. Fig. 3/14/15 examples] The final reasoning prompt asks for 'ONLY the letter (A, B, C, or D)', but the qualitative examples show long reasoning chains. Please clarify whether the reported numbers come from the letter-only protocol or from a separate reasoning-then-answer setting.
  4. [Appendix B, 'Current limitations'] The two limitation cases—egocentric frame transformation and fine-grained body-part relations—are honestly reported and align with the method's design. It would strengthen the paper to add one sentence in the main text referencing these boundaries.

Circularity Check

2 steps flagged

Empirical validation is partly circular: DTG supplies answer-relevant metric values in the prompt, and the primary benchmark Dyn-Bench is self-authored and cited as validating the paper's own premise.

specific steps
  1. self definitional [Sec. 3.3 Eq. (10); Sec. 4.1; Fig. 14(b)]
    "During inference, DynTrace feeds the user query q, the DTV collection D, and the serialized DTG G into the target MLLM: \hat{y}= MLLM(q,D,G). ... Rather than feeding original video frames to the target MLLM, the DynTrace-enhanced version feeds DTV together with the serialized DTG and the query. ... The provided Dynamic Trace Graph (DTG) evidence is the only source of quantitative motion data. The 'Cues' section for Object [ID: obj1] in each window provides the object's speed at the end of that time interval. The question asks for the average speed over the entire video duration."

    DTG serializes end-of-window positions, speeds, headings, distances, bearings, and key moments such as CLOSEST_POINT/TURN_POINT. Many Dyn-Bench, VLM4D, and DSI-Bench questions ask for exactly these quantities (e.g., 'At the 1.5-second mark, approximately how far is the person from the camera?'; 'What is the average speed of the silver car?'). Since Eq. (10) feeds DTG to the answering MLLM while the baseline receives only raw frames, the accuracy gain is largely a read-the-answer-from-text effect. The paper's conclusion that tracking dynamic object evidence improves reasoning is therefore self-definitional: the 'evidence' is defined to include the queried metric, so the improved answer is forced by construction.

  2. self citation load bearing [Sec. 1; Sec. 4.2 (Dyn-Bench)]
    "Dyn-Bench [10] empirically validates this coupled mechanism, showing that effective dynamic-scene reasoning depends on jointly understanding camera movement and tracking evolving objects in a unified and object-centered manner. ... Dyn-Bench serves as our primary benchmark ... This framework directly instantiates our central premise: accurate MLLM reasoning requires continuous, interpretable dynamic object evidence."

    Dyn-Bench [10] is authored by an overlapping set of authors (Yuzhi Huang, Rongxin Gao, Dongxuan Liu, Jie Wu, Shuzhao Xie, Jingyan Jiang, Xinghao Ding, Yue Huang). The paper invokes this citation as empirical validation of its own central premise, then uses the same benchmark as the primary evaluation, calling it 'the strongest and most direct support.' Because the benchmark was designed around the same premise, the citation cannot independently justify that premise; the support loop closes on the authors' own prior work.

full rationale

The geometric construction of DTV/DTG is not itself circular: Eqs. (5)/(22) lift tracked masks to world coordinates, and DTV/DTG are derived from those trajectories. The circularity lies in the empirical validation. First, DTG explicitly contains the metric values that the benchmark questions ask about (position, speed, distance, bearing, key moments), and the DynTrace condition replaces raw frames with DTV+DTG (Eq. 10), so the MLLM can read the answer from the text; the ablation (DTG alone +2.2 vs DTV alone +0.9) is consistent with this read-the-answer effect. Second, the primary benchmark Dyn-Bench [10] is authored by overlapping authors and is cited as validating the paper's own premise, creating a self-citation loop. VLM4D and DSI-Bench provide some independent benchmark variety, but they share the same DTG input-leak issue, so the central 'improvement' claim is substantially forced by the input containing answer-relevant values. Score 6 reflects partial circularity in the central empirical claim, not a claim that the geometric derivation is tautological.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 0 invented entities

The working machinery is standard computer-vision tooling (optical flow, affine stabilization, SAM tracking, monocular unprojection, tokenized graphs); the paper adds no new physical or mathematical postulates. The audit therefore focuses on the measurement assumptions that make the metric evidence trustworthy, and on the handful of unstated thresholds that shape the pipeline.

free parameters (4)
  • Flow threshold tau_f = unspecified
    Eq. 15 thresholds residual flow magnitude to declare motion regions; the value is never reported, yet it controls which objects get extracted.
  • Probe length lambda in DTV = adaptive, unspecified
    Appendix A.2: lambda is enlarged until the projected displacement is 'visually distinguishable'; no fixed criterion, per-object hand-tuning affects arrow semantics.
  • Number of temporal windows K for DTG = unspecified
    Section 3.2 partitions each video into K windows; K and window length are never given, and DTG granularity depends on them.
  • Sampled frame count = 16 frames
    Section 4.1/Fig. 4: 16 is the optimum among {8,16,24,32} and is used for all main results; the optimum itself is a chosen operating point.
axioms (5)
  • domain assumption Camera-induced frame-to-frame motion is well-approximated by a global 2D affine transform fitted to dominant flow (Eqs. 12-14).
    Used to build residual flow and extract dynamic objects; affine models fail under translational parallax, so true object motion can be misestimated in complex scenes.
  • domain assumption DA3 monocular depth (median over mask) and camera pose are sufficiently metric and temporally aligned to yield real-world positions and speeds (Eq. 5/22).
    DTG reports absolute meters and m/s to the MLLM; monocular scale/pose errors propagate directly into answers because raw frames are not shown to the model.
  • domain assumption Each dynamic object is well represented by its mask centroid plus median depth (Eq. 5).
    Single-point object model fails for extended objects and part-level motion, which the paper itself lists as a current limitation (Fig. 9b).
  • standard math Official benchmark protocols and ground truths of Dyn-Bench, VLM4D, and DSI-Bench are acceptable yardsticks.
    All accuracy numbers rest on them; for Dyn-Bench this is an author-built benchmark with overlapping authorship (flagged separately).
  • domain assumption SAM3 with text prompts and point seeds yields temporally consistent masks across all frames (Eqs. 4, 20-21).
    Tracking failure would corrupt trajectories; occlusion robustness is only tested on 75 manually selected videos.

pith-pipeline@v1.3.0-alltime-deepseek · 44638 in / 16488 out tokens · 164614 ms · 2026-08-02T06:28:12.718587+00:00 · methodology

0 comments
read the original abstract

4D spatio-temporal reasoning, jointly modeling 3D spatial structure and temporal evolution, is essential for understanding dynamic worlds and enabling embodied interaction. While current Multimodal Large Language Models (MLLMs) show strong capabilities in static scene understanding and coarse-grained 4D tasks, they still have notable limitations in continuous dynamic scene perception, especially in tracking dynamic object evidence for coherent 4D spatio-temporal reasoning. This shortcoming stems mainly from relying on sparse frame-level observations, fragmenting continuous dynamic cues and leaving models unable to disentangle genuine object dynamics from camera-induced apparent motion. Inspired by humans tracking dynamic cues while compensating for viewpoint changes, we propose DynTrace, a training-free framework for 4D spatio-temporal reasoning with two complementary components. Dynamic Trajectory Visualization (DTV) reprojects world-coordinate trajectories onto the image plane, providing geometry-informed visual priors that disentangle genuine object dynamics from camera-induced apparent motion. Meanwhile, the Dynamic Trace Token (DT-Token), organized into a Dynamic Trace Graph (DTG), tracks object-level dynamic cues, trace evolution, and key moments, maintaining continuous dynamic object evidence for coherent 4D reasoning. Together, these two components equip MLLMs with continuously tracked dynamic object evidence, grounded in geometry-informed visual priors and structured spatio-temporal traces. DynTrace consistently improves open-source MLLMs, achieving state-of-the-art results on Dyn-Bench, VLM4D, and DSI-Bench, validating the importance of tracking dynamic object evidence for robust 4D spatio-temporal reasoning.

Figures

Figures reproduced from arXiv: 2607.12503 by Chu Li, Dongxuan Liu, Jie Wu, Jingyan Jiang, Rongxin Gao, Shuzhao Xie, Xiaotong Tu, Xinghao Ding, Yue Huang, Yuzhi Huang, Zhenye Wang.

Figure 1
Figure 1. Figure 1: Failure modes of MLLMs in 4D spatio-temporal reasoning. Uniform sampling yields fragmented key frames that weaken dy [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of DynTrace. Given a video and a language query, DynTrace follows three stages: (1) Dynamic Objects Extraction [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Qualitative results of DynTrace on two representative cases. The figure shows how DynTrace changes the evidence path for [PITH_FULL_IMAGE:figures/full_fig_p008_3.png] view at source ↗
Figure 5
Figure 5. Figure 5: Performance on the Dynamic Source Confusion and Dy [PITH_FULL_IMAGE:figures/full_fig_p009_5.png] view at source ↗
Figure 4
Figure 4. Figure 4: Effect of frame sampling on Dyn-Bench and VLM4D. [PITH_FULL_IMAGE:figures/full_fig_p009_4.png] view at source ↗
Figure 7
Figure 7. Figure 7: Example of serialized DTG. The final stage serializes the window-level graph se￾quence G into textual DTG evidence Ge, which contains a global camera block and window-aligned object and rela￾tion blocks. An example of this representation is shown in [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Failure-mode analysis on VLM4D and DSI-Bench. [PITH_FULL_IMAGE:figures/full_fig_p017_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Representative limitation cases of DynTrace. [PITH_FULL_IMAGE:figures/full_fig_p018_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Complete DTV and DTG visualization for Case 1 (bmx-trees). [PITH_FULL_IMAGE:figures/full_fig_p020_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Complete DTV and DTG visualization for Case 2 (mallard-fly). [PITH_FULL_IMAGE:figures/full_fig_p021_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Complete DTV and DTG visualization for Case 3 (crossing). [PITH_FULL_IMAGE:figures/full_fig_p022_12.png] view at source ↗
Figure 13
Figure 13. Figure 13: Attention comparison between the baseline and DynTrace on representative [PITH_FULL_IMAGE:figures/full_fig_p023_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Additional baseline-versus-DynTrace comparisons on viewpoint-sensitive motion and metric reasoning questions. [PITH_FULL_IMAGE:figures/full_fig_p024_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Additional baseline-versus-DynTrace comparisons on trajectory change and interaction distance reasoning questions. [PITH_FULL_IMAGE:figures/full_fig_p024_15.png] view at source ↗
Figure 16
Figure 16. Figure 16: Prompt template for query-guided entity discovery. [PITH_FULL_IMAGE:figures/full_fig_p025_16.png] view at source ↗
Figure 17
Figure 17. Figure 17: Prompt template for multi-frame box verification and retained-box selection. [PITH_FULL_IMAGE:figures/full_fig_p026_17.png] view at source ↗
Figure 18
Figure 18. Figure 18: Prompt template for Representation Integration & Reasoning. [PITH_FULL_IMAGE:figures/full_fig_p027_18.png] view at source ↗
Figure 19
Figure 19. Figure 19: Prompt template for the 15-rule failure-mode selection. [PITH_FULL_IMAGE:figures/full_fig_p028_19.png] view at source ↗

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

67 extracted references · 18 linked inside Pith

  1. [1]

    Llava-onevision-1.5: Fully open framework for de- mocratized multimodal training

    Xiang An, Yin Xie, Kaicheng Yang, Wenkang Zhang, Xiuwei Zhao, Zheng Cheng, Yirui Wang, Songcen Xu, Changrui Chen, Chunsheng Wu, Huajie Tan, Chunyuan Li, Jing Yang, Jie Yu, Xiyao Wang, Bin Qin, Yumeng Wang, Zizhen Yan, Ziyong Feng, Ziwei Liu, Bo Li, and Jiankang Deng. Llava-onevision-1.5: Fully open framework for de- mocratized multimodal training. InarXiv...

    2025

  2. [2]

    Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025

    Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhao- hai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Jun- yang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shix...

    Pith/arXiv arXiv 2025

  3. [3]

    Spatialthinker: Reinforcing 3d rea- soning in multimodal llms via spatial rewards, 2025

    Hunar Batra, Haoqin Tu, Hardy Chen, Yuanze Lin, Cihang Xie, and Ronald Clark. Spatialthinker: Reinforcing 3d rea- soning in multimodal llms via spatial rewards, 2025. 2, 5, 16

    2025

  4. [4]

    Sam 3: Segment anything with concepts, 2025

    Nicolas Carion, Laura Gustafson, Yuan-Ting Hu, Shoub- hik Debnath, Ronghang Hu, Didac Suris, Chaitanya Ryali, Kalyan Vasudev Alwala, Haitham Khedr, Andrew Huang, Jie Lei, Tengyu Ma, Baishan Guo, Arpit Kalla, Markus Marks, Joseph Greer, Meng Wang, Peize Sun, Roman R¨adle, Triantafyllos Afouras, Effrosyni Mavroudi, Kather- ine Xu, Tsung-Han Wu, Yu Zhou, Lil...

    2025

  5. [5]

    Exploring embodied multimodal large models: Development, datasets, and future directions.Inf

    Shoubin Chen, Zehao Wu, Kai Zhang, Chunyu Li, Baiyang Zhang, Fei Ma, Fei Richard Yu, and Qingquan Li. Exploring embodied multimodal large models: Development, datasets, and future directions.Inf. Fusion, 122(C), 2025. 1

    2025

  6. [6]

    Toward embodied intelligence-enabled human-robot symbiotic manufacturing: A large language model-based perspective.J

    Wenhang Dong, Shufei Li, and Pai Zheng. Toward embodied intelligence-enabled human-robot symbiotic manufacturing: A large language model-based perspective.J. Comput. Inf. Sci. Eng., 25, 2025. 1

    2025

  7. [7]

    Tool-augmented spatiotemporal reasoning for streamlining video question answering task

    Sunqi Fan, Jiashuo Cui, Meng-Hao Guo, and Shuojin Yang. Tool-augmented spatiotemporal reasoning for streamlining video question answering task. InThe Thirty-ninth An- nual Conference on Neural Information Processing Systems,

  8. [8]

    Ma-lmm: Memory-augmented large multimodal model for long-term video understanding

    Bo He, Hengduo Li, Young Kyun Jang, Menglin Jia, Xue- fei Cao, Ashish Shah, Abhinav Shrivastava, and Ser-Nam Lim. Ma-lmm: Memory-augmented large multimodal model for long-term video understanding. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 13504–13514, 2024. 2

    2024

  9. [9]

    Vipe: Video pose engine for 3d geometric perception

    Jiahui Huang, Qunjie Zhou, Hesam Rabeti, Aleksandr Ko- rovko, Huan Ling, Xuanchi Ren, Tianchang Shen, Jun Gao, Dmitry Slepichev, Chen-Hsuan Lin, Jiawei Ren, Kevin Xie, Joydeep Biswas, Laura Leal-Taixe, and Sanja Fidler. Vipe: Video pose engine for 3d geometric perception. InNVIDIA Research Whitepapers arXiv:2508.10934, 2025. 2, 3

    Pith/arXiv arXiv 2025

  10. [10]

    Think- ing in dynamics: How multimodal large language models perceive, track, and reason dynamics in physical 4d world,

    Yuzhi Huang, Kairun Wen, Rongxin Gao, Dongxuan Liu, Yibin Lou, Jie Wu, Jing Xu, Jian Zhang, Zheng Yang, Yun- long Lin, Chenxin Li, Panwang Pan, Junbin Lu, Jingyan Jiang, Xinghao Ding, Yue Huang, and Zhi Wang. Think- ing in dynamics: How multimodal large language models perceive, track, and reason dynamics in physical 4d world,

  11. [11]

    Robostream: Weaving spatio-temporal reasoning with memory in vision-language models for robotics.arXiv preprint arXiv:2603.12939, 2026

    Yuzhi Huang, Jie Wu, Weijue Bu, Ziyi Xiong, Gaoyang Jiang, Ye Li, Kangye Ji, Shuzhao Xie, Yue Huang, Chenglei Wu, Jingyan Jiang, and Zhi Wang. Robostream: Weaving spatio-temporal reasoning with memory in vision-language models for robotics.arXiv preprint arXiv:2603.12939, 2026. 1, 2

    Pith/arXiv arXiv 2026

  12. [12]

    Storm: Token-efficient long video understanding for multimodal llms.arXiv preprint arXiv:2503.04130, 2025

    Jindong Jiang, Xiuyu Li, Zhijian Liu, Muyang Li, Guo Chen, Zhiqi Li, De-An Huang, Guilin Liu, Zhiding Yu, Kurt Keutzer, Sungjin Ahn, Jan Kautz, Hongxu Yin, Yao Lu, Song Han, and Wonmin Byeon. Storm: Token-efficient long video understanding for multimodal llms.arXiv preprint arXiv:2503.04130, 2025. 3

    arXiv 2025

  13. [13]

    Mosca: Dynamic gaussian fusion from casual videos via 4d motion scaffolds.arXiv preprint arXiv:2405.17421, 2024

    Jiahui Lei, Yijia Weng, Adam Harley, Leonidas Guibas, and Kostas Daniilidis. Mosca: Dynamic gaussian fusion from casual videos via 4d motion scaffolds.arXiv preprint arXiv:2405.17421, 2024. 1

    Pith/arXiv arXiv 2024

  14. [14]

    Llava-st: A multimodal large language model for fine-grained spatial- temporal understanding, 2025

    Hongyu Li, Jinyu Chen, Ziyu Wei, Shaofei Huang, Tian- rui Hui, Jialin Gao, Xiaoming Wei, and Si Liu. Llava-st: A multimodal large language model for fine-grained spatial- temporal understanding, 2025. 5, 16

    2025

  15. [15]

    See&trek: Training-free spatial prompting for multimodal large lan- guage model.arXiv preprint arXiv:2509.16087, 2025

    Pengteng Li, Pinhao Song, Wuyang Li, Weiyu Guo, Huizai Yao, Yijie Xu, Dugang Liu, and Hui Xiong. See&trek: Training-free spatial prompting for multimodal large lan- guage model.arXiv preprint arXiv:2509.16087, 2025. 5, 7, 16

    arXiv 2025

  16. [16]

    4k4dgen: Panoramic 4d generation at 4k resolution

    Renjie Li, Panwang Pan, Bangbang Yang, Dejia Xu, Shi- jie Zhou, Xuanyang Zhang, Zeming Li, Achuta Kadambi, Zhangyang Wang, Zhengzhong Tu, et al. 4k4dgen: Panoramic 4d generation at 4k resolution. InInternational Conference on Learning Representations (ICLR). ICLR,

  17. [17]

    Videochat- flash: Hierarchical compression for long-context video mod- eling.arXiv preprint arXiv:2501.00574, 2025

    Xinhao Li, Yi Wang, Jiashuo Yu, Xiangyu Zeng, Yuhan Zhu, Haian Huang, Jianfei Gao, Kunchang Li, Yinan He, Chent- ing Wang, Yu Qiao, Yali Wang, and Limin Wang. Videochat- flash: Hierarchical compression for long-context video mod- eling.arXiv preprint arXiv:2501.00574, 2025. 3

    Pith/arXiv arXiv 2025

  18. [18]

    Sti-bench: Are mllms ready for precise spatial-temporal world understanding? InProceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 5622–5632, 2025

    Yun Li, Yiming Zhang, Tao Lin, XiangRui Liu, Wenxiao Cai, Zheng Liu, and Bo Zhao. Sti-bench: Are mllms ready for precise spatial-temporal world understanding? InProceed- ings of the IEEE/CVF International Conference on Com- puter Vision, pages 5622–5632, 2025. 1

    2025

  19. [19]

    Movies: Motion-aware 4d dynamic view synthesis in one second

    Chenguo Lin, Yuchen Lin, Panwang Pan, Yifan Yu, Hon- glei Yan, Katerina Fragkiadaki, and Yadong Mu. Movies: Motion-aware 4d dynamic view synthesis in one second. In The IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2026. 3

    2026

  20. [20]

    Chen, Zhenyu Li, Guang Shi, Jiashi Feng, and Bingyi Kang

    Haotong Lin, Sili Chen, Jun Hao Liew, Donny Y . Chen, Zhenyu Li, Guang Shi, Jiashi Feng, and Bingyi Kang. Depth anything 3: Recovering the visual space from any views. arXiv preprint arXiv:2511.10647, 2025. 2, 4

    Pith/arXiv arXiv 2025

  21. [21]

    Mmsi-video-bench: A holistic bench- mark for video-based spatial intelligence.arXiv preprint arXiv:2512.10863, 2025

    Jingli Lin, Runsen Xu, Shaohao Zhu, Sihan Yang, Peizhou Cao, Yunlong Ran, Miao Hu, Chenming Zhu, Yiman Xie, Yilin Long, et al. Mmsi-video-bench: A holistic bench- mark for video-based spatial intelligence.arXiv preprint arXiv:2512.10863, 2025. 1

    arXiv 2025

  22. [22]

    Na vid-4d: Unleashing spatial intelligence in egocentric rgb-d videos for vision-and- language navigation

    Haoran Liu, Weikang Wan, Xiqian Yu, Minghan Li, Ji- azhao Zhang, Bo Zhao, Zhibo Chen, Zhongyuan Wang, Zhizheng Zhang, and He Wang. Na vid-4d: Unleashing spatial intelligence in egocentric rgb-d videos for vision-and- language navigation. In2025 IEEE International Conference on Robotics and Automation (ICRA), pages 10607–10615,

  23. [23]

    Bolt: Boost large vision-language model without training for long-form video understanding

    Shuming Liu, Chen Zhao, Tianqi Xu, and Bernard Ghanem. Bolt: Boost large vision-language model without training for long-form video understanding. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025. 2, 3

    2025

  24. [24]

    Spatial-ssrl: Enhancing spatial understanding via self-supervised reinforcement learning.arXiv preprint arXiv:2510.27606, 2025

    Yuhong Liu, Beichen Zhang, Yuhang Zang, Yuhang Cao, Long Xing, Xiaoyi Dong, Haodong Duan, Dahua Lin, and Jiaqi Wang. Spatial-ssrl: Enhancing spatial understanding via self-supervised reinforcement learning.arXiv preprint arXiv:2510.27606, 2025. 2, 5, 7, 16

    arXiv 2025

  25. [25]

    Llava-mr: Large language-and-vision assistant for video moment retrieval, 2024

    Weiheng Lu, Jian Li, An Yu, Ming-Ching Chang, Sheng- peng Ji, and Min Xia. Llava-mr: Large language-and-vision assistant for video moment retrieval, 2024. 3

    2024

  26. [26]

    Dynamic 3d gaussians: Tracking by per- sistent dynamic view synthesis

    Jonathon Luiten, Georgios Kopanas, Bastian Leibe, and Deva Ramanan. Dynamic 3d gaussians: Tracking by per- sistent dynamic view synthesis. In3DV, 2024. 1

    2024

  27. [27]

    Spatialreasoner: Towards explicit and generalizable 3d spatial reasoning

    Wufei Ma, Yu-Cheng Chou, Qihao Liu, Xingrui Wang, Celso M de Melo, Jianwen Xie, and Alan Yuille. Spatialreasoner: Towards explicit and generalizable 3d spatial reasoning. In Advances in Neural Information Processing Systems, 2025. 5, 16

    2025

  28. [28]

    Large language models and 3d vision for intelligent robotic perception and autonomy.Sensors (Basel, Switzerland), 25,

    Vinit Mehta, Charu Sharma, and Karthick Thiyagarajan. Large language models and 3d vision for intelligent robotic perception and autonomy.Sensors (Basel, Switzerland), 25,

  29. [29]

    Open-o3 video: Grounded video reasoning with explicit spatio-temporal evi- dence.arXiv preprint arXiv:2510.20579, 2025

    Jiahao Meng, Xiangtai Li, Haochen Wang, Yue Tan, Tao Zhang, Lingdong Kong, Yunhai Tong, Anran Wang, Zhiyang Teng, Yujing Wang, and Zhuochen Wang. Open-o3 video: Grounded video reasoning with explicit spatio-temporal evi- dence.arXiv preprint arXiv:2510.20579, 2025. 1

    arXiv 2025

  30. [30]

    Spacer: Rein- forcing mllms in video spatial reasoning.arXiv preprint arXiv:2504.01805, 2025

    Kun Ouyang, Yuanxin Liu, Haoning Wu, Yi Liu, Hao Zhou, Jie Zhou, Fandong Meng, and Xu Sun. Spacer: Rein- forcing mllms in video spatial reasoning.arXiv preprint arXiv:2504.01805, 2025. 2, 5, 7, 16

    Pith/arXiv arXiv 2025

  31. [31]

    Diff4splat: Controllable 4d scene generation with latent dynamic reconstruction models

    Panwang Pan, Chenguo Lin, Jingjing Zhao, Chenxin Li, Yuchen Lin, Haopeng Li, Honglei Yan, Kairun Wen, Yun- long Lin, Yixuan Yuan, et al. Diff4splat: Controllable 4d scene generation with latent dynamic reconstruction models. arXiv preprint arXiv:2511.00503, 2025. 3

    Pith/arXiv arXiv 2025

  32. [32]

    Kim, Bilge Soran, Raghuraman Krishnamoor- thi, Mohamed Elhoseiny, and Vikas Chandra

    Xiaoqian Shen, Yunyang Xiong, Changsheng Zhao, Lemeng Wu, Jun Chen, Chenchen Zhu, Zechun Liu, Fanyi Xiao, Bal- akrishnan Varadarajan, Florian Bordes, Zhuang Liu, Hu Xu, Hyunwoo J. Kim, Bilge Soran, Raghuraman Krishnamoor- thi, Mohamed Elhoseiny, and Vikas Chandra. Longvu: Spa- tiotemporal adaptive compression for long video-language understanding.arXiv pr...

    Pith/arXiv arXiv 2024

  33. [33]

    Corso, and Eric Sax.r 4: Retrieval-augmented reasoning for vision- language models in 4d spatio-temporal space.arXiv preprint arXiv:2512.15940, 2025

    Tin Stribor Sohn, Maximilian Dillitzer, Jason J. Corso, and Eric Sax.r 4: Retrieval-augmented reasoning for vision- language models in 4d spatio-temporal space.arXiv preprint arXiv:2512.15940, 2025. 1, 3

    arXiv 2025

  34. [34]

    Corso, and Eric Sax

    Tin Stribor Sohn, Maximilian Dillitzer, Jason J. Corso, and Eric Sax. Snow: Spatio-temporal scene understanding with world knowledge for open-world embodied reasoning.arXiv preprint arXiv:2512.16461, 2025. 1, 3

    arXiv 2025

  35. [35]

    Moviechat: From dense token to sparse memory for long video understanding

    Enxin Song, Wenhao Chai, Guanhong Wang, Yucheng Zhang, Haoyang Zhou, Feiyang Wu, Haozhe Chi, Xun Guo, Tian Ye, Yanting Zhang, et al. Moviechat: From dense token to sparse memory for long video understanding. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 18221–18232, 2024. 2

    2024

  36. [36]

    Adaptive keyframe sampling for long video understanding.arXiv preprint arXiv:2502.21271, 2025

    Xi Tang, Jihao Qiu, Lingxi Xie, Yunjie Tian, Jianbin Jiao, and Qixiang Ye. Adaptive keyframe sampling for long video understanding.arXiv preprint arXiv:2502.21271, 2025. 2, 3

    Pith/arXiv arXiv 2025

  37. [37]

    Spatial4d-bench: A versa- tile 4d spatial intelligence benchmark.arXiv preprint arXiv:2601.00092, 2025

    Pan Wang, Yang Liu, Guile Wu, Eduardo R Corral-Soto, Chengjie Huang, Binbin Xu, Dongfeng Bai, Xu Yan, Yuan Ren, Xingxin Chen, et al. Spatial4d-bench: A versa- tile 4d spatial intelligence benchmark.arXiv preprint arXiv:2601.00092, 2025. 1

    arXiv 2025

  38. [38]

    Alvarez, Lei Zhang, and Zhiding Yu

    Shihao Wang, Guo Chen, De an Huang, Zhiqi Li, Minghan Li, Guilin Liu, Jose M. Alvarez, Lei Zhang, and Zhiding Yu. Videoitg: Multimodal video understanding with instructed temporal grounding, 2025. 3

    2025

  39. [39]

    Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency

    Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Sheng- long Ye, Jie Shao, et al. Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265, 2025. 1, 5, 6, 7, 16

    Pith/arXiv arXiv 2025

  40. [40]

    Waft: Warping-alone field trans- forms for optical flow.arXiv preprint arXiv:2506.21526,

    Yihan Wang and Jia Deng. Waft: Warping-alone field trans- forms for optical flow.arXiv preprint arXiv:2506.21526,

    arXiv

  41. [41]

    Internvideo2.5: Empower- 11 ing video mllms with long and rich context modeling.arXiv preprint arXiv:2501.12386, 2025

    Yi Wang, Xinhao Li, Ziang Yan, Yinan He, Jiashuo Yu, Xi- angyu Zeng, Chenting Wang, Changlian Ma, Haian Huang, Jianfei Gao, Min Dou, Kai Chen, Wenhai Wang, Yu Qiao, Yali Wang, and Limin Wang. Internvideo2.5: Empower- 11 ing video mllms with long and rich context modeling.arXiv preprint arXiv:2501.12386, 2025. 1, 3, 5, 7, 16

    Pith/arXiv arXiv 2025

  42. [42]

    Videotree: Adaptive tree-based video representation for llm reasoning on long videos.arxiv, 2024

    Ziyang Wang, Shoubin Yu, Elias Stengel-Eskin, Jaehong Yoon, Feng Cheng, Gedas Bertasius, and Mohit Bansal. Videotree: Adaptive tree-based video representation for llm reasoning on long videos.arxiv, 2024. 2, 3

    2024

  43. [43]

    Dynamicverse: A physically-aware multimodal framework for 4d world mod- eling, 2025

    Kairun Wen, Yuzhi Huang, Runyu Chen, Hui Zheng, Yun- long Lin, Panwang Pan, Chenxin Li, Wenyan Cong, Jian Zhang, Junbin Lu, Chenguo Lin, Dilin Wang, Zhicheng Yan, Hongyu Xu, Justin Theiss, Yue Huang, Xinghao Ding, Rakesh Ranjan, and Zhiwen Fan. Dynamicverse: A physically-aware multimodal framework for 4d world mod- eling, 2025. 1, 3

    2025

  44. [44]

    Spatial-mllm: Boosting mllm capabilities in visual-based spatial intelligence.arXiv preprint arXiv:2505.23747, 2025

    Diankun Wu, Fangfu Liu, Yi-Hsin Hung, and Yueqi Duan. Spatial-mllm: Boosting mllm capabilities in visual-based spatial intelligence.arXiv preprint arXiv:2505.23747, 2025. 2

    Pith/arXiv arXiv 2025

  45. [45]

    St- think: How multimodal large language models reason about 4d worlds from ego-centric videos

    Peiran Wu, Yunze Liu, Miao Liu, and Junxiao Shen. St- think: How multimodal large language models reason about 4d worlds from ego-centric videos. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 5174–5183, 2026. 1

    2026

  46. [46]

    St-vla: Enabling 4d-aware spatiotem- poral understanding for general robot manipulation.arXiv preprint arXiv:2603.13788, 2026

    You Wu, Zixuan Chen, Cunxu Ou, Wenxuan Wang, Wenbo Huang, Lin Cao, Yangtao Chen, Weichao Qiu, Xingyue Quan, Jieqi Shi, et al. St-vla: Enabling 4d-aware spatiotem- poral understanding for general robot manipulation.arXiv preprint arXiv:2603.13788, 2026. 1

    arXiv 2026

  47. [47]

    A training-free framework for long video understanding via video-query-options similarity

    Zhirong Wu, Xiaodong Wang, Langling Huang, Teng Xu, and Peixi Peng. A training-free framework for long video understanding via video-query-options similarity. InThe F ourteenth International Conference on Learning Represen- tations, 2026. 3

    2026

  48. [48]

    Qwen3-omni technical report.arXiv preprint arXiv:2509.17765, 2025

    Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, Yuanjun Lv, Yongqi Wang, Dake Guo, He Wang, Linhan Ma, Pei Zhang, Xinyu Zhang, Hongkun Hao, Zishan Guo, Baosong Yang, Bin Zhang, Ziyang Ma, Xipin Wei, Shuai Bai, Keqin Chen, Xuejing Liu, Peng Wang, Mingkun Yang, Dayiheng Liu, Xingzhang Ren, Bo ...

    Pith/arXiv arXiv 2025

  49. [49]

    Learn- ing to inference adaptively for multimodal large language models, 2025

    Zhuoyan Xu, Khoi Duc Nguyen, Preeti Mukherjee, Saurabh Bagchi, Somali Chaterji, Yingyu Liang, and Yin Li. Learn- ing to inference adaptively for multimodal large language models, 2025. 1

    2025

  50. [50]

    Yeh, Yu-Chiang Frank Wang, and Min-Hung Chen

    Chiao-An Yang, Ryo Hachiuma, Sifei Liu, Subhashree Rad- hakrishnan, Raymond A. Yeh, Yu-Chiang Frank Wang, and Min-Hung Chen. 4d-rgpt: Toward region-level 4d under- standing via perceptual distillation. InProc. CVPR, 2026. 1, 2, 3

    2026

  51. [51]

    Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie

    Jihan Yang, Shusheng Yang, Anjali W. Gupta, Rilyn Han, Li Fei-Fei, and Saining Xie. Thinking in Space: How Multi- modal Large Language Models See, Remember and Recall Spaces.arXiv preprint arXiv:2412.14171, 2024. 1

    Pith/arXiv arXiv 2024

  52. [52]

    Visual spatial tuning

    Rui Yang, Ziyu Zhu, Yanwei Li, Jingjia Huang, Shen Yan, Siyuan Zhou, Zhe Liu, Xiangtai Li, Shuangye Li, Wenqian Wang, Yi Lin, and Hengshuang Zhao. Visual spatial tuning. arXiv preprint arXiv:2511.05491, 2025. 5, 7, 16

    arXiv 2025

  53. [53]

    Mllm-4d: Towards visual-based spatial-temporal intelligence.arXiv preprint arXiv:2603.00515, 2026

    Xingyilang Yin, Chengzhengxu Li, Jiahao Chang, Chi- Man Pun, and Xiaodong Cun. Mllm-4d: Towards visual-based spatial-temporal intelligence.arXiv preprint arXiv:2603.00515, 2026. 2, 5, 7, 16

    arXiv 2026

  54. [54]

    Unicomp: Rethinking video compression through informational uniqueness, 2026

    Chao Yuan, Shimin Chen, Minliang Lin, Limeng Qiao, Guanglu Wan, and Lin Ma. Unicomp: Rethinking video compression through informational uniqueness, 2026. 3

    2026

  55. [55]

    Think with videos for agentic long-video understanding, 2025

    Huaying Yuan, Zheng Liu, Junjie Zhou, Hongjin Qian, Yan Shu, Nicu Sebe, Ji-Rong Wen, and Zhicheng Dou. Think with videos for agentic long-video understanding, 2025. 3

    2025

  56. [56]

    Videorefer suite: Advancing spatial- temporal object understanding with video llm

    Yuqian Yuan, Hang Zhang, Wentong Li, Zesen Cheng, Bo- qiang Zhang, Long Li, Xin Li, Deli Zhao, Wenqiao Zhang, Yueting Zhuang, et al. Videorefer suite: Advancing spatial- temporal object understanding with video llm. InProceed- ings of the Computer Vision and Pattern Recognition Con- ference, pages 18970–18980, 2025. 1, 5, 16

    2025

  57. [57]

    Pixelrefer: A unified framework for spatio-temporal object referring with arbitrary granularity.arXiv, 2025

    Yuqian Yuan, Wenqiao Zhang, Xin Li, Shihao Wang, Kehan Li, Wentong Li, Jun Xiao, Lei Zhang, and Beng Chin Ooi. Pixelrefer: A unified framework for spatio-temporal object referring with arbitrary granularity.arXiv, 2025. 1

    2025

  58. [58]

    Geometrically-constrained agent for spatial reasoning

    Chen Zeren, Lu Xiaoya, Zheng Zhijie, Li Pengrui, He Lehan, Zhou Yijin, Shao Jing, Zhuang Bohan, and Sheng Lu. Geometrically-constrained agent for spatial reasoning. arXiv preprint arXiv:2511.22659, 2025. 2, 3

    arXiv 2025

  59. [59]

    Videollama 3: Frontier multi- modal foundation models for image and video understand- ing.arXiv preprint arXiv:2501.13106, 2025

    Boqiang Zhang, Kehan Li, Zesen Cheng, Zhiqiang Hu, Yuqian Yuan, Guanzheng Chen, Sicong Leng, Yuming Jiang, Hang Zhang, Xin Li, Peng Jin, Wenqi Zhang, Fan Wang, Li- dong Bing, and Deli Zhao. Videollama 3: Frontier multi- modal foundation models for image and video understand- ing.arXiv preprint arXiv:2501.13106, 2025. 1, 5, 7, 16

    Pith/arXiv arXiv 2025

  60. [60]

    Dsi- bench: A benchmark for dynamic spatial intelligence, 2025

    Ziang Zhang, Zehan Wang, Guanghao Zhang, Weilong Dai, Yan Xia, Ziang Yan, Minjie Hong, and Zhou Zhao. Dsi- bench: A benchmark for dynamic spatial intelligence, 2025. 2, 6, 17

    2025

  61. [61]

    Learning from videos for 3d world: Enhancing mllms with 3d vision geometry priors.arXiv preprint arXiv:2505.24625,

    Duo Zheng, Shijia Huang, Yanyang Li, and Liwei Wang. Learning from videos for 3d world: Enhancing mllms with 3d vision geometry priors.arXiv preprint arXiv:2505.24625,

    arXiv

  62. [62]

    Llava-4d: Embedding spa- tiotemporal prompt into lmms for 4d scene understanding

    Hanyu Zhou and Gim Hee Lee. Llava-4d: Embedding spa- tiotemporal prompt into lmms for 4d scene understanding. arXiv preprint arXiv:2505.12253, 2025. 1, 3

    Pith/arXiv arXiv 2025

  63. [63]

    Learning to reason in 4d: Dynamic spatial understanding for vision language models, 2025

    Shengchao Zhou, Yuxin Chen, Yuying Ge, Wei Huang, Jiehong Lin, Ying Shan, and Xiaojuan Qi. Learning to reason in 4d: Dynamic spatial understanding for vision language models, 2025. 1, 3, 5, 7, 16

    2025

  64. [64]

    Feature4x: Bridging any monocular video to 4d agentic ai with versatile gaussian feature fields

    Shijie Zhou, Hui Ren, Yijia Weng, Shuwang Zhang, Zhen Wang, Dejia Xu, Zhiwen Fan, Suya You, Zhangyang Wang, Leonidas Guibas, et al. Feature4x: Bridging any monocular video to 4d agentic ai with versatile gaussian feature fields. InProceedings of the Computer Vision and Pattern Recogni- tion Conference, pages 14179–14190, 2025. 1

    2025

  65. [65]

    Vlm4d: To- wards spatiotemporal awareness in vision language models

    Shijie Zhou, Alexander Vilesov, Xuehai He, Ziyu Wan, Shuwang Zhang, Aditya Nagachandra, Di Chang, Dongdong Chen, Eric Xin Wang, and Achuta Kadambi. Vlm4d: To- wards spatiotemporal awareness in vision language models. 12 InProceedings of the IEEE/CVF international conference on computer vision, pages 8600–8612, 2025. 2, 3, 6, 17

    2025

  66. [66]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shen- glong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025. 1

    Pith/arXiv arXiv 2025

  67. [67]

    C u e s

    Wenxuan Zhu, Bing Li, Cheng Zheng, Jinjie Mai, Jun Chen, Letian Jiang, Abdullah Hamdi, Sara Rojas Martinez, Chia- Wen Lin, Mohamed Elhoseiny, and Bernard Ghanem. 4d- bench: Benchmarking multi-modal large language models for 4d object understanding, 2025. 1 13 This supplementary material complements the main pa- per”DynTrace: Tracking Dynamic Object Eviden...

    2025