Pith. sign in

REVIEW 4 major objections 6 minor 61 references

Interaction graphs, not per-object trajectories, can control multi-object video generation with far less training data.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-01 07:00 UTC pith:32ZJBOFP

load-bearing objection Well-engineered graph-conditioned video generation, but the headline numbers rest on a VLM-filtered test set that shares a pipeline with the training data. the 4 major comments →

arxiv 2607.21580 v1 pith:32ZJBOFP submitted 2026-07-23 cs.CV cs.AI

GraphVid: Interactive Graph-Controllable Video Generation

Vedant Shah , Onkar Susladkar , Tushar Prakash , Kiet Nguyen , Tianjio Yu , Adheesh Juvekar , Muntasir Waheed , Ismini Lourentzou This is my paper
classification cs.CV cs.AI
keywords controllable video generationscene graphsinteraction graphsgraph neural networksdiffusion transformersLoRAimage-to-video generationrelational video dynamics
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

GraphVid sets out to make controllable video generation as simple as editing a directed graph of interactions among the objects in a scene. Its central claim is that a structured interaction graph—nodes for entities, typed directed edges for relations such as push, pull, or hold—can replace per-object trajectory drawings or physics labels as the control signal for an image-to-video model. The paper shows that injecting these graphs into a frozen video diffusion transformer through an edge-aware graph network and lightweight LoRA adapters is enough to match or beat much larger motion-control systems while training on roughly 27,000 clips and only 0.6 billion trainable parameters. If true, this would make multi-object video control more intuitive, less data-hungry, and easier to interactively edit, and it points to interaction semantics as a reusable inductive bias for generative video models.

Core claim

The paper's core discovery is that a directed scene graph annotated with open-vocabulary interaction descriptors is a sufficient conditioning interface for guiding a pretrained video diffusion transformer. Objects detected in the first frame become graph nodes; user-specified or automatically extracted relations become directed, typed edges that carry textual semantic cues. An edge-aware graph isomorphism network (GINEConv) propagates these edge semantics through message passing, so each entity's representation reflects its relational role; a graph-to-token adapter then feeds these representations into the frozen transformer's attention, with LoRA adapters tailoring the attention to the new

What carries the argument

The central mechanism is the interaction graph plus the edge-aware graph reasoning module. Nodes carry fused visual, textual, and spatial embeddings of detected objects; directed edges carry textual descriptors of the intended interaction. A GINEConv network injects these edge attributes directly into message passing, yielding interaction-aware node embeddings; a projection MLP converts them into conditioning tokens for a frozen video diffusion transformer, and LoRA adapters in the attention layers allow the pretrained weights to absorb the graph conditioning without being updated. The same graph interface also converts user edits into updated conditioning tokens at inference.

Load-bearing premise

The load-bearing assumption is that the synthetic interaction graphs used for training and evaluation—generated by a vision-language model rather than by humans—are accurate and representative of the graphs a real user would draw; if they are noisy or the test subset is biased toward interaction-clean clips, the reported quality and controllability gains could shrink for genuine user edits.

What would settle it

Distribute the same source images to naive users, let them draw their own interaction graphs, generate videos, and have independent annotators judge whether the intended interaction actually occurs; then compare GraphVid's motion-accuracy and human-preference scores on these user graphs against the same metrics on the paper's synthetic-graph test set. If user-graph controllability drops to near the text-only baseline, the central claim of intuitive, precise graph control would be undermined.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Control becomes compositional: adding, removing, or changing an edge changes the motion of the connected objects, so a user can direct a scene by editing relations rather than drawing tracks.
  • If interaction graphs carry the control signal, video generation models may need far fewer supervised training samples, since relational structure supplies an inductive bias that low-level motion cues lack.
  • The approach appears backbone-agnostic: swapping the pretrained video transformer from a 2B to a 5B model yields similar quality, suggesting graph conditioning is a general interface rather than a quirk of one architecture.
  • The improved motion accuracy (lower endpoint error on interaction-centric tests) implies graph conditioning binds motion to the correct entities, reducing the drifting and desynchronization common in object-level trajectory control.
  • Fastest inference among compared methods (about 200 seconds per clip) makes interactive, graph-based control practical in a feedback loop.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • My inference: The graph interface could extend beyond video generation; the same relational conditioning may help text-to-video, 3D simulation, and embodied-AI planners treat interactions as first-class editable controls.
  • My inference: A natural test is whether the model generalizes to interaction verbs and graph topologies that never appear in GraphVid-Bench; the paper's compositionality argument predicts it will, but does not yet measure this.
  • My inference: The training curves rely on a vision-language model to synthesize graphs from captions; if real user edits have different edge vocabularies or sparsity, the reported controllability may not directly transfer until the pipeline is fine-tuned on human edits.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper proposes GraphVid, an image-to-video generation model that conditions generation on a directed interaction scene graph. Given an input frame, a detector and an MLLM build an initial scene graph; users edit edges to specify interactions; an edge-aware GINE message-passing module produces node tokens that are injected into a frozen LTX-Video DiT via LoRA adapters. The authors introduce GraphVid-Bench, a ~27K-clip dataset with machine-generated graph annotations, and report experiments on VLM-filtered MoveBench and DAVIS subsets. The headline claim is that GraphVid matches or outperforms trajectory- and physics-conditioned baselines (e.g., FID 17.02 vs 28.32 for Motion-I2V, EPE 2.9 vs 3.9) while using 0.6B trainable parameters and 27K training videos. Ablations cover backbone generalization, node capacity, LoRA rank, edge semantics, and graph representation.

Significance. If the central claim is established, graph-structured semantic conditioning is a genuinely promising interface for controllable multi-object video generation: it is parameter-efficient, keeps the backbone frozen, and the paper provides a substantial dataset and detailed curation prompts. The paper has several strengths: the edge-aware GNN formulation is simple and plausible, the backbone generalization experiment (LTX vs Wan 2.2) is a useful sanity check, the ablations are informative, and the failure-case analysis is candid. However, the headline comparative results rest on an evaluation protocol that is partially closed around the same VLM family used to create the training signal, and no uncertainties or held-out validation are reported. The contribution is therefore significant only if the evaluation concerns can be addressed; the method itself is not invalidated by them.

major comments (4)
  1. [Appendix A.1; Tables 2, 3, 8] The headline comparisons are computed on VLM-filtered subsets. Appendix A.1 states that MoveBench videos are filtered by Qwen3-VL to retain only clips with discrete object-level physical interactions, and the DAVIS subset receives the same filtering. The same model family (Qwen3-VL) is used to generate the training graph edits (Appendix A, 'Synthetic Graph Update') and to build the conditioning graphs. This creates a distributional alignment between GraphVid's training data and the evaluation subsets that may not hold for trajectory-based baselines, whose training distributions are different. No results are reported on unfiltered MoveBench/DAVIS, and no confidence intervals or significance tests are given. The claimed FID 28.32→17.02 and FVD 159.32→99.42 over Motion-I2V in Table 2 cannot yet be interpreted as a general quality advantage; an unfiltered evaluation, or at least per-category
  2. [§5.6, Tables 5 and 7] Two key hyperparameters — maximum node capacity N_max=30 and LoRA rank=128 — are selected by ablations on the same filtered MoveBench subset used for the main comparisons. There is no separate validation split. The FID difference between N_max=30 and N_max=50 in Table 5 is only 0.15, so the choice may be tuning noise; Table 7 shows monotonic improvement with rank, so selecting the endpoint on the evaluation set is fragile. The reported efficiency/quality advantage needs a validation protocol that does not reuse the test set for hyperparameter selection.
  3. [Appendix A; Section F; Appendix D] Training graph edits are synthesized by prompting Qwen3-VL with ground-truth captions ('Synthetic Graph Update'), and there is no human validation of edge direction, label, or physics attributes. Section F documents that the detector can fail to isolate small objects, causing incorrect interaction assignment. This means the training signal may be noisy in ways that are not quantified. The controllability claim for genuine user edits is supported only by a small human preference study (10 scenarios, 60 participants, no significance testing). At minimum, the paper should report annotation-quality statistics (e.g., human agreement on a sample) and the distribution of graph edits, and should discuss how annotation noise affects the results.
  4. [Tables 2, 3, 8] No measure of uncertainty is reported for any metric. FVD and FID are known to have high variance on small samples; the filtered MoveBench subset appears to contain only a few hundred clips, judging from Figure 7. Without confidence intervals or multiple seeds, differences such as FID 17.02 vs 19.02 (GraphVid vs FlashMotion) or DAVIS FVD 98 vs 100 may not be significant. Report bootstrap confidence intervals and the exact number of videos in each evaluation subset.
minor comments (6)
  1. [Appendix B; Table 7] The LoRA scaling factor is stated as α=32, but the rank ablation in Table 7 does not state whether α was scaled with rank. If α was fixed, increasing rank also changes the effective update magnitude, confounding the interpretation. Please clarify.
  2. [Table 2; Figure 5] The 'Inference (s)' column lacks a footnote describing hardware, resolution, number of frames, and batch size for each method. Figure 5 is difficult to read without numerical labels for GraphVid's throughput.
  3. [Section 5.3] The phrase 'reduces FID by up to 39.9%' refers to a single baseline (Motion-I2V) but the sentence does not name it until later. Make the comparison explicit at the first use.
  4. [Appendix D; Figure 8] The aggregate panel in Figure 8 is confusing: '60% 30% 10%' is described as Wins/Ties/Losses, but the definitions of a tie and a loss in the aggregate are not fully specified. Clarify the aggregation procedure.
  5. [Section 4] Table 1 uses N=0 for no-interaction samples while the text elsewhere uses N for the number of interactions. Define the notation consistently to avoid ambiguity.
  6. [Section 6; Data Availability] The paper does not state whether GraphVid-Bench and the code will be released. This is important for reproducibility of the dataset curation pipeline; please add an explicit data/code availability statement.

Circularity Check

1 steps flagged

Headline gains rest on an interaction-centric test subset filtered by the same Qwen3-VL pipeline that generates GraphVid's training graphs; the relative comparisons are not fully external.

specific steps
  1. other [Appendix A and A.1; Tables 2/3 and Section 5.2]
    "We retain only videos classified as exhibiting discrete object interactions and discard sequences dominated by camera panning, background motion, or fluid/gaseous environmental effects. ... MoveBench: Videos are first filtered by the VLM to retain only clips exhibiting discrete object-level physical interactions."

    The same Qwen3-VL classifier is used to define the training distribution (discrete object interactions) and to distill the MoveBench/DAVIS test sets used for the headline quantitative claims. Because GraphVid's conditioning graphs at inference are also produced by Qwen3-VL (Inference-Time User Intent Translation), the model is both trained and evaluated inside a Qwen3-VL-selected distribution, while trajectory/physics baselines are not adapted to this filter. The reported 39.9% FID / 37.6% FVD gains over Motion-I2V are therefore not independent of the model's own annotation pipeline; an unfiltered or human-validated evaluation is needed. This is benchmark circularity, not a definitional tautology.

full rationale

The core architecture is not circular: the flow-matching loss (Eq. 2) is a standard training objective, the GINE message passing (Eq. 1) is an external formulation, and the frozen LTX backbone plus LoRA adaptation are standard components. No prediction is derived by construction from a fitted parameter, and no load-bearing self-citation chain is present. The circularity burden is concentrated in the evaluation protocol: GraphVid-Bench training graphs are synthesized by Qwen3-VL from ground-truth captions (Synthetic Graph Update), and the MoveBench/DAVIS evaluation subsets are filtered by the same Qwen3-VL motion classifier into interaction-centric clips. Since GraphVid is the only method whose conditioning modality is generated by that VLM and whose training distribution is defined by that filter, the quantitative comparison on Tables 2 and 3 is an in-family evaluation. Appendix F documents VLM grounding failures, confirming that the VLM-generated graphs are not ground truth. This does not invalidate the architectural contribution, but it means the headline efficiency/quality gains are partly an artifact of the closed-loop benchmark, justifying a borderline score.

Axiom & Free-Parameter Ledger

2 free parameters · 5 axioms · 0 invented entities

The central claim rests on two classes of unverified inputs: synthetic MLLM graph annotations without human validation, and a VLM-curated evaluation set. Two hyperparameters were tuned on the evaluation benchmark. No new physical entities are introduced.

free parameters (2)
  • Max node capacity N_max = 30
    Selected from Table 5 ablation on the evaluation benchmark; larger capacities degrade FVD. This is a hyperparameter chosen using test-set performance.
  • LoRA rank and alpha = r=128, alpha=32
    Selected from Table 7 ablation on the evaluation benchmark; rank 128 gives the best FID/FVD/PSNR/SSIM. Same test-set tuning issue.
axioms (5)
  • domain assumption GINEConv update (Eq. 1) with edge attributes captures interaction semantics needed for video control.
    The entire edge-aware reasoning claim rests on this graph operator and its chosen hyperparameters; no proof or independent validation is offered.
  • domain assumption Qwen3-VL object detection and Qwen3-Embedding edge text provide sufficient grounding for scene graphs.
    Node and edge features are taken from these models; Appendix F shows failures on small objects, so grounding is not guaranteed.
  • ad hoc to paper MLLM-generated graph edits from ground-truth captions are accurate training supervision.
    Appendix A uses the 'Synthetic Graph Update' prompt with no human validation of the generated edges.
  • ad hoc to paper VLM-filtered interaction-centric test subsets are representative of controllable generation quality.
    Appendix A.1 distills MoveBench and DAVIS with a Qwen3-VL classifier; the selection may favor graph-conditioned models and affect reported metrics.
  • domain assumption A frozen LTX-Video flow-matching DiT can be conditioned by graph tokens through LoRA without retraining the backbone.
    The method relies on the backbone's generative priors remaining intact; no analysis of capacity limits or failure modes is given.

pith-pipeline@v1.3.0-alltime-deepseek · 22177 in / 11535 out tokens · 109042 ms · 2026-08-01T07:00:17.753962+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of GraphVid: Interactive Graph-Controllable Video Generation." pith.science (2026). https://pith.science/paper/32ZJBOFP

@misc{pith2026260721580,
  author       = {Pith},
  title        = {Pith review of: GraphVid: Interactive Graph-Controllable Video Generation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/32ZJBOFP}},
  note         = {Machine review of arXiv:2607.21580}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Controllable video generation remains challenging due to the difficulty of specifying precise multi-object interactions using text prompts or motion-control inputs that primarily constrain pixel movement. In practice, trajectory-based control often requires users to draw accurate tracks for multiple objects, which scales poorly with scene complexity and becomes ambiguous under occlusion or overlap. To enable flexible yet precise multi-subject control, we introduce $\textbf{GraphVid}$, a graph-conditioned image-to-video generation model that enables interactive control through structured interaction graphs. We further curate $\textbf{GraphVid-Bench}$, a large-scale interaction-centric video dataset with structured relational annotations to enable training of interaction-aware video generation models. Despite using substantially less training data and fewer trainable parameters than prior motion-control methods, GraphVid delivers strong controllability and video quality. Compared with Motion-I2V, GraphVid reduces FID by up to 39.9% and FVD by 37.6%, while improving PSNR (9.87=>15.98) and SSIM (0.38=>0.61). Our results highlight the potential of structured semantic interfaces as a powerful paradigm for controllable video generation.

Figures

Figures reproduced from arXiv: 2607.21580 by Adheesh Juvekar, Ismini Lourentzou, Kiet Nguyen, Muntasir Waheed, Onkar Susladkar, Tianjio Yu, Tushar Prakash, Vedant Shah.

Figure 1
Figure 1. Figure 1: GraphVid enables controllable multi-object image-to-video genera [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: GraphVid Overview. From an input image, objects are detected and encoded into node embeddings, while an MLLM extracts relational edges to construct an initial interaction graph. Edge-aware graph reasoning converts node and edge representations into conditioning tokens that guide a frozen video diffusion transformer to generate interaction-consistent video dynamics. with an interaction-grounded graph and le… view at source ↗
Figure 3
Figure 3. Figure 3: GraphVid Edge-Aware Graph Reasoning. Node and edge embeddings are encoded into temporal latents and processed by our proposed edge-aware GNN that injects interaction semantics directly into message passing. The resulting graph embed￾dings capture relational dynamics between entities and are mapped into conditioning tokens that modulate a frozen diffusion video backbone via LoRA adapters. Node Representatio… view at source ↗
Figure 4
Figure 4. Figure 4: Interaction statistics in GraphVid-Bench dataset. (a) Frequency of interaction primitives, grouped into four categories (force & manipulation, kinematics & motion, physical contact, and support & position). (b) Interaction co-occurrence matrix captures interaction primitives appearing together within the same video se￾quences. The co-occurrence of manipulation and motion primitives shows that real￾world dy… view at source ↗
Figure 5
Figure 5. Figure 5: Model efficiency comparison. Trade-off between inference throughput and scale. Bubble size reflects model scale [PITH_FULL_IMAGE:figures/full_fig_p012_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative results of GraphVid for controllable interaction-based [PITH_FULL_IMAGE:figures/full_fig_p013_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Category distribution of the MoveBench dataset and the curated evaluation subset. The figure shows the number of samples per semantic category in the original MoveBench dataset (light bars) and the subset selected for evaluation in our experiments (dark bars). Categories are ordered by decreasing subset frequency. Our curated evaluation split preserves broad semantic coverage while reducing extreme long-ta… view at source ↗
Figure 8
Figure 8. Figure 8: User preference study across different dimensions [PITH_FULL_IMAGE:figures/full_fig_p024_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: Qualitative comparison on multi-object interaction control. [PITH_FULL_IMAGE:figures/full_fig_p025_9.png] view at source ↗
Figure 10
Figure 10. Figure 10: Qualitative comparison on interaction-driven video generation. [PITH_FULL_IMAGE:figures/full_fig_p026_10.png] view at source ↗
Figure 11
Figure 11. Figure 11: Failure case: semantic grounding error for small symbolic objects. [PITH_FULL_IMAGE:figures/full_fig_p027_11.png] view at source ↗
Figure 12
Figure 12. Figure 12: Additional qualitative results on diverse interaction scenarios. [PITH_FULL_IMAGE:figures/full_fig_p028_12.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

61 extracted references · 15 linked inside Pith

  1. [1]

    In: IEEE Conf

    Armeni, I., He, Z.Y., Gwak, J., Zamir, A.R., Fischer, M., Malik, J., Savarese, S.: 3d scene graph: A structure for unified semantics, 3d space, and camera. In: IEEE Conf. Comput. Vis. Pattern Recog. (2019) 16 V. Shah et al

    2019

  2. [2]

    arXiv preprint arXiv:2511.21631 (2025)

    Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., et al.: Qwen3-vl technical report. arXiv preprint arXiv:2511.21631 (2025)

    Pith/arXiv arXiv 2025

  3. [3]

    Bar, A., Herzig, R., Wang, X., Rohrbach, A., Chechik, G., Darrell, T., Globerson, A.: Compositional video synthesis with action graphs. In: Int. Conf. Mach. Learn. (2021)

    2021

  4. [4]

    In: SIGGRAPH Asia Conference Papers (2024)

    Bar-Tal, O., Chefer, H., Tov, O., Herrmann, C., Paiss, R., Zada, S., Ephrat, A., Hur, J., Liu, G., Raj, A., et al.: Lumiere: A space-time diffusion model for video generation. In: SIGGRAPH Asia Conference Papers (2024)

    2024

  5. [5]

    arXiv preprint arXiv:2311.15127 (2023)

    Blattmann, A., Dockhorn, T., Kulal, S., Mendelevitch, D., Kilian, M., Lorenz, D., Levi, Y., English, Z., Voleti, V., Letts, A., et al.: Stable video diffusion: Scaling latent video diffusion models to large datasets. arXiv preprint arXiv:2311.15127 (2023)

    Pith/arXiv arXiv 2023

  6. [6]

    In: Practice and Experience in Advanced Research Computing (PEARC) (2023)

    Boerner, T.J., Deems, S., Furlani, T.R., Knuth, S.L., Towns, J.: ACCESS: Advanc- ingInnovation:NSF’sAdvancedCyberinfrastructureCoordinationEcosystem:Ser- vices & Support. In: Practice and Experience in Advanced Research Computing (PEARC) (2023)

    2023

  7. [7]

    IEEE Transactions on Pattern Analysis and Machine Intelligence45(1), 1–26 (2021)

    Chang, X., Ren, P., Xu, P., Li, Z., Chen, X., Hauptmann, A.: A comprehensive survey of scene graphs: Generation and application. IEEE Transactions on Pattern Analysis and Machine Intelligence45(1), 1–26 (2021)

    2021

  8. [8]

    Chu, R., He, Y., Chen, Z., Zhang, S., Xu, X., Xia, B., WANG, D., Yi, H., Liu, X., Zhao, H., et al.: Wan-move: Motion-controllable video generation via latent trajectory guidance. In: Adv. Neural Inform. Process. Syst. (2025)

    2025

  9. [9]

    arXiv preprint arXiv:2506.08189 (2025)

    Dutta, A., Mehrab, K.S., Sawhney, M., Neog, A., Khurana, M., Fatemi, S., Prad- han, A., Maruf, M., Lourentzou, I., Daw, A., et al.: Open world scene graph gen- eration using vision language models. arXiv preprint arXiv:2506.08189 (2025)

    Pith/arXiv arXiv 2025

  10. [10]

    In: IEEE Conf

    Geng, D., Herrmann, C., Hur, J., Cole, F., Zhang, S., Pfaff, T., Lopez-Guevara, T., Aytar, Y., Rubinstein, M., Sun, C., et al.: Motion prompting: Controlling video generation with motion trajectories. In: IEEE Conf. Comput. Vis. Pattern Recog. (2025)

    2025

  11. [11]

    arXiv preprint arXiv:2307.10373 (2023)

    Geyer, M., Bar-Tal, O., Bagon, S., Dekel, T.: Tokenflow: Consistent diffusion fea- tures for consistent video editing. arXiv preprint arXiv:2307.10373 (2023)

    Pith/arXiv arXiv 2023

  12. [12]

    Gillman, N., Herrmann, C., Freeman, M., Aggarwal, D., Luo, E., Sun, D., Sun, C.: Force prompting: Video generation models can learn and generalize physics-based control signals. In: Adv. Neural Inform. Process. Syst. (2025)

    2025

  13. [13]

    something something

    Goyal, R., Ebrahimi Kahou, S., Michalski, V., Materzynska, J., Westphal, S., Kim, H., Haenel, V., Fruend, I., Yianilos, P., Mueller-Freitag, M., et al.: The" something something" video database for learning and evaluating visual common sense. In: IEEE Conf. Comput. Vis. Pattern Recog. (2017)

    2017

  14. [14]

    Guo, Y., Yang, C., Rao, A., Agrawala, M., Lin, D., Dai, B.: Sparsectrl: Adding sparse controls to text-to-video diffusion models. In: Eur. Conf. Comput. Vis. (2024)

    2024

  15. [15]

    Guo, Y., Yang, C., Rao, A., Liang, Z., Wang, Y., Qiao, Y., Agrawala, M., Lin, D., Dai, B.: Animatediff: Animate your personalized text-to-image diffusion models without specific tuning. Int. Conf. Learn. Represent. (2024)

    2024

  16. [16]

    arXiv preprint arXiv:2501.00103 (2024)

    HaCohen, Y., Chiprut, N., Brazowski, B., Shalem, D., Moshe, D., Richardson, E., Levin, E., Shiran, G., Zabari, N., Gordon, O., et al.: Ltx-video: Realtime video latent diffusion. arXiv preprint arXiv:2501.00103 (2024)

    Pith/arXiv arXiv 2024

  17. [17]

    Hamilton, W., Ying, Z., Leskovec, J.: Inductive representation learning on large graphs. Adv. Neural Inform. Process. Syst. (2017) GraphVid: Interactive Graph-Controllable Video Generation 17

    2017

  18. [18]

    arXiv preprint arXiv:2405.19334 (2024)

    He, Y., Liu, Z., Chen, J., Tian, Z., Liu, H., Chi, X., Liu, R., Yuan, R., Xing, Y., Wang, W., et al.: Llms meet multimodal generation and editing: A survey. arXiv preprint arXiv:2405.19334 (2024)

    Pith/arXiv arXiv 2024

  19. [19]

    Heusel,M.,Ramsauer,H.,Unterthiner,T.,Nessler,B.,Hochreiter,S.:Ganstrained by a two time-scale update rule converge to a local nash equilibrium. Adv. Neural Inform. Process. Syst. (2017)

    2017

  20. [20]

    In: AAAI (2024)

    Holla, M., Lourentzou, I.: Commonsense for zero-shot natural language video lo- calization. In: AAAI (2024)

    2024

  21. [21]

    Hu, E.J., Shen, Y., Wallis, P., Allen-Zhu, Z., Li, Y., Wang, S., Wang, L., Chen, W., et al.: Lora: Low-rank adaptation of large language models. Int. Conf. Learn. Represent. (2022)

    2022

  22. [22]

    Hu, Y., Chen, Z., Luo, C.: Lamd: Latent motion diffusion for image-conditional video generation. Int. J. Comput. Vis. (2025)

    2025

  23. [23]

    In: IEEE Conf

    Huang, Z., He, Y., Yu, J., Zhang, F., Si, C., Jiang, Y., Zhang, Y., Wu, T., Jin, Q., Chanpaisit, N., et al.: Vbench: Comprehensive benchmark suite for video genera- tive models. In: IEEE Conf. Comput. Vis. Pattern Recog. (2024)

    2024

  24. [24]

    Karaev, N., Rocco, I., Graham, B., Neverova, N., Vedaldi, A., Rupprecht, C.: Cotracker: It is better to track together. In: Eur. Conf. Comput. Vis. (2024)

    2024

  25. [25]

    In: International Conference on Medical Image Computing and Computer-Assisted Intervention (2022)

    Karwande, G., Mbakwe, A.B., Wu, J.T., Celi, L.A., Moradi, M., Lourentzou, I.: Chexrelnet: An anatomy-aware model for tracking longitudinal relationships be- tween chest x-rays. In: International Conference on Medical Image Computing and Computer-Assisted Intervention (2022)

    2022

  26. [26]

    Neurosymbolic Artificial Intelligence (2025)

    Khan, M.J., Ilievski, F., Breslin, J.G., Curry, E.: A survey of neurosymbolic vi- sual reasoning with scene graphs and common sense knowledge. Neurosymbolic Artificial Intelligence (2025)

    2025

  27. [27]

    arXiv preprint arXiv:1609.02907 (2016)

    Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)

    Pith/arXiv arXiv 2016

  28. [28]

    arXiv preprint arXiv:2403.14468 (2024)

    Ku, M., Wei, C., Ren, W., Yang, H., Chen, W.: Anyv2v: A tuning-free framework for any video-to-video editing tasks. arXiv preprint arXiv:2403.14468 (2024)

    Pith/arXiv arXiv 2024

  29. [29]

    In: IEEE Conf

    Lei, G., Wang, C., Zhang, R., Wang, Y., Li, H., Xu, W.: Animateanything: Con- sistent and controllable animation for video generation. In: IEEE Conf. Comput. Vis. Pattern Recog. (2025)

    2025

  30. [30]

    Neurocomputing (2024)

    Li, H., Zhu, G., Zhang, L., Jiang, Y., Dang, Y., Hou, H., Shen, P., Zhao, X., Shah, S.A.A., Bennamoun, M.: Scene graph generation: A comprehensive survey. Neurocomputing (2024)

    2024

  31. [31]

    In: IEEE Conf

    Li, Q., Xing, Z., Wang, R., Cao, H., Dai, Q., Dong, D., Wu, Z.: Flashmotion: Few-step controllable video generation with trajectory guidance. In: IEEE Conf. Comput. Vis. Pattern Recog. (2026)

    2026

  32. [32]

    Li, Q., Xing, Z., Wang, R., Zhang, H., Dai, Q., Wu, Z.: Magicmotion: Controllable videogenerationwithdense-to-sparsetrajectoryguidance.In:IEEEConf.Comput. Vis. Pattern Recog. (2025)

    2025

  33. [33]

    In: IEEE Conf

    Li, X., Ma, C., Yang, X., Yang, M.H.: Vidtome: Video token merging for zero-shot video editing. In: IEEE Conf. Comput. Vis. Pattern Recog. (2024)

    2024

  34. [34]

    In: AAAI (2025)

    Li, Y., Wang, X., Zhang, Z., Wang, Z., Yuan, Z., Xie, L., Shan, Y., Zou, Y.: Image conductor: Precision control for interactive video synthesis. In: AAAI (2025)

    2025

  35. [35]

    arXiv preprint arXiv:2412.00131 (2024)

    Lin, B., Ge, Y., Cheng, X., Li, Z., Zhu, B., Wang, S., He, X., Ye, Y., Yuan, S., Chen, L., et al.: Open-sora plan: Open-source large video generation model. arXiv preprint arXiv:2412.00131 (2024)

    Pith/arXiv arXiv 2024

  36. [36]

    Ogunleye, M.A., Abdelrahman, E., Lourentzou, I.: 3d-vcd: Hallucination mitiga- tionin3d-llmembodiedagentsthroughvisualcontrastivedecoding.In:IEEEConf. Comput. Vis. Pattern Recog. (2026) 18 V. Shah et al

    2026

  37. [37]

    In: IEEE Conf

    Peebles, W., Xie, S.: Scalable diffusion models with transformers. In: IEEE Conf. Comput. Vis. Pattern Recog. (2023)

    2023

  38. [38]

    arXiv preprint arXiv:1704.00675 (2017)

    Pont-Tuset, J., Perazzi, F., Caelles, S., Arbeláez, P., Sorkine-Hornung, A., Van Gool, L.: The 2017 davis challenge on video object segmentation. arXiv preprint arXiv:1704.00675 (2017)

    Pith/arXiv arXiv 2017

  39. [39]

    arXiv preprint arXiv:2510.02284 (2025)

    Romero, D., Bermudez, A., Li, H., Pizzati, F., Laptev, I.: Learning to generate rigid body interactions with video diffusion models. arXiv preprint arXiv:2510.02284 (2025)

    arXiv 2025

  40. [40]

    arXiv preprint arXiv:2603.20169 (2026)

    Shen, Y., Liu, J., Li, X., Liu, Y., Li, B., Yang, H., Jia, W., Li, Y., Yu, T., Rehg, J.M., Cao, X., Lourentzou, I.: Egoforge: Goal-directed egocentric world simulator. arXiv preprint arXiv:2603.20169 (2026)

    arXiv 2026

  41. [41]

    In: IEEE Conf

    Shen, Y., Xiong, J., Yu, T., Lourentzou, I.: Phantom: Physics-infused video gen- eration via joint modeling of visual and latent physical dynamics. In: IEEE Conf. Comput. Vis. Pattern Recog. (2026)

    2026

  42. [42]

    Shi, X., Huang, Z., Wang, F.Y., Bian, W., Li, D., Zhang, Y., Zhang, M., Cheung, K.C., See, S., Qin, H., et al.: Motion-i2v: Consistent and controllable image-to- videogenerationwithexplicitmotionmodeling.In:SIGGRAPHConferencePapers (2024)

    2024

  43. [43]

    arXiv preprint arXiv:2209.14792 (2022)

    Singer, U., Polyak, A., Hayes, T., Yin, X., An, J., Zhang, S., Hu, Q., Yang, H., Ashual, O., Gafni, O., et al.: Make-a-video: Text-to-video generation without text- video data. arXiv preprint arXiv:2209.14792 (2022)

    Pith/arXiv arXiv 2022

  44. [44]

    In: IEEE Conf

    Susladkar, O., Prakash, T., Juvekar, A., Nguyen, K.A., Jang, D.H., Dhillon, I.S., Lourentzou, I.: Pyratok: Language-aligned pyramidal tokenizer for video under- standing and generation. In: IEEE Conf. Comput. Vis. Pattern Recog. (2026)

    2026

  45. [45]

    Susladkar, O., Sen Gupta, J., Sehgal, C., Mittal, S., Singhal, R.: Motionaura: Gen- erating high-quality and motion consistent videos using discrete diffusion. In: Int. Conf. Learn. Represent. (2025)

    2025

  46. [46]

    Unterthiner, T., Van Steenkiste, S., Kurach, K., Marinier, R., Michalski, M., Gelly, S.: Fvd: A new metric for video generation. Int. Conf. Learn. Represent. Worksh. DeepGenStruct (2019)

    2019

  47. [47]

    In: IEEE Conf

    Wald, J., Dhamo, H., Navab, N., Tombari, F.: Learning 3d semantic scene graphs from 3d indoor reconstructions. In: IEEE Conf. Comput. Vis. Pattern Recog. (2020)

    2020

  48. [48]

    arXiv preprint arXiv:2503.20314 (2025)

    Wan, T., Wang, A., Ai, B., Wen, B., Mao, C., Xie, C.W., Chen, D., Yu, F., Zhao, H., Yang, J., et al.: Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314 (2025)

    Pith/arXiv arXiv 2025

  49. [49]

    Wang, J., Ma, A., Cao, K., Zheng, J., Feng, J., Zhang, Z., Pang, W., Liang, X.: Wisa: World simulator assistant for physics-aware text-to-video generation. In: Adv. Neural Inform. Process. Syst. (2025)

    2025

  50. [50]

    Wang, X., Yuan, H., Zhang, S., Chen, D., Wang, J., Zhang, Y., Shen, Y., Zhao, D., Zhou, J.: Videocomposer: Compositional video synthesis with motion control- lability. Adv. Neural Inform. Process. Syst. (2023)

    2023

  51. [51]

    IEEE Int

    Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Int. Conf. Image Process. (2004)

    2004

  52. [52]

    In: IEEE Conf

    Wei, Y., Zhang, S., Yuan, H., Gong, B., Tang, L., Wang, X., Qiu, H., Li, H., Tan, S., Zhang, Y., et al.: Dreamrelation: Relation-centric video customization. In: IEEE Conf. Comput. Vis. Pattern Recog. (2025)

    2025

  53. [53]

    Wu, W., Li, Z., Gu, Y., Zhao, R., He, Y., Zhang, D.J., Shou, M.Z., Li, Y., Gao, T., Zhang, D.: Draganything: Motion control for anything using entity representation. In: Eur. Conf. Comput. Vis. (2024) GraphVid: Interactive Graph-Controllable Video Generation 19

    2024

  54. [54]

    Xu, K., Hu, W., Leskovec, J., Jegelka, S.: How powerful are graph neural networks? arXiv preprint arXiv:1810.00826 (2018)

    Pith/arXiv arXiv 2018

  55. [55]

    Yang, X., He, C., Ma, J., Zhang, L.: Motion-guided latent diffusion for temporally consistent real-world video super-resolution. In: Eur. Conf. Comput. Vis. (2024)

    2024

  56. [56]

    Yang, Z., Teng, J., Zheng, W., Ding, M., Huang, S., Xu, J., Yang, Y., Hong, W., Zhang, X., Feng, G., et al.: Cogvideox: Text-to-video diffusion models with an expert transformer. In: Int. Conf. Learn. Represent. (2025)

    2025

  57. [57]

    Yin, S., Wu, C., Liang, J., Shi, J., Li, H., Ming, G., Duan, N.: Dragnuwa: Fine- grainedcontrolinvideogenerationbyintegratingtext,image,andtrajectory.arXiv preprint arXiv:2308.08089 (2023)

    Pith/arXiv arXiv 2023

  58. [58]

    In: IEEE Conf

    Zhang, L., Rao, A., Agrawala, M.: Adding conditional control to text-to-image diffusion models. In: IEEE Conf. Comput. Vis. Pattern Recog. (2023)

    2023

  59. [59]

    arXiv preprint arXiv:2506.05176 (2025)

    Zhang, Y., Li, M., Long, D., Zhang, X., Lin, H., Yang, B., Xie, P., Yang, A., Liu, D., Lin, J., et al.: Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176 (2025)

    Pith/arXiv arXiv 2025

  60. [60]

    Wins” denotes the percentage of times a model was chosen as the absolute best for a given metric, “Ties

    Zhang, Z., Liao, J., Li, M., Dai, Z., Qiu, B., Zhu, S., Qin, L., Wang, W.: Tora: Trajectory-oriented diffusion transformer for video generation. In: IEEE Conf. Comput. Vis. Pattern Recog. (2025) 20 V. Shah et al. AGraphVid-BenchDetails and Curation Pipeline DatasetProvenanceandAggregation.WeconstructourGraphVid-Bench training data from three complementary...

    2025

  61. [61]

    {caption}

    A caption describing the video: “{caption}” Your task is to isolate true, discrete object-to-object physics from camera-induced motion and ambient environmental motion. Motion Categories –discrete_object_physics: Solid, distinct objects (e.g., people, cars, bats, balls) moving due to explicit physical forces, collisions, or locomotion. The motion can be e...