Pith. sign in

REVIEW 3 major objections 5 minor 45 references

Video instruction following is best measured by decomposing prompts into dependency graphs and propagating failures along causal chains, which reveals that current models collapse on deep and late constraints.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-02 04:52 UTC pith:CGGXTFU5

load-bearing objection ST-DAG-based evaluation is a genuine step forward, but the headline depth-sensitivity result is partly an artifact of the short-circuit scoring rule. the 3 major comments →

arxiv 2607.13527 v1 pith:CGGXTFU5 submitted 2026-07-15 cs.CV

VGIF-Score: Interpretable and Diagnostic Evaluation of Spatio-Temporal Instruction Following in Video Generation

Songyu Xu , Xin Wang , Qiang Chen , Xinran Wang , Muxi Diao , Yuxuan Zhang , Kongming Liang , Rui Lin
show 1 more author
Zhanyu Ma
This is my paper
classification cs.CV
keywords video generation evaluationinstruction followingspatio-temporal dependency graphdependency-aware QAshort-circuit diagnosticsvideo generation benchmarkcausal reasoningVLM-based evaluation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper's central claim is that instruction following in video generation can be measured at the level of individual semantic constraints, and that doing so exposes failures that aggregate quality scores hide. To show this, the authors introduce VGIF-Score, which parses each prompt into a spatio-temporal directed acyclic graph (ST-DAG) of entities, actions, states, and causal relations; verifies each node through binary question-answering against the generated video; and propagates failures along dependency edges via a short-circuit rule. This objective branch is paired with a per-prompt auto-generated rubric for cinematography, visual purity, motion smoothness, and physics adherence, and both components are checked against human judgments on a 200-video sample. The framework is instantiated as VGIF-Bench, containing 223 long, dependency-rich prompts that yield about 4.3K fine-grained evaluation items, and applied to 14 video generation models. The paper's main empirical finding is that current models rarely follow causal constraints, and that accuracy collapses both with dependency depth (roughly 19 percentage points per level) and with constraint position in the prompt (a 6.7-fold drop from the first fifth to the last fifth).

Core claim

On its own terms, VGIF-Score establishes that long, compositional video instructions can be decomposed into a dependency graph and evaluated automatically, constraint by constraint, with results that align with human judgments: 96.3% agreement on the QA items, a rank correlation of 0.87 between the combined score and human overall ratings, and stronger human alignment for the combined score than for either branch alone. Applied to 14 proprietary and open-source models on more than 3,000 generated videos, the framework finds that the bottleneck in video generation is not visual fidelity but structured execution: causal-relation accuracy stays below about 4.2 for commercial models and near zer

What carries the argument

The load-bearing object is the Spatio-Temporal Directed Acyclic Graph (ST-DAG): each node is an atomic semantic unit—entity, attribute, location, action, state, or causal—and each directed edge is a dependency, either a compositional prerequisite or a causal consequence. The graph is turned into binary QA pairs, and per-node correctness is computed recursively: a node is credited only if the evaluator's answer is correct and the node's dependency expression (using AND/OR connectives over upstream nodes) is satisfied. This short-circuit rule is what converts a flat accuracy number into a failure-localization tool: once a prerequisite fails, every downstream node is automatically marked incorr

Load-bearing premise

A single proprietary vision-language model is assumed to answer the binary ST-DAG questions and rubric ratings accurately enough that the reported model rankings, the 6.7-fold position cliff, and the depth falloff reflect the generators rather than the evaluator's own biases or instability; the supporting human check covers 200 sampled videos, not the full benchmark.

What would settle it

Run the full 223-prompt, 4.3K-item protocol twice with two different strong vision-language evaluators, or with complete human annotation of all items, and compare per-dimension and per-model scores; if the 14-model ranking, the 6.7-fold position decline, or the depth falloff changes materially across evaluators, the reported failure modes are evaluator artifacts rather than generator properties.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • If the paper is right, evaluations of text-to-video models should report position- and depth-stratified accuracies, because aggregate scores mask a 6.7-fold recency cliff and a roughly 19-point-per-depth-level falloff.
  • Causal chains are the binding constraint: on the paper's numbers no tested model exceeds a causal QA score of 4.21, so progress in instruction following should be tracked on causal and state dimensions rather than entity or location accuracy.
  • A single failed prerequisite can invalidate all downstream constraints, so diagnostic dashboards built on short-circuit propagation can point to the exact gating edge that needs improvement in a model.
  • Combining objective QA with a per-prompt rubric correlates better with human overall judgment (0.87) than either branch alone, supporting the paper's dual-branch design as a standard evaluation protocol.
  • Benchmark prompts need deep dependency structures—VGIF-Bench averages 16.4 nodes and 17.7 edges per prompt—to separate models that merely render scenes from models that execute event sequences.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • An extension the paper leaves implicit: because the short-circuit rule zeroes every downstream node once an upstream gate fails, the score measures minimal achievable completion, not partial credit; a future variant could report both to distinguish 'never attempted' from 'attempted but wrong'.
  • The same protocol could be turned into a training signal: ST-DAG failure traces identify which prompt segments a generator ignores, suggesting a way to reweight late and deep clauses in reinforcement learning or reward modeling.
  • A direct stress test of the paper's position finding would shuffle clause order while keeping the dependency graph fixed; the paper reports position sensitivity but does not separate lexical recency from structural ordering, so a shuffle experiment could clarify the cause of the 6.7-fold decline.
  • The dependency-aware evaluation assumes the vision-language evaluator can perceive cause-effect in video; a supplementary test using temporally edited clips—removing a causal event while keeping the rest—would check whether the evaluator's causal answers truly track event presence.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes VGIF-Score, a framework for evaluating instruction following in text-to-video generation. Prompts are parsed by an LLM into a Spatio-Temporal Directed Acyclic Graph (ST-DAG), from which dependency-aware binary QA pairs are derived and answered by a VLM. A second subjective branch uses an instruction-conditioned AutoRubric to score cinematography, visual purity, motion smoothness, and physics adherence. The two branches are equally weighted into a final score. The authors instantiate the framework on VGIF-Bench (223 prompts, ~4.3K evaluation items), evaluate 14 proprietary and open-source video generation models, and report two systematic failure modes: weak causal instruction following and strong sensitivity to dependency depth and prompt position. Human validation on 200 videos indicates high agreement with the VLM evaluator.

Significance. If the claims were fully supported, VGIF-Score would be a valuable contribution: it moves video-generation evaluation from holistic, aggregate scoring toward fine-grained, diagnostic, and interpretable assessment, and it introduces a dependency-aware graph structure that is not present in prior benchmarks such as VBench, T2V-CompBench, or ChronoMagic. The explicit ST-DAG representation and the attempt to localize failures to specific dependency edges are genuinely useful design ideas. The benchmark construction pipeline, which combines LLM drafting with human verification, is also practical and scalable. However, the central diagnostic findings — depth sensitivity and, to a significant degree, the causal bottleneck — are confounded by the short-circuit propagation rule in Eq. (2), and the entire evaluation rests on a single proprietary VLM with only a limited human-validation sample. The interpretive value of the paper is therefore not yet established.

major comments (3)
  1. [Section 3.2, Eq. (2) and Section 5.4, Fig. 3b] The headline finding of 'strong sensitivity to dependency depth' is largely an artifact of the scoring rule. Eq. (2) defines c_i = 1[answer_i = a_i] AND eval(dep(i), {c_j}_{j<i}), so a node at depth d is marked incorrect whenever any ancestor is incorrect, even if the VLM correctly answers that node's own question. The 'accuracy at depth d' plotted in Fig. 3b is therefore the joint probability that the node and all its ancestors are correct, not a per-node capability estimate. Under any evaluator with per-question error rate epsilon > 0, accuracy is guaranteed to decline monotonically with depth. The observed depth 0→1 drop from 80.6% to 58.5% is close to 0.806^2 ≈ 0.65, consistent with pure propagation. The same mechanism inflates the causal bottleneck in Table 2, since causal nodes are typically downstream of prerequisite actions/states. To support the claimed diagnosis, the authors mu
  2. [Section 5.4, Fig. 3a] The prompt-position effect is confounded with node type and dependency depth. The paper states that accuracy drops from 67.9% in the first 20% of the prompt to 10.1% in the final 20%, and later that 'position sensitivity is most severe for semantically complex constraints' — but no control is reported for the fact that later positions likely contain more state, causal, and deeply dependent nodes. Without stratifying by node type and depth (or performing a regression with those covariates), the conclusion that position and depth are 'two orthogonal axes' is not established. The authors should report, e.g., position curves within each node type and within fixed depth bands.
  3. [Section 5.6 and Section 5.1] All benchmark conclusions, including all model rankings, error analyses, and structural findings, are produced by a single proprietary VLM (Gemini-3.1-Pro). The human validation uses only 200 videos, with no breakdown by model, category, depth, or question type, and no analysis of the VLM's error patterns (e.g., false positives vs. false negatives on QA, or ceiling/floor effects). The reported aggregate agreement (96.3%, κ=0.92) is encouraging, but it does not demonstrate that the VLM's judgments are reliable enough to support the specific depth/position/causal claims, which involve small differences at extreme ranges. The authors should provide per-model or per-stratum human agreement, a second evaluator cross-check, or at least an error analysis of the VLM on the validated sample.
minor comments (5)
  1. [Abstract and Introduction] Typo: 'depencey-depth' should be 'dependency-depth'.
  2. [Table 1] VGIF-Bench contains only 223 prompts, far fewer than several comparison benchmarks. The paper justifies this by structural complexity, but a discussion of potential selection bias and statistical power would be useful.
  3. [Table 4] The table reports 'VGIF-Score 0.71 0.83 0.89' without explicitly stating which columns are Spearman correlations and which are Pearson. It later says 'Spearman rank correlation' for rating-based scores, but the table caption should be self-contained.
  4. [Section 5.4, Fig. 3a] The text says the position effect is 'universal across all 14 VGMs', but only aggregate numbers and one example (PixVerse-V6) are shown. A per-model plot or supplementary table would support this strong claim.
  5. [Eq. (2)] The notation eval(dep(i), {c_j}_{j<i}) is explained in prose but would benefit from a formal definition of the Boolean evaluation function, especially for OR dependencies with multiple parents.

Circularity Check

2 steps flagged

Depth-sensitivity 'finding' is an artifact of the short-circuit scoring in Eq. (2); the causal bottleneck is partly forced by the same propagation rule. Otherwise the benchmark and evaluator setup are self-contained.

specific steps
  1. self definitional [Section 3.2, Eq. (2); Section 5.4 'Dependency depth' and Fig. 3b]
    "Formally, let âi be the VLM’s answer for question qi, and let dep(i) denote its dependency expression over upstream QA indices. The per-node correctness is defined recursively: ci = 1[âi = ai] ∧ eval(dep(i), {cj}j<i), (2) ... If the dependency expression evaluates to false, the node is marked incorrect regardless of its own answer, and this failure propagates to all downstream nodes along the dependency chain."

    By Eq. (2), ci includes the correctness of all ancestors; any upstream failure forces every downstream node to 0. The reported 'accuracy at dependency depth' in Fig. 3b is therefore the joint probability that a node and its entire dependency chain succeed, which is non-increasing in depth for any evaluator. The observed decay (80.6→58.5→36.4→15.1→5.6), including the depth 0→1 ratio close to 0.806^2, is a mechanical consequence of the propagation rule, not an independent measurement of VGM capability. The claim that VGIF-Score reveals 'strong sensitivity to dependency depth' thus restates the definition of Eq. (2) rather than an independent empirical discovery.

  2. self definitional [Section 5.2 'Causal reasoning bottleneck'; Section 3.2, Eq. (2)]
    "Importantly, causal failures are not isolated errors: once a triggering action or prerequisite state is missed, downstream state changes and causal outcomes often become impossible to realize. This explains why causal scores are substantially lower than entity or location scores, and motivates the dependency-aware short-circuit design."

    Causal and state nodes are, by construction, downstream of prerequisite action nodes in the ST-DAG. Under Eq. (2), any miss of an upstream trigger sets ci=0 for all descendants regardless of the VLM's own answer to the causal question. Since action accuracy is already low in Table 2 (about 8.95–35.57), the reported causal scores near 0–5 are partly forced by the same propagation rule, so the 'causal reasoning bottleneck' is not measured independently of the metric definition. The paper's own explanation invokes the short-circuit mechanism that created the numbers.

full rationale

Most of VGIF-Score is not circular: QA pairs and rubrics are generated from the prompt text by GPT-5.2 and human-verified, not fitted to the target videos; the VLM evaluator answers the same questions on generated videos; model rankings are contingent empirical outputs; and the 200-video human validation (Table 4) provides independent evidence for evaluator agreement. No load-bearing self-citation chain was found. The central circularity is confined to the structural diagnostics. Eq. (2) defines node correctness as own answer AND all ancestor answers, so the 'accuracy vs. dependency depth' curve in Fig. 3b is guaranteed to decline with depth by construction. The reported causal bottleneck is partly the same mechanism, because causal nodes are downstream of prerequisites and are zeroed whenever any ancestor fails. Thus the headline claim that VGIF-Score 'reveals' strong depth sensitivity is a restatement of the scoring rule, while the depth-0 and raw per-dimension numbers remain informative. Score 6 reflects partial, construction-level circularity in one of the two headline failure modes.

Axiom & Free-Parameter Ledger

2 free parameters · 3 axioms · 0 invented entities

The method introduces no physical entities. Its assumptions center on the reliability of a single VLM evaluator and the faithfulness of the ST-DAG representation. There are two equal-weight choices that are free parameters in the scoring formula and are not justified by sensitivity analysis.

free parameters (2)
  • Equal weights in final score = 0.5 objective + 0.5 subjective
    The final VGIF-Score is an unweighted average of objective and subjective branches (Eq. 5), chosen without justification or sensitivity analysis.
  • Equal weights across rubric dimensions = 0.25 each
    Cinematography, visual purity, motion smoothness, and physics adherence are equally weighted (Eq. 4), with no evidence these weights match human preferences.
axioms (3)
  • domain assumption Gemini-3.1-Pro correctly answers binary QA about video content and scores rubrics accurately.
    The entire evaluation pipeline depends on this one VLM evaluator; human validation on 200 videos is limited evidence.
  • domain assumption The ST-DAG built from a prompt captures the intended semantics well enough for QA evaluation.
    Graph construction is done by GPT-5.2 plus human verification, but small errors in graph structure alter the QA answers and short-circuit propagation.
  • domain assumption Depicted video content is unambiguous for binary QA (ground-truth answers are well-defined).
    Questions like 'does the glass shatter?' can be subjective if the video is ambiguous; the paper assumes annotators agree, with reported kappa 0.92 on a subset.

pith-pipeline@v1.3.0-alltime-deepseek · 11748 in / 3193 out tokens · 33453 ms · 2026-08-02T04:52:55.516743+00:00 · methodology

0 comments
read the original abstract

Recent video generation models (VGMs) have made substantial progress in visual fidelity, yet their ability to follow long, compositional instructions remains insufficiently evaluated. Existing evaluation protocols often rely on prompts that are short and semantically shallow, with limited atomic constraints and weak spatio-temporal dependencies. They also frequently depend on costly human evaluation or handcrafted vision pipelines, while providing little diagnostic insight into which instruction constraints succeed or fail. To address this gap, we propose VGIF-Score, a highly automated and interpretable framework for evaluating instruction following in video generation. VGIF-Score consists of two complementary components: an objective completion branch that parses prompts into a Spatio-Temporal Directed Acyclic Graph (ST-DAG) and performs dependency-aware QA with short-circuit diagnostics, and a subjective satisfaction branch that uses instruction-conditioned AutoRubric to assess cinematography, visual purity, motion smoothness, and physics adherence. Together, these components produce a unified score that captures both objective completion and perceptual satisfaction. We instantiate this framework on VGIF-Bench, a benchmark of 223 long, structurally entangled prompts paired with approximately 4.3K fine-grained evaluation items. Experiments on 14 proprietary and open-source VGMs across more than 3K generated videos show that VGIF-Score provides reliable, interpretable, and diagnostically useful evaluation of video generation instruction following. The code will be available at https://github.com/PRIS-CV/VGIF-SCORE.

Figures

Figures reproduced from arXiv: 2607.13527 by Kongming Liang, Muxi Diao, Qiang Chen, Rui Lin, Songyu Xu, Xinran Wang, Xin Wang, Yuxuan Zhang, Zhanyu Ma.

Figure 1
Figure 1. Figure 1: Overview of VGIF-Score. The framework evaluates spatio-temporal instruction following via objective QA-based scoring and subjective rubric-based assessment [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of VGIF-Bench. The figure presents (a) the hierarchical prompt taxonomy, (b) graph depth distribution, (c) ST-DAG node-type composition, and (d) multi-parent node distribution. These statistics illustrate the coverage and structural complexity of the benchmark. For a benchmark of M prompt-video pairs, the overall score is: VGIF-Score = 1 M X M m=1 SVGIF(pm, xm). (6) 4 VGIF-Bench VGIF-Bench is desi… view at source ↗
Figure 3
Figure 3. Figure 3: Structural factors governing instruction-following accuracy. [PITH_FULL_IMAGE:figures/full_fig_p010_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Dependency-aware causal chain diagnosis. [PITH_FULL_IMAGE:figures/full_fig_p011_4.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 16 linked inside Pith

  1. [1]

    ai, S., Teng, H., Jia, H., Sun, L., Li, L., Li, M., Tang, M., Han, S., Zhang, T., Zhang, W.Q., Luo, W., Kang, X., Sun, Y., Cao, Y., Huang, Y., Lin, Y., Fang, Y., Tao, Z., Zhang, Z., Wang, Z., Liu, Z., Shi, D., Su, G., Sun, H., Pan, H., Wang, J., Sheng, J., Cui, M., Hu, M., Yan, M., Yin, S., Zhang, S., Liu, T., Yin, X., Yang, X., Song, X., Hu, X., Zhang, Y...

    Pith/arXiv arXiv 2025

  2. [2]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition

    Chen, J., Zhou, Z., Tong, Y., Chang, D., Luo, Y., Ma, Z.: Seeing as experts do: A knowledge-augmented agent for open-set fine-grained visual understanding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition. pp. 41446–41455 (2026)

    2026

  3. [3]

    In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision

    Chen, Y., Guo, X., Shi, Z., Song, Z., Zhang, J.: T2vworldbench: A benchmark for evaluating world knowledge in text-to-video generation. In: Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. pp. 6474–6485 (2026)

    2026

  4. [4]

    In: The Twelfth International Conference on Learning Representations

    Cho, J., Hu, Y., Baldridge, J.M., Garg, R., Anderson, P., Krishna, R., Bansal, M., Pont-Tuset, J., Wang, S.: Davidsonian scene graph: Improving reliability in fine-grained evaluation for text-to-image generation. In: The Twelfth International Conference on Learning Representations

  5. [5]

    Educational and psycho- logical measurement20(1), 37–46 (1960)

    Cohen, J.: A coefficient of agreement for nominal scales. Educational and psycho- logical measurement20(1), 37–46 (1960)

    1960

  6. [6]

    arXiv preprint arXiv:2510.24717 (2025)

    Deng, H., Pan, T., Zhang, F., Liu, Y., Luo, Z., Cui, Y., Shen, C., Shan, S., Zhang, Z., Wang, X.: Uniform discrete diffusion with metric path for video generation. arXiv preprint arXiv:2510.24717 (2025)

    arXiv 2025

  7. [7]

    arXiv preprint arXiv:2406.08656 (2024)

    Feng, W., Li, J., Saxon, M., Fu, T.j., Chen, W., Wang, W.Y.: Tc-bench: Bench- marking temporal compositionality in text-to-video and image-to-video generation. arXiv preprint arXiv:2406.08656 (2024)

    Pith/arXiv arXiv 2024

  8. [8]

    IEEE Transactions on Image Processing35, 3395–3410 (2026)

    Gao, Y., Chang, D., Yu, B., Qin, H., Diao, M., Chen, L., Liang, K., Ma, Z.: Toward generalizable forgery detection and reasoning. IEEE Transactions on Image Processing35, 3395–3410 (2026)

    2026

  9. [9]

    In: BMVC

    Gao, Y., Lin, W., Xu, J., Xu, W., Chen, P.: Self-supervised adversarial training for robust face forgery detection. In: BMVC. p. 718 (2023)

    2023

  10. [10]

    Advances in neural in- formation processing systems27(2014)

    Goodfellow, I.J., Pouget-Abadie, J., Mirza, M., Xu, B., Warde-Farley, D., Ozair, S., Courville, A., Bengio, Y.: Generative adversarial nets. Advances in neural in- formation processing systems27(2014)

    2014

  11. [11]

    Google DeepMind: Gemini 3.1 pro (2026),https://deepmind.google/models/ model-cards/gemini-3-1-pro/

    2026

  12. [12]

    arXiv preprint arXiv:2601.03233 (2026)

    HaCohen, Y., Brazowski, B., Chiprut, N., Bitterman, Y., Kvochko, A., Berkowitz, A., Shalem, D., Lifschitz, D., Moshe, D., Porat, E., et al.: Ltx-2: Efficient joint audio-visual foundation model. arXiv preprint arXiv:2601.03233 (2026)

    Pith/arXiv arXiv 2026

  13. [13]

    In: Proceedings of the 2021 conference on empirical methods in natural language processing

    Hessel, J., Holtzman, A., Forbes, M., Le Bras, R., Choi, Y.: Clipscore: A reference- free evaluation metric for image captioning. In: Proceedings of the 2021 conference on empirical methods in natural language processing. pp. 7514–7528 (2021) 14 S. Xu et al

    2021

  14. [14]

    Advances in neural information processing systems35, 8633– 8646 (2022)

    Ho, J., Salimans, T., Gritsenko, A., Chan, W., Norouzi, M., Fleet, D.J.: Video diffusion models. Advances in neural information processing systems35, 8633– 8646 (2022)

    2022

  15. [15]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Huang, Z., He, Y., Yu, J., Zhang, F., Si, C., Jiang, Y., Zhang, Y., Wu, T., Jin, Q., Chanpaisit, N., et al.: Vbench: Comprehensive benchmark suite for video gener- ative models. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 21807–21818 (2024)

    2024

  16. [16]

    IEEE Transactions on Pattern Analysis and Machine Intelligence (2025)

    Huang, Z., Zhang, F., Xu, X., He, Y., Yu, J., Dong, Z., Ma, Q., Chanpaisit, N., Si, C., Jiang, Y., et al.: Vbench++: Comprehensive and versatile benchmark suite for video generative models. IEEE Transactions on Pattern Analysis and Machine Intelligence (2025)

    2025

  17. [17]

    arXiv preprint arXiv:1312.6114 (2013)

    Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)

    Pith/arXiv arXiv 2013

  18. [18]

    arXiv preprint arXiv:2406.13743 (2024)

    Li, B., Lin, Z., Pathak, D., Li, J., Fei, Y., Wu, K., Ling, T., Xia, X., Zhang, P., Neubig, G., et al.: Genai-bench: Evaluating and improving compositional text-to- visual generation. arXiv preprint arXiv:2406.13743 (2024)

    Pith/arXiv arXiv 2024

  19. [19]

    In: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Ling, X., Zhu, C., Wu, M., Li, H., Feng, X., Yang, C., Hao, A., Zhu, J., Wu, J., Chu, X.: Vmbench: A benchmark for perception-aligned video motion generation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. pp. 13087–13098 (2025)

    2025

  20. [20]

    Liu, J., Han, J., Yan, B., Wu, H., Zhu, F., Wang, X., Jiang, Y., Peng, B., Yuan, Z.: Infinitystar: Unified spacetime autoregressive modeling for visual generation (2025),https://arxiv.org/abs/2511.04675

    arXiv 2025

  21. [21]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition

    Liu, Y., Cun, X., Liu, X., Wang, X., Zhang, Y., Chen, H., Liu, Y., Zeng, T., Chan, R., Shan, Y.: Evalcrafter: Benchmarking and evaluating large video generation models. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. pp. 22139–22149 (2024)

    2024

  22. [22]

    OpenAI: Gpt-5.2.https://openai.com/index/introducing-gpt-5-2/(2025), large language model

    2025

  23. [23]

    In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition

    Qin, H., Chang, D., Gao, Y., Tan, Y., Chen, L., Ma, Z.: Increfa: Breaking the static wall of generative model attribution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 35405–35415 (2026)

    2026

  24. [24]

    arXiv preprint arXiv:2505.15217 (2025)

    Qin, H., Chang, D., Gao, Y., Yu, B., Chen, L., Ma, Z.: Multimodal conditional in- formation bottleneck for generalizable ai-generated image detection. arXiv preprint arXiv:2505.15217 (2025)

    Pith/arXiv arXiv 2025

  25. [25]

    Spearman, C.: The proof and measurement of association between two things. (1961)

    1961

  26. [26]

    In: Proceed- ings of the Computer Vision and Pattern Recognition Conference

    Sun, K., Huang, K., Liu, X., Wu, Y., Xu, Z., Li, Z., Liu, X.: T2v-compbench: A comprehensive benchmark for compositional text-to-video generation. In: Proceed- ings of the Computer Vision and Pattern Recognition Conference. pp. 8406–8416 (2025)

    2025

  27. [27]

    In: International Conference on Pattern Recognition

    Tang, Z., Wang, Z., Peng, B., Dong, J.: Clip-agiqa: Boosting the performance of ai-generated image quality assessment with clip. In: International Conference on Pattern Recognition. pp. 48–61. Springer (2024)

    2024

  28. [28]

    In: Proceedings of the AAAI Conference on Artificial Intelligence

    Tang, Z., Yang, S., Peng, B., Wang, Z., Dong, J.: Revisiting mllm based image quality assessment: Errors and remedy. In: Proceedings of the AAAI Conference on Artificial Intelligence. vol. 40, pp. 9475–9483 (2026)

    2026

  29. [29]

    arXiv preprint arXiv:2601.20305 (2026) VGIF-Score 15

    Tang, Z., Yang, S., Wang, Z., Peng, B., Li, Y., Dong, B., Dong, J.: Endogenous re- prompting: Self-evolving cognitive alignment for unified multimodal models. arXiv preprint arXiv:2601.20305 (2026) VGIF-Score 15

    arXiv 2026

  30. [30]

    Team, G.: Mochi 1.https://github.com/genmoai/models(2024)

    2024

  31. [31]

    arXiv preprint arXiv:2512.16776 (2025)

    Team, K., Chen, J., Ci, Y., Du, X., Feng, Z., Gai, K., Guo, S., Han, F., He, J., He, K., et al.: Kling-omni technical report. arXiv preprint arXiv:2512.16776 (2025)

    Pith/arXiv arXiv 2025

  32. [32]

    arXiv preprint arXiv:2510.22200 (2025)

    Team, M.L., Cai, X., Huang, Q., Kang, Z., Li, H., Liang, S., Ma, L., Ren, S., Wei, X., Xie, R., et al.: Longcat-video technical report. arXiv preprint arXiv:2510.22200 (2025)

    arXiv 2025

  33. [33]

    arXiv preprint arXiv:2502.01719 (2025)

    Tong, H., Wang, Z., Chen, Z., Ji, H., Qiu, S., Han, S., Geng, K., Xue, Z., Zhou, Y., Xia, P., et al.: Mj-video: Fine-grained benchmarking and rewarding video pref- erences in video generation. arXiv preprint arXiv:2502.01719 (2025)

    Pith/arXiv arXiv 2025

  34. [34]

    arXiv preprint arXiv:1812.01717 (2018)

    Unterthiner, T., Van Steenkiste, S., Kurach, K., Marinier, R., Michalski, M., Gelly, S.: Towards accurate generative models of video: A new metric & challenges. arXiv preprint arXiv:1812.01717 (2018)

    Pith/arXiv arXiv 2018

  35. [35]

    arXiv preprint arXiv:2503.20314 (2025)

    Wan, T., Wang, A., Ai, B., Wen, B., Mao, C., Xie, C.W., Chen, D., Yu, F., Zhao, H., Yang, J., et al.: Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314 (2025)

    Pith/arXiv arXiv 2025

  36. [36]

    arXiv preprint arXiv:2604.18518 (2026)

    Wang, J., Deng, H., Pan, T., Liu, Y., Wang, C., Zhang, F., Qi, Y., Wang, X.: Udm- grpo: Stable and efficient group relative policy optimization for uniform discrete diffusion models. arXiv preprint arXiv:2604.18518 (2026)

    Pith/arXiv arXiv 2026

  37. [37]

    Advances in Neural Information Processing Systems38(2026)

    Wang, X., Xu, S., Xiangxuan, S., Zhang, Y., Diao, M., Duan, X., Liang, K., Ma, Z., et al.: Cinetechbench: A benchmark for cinematographic technique understanding and generation. Advances in Neural Information Processing Systems38(2026)

    2026

  38. [38]

    Wang, X., Zhang, Y., Zhang, X., Yan, H., Diao, M., Xu, S., Yan, Z., Li, H., Liang, K., Ma, Z.: Detailverifybench: A benchmark for dense hallucination localization in long image captions (2026),https://arxiv.org/abs/2604.05623

    Pith/arXiv arXiv 2026

  39. [39]

    arXiv preprint arXiv:2511.18870 (2025)

    Wu, B., Zou, C., Li, C., Huang, D., Yang, F., Tan, H., Peng, J., Wu, J., Xiong, J., Jiang, J., et al.: Hunyuanvideo 1.5 technical report. arXiv preprint arXiv:2511.18870 (2025)

    Pith/arXiv arXiv 2025

  40. [40]

    arXiv preprint arXiv:2510.17314 (2025)

    Xie, L., Huang, S., Zhang, Z., Zou, A., Zhai, Y., Ren, D., Zhang, K., Hu, H., Liu, B., Chen, H., et al.: Auto-rubric: Learning from implicit weights to explicit rubrics for reward modeling. arXiv preprint arXiv:2510.17314 (2025)

    arXiv 2025

  41. [41]

    arXiv preprint arXiv:2104.10157 (2021)

    Yan, W., Zhang, Y., Abbeel, P., Srinivas, A.: Videogpt: Video generation using vq-vae and transformers. arXiv preprint arXiv:2104.10157 (2021)

    Pith/arXiv arXiv 2021

  42. [42]

    Yang, S., Zhong, H., Zhang, R., Zhao, X., Li, S., Zheng, K., Yang, X., Wang, Z., Tang, Z., Li, Y., Gu, B., Peng, Z., Huang, Y., Luo, M., Bo, Y., Feng, D., Zhang, Y., Ma, J., Wang, R., Zhang, L., Guo, Y., Guan, F., Agrawala, M., Fu, H., Zhao, A., Rao, A.: Evalverse: Pipeline-aware and expert-calibrated benchmarking for professional cinematic video generati...

    2026

  43. [43]

    arXiv preprint arXiv:2408.06072 (2024)

    Yang, Z., Teng, J., Zheng, W., Ding, M., Huang, S., Xu, J., Yang, Y., Hong, W., Zhang, X., Feng, G., et al.: Cogvideox: Text-to-video diffusion models with an expert transformer. arXiv preprint arXiv:2408.06072 (2024)

    Pith/arXiv arXiv 2024

  44. [44]

    Advances in Neural Information Processing Systems37, 21236–21270 (2024)

    Yuan, S., Huang, J., Xu, Y., Liu, Y., Zhang, S., Shi, Y., Zhu, R., Cheng, X., Luo, J., Yuan, L.: Chronomagic-bench: A benchmark for metamorphic evaluation of text-to-time-lapse video generation. Advances in Neural Information Processing Systems37, 21236–21270 (2024)

    2024

  45. [45]

    arXiv preprint arXiv:2503.21755 (2025)

    Zheng, D., Huang, Z., Liu, H., Zou, K., He, Y., Zhang, F., Gu, L., Zhang, Y., He, J., Zheng, W.S., et al.: Vbench-2.0: Advancing video generation benchmark suite for intrinsic faithfulness. arXiv preprint arXiv:2503.21755 (2025)

    Pith/arXiv arXiv 2025