Pith. sign in

REVIEW 3 major objections 5 minor 75 references

A learned manifold of real human motion lets two distance scores judge AI-generated actions and their temporal coherence more reliably than existing metrics or multimodal judges.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-03 19:06 UTC pith:YZFYLRCZ

load-bearing objection A solid evaluation method and benchmark undermined by an in-distribution S_cons test; the temporal metric shows real external signal, and the action-consistency claim needs held-out validation before it should be trusted. the 3 major comments →

arxiv 2512.01803 v3 pith:YZFYLRCZ submitted 2025-12-01 cs.CV

Generative Action Tell-Tales: Assessing Human Motion in Synthesized Videos

Xavier Thomas , Youngsun Lim , Ananya Srinivasan , Audrey Zheng , Deepti Ghadiyaram This is my paper
classification cs.CV
keywords action realismvideo generation evaluationhuman motion manifoldtemporal coherenceskeletal geometryreference-free metrichuman perceptiongenerated video benchmark
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper tries to establish that the realism of human actions in AI-generated videos can be scored by measuring distance from a compact latent manifold learned from real-world human motion. The manifold is built from per-frame 3D body shape, pose, and orientation, 2D joint keypoints, and visual features, plus their first-order temporal differences, trained so that real videos of the same action cluster tightly while temporally distorted videos are pushed away. From this manifold the paper derives two simple metrics — action consistency, the distance to a class centroid, and temporal coherence, the average jump between consecutive frame embeddings — and validates them on a new benchmark with thousands of human ratings. On that benchmark the two metrics reach rank correlations of about 0.61 and 0.64 with human judgments, beating the strongest multimodal-language-model judge by large margins, so a sympathetic reader would take the core claim as serious.

Core claim

The paper's central claim is that a video shows a 'real' human action exactly when its embedding lands close to where real videos of that action cluster, and moves through that space at a smooth, plausible pace. The authors demonstrate this by training a small transformer encoder on windows of real video using two objectives: a supervised contrastive loss that pulls same-action windows together, and a hard-negative loss that shoves temporally corrupted windows (shuffled, frozen, or reversed frames) away. The resulting space is called the action manifold. A generated video is scored for action consistency by the L2 distance between its averaged window embedding and the action-class centroid,

What carries the argument

The load-bearing object is the learned action manifold: a latent space in which each temporal window of real human motion is encoded by a transformer that fuses per-frame static features (3D body pose, shape, global orientation, 2D keypoints, visual appearance) with first-order temporal-derivative features. The training uses a supervised contrastive loss for same-action clustering and a hard-negative loss that repels temporally distorted windows, so the space is simultaneously semantic and motion-sensitive. Two derived scores carry the argument: S_cons measures distance to the nearest action-class centroid, and S_temp measures average jump size between consecutive frame embeddings; their joi

Load-bearing premise

The main load-bearing premise is that the benchmark's human-ratings comparison is fair: the encoder is trained on the same ten action classes used in the benchmark, so the measured advantage over zero-shot baselines may not hold for actions the encoder has never seen.

What would settle it

Generate videos of human actions outside the ten classes used in training, collect human ratings, estimate centroids from real videos of those new classes, and recompute S_cons; if its rank correlation with human judgments drops to the level of the strongest baseline (around 0.45), the reported gains are an artifact of the training-class overlap.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • Generated videos can be scored without any reference clip or text prompt, purely by how their embeddings sit relative to a real-action manifold.
  • The temporal-coherence score generalizes to actions outside the training set and reproduces the human ranking of text-to-video models on an external benchmark.
  • Removing the motion-derivative features causes the largest drop in accuracy, so temporal dynamics are the primary signal, not appearance.
  • The benchmark's human ratings expose that current video generators still fail systematically on whole-body rotational actions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the benchmark scores the metric on the same ten action classes used to train the encoder, so the large gain over zero-shot multimodal judges may partly reflect an in-distribution advantage; whether it holds on arbitrary unseen actions is the natural next test.
  • Editorial inference: the same manifold could be turned into a debugging tool — a video's per-window trajectory shows where it leaves the manifold, localizing which seconds of generation are anatomically or temporally wrong.
  • Editorial inference: because S_temp only measures smoothness, a frozen or duplicated-frame video can receive an artificially good score (the paper's own sensitivity analysis shows this exception); pairing it with S_cons or a stillness detector would close that hole.
  • Editorial inference: if centroids can be computed cheaply from a few real videos of a new action class, the metric becomes a practical plug-in for model selection during video-generation development.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper proposes a learned latent manifold for evaluating human actions in AI-generated videos. The approach extracts human-centric features (SMPL pose/shape/orientation, 2D keypoints, visual features) plus first-order temporal derivatives, trains a transformer encoder with supervised contrastive and hard-negative losses on 10 UCF-101 action classes, and derives two metrics: S_cons (distance of a generated video's embedding to a class centroid of real videos) and S_temp (mean inter-frame embedding distance). The authors introduce TAG-Bench, a benchmark of 300 generated videos with human ratings on Action Consistency and Temporal Coherence, and report that S_cons reaches Spearman ρ=0.61 and S_temp ρ=0.64, outperforming the best baseline (GPT-5) by 35.6% and 68.4% respectively. External validation on VBench-2.0 is performed with S_temp only, where it ranks four text-to-video models in agreement with humans.

Significance. If the claims hold, a learned human-action manifold metric would be a valuable tool for video generation evaluation, and TAG-Bench with human ratings is a useful community resource. The paper's strengths include a well-motivated combination of 3D/2D/visual features and temporal derivatives, thorough ablations (loss terms, input features, visual backbone, window length), and a serious human-study protocol with subject rejection and convergence analysis. The code is promised publicly. However, the central claim of a general-purpose human-action fidelity metric is not yet supported: the main S_cons result is evaluated on the same action classes used for training, and the only external benchmark test excludes S_cons entirely. Additionally, S_temp as defined in Eq. (4) assigns a perfect score to a static video, a known failure mode that the supplementary material concedes.

major comments (3)
  1. [Sec. 5.1 / Sec. 3.3 / Sec. 5.3] The headline comparison for S_cons is in-distribution with respect to action classes. Sec. 5.1 states the encoder is trained from scratch on the same 10 UCF-101 categories used in TAG-Bench, and Sec. 3.3 defines S_cons as a distance to class centroids c_k fitted from real videos of those exact classes. Thus the 0.61 vs 0.45 (GPT-5) and the claimed +35.6%/+68.4% gains in Table 1 show advantages on a test whose action vocabulary is identical to the training vocabulary, whereas the MLLM baselines are zero-shot. The external validation in Sec. 5.3 explicitly excludes S_cons because 'S_cons requires a corresponding action centroid,' so no evidence is presented that S_cons transfers to unseen actions. The abstract's claim of a general human-action fidelity metric therefore rests on missing evidence. I request an evaluation on held-out action classes (e.g., remaining UCF-101 or Kinetics action
  2. [Eq. (4) / Supp. Fig. 12] S_temp is defined as the average inter-frame embedding distance; identical adjacent frames yield zero distance. A video that is frozen or nearly static is therefore scored as perfectly temporally coherent. The supplementary material (Fig. 12 and surrounding text) explicitly acknowledges this 'Copy' exception, noting that S_temp does not increase because identical adjacent frames produce low embedding differences. This is not a peripheral artifact: static or near-static frames are a common failure mode of video generators, and the paper's own hard-negative training (Sec. 3.2.2) includes repeated-frame distortions as negatives. As defined, the metric cannot distinguish a static frame from smooth coherent motion. This limitation should be addressed in the metric itself or clearly stated as a scope restriction in the main paper, and the claim that S_temp measures temporal coherence should be
  3. [Table 1 / Fig. 3] The paper reports Spearman correlations without confidence intervals or significance tests. The differences between S_cons/S_temp and GPT-5 (0.61 vs 0.45; 0.64 vs 0.38) on 300 videos may be affected by sampling variability, especially given human-rating noise (inter-rater agreement is 0.72/0.71). In the VBench-2.0 external validation, Fig. 3 reports ρ=1.0000 based on only four models (Table 7), which is not statistically meaningful by itself. I recommend adding bootstrap confidence intervals for the correlations and, for the win-ratio comparison, reporting the number of prompts/comparisons and a test of ranking agreement that accounts for the small N.
minor comments (5)
  1. [Throughout] Several typos and formatting artifacts: in Table 1, '/uk' and 'ὑ2' symbols appear to be rendering issues; the abstract and Sec. 1 use '68%' without specifying the two metrics separately; Fig. 3 caption says 'ρ=1.0000' but the axis labels and fitting line are not defined in the caption.
  2. [Sec. 4 / Supp. B.2] The number of retained raters differs between the main text (Sec. 4: after subject rejection, average inter-rater correlation 0.716/0.710) and the appendix (121 retained for Action Consistency, 141 for Temporal Coherence). Please reconcile these numbers or clarify the exact sequence of filtering.
  3. [Sec. 5.3] For the VBench-2.0 external evaluation, the paper says 'we evaluate only videos with a single visible person per frame,' but does not report how many videos/prompts were retained after filtering. This information is needed to assess the robustness of the win-ratio result.
  4. [Sec. 5.5, Table 4] When replacing ViT features with CLIP or DINOv2, the paper says 'we train the encoder from scratch' but does not specify whether the same training budget and hyperparameters were used; if not, the comparison may reflect tuning rather than feature quality.
  5. [Sec. 3.2.1] The notation in Eq. (2) for the attention weights is slightly confusing: the softmax is over k, but the query q and key projection are shared across all k. Clarify that a single query is used per frame.

Circularity Check

0 steps flagged

No significant circularity; the S_cons in-distribution train/test overlap is an evaluation-scope caveat, not a derivation that reduces to its inputs.

full rationale

The paper's derivation is not circular by construction. The action-consistency metric is defined (Sec. 3.3) as the L2 distance between a generated video's mean [CLS] embedding and a class centroid c_k averaged over real videos; the encoder is trained with a supervised contrastive loss (Sec. 3.2.2, 5.1). This makes S_cons a fitted nearest-centroid score, but the reported 0.61 Spearman correlation is measured against independently collected human ratings of generated videos (Sec. 4, 5.2), not against the class labels used to fit the encoder/centroids. The correlation is therefore an empirical result, not an identity. S_temp (Eq. 4) is a self-contained inter-frame embedding-distance measure and is externally validated on VBench-2.0 prompts not in the training classes (Sec. 5.3), giving the temporal-coherence claim independent support. The main caveat is evaluation scope, not circularity: Sec. 5.1 trains on 'the same 10 UCF101 action categories' used in TAG-Bench (Sec. 4), and Sec. 5.3 admits S_cons is not applied to VBench-2.0 because 'Scons requires a corresponding action centroid.' Thus S_cons's generalization to unseen action classes is untested, and the +35.6%/+68.4% gains over zero-shot GPT-5 (Table 1) may be inflated by in-distribution fitting. The supplementary also admits a 'Copy' failure mode for S_temp (Fig. 12). These are validity/robustness limitations, not examples of a prediction being equivalent to its input by construction. No load-bearing self-citation chain, uniqueness import, or ansatz-via-citation is present.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

The central claim rests on three domain assumptions: UCF-101 labels define the action semantics; off-the-shelf human keypoint/mesh extractors work on generated videos; and filtered MTurk ratings are valid ground truth. Beyond hyperparameters λ, T, and rejection cutoffs, the only fitted values that enter the metric equations are the class centroids in S_cons.

free parameters (4)
  • Action class centroids c_k = 10×256-D vectors (one per class)
    Averaged window-level embeddings of real training videos per class; S_cons = ||z_gen − c_k|| (Sec. 3.3). Fitted to the same classes used in evaluation.
  • Hard-negative loss weight λ = 10
    Weight in Eq. 3 balancing L_supcon and L_hard-neg; set without reported sensitivity analysis.
  • Temporal window length T = 32 frames
    Window size for training and metrics; chosen from Table 10 sweep; all scores depend on this temporal scale.
  • Subject-rejection thresholds = 95th percentile, R1/R2 criteria, ρ<0.55
    MTurk filtering in App. B.2 determines the human MOS ground truth; cutoffs chosen empirically and affect all correlation results.
axioms (4)
  • domain assumption UCF-101 class labels for the 10 selected actions are a valid semantic ground truth for action consistency.
    The encoder's supervised contrastive loss (Sec. 3.2.2) and S_cons centroids (Sec. 3.3) both assume these 10 classes cover the relevant space of 'real-world human actions'.
  • domain assumption SMPL parameters, 2D keypoints, and ViT appearance features extracted from generated videos are sufficiently reliable to represent motion and anatomy.
    Sec. 3.1 and Sec. 5.1 rely on TokenHMR and DW-Pose to obtain features; if these extractors fail on generated artifacts, the learned manifold distances become uninformative. The paper's use of 2D keypoints partly mitigates SMPL's anatomical prior but does not eliminate the dependency.
  • domain assumption Human ratings on AMT after multi-stage subject rejection and z-scoring reflect true action consistency and temporal coherence.
    TAG-Bench (Sec. 4, App. B.2) uses heavily filtered MTurk labels as ground truth; the central correlations in Table 1 inherit any bias from this filtering.
  • domain assumption The learned embedding space is structured so that real videos of an action form compact clusters and generated videos of the same action should lie near the real centroid.
    This is the core postulate in Sec. 3.3 (Fig. 1); the supervised contrastive loss is trained to encourage it, so it is an assumption built into the method.

pith-pipeline@v1.3.0-alltime-deepseek · 22325 in / 14240 out tokens · 135596 ms · 2026-08-03T19:06:09.473645+00:00 · methodology

0 comments
read the original abstract

Despite rapid advances in video generative models, robust metrics for evaluating visual and temporal correctness of complex human actions remain elusive. Critically, existing pure-vision encoders and Multimodal Large Language Models (MLLMs) are strongly appearance-biased, lack temporal understanding, and thus struggle to discern intricate motion dynamics and anatomical implausibilities in generated videos. We tackle this gap by introducing a novel evaluation metric derived from a learned latent space of real-world human actions. Our method first captures the nuances, constraints, and temporal smoothness of real-world motion by fusing appearance-agnostic human skeletal geometry features with appearance-based features. We posit that this combined feature space provides a robust representation of action plausibility. Given a generated video, our metric quantifies its action quality by measuring the distance between its underlying representations and this learned real-world action distribution. For rigorous validation, we develop a new multi-faceted benchmark specifically designed to probe temporally challenging aspects of human action fidelity. Through extensive experiments, we show that our metric achieves substantial improvement of more than 68% compared to existing state-of-the-art methods on our benchmark, performs competitively on established external benchmarks, and has a stronger correlation with human perception. Our in-depth analysis reveals critical limitations in current video generative models and establishes a new standard for advanced research in video generation.

Figures

Figures reproduced from arXiv: 2512.01803 by Ananya Srinivasan, Audrey Zheng, Deepti Ghadiyaram, Xavier Thomas, Youngsun Lim.

Figure 1
Figure 1. Figure 1: What are the telltale signs of a generative action video? We an￾swer this by learning a robust manifold based on appearance and anatomi￾cal coherence exhibited by humans performing actions across several real￾world videos. This manifold serves as anchors against which we project the features of a generated video in question and assess its realism. derstanding allows humans to effortlessly recognize motion … view at source ↗
Figure 2
Figure 2. Figure 2: Architectural overview of the encoder we train to learn the real-world action manifold. We extract per-frame static human-centric and temporal motion features (Fig. (a)) (Sec. 3.1), and aggregate them, yielding one embedding for each frame (Fig. (b)) (Sec. 3.2.1). We prepend a [CLS] token to the per-frame tokens and pass as input to a 4-layer trans￾former encoder (Fig. (c)) (Sec. 3.2.1). Our aim is to enco… view at source ↗
Figure 3
Figure 3. Figure 3: Model comparisons on TAG-Bench and VBench-2.0 Human Anatomy. We compare models pairwise for the same input prompt; for each pair, the model with the higher score (human or metric) is the winner. We then plot the win ratios (see Sec. 5.3) of human scores (x-axis) against win ratios from our metric (y-axis). Our metrics (Scons and Stemp) observe the same ranking of models as humans on both benchmarks. align … view at source ↗
Figure 4
Figure 4. Figure 4: Comparing generative models. We plot the mean Scons and Stemp scores (Sec. 3.3) (lower is better) for each generative model across different actions. Wan2.2 performs best among the other models (low scores in both Scons and Stemp). Shotput and JumpingJack challenge all models, yielding high scores across both metrics [PITH_FULL_IMAGE:figures/full_fig_p008_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: t-SNE visualization of the embeddings of generated videos along with train centroids. We project the zCLS embeddings of generated videos (colored markers) from TAG-Bench and the corresponding train￾ing class centroids (white crosses) using t-SNE [56]. Realistic generated videos cluster near their respective class centroids (e.g., Wan2.2 videos for “PullUps”, with an average human rating of: 8.41 for Action… view at source ↗
Figure 6
Figure 6. Figure 6: User interface used for the human evaluation study. Participants were asked to rate each AI-generated video along two dimensions: Action Consistency (how accurately the motion matches the described action) and Temporal Coherence (how natural and physically realistic the motion appears). Each participant viewed 30 videos and provided ratings on a scale from 0 to 10. 1 [PITH_FULL_IMAGE:figures/full_fig_p013… view at source ↗
Figure 7
Figure 7. Figure 7: Standard deviation vs. number of raters for five randomly selected videos (A [PITH_FULL_IMAGE:figures/full_fig_p015_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Standard deviation vs. number of raters for five randomly selected videos ( [PITH_FULL_IMAGE:figures/full_fig_p015_8.png] view at source ↗
Figure 9
Figure 9. Figure 9: TokenHMR-based feature extraction. Each input frame is processed by Detectron2 [63] to obtain a bounding box of the person, which is cropped and passed to TokenHMR [19]. TokenHMR uses a ViT-H/16 backbone, followed by transformer blocks and task-specific heads, to predict SMPL parameters: pose (θ), global orientation (go), and body shape (β) in Sec. 3.1.1. Intermediate features from the ViT backbone are use… view at source ↗
Figure 10
Figure 10. Figure 10: Human-centric input features. From each frame, we extract (a) 3D pose, body shape, and global orientation (Sec. 3.1.1), (b) 2D keypoints (Sec. 3.1.2), and (c) visual appearance features (Sec. 3.1.3) to describe the body’s state in that frame. (d) We additionally compute temporal differences of each feature to capture frame-to-frame motion dynamics (Sec. 3.1.4) Feature Pose Global orientation Body shape Ke… view at source ↗
Figure 11
Figure 11. Figure 11: t-SNE visualization of the embeddings of unseen test videos of diverse actions. We project the zCLS embeddings of unseen real test videos (colored markers) and the corresponding training class centroids (crosses) using t-SNE [56]. It is evident that unseen test videos cluster around their respective class centroids, indicating that the learned embedding space captures compact and semantically meaningful a… view at source ↗
Figure 12
Figure 12. Figure 12: Sensitivity of Action Consistency (Scons) and Temporal Coherence (Stemp) to controlled temporal distortions. The mean value across all test samples is plotted for each distortion type and severity. Sensitivity to temporal distortions. We measure whether the learned embedding space and metrics are temporally sensitive. For this, we apply controlled temporal corruptions to unseen real videos as done on trai… view at source ↗
Figure 13
Figure 13. Figure 13: Average attention weights averaged over all real test videos. [PITH_FULL_IMAGE:figures/full_fig_p019_13.png] view at source ↗
Figure 14
Figure 14. Figure 14: Per-class attention weights, averaged over all real test videos. [PITH_FULL_IMAGE:figures/full_fig_p019_14.png] view at source ↗
Figure 15
Figure 15. Figure 15: Example of the 4×10 grid-panel layout used to prompt MLLMs. Shown here is a video generated by Hunyuan [32] for the action class BodyWeightSquats. We uniformly sample 40 frames and place them in row-major order. Each cell overlays its grid coordinates (row,col) in the top-left; when the video duration is available, the timestamp is also shown (e.g., (0,3) 1.2s). This grid preserves temporal progression an… view at source ↗
Figure 16
Figure 16. Figure 16: Per-video Qwen3-VL scores for Action Consistency (orange) and Temporal Coherence (green) before (left) and after (right) applying in-context learning. Individual dots correspond to the model prediction for each generated video. Scores are highly saturated near 0 or 1 in both cases, indicating binary-like decisions rather than nuanced motion reasoning [PITH_FULL_IMAGE:figures/full_fig_p022_16.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

75 extracted references · 26 linked inside Pith

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ah- mad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774,

    Pith/arXiv arXiv

  2. [2]

    Kuhl, and Andrew N

    Alison Gopnik, Patricia K. Kuhl, and Andrew N. Meltzoff. The scientist in the crib: Minds, brains, and how children learn, 1999. 1

    1999

  3. [3]

    Di- rect motion models for assessing generated videos.arXiv preprint arXiv:2505.00209, 2025

    Kelsey Allen, Carl Doersch, Guangyao Zhou, Mohammed Suhail, Danny Driess, Ignacio Rocco, Yulia Rubanova, Thomas Kipf, Mehdi SM Sajjadi, Kevin Murphy, et al. Di- rect motion models for assessing generated videos.arXiv preprint arXiv:2505.00209, 2025. 6, 8

    Pith/arXiv arXiv 2025

  4. [4]

    Amazon mechanical turk

    Amazon Web Services, Inc. Amazon mechanical turk. https://www.mturk.com/. Accessed: November

  5. [5]

    Tuned con- trastive learning.arXiv preprint arXiv:2305.10675, 2023

    Chaitanya Animesh and Manmohan Chandraker. Tuned con- trastive learning.arXiv preprint arXiv:2305.10675, 2023. 4

    Pith/arXiv arXiv 2023

  6. [6]

    Videophy: Evaluating physical commonsense for video generation.arXiv preprint arXiv:2406.03520, 2024

    Hritik Bansal, Zongyu Lin, Tianyi Xie, Zeshun Zong, Michal Yarom, Yonatan Bitton, Chenfanfu Jiang, Yizhou Sun, Kai- Wei Chang, and Aditya Grover. Videophy: Evaluating physical commonsense for video generation.arXiv preprint arXiv:2406.03520, 2024. 2

    Pith/arXiv arXiv 2024

  7. [7]

    Videophy-2: A challenging action-centric physical commonsense evaluation in video generation.arXiv preprint arXiv:2503.06800, 2025

    Hritik Bansal, Clark Peng, Yonatan Bitton, Roman Golden- berg, Aditya Grover, and Kai-Wei Chang. Videophy-2: A challenging action-centric physical commonsense evaluation in video generation.arXiv preprint arXiv:2503.06800, 2025. 6, 8

    Pith/arXiv arXiv 2025

  8. [8]

    Row, Peterson and Company, 1954

    Roger G Barker and Herbert F Wright.Midwest and the USA. Row, Peterson and Company, 1954. 1

    1954

  9. [9]

    Is space-time attention all you need for video understanding? International Conference on Machine Learning, 2021

    Gedas Bertasius, Heng Wang, and Lorenzo Torresani. Is space-time attention all you need for video understanding? International Conference on Machine Learning, 2021. 4

    2021

  10. [10]

    Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023

    Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram V oleti, Adam Letts, et al. Stable video diffusion: Scaling latent video diffusion models to large datasets.arXiv preprint arXiv:2311.15127, 2023. 2

    Pith/arXiv arXiv 2023

  11. [11]

    Realtime multi-person 2d pose estimation using part affin- ity fields

    Zhe Cao, Tomas Simon, Shih-En Wei, and Yaser Sheikh. Realtime multi-person 2d pose estimation using part affin- ity fields. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2017. 3

    2017

  12. [12]

    Emerg- ing properties in self-supervised vision transformers.2021 IEEE/CVF International Conference on Computer Vision,

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv’e J’egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers.2021 IEEE/CVF International Conference on Computer Vision,

    2021

  13. [13]

    Emerg- ing properties in self-supervised vision transformers

    Mathilde Caron, Hugo Touvron, Ishan Misra, Herv ´e J´egou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerg- ing properties in self-supervised vision transformers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2021. 6

    2021

  14. [14]

    A short note on the kinetics-700 human action dataset.arXiv preprint arXiv:1907.06987, 2019

    Joao Carreira, Eric Noland, Chloe Hillier, and Andrew Zis- serman. A short note on the kinetics-700 human action dataset.arXiv preprint arXiv:1907.06987, 2019. 9

    Pith/arXiv arXiv 1907

  15. [15]

    Sora as an agi world model? a complete survey on text-to-video generation.arXiv preprint arXiv:2403.05131, 2024

    Joseph Cho, Fachrina Dewi Puspitasari, Sheng Zheng, Jingyao Zheng, Lik-Hang Lee, Tae-Ho Kim, Choong Seon Hong, and Chaoning Zhang. Sora as an agi world model? a complete survey on text-to-video generation.arXiv preprint arXiv:2403.05131, 2024. 2, 5

    arXiv 2024

  16. [16]

    Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025

    Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, Noveen Sachdeva, Inderjit Dhillon, Marcel Blis- tein, Ori Ram, Dan Zhang, Evan Rosen, et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities.arXiv preprint arXiv:2507.06261, 2025. 6, 7

    Pith/arXiv arXiv 2025

  17. [17]

    Arcface: Additive angular margin loss for deep face recognition

    Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, 2019. 8

    2019

  18. [18]

    An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020. 3

    Pith/arXiv arXiv 2010

  19. [19]

    Tokenhmr: Advancing human mesh recov- ery with a tokenized pose representation

    Sai Kumar Dwivedi, Yu Sun, Priyanka Patel, Yao Feng, and Michael J Black. Tokenhmr: Advancing human mesh recov- ery with a tokenized pose representation. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 5, 9, 4, 7

    2024

  20. [20]

    Smpl made simple faqs.https : / / files

    Perceiving Systems Department MPI for Intelligent Sys- tems. Smpl made simple faqs.https : / / files . is . tue . mpg . de / black / talks / SMPL - made - simple-FAQs.pdf. 3

  21. [21]

    Ava: A video dataset of spatio-temporally localized atomic visual actions

    Chunhui Gu, Chen Sun, David A Ross, Carl V ondrick, Car- oline Pantofaru, Yeqing Li, Sudheendra Vijayanarasimhan, George Toderici, Susanna Ricco, Rahul Sukthankar, et al. Ava: A video dataset of spatio-temporally localized atomic visual actions. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018. 1

    2018

  22. [22]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2016. 6

    2016

  23. [23]

    Videoscore: Building automatic metrics to simulate fine-grained human feedback for video genera- tion

    Xuan He, Dongfu Jiang, Ge Zhang, Max Ku, Achint Soni, Sherman Siu, Haonan Chen, Abhranil Chandra, Ziyan Jiang, Aaran Arulraj, et al. Videoscore: Building automatic metrics to simulate fine-grained human feedback for video genera- tion. InProceedings of the Conference on Empirical Meth- ods in Natural Language Processing, 2024. 2, 6, 8

    2024

  24. [24]

    Videoscore2: Think before you score in generative video evaluation.arXiv preprint arXiv:2509.22799, 2025

    Xuan He, Dongfu Jiang, Ping Nie, Minghao Liu, Zhengxuan Jiang, Mingyi Su, Wentao Ma, Junru Lin, Chun Ye, Yi Lu, et al. Videoscore2: Think before you score in generative video evaluation.arXiv preprint arXiv:2509.22799, 2025. 5, 6, 8

    arXiv 2025

  25. [25]

    Clipscore: A reference-free evaluation met- ric for image captioning

    Jack Hessel, Ari Holtzman, Maxwell Forbes, Ronan Le Bras, and Yejin Choi. Clipscore: A reference-free evaluation met- ric for image captioning. InProceedings of the Confer- ence on Empirical Methods in Natural Language Processing,

  26. [26]

    Video dif- fusion models.Advances in Neural Information Processing Systems, 2022

    Jonathan Ho, Tim Salimans, Alexey Gritsenko, William Chan, Mohammad Norouzi, and David J Fleet. Video dif- fusion models.Advances in Neural Information Processing Systems, 2022. 2 10

    2022

  27. [27]

    Cogvideo: Large-scale pretraining for text-to-video generation via transformers.arXiv preprint arXiv:2205.15868, 2022

    Wenyi Hong, Ming Ding, Wendi Zheng, Xinghan Liu, and Jie Tang. Cogvideo: Large-scale pretraining for text-to-video generation via transformers.arXiv preprint arXiv:2205.15868, 2022. 2, 7, 5

    Pith/arXiv arXiv 2022

  28. [28]

    Image quality metrics: Psnr vs

    Alain Hore and Djemel Ziou. Image quality metrics: Psnr vs. ssim. InProceedings of the 20th IEEE International Confer- ence on Pattern Recognition, 2010. 2

    2010

  29. [29]

    Methodologies for the subjective assessment of the quality of television pic- tures

    International Telecommunication Union. Methodologies for the subjective assessment of the quality of television pic- tures. Technical report, ITU Radiocommunication Sector (ITU-R), 2019. 2

    2019

  30. [30]

    Black, David W

    Angjoo Kanazawa, Michael J. Black, David W. Jacobs, and Jitendra Malik. End-to-end recovery of human shape and pose. InProceedings of the IEEE/CVF Conference on Com- puter Vision and Pattern Recognition, 2018. 3

    2018

  31. [31]

    Vibe: Video inference for human body pose and shape estimation

    Muhammed Kocabas, Nikos Athanasiou, and Michael J Black. Vibe: Video inference for human body pose and shape estimation. InProceedings of the IEEE/CVF Confer- ence on Computer Vision and Pattern Recognition, 2020. 3

    2020

  32. [32]

    Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024

    Weijie Kong, Qi Tian, Zijian Zhang, Rox Min, Zuozhuo Dai, Jin Zhou, Jiangfeng Xiong, Xin Li, Bo Wu, Jianwei Zhang, et al. Hunyuanvideo: A systematic framework for large video generative models.arXiv preprint arXiv:2412.03603, 2024. 2, 5, 7, 9

    Pith/arXiv arXiv 2024

  33. [33]

    Kling: High-fidelity and temporally consistent text-to-video generation.Technical Report, 2024

    Kuaishou Technology. Kling: High-fidelity and temporally consistent text-to-video generation.Technical Report, 2024. https://kling.kuaishou.com. 7, 5

    2024

  34. [34]

    What matters when building vision-language models? Advances in Neural Information Processing Systems, 2024

    Hugo Laurenc ¸on, L´eo Tronchon, Matthieu Cord, and Victor Sanh. What matters when building vision-language models? Advances in Neural Information Processing Systems, 2024. 6

    2024

  35. [35]

    Genmo: A generalist model for human motion.arXiv preprint arXiv:2505.01425,

    Jiefeng Li, Jinkun Cao, Haotian Zhang, Davis Rempe, Jan Kautz, Umar Iqbal, and Ye Yuan. Genmo: A generalist model for human motion.arXiv preprint arXiv:2505.01425,

    Pith/arXiv arXiv

  36. [36]

    Visual instruction tuning.Advances in Neural Information Processing Systems, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in Neural Information Processing Systems, 2023. 6

    2023

  37. [37]

    Llava-next: Im- proved reasoning, ocr, and world knowledge for large mul- timodal models (llava v1.6).https : / / llava - vl

    Haotian Liu, Chunyuan Li, Yuheng Li, Bo Li, Yuanhan Zhang, Sheng Shen, and Yong Jae Lee. Llava-next: Im- proved reasoning, ocr, and world knowledge for large mul- timodal models (llava v1.6).https : / / llava - vl . github . io / blog / 2024 - 01 - 30 - llava - next/,

    2024

  38. [38]

    Evalcrafter: Benchmarking and eval- uating large video generation models

    Yaofang Liu, Xiaodong Cun, Xuebo Liu, Xintao Wang, Yong Zhang, Haoxin Chen, Yang Liu, Tieyong Zeng, Raymond Chan, and Ying Shan. Evalcrafter: Benchmarking and eval- uating large video generation models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 2

    2024

  39. [39]

    Smpl: A skinned multi- person linear model.Seminal Graphics Papers: Pushing the Boundaries, Volume 2, 2023

    Matthew Loper, Naureen Mahmood, Javier Romero, Gerard Pons-Moll, and Michael J Black. Smpl: A skinned multi- person linear model.Seminal Graphics Papers: Pushing the Boundaries, Volume 2, 2023. 3

    2023

  40. [40]

    Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017

    Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization.arXiv preprint arXiv:1711.05101, 2017. 6

    Pith/arXiv arXiv 2017

  41. [41]

    X-clip: End-to-end multi-grained con- trastive learning for video-text retrieval, 2022

    Yiwei Ma, Guohai Xu, Xiaoshuai Sun, Ming Yan, Ji Zhang, and Rongrong Ji. X-clip: End-to-end multi-grained con- trastive learning for video-text retrieval, 2022. 6, 8

    2022

  42. [42]

    No-reference image quality assessment in the spa- tial domain.IEEE Transactions on Image Processing, 2012

    Anish Mittal, Anush Krishna Moorthy, and Alan Conrad Bovik. No-reference image quality assessment in the spa- tial domain.IEEE Transactions on Image Processing, 2012. 6, 8

    2012

  43. [43]

    Do generative video models understand physical principles?arXiv preprint arXiv:2501.09038, 2025

    Saman Motamed, Laura Culp, Kevin Swersky, Priyank Jaini, and Robert Geirhos. Do generative video models understand physical principles?arXiv preprint arXiv:2501.09038, 2025. 1, 2

    Pith/arXiv arXiv 2025

  44. [44]

    Sora: A large-scale diffusion transformer for text- to-video generation.Technical Report, 2024.https:// openai.com/research/sora

    OpenAI. Sora: A large-scale diffusion transformer for text- to-video generation.Technical Report, 2024.https:// openai.com/research/sora. 2, 7, 5

    2024

  45. [45]

    Gpt-5 system card

    OpenAI. Gpt-5 system card. Technical report, OpenAI,

  46. [46]

    Dinov2: Learning robust visual features without supervision

    Maxime Oquab, Timoth ´ee Darcet, Th ´eo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision. arXiv preprint arXiv:2304.07193, 2023. 9

    Pith/arXiv arXiv 2023

  47. [47]

    Expressive body capture: 3d hands, face, and body from a single image

    Georgios Pavlakos, Vasileios Choutas, Nima Ghorbani, Timo Bolkart, Ahmed AA Osman, Dimitrios Tzionas, and Michael J Black. Expressive body capture: 3d hands, face, and body from a single image. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2019. 3

    2019

  48. [48]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. InInternational Conference on Machine Learning. PmLR, 2021. 3, 6, 9, 8

    2021

  49. [49]

    Two-stream con- volutional networks for action recognition in videos.Ad- vances in Neural Information Processing Systems, 2014

    Karen Simonyan and Andrew Zisserman. Two-stream con- volutional networks for action recognition in videos.Ad- vances in Neural Information Processing Systems, 2014. 4

    2014

  50. [50]

    Ucf101: A dataset of 101 human actions classes from videos in the wild.arXiv preprint arXiv:1212.0402, 2012

    Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. Ucf101: A dataset of 101 human actions classes from videos in the wild.arXiv preprint arXiv:1212.0402, 2012. 5, 9

    Pith/arXiv arXiv 2012

  51. [51]

    Runway gen-4: Advancing realistic text-to-video generation.Technical Report, 2024.https: //research.runwayml.com/gen4

    Runway Research Team. Runway gen-4: Advancing realistic text-to-video generation.Technical Report, 2024.https: //research.runwayml.com/gen4. 1, 2, 5, 7

    2024

  52. [52]

    Human motion dif- fusion model.arXiv preprint arXiv:2209.14916, 2022

    Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Daniel Cohen-Or, and Amit H Bermano. Human motion dif- fusion model.arXiv preprint arXiv:2209.14916, 2022. 2

    Pith/arXiv arXiv 2022

  53. [53]

    What’s in a latent? leveraging diffusion latent space for domain generalization

    Xavier Thomas and Deepti Ghadiyaram. What’s in a latent? leveraging diffusion latent space for domain generalization. arXiv preprint arXiv:2503.06698, 2025. 6

    Pith/arXiv arXiv 2025

  54. [54]

    Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training.Advances in Neural Information Processing Systems, 2022

    Zhan Tong, Yibing Song, Jue Wang, and Limin Wang. Videomae: Masked autoencoders are data-efficient learners for self-supervised video pre-training.Advances in Neural Information Processing Systems, 2022. 6

    2022

  55. [55]

    To- wards accurate generative models of video: A new metric & challenges.arXiv preprint arXiv:1812.01717, 2018

    Thomas Unterthiner, Sjoerd Van Steenkiste, Karol Kurach, Raphael Marinier, Marcin Michalski, and Sylvain Gelly. To- wards accurate generative models of video: A new metric & challenges.arXiv preprint arXiv:1812.01717, 2018. 2 11

    Pith/arXiv arXiv 2018

  56. [56]

    Visualizing data using t-sne.Journal of Machine Learning Research,

    Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne.Journal of Machine Learning Research,

  57. [57]

    Attention is all you need.Advances in Neural Information Processing Systems, 30, 2017

    Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszko- reit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need.Advances in Neural Information Processing Systems, 30, 2017. 4

    2017

  58. [58]

    Praneeth, Sumohana S

    Narasimhan Venkatanath, D. Praneeth, Sumohana S. Chan- nappayya, and Swarup S. Medasani. Blind image quality evaluation using perception-based features. InProceedings of the 2015 Twenty First National Conference on Communi- cations, 2015. 6, 8

    2015

  59. [59]

    Wan: Open and advanced large-scale video gen- erative models.arXiv preprint arXiv:2503.20314, 2025

    Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video gen- erative models.arXiv preprint arXiv:2503.20314, 2025. 1, 2, 5, 7

    Pith/arXiv arXiv 2025

  60. [60]

    Bovik, H.R

    Zhou Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli. Image quality assessment: from error visibility to structural similarity.IEEE Transactions on Image Processing, 2004. 6, 8

    2004

  61. [61]

    Image quality assessment: from error visibility to structural similarity.IEEE Transactions on Image Process- ing, 2004

    Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Si- moncelli. Image quality assessment: from error visibility to structural similarity.IEEE Transactions on Image Process- ing, 2004. 2, 6, 8

    2004

  62. [62]

    Video models are zero-shot learn- ers and reasoners.arXiv preprint arXiv:2509.20328, 2025

    Thadd ¨aus Wiedemer, Yuxuan Li, Paul Vicol, Shixiang Shane Gu, Nick Matarese, Kevin Swersky, Been Kim, Priyank Jaini, and Robert Geirhos. Video models are zero-shot learn- ers and reasoners.arXiv preprint arXiv:2509.20328, 2025. 2

    Pith/arXiv arXiv 2025

  63. [63]

    Detectron2.https://github

    Yuxin Wu, Alexander Kirillov, Francisco Massa, Wan-Yen Lo, and Ross Girshick. Detectron2.https://github. com/facebookresearch/detectron2, 2019. 5, 4

    2019

  64. [64]

    A survey on video diffusion models.ACM Computing Surveys, 2024

    Zhen Xing, Qijun Feng, Haoran Chen, Qi Dai, Han Hu, Hang Xu, Zuxuan Wu, and Yu-Gang Jiang. A survey on video diffusion models.ACM Computing Surveys, 2024. 2

    2024

  65. [65]

    Seeing the ar- row of time in large multimodal models.arXiv preprint arXiv:2506.03340, 2025

    Zihui Xue, Mi Luo, and Kristen Grauman. Seeing the ar- row of time in large multimodal models.arXiv preprint arXiv:2506.03340, 2025. 7

    arXiv 2025

  66. [66]

    Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025

    An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025. 6

    Pith/arXiv arXiv 2025

  67. [67]

    Effec- tive whole-body pose estimation with two-stages distillation

    Zhendong Yang, Ailing Zeng, Chun Yuan, and Yu Li. Effec- tive whole-body pose estimation with two-stages distillation. Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023. 5

    2023

  68. [68]

    The unreasonable effectiveness of deep features as a perceptual metric

    Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shecht- man, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2018. 2

    2018

  69. [69]

    I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models.arXiv preprint arXiv:2311.04145, 2023

    Shiwei Zhang, Jiayu Wang, Yingya Zhang, Kang Zhao, Hangjie Yuan, Zhiwu Qin, Xiang Wang, Deli Zhao, and Jingren Zhou. I2vgen-xl: High-quality image-to-video synthesis via cascaded diffusion models.arXiv preprint arXiv:2311.04145, 2023. 2

    Pith/arXiv arXiv 2023

  70. [70]

    Video instruction tuning with synthetic data.arXiv preprint arXiv:2410.02713, 2024

    Yuanhan Zhang, Jinming Wu, Wei Li, Bo Li, Zejun Ma, Zi- wei Liu, and Chunyuan Li. Video instruction tuning with synthetic data.arXiv preprint arXiv:2410.02713, 2024. 8

    Pith/arXiv arXiv 2024

  71. [71]

    Vbench-2.0: Advancing video generation benchmark suite for intrinsic faithfulness.arXiv preprint arXiv:2503.21755, 2025

    Dian Zheng, Ziqi Huang, Hongbo Liu, Kai Zou, Yinan He, Fan Zhang, Lulu Gu, Yuanhan Zhang, Jingwen He, Wei- Shi Zheng, et al. Vbench-2.0: Advancing video generation benchmark suite for intrinsic faithfulness.arXiv preprint arXiv:2503.21755, 2025. 2, 5, 6, 7, 8

    Pith/arXiv arXiv 2025

  72. [72]

    HulaHoop

    Zangwei Zheng, Xiangyu Peng, Tianji Yang, Chenhui Shen, Shenggui Li, Hongxin Liu, Yukun Zhou, Tianyi Li, and Yang You. Open-sora: Democratizing efficient video production for all.arXiv preprint arXiv:2412.20404, 2024. 5 12 Generative Action Tell-Tales: Assessing Human Motion in Synthesized Videos Supplementary Material Table of Contents A.Human Evaluation...

    Pith/arXiv arXiv 2024

  73. [75]

    - Check pose, motion pattern, timing, and repeated evidence of that action

    Action Consistency (action_consistency): How well does the visible action in the frames match the described target action? - Focus only on what is clearly shown. - Check pose, motion pattern, timing, and repeated evidence of that action. - Do NOT guess intentions outside the frames

  74. [76]

    action_consistency

    Temporal Coherence (temporal_coherence): How physically realistic / plausible are the motions and body configurations? - Look for broken limbs, impossible joint angles, teleporting limbs, limbs merging into objects, obvious gravity violations, ghost artifacts (extra arms / missing torso), etc. - Minor render glitches are OK if motion is still basically hu...

  75. [2025]

    Accessed: 2025-11-10. 6

    2025