A two-stage alignment framework that first fuses visual modalities (RGB, flow, skeleton) then introduces text, achieving 21% SRCC improvement on a new clinical AQA dataset and gains on two public benchmarks.
Gaia: Rethinking action quality assessment for ai-generated videos
2 Pith papers cite this work. Polarity classification is still indexing.
2
Pith papers citing it
abstract
Assessing action quality is both imperative and challenging due to its significant impact on the quality of AI-generated videos, further complicated by the inherently ambiguous nature of actions within AI-generated video (AIGV). Current action quality assessment (AQA) algorithms predominantly focus on actions from real specific scenarios and are pre-trained with normative action features, thus rendering them inapplicable in AIGVs. To address these problems, we construct GAIA, a Generic AI-generated Action dataset, by conducting a large-scale subjective evaluation from a novel causal reasoning-based perspective, resulting in 971,244 ratings among 9,180 video-action pairs. Based on GAIA, we evaluate a suite of popular text-to-video (T2V) models on their ability to generate visually rational actions, revealing their pros and cons on different categories of actions. We also extend GAIA as a testbed to benchmark the AQA capacity of existing automatic evaluation methods. Results show that traditional AQA methods, action-related metrics in recent T2V benchmarks, and mainstream video quality methods perform poorly with an average SRCC of 0.454, 0.191, and 0.519, respectively, indicating a sizable gap between current models and human action perception patterns in AIGVs. Our findings underscore the significance of action quality as a unique perspective for studying AIGVs and can catalyze progress towards methods with enhanced capacities for AQA in AIGVs.
citation-role summary
dataset 1
citation-polarity summary
fields
cs.CV 2roles
dataset 1polarities
use dataset 1representative citing papers
This survey proposes a modality-driven hierarchical taxonomy for AQA methods, establishes a unified benchmark for video-based approaches across datasets, and outlines research trends and challenges.
citing papers explorer
-
Two-Stage Multi-Modal Fusion with Adaptive Alignment for Action Quality Assessment
A two-stage alignment framework that first fuses visual modalities (RGB, flow, skeleton) then introduces text, achieving 21% SRCC improvement on a new clinical AQA dataset and gains on two public benchmarks.
-
A Comprehensive Survey of Action Quality Assessment: Method and Benchmark
This survey proposes a modality-driven hierarchical taxonomy for AQA methods, establishes a unified benchmark for video-based approaches across datasets, and outlines research trends and challenges.