REVIEW 10 cited by
Mora: Enabling Generalist Video Generation via A Multi-Agent Framework
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Text-to-video generation has made significant strides, but replicating the capabilities of advanced systems like OpenAI Sora remains challenging due to their closed-source nature. Existing open-source methods struggle to achieve comparable performance, often hindered by ineffective agent collaboration and inadequate training data quality. In this paper, we introduce Mora, a novel multi-agent framework that leverages existing open-source modules to replicate Sora functionalities. We address these fundamental limitations by proposing three key techniques: (1) multi-agent fine-tuning with a self-modulation factor to enhance inter-agent coordination, (2) a data-free training strategy that uses large models to synthesize training data, and (3) a human-in-the-loop mechanism combined with multimodal large language models for data filtering to ensure high-quality training datasets. Our comprehensive experiments on six video generation tasks demonstrate that Mora achieves performance comparable to Sora on VBench, outperforming existing open-source methods across various tasks. Specifically, in the text-to-video generation task, Mora achieved a Video Quality score of 0.800, surpassing Sora 0.797 and outperforming all other baseline models across six key metrics. Additionally, in the image-to-video generation task, Mora achieved a perfect Dynamic Degree score of 1.00, demonstrating exceptional capability in enhancing motion realism and achieving higher Imaging Quality than Sora. These results highlight the potential of collaborative multi-agent systems and human-in-the-loop mechanisms in advancing text-to-video generation. Our code is available at \url{https://github.com/lichao-sun/Mora}.
Forward citations
Cited by 10 Pith papers
-
AgentHOI: Multi-Agent Reasoning for Human-Object-Interaction Video Generation via Implicit Representation Alignment
AgentHOI generates human-object interaction videos from text plus one human image and one object image, using multi-agent action planning and implicit text-to-motion feature alignment inside a video diffusion model.
-
When Cultures Move: Measuring and Improving Multicultural Text-to-Video Generation
Parallel, role-specialized prompt agents improve cultural relevance in text-to-video generation, with a new cross-cultural benchmark showing the largest gains for location cues.
-
Detecting AI-Generated Videos with Spiking Neural Networks
MAST with spiking neural networks achieves 93.14% mean accuracy detecting AI-generated videos from 10 unseen generators by exploiting smoother pixel residuals and compact semantic trajectories.
-
OmniHuman-1.5: Instilling an Active Mind in Avatars via Cognitive Simulation
OmniHuman-1.5 combines MLLM-based planning with a multimodal diffusion transformer and pseudo-last-frame identity conditioning to generate context-aware avatar videos from audio and a reference image.
-
AmbiGraph-Eval: Can LLMs Effectively Handle Ambiguous Graph Queries?
A benchmark organized by a six-type taxonomy of ambiguous graph queries reportedly shows that nine LLMs, including top models, frequently produce wrong query translations.
-
TokensGen: Harnessing Condensed Tokens for Long Video Generation
TokensGen generates consistent long videos by representing each clip as condensed semantic tokens, generating all tokens jointly from text, and stitching clips with adaptive FIFO denoising.
-
SPAgent: Adaptive Task Decomposition and Model Selection for General Video Generation and Editing
SPAgent is an MLLM-based coordinator that decomposes user instructions, plans execution routes, and selects among open-source video generation and editing models, outperforming single models in MOS.
-
ViBe: A Text-to-Video Benchmark for Evaluating Hallucination in Large Multimodal Models
The paper builds and human-labels a 3,782-video dataset spanning five hallucination categories in text-to-video outputs, and shows that standard classifiers reach only about 35% accuracy on the resulting classification task.
-
BounTCHA: A CAPTCHA Utilizing Boundary Identification in Guided Generative AI-extended Videos
BounTCHA uses human ability to spot the boundary between a real video and its AI-generated extension as a new CAPTCHA test, with about 83% human accuracy versus under 20% for tested AI models.
-
Visual Large Language Models for Generalized and Specialized Applications
This paper reviews and taxonomizes VLLM applications into vision-to-text, vision-to-action, and text-to-vision, adding ethics and future-work discussion.
Discussion (0). Continue with ORCID to comment.