GEAR jointly trains VQ tokenizer and AR generator end-to-end via dual hard/soft read-out and representation alignment, achieving up to 10x faster ImageNet gFID convergence than LlamaGen-REPA while generalizing across quantizers and to text-to-image.
super hub Mixed citations
Scalable diffusion models with transformers
Mixed citation behavior. Most common role is method (47%).
101
Pith papers citing it
Method
47% of classified citations
hub tools
citation-role summary
method 8
background 6
baseline 3
citation-polarity summary
claims ledger
- baseline 44 57.35 58.66 93.07 AnimateDiff-V2 [35] - 82.90 69.75 95.30 97.68 98.75 97.76 40.83 67.16 70.10 90.90 VideoCrafter-2.0 [10] - 82.20 73.42 96.85 98.22 98.41 97.73 42.50 63.13 67.22 92.55 CogVideoX [140] 5B 82.75 77.04 96.23 96.52 98.66 96.92 70.97 61.98 62.90 85.23 Kling [50] - 83.39 75.68 98.33 97.60 99.30 99.40 46.94 61.21 65.62 87.24 Open-Sora-2.0 [90] - 82.10 80.14 98.75 98.00 99.40 99.49 20.74 64.33 65.62 94.50 Gen-3 [96] - 84.11 75.17 97.10 96.62 98.61 99.23 60.14 63.34 66.82 87.81 Step-Vi
- background The code is available at https://github.com/ywlq/MotionCache. 1 Introduction Video generation models [12, 18, 24, 26, 34, 41, 47] have achieved remarkable success, facilitating applications ranging from autonomous driving [10, 11, 37] and cinematic creation [6, 38] to social media [3]. While architectures have evolved from U-Nets [2, 27, 29] to scalable Diffusion Transformers (DiTs) [25], practical deployment is hindered by the prohibitive costs of iterative denoising. Moreover, the quadratic co
- method DINOv2 [46] to extract frame-wise visual features. These features are further processed by a spatial-temporal transformer [ 47]. The decoder is implemented with standard transformer [ 48]. The latent action dimension n is set to 16. For RotVLA, the VLM backbone is initialized from InternVL3.5-1B [38], followed by an action expert implemented as a 24-layer Diffusion Transformer (DiT) [49]. The full model contains approximately 1.7B, including 304M in the vision encoder, 752M in the language model
- method reconstruction by enforcing finer supervision on local textures and detailed structures. Taking into account appearance and motion modeling simultaneously, we apply a hybrid discriminator with an architecture similar to that used in PatchGAN [11]. 2.2 Diffusion Transformer With the visual tokens encoded by VAE and text tokens generated by a text encoder, we employ the transformer as our diffusion backbone [20], where a fine-tuned decoder-only LLM as the text encoder. The visual tokens are then c
- baseline ConvNeXt-UNet Backbone for Microscopic Locality and Multi-Scale Structure The diffusion backbone determines how effectively masked-diffusion pretraining captures pathology morphology. For cell-level dense prediction, features must preserve nuclear contours, chromatin texture, thin boundaries, and multi-scale tissue context. We therefore compare DiT [32], Attention U-Net [33], and ConvNeXt-UNet under the same pretraining and evaluation protocol. DiT offers scalable global modeling, but patch toke
- method For both G and O, we can use the empirical training data distribution ˆpdata(G) and ˆpdata(O|G). By definingX :=(A,W,F, ℓ), our model can be written as pθ(G, O,Z,X) = ˆpdata(G) ˆpdata(O|G)p θFM(Z|G, O)p θD(X|Z, G, O),(5) where θ= (θ FM, θD) are the parameters of the flow matching and decoder neural networks. Similar to ADiT, we use a Diffusion Transformer (DiT) [30] as our denoiser network F, where conditional information gets added through adaptive layer norm. Moreover, we use self-conditioning
co-cited works
representative citing papers
MADField is a multi-fidelity amortized model for predicting density fields to improve accuracy and speed of adsorption calculations in nanoporous materials for high-throughput screening.
BloGDiT introduces blocked Gibbs-style denoising in diffusion transformers to enable large targeted edits for constraint satisfaction and optimization, matching or exceeding prior methods on Sudoku, graph coloring, MIS, and MaxCut.
Text tokens in FLUX.2 absorb reference image properties like color and style to influence outputs while pixel-exact details bypass them, localized to padding tokens via causal interventions.
StableHand introduces a quality-aware flow matching framework conditioned on predicted four-channel per-frame hand observation quality to estimate dual-hand world-space motion from egocentric video, achieving SOTA results with 20-25% W-MPJPE reduction on HOT3D and ARCTIC benchmarks.
KAN-SAE applies nonlinear per-feature B-spline activations in sparse autoencoders to discover 72% more alive climate features and interpretable patterns such as European heatwaves and Pacific typhoons in deep learning weather models.
StreamingEffect enables real-time 720p human-centric video effect generation on one GPU via teacher-student distillation, keyframe control, and a new 130K video dataset.
Echo-Forcing decouples stable anchors, compressed history, and recent dynamics in video diffusion KV caches using hierarchical memory, scene recall frames, and difference-aware decay to support interactive long video generation under bounded cache.
MechVerse benchmark shows current video generation models preserve appearance but fail at mechanically admissible motion, with errors rising as coupling complexity increases.
MM-SOLD is a training-free particle sampler whose large-particle limit converges to a moment-matched Gibbs distribution obtained by exponentially tilting a score-smoothed target.
RotVLA models latent actions as continuous SO(n) rotations with triplet-frame supervision and flow-matching to reach 98.2% success on LIBERO and 89.6%/88.5% on RoboTwin2.0 using a 1.7B-parameter model.
CausalCine enables real-time causal autoregressive multi-shot video generation via multi-shot training, content-aware memory routing for coherence, and distillation to few-step inference.
DRoRAE adaptively fuses multi-layer features from vision encoders via energy-constrained routing to enrich visual tokens, cutting rFID from 0.57 to 0.29 and generation FID from 1.74 to 1.65 on ImageNet-256 while revealing a log-linear scaling law with fusion capacity.
NoiseGate learns per-latent timestep schedules as an information-gating policy in diffusion-based world action models, yielding consistent gains on RoboTwin manipulation tasks.
ArenaPO infers Gaussian capability distributions from pairwise preferences and applies truncated-normal latent inference to derive fine-grained offline rewards for preference optimization of text-to-image diffusion models.
HDP3 is a pocket-scale 3D diffusion policy with a Diffusion Mixer decoder that achieves state-of-the-art visuomotor control using two-step DDIM inference and under 1% of the parameters of prior 3D diffusion policies.
Dream-Cubed releases a billion-scale voxel dataset and 3D diffusion models that generate controllable Minecraft worlds by operating directly on blocks.
Immune2V immunizes images against dual-stream I2V generation by enforcing temporally balanced latent divergence and aligning generative features to a precomputed collapse trajectory, yielding stronger persistent degradation than image-level baselines.
A standardized 16-task memory benchmark and 14 controlled VLA variants show perceptual memory (especially frame sampling + AdaLN modulation) is strongest overall, while symbolic subgoals win on counting and short-horizon reasoning.
InstructMoLE replaces per-token routing with instruction-guided global routing for mixture-of-low-rank-experts in diffusion transformers and adds an output-space orthogonality loss to improve multi-conditional image generation.
Dreamer 4 is the first agent to obtain diamonds in Minecraft from only offline data by reinforcement learning inside a scalable world model that accurately predicts game mechanics.
A feed-forward feature-Gaussian plus one-step geometry-aware pixel-flow simulator converts large image collections into 20K interactive scenes and 10M+ navigation samples that improve zero-shot Habitat and real-robot performance.
Fine-tuning a video generator on a new 17K reasoning-video dataset improves Chain-of-Frame reasoning, and adding learnable visual/textual reasoning tokens yields further gains on external benchmarks.
GORMPO uses generative models for density-based regularization in model-based offline RL, outperforming baselines by 17% on a medical dataset while providing theoretical guarantees under mild assumptions.
citing papers explorer
-
GEAR: Guided End-to-End AutoRegression for Image Synthesis
GEAR jointly trains VQ tokenizer and AR generator end-to-end via dual hard/soft read-out and representation alignment, achieving up to 10x faster ImageNet gFID convergence than LlamaGen-REPA while generalizing across quantizers and to text-to-image.
-
MADField: Multi-fidelity Amortized Density Field for Adsorption in Nanoporous Materials
MADField is a multi-fidelity amortized model for predicting density fields to improve accuracy and speed of adsorption calculations in nanoporous materials for high-throughput screening.
-
Blocked Gibbs meets Diffusion Transformers: Unsupervised Learning for Constraint Optimization
BloGDiT introduces blocked Gibbs-style denoising in diffusion transformers to enable large targeted edits for constraint satisfaction and optimization, matching or exceeding prior methods on Sudoku, graph coloring, MIS, and MaxCut.
-
Vision-Language Binding in In-Context Image Generation
Text tokens in FLUX.2 absorb reference image properties like color and style to influence outputs while pixel-exact details bypass them, localized to padding tokens via causal interventions.
-
StableHand: Quality-Aware Flow Matching for World-Space Dual-Hand Motion Estimation from Egocentric Video
StableHand introduces a quality-aware flow matching framework conditioned on predicted four-channel per-frame hand observation quality to estimate dual-hand world-space motion from egocentric video, achieving SOTA results with 20-25% W-MPJPE reduction on HOT3D and ARCTIC benchmarks.
-
Beyond Linear Superposition: Discovering Climate Features in AI Weather Models with KAN-SAE
KAN-SAE applies nonlinear per-feature B-spline activations in sparse autoencoders to discover 72% more alive climate features and interpretable patterns such as European heatwaves and Pacific typhoons in deep learning weather models.
-
StreamingEffect: Real-Time Human-Centric Video Effect Generation
StreamingEffect enables real-time 720p human-centric video effect generation on one GPU via teacher-student distillation, keyframe control, and a new 130K video dataset.
-
Echo-Forcing: A Scene Memory Framework for Interactive Long Video Generation
Echo-Forcing decouples stable anchors, compressed history, and recent dynamics in video diffusion KV caches using hierarchical memory, scene recall frames, and difference-aware decay to support interactive long video generation under bounded cache.
-
MechVerse: Evaluating Physical Motion Consistency in Video Generation Models
MechVerse benchmark shows current video generation models preserve appearance but fail at mechanically admissible motion, with errors rising as coupling complexity increases.
-
Training-Free Generative Sampling via Moment-Matched Score Smoothing
MM-SOLD is a training-free particle sampler whose large-particle limit converges to a moment-matched Gibbs distribution obtained by exponentially tilting a score-smoothed target.
-
RotVLA: Rotational Latent Action for Vision-Language-Action Model
RotVLA models latent actions as continuous SO(n) rotations with triplet-frame supervision and flow-matching to reach 98.2% success on LIBERO and 89.6%/88.5% on RoboTwin2.0 using a 1.7B-parameter model.
-
CausalCine: Real-Time Autoregressive Generation for Multi-Shot Video Narratives
CausalCine enables real-time causal autoregressive multi-shot video generation via multi-shot training, content-aware memory routing for coherence, and distillation to few-step inference.
-
Beyond the Last Layer: Multi-Layer Representation Fusion for Visual Tokenization
DRoRAE adaptively fuses multi-layer features from vision encoders via energy-constrained routing to enrich visual tokens, cutting rFID from 0.57 to 0.29 and generation FID from 1.74 to 1.65 on ImageNet-256 while revealing a log-linear scaling law with fusion capacity.
-
NoiseGate: Learning Per-Latent Timestep Schedules as Information Gating in World Action Models
NoiseGate learns per-latent timestep schedules as an information-gating policy in diffusion-based world action models, yielding consistent gains on RoboTwin manipulation tasks.
-
Arena as Offline Reward: Efficient Fine-Grained Preference Optimization for Diffusion Models
ArenaPO infers Gaussian capability distributions from pairwise preferences and applies truncated-normal latent inference to derive fine-grained offline rewards for preference optimization of text-to-image diffusion models.
-
Hyper-DP3: Frequency-Aware Right-Sizing of 3D Diffusion Policies for Visuomotor Control
HDP3 is a pocket-scale 3D diffusion policy with a Diffusion Mixer decoder that achieves state-of-the-art visuomotor control using two-step DDIM inference and under 1% of the parameters of prior 3D diffusion policies.
-
Dream-Cubed: Controllable Generative Modeling in Minecraft by Training on Billions of Cubes
Dream-Cubed releases a billion-scale voxel dataset and 3D diffusion models that generate controllable Minecraft worlds by operating directly on blocks.
-
Immune2V: Image Immunization Against Dual-Stream Image-to-Video Generation
Immune2V immunizes images against dual-stream I2V generation by enforcing temporally balanced latent divergence and aligning generative features to a precomputed collapse trajectory, yielding stronger persistent degradation than image-level baselines.
-
RoboMME: Benchmarking and Understanding Memory for Robotic Generalist Policies
A standardized 16-task memory benchmark and 14 controlled VLA variants show perceptual memory (especially frame sampling + AdaLN modulation) is strongest overall, while symbolic subgoals win on counting and short-horizon reasoning.
-
InstructMoLE: Instruction-Guided Mixture of Low-rank Experts for Multi-Conditional Image Generation
InstructMoLE replaces per-token routing with instruction-guided global routing for mixture-of-low-rank-experts in diffusion transformers and adds an output-space orthogonality loss to improve multi-conditional image generation.
-
Training Agents Inside of Scalable World Models
Dreamer 4 is the first agent to obtain diamonds in Minecraft from only offline data by reinforcement learning inside a scalable world model that accurately predicts game mechanics.
-
Image2Sim: Scaling Embodied Navigation via Generative Neural Simulator
A feed-forward feature-Gaussian plus one-step geometry-aware pixel-flow simulator converts large image collections into 20K interactive scenes and 10M+ navigation samples that improve zero-shot Habitat and real-robot performance.
-
OpenCoF: Learning to Reason Through Video Generation
Fine-tuning a video generator on a new 17K reasoning-video dataset improves Chain-of-Frame reasoning, and adding learnable visual/textual reasoning tokens yields further gains on external benchmarks.
-
Generative OOD-regularized Model-based Policy Optimization
GORMPO uses generative models for density-based regularization in model-based offline RL, outperforming baselines by 17% on a medical dataset while providing theoretical guarantees under mild assumptions.
-
Complete-muE: Optimal Hyperparameter Transfer and Scaling for MoE Models
Complete-muE combines active-width μP and activated-expert scaling to transfer hyperparameters across dense FFN, dense MoE, and sparse MoE while covering changes in experts, capacity, width, depth, batch size, and duration.
-
SCOPE: Simulating Cross-game Operations in Playable Environments for FPS World Models
SCOPE adds per-pixel action conditioning to pretrained video diffusion models and releases the CrossFPS multi-game dataset to support cross-game FPS world model simulation with zero-shot transfer.
-
SEGA: Spectral-Energy Guided Attention for Resolution Extrapolation in Diffusion Transformers
SEGA adaptively scales RoPE attention components using spectral-energy guidance from the latent to improve structural coherence and fine details in high-resolution DiT synthesis.
-
UniVL: Unified Vision-Language Embedding for Spatially Grounded Contextual Image Generation
UniVL unifies vision and language into one mask-rendered input processed by an OCR backbone to condition diffusion models for spatially grounded image generation without a standalone text encoder.
-
Spatial Gram Alignment for Ultra-High-Resolution Image Synthesis
Spatial Gram Alignment aligns internal self-similarities of LDM features with foundation priors to reconcile global structure and fine details in ultra-high-resolution text-to-image synthesis.
-
Rethinking Cross-Layer Information Routing in Diffusion Transformers
DAR replaces residual addition in DiTs with learnable, timestep-adaptive aggregation of sublayer outputs, yielding 2.11 FID improvement on SiT-XL/2 and 8.75x faster convergence on ImageNet 256x256.
-
Accelerating Video Inverse Problem Solvers with Autoregressive Diffusion Models
AVIS applies autoregressive diffusion models to video inverse problems by streaming restoration with measurement-consistent initialization, reducing latency from 114s to 4s and raising throughput to 1.18 FPS (or 5.91 FPS in the Flash variant).
-
CPC-VAR:Continual Personalized and Compositional Generation in Visual Autoregressive Models
CPC-VAR adds Gradient-based Concept Neuron Selection for continual single-concept learning and a context-aware multi-branch composition strategy to reduce forgetting and entanglement in VAR-based personalized image generation.
-
Rebalancing Reference Frame Dominance to Improve Motion in Image-to-Video Models
Reference-frame dominance in self-attention suppresses motion in image-to-video models; DyMoS rebalances attention from generated frames to the reference during initial denoising steps to improve dynamics while preserving fidelity.
-
Lance: Unified Multimodal Modeling by Multi-Task Synergy
Lance presents a dual-stream mixture-of-experts model with modality-aware positional encoding and staged multi-task training that outperforms prior open-source unified models on image and video generation while keeping strong understanding performance.
-
VISTA: Triplet-Supervised Video Style Transfer with Diffusion Transformers
VISTA introduces a new synthetic triplet dataset and diffusion-transformer framework with style adapter that jointly models style, content, and motion to achieve state-of-the-art video style transfer.
-
SemaVoice: Semantic-Aware Continuous Autoregressive Speech Synthesis
SemaVoice adds SFM-guided alignment to refine continuous speech representations in autoregressive TTS, reporting 1.71% English WER on Seed-TTS and competitiveness with open-source SOTA.
-
Beyond Point-Wise Matching: Structural Representation Alignment for Accelerating Diffusion Transformers
sREPA enforces structural consistency in relational geometry of pre-trained vision features to accelerate DiT training and improve generation quality.
-
DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformers
DiRotQ uses PCA-based rotation-aware activation quantization combined with GPTQ to achieve better FID and PSNR in 4-bit diffusion transformers than prior methods like SVDQuant.
-
Flash-GRPO: Efficient Alignment for Video Diffusion via One-Step Policy Optimization
Flash-GRPO is a one-step GRPO framework for video diffusion alignment that applies iso-temporal grouping and temporal gradient rectification to achieve higher alignment quality and stability than full-trajectory training under low compute budgets on 1.3B-14B models.
-
Efficient Image Synthesis with Sphere Latent Encoder
Decouples Sphere Encoder into fixed pretrained encoder and spherical latent denoiser, yielding higher quality and faster inference than the joint original on Animal-Faces, Oxford-Flowers and ImageNet-1K.
-
ReactiveGWM: Steering NPC in Reactive Game World Models
ReactiveGWM introduces a decoupled diffusion architecture for player-NPC interactions that learns game-agnostic response logic for zero-shot strategy transfer across games.
-
Nonlinear Bipolar Compensation: Handling Outliers in Post-Training Quantization
Nonlinear Bipolar Compensation with Bipolar Logarithmic Transformation reduces outlier effects in post-training quantization by performing compensation in a compressed transformed space.
-
Crys-JEPA: Accelerating Crystal Discovery via Embedding Screening and Generative Refinement
Crys-JEPA introduces a joint embedding predictive architecture that creates an energy-aware latent space, enabling embedding-based stability screening and a refinement pipeline that yields up to 72.7% gains on the V.S.U.N. metric for crystal generation.
-
Early Semantic Grounding in Image Editing Models for Zero-Shot Referring Image Segmentation
Pretrained instruction-based image editing models exhibit early foreground-background separability that enables a training-free framework for zero-shot referring image segmentation using a single denoising step.
-
SocialDirector: Training-Free Social Interaction Control for Multi-Person Video Generation
SocialDirector uses spatiotemporal actor masking and directional reweighting on cross-attention maps to reduce actor-action mismatches and improve target-directed interactions in generated multi-person videos.
-
Yeti: A compact protein structure tokenizer for reconstruction and multi-modal generation
Yeti is a compact tokenizer for protein structures that delivers strong codebook use, token diversity, and reconstruction while enabling from-scratch multimodal generation of plausible sequences and structures with 10x fewer parameters than ESM3.
-
What Matters for Diffusion-Friendly Latent Manifold? Prior-Aligned Autoencoders for Latent Diffusion
Prior-Aligned AutoEncoders shape latent manifolds with spatial coherence, local continuity, and global semantics to improve latent diffusion, achieving SOTA gFID 1.03 on ImageNet 256x256 with up to 13x faster convergence.
-
Continuous Latent Diffusion Language Model
Cola DLM proposes a hierarchical latent diffusion model that learns a text-to-latent mapping, fits a global semantic prior in continuous space with a block-causal DiT, and performs conditional decoding, establishing latent prior modeling as an alternative to token-level autoregressive language model
-
Beyond Autoregressive RTG: Conditioning via Injection Outside Sequential Modeling in Decision Transformer
Injecting RTG into states outside the autoregressive sequence yields shorter, more efficient Decision Transformers that outperform the original on offline RL tasks.
-
SDFlow: Similarity-Driven Flow Matching for Time Series Generation
SDFlow learns a global transport map via similarity-driven flow matching in VQ latent space, using low-rank manifold decomposition and a categorical posterior to handle discreteness, yielding SOTA long-horizon performance and inference speedups.