{"work":{"id":"a3a05169-18b1-42bb-8775-eada50163437","openalex_id":"https://openalex.org/W4312057601","doi":"10.48550/arxiv.2212.09748","arxiv_id":"2212.09748","raw_key":null,"title":"Scalable Diffusion Models with Transformers","authors":null,"authors_text":"William Peebles, Saining Xie","year":2022,"venue":"cs.CV","abstract":"We explore a new class of diffusion models based on the transformer architecture. We train latent diffusion models of images, replacing the commonly-used U-Net backbone with a transformer that operates on latent patches. We analyze the scalability of our Diffusion Transformers (DiTs) through the lens of forward pass complexity as measured by Gflops. We find that DiTs with higher Gflops -- through increased transformer depth/width or increased number of input tokens -- consistently have lower FID. In addition to possessing good scalability properties, our largest DiT-XL/2 models outperform all prior diffusion models on the class-conditional ImageNet 512x512 and 256x256 benchmarks, achieving a state-of-the-art FID of 2.27 on the latter.","external_url":"https://arxiv.org/abs/2212.09748","cited_by_count":42,"metadata_source":"pith","metadata_fetched_at":"2026-08-05T02:28:24.338817+00:00","pith_arxiv_id":"2212.09748","created_at":"2026-05-09T06:25:47.538776+00:00","updated_at":"2026-08-05T02:28:24.338817+00:00","title_quality_ok":true,"display_title":"Scalable Diffusion Models with Transformers","render_title":"Scalable Diffusion Models with Transformers"},"hub":{"state":{"work_id":"a3a05169-18b1-42bb-8775-eada50163437","tier":"super_hub","tier_reason":"100+ Pith inbound or 10,000+ external citations","pith_inbound_count":123,"external_cited_by_count":42,"distinct_field_count":13,"first_pith_cited_at":"2023-07-04T23:04:57+00:00","last_pith_cited_at":"2026-07-09T17:59:11+00:00","author_build_status":"needed","summary_status":"needed","contexts_status":"needed","graph_status":"needed","ask_index_status":"needed","reader_status":"not_needed","recognition_status":"not_needed","updated_at":"2026-08-21T19:49:24.536192+00:00","tier_text":"super_hub"},"tier":"super_hub","role_counts":[{"context_role":"background","n":10},{"context_role":"method","n":8},{"context_role":"baseline","n":1},{"context_role":"dataset","n":1}],"polarity_counts":[{"context_polarity":"background","n":10},{"context_polarity":"use_method","n":8},{"context_polarity":"baseline","n":1},{"context_polarity":"use_dataset","n":1}],"runs":{"ask_index":{"job_type":"ask_index","status":"succeeded","result":{"title":"Scalable Diffusion Models with Transformers","claims":[{"claim_text":"We explore a new class of diffusion models based on the transformer architecture. We train latent diffusion models of images, replacing the commonly-used U-Net backbone with a transformer that operates on latent patches. We analyze the scalability of our Diffusion Transformers (DiTs) through the lens of forward pass complexity as measured by Gflops. We find that DiTs with higher Gflops -- through increased transformer depth/width or increased number of input tokens -- consistently have lower FID. In addition to possessing good scalability properties, our largest DiT-XL/2 models outperform all ","claim_type":"abstract","evidence_strength":"source_metadata"},{"claim_text":"From Syntax to Semantics: Unveiling the Emergence of Chirality in SMILES Translation Models decreased more slowly than ZINC20 ( ∼100) to maintain structural diversity. Detailed algorithms are provided in Supplementary Section. A. Training was conducted across four NVIDIA H100 GPUs with a global batch size of 2048 SMILES per step. We utilized the RAdamScheduleFree optimizer [48] (lr=3.0e-4, r=0.5, weight_decay=0.0) combined with ZClip [49] for adaptive gradient clipping. We defined one epoch as 1","claim_type":"method","confidence":0.95,"evidence_strength":"citation_context"},{"claim_text":"DiT-XL/2Δ𝑔∗ [53] 2.3 278.2 675M 250 59.5k MDTΔ𝑔∗ [18] 1.8 283.0 676M 250>59k MaskDiTΔ𝑔∗ [82] 2.3 276.6 736M 250>28k CDMΔ [30] 4.9 158.7 - 8100 - RINΔ [36] 3.4 182.0 410M 1000 334k Simple DiffusionΔ𝑔 [33] 2.4 256.3 2B 512 - VDM++Δ𝑔 [39] 2.1 267.7 2B 512 - EDiffΔ𝑔 [25] 2.1 - 450M 50 119k LPDM-ADMΔ𝑔 [72] 2.7 - - 50 7.8k MARΔ𝑔 [44]✓1.8 296.0 479M 128 - VQVAE-2Δ [55]✓31.1∼45 13.5B 5120 - VQGANΔ [15]✓15.8 78.3 1.4B 256 - MaskGITΔ[7] 6.2 182.1 227M 8 647 Mo-VQGANΔ [81] 7.2 130.1 389M 12∼1k MaskBitΔ𝑔 [7","claim_type":"baseline","confidence":0.95,"evidence_strength":"citation_context"},{"claim_text":"Moreover, following [11], we perform the LoRA-based distillation method to accelerate the decoding by about 10×,i.e., from50denoised steps to4, without sacrificing performance. Video decoder.We extend Emu3.5 to generate continuous videos with a diffusion-based video decoder condi- tioned on the generated keyframe tokens. Our video decoder is built upon the mainstream DiT [69] architecture. We utilize quantized embeddings from the VQ quantizer to provide fine-grained visual details, while optiona","claim_type":"method","confidence":0.9,"evidence_strength":"citation_context"},{"claim_text":"Video Diffusion Models [16] and CogVideoX [17]. Recent open-source implementations including HunyuanVideo [18], StepVideo [19], SkyReels-V1[20], OpenSora-2.0 [21], and Wan2.1 [5] demonstrate progressively diminishing quality disparities with their proprietary counterparts. These improvements stem from multi-faceted innovations: architectural transitions from U-Net [ 22] to DiT [23] or MMDiT [24] structures, enhanced V AE implementations [25, 26, 27, 28, 29, 18, 5], upgraded text encoders [30, 31","claim_type":"background","confidence":0.9,"evidence_strength":"citation_context"},{"claim_text":"0 form a comprehensive benchmarking system, driving the field beyond superficial faithfulness towards truly intrinsically faithful video generation. II. R ELATED WORKS A. Video Generative Models With the advancements in diffusion models [33]-[44], vari- ational autoencoder-based compression techniques [45]-[49], and transformer architectures [50], [51], video generation has emerged as one of the most dynamic frontiers in artificial in- telligence research. Prior to Sora's breakthrough, predomina","claim_type":"background","confidence":0.9,"evidence_strength":"citation_context"},{"claim_text":"nally, we apply depth-threshold filtering and VLM verification [40] to remove samples with incorrect masks or insufficient 3D dis- placement. Each pair is further annotated with expanded prompts that describe non-geometric appearance changes (e.g., color or background variation), facilitating better text-image alignment for subsequent training. 4 Experiments 4.1 Training Details We use Qwen-Image-Edit [38] as the DiT editing backbone and Depth-Anything-3 [26] as the 3D foundation model. We fine-","claim_type":"method","confidence":0.9,"evidence_strength":"citation_context"}],"why_cited":"Pith tracks Scalable Diffusion Models with Transformers because it crossed a citation-hub threshold. Current citing contexts most often use it as background evidence (10 contexts).","role_counts":[{"n":10,"context_role":"background"},{"n":8,"context_role":"method"},{"n":1,"context_role":"baseline"},{"n":1,"context_role":"dataset"}]},"error":null,"updated_at":"2026-07-01T23:32:01.132450+00:00"},"author_expand":{"job_type":"author_expand","status":"succeeded","result":{"authors_linked":[{"id":"5b0a1abf-4de4-4e09-988e-cbfaf6809f9c","orcid":null,"display_name":"William Peebles"},{"id":"b66e630a-75b7-48e2-b814-2ce458daf5da","orcid":null,"display_name":"Saining Xie"}]},"error":null,"updated_at":"2026-07-01T23:32:01.124168+00:00"},"context_extract":{"job_type":"context_extract","status":"succeeded","result":{"enqueued_papers":25},"error":null,"updated_at":"2026-05-14T18:29:55.959872+00:00"},"graph_features":{"job_type":"graph_features","status":"succeeded","result":{"co_cited":[{"title":"Classifier-Free Diffusion Guidance","work_id":"acf2c588-c088-4a6c-938e-150ad7c666d7","shared_citers":7},{"title":"High-Resolution Image Synthesis with Latent Diffusion Models","work_id":"f0270d36-2952-47fb-84c1-95e3ec341126","shared_citers":7},{"title":"Wan: Open and Advanced Large-Scale Video Generative Models","work_id":"ad3ebc3b-4224-46c9-b61d-bcf135da0a7c","shared_citers":7},{"title":"CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer","work_id":"f38fc088-12aa-4bf4-9ecd-08d3e797ccb7","shared_citers":6},{"title":"Flow Matching for Generative Modeling","work_id":"6edb71c4-5d64-40af-a394-9757ea051a36","shared_citers":6},{"title":"$\\pi_0$: A Vision-Language-Action Flow Model for General Robot Control","work_id":"f790abdc-a796-482f-a40d-f8ee035ecfc2","shared_citers":5},{"title":"Hierarchical Text-Conditional Image Generation with CLIP Latents","work_id":"0c6a768b-70b8-4242-bb0e-459f1008c9fc","shared_citers":5},{"title":"Denoising Diffusion Implicit Models","work_id":"8fa2128b-d18c-405c-ac92-0e669cf89ac0","shared_citers":4},{"title":"Denoising Diffusion Probabilistic Models","work_id":"dc023f4e-7c79-471c-b713-deeb559ba010","shared_citers":4},{"title":"OpenVLA: An Open-Source Vision-Language-Action Model","work_id":"3e7e65c5-5aed-4fe9-8414-2092bcb31cc7","shared_citers":4},{"title":"Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding","work_id":"af16442b-a46f-469d-8818-c37b53a504c7","shared_citers":4},{"title":"Scaling Rectified Flow Transformers for High-Resolution Image Synthesis","work_id":"4dc55d76-271e-42dd-878f-c20546599c69","shared_citers":4},{"title":"Score-Based Generative Modeling through Stochastic Differential Equations","work_id":"d9110e53-a5d4-4794-a4c5-a575e91c31ad","shared_citers":4},{"title":"SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis","work_id":"8034c587-fba6-4941-87ba-c98f2ac962cb","shared_citers":4},{"title":"$\\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization","work_id":"d1ad7304-d09a-49bc-809e-846439f6aff9","shared_citers":3},{"title":"arXiv preprint arXiv:2407.02371 (2024)","work_id":"00dd95a8-d503-4a41-9603-785dcf4a0b8e","shared_citers":3},{"title":"Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation","work_id":"41efe203-9377-4c63-b1d6-e499cd6e46f6","shared_citers":3},{"title":"Chameleon: Mixed-Modal Early-Fusion Foundation Models","work_id":"2661b9a6-25cc-41a1-8100-612d2b801289","shared_citers":3},{"title":"ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment","work_id":"94248955-4bc5-4517-98a0-66224a36d865","shared_citers":3},{"title":"Emerging Properties in Unified Multimodal Pretraining","work_id":"e0cfd82c-f5d4-44fd-b531-ec73ab0a805b","shared_citers":3},{"title":"Gemini: A Family of Highly Capable Multimodal Models","work_id":"83f7c85b-3f11-450f-ac0c-64d9745220b2","shared_citers":3},{"title":"Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling","work_id":"67d9e391-26d1-459e-ab56-07e60511c886","shared_citers":3},{"title":"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware","work_id":"6fe159e0-fa73-481a-88d4-4719c15140be","shared_citers":3},{"title":"LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning","work_id":"662203ad-084f-42c4-8e60-977b3173755b","shared_citers":3}],"time_series":[{"n":2,"year":2023},{"n":1,"year":2024},{"n":2,"year":2025},{"n":27,"year":2026}],"dependency_candidates":[]},"error":null,"updated_at":"2026-05-14T18:30:00.005902+00:00"},"identity_refresh":{"job_type":"identity_refresh","status":"succeeded","result":{"items":[{"title":"Qwen3 Technical Report","outcome":"unchanged","work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e","resolver":"local_arxiv","confidence":0.98,"old_work_id":"25a4e30c-1232-48e7-9925-02fa12ba7c9e"}],"counts":{"fixed":0,"merged":0,"unchanged":1,"quarantined":0,"needs_external_resolution":0},"errors":[],"attempted":1},"error":null,"updated_at":"2026-05-14T18:29:24.764062+00:00"},"role_polarity":{"job_type":"role_polarity","status":"succeeded","result":{"title":"Scalable Diffusion Models with Transformers","claims":[{"claim_text":"We explore a new class of diffusion models based on the transformer architecture. We train latent diffusion models of images, replacing the commonly-used U-Net backbone with a transformer that operates on latent patches. We analyze the scalability of our Diffusion Transformers (DiTs) through the lens of forward pass complexity as measured by Gflops. We find that DiTs with higher Gflops -- through increased transformer depth/width or increased number of input tokens -- consistently have lower FID. In addition to possessing good scalability properties, our largest DiT-XL/2 models outperform all ","claim_type":"abstract","evidence_strength":"source_metadata"},{"claim_text":"From Syntax to Semantics: Unveiling the Emergence of Chirality in SMILES Translation Models decreased more slowly than ZINC20 ( ∼100) to maintain structural diversity. Detailed algorithms are provided in Supplementary Section. A. Training was conducted across four NVIDIA H100 GPUs with a global batch size of 2048 SMILES per step. We utilized the RAdamScheduleFree optimizer [48] (lr=3.0e-4, r=0.5, weight_decay=0.0) combined with ZClip [49] for adaptive gradient clipping. We defined one epoch as 1","claim_type":"method","confidence":0.95,"evidence_strength":"citation_context"},{"claim_text":"DiT-XL/2Δ𝑔∗ [53] 2.3 278.2 675M 250 59.5k MDTΔ𝑔∗ [18] 1.8 283.0 676M 250>59k MaskDiTΔ𝑔∗ [82] 2.3 276.6 736M 250>28k CDMΔ [30] 4.9 158.7 - 8100 - RINΔ [36] 3.4 182.0 410M 1000 334k Simple DiffusionΔ𝑔 [33] 2.4 256.3 2B 512 - VDM++Δ𝑔 [39] 2.1 267.7 2B 512 - EDiffΔ𝑔 [25] 2.1 - 450M 50 119k LPDM-ADMΔ𝑔 [72] 2.7 - - 50 7.8k MARΔ𝑔 [44]✓1.8 296.0 479M 128 - VQVAE-2Δ [55]✓31.1∼45 13.5B 5120 - VQGANΔ [15]✓15.8 78.3 1.4B 256 - MaskGITΔ[7] 6.2 182.1 227M 8 647 Mo-VQGANΔ [81] 7.2 130.1 389M 12∼1k MaskBitΔ𝑔 [7","claim_type":"baseline","confidence":0.95,"evidence_strength":"citation_context"},{"claim_text":"Moreover, following [11], we perform the LoRA-based distillation method to accelerate the decoding by about 10×,i.e., from50denoised steps to4, without sacrificing performance. Video decoder.We extend Emu3.5 to generate continuous videos with a diffusion-based video decoder condi- tioned on the generated keyframe tokens. Our video decoder is built upon the mainstream DiT [69] architecture. We utilize quantized embeddings from the VQ quantizer to provide fine-grained visual details, while optiona","claim_type":"method","confidence":0.9,"evidence_strength":"citation_context"},{"claim_text":"Video Diffusion Models [16] and CogVideoX [17]. Recent open-source implementations including HunyuanVideo [18], StepVideo [19], SkyReels-V1[20], OpenSora-2.0 [21], and Wan2.1 [5] demonstrate progressively diminishing quality disparities with their proprietary counterparts. These improvements stem from multi-faceted innovations: architectural transitions from U-Net [ 22] to DiT [23] or MMDiT [24] structures, enhanced V AE implementations [25, 26, 27, 28, 29, 18, 5], upgraded text encoders [30, 31","claim_type":"background","confidence":0.9,"evidence_strength":"citation_context"},{"claim_text":"0 form a comprehensive benchmarking system, driving the field beyond superficial faithfulness towards truly intrinsically faithful video generation. II. R ELATED WORKS A. Video Generative Models With the advancements in diffusion models [33]-[44], vari- ational autoencoder-based compression techniques [45]-[49], and transformer architectures [50], [51], video generation has emerged as one of the most dynamic frontiers in artificial in- telligence research. Prior to Sora's breakthrough, predomina","claim_type":"background","confidence":0.9,"evidence_strength":"citation_context"},{"claim_text":"nally, we apply depth-threshold filtering and VLM verification [40] to remove samples with incorrect masks or insufficient 3D dis- placement. Each pair is further annotated with expanded prompts that describe non-geometric appearance changes (e.g., color or background variation), facilitating better text-image alignment for subsequent training. 4 Experiments 4.1 Training Details We use Qwen-Image-Edit [38] as the DiT editing backbone and Depth-Anything-3 [26] as the 3D foundation model. We fine-","claim_type":"method","confidence":0.9,"evidence_strength":"citation_context"}],"why_cited":"Pith tracks Scalable Diffusion Models with Transformers because it crossed a citation-hub threshold. Current citing contexts most often use it as background evidence (10 contexts).","role_counts":[{"n":10,"context_role":"background"},{"n":8,"context_role":"method"},{"n":1,"context_role":"baseline"},{"n":1,"context_role":"dataset"}]},"error":null,"updated_at":"2026-07-01T23:32:01.129123+00:00"},"summary_claims":{"job_type":"summary_claims","status":"succeeded","result":{"title":"Scalable Diffusion Models with Transformers","claims":[{"claim_text":"We explore a new class of diffusion models based on the transformer architecture. We train latent diffusion models of images, replacing the commonly-used U-Net backbone with a transformer that operates on latent patches. We analyze the scalability of our Diffusion Transformers (DiTs) through the lens of forward pass complexity as measured by Gflops. We find that DiTs with higher Gflops -- through increased transformer depth/width or increased number of input tokens -- consistently have lower FID. In addition to possessing good scalability properties, our largest DiT-XL/2 models outperform all ","claim_type":"abstract","evidence_strength":"source_metadata"}],"why_cited":"Pith tracks Scalable Diffusion Models with Transformers because it crossed a citation-hub threshold.","role_counts":[]},"error":null,"updated_at":"2026-05-14T18:29:39.056531+00:00"}},"summary":{"title":"Scalable Diffusion Models with Transformers","claims":[{"claim_text":"We explore a new class of diffusion models based on the transformer architecture. We train latent diffusion models of images, replacing the commonly-used U-Net backbone with a transformer that operates on latent patches. We analyze the scalability of our Diffusion Transformers (DiTs) through the lens of forward pass complexity as measured by Gflops. We find that DiTs with higher Gflops -- through increased transformer depth/width or increased number of input tokens -- consistently have lower FID. In addition to possessing good scalability properties, our largest DiT-XL/2 models outperform all ","claim_type":"abstract","evidence_strength":"source_metadata"}],"why_cited":"Pith tracks Scalable Diffusion Models with Transformers because it crossed a citation-hub threshold.","role_counts":[]},"graph":{"co_cited":[{"title":"Classifier-Free Diffusion Guidance","work_id":"acf2c588-c088-4a6c-938e-150ad7c666d7","shared_citers":7},{"title":"High-Resolution Image Synthesis with Latent Diffusion Models","work_id":"f0270d36-2952-47fb-84c1-95e3ec341126","shared_citers":7},{"title":"Wan: Open and Advanced Large-Scale Video Generative Models","work_id":"ad3ebc3b-4224-46c9-b61d-bcf135da0a7c","shared_citers":7},{"title":"CogVideoX: Text-to-Video Diffusion Models with An Expert Transformer","work_id":"f38fc088-12aa-4bf4-9ecd-08d3e797ccb7","shared_citers":6},{"title":"Flow Matching for Generative Modeling","work_id":"6edb71c4-5d64-40af-a394-9757ea051a36","shared_citers":6},{"title":"$\\pi_0$: A Vision-Language-Action Flow Model for General Robot Control","work_id":"f790abdc-a796-482f-a40d-f8ee035ecfc2","shared_citers":5},{"title":"Hierarchical Text-Conditional Image Generation with CLIP Latents","work_id":"0c6a768b-70b8-4242-bb0e-459f1008c9fc","shared_citers":5},{"title":"Denoising Diffusion Implicit Models","work_id":"8fa2128b-d18c-405c-ac92-0e669cf89ac0","shared_citers":4},{"title":"Denoising Diffusion Probabilistic Models","work_id":"dc023f4e-7c79-471c-b713-deeb559ba010","shared_citers":4},{"title":"OpenVLA: An Open-Source Vision-Language-Action Model","work_id":"3e7e65c5-5aed-4fe9-8414-2092bcb31cc7","shared_citers":4},{"title":"Photorealistic Text-to-Image Diffusion Models with Deep Language Understanding","work_id":"af16442b-a46f-469d-8818-c37b53a504c7","shared_citers":4},{"title":"Scaling Rectified Flow Transformers for High-Resolution Image Synthesis","work_id":"4dc55d76-271e-42dd-878f-c20546599c69","shared_citers":4},{"title":"Score-Based Generative Modeling through Stochastic Differential Equations","work_id":"d9110e53-a5d4-4794-a4c5-a575e91c31ad","shared_citers":4},{"title":"SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis","work_id":"8034c587-fba6-4941-87ba-c98f2ac962cb","shared_citers":4},{"title":"$\\pi_{0.5}$: a Vision-Language-Action Model with Open-World Generalization","work_id":"d1ad7304-d09a-49bc-809e-846439f6aff9","shared_citers":3},{"title":"arXiv preprint arXiv:2407.02371 (2024)","work_id":"00dd95a8-d503-4a41-9603-785dcf4a0b8e","shared_citers":3},{"title":"Autoregressive Model Beats Diffusion: Llama for Scalable Image Generation","work_id":"41efe203-9377-4c63-b1d6-e499cd6e46f6","shared_citers":3},{"title":"Chameleon: Mixed-Modal Early-Fusion Foundation Models","work_id":"2661b9a6-25cc-41a1-8100-612d2b801289","shared_citers":3},{"title":"ELLA: Equip Diffusion Models with LLM for Enhanced Semantic Alignment","work_id":"94248955-4bc5-4517-98a0-66224a36d865","shared_citers":3},{"title":"Emerging Properties in Unified Multimodal Pretraining","work_id":"e0cfd82c-f5d4-44fd-b531-ec73ab0a805b","shared_citers":3},{"title":"Gemini: A Family of Highly Capable Multimodal Models","work_id":"83f7c85b-3f11-450f-ac0c-64d9745220b2","shared_citers":3},{"title":"Janus-Pro: Unified Multimodal Understanding and Generation with Data and Model Scaling","work_id":"67d9e391-26d1-459e-ab56-07e60511c886","shared_citers":3},{"title":"Learning Fine-Grained Bimanual Manipulation with Low-Cost Hardware","work_id":"6fe159e0-fa73-481a-88d4-4719c15140be","shared_citers":3},{"title":"LIBERO: Benchmarking Knowledge Transfer for Lifelong Robot Learning","work_id":"662203ad-084f-42c4-8e60-977b3173755b","shared_citers":3}],"time_series":[{"n":2,"year":2023},{"n":1,"year":2024},{"n":2,"year":2025},{"n":27,"year":2026}],"dependency_candidates":[]},"authors":[{"id":"b66e630a-75b7-48e2-b814-2ce458daf5da","orcid":null,"display_name":"Saining Xie","source":"manual","import_confidence":0.72},{"id":"5b0a1abf-4de4-4e09-988e-cbfaf6809f9c","orcid":null,"display_name":"William Peebles","source":"manual","import_confidence":0.72}]}}