{"paper":{"title":"DiRotQ: Rotation-Aware Quantization for 4-bit Diffusion Transformers","license":"http://creativecommons.org/licenses/by-nc-sa/4.0/","headline":"DiRotQ rotates activations into a PCA-derived basis to protect dominant variance directions during 4-bit quantization of diffusion transformers.","cross_cats":["cs.LG"],"primary_cat":"cs.CV","authors_text":"Hesham Mostafa, Mahsa Salmani, Sayeh Sharify","submitted_at":"2026-05-16T00:52:00Z","abstract_excerpt":"Diffusion Transformers (DiTs) achieve state-of-the-art image generation quality but incur substantial memory and computational costs at inference. While aggressive Post-Training Quantization (PTQ) to 4-bit precision offers significant efficiency gains, it typically results in severe quality degradation. Existing approaches, including smoothing-based methods, mixed-precision schemes, rotation techniques, and low-rank residual methods, partially mitigate this issue but still leave a noticeable gap to FP16/BF16 performance. In this work, we introduce DiRotQ, a W4A4 PTQ framework that mitigates th"},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"DiRotQ achieves an FID of 15.9 and PSNR of 19.1 dB on PixArt-Σ over the MJHQ-30K dataset, outperforming the prior state-of-the-art SVDQuant (FID 18.9, PSNR 17.6) under the same INT W4A4 setting.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"That a low-rank subspace identified via PCA on calibration data captures the dominant activation variance sufficiently to allow safe 4-bit quantization of the remaining components, with the rotation and fusion preserving overall model behavior.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"DiRotQ uses PCA-based rotation-aware activation quantization combined with GPTQ to achieve better FID and PSNR in 4-bit diffusion transformers than prior methods like SVDQuant.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"DiRotQ rotates activations into a PCA-derived basis to protect dominant variance directions during 4-bit quantization of diffusion transformers.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"d9912b65fa94ef32d0c9cb0baa7732eff7c6d72b4fe1eea23ac9a6a814e1a491"},"source":{"id":"2605.16732","kind":"arxiv","version":1},"verdict":{"id":"aef830d5-40f5-40fd-b028-09e5d4be9f53","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-19T21:43:48.527133Z","strongest_claim":"DiRotQ achieves an FID of 15.9 and PSNR of 19.1 dB on PixArt-Σ over the MJHQ-30K dataset, outperforming the prior state-of-the-art SVDQuant (FID 18.9, PSNR 17.6) under the same INT W4A4 setting.","one_line_summary":"DiRotQ uses PCA-based rotation-aware activation quantization combined with GPTQ to achieve better FID and PSNR in 4-bit diffusion transformers than prior methods like SVDQuant.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"That a low-rank subspace identified via PCA on calibration data captures the dominant activation variance sufficiently to allow safe 4-bit quantization of the remaining components, with the rotation and fusion preserving overall model behavior.","pith_extraction_headline":"DiRotQ rotates activations into a PCA-derived basis to protect dominant variance directions during 4-bit quantization of diffusion transformers."},"integrity":{"clean":true,"summary":{"advisory":0,"critical":0,"by_detector":{},"informational":0},"endpoint":"/pith/2605.16732/integrity.json","findings":[],"available":true,"detectors_run":[{"name":"doi_title_agreement","ran_at":"2026-05-19T22:01:19.927534Z","status":"completed","version":"1.0.0","findings_count":0},{"name":"doi_compliance","ran_at":"2026-05-19T21:51:05.260680Z","status":"completed","version":"1.0.0","findings_count":0},{"name":"claim_evidence","ran_at":"2026-05-19T19:01:56.341113Z","status":"completed","version":"1.0.0","findings_count":0},{"name":"ai_meta_artifact","ran_at":"2026-05-19T18:33:26.468811Z","status":"skipped","version":"1.0.0","findings_count":0}],"snapshot_sha256":"abfefa1104e06423f04afcd00b991538097cc5c9024f72f02ed87793db7d168f"},"references":{"count":88,"sample":[{"doi":"","year":2024,"title":"QuaRot: Outlier-free 4-bit inference in rotated LLMs","work_id":"ab5f8849-990f-4007-8756-dd0c2c3abb63","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2022,"title":"Blended diffusion for text-driven editing of natural images","work_id":"1032523a-0ace-4af7-aef1-c93f591d8907","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2022,"title":"eDiff-I: Text-to-Image Diffusion Models with an Ensemble of Expert Denoisers","work_id":"2cd7b629-ab37-4ce5-b51e-aa4d99547468","ref_index":3,"cited_arxiv_id":"2211.01324","is_internal_anchor":true},{"doi":"","year":2023,"title":"All are worth words: A ViT backbone for diffusion models","work_id":"90c08521-811b-46c4-a99e-ecb7e85825e5","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2022,"title":"Token Merging: Your ViT But Faster","work_id":"528509bc-2611-4e7f-a772-ea14d25b6dae","ref_index":5,"cited_arxiv_id":"2210.09461","is_internal_anchor":true}],"resolved_work":88,"snapshot_sha256":"6e08cace26c966b27aeccf235baf6f1b52a60f28cbbacfe11d3daf5708dd47d1","internal_anchors":16},"formal_canon":{"evidence_count":1,"snapshot_sha256":"9007d3da0dd2a29afcc76c7cd7562c4b0b2fc1dd207c30d008ad42d980f6ce22"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}