{"id":"bd4d91e8-d407-4c40-a7c6-30fef25f7e4b","arxiv_id":"2505.13219","paper_version":5,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"PiT replaces global attention in diffusion transformers with window attention plus a depthwise-conv bridging branch and a progressive channel schedule, achieving better FID than DiT and U-DiT at lower or similar compute.","lead":"Diffusion Transformer image generators spend most of their computation on far-range attention that the authors find is rarely needed. Their new design mixes window attention with a convolution branch and a channel-allocation schedule, reporting better ImageNet FID scores at lower FLOPs.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"PCCA's no-cost claim is unverified: transferring channels from DWConv to window attention can raise FLOPs, confounding the Kth-order ablation with added attention compute.","rationale":"I read the paper as making two linked claims: a full-architecture efficiency claim (PiT-L beats DiT-XL/2 with less FLOPs) and a mechanism claim (PSWA's high-frequency bridging and PCCA's Kth-order attention are responsible). The full-architecture claim is credible: Table 4 shows a controlled swap of MHSA for PSWA improves isotropic DiTs by roughly 8-9% FID at lower FLOPs, and Table 5 shows modest gains at 512. The weak spot is PCCA. The paper never reports FLOPs for the Table 7 ablations, and the direction of the best schedule aligns with spending more compute on attention, not with the proposed Kth-order aggregation per se. This is load-bearing because the paper explicitly claims the strategy adds no computational cost, and because the marginal gain over U-DiT-L is small enough that a compute confound could account for it. The reader's verdict of CONDITIONAL is appropriate; I would add the FLOPs/control check to the conditions rather than changing the verdict. I do not see grounds for REJECT: the core architecture is plausible and partially supported by controlled experiments, and no result in the paper is internally contradictory except the supplementary theorem already flagged by the reader.","tokens_in":13744,"tokens_out":14425,"duration_ms":139138,"concrete_test":"Run the five Table 7 channel allocations in the released (or reimplemented) code with per-layer FLOPs logged; verify whether the total FLOPs of the Cwin-increasing schedule exceed those of the equal or Cwin-decreasing schedules. Then run a control with the same PCCA schedule but feed the transferred channels from a linear copy or a random permutation of the previous layer's conv features instead of the 5x5 DWConv output, keeping FLOPs identical; if this control matches the row-5 FID within noise, the gain does not come from Kth-order similarity. If code is unavailable, an analytic FLOP derivation using the actual window size and 5x5 kernel can already settle the first part.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Section PCCA asserts that PCCA captures Kth-order attention without any additional computational cost, and Equation 11 credits the FID gain to Kth-order similarity carried by channels transferred from the 5x5 DWConv branch to the static window attention branch. The two branches have different per-channel costs: window attention costs roughly window_size^2 multiply-adds per token per channel, whereas a 5x5 depthwise convolution costs 25 multiply-adds per token per channel. For any window larger than 5x5, each reassigned channel increases that layer's FLOPs, so the no-cost claim is not automatic and is likely false for the configurations used. Table 7 compares five channel-allocation schedules but reports only FID and never FLOPs; the best schedule (Cwin/C increasing) is precisely the one that spends more channels on the more expensive attention branch in deeper layers. Thus the improvement attributed to Kth-order neighborhood aggregation is confounded with a shift of compute toward attention. Since the headline claim is about matching or exceeding DiT/U-DiT at lower FLOPs, and PiT-L's margin over U-DiT-L is only about 0.9 FID, this confound determines whether PCCA's contribution is a semantic mechanism or simply a favourable compute allocation.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes PiT, a family of diffusion transformers that replace global multi-head attention with static window attention (PSWA) plus a depthwise-convolution high-frequency bridging branch, and introduce PCCA, a channel-allocation schedule that progressively shifts channels from the convolution branch to the window-attention branch across layers. The authors claim PCCA realizes 'Kth-order attention' at no extra cost. They report ImageNet 256×256 FID improvements over DiT and U-DiT at lower or comparable FLOPs for S/B/L sizes, a 512×512 comparison, training-curve tables, MHSA-replacement experiments, channel-allocation and K-order ablations, and a supplementary theorem on the low-frequency bias of attention.","tokens_in":14050,"tokens_out":8330,"duration_ms":83340,"significance":"If the empirical FID results are reproducible, the paper would strengthen the case that local window attention plus a cheap convolutional bridge can outperform full global attention in diffusion transformers under equal training budgets, which is of practical significance for scaling image generation. The paper's strengths include the systematic size sweep (S/B/L), the iteration-by-iteration comparison in Table 3, the direct MHSA replacement in Table 4, and the channel-allocation/K ablations. However, the interpretive layer around PCCA and Kth-order attention is not yet supported by the reported equations and ablations, and the supplementary theorem's proof is invalid. The central empirical claim is plausible but needs the confounds below addressed.","major_comments":[{"comment":"The statement in §PCCA that PCCA captures Kth-order attention 'without any additional computational cost' is not substantiated and is likely false as stated. A channel reassigned from the 5×5 DWConv branch (25 multiply-adds per token per channel for the convolution itself) to the window-attention branch incurs the attention branch's per-channel interaction cost, which for typical window sizes is larger than 25. Table 7 compares five allocation schedules but reports only FID, never FLOPs, so row 5's best FID is confounded with a shift of compute toward the attention branch in deeper layers. Please report per-schedule FLOPs (and ideally per-layer channel budgets) and either verify the no-cost claim or rewrite it as a limited-cost claim.","section":"§PCCA and Table 7"},{"comment":"Equation (11) is not a derivation of the implemented computation. The channels in the red dashed box were convolved by 5×5 DWConv at layer k, but at layer k+1 they pass through LayerNorm, QKV projections, and window attention before any similarity is computed; the expression S^3_ij = Σ_c Conv5(X_i^c)·Conv5(X_j^c) omits these operations and therefore does not show that the network literally computes third-order similarity. The Kth-order interpretation should be presented as an analogy or supported by a derived identity that accounts for the actual operations.","section":"§PCCA, Eq. (11)"},{"comment":"The proof is invalid. In Eqs. (22)–(23), the paper derives E_L(t) ≥ (1-C)^t E_L(0) and, for C<1, concludes this remains bounded below by c>0; but if 0<C<1, (1-C)^t decays to zero, so c=0. The assumptions C<1 in Eq. (21) and the bound in Eq. (24) are also introduced without proof. The theorem therefore does not establish the claimed low-frequency propagation. This does not affect the empirical comparisons, but the supplementary claim should be corrected or removed.","section":"Supplementary Theorem 1"},{"comment":"The locality analysis uses attention maps from one pre-trained DiT-B/2 at a single 16×16 latent resolution. The paper generalizes this to 'DiTs' as a class and uses it to motivate replacing global attention at all scales. Since the transfer of the locality conclusion to larger latent resolutions (e.g., 32×32 in the 512×512 setting), larger models, and earlier training stages is not shown, the claim should be explicitly scoped or supported by additional measurements; the current sentence 'DiT does not rely on long-distance attention modeling' is stronger than the evidence.","section":"§Revisiting, Figure 1"}],"minor_comments":[{"comment":"The sentence 'PiT-L with a smaller model size surpasses the metrics of DiT-XL/2 at 7M iterations when it reaches 1M iterations' should be rewritten; the intended comparison is PiT-L at 1M steps versus DiT-XL/2 at 7M steps.","section":"Table 3 text"},{"comment":"The abstract and text report '54% FID improvement' and '10.68' absolute FID gain; from Table 2 these values depend on whether the baseline is DiT-XL/2 (19.47) or DiT-XL/2* (19.86). Please state the baseline explicitly.","section":"Abstract and Table 2"},{"comment":"Figure 1 and the text differ on whether the empirical sample is 'one million samples' or 'one million attention maps'; please reconcile and specify the number of images, tokens, heads, and layers.","section":"Figure 1 caption"},{"comment":"The supplementary section title contains an unfinished placeholder ('Propagation of Low-Frequency Theorem??'); please complete it.","section":"Supplementary section title"},{"comment":"The main FID tables do not report error bars or multiple seeds; given the small gaps in Table 2 (PiT-B vs U-DiT-B: 15.56 vs 16.64), a note on run-to-run variance would be helpful.","section":"Tables 2 and 3"}],"recommendation":"major_revision","confidential_remarks":"The main empirical result is likely publishable after the PCCA FLOPs confound is resolved and the invalid theorem is removed or corrected. The contribution is currently oversold by the 'no additional cost' and 'theoretically demonstrated' language; with those fixed, the paper should be reconsidered."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I read PiT with interest. The headline result is solid and useful: swapping MHSA for a static window attention plus a depthwise-conv bridge gives a consistent FID improvement over both DiT and U-DiT across model sizes and training budgets. The controlled replacement in Table 4 is a clean experiment, and the step-by-step training curves in Table 3 add confidence that the gains are not a fluke. The attention-distance measurement on a pretrained DiT is a nice empirical observation, and the throughput numbers in Table 6 are concrete.\n\nWhat is genuinely new is the PCCA schedule: to my knowledge, no one else has proposed progressively shifting channels from a conv branch to a window-attention branch as a way to increase receptive-field coverage in deeper layers. The PSWA block itself is a recombination of known parts, but the combination is effective and the ablations in Tables 7 and 8 support the design choices.\n\nThe soft spots are all in the interpretation and the supplementary material. The abstract and Section 4 claim that PCCA captures Kth-order attention 'without any additional computational cost.' But Table 7 reports only FID, not FLOPs, for the five allocation schedules. Window attention and 5x5 depthwise conv have different per-channel costs; shifting channels from conv to attention changes the layer's compute. If attention is the more expensive branch, the best schedule (Cwin increasing) may just be a favorable compute reallocation, not proof of a Kth-order mechanism. The paper needs to report FLOPs per schedule, or directly implement the Kth-order aggregation and compare. This does not kill the empirical result, but it undercuts the semantic story.\n\nMore serious is the supplementary theorem. The proof claims that (1-C)^t does not decay to zero when C<1, which is plainly false for 0<C<1. The lower bound c>0 collapses, so the theorem as written is not proved. It should be corrected or removed. The theorem is not load-bearing for the main experiments, but leaving a false equation in the supplement is a credibility problem.\n\nMinor notes: no code or error bars, and the U-DiT comparisons may use published numbers rather than reruns under identical settings. The DiT* reruns in Table 4 show some control, but the direct U-DiT apples-to-apples comparison is unclear.\n\nOverall, the paper is for researchers working on efficient diffusion backbones. The empirical claim is likely true and worth engaging with. I would send it to a serious venue, but with clear instructions to fix the FLOPs confound in the PCCA ablation and correct or delete the theorem. If those are addressed, it is a publishable efficiency contribution.","headline":"The empirical case for replacing global attention with window attention plus a conv bridge in DiTs is credible and consistently shown, but the paper oversells the Kth-order attention story and includes a wrong theorem in the supplement.","tokens_in":14560,"tokens_out":6224,"would_cite":true,"duration_ms":61734,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that diffusion transformers barely rely on long-range attention, and that a windowed attention plus convolutional bridge architecture (PiT) outperforms full-attention DiT and U-DiT models at the same training budget.","keywords":["diffusion transformer","window attention","image generation","Pseudo Shifted Window Attention","Kth-order attention","low-frequency bias","channel allocation"],"falsifier":"Measure the row and column attention distance of Equation 4 on attention maps from a larger pretrained DiT, or from DiT-B/2 operating at a 32×32 latent resolution; if the 99th-percentile distance approaches the full map size, the locality premise behind PiT fails and window-attention would be expected to lose to global attention on those scales.","tokens_in":13522,"feed_emoji":"🖼️","tokens_out":9460,"duration_ms":83684,"temperature":0.7,"pith_summary":"Diffusion Transformers spend most of their compute on full global self-attention, but this paper argues that most of that global computation is redundant. The authors measure attention maps from a pretrained DiT and find that 99% of tokens attend mainly to neighbors within a few pixels, so they replace global attention with a cheaper static-window attention plus a convolutional high-frequency bridging branch (PSWA). They add a channel-allocation schedule (PCCA) that shifts channels from the convolutional branch to the attention branch as layers deepen, which they present as capturing higher-order similarity at no extra cost. The resulting PiT models beat full-attention DiT and U-DiT baselines at the same training budget; PiT-L reaches FID 9.18 versus 19.47 for DiT-XL/2 while using fewer FLOPs. If the paper is right, window attention plus convolution is not just a cheaper approximation but a better inductive bias for diffusion image generation.","feed_headline":"Diffusion transformer cuts FID 54% by dropping global attention","feed_subtitle":"PiT-L reaches FID 9.18 versus 19.47 for DiT-XL/2 while using fewer FLOPs.","key_machinery":"The load-bearing machinery is Pseudo Shifted Window Attention (PSWA), a channel-split attention block that replaces global multi-head self-attention in diffusion transformers. PSWA runs static window attention on one set of channels and a depthwise separable convolution on another, then concatenates the two branches; the convolution both captures high-frequency information and links adjacent windows without the cost of shifted-window overlaps. The second component, Progressive Coverage Channel Allocation (PCCA), is a per-layer schedule that grows the window-attention channel fraction and shrinks the convolution fraction as depth increases, so tokens that were locally aggregated by large convolution kernels in one layer feed the attention of the next. Because a $5\\times5$ convolution covers a third-order neighborhood, attention computed on those transferred channels becomes $K$-th order attention, defined as similarity between neighborhood aggregates rather than between raw tokens. This machinery converts the empirical locality observation into a cheaper architecture, and the supplement's Fourier argument is meant to justify why the convolution branch is needed.","core_discovery":"The paper's central claim is that diffusion transformers do not actually need the global receptive field full self-attention provides. Using row and column attention distances averaged over one million attention maps from a pretrained DiT-B/2 at 16×16 latent resolution, the authors report that 99% of tokens attend within a row/column distance of 2–6, so local interactions dominate. On that basis they assert that static window attention is sufficient, and they design PSWA to split channels between window attention and a depthwise convolution branch; the convolution supplies high-frequency detail and bridges window boundaries. PCCA then progressively increases the attention-branch channel share across layers, so features aggregated over neighborhoods in earlier layers enter later attention computations, yielding what the paper calls $K$-th order attention at no extra cost. The authors also argue, with a Fourier-domain analysis in the supplement, that attention mechanisms carry a low-frequency bias that overlap-based window connections inherit, and that PSWA's parallel convolution branch is a robust way around it. Empirically, PiT-L obtains FID 9.18 at 400K iterations, a 54% improvement over DiT-XL/2's 19.47 with fewer FLOPs.","pith_inferences":["Beyond the paper's experiments, the locality measurement comes from a single pretrained DiT-B/2 at a 16×16 latent resolution; if larger models or higher latent resolutions develop longer-range dependencies, the window size and PCCA schedule would need to scale and the efficiency gap could close.","The DWConv bridge is effectively a cheap convolutional inductive bias, so the same channel-split design could be tried in text-to-image or video diffusion transformers; the paper does not test those settings.","The supplementary low-frequency theorem is only a sketch: the proof drops the nonlinear Softmax term and the theorem name appears unfinished, so the theoretical case for the DWConv branch is weaker than the empirical evidence.","The reported 54% improvement suggests training compute, not model capacity, was the bottleneck for the DiT-XL/2 baseline; a direct test would be to give DiT-XL/2 the same FLOPs-reduced training recipe and see whether the gap persists."],"forward_implications":["At a fixed 400K-iteration budget on ImageNet at 256×256, PiT-L reaches FID 9.18 versus 19.47 for DiT-XL/2 while using 100.80 G FLOPs instead of 114.46 G.","Simply replacing MHSA with PSWA in standard isotropic DiT models improves FID at every tested size (PiT-S/2, PiT-B/2, PiT-L/2) while lowering FLOPs.","PiT-B also outperforms U-DiT-B at 512×512 resolution under the same training settings, so the gain is not limited to the 256×256 benchmark.","Because the attention is windowed, throughput rises sharply: PiT-B/2 processes about 618 images per second versus 272 for DiT-B/2 on the same GPUs."],"supporting_citations":[{"why":"Defines the DiT baseline architecture and the 400K-iteration training budget that all comparisons share.","marker":"Peebles and Xie 2023"},{"why":"Introduces shifted-window attention, the standard inter-window connection approach that PSWA's convolution bridge replaces.","marker":"Liu et al. 2021"},{"why":"Provides the U-shaped macro architecture (U-DiT) that PiT scales and the strongest competing baseline.","marker":"Tian et al. 2024"},{"why":"Supplies the denoising diffusion framework in which all PiT models are trained.","marker":"Ho, Jain, and Abbeel 2020"},{"why":"MDT is a state-of-the-art diffusion transformer baseline against which PiT reports FID improvements.","marker":"Gao et al. 2023a"},{"why":"PixArt-α is a strong XL-scale diffusion transformer baseline compared in the main FID table.","marker":"Chen et al. 2023"}],"fun_headline_variants":["Local attention cuts diffusion FID by 54%","Diffusion transformers don't need global attention","PiT: 54% FID gain with local attention","Local attention replaces global for better diffusion","PiT slashes FID 54% using less compute"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire efficiency argument rests on the measured claim that 99% of DiT attention stays within a 2–6 token distance at 16×16 latent resolution; if that locality is an artifact of one model size and one resolution, the window-attention advantage may not transfer.","fun_headline_variants_meta":{"raw":{"variants":["Local attention cuts diffusion FID by 54%","Diffusion transformers don't need global attention","PiT: 54% FID gain with local attention","Local attention replaces global for better diffusion","PiT slashes FID 54% using less compute"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000467,"raw_usage":{"total_tokens":2352,"prompt_tokens":995,"completion_tokens":1357,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":611,"completion_tokens_details":{"reasoning_tokens":1282}},"tokens_in":611,"tokens_out":1357,"duration_ms":10803,"temperature":1.0,"reasoning_tokens":1282,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T20:16:53.448259+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Measure the row and column attention distance of Equation 4 on attention maps from a larger pretrained DiT, or from DiT-B/2 operating at a 32×32 latent resolution; if the 99th-percentile distance approaches the full map size, the locality premise behind PiT fails and window-attention would be expected to lose to global attention on those scales.","supporting_citations":[],"review_version":1}