{"id":"a4734cc8-3099-4050-a7fc-7be80cd74daa","arxiv_id":"2507.23268","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"PixNerd is a single-stage pixel-space diffusion transformer that uses predicted neural field weights to decode large patches, reaching 2.15 FID on ImageNet 256 without a VAE.","lead":"PixNerd replaces the usual VAE in a diffusion transformer with a small neural field that decodes large pixel patches, letting the model generate images directly in pixel space. It reports ImageNet FID scores around 2.15 at 256 and 2.84 at 512, without a cascade or latent autoencoder.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The headline FID/DPG results are not independently checkable from the preprint: the 512 fine-tuning schedule is an unfilled placeholder ('abc steps') and the DPG score is reported as both 0.82 and 80.9; the central empirical claim therefore needs a reproducibility check before acceptance.","rationale":"The paper's contribution is an architecture plus a set of empirical claims. The architecture is plausible: the neural-field decoder is a clean replacement for the final linear projection, and the ablations in Figure 7 give real support to the MLP-capacity assumption by showing that width, depth, normalization, and coordinate encoding all matter. I therefore do not think the weakest point is decoder expressiveness; even if the MLP were capacity-limited, the baseline comparisons would still show that the neural-field design helps. The truly load-bearing condition is that the reported headline numbers — especially the 512 FID — were produced by a well-defined, reproducible procedure. The unfilled 'abc steps' in Section 4.4 and the inconsistent DPG scores (80.9 versus 0.82) are manuscript-internal evidence that this condition is not currently met. Since no code or checkpoints are provided, a reviewer cannot distinguish a genuine architectural result from an unreproducible or misreported evaluation. This justifies preserving the reader's CONDITIONAL verdict. It does not require moving to REJECT, because the inconsistencies are fixable, the ablations support the method, and the overall design is coherent enough that a properly documented release could resolve the concern.","tokens_in":16305,"tokens_out":10625,"duration_ms":126234,"concrete_test":"Obtain or release the 512 checkpoints and configuration, fix the 'abc steps' placeholder to the exact fine-tuning schedule, and recompute ImageNet 512 FID using Euler-100 with CFG 3.5 and interval guidance [0.3, 1.0] on the full validation set. If the recomputed FID matches 2.84 within tolerance, the claim is supported; if the schedule was never fixed or the checkpoint is unavailable, the 512 result should remain unverified and the paper should stay conditional.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is empirical: a single-scale pixel-space DiT with per-patch neural-field decoding reaches 2.15 FID on ImageNet 256 and 2.84 FID on ImageNet 512 without a VAE or cascade. For this to hold, the reported evaluation must be reproducible and internally consistent. The manuscript does not currently permit that check. In Section 4.4, the ImageNet 512 model is said to be fine-tuned 'for abc steps' — the step count is literally a placeholder — so the second headline number cannot be reproduced or checked for early stopping or validation-set selection. In the DPG paragraph, the text claims a '0.82 overall score' while the abstract and Table 5 report 80.9; both cannot be correct, so at least one reported headline result is mis-stated. No code or checkpoints are available in the preprint to resolve these discrepancies. The reader's MLP-capacity concern is reasonable, and the ablations in Figure 7 partially address it, but the more immediate blocker for the central claim is that the headline numbers themselves cannot be verified from the paper as written.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"PixNerd proposes a pixel-space diffusion transformer in which the final linear patch decoder is replaced by a per-patch neural field. The transformer's last hidden state for each patch is linearly projected into MLP weights (Eq. 9), and each pixel's diffusion velocity is decoded from DCT coordinate encodings plus the noisy pixel value (Eq. 10). The model is trained end-to-end at a single scale with patch size 16, avoiding VAE compression and cascade pipelines. On class-conditional ImageNet, PixNerd-XL/16 reports FID 2.15 at 256×256 and 2.84 at 512×512; on text-to-image, PixNerd-XXL/16 reports GenEval 0.73 and DPG 80.9. The paper includes a controlled comparison against a linear-decoder baseline (Baseline-L/16) and component ablations supporting the neural-field design.","tokens_in":16621,"tokens_out":10154,"duration_ms":100418,"significance":"If the results hold, PixNerd is a significant advance for pixel-space generation: it shows that a single-scale DiT with patch size 16 can approach latent DiT performance without a VAE, and the resource comparison in Table 1 suggests large latency gains over cascade pixel models. The controlled Baseline-L/16 comparison and the loss curves in Fig. 6 provide direct evidence that the neural-field decoder is responsible for the improvement, and the ablations in Fig. 7 give useful design guidance. The method is simple and does not rely on adversarial VAE training. However, the empirical claims are currently undermined by incomplete training details and inconsistent benchmark numbers, so the result cannot yet be fully assessed.","major_comments":[{"comment":"The 512×512 result is not reproducible as reported. The text in §4.4 says the ImageNet 512 model is fine-tuned from the 256 model 'for abc steps' — a literal placeholder — and states the guidance interval is [0.3, 1.0], while the caption of Table 3 says interval [0.1, 1] with CFG 3.5. Please provide the actual step/epoch count, reconcile the interval, and state whether the reported FID was selected on a validation set or is a single test-set evaluation. The manuscript lists repository URLs but does not state whether code or checkpoints will be available, so these ambiguities cannot be resolved from the paper.","section":"§4.4, Table 3"},{"comment":"The DPG result is internally inconsistent: the text says PixNerd-XXL/16 'achieves 0.82 overall score', while the abstract and Table 5 report 80.9. Since all DPG scores in Table 5 are on a 0–100 scale, 80.9 is presumably the correct value, but the manuscript currently states two different headline numbers for the same benchmark. Correct all occurrences and double-check the GenEval/DPG tables for further such discrepancies.","section":"§4.5, DPG paragraph"},{"comment":"The headline FID numbers are obtained after tuning hyperparameters on the target metric: Fig. 7e sweeps CFG values 3.0–5.0 and picks 3.4–3.6 on FID10k, and the paper then chooses 3.5 as the default for the FID50k results in Tables 2 and 3. This is selection on the evaluation benchmark and makes the reported 2.15/2.84 FID optimistic. Please report results for a fixed CFG/interval (or a validation split), and clearly state that the final numbers use the tuned schedule.","section":"§4.3, Fig. 7e and Table 2"},{"comment":"The central architectural assumption is that a linear projection of the patch hidden state predicts neural-field weights sufficient to decode all pixels in a 16×16 patch, with DCT coordinates plus the noisy pixel value as complete local conditioning. The Baseline-L/16 comparison and the width/depth ablations support this choice empirically, but they do not test the capacity ceiling of the decoder. A concrete upper-bound experiment (e.g., larger MLP hidden dimension, deeper MLP, or a small transformer-based per-patch decoder) would clarify how much of the remaining gap to latent DiT is due to decoder capacity.","section":"Eqs. (9)–(11)"}],"minor_comments":[{"comment":"The paper contains several typos, including 'Pixel Nerual FieldDiffusion' in the introduction, 'Basline-L/16' in Fig. 5, and a stray space in 'V AE' in multiple places; please proofread.","section":"§1, Fig. 5"},{"comment":"The solver is referred to as 'Adam2', 'Adams-2nd', and 'Adams-2' interchangeably in different parts of the paper; please use one consistent name.","section":"§4.3, §4.5, Fig. 3"},{"comment":"The definitions of f(t) and g(t) are garbled: the expression for g(t) lacks the standard square-root/brace structure, and Eq. (7) prints '20πi' where superscripts such as 2^0πi appear intended. Eq. (12) should also define the range and normalization of k1 and k2.","section":"§3.1, Eqs. (2), (7), (12)"},{"comment":"The text says CFG was swept from 3.0 to 5.0, but the horizontal axis only shows 3.2–4.8; please clarify the exact range and state whether the FID10k values are single runs or averaged over multiple seeds.","section":"Fig. 7e"},{"comment":"The statement that PixNerd is 'nearly 8× faster than other pixel diffusion models' is not directly supported by the table: ADM-G is 4.21s versus 0.65s (about 6.5×) and PixelFlow is 10.1s versus 0.65s (about 15.5×). Please specify the hardware and clarify which comparison the factor refers to.","section":"Table 1"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is promising and the core idea is well motivated, but the missing fine-tuning step count and the DPG score discrepancy are severe for a primarily empirical paper. I recommend major revision and asking the authors to supply the missing training/evaluation details, reconcile the contradictory guidance intervals, and correct the benchmark tables. I do not see a fundamental flaw in the method that would require rejection, provided the numbers can be verified."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe core idea here is good. PixNerd replaces the final linear projection in a pixel-space DiT with a per-patch neural field: the last hidden state predicts MLP weights, and a DCT coordinate encoding plus the noisy pixel value is fed into that MLP to decode velocity per pixel. That exact combination is not in the cited prior work. The ablation against their own Baseline-L/16—same initialization, same training budget, linear projection—is the right test, and the loss curves plus Figure 5 show the neural field is doing real work, not just adding parameters. The resource numbers also look plausible: a single-scale, single-stage pixel model with only 256 tokens on ImageNet-256 is a legitimately different operating point than cascade pixel models.\n\nThe empirical claim is not checkable from the preprint. Section 4.4 says the 512 model was fine-tuned 'for abc steps'—that's a literal placeholder, so the 2.84 FID cannot be reproduced or checked for early stopping. The DPG paragraph says '0.82 overall score' while the abstract and Table 5 report 80.9; both cannot be right. There is also a mismatch on the 512 interval guidance: the text says interval [0.3,1.0] with CFG 3.5, but the table caption says [0.1,1]. These are fixable but they undermine the headline numbers as submitted.\n\nOn methodology, the CFG and interval bounds were chosen on FID10k (Figure 7e) and then used for the FID50k numbers in Table 2. That is selection on the evaluation metric; not disqualifying, but it should be disclosed. The MLP-capacity concern—can a two-layer, 64-channel MLP decode a 16x16 patch—is partially addressed by the Baseline-L/16 comparison, but the ablations do not directly test whether a wider or deeper decoder would do better. That is a minor gap, not a fatal one.\n\nWho should read this: anyone working on pixel-space generation or large-patch diffusion. The architecture is simple enough to implement, and the idea is worth discussing even if the final numbers shift after a proper re-run. I would bring it to a reading group.\n\nRecommendation: send it to review. A serious referee can verify the claims and catch the placeholder and inconsistency. This is a conditional accept in its current form, not a reject.","headline":"PixNerd's neural-field decoder is genuinely new and the ablations support it, but the manuscript's internal inconsistencies (an 'abc steps' placeholder and DPG score reported as both 0.82 and 80.9) make the headline numbers unverifiable as written.","tokens_in":17088,"tokens_out":4088,"would_cite":true,"duration_ms":39666,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"PixNerd claims that a pixel diffusion transformer with a per-patch neural field decoder reaches 2.15 FID on ImageNet 256x256 and 2.84 FID on ImageNet 512x512 without a VAE or cascade pipeline.","keywords":["diffusion transformer","pixel-space diffusion","neural field","implicit neural representation","image generation","class-conditional generation","text-to-image generation","VAE-free generation"],"falsifier":"Take the trained PixNerd-L/16 model and evaluate it with the DCT coordinate encoding replaced by zeros, so the decoder sees only the noisy pixel value; if FID at 400k training steps stays close to the full model, then coordinates are doing little and the claimed mechanism is not the source of the gain. Conversely, scale the neural field MLP to four layers and 128 channels: if the FID curve does not improve beyond the 2-layer/64-channel default, then the simple-MLP capacity assumption is not the binding constraint.","tokens_in":16125,"feed_emoji":"🎨","tokens_out":8620,"duration_ms":89298,"temperature":0.7,"pith_summary":"Diffusion transformers currently owe most of their image quality to a pre-trained variational autoencoder (VAE) that compresses pixels into a latent space; this paper argues that the VAE is a workaround, not a requirement. PixNerd is a single-scale, single-stage diffusion transformer that denoises raw pixels directly, using patch size 16 to keep the token count the same as a latent model. The authors report 2.15 FID on ImageNet 256x256 and 2.84 FID on ImageNet 512x512 with no VAE, no cascade pipeline, and no multi-scale training, bringing the quality close to latent DiT-class models under similar compute. The claim matters because it shifts the difficulty of pixel diffusion from \"the space is too hard\" to \"the decoder was too weak\": the paper's evidence is that replacing the final linear projection with a per-patch neural field lowers training loss and closes most of the FID gap. The same design is extended to text-to-image generation, where it reports 0.73 on GenEval and 80.9 on DPG.","feed_headline":"Pixel-space transformer hits 2.15 FID on ImageNet with no VAE","feed_subtitle":"A per-patch neural field decoder lets a single-stage diffusion model match latent models at 256x256 and 512x512.","key_machinery":"The load-bearing mechanism is the patch-wise neural field decoder. Given the last hidden state $X_n$ of patch $n$, two linear layers predict the row-normalized weights $\\{W_1^n, W_2^n\\}$ of a two-layer MLP (Eq. 9). The MLP is evaluated at every pixel $(i,j)$ inside the patch: its input is the concatenation of a DCT coordinate encoding $\\cos(k_1 i)\\cos(k_2 j)$ for $k_1,k_2\\in(0,K]$ and the noisy pixel value $x_n(i,j)$, and its output is passed through a final linear layer to give the velocity $v_n(i,j)$ (Eqs. 10-11). Because the decoder is a coordinate-based function, it can be queried at arbitrary locations, which is what enables training-free arbitrary-resolution generation: the token grid stays fixed at the pretraining resolution while the neural field coordinates are interpolated.","core_discovery":"The central claim is that a large-patch pixel-space diffusion transformer can be competitive with latent-space diffusion if the patch decoder is a coordinate-conditioned neural field rather than a linear layer. Concretely, the transformer's last hidden state for a 16x16 patch predicts the weights of a two-layer MLP; that MLP then takes, for each pixel in the patch, a DCT-basis encoding of the pixel's local coordinates concatenated with the noisy pixel value, and outputs the diffusion velocity. This gives the decoder an explicit sense of spatial locality that a linear projection on flattened patches does not have, which the paper identifies as the reason it can decode large patches without a cascade. The paper presents this as evidence that pixel-space diffusion is viable as a single-scale, single-stage, end-to-end system, and that the VAE's two-stage training is an optional shortcut rather than a structural necessity.","pith_inferences":["An implication the paper leaves implicit is that the same coordinate-based decoder should transfer to other dense-output domains, such as video or 3D, where a transformer emits patch tokens that must be expanded into many output points.","A testable extension the paper does not run is to measure resolution extrapolation with spectral statistics; if the coordinate interpolation preserves low frequencies but loses high-frequency structure, the claimed training-free arbitrary resolution has a boundary.","The paper's comparative framing implies a stronger claim than the one it proves: that the VAE contributes artifacts rather than being a necessary compression. A direct test would be to fine-tune PixNerd at 512x512 with a high-resolution post-training stage and compare against a latent model post-trained identically.","The paper itself concedes some generated samples show unclear details, so the fair reading is that the gap to latent models has narrowed enough to make the VAE optional, not that pixel space has become strictly better."],"forward_implications":["A single-stage pixel-space diffusion model is computationally practical at ImageNet scale: with patch size 16 it keeps the same token count as a latent DiT, and the paper reports roughly 8x lower per-step latency than ADM-G and PixelFlow while training faster per iteration than a latent SiT-L/2 baseline.","The model reaches its headline image quality without any cascade: 2.15 FID on ImageNet 256x256 with 100 Euler steps, and 2.84 FID on ImageNet 512x512 after fine-tuning from the 256x256 checkpoint.","Large patches stop being a barrier to pixel-space learning: at identical training steps, PixNerd-L/16 reports consistently lower flow-matching loss and lower DINOv2 representation-alignment loss than the linear-projection Baseline-L/16.","Arbitrary-resolution generation is a by-product of the coordinate decoder: keeping the number of tokens constant and interpolating neural field coordinates produces multi-resolution outputs without fine-tuning.","The same decoder transfers to text-to-image: PixNerd-XXL/16 reaches 0.73 on GenEval and 80.9 on DPG with a 45M-image training set, outperforming the previous pixel-space PixelFlow on both benchmarks in the paper's tables."],"supporting_citations":[{"why":"Supplies the diffusion transformer backbone with AdaLN conditioning that PixNerd keeps and modifies.","marker":"[13]"},{"why":"Defines the VAE-based latent diffusion paradigm that PixNerd removes, and provides the latent baselines it compares against.","marker":"[1]"},{"why":"Establishes the transformer-plus-flow-matching setting whose velocity objective PixNerd adopts.","marker":"[14]"},{"why":"The strongest pixel-space cascade baseline; PixNerd claims to match or beat it with a single scale and no cascade.","marker":"[9]"},{"why":"Another cascade pixel diffusion approach that motivates the single-stage design.","marker":"[10]"},{"why":"The coordinate-MLP pattern of implicit neural fields that the patch decoder adapts.","marker":"[11]"},{"why":"Periodic-activation implicit neural representations that inform the coordinate encoding design choices.","marker":"[12]"},{"why":"Representation alignment loss used in training; the reported loss curves show PixNerd-L/16 improves on it over the linear baseline.","marker":"[54]"},{"why":"Interval guidance, the sampling technique whose ablations determine the CFG settings behind the headline FID results.","marker":"[57]"}],"fun_headline_variants":["Pixel diffusion ditches VAE with neural field decoder","Neural field decoder lets pixel-space diffusion rival VAEs","Single-stage pixel diffusion hits 2.15 FID, no VAE needed","Patch-wise neural field makes pixel-space diffusion competitive","PixelNerd: VAE-free diffusion with per-patch neural decoding"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a two-layer MLP whose weights are predicted per patch can faithfully decode the velocity of all 256 pixels inside a 16x16 patch from just a coordinate encoding and the noisy pixel value, and the paper shows ablations over MLP width, depth, and normalization but does not prove that this decoder class is expressive enough to be the right one.","fun_headline_variants_meta":{"raw":{"variants":["Pixel diffusion ditches VAE with neural field decoder","Neural field decoder lets pixel-space diffusion rival VAEs","Single-stage pixel diffusion hits 2.15 FID, no VAE needed","Patch-wise neural field makes pixel-space diffusion competitive","PixelNerd: VAE-free diffusion with per-patch neural decoding"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000155,"raw_usage":{"total_tokens":1199,"prompt_tokens":912,"completion_tokens":287,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":528,"completion_tokens_details":{"reasoning_tokens":201}},"tokens_in":528,"tokens_out":287,"duration_ms":3345,"temperature":1.0,"reasoning_tokens":201,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T10:53:11.143533+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take the trained PixNerd-L/16 model and evaluate it with the DCT coordinate encoding replaced by zeros, so the decoder sees only the noisy pixel value; if FID at 400k training steps stays close to the full model, then coordinates are doing little and the claimed mechanism is not the source of the gain. Conversely, scale the neural field MLP to four layers and 128 channels: if the FID curve does not improve beyond the 2-layer/64-channel default, then the simple-MLP capacity assumption is not the binding constraint.","supporting_citations":[{"cited_title":"Scalable diffusion models with transformers","cited_arxiv_id":null,"evidence_quote":"Supplies the diffusion transformer backbone with AdaLN conditioning that PixNerd keeps and modifies."},{"cited_title":"High-resolution image synthesis with latent diffusion models","cited_arxiv_id":null,"evidence_quote":"Defines the VAE-based latent diffusion paradigm that PixNerd removes, and provides the latent baselines it compares against."},{"cited_title":"Nerf: Representing scenes as neural radiance fields for view synthesis.Communications of the ACM, 65(1):99–106, 2021","cited_arxiv_id":null,"evidence_quote":"The coordinate-MLP pattern of implicit neural fields that the patch decoder adapts."},{"cited_title":"Implicit neural representations with periodic activation functions.Advances in neural information processing systems, 33:7462–7473, 2020","cited_arxiv_id":null,"evidence_quote":"Periodic-activation implicit neural representations that inform the coordinate encoding design choices."}],"review_version":1}