{"id":"2c279da9-69c3-427e-8614-f52e6f9822fc","arxiv_id":"2501.18593","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"A single diffusion L2 loss can train scalable image tokenizers that match or outperform GAN-LPIPS tokenizers for reconstruction and downstream generation.","lead":"A simple image tokenizer, DiTo, trains an encoder and a diffusion decoder with a single L2 loss, no adversarial or perceptual losses. It matches or beats a widely used GAN-LPIPS tokenizer on reconstruction and generation once scaled up, while being simpler and self-supervised.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"DiTo-XL's 52.44% human-preference 'win' is statistically indistinguishable from a tie (95% CI includes 50%) and conflicts with rFID; the better-reconstruction claim is therefore not load-bearing.","rationale":"The paper's central contribution is credible in its weaker form: DiTo is a simpler, entirely L2-loss tokenizer, and it produces latents on which a DiT can be trained to competitive gFID (7.57 vs 7.49 at XL; 6.29 with noise synchronization). The scaling curves and the provided code link give some independent support for the 'scalable' part. What is not established is the stronger 'better reconstruction' claim. The only quantitative evidence for it is the XL human-preference bar, which is a 52.44% ties-as-half rate over 900 MTurk judgments with no reported error bars or significance test. The raw breakdown is 42.78% DiTo, 37.89% GLPTo, 19.33% ties, so the preference margin is small and, under a standard binomial test, well inside sampling noise. This conflicts with rFID@5K (7.95 vs 4.14) and with LPIPS and Inception Score (Table 6), which favor GLPTo. The paper's response is that FID misaligns with human judgment, which makes the human protocol the load-bearing piece of evidence; but that evidence is too weak to carry the claim. A formal confidence-interval calculation on the existing Table 4 counts would settle whether the preference is real; if it is not, the paper should be read as showing competitiveness, not superiority, in reconstruction. This does not change the reader's CONDITIONAL verdict, but it sharpens the condition and clarifies what additional evidence would be needed.","tokens_in":17988,"tokens_out":7796,"duration_ms":78760,"concrete_test":"Run an exact two-sided binomial test on the DiTo-XL versus GLPTo-XL row of Table 4 using the raw counts behind the reported percentages (N=900, with ties either excluded or split half). If the 95% confidence interval for the probability that a rater prefers DiTo includes 0.50, then the paper's statement that DiTo-XL 'outperforms' GLPTo-XL in reconstruction is not supported and should be weakened to 'statistically tied' unless additional human data are collected.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The reconstruction-quality claim rests almost entirely on the XL-scale human-preference result in Figure 4 and Table 4. For DiTo-XL vs GLPTo-XL, the raw breakdown is 42.78% prefer DiTo, 37.89% prefer GLPTo, and 19.33% ties; the reported 52.44% counts ties as half. Using N=900 and splitting ties, the standard error of the preference proportion is about 1.66 percentage points, so the 95% confidence interval for the true preference rate is roughly [49.2%, 55.7%] and includes 50%; excluding ties gives a similar interval that also includes 50%. Thus the human evidence is statistically indistinguishable from chance. This is the only evidence for 'better reconstruction' in the self-supervised setting, and it directly conflicts with rFID@5K (7.95 vs 4.14) and with the LPIPS and Inception Score numbers in Table 6, where the supervised baseline wins. The paper's argument that FID is not aligned with human judgment makes the human protocol the load-bearing piece of evidence, but that protocol as reported cannot carry the claim. If the preference is not real, the strongest surviving claim is 'competitive,' not 'better,' reconstruction.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes DiTo, a diffusion autoencoder image tokenizer trained with a single Flow Matching diffusion L2 loss, without GAN or perceptual losses. The encoder is the standard LDM encoder, and the decoder is a diffusion UNet conditioned on the latent. The authors argue that a single ELBO-style diffusion objective is sufficient to learn competitive image tokens, and they introduce a noise-synchronization regularization to improve downstream latent diffusion training. They compare DiTo to an LDM-style GAN-LPIPS tokenizer (GLPTo) at three scales (B, L, XL) for reconstruction and for training a DiT latent generator, reporting competitive or better rFID/gFID and a human-preference study. The paper includes theory grounding via the Kingma-Gao ELBO result, ablations of training objectives, LayerNorm versus KL regularization, and a zero-shot higher-resolution evaluation.","tokens_in":18240,"tokens_out":3769,"duration_ms":38469,"significance":"If the central claim holds, a simple self-supervised diffusion objective can replace the complex GAN-LPIPS recipe used by most current latent image/video models, simplifying tokenizer training and removing dependence on pretrained supervised features. The paper is well grounded theoretically, releases code, and evaluates at nontrivial scales. Its strengths include the clean architecture, the connection to ELBO theory, the LayerNorm analysis, and the honest reporting of rFID gaps. However, the empirical support for the headline 'better reconstruction' claim is fragile: the only direct evidence in the self-supervised setting is a human-preference result at XL that is statistically indistinguishable from a tie and conflicts with the reported rFID and LPIPS/IS numbers. The generation results are promising but are reported without repeated runs. The core idea is valuable, but the claims need to be tightened and supported by additional statistical evidence.","major_comments":[{"comment":"The claim that DiTo-XL achieves better reconstruction than GLPTo-XL is based on a human-preference rate of 52.44% when ties are counted as half, from 900 MTurk comparisons. With N=900, the standard error of this proportion is about 1.67 percentage points, so the 95% confidence interval is approximately [49.2%, 55.7%] and includes 50%; the result is statistically indistinguishable from a tie. This matters because the authors explicitly argue that FID is not aligned with human judgment, making the human protocol the load-bearing evidence for the better-reconstruction claim. As reported, the protocol cannot carry that claim. I recommend reporting confidence intervals or pairwise significance tests, increasing the number of comparisons, and providing a robustness analysis with respect to how ties are handled. Without this, the claim should be moderated to 'competitive' rather than 'better.'","section":"Section 5.1, Figure 4 and Table 4"},{"comment":"The generation results are based on a single training run of each DiT model. The differences in gFID@50K (7.49 for GLPTo-XL, 7.57 for DiTo-XL, 6.29 for DiTo-XL with noise sync) are small relative to typical run-to-run variance for large generative models. Without multiple seeds or at least a measure of variance, the statement that noise synchronization 'outperforms' GLPTo is not established. Please report multiple runs with error bars or clearly label these as single-run comparisons and adjust the wording accordingly.","section":"Section 5.2, Table 2"},{"comment":"Noise synchronization is presented as a key contribution, but the paper does not ablate its design choices: the probability p=0.1, the sampling range of tau, and the exact conditioning of the decoder on z_tau with t in [tau,1]. It is therefore unclear whether the gFID improvement in Table 2 is attributable to the proposed regularization or to a particular choice of hyperparameters. Please provide an ablation of p and tau, and clarify the notation around 'x_tau...1' and the used loss, as the current description is difficult to follow.","section":"Section 4, Noise synchronization"},{"comment":"The comparison of ELBO versus non-ELBO diffusion objectives (Flow Matching, v-prediction cosine, epsilon-prediction, EDM) is reported only with qualitative examples. The paper claims that non-ELBO objectives lead to suboptimal latent representations, but no quantitative rFID or generation numbers are given for these variants. Please add quantitative results to support the claim that the ELBO-form objective is important for learning useful latents, especially since this is one of the primary design justifications.","section":"Section 5.3, Figure 5"}],"minor_comments":[{"comment":"The phrase 'competitive or better quality' should be qualified in light of the statistical evidence; the current wording overstates the reconstruction result.","section":"Abstract and Conclusion"},{"comment":"The row labeled 'Supervised (Rombach et al., 2022)' is ambiguous; please clarify that this is the original Stable Diffusion tokenizer, presumably evaluated under the same protocol, and note any differences in training data.","section":"Table 1"},{"comment":"The preference plot would benefit from error bars or confidence intervals; as it stands, the reader cannot assess the uncertainty of the 52.44% point estimate.","section":"Figure 4"},{"comment":"The derivation of the sample prediction x-bar_theta is concise but would be easier to follow if the matrix inverse and the use of the L2-optimal network prediction were motivated more explicitly.","section":"Appendix E, Equations (10)--(13)"},{"comment":"Please state whether the 512x512 rFID evaluation is performed on the same validation image set as the 256x256 evaluation, since the zero-shot generalization claim depends on the comparability of the image distributions.","section":"Appendix F, Table 8"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about the human-preference result does land: the reported 52.44% with N=900 is not statistically distinguishable from chance, and it conflicts with rFID and with LPIPS/IS. This is the load-bearing issue for the reconstruction claim. The paper is technically competent and the generation results are promising, but the authors need to either strengthen the human-evaluation evidence or moderate the 'better reconstruction' claim. The missing error bars for generation FID should also be addressed. With those changes, the paper would be suitable for publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid, well-executed paper that deserves serious review, but the abstract and conclusion overreach on reconstruction quality. The core idea—train a diffusion autoencoder with a single L2/flow-matching loss and scale it to ImageNet tokenization—is genuinely new in this form. Previous diffusion autoencoders were small-scale; concurrent works mix diffusion with LPIPS or GAN losses. DiTo shows a single self-supervised objective can produce latents that train a DiT to a 6.29 gFID, beating GLPTo's 7.49 with noise synchronization. That is the paper's strongest result and worth taking seriously.\n\nWhat it does well: the ELBO connection is used correctly to justify flow matching over epsilon-prediction; the ablation showing non-ELBO objectives cause color shift is informative; the LayerNorm vs KL ablation is clean; and the authors report compute and give a code link. The zero-shot 512x512 generalization is a nice bonus.\n\nSoft spots, in order of importance. First, the human-preference evidence for 'better reconstruction' is statistically indistinguishable from a tie. For DiTo-XL vs GLPTo-XL, the raw counts are 42.78% vs 37.89% with 19.33% ties; counting ties as half gives 52.44% for N=900. The standard error is about 1.7 percentage points, so the 95% CI includes 50%. The paper says 'significantly improves' and 'outperforms' but the data do not support that. This matters because rFID@5K is worse (7.95 vs 4.14) and the FID-vs-human-alignment argument makes the human study load-bearing. The claim should be softened to 'competitive' or backed by more comparisons.\n\nSecond, no error bars or multiple seeds anywhere; FID differences of 0.1-0.5 are treated as decisive. Not fatal, but common for this field and easily fixable.\n\nThird, the noise-synchronization result is a single configuration (p=0.1). The paper doesn't ablate p or show sensitivity, so the 6.29 vs 7.49 could be luck. Needs a small study.\n\nFourth, the 'self-supervised' framing is honest—the base DiTo uses no supervised losses—but the DiTo+LPIPS variant, which beats GLPTo on rFID, reintroduces the supervised component. The paper is clear about this, so it's not a flaw, but it should be highlighted in the conclusion.\n\nI disagree with the stress-test note's conclusion that the better-reconstruction claim is 'not load-bearing' for the whole paper. The main contribution is the method and generation results, which stand even if reconstruction is merely competitive. The human-eval problem is a real overclaim, but it is localized and correctable.\n\nRecommendation: send it to review. The core result is novel and the experiments, despite the blemishes, are careful. A good referee will push for the human-eval fix and a noise-sync ablation. I'd cite this if I worked on latent diffusion or tokenizers.","headline":"DiTo's real contribution is a simpler self-supervised tokenizer with competitive generation FID; the 'better reconstruction' claim rests on a human eval that is statistically a tie.","tokens_in":18789,"tokens_out":2809,"would_cite":true,"duration_ms":25441,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that a single diffusion L2 loss—Flow Matching, an evidence-lower-bound objective—can train scalable image tokenizers, and that these tokenizers match or outperform the standard GAN-LPIPS tokenizer (GLPTo) for both image…","keywords":["image tokenizers","diffusion autoencoders","flow matching","latent diffusion models","self-supervised learning","image reconstruction","image generation","ELBO objective"],"falsifier":"A direct test: collect more human preferences between DiTo-XL and GLPTo-XL at both 256px and 512px; if DiTo no longer wins at either resolution, or if a text-reading metric on symbol-heavy crops shows no advantage, the reconstruction claim fails. For the generation claim, retrain a DiT-XL/2 on DiTo-XL latents with and without noise synchronization over multiple seeds and check whether the gFID 6.29 versus 7.49 improvement reproduces within noise.","tokens_in":17776,"feed_emoji":"🖼️","tokens_out":9911,"duration_ms":80300,"temperature":0.7,"pith_summary":"This paper asks whether image tokenizers—the autoencoders that compress pixels into compact latents for image generation—need the complex multi-loss recipe used by today's best systems, an autoencoder combining L1, LPIPS, and GAN losses. It argues that one self-supervised diffusion L2 loss, Flow Matching, an evidence-lower-bound objective, is enough to train a tokenizer end to end. The resulting diffusion tokenizer (DiTo) matches or improves on that GAN-LPIPS tokenizer (GLPTo) in reconstruction quality and in the quality of image generators trained on its latents. If correct, this removes the need to balance L1, LPIPS, and adversarial losses, and makes tokenizer quality improve predictably with model scale.","feed_headline":"One diffusion loss trains image tokenizers that match GAN recipes","feed_subtitle":"DiTo replaces the supervised L1+LPIPS+GAN mix with self-supervised Flow Matching, scaling up to match or beat it.","key_machinery":"The central object is the diffusion tokenizer: an encoder plus a diffusion decoder trained with a single $L^2$ Flow Matching loss, which the paper justifies as an ELBO (evidence lower bound) maximization objective. The decoder is a convolutional UNet with timestep conditioning that takes the upsampled latent $z$ as a condition and denoises $x_t$ toward $x$; at inference it samples the reconstruction with an Euler ODE solver. Two design choices carry the argument: using an ELBO-form objective (Flow Matching or v-prediction with cosine schedule) rather than non-ELBO alternatives such as $\\epsilon$-prediction or EDM, which cause color shifts in jointly trained latents; and noise synchronization, which during training noises $z$ at a random time $\\tau$ with probability 0.1 and asks the decoder to denoise pixel noises from $t \\in [\\tau,1]$, aligning the latent-space diffusion path with the pixel-space path. DiTo also replaces the KL regularizer on $z$ with LayerNorm.","core_discovery":"DiTo is a diffusion autoencoder trained jointly from scratch: an encoder maps an image $x$ to a compact latent $z$, and a decoder models $p(x|z)$ using a Flow Matching objective, which the paper connects to ELBO theory. The discovery is that this single L2 loss learns latents that are competitive with or better than the GAN-LPIPS recipe. At the XL scale, DiTo's reconstruction FID at 5K samples is 7.95 versus GLPTo-XL's 4.14, yet human side-by-side comparisons favor DiTo 52.44% when ties are counted as half, and DiTo-XL has the best PSNR and SSIM among the compared tokenizers; adding the LPIPS loss already used in GLPTo gives DiTo-XL the best rFID (3.53). On generation, a DiT-XL/2 trained on DiTo latents with noise synchronization reaches gFID 6.29 versus 7.49 for the same generator trained on GLPTo-XL latents. The paper also reports that DiTo's reconstruction quality keeps improving as the decoder grows, with no loss-weight rebalancing needed.","pith_inferences":["If the single-loss recipe holds, the same diffusion objective could plausibly train tokenizers for video, audio, and other continuous signals, since the loss itself is modality-agnostic.","The human-preference result suggests that reconstruction FID, computed in a supervised feature space, may be a misleading yardstick for tokenizer quality; preference tests or task-specific metrics could become the primary evidence.","Noise synchronization is a transferable idea for any latent diffusion pipeline: aligning how noise destroys latents with how it destroys pixels could improve other autoencoder-plus-diffusion stacks.","DiTo's zero-shot resolution generalization hints that fully convolutional diffusion decoders may eventually let one tokenizer serve multiple resolutions without retraining."],"forward_implications":["Tokenizer training can use a single self-supervised diffusion L2 loss, removing the need to tune weights among L1, LPIPS, and GAN losses.","Scaling up the diffusion decoder improves reconstruction faithfulness and image-generation quality without rebalancing loss hyperparameters.","Latent diffusion models trained on DiTo latents with noise synchronization outperform those trained on GLPTo-XL latents (gFID 6.29 versus 7.49).","Using ELBO-form diffusion objectives such as Flow Matching matters for learning the latent representation itself, not just for decoding.","DiTo transfers to higher-resolution inputs zero-shot: the rFID gap versus GLPTo narrows from 7.95 versus 4.14 at 256 pixels to 2.32 versus 2.13 at 512 pixels."],"supporting_citations":[{"why":"Provides the GAN-LPIPS tokenizer baseline and the encoder architecture that DiTo builds on.","marker":"Rombach et al., 2022"},{"why":"Supplies the ELBO theory that justifies which diffusion objectives maximize likelihood lower bounds.","marker":"Kingma & Gao, 2024"},{"why":"Defines the Flow Matching objective used as DiTo's single training loss.","marker":"Lipman et al., 2023"},{"why":"Provides the DiT latent diffusion model used to evaluate generation quality on DiTo and GLPTo latents.","marker":"Peebles & Xie, 2023"},{"why":"Defines the supervised perceptual loss used in the GLPTo recipe.","marker":"Zhang et al., 2018"},{"why":"Defines the adversarial loss used in the GLPTo recipe.","marker":"Goodfellow et al., 2020"},{"why":"Provides the convolutional UNet decoder architecture adapted by DiTo.","marker":"Song et al., 2023"},{"why":"Defines the Fréchet Inception Distance used for the reconstruction and generation metrics.","marker":"Heusel et al., 2017"},{"why":"Provides the ImageNet dataset used for training and evaluation.","marker":"Deng et al., 2009"}],"fun_headline_variants":["One L2 loss trains tokenizers that match GAN recipes","DiTo: self-supervised tokenizer with a single diffusion loss","Single loss, scalable: diffusion tokenizer beats GAN mix","DiTo replaces heuristic losses with one diffusion objective","Self-supervised DiTo: competitive tokenization via one loss"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that side-by-side human preference is the right test of reconstruction quality when it conflicts with reconstruction FID, because DiTo-XL's rFID@5K (7.95) is nearly double GLPTo-XL's (4.14) while the paper's better-reconstruction claim rests on a 52.44% preference rate.","fun_headline_variants_meta":{"raw":{"variants":["One L2 loss trains tokenizers that match GAN recipes","DiTo: self-supervised tokenizer with a single diffusion loss","Single loss, scalable: diffusion tokenizer beats GAN mix","DiTo replaces heuristic losses with one diffusion objective","Self-supervised DiTo: competitive tokenization via one loss"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000173,"raw_usage":{"total_tokens":1292,"prompt_tokens":971,"completion_tokens":321,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":587,"completion_tokens_details":{"reasoning_tokens":237}},"tokens_in":587,"tokens_out":321,"duration_ms":3678,"temperature":1.0,"reasoning_tokens":237,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T22:52:21.313538+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct test: collect more human preferences between DiTo-XL and GLPTo-XL at both 256px and 512px; if DiTo no longer wins at either resolution, or if a text-reading metric on symbol-heavy crops shows no advantage, the reconstruction claim fails. For the generation claim, retrain a DiT-XL/2 on DiTo-XL latents with and without noise synchronization over multiple seeds and check whether the gFID 6.29 versus 7.49 improvement reproduces within noise.","supporting_citations":[{"cited_title":"and Gao, R","cited_arxiv_id":null,"evidence_quote":"Supplies the ELBO theory that justifies which diffusion objectives maximize likelihood lower bounds."},{"cited_title":"Consistency models","cited_arxiv_id":null,"evidence_quote":"Provides the convolutional UNet decoder architecture adapted by DiTo."}],"review_version":1}