{"id":"37dc15b4-537b-4b51-a3a3-72e6badc15ea","arxiv_id":"2505.07812","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"Energy-based AutoRegression (EAR) uses a strictly proper energy score to train a masked autoregressive Transformer on continuous image tokens, reaching ImageNet 256x256 FID 1.97 at 937M parameters while generating images in about one second.","lead":"This paper introduces EAR, an autoregressive image model that predicts continuous visual tokens with an 'energy score' objective, avoiding both vector quantization and the many denoising steps of per-token diffusion. On ImageNet 256x256, EAR reaches FID 1.97 and generates an image in about one second, roughly ten times faster than the comparable diffusion-loss model MAR.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claim that continuous AR beats discrete AR is confounded by tokenizer quality; a matched-tokenizer discrete baseline is needed before attributing the gain to the energy-score objective.","rationale":"The reader's weakest assumption picks out exactly the confounding of objective and tokenizer in the headline comparison, and this is the most load-bearing concern because it directly affects the central claim of superiority over discrete autoregressive generation. The energy-loss framework is theoretically sound and empirically demonstrated, and the paper releases code, so there is no reason to reject or to demand a higher standard than conditional acceptance. The remaining concerns raised by the reader (ad hoc stabilizers lambda and tau_train, missing error bars) are secondary: they affect reproducibility and confidence intervals but do not threaten the core feasibility of the method. A matched-tokenizer discrete baseline would settle whether the empirical advantage comes from the continuous tokenizer or from the energy-score objective. Until that check is run, the paper should frame its contribution as 'continuous AR is viable and efficient' rather than 'continuous AR outperforms discrete AR.' This does not change the reader's CONDITIONAL verdict.","tokens_in":18734,"tokens_out":7983,"duration_ms":88339,"concrete_test":"Run a matched-tokenizer discrete-AR ablation. Train a discrete-AR Transformer at the same scales as EAR-B and EAR-H (about 205M and 937M parameters, same masked-AR schedule, CFG, and sampling settings) on KL-16 features quantized with a scheme whose reconstruction FID is close to 1.22 (for example, finite scalar quantization or a high-rate residual VQ). If this discrete model reaches FID around 1.97 or better, then the advantage attributed to the energy score is actually due to the KL tokenizer; if it remains clearly worse (e.g., FID above 3), the energy-score objective is the primary driver. A cheaper secondary check is to report the VQ-16 baseline's reconstruction FID alongside its generation FID and compute the gap to EAR under the same VQ-16 tokenizer.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central comparison in Section 5.2 (Table 1, Figure 3) varies two factors simultaneously: the training objective (energy score vs. cross-entropy) and the tokenizer (continuous KL-16 with reconstruction FID 1.22 vs. discrete VQ-16 with reconstruction FID 5.87). Since the tokenizer's reconstruction quality upper-bounds any downstream generation quality, the large FID gap in Figure 3 is largely expected even before the objective is changed. The paper acknowledges this bottleneck in Section 5.2, yet concludes that 'continuous tokenization with the energy loss consistently outperforms discrete tokenization with the cross-entropy loss.' That attribution is not established by the presented experiments. The energy-score method itself is not in question: EAR-H reaches a strong FID of 1.97 and offers a large speed advantage over MAR. What is unsupported is the broader claim of superiority over discrete autoregressive models. Table 1 itself lists discrete AR systems with better tokenizers (MAGVIT-v2 at FID 1.78, VAR-d30 at FID 1.92) that match or beat EAR-H's FID, so 'continuous beats discrete AR' is not generally true. The defensible central claim should be narrowed to: continuous autoregressive generation with an energy loss is a practical, efficient alternative whose quality approaches the tokenizer's reconstruction ceiling.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper introduces a Continuous Visual Autoregressive (VAR) framework that trains autoregressive models with strictly proper scoring rules, avoiding vector quantization. The main instantiation, Energy-based AutoRegression (EAR), maximizes the energy score for alpha in (0,2) via a two-sample unbiased estimator, with an MLP generator that samples continuous tokens in a single forward pass. Experiments on ImageNet 256x256 report EAR-H at FID 1.97 with 937M parameters and roughly one second per image, and the paper shows that alpha=2 (non-strictly proper) fails while alpha<2 succeeds. The paper also positions GIVT and diffusion loss as instances of the same framework via the logarithmic and Hyvärinen scores, respectively.","tokens_in":19033,"tokens_out":8458,"duration_ms":75696,"significance":"If the claims hold, the paper offers a principled, likelihood-free alternative to per-token diffusion for continuous autoregressive generation, with a large inference speed advantage. The theoretical connection between strictly proper scoring rules and autoregressive training is clearly presented, and the alpha=2 ablation is a clean empirical validation of strict propriety. The code release and detailed ablations (temperature, noise, CFG, learning rate, masking) support reproducibility. However, the headline comparison against discrete autoregressive models is not yet established because it conflates the training objective with tokenizer quality.","major_comments":[{"comment":"The comparison supporting the claim of superiority over discrete autoregressive generation is confounded by tokenizer quality. The continuous KL-16 tokenizer used for EAR has a reconstruction FID of 1.22, while the discrete VQ-16 tokenizer used for the baseline has a reconstruction FID of 5.87. Because the tokenizer's reconstruction error upper-bounds generation quality, the gap in Figure 3 is expected even if the energy-score objective and cross-entropy were equally effective. Moreover, Table 1 lists discrete autoregressive models with better tokenizers (MAGVIT-v2 at FID 1.78 and VAR-d30 at FID 1.92) that match or beat EAR-H's 1.97. The paper acknowledges the reconstruction bottleneck in Section 5.2 but still concludes that continuous tokenization with the energy loss consistently outperforms discrete tokenization with cross-entropy. I request either a matched-tokenizer discrete baseline (for example, a discrete autoregressive model trained on the same KL-16 latent space after quantization, or a cross-entropy model using MAGVIT-v2 or FSQ tokens) or a clearly qualified claim that restricts the comparison to the specific VQ-16 baseline and presents EAR as a practical alternative rather than a generally superior paradigm.","section":"Section 5.2, Figure 3, Table 1"},{"comment":"The actual training objective used for the final model is not exactly the strictly proper energy score. With tau_train=0.99, the fine-tuning loss becomes L = |x1-y|^alpha + |x2-y|^alpha - tau_train|x1-x2|^alpha, which is not an unbiased estimator of the negative energy score; the extra term (1-tau_train) E|x1-x2|^alpha penalizes the spread of the model distribution and breaks the strict-propriety rationale. The paper labels this as a fine-tuning heuristic, but the abstract's statement that 'all we need is to select a strictly proper score and set it as the training objective to optimize' is then not literally true for the reported final checkpoint. Please clarify how the temperature-weighted loss is justified within the proposed framework, or explicitly classify the temperature modification as an orthogonal regularization technique that falls outside the scoring-rule theory.","section":"Section 4.3 (Temperature) and Section 5.1"}],"minor_comments":[{"comment":"The caption contains a typo: 'gudiance' should be 'guidance'.","section":"Figure 6 caption"},{"comment":"The description of EAR-H as having a 'relatively modest model size among the leading systems' is misleading given that 937M parameters is larger than DiT-XL (675M), MAGVIT-v2 (307M), and comparable to MAR (943M); please rephrase to specify the comparison set for which the size is modest.","section":"Section 5.2"},{"comment":"The conclusion that dnoise=64 performs better rests on small differences (e.g., w/ cfg Uniform: 3.55 at dnoise=64 vs. 3.89 at dnoise=32); please report multiple seeds or confidence intervals to support this choice.","section":"Table 3"},{"comment":"The reference 'Alpha-VLLM' points to a GitHub repository rather than an archival publication; please cite the corresponding paper if available, or specify the exact version and commit used.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The core theoretical contribution is sound, and the experimental results are competitive, but the central comparison against discrete autoregressive models needs to be made fair before the paper can be accepted. The framework is a natural extension of the authors' ICML 2024 work on scoring rules for language modeling; the novelty here is the application to continuous visual generation and the energy-score estimator, which is sufficient for a full paper if the claims are properly scoped. I would be willing to look at a revised version."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: this paper is worth a serious look. The energy-score objective for continuous autoregressive image generation is a genuine new idea, and it works well enough to report FID 1.97 on ImageNet 256 with a 937M model and roughly 10x lower inference latency than the per-token diffusion baseline MAR. The theoretical grounding is correct: the energy score is strictly proper for alpha in (0,2), Eq. 10 is an unbiased two-sample estimator, and the alpha=2 ablation (FID 188.1) gives a clean empirical confirmation that strict propriety matters.\n\nWhat is new: previous continuous AR methods either constrained to a parametric family (GIVT with GMM) or needed many denoising steps (diffusion loss). The energy loss only requires two samples from the model distribution, so it enables a single-forward-pass MLP generator. The paper also frames GIVT and diffusion loss as instances of strictly proper score maximization (log score and Hyvarinen score, respectively), which is a nice unifying perspective. Code is released, which helps reproducibility.\n\nSoft spots, in order of importance. The headline claim in Section 5.2 that \"continuous tokenization with the energy loss consistently outperforms discrete tokenization with the cross-entropy loss\" is confounded. The comparison in Figure 3 varies both the objective and the tokenizer: KL-16 continuous has reconstruction FID 1.22, while the VQ-16 discrete has 5.87. Since reconstruction quality upper-bounds generation quality, much of the gap is likely due to the tokenizer, not the objective. The paper does acknowledge the reconstruction gap, but then still draws the stronger conclusion. Table 1 itself shows discrete models with better tokenizers (MAGVIT-v2 at 1.78, VAR-d30 at 1.92) beating EAR-H's 1.97, so \"continuous beats discrete\" is not generally true. A matched-tokenizer experiment or explicit decomposition is needed to support the attribution. Second, several techniques (tau_train, lambda=0.25 for the MLP generator's learning rate, noise type/dimension) are empirical stabilizers that are not derived from the scoring-rule framework. That is not fatal, but it weakens the \"all we need is to pick a proper score\" narrative. Third, FID/IS are reported without error bars; multiple seeds would make the main table more convincing.\n\nWho this is for: anyone working on autoregressive image generation, continuous tokenization, or scoring-rule objectives. The contribution is real and the experiments are honest in most places. It deserves a serious referee. My recommendation: send it to review, and ask the authors to re-run the discrete baseline with a matched-reconstruction tokenizer, or at least to soften the \"beats discrete\" claim to what the data actually support.","headline":"Energy-score autoregression works and is fast, but the paper overclaims 'continuous beats discrete' by confounding the objective with tokenizer quality.","tokens_in":19596,"tokens_out":2676,"would_cite":true,"duration_ms":25180,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Training an autoregressive Transformer to maximize the energy score generates visual tokens directly, reaching FID 1.97 on ImageNet 256x256 without vector quantization and cutting inference time to about one second.","keywords":["continuous autoregressive image generation","energy score","strictly proper scoring rules","vector quantization","visual autoregressive modeling","diffusion loss","masked autoregressive Transformer","image generation efficiency"],"falsifier":"Retrain the discrete autoregressive Transformer on a discrete tokenization that matches the KL tokenizer's reconstruction quality (reconstruction FID near 1.22 rather than 5.87), with the same model size, steps, and guidance, and compare FID with EAR-H's 1.97 on ImageNet 256x256; a match or a win for the discrete model would falsify the paper's central claim that continuous autoregression is what delivers the gain.","tokens_in":18509,"feed_emoji":"🖼️","tokens_out":9392,"duration_ms":83132,"temperature":0.7,"pith_summary":"The paper claims that an autoregressive Transformer can generate continuous visual tokens directly, with no vector quantization, if it is trained to maximize the energy score, a strictly proper scoring rule whose expected value is uniquely maximized by the true distribution. On ImageNet 256x256 class-conditional generation, the largest model (EAR-H, 937M parameters) reaches FID 1.97, and a single image is produced in roughly one second, about ten times faster than per-token diffusion (MAR) at comparable quality. The energy loss needs only two samples from the model's own predictive distribution, so the model is free to define that distribution implicitly through a small MLP generator rather than through a constrained parametric family. This matters because quantization error in discrete tokens upper-bounds image quality, and per-token diffusion is slow; the framework also recovers GIVT and diffusion loss as special cases under different strictly proper scores.","feed_headline":"Continuous autoregression hits ImageNet FID 1.97 without quantization","feed_subtitle":"Training on a strictly proper energy score removes vector quantization and cuts image generation to about one second.","key_machinery":"The central object is the energy score as a strictly proper scoring rule, paired with an MLP generator as the implicit sampler. For $\\alpha\\in(0,2)$, the energy score $S(p,y)=E[|x_1-x_2|^\\alpha]-2E[|x-y|^\\alpha]$ is strictly proper because the generalized energy distance $2E[|x-y|^\\alpha]-E[|x_1-x_2|^\\alpha]-E[|y_1-y_2|^\\alpha]$ is nonnegative and zero only when $p=q$; this unique optimizer is what turns the negative score into a principled loss rather than an ad hoc distance. The MLP generator consumes the Transformer hidden state and a random noise vector, injecting the noise through adaptive layer normalization in residual blocks, so that the conditional distribution of the next continuous token is defined by the sampling process itself. The energy loss then estimates the score with two draws from this sampler, making the whole objective likelihood-free and single-pass at inference.","core_discovery":"On the paper's own terms, the discovery is that continuous visual autoregressive generation can be trained as score maximization: select a strictly proper scoring rule and optimize its negative expectation at every autoregressive step. The concrete instantiation is the energy score $S(p,y)=E[|x_1-x_2|^\\alpha]-2E[|x-y|^\\alpha]$ with $\\alpha\\in(0,2)$, whose negative expected value is the generalized energy distance and equals zero only when the predicted distribution $p$ and the true conditional distribution $q$ coincide. Because the score can be estimated from two independent samples of the model distribution, the loss $\\mathcal{L}=|x_1-y|^\\alpha+|x_2-y|^\\alpha-|x_1-x_2|^\\alpha$ requires no explicit density, so an MLP generator with injected noise can define an implicit predictive distribution. Trained with masked autoregression, classifier-free guidance, and temperature scheduling on ImageNet 256x256, the largest model (EAR-H, 937M parameters) reaches FID 1.97 with a generation time of roughly one second, compared with about ten seconds for the per-token diffusion baseline (MAR) at comparable FID. The paper further shows GIVT and diffusion loss are special cases of the same framework via the logarithmic score and the Hyvarinen score, respectively.","pith_inferences":["The reported advantage over discrete autoregression is a bundled comparison: the energy objective and the continuous KL tokenizer (reconstruction FID 1.22) change together against a VQ tokenizer with reconstruction FID 5.87. A discrete autoregressive model trained on an equally strong tokenizer would isolate whether continuous autoregression itself, rather than the better tokenizer, is what drives","The two-sample energy estimator is high-variance, and the paper's fixes, including a reduced learning rate for the MLP generator and the collapse for $\\alpha<1$, suggest that estimator variance, not just strict propriety, is what actually limits trainability.","A natural testable extension is applying the energy objective to continuous latent language modeling or video and audio tokens, where the same single-pass advantage could be measured against per-token diffusion baselines."],"forward_implications":["Discrete image tokenizers are no longer required for autoregressive generation; continuous tokens can be supervised directly, avoiding the reconstruction bottleneck imposed by vector quantization.","Inference is one forward pass per token: a 256x256 image is generated in about one second, roughly ten times faster than per-token diffusion at comparable quality.","Any strictly proper score for continuous variables is a valid training objective, so the framework opens a family of likelihood-free losses beyond the energy score.","The framework absorbs GIVT and diffusion loss as special cases, which means improvements in scoring-rule theory translate into concrete new autoregressive objectives for visual data.","Generation quality improves monotonically from EAR-B to EAR-H, so scaling the energy Transformer is a direct route to further FID reduction."],"supporting_citations":[{"why":"Defines the energy score and establishes strict propriety for α in (0,2), the theoretical foundation of the energy loss.","marker":"(Székely, 2003)"},{"why":"Supplies the theory of strictly proper scoring rules that frames the Continuous VAR objective.","marker":"(Gneiting & Raftery, 2007)"},{"why":"Introduces the sequence-level scoring-rule loss for autoregressive models that this paper adapts to continuous visual tokens.","marker":"(Shao et al., 2024)"},{"why":"Provides the closest baseline MAR with diffusion loss, the KL-16 continuous tokenizer, and the masked autoregressive and scaling setup used by EAR.","marker":"(Li et al., 2024)"},{"why":"GIVT is the prior continuous autoregressive approach that the paper re-derives as logarithmic-score maximization.","marker":"(Tschannen et al., 2023)"},{"why":"Introduces the Hyvärinen score used to identify per-token diffusion loss as another instance of the framework.","marker":"(Hyvärinen, 2005)"},{"why":"Supplies the VQ-16 discrete tokenizer and its reconstruction FID (5.87), the comparison point that motivates continuous tokens.","marker":"(Rombach et al., 2022)"},{"why":"Classifier-free guidance at inference is essential to the reported FID gains for EAR.","marker":"(Ho & Salimans, 2022)"}],"fun_headline_variants":["Continuous autoregression without quantization hits FID 1.97","Score maximization yields quantization-free visual autoregression","Energy score training: FID 1.97 without vector quantization","Continuous VAR via score maximization beats quantization","Autoregressive generation goes continuous: FID 1.97 on ImageNet"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the quality gain over discrete autoregressive models comes from the continuous energy-score framework and not from the much better reconstruction of the KL tokenizer, since the head-to-head comparison changes both the objective and the tokenizer at once.","fun_headline_variants_meta":{"raw":{"variants":["Continuous autoregression without quantization hits FID 1.97","Score maximization yields quantization-free visual autoregression","Energy score training: FID 1.97 without vector quantization","Continuous VAR via score maximization beats quantization","Autoregressive generation goes continuous: FID 1.97 on ImageNet"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000203,"raw_usage":{"total_tokens":1412,"prompt_tokens":999,"completion_tokens":413,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":615,"completion_tokens_details":{"reasoning_tokens":329}},"tokens_in":615,"tokens_out":413,"duration_ms":3756,"temperature":1.0,"reasoning_tokens":329,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:08:16.389755+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain the discrete autoregressive Transformer on a discrete tokenization that matches the KL tokenizer's reconstruction quality (reconstruction FID near 1.22 rather than 5.87), with the same model size, steps, and guidance, and compare FID with EAR-H's 1.97 on ImageNet 256x256; a match or a win for the discrete model would falsify the paper's central claim that continuous autoregression is what delivers the gain.","supporting_citations":[],"review_version":1}