Pith. sign in

REVIEW 3 major objections 4 minor 38 references

Frequency-Aware Autoregressive Modeling for Efficient High-Resolution Image Synthesis

T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SparseVAR claims that low-frequency tokens at high-resolution stages can be safely skipped, cutting next-scale image-model latency roughly in half with negligible quality loss.

desk verdict Useful, honest efficiency paper: the 2x training-free speedup on Infinity-2B holds at its chosen operating point, but the frequency-awareness mechanism is under-tested without a random-mask control and tau needs per-model tuning. read the letter →

arxiv 2507.20454 v1 pith:ES4NHBDI submitted 2025-07-28 cs.CV cs.LG

Zhuokun Chen , Jugang Fan , Zhuowei Yu , Bohan Zhuang , Mingkui Tan This is my paper · ORCID
classification cs.CVcs.LG
keywords next-scalepredictionvisualautoregressivemodeltokenreductioninferenceaccelerationlow-frequencytokensanchortext-to-imagegenerationsparse
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper sets out to show that the high-resolution stages of next-scale prediction image models spend most of their computation on redundant tokens. Decoded-image differences between consecutive stages concentrate in high-frequency regions such as edges, while smooth background areas barely change, so low-frequency tokens can be dropped once a model reaches higher resolutions. SparseVAR identifies those tokens from the mean-squared change of features in a selected transformer block, excludes them from the current and all later stages, and keeps a small regular grid of anchor tokens whose logits are copied into the excluded positions when similar enough. The key result is that this training-free mask cuts Infinity-2B inference from 2.78 s to 1.36 s at threshold $\tau=0.7$ while the GenEval overall score moves only from 0.7260 to 0.7209. This matters because late-stage high-resolution tokens dominate inference cost in visual autoregressive models, so a cheap frequency mask offers a practical speedup without a new training run.

What carries the argument

The load-bearing object is the MSE change map computed inside one block of the model: $$\$\Delta$ $F^{{s}}$_{k-1}(i,j) = \frac{1}{C}\sum_{c=1}^{C}\left($F^{{s}}$_{k-1}(i,j,c) - $F^{{s-1}}$_{k-1}(i,j,c)\right)^2,$$ where $s$ is the selected block and $C$ the channel dimension. After interpolation to stage $k$, the exclusion set $$\mathcal{M}^{k}_{\mathrm{low}} = \{(i,j) \mid \tilde{F}_k(i,j) < \tau \max(\tilde{F}_k)\}$$ sends those positions out of the computation for stage $k$ and later stages. Anchor retention is the second mechanism: the top-left corner of every $\alpha \times \alpha$ grid is kept, and each excluded token receives the logits of its most similar anchor when the cosine similarity of the interpolated previous-stage logits exceeds threshold $\beta$, otherwise its residual is set to zero. These two mechanisms turn the sparsity decision into a lightweight per-image choice of one block's MSE map and a threshold, avoiding clustering or attention ranking that would itself be expensive at high resolution.

What would settle it

Take a set of generated images and, for each, compute the interpolated MSE map at the selected block, threshold it at $\tau$, then run inference with the excluded tokens forced to zero residual or to anchor-copied logits. If images with fine detail inside the excluded regions such as text, faces, or texture on a non-flat background degrade visibly while the mask still excludes most tokens, the frequency proxy is misaligned. Quantitatively, one could measure the correlation between each token's interpolated block-MSE value and the actual change that removing that token produces in the final decoded image; a low or negative correlation would refute the paper's premise.

Watch

Extended reading notes

Core claim

SparseVAR's central claim is that low-frequency tokens are largely inert during the high-resolution stages of next-scale prediction and can be excluded with almost no image-quality cost. The method rests on two observations: the per-stage residuals of HART, Infinity, and FlexVAR concentrate their effect in high-frequency regions, and different transformer blocks within the model attend to different spatial frequencies. A selected block's feature MSE change map from stage $k-1$ is interpolated to stage $k$; tokens whose interpolated values fall below $\tau$ times the map maximum are marked low-frequency and skipped, while the skipped positions either receive the logits of their most similar anchor token or a zero residual. At $\tau=0.7$ on Infinity-2B the latency drops from 2.78 s to 1.36 s with GenEval moving from 0.7260 to 0.7209, and at $\tau=0.6$ on HART-0.7B the latency drops from 1.32 s to 0.99 s with the GenEval score essentially flat. The appendix applies the same procedure to FlexVAR, supporting the stronger claim that the redundancy is a property of multi-stage refinement itself, not of the residual VAE designs used by HART and Infinity.

Load-bearing premise

The acceleration stands or falls with the assumption that a selected block's interpolated feature MSE change at the previous scale correctly identifies which tokens can be skipped at the current and later scales without visible harm; the paper supports this with qualitative visualizations and a block-selection experiment rather than a direct per-token measurement.

Editorial extensions

If this is right

  • Any next-scale prediction model can be accelerated at inference time by adding this mask and a small number of anchor copies, with no retraining and no change to the pretrained weights.
  • At 1024x1024, users of Infinity-2B can expect roughly a doubling of generative speed at threshold $\tau=0.7$ while text-to-image alignment on GenEval shifts by only about half a percent.
  • Because the redundancy appears in HART, Infinity, and FlexVAR, the speedup should become larger at even higher resolutions, where late stages process many more tokens and low-frequency regions dominate the token count.
  • The quality of the acceleration depends on reading the right block's MSE map, so a practical deployment should search the block index once per model family rather than treating the same block as universally valid.
  • Anchor tokens are load-bearing for HART's residual diffusion path: without them, low-frequency regions lose the inputs the diffusion refinement expects, and quality degrades more than it does for direct next-scale models.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural untested extension is to combine frequency masking with KV-cache or attention sparsification, since the skipped low-frequency tokens are precisely the positions whose long-range attention is weakest.
  • Because anchors are placed on a fixed grid, an adaptive scheme that places anchors along detected edges or texture boundaries would likely match or exceed the same quality while using fewer anchor tokens.
  • The reported latencies are single-GPU, batch-1 measurements on an NVIDIA 3090; production serving with larger batches or tensor-parallel execution may compress the wall-clock gain, even though the reduction in computed tokens should transfer.
  • The same frequency-proxy idea may carry over to high-resolution diffusion transformers, which also waste computation on flat backgrounds, though the paper deliberately restricts its claim to next-scale autoregressive models.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes SparseVAR, a training-free inference acceleration framework for next-scale prediction visual autoregressive models. The method uses the per-token MSE change of features in a selected transformer block to identify low-frequency tokens, excludes those tokens from computation in high-resolution stages, and retains a small set of uniformly sampled anchor tokens whose logits are copied to excluded tokens when cosine similarity is sufficiently high. Experiments on Infinity-2B and HART-0.7B report roughly 2x speedup on Infinity with a GenEval overall score drop from 0.7260 to 0.7209 at tau=0.7, as well as smaller speedups on HART and FlexVAR. The paper also compares against token-reduction baselines and ablates the metric choice, anchor grid size, and starting stage.

Significance. If the frequency-aware exclusion claim is validated, SparseVAR is a practically useful plug-and-play accelerator for high-resolution autoregressive image generation: it requires no retraining, ships with code, and reports comparisons against ToMe, PiToMe, ZipVL, and SparseViT. The additional FlexVAR experiment is a thoughtful check that the redundancy is not solely an artifact of the residual VAE used by HART and Infinity. However, the manuscript's central conceptual claim, that low-frequency tokens are intrinsically skippable, is not yet directly supported by a same-budget randomized control, and a specification inconsistency in the block-selection setup must be fixed before the results are reproducible.

major comments (3)
  1. [Section 5, Figures 6 and 10] The experiments do not include a same-budget control that randomizes which tokens are excluded while holding the exclusion count, anchor placement, and anchor-copy repair mechanism fixed. Figure 6 compares MSE, logits, and L1 as scoring metrics, and Figure 10 compares against ToMe, PiToMe, ZipVL, and SparseViT, but no experiment preserves the sparsity ratio and anchor strategy while varying only the identity of the dropped tokens. Without such a control, the reported quality-latency tradeoff could in principle be produced by the anchor-copying mechanism in Section 4.3 with any sparsity pattern, leaving the paper's central frequency-awareness premise untested. I ask the authors to add a random-mask ablation at matched latency/sparsity operating points.
  2. [Section 5, 'Impact of block selection'; Appendix E.2; Figure 8] The selected block index s, which controls the MSE map in Eq. (1), is specified inconsistently. The main text states that experiments were conducted on Infinity with tau=0.6 and that the 16th block achieved the best results, but the caption of Figure 8 says the evaluation uses HART-0.7B, and Appendix E.2 states that the 3rd block is selected for Infinity-2B while the 16th block is selected for HART-0.7B. Since s is a free parameter that affects the exclusion mask and the reported latency, the manuscript must state unambiguously which block is used for each model and reconcile the text, figure caption, and appendix.
  3. [Tables 1, 3, 4, 5; Figure 8] The headline operating point is selected on the evaluation benchmarks, and the tau threshold does not transfer across models. Table 1 reports tau=0.7 for Infinity, but Table 3 shows that HART-0.7B at tau=0.7 collapses on human-preference metrics (ImageReward drops from 0.8656 to 0.4333 and HPSv2.1 from 29.53 to 25.71). Because tau, alpha, beta, P, and the block index are all chosen after inspecting GenEval/DPG-Bench results, the 'minimal degradation' claim is at risk of overfitting to the reported settings. The authors should either tune on a separate calibration split or report a selection rule based on a target latency budget, and they should report the per-prompt distribution of the exclusion ratio and quality change for the chosen operating point.
minor comments (4)
  1. [Section 3 heading] The heading 'Empircal Insights' contains a typo and should read 'Empirical Insights'.
  2. [Section 5, 'Implementation details'] The implementation details state beta=0.9 and P=10 but do not state the value of alpha in the main text; alpha=4 appears only in the Table 1 caption. Please state all fixed hyperparameters for each model in Section 5.
  3. [Section 4.2, Eq. (2)] Please specify the interpolation method used to map Delta F^s_{k-1} to the resolution of stage k, and clarify that the max in Eq. (2) is taken per image over the interpolated map; this affects the reproducibility of the exclusion masks.
  4. [Algorithm 1 and Section 4.3] Please clarify whether Eq. (1) is computed on the pruned feature map or the full feature map after anchor-copy assignment, since this affects the exclusion mask propagated to the next stage. Also, the '-' row in Table 4 should explicitly state that excluded tokens receive zero residuals when no anchors are kept.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: SparseVAR's quality numbers are measured on external benchmarks; the method's components are empirical heuristics with hyperparameters selected by ablations, not quantities derived from the reported results by construction.

full rationale

The claimed derivation chain is an empirical heuristic rather than a formal derivation. Observations 1-2 motivate skipping tokens whose interpolated block-wise MSE change is small (Eqs. 1-2); anchor tokens then copy previous-stage logits (Eq. 3, Sec. 4.3); and the resulting quality is measured on GenEval, DPG-Bench, ImageReward, and HPSv2.1. None of the reported scores follows algebraically from Eq. (1)-(3), so the conclusions do not reduce to the method's definitions by construction. The selected block index s is chosen via the ablation in Fig. 8, and tau, alpha, beta are chosen via Tables 1-5; this is hyperparameter selection on evaluation data, not a fitted parameter being renamed as a prediction, because the reported latencies and GenEval/DPG/HPS scores are actual measurements rather than consequences of the selection criterion. The only author-overlapping citations ([11] ZipAR and [12] ZipVL) are used as related work, attention-locality context, and baseline comparisons, not as a load-bearing uniqueness theorem or as justification of the central claim. A missing same-budget random-mask control would weaken the causal attribution of the speed/quality tradeoff to low-frequency identity, but that is an experimental-control limitation, not circular derivation. The paper is best scored as containing no significant circularity.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method introduces no new physical or architectural entities. Its contribution is a set of heuristics (MSE frequency mask, uniform anchors, logit copying) governed by five tuned hyperparameters. The central premise, that an interpolated block-level MSE map reliably flags excludable low-frequency tokens, is ad hoc and validated only by qualitative evidence and benchmark tuning.

free parameters (5)
  • tau (exclusion threshold) = 0.7 for Infinity, 0.6 for HART (swept 0.4-0.7)
    Controls the fraction of tokens excluded as low-frequency; headline results use the value that best balances latency and quality on GenEval/DPG-Bench.
  • alpha (anchor grid size) = 4
    Uniform anchor sampling interval; chosen from {2,3,4,5} for best quality-efficiency trade-off on GenEval.
  • beta (similarity threshold) = 0.9
    Threshold for copying the most similar anchor logits; set without a sensitivity study.
  • P (first excluded stage) = 10
    Stage at which token exclusion begins; chosen from {6,8,9,10,11,12} on GenEval.
  • s (selected block for MSE map) = 16 for HART, 3 for Infinity
    Block used to compute the MSE change map; selected by maximizing GenEval score across blocks (Figure 8).
assumptions (5)
  • ad hoc to paper The MSE change inside a selected transformer block, interpolated across scales, is a faithful proxy for image-space residual importance.
    Core heuristic of SparseVAR; supported only by qualitative visualizations and the block-selection experiment, with no independent theoretical or quantitative validation.
  • ad hoc to paper Low-frequency tokens in high-resolution stages produce negligible residuals that can be replaced by logits copied from the most similar anchor token.
    Depends on the local-similarity observation in Appendix E.3; the fixed beta=0.9 threshold and uniform anchor sampling are justified post hoc.
  • domain assumption Next-scale prediction models operate by predicting logits, mapping them to residuals via a codebook, and interpolating accumulated residuals.
    Standard framework inherited from VAR/HART/Infinity and stated in Section 4.1.
  • domain assumption Inference latency is dominated by the later high-resolution stages where token count is large.
    Cost grows with the square of the token count; this motivates excluding tokens from stage P onward and underpins the complexity analysis.
  • domain assumption Model weights and tokenizers of HART and Infinity are treated as fixed, well-behaved inputs.
    SparseVAR does not modify or train the base model; correctness depends on the base model continuing to behave as reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Frequency-Aware Autoregressive Modeling for Efficient High-Resolution Image Synthesis." pith.science (2026). https://pith.science/paper/ES4NHBDI

@misc{pith2026250720454,
  author       = {Pith},
  title        = {Pith review of: Frequency-Aware Autoregressive Modeling for Efficient High-Resolution Image Synthesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ES4NHBDI}},
  note         = {Machine review of arXiv:2507.20454}
}
read the original abstract

Visual autoregressive modeling, based on the next-scale prediction paradigm, exhibits notable advantages in image quality and model scalability over traditional autoregressive and diffusion models. It generates images by progressively refining resolution across multiple stages. However, the computational overhead in high-resolution stages remains a critical challenge due to the substantial number of tokens involved. In this paper, we introduce SparseVAR, a plug-and-play acceleration framework for next-scale prediction that dynamically excludes low-frequency tokens during inference without requiring additional training. Our approach is motivated by the observation that tokens in low-frequency regions have a negligible impact on image quality in high-resolution stages and exhibit strong similarity with neighboring tokens. Additionally, we observe that different blocks in the next-scale prediction model focus on distinct regions, with some concentrating on high-frequency areas. SparseVAR leverages these insights by employing lightweight MSE-based metrics to identify low-frequency tokens while preserving the fidelity of excluded regions through a small set of uniformly sampled anchor tokens. By significantly reducing the computational cost while maintaining high image generation quality, SparseVAR achieves notable acceleration in both HART and Infinity. Specifically, SparseVAR achieves up to a 2 times speedup with minimal quality degradation in Infinity-2B.

Figures

Figures reproduced from arXiv: 2507.20454 by the authors.

Figure 1
Figure 1. Existing visual autoregressive models allocate uniform [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. High-resolution stages have minimal impact on low-frequency regions. We visualize the images generated by the last five higher-resolution stages of HART-0.7B (top), along with the ℓ1 difference between each stage and its previous stage (bottom). The redder areas indicate where the ℓ1 difference is larger, and these areas are mostly concentrated in the high-frequency regions of the foreground. In contrast, the ℓ1 cha… view at source ↗
Figure 3
Figure 3. Different blocks in next-scale prediction models tend to focus on distinct regions. We visualize the MSE changes before and after feature inference at the 16th and 21st blocks during stages 10-12 of HART-0.7B. It is clear that different blocks exhibit distinct regional focus tendencies. duces a learnable token merging technique for long-form video inputs, considering both token similarity and saliency. LLava-PruMerg… view at source ↗
Figures from the paper (16 more)
Figure 4
Figure 4. Figure 4: Overview of dynamic exclusion in SparseVAR. SparseVAR dynamically identifies and excludes low-frequency tokens starting [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Overview of retention of anchor tokens in Sparse [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]
Figure 6
Figure 6. Figure 6: Comparison of different met￾rics for distinguishing high- and low￾frequency regions. efficiency, we set P = 10. Comparisons with different early exiting metrics. To com￾pare the effectiveness of different metrics for distinguishing high- and low-frequency regions for e…
Figure 7
Figure 7. Figure 7: Qualitative visualizations of SparseVAR. The yellow and purple colors represent the tokens identified as retained and early-exited [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Impact of block selection on MSE-based frequency [PITH_FULL_IMAGE:figures/full_fig_p008_8.png]
Figure 9
Figure 9. Figure 9: ℓ1 difference between consecutive stages in FlexVAR. Redder areas indicate greater refinement. Low-frequency regions show minimal change in later stages. We then apply SparseVAR to FlexVAR by skipping token prediction in low-frequency regions and using interpolated val…
Figure 10
Figure 10. Figure 10: Comparison of token reduction methods on GenEval using HART-0.7B. [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Visualization results on complex scenes with multiple subjects and high-frequency details (e.g., text, faces, groups). SparseVAR [PITH_FULL_IMAGE:figures/full_fig_p012_11.png]
Figure 12
Figure 12. Figure 12: More visualizations of HART-0.7B. Stage 9 Stage 10 Stage 11 Stage 12 Generated Image [PITH_FULL_IMAGE:figures/full_fig_p013_12.png]
Figure 13
Figure 13. Figure 13: More visualizations of Infinity-2B. E.2. Distinct Regional Focus of Blocks in Next-Scale Prediction Models In this section, we present more detailed visualizations of the MSE changes for some blocks in the final five stages. As shown in [PITH_FULL_IMAGE:figures/full_…
Figure 14
Figure 14. Figure 14: More visualizations of HART-0.7B. Stage 8 Block 2 Block 3 Block 4 Block 5 Block 6 Block 7 Stage 9 Stage 10 Stage 11 Stage 12 Stage 13 [PITH_FULL_IMAGE:figures/full_fig_p014_14.png]
Figure 15
Figure 15. Figure 15: More visualizations of Infinity-2B. in low-frequency regions, we visualize the average cosine similarity of logits between each token and its neighboring 3×3 region at stages 8-11 of HART and Infinity. As shown in [PITH_FULL_IMAGE:figures/full_fig_p014_15.png]
Figure 16
Figure 16. Figure 16: More visualizations of HART-0.7B. Generated Image Stage 8 Stage 9 Stage 10 Stage 11 [PITH_FULL_IMAGE:figures/full_fig_p015_16.png]
Figure 17
Figure 17. Figure 17: More visualizations of Infinity-2B [PITH_FULL_IMAGE:figures/full_fig_p015_17.png]
Figure 18
Figure 18. Figure 18: Visualization of attention maps of SparseVAR. The x-axis represents the indices of all tokens in the current stage and the KV [PITH_FULL_IMAGE:figures/full_fig_p016_18.png]
Figure 19
Figure 19. Figure 19: Qualitative visualizations of SparseVAR. [PITH_FULL_IMAGE:figures/full_fig_p017_19.png]

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

38 extracted references · 24 canonical work pages

  1. [1]

    Arif, K. H. I., Yoon, J., Nikolopoulos, D. S., Vandierendonck, H., John, D., and Ji, B. Hired: Attention-guided token dropping for efficient infer- ence of high-resolution vision-language models in resource-constrained environments. arXiv preprint arXiv:2408.10945, 2024. 2, 3

    arXiv 2024

  2. [2]

    Improving image generation with better captions

    Betker, J., Goh, G., Jing, L., Brooks, T., Wang, J., Li, L., Ouyang, L., Zhuang, J., Lee, J., Guo, Y ., et al. Improving image generation with better captions. Com- puter Science. https://cdn. openai. com/papers/dall-e-3. pdf, 2(3):8, 2023. 3

    work page 2023

  3. [3]

    Token merging: Your vit but faster

    Bolya, D., Fu, C., Dai, X., Zhang, P., Feichtenhofer, C., and Hoffman, J. Token merging: Your vit but faster. In ICLR. OpenReview.net, 2023. 2, 3, 11

    work page 2023

  4. [4]

    T., Luo, P., Lu, H., and Li, Z

    Chen, J., Yu, J., Ge, C., Yao, L., Xie, E., Wang, Z., Kwok, J. T., Luo, P., Lu, H., and Li, Z. Pixart- α: Fast training of diffusion transformer for photorealistic text-to-image synthesis. In ICLR, 2024. 3

    work page 2024

  5. [5]

    An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models

    Chen, L., Zhao, H., Liu, T., Bai, S., Lin, J., Zhou, C., and Chang, B. An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. In ECCV, pp. 19–35. Springer,

    work page

  6. [6]

    Optical flow distillation: Towards efficient and stable video style transfer

    Chen, X., Zhang, Y ., Wang, Y ., Shu, H., Xu, C., and Xu, C. Optical flow distillation: Towards efficient and stable video style transfer. In Computer Vision– ECCV 2020: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part VI 16, pp. 614–

    work page 2020

  7. [7]

    Cogview: Mastering text-to-image generation via trans- formers

    Ding, M., Yang, Z., Hong, W., Zheng, W., Zhou, C., Yin, D., Lin, J., Zou, X., Shao, Z., Yang, H., et al. Cogview: Mastering text-to-image generation via trans- formers. Advances in neural information processing systems, 34:19822–19835, 2021. 1

    work page 2021

  8. [8]

    Taming trans- formers for high-resolution image synthesis

    Esser, P., Rombach, R., and Ommer, B. Taming trans- formers for high-resolution image synthesis. In CVPR, pp. 12873–12883, 2021. 3

    work page 2021

Show all 38 references
  1. [9]

    Geneval: An object-focused framework for evaluating text-to-image alignment

    Ghosh, D., Hajishirzi, H., and Schmidt, L. Geneval: An object-focused framework for evaluating text-to-image alignment. NeurIPS, 36, 2024. 5

    2024

  2. [10]

    Infinity: Scaling bitwise autore- gressive modeling for high-resolution image synthesis

    Han, J., Liu, J., Jiang, Y ., Yan, B., Zhang, Y ., Yuan, Z., Peng, B., and Liu, X. Infinity: Scaling bitwise autore- gressive modeling for high-resolution image synthesis. arXiv preprint arXiv:2412.04431, 2024. 1, 3, 5

    2024 arXiv

  3. [11]

    Zipar: Accelerating autoregressive im- age generation through spatial locality

    He, Y ., Chen, F., He, Y ., He, S., Zhou, H., Zhang, K., and Zhuang, B. Zipar: Accelerating autoregressive im- age generation through spatial locality. arXiv preprint arXiv:2412.04062, 2024. 2

    2024 arXiv

  4. [12]

    Zipvl: Efficient large vision-language models with dynamic token sparsification and kv cache compression

    He, Y ., Chen, F., Liu, J., Shao, W., Zhou, H., Zhang, K., and Zhuang, B. Zipvl: Efficient large vision-language models with dynamic token sparsification and kv cache compression. arXiv preprint arXiv:2410.08584, 2024. 2, 3, 11

    2024 arXiv

  5. [13]

    Ella: Equip diffusion models with llm for enhanced semantic alignment

    Hu, X., Wang, R., Fang, Y ., Fu, B., Cheng, P., and Yu, G. Ella: Equip diffusion models with llm for enhanced semantic alignment. arXiv preprint arXiv:2403.05135,

    arXiv

  6. [14]

    Autoregressive image generation using residual quantization

    Lee, D., Kim, C., Kim, S., Cho, M., and Han, W.- S. Autoregressive image generation using residual quantization. In CVPR, pp. 11523–11532, 2022. 1

    2022

  7. [15]

    Autoregressive image generation using residual quantization

    Lee, D., Kim, C., Kim, S., Cho, M., and Han, W.- S. Autoregressive image generation using residual quantization. In CVPR, pp. 11523–11532, 2022. 3

    2022

  8. [16]

    Video token merging for long-form video understand- ing

    Lee, S.-H., Wang, J., Zhang, Z., Fan, D., and Li, X. Video token merging for long-form video understand- ing. arXiv preprint arXiv:2410.23782, 2024. 2, 3

    2024 arXiv

  9. [17]

    Control- lable text-to-image generation

    Li, B., Qi, X., Lukasiewicz, T., and Torr, P. Control- lable text-to-image generation. NeurIPS, 32, 2019. 1

    2019

  10. [18]

    Imagefolder: Autoregressive image generation with folded tokens

    Li, X., Qiu, K., Chen, H., Kuen, J., Gu, J., Raj, B., and Lin, Z. Imagefolder: Autoregressive image generation with folded tokens. arXiv preprint arXiv:2410.01756,

    arXiv

  11. [19]

    Controlvar: Exploring control- lable visual autoregressive modeling

    Li, X., Qiu, K., Chen, H., Kuen, J., Lin, Z., Singh, R., and Raj, B. Controlvar: Exploring control- lable visual autoregressive modeling. arXiv preprint arXiv:2406.09750, 2024

    2024 arXiv

  12. [20]

    Finite scalar quantization: Vq-vae made simple

    Mentzer, F., Minnen, D., Agustsson, E., and Tschannen, M. Finite scalar quantization: Vq-vae made simple. In ICLR. 1

  13. [21]

    SDXL: improving latent diffusion models for high-resolution image synthesis

    Podell, D., English, Z., Lacey, K., Blattmann, A., Dock- horn, T., M¨uller, J., Penna, J., and Rombach, R. SDXL: improving latent diffusion models for high-resolution image synthesis. In ICLR, 2024. 3

    2024

  14. [22]

    Zero-shot text-to-image generation

    Ramesh, A., Pavlov, M., Goh, G., Gray, S., V oss, C., Radford, A., Chen, M., and Sutskever, I. Zero-shot text-to-image generation. In ICML, pp. 8821–8831. Pmlr, 2021. 1

    2021

  15. [23]

    Gen- erating diverse high-fidelity images with vq-vae-2

    Razavi, A., Van den Oord, A., and Vinyals, O. Gen- erating diverse high-fidelity images with vq-vae-2. NeurIPS, 32, 2019. 3

    2019

  16. [24]

    Generative adversarial text to image synthesis

    Reed, S., Akata, Z., Yan, X., Logeswaran, L., Schiele, B., and Lee, H. Generative adversarial text to image synthesis. In ICML, pp. 1060–1069. PMLR, 2016. 1

    2016

  17. [25]

    High-resolution image synthesis with latent diffusion models

    Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. High-resolution image synthesis with latent diffusion models. In CVPR, pp. 10684–10695, 2022. 1

    2022

  18. [26]

    J., and Yan, Y

    Shang, Y ., Cai, M., Xu, B., Lee, Y . J., and Yan, Y . Llava- prumerge: Adaptive token reduction for efficient large multimodal models. arXiv preprint arXiv:2403.15388,

  19. [27]

    Autoregressive model beats diffusion: Llama for scalable image generation

    Sun, P., Jiang, Y ., Chen, S., Zhang, S., Peng, B., Luo, P., and Yuan, Z. Autoregressive model beats diffusion: Llama for scalable image generation. arXiv preprint arXiv:2406.06525, 2024. 1

    2024 arXiv

  20. [28]

    Hart: Ef- ficient visual generation with hybrid autoregressive transformer

    Tang, H., Wu, Y ., Yang, S., Xie, E., Chen, J., Chen, J., Zhang, Z., Cai, H., Lu, Y ., and Han, S. Hart: Ef- ficient visual generation with hybrid autoregressive transformer. arXiv preprint arXiv:2410.10812, 2024. 1, 2, 3, 5

    2024 arXiv

  21. [29]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction

    Tian, K., Jiang, Y ., Yuan, Z., Peng, B., and Wang, L. Visual autoregressive modeling: Scalable image generation via next-scale prediction. 2024. 1, 2, 3, 11

    2024

  22. [30]

    Neural discrete representation learning

    Van Den Oord, A., Vinyals, O., et al. Neural discrete representation learning. NeurIPS, 30, 2017. 3

    2017

  23. [31]

    Tag dis- entangled generative adversarial networks for object image re-rendering

    Wang, C., Wang, C., Xu, C., and Tao, D. Tag dis- entangled generative adversarial networks for object image re-rendering. In International joint conference on artificial intelligence (IJCAI), 2017. 1

    2017

  24. [32]

    Emu3: Next-token prediction is all you need

    Wang, X., Zhang, X., Luo, Z., Sun, Q., Cui, Y ., Wang, J., Zhang, F., Wang, Y ., Li, Z., Yu, Q., et al. Emu3: Next-token prediction is all you need. arXiv preprint arXiv:2409.18869, 2024. 1

    2024 arXiv

  25. [33]

    Human preference score v2: A solid benchmark for evaluating human preferences of text- to-image synthesis

    Wu, X., Hao, Y ., Sun, K., Chen, Y ., Zhu, F., Zhao, R., and Li, H. Human preference score v2: A solid benchmark for evaluating human preferences of text- to-image synthesis. arXiv preprint arXiv:2306.09341,

    arXiv

  26. [34]

    Imagereward: Learning and evalu- ating human preferences for text-to-image generation

    Xu, J., Liu, X., Wu, Y ., Tong, Y ., Li, Q., Ding, M., Tang, J., and Dong, Y . Imagereward: Learning and evalu- ating human preferences for text-to-image generation. NeurIPS, 36, 2024. 5

    2024

  27. [35]

    Attngan: Fine-grained text to image generation with attentional generative adversar- ial networks

    Xu, T., Zhang, P., Huang, Q., Zhang, H., Gan, Z., Huang, X., and He, X. Attngan: Fine-grained text to image generation with attentional generative adversar- ial networks. In CVPR, pp. 1316–1324, 2018. 1

    2018

  28. [36]

    Y ., Zhang, H., Pang, R., Qin, J., Ku, A., Xu, Y ., Baldridge, J., and Wu, Y

    Yu, J., Li, X., Koh, J. Y ., Zhang, H., Pang, R., Qin, J., Ku, A., Xu, Y ., Baldridge, J., and Wu, Y . Vector- quantized image modeling with improved vqgan.arXiv preprint arXiv:2110.04627, 2021. 3

    2021 arXiv

  29. [37]

    B., Versari, L., Sohn, K., Minnen, D., Cheng, Y ., Gupta, A., Gu, X., Hauptmann, A

    Yu, L., Lezama, J., Gundavarapu, N. B., Versari, L., Sohn, K., Minnen, D., Cheng, Y ., Gupta, A., Gu, X., Hauptmann, A. G., et al. Language model beats diffusion-tokenizer is key to visual generation. InICLR. 1

  30. [38]

    B., Versari, L., Sohn, K., Minnen, D., Cheng, Y ., Gupta, A., Gu, X., Hauptmann, A

    Yu, L., Lezama, J., Gundavarapu, N. B., Versari, L., Sohn, K., Minnen, D., Cheng, Y ., Gupta, A., Gu, X., Hauptmann, A. G., Gong, B., Yang, M., Essa, I., Ross, D. A., and Jiang, L. Language model beats diffusion - tokenizer is key to visual generation. In ICLR, 2024. 3 Appendi...

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.