Pith. sign in

REVIEW 4 major objections 6 minor 3 cited by

Native-Resolution Image Synthesis

T0 review · 4 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper argues that training a diffusion transformer directly on images at their native resolutions and aspect ratios yields a single model that is state-of-the-art at both standard square benchmarks and capable of zero-shot generation…

desk verdict A real single-model resolution-generalization result, but the paper's explanation of why it works is ahead of its ablations. read the letter →

arxiv 2506.03131 v1 pith:NF4Z7NYG submitted 2025-06-03 cs.CV cs.LG

Zidong Wang , Lei Bai , Xiangyu Yue , Wanli Ouyang , Yiyuan Zhang This is my paper
classification cs.CVcs.LG
keywords native-resolutionimagesynthesisdiffusiontransformervariable-lengthsequencemodelinggenerationresolutiongeneralizationaspectratio2Drotarypositionembeddingzero-shot
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper tries to establish that diffusion models for images can be released from the fixed square training sizes that dominate the field, and that training directly on images' native resolutions and aspect ratios, the way large language models train on variable-length text, gives them the same kind of zero-shot flexibility. For this it introduces the Native-resolution diffusion Transformer (NiT), a DiT-style model that tokenizes each image at its own dimensions, packs the resulting variable-length token sequences without padding, and uses axial 2D rotary position embeddings to keep 2D structure. A single NiT-XL model is reported to reach state-of-the-art Fréchet inception distance (FID) on both ImageNet 256x256 (2.03) and 512x512 (1.45), and to generate high-fidelity images at unseen 1024x1024 (FID 4.52) and 1536x1536 (FID 6.51) and at unseen aspect ratios. If these results hold, resizing and center-cropping are not neutral preprocessing but a cause of poor generalization, and the LLM recipe of native-format training transfers directly to generative vision.

What carries the argument

The load-bearing mechanism is variable-length packed sequence processing inside a diffusion transformer. Each native-resolution image contributes a number of visual tokens equal to its latent height times width, and images are concatenated into one sequence with cumulative sequence lengths that tell a variable-length attention kernel to attend only within each image, eliminating padding and masking. Axial 2D rotary position embeddings factorize each token's position into separate height and width rotation frequencies, so relative 2D position is preserved across resolution changes, while Packed Adaptive Layer Normalization broadcasts per-image class or text conditioning to the right tokens. Together these make the model's effective computation independent of a fixed grid, which is what the paper claims lets it generate at unseen resolutions and aspect ratios.

What would settle it

Train the same NiT architecture on a fixed mixture of only 256x256 and 512x512 images for enough steps to match the native-resolution model's training budget; if that model also reaches FID near 4 on 1024x1024 or near 4 on 9:16, the resolution-generalization result is not caused by native-resolution data. Conversely, train a fixed-resolution architecture such as DiT with absolute positional embeddings on the native-resolution mixture; if it too generalizes, the effect is data-driven rather than architectural.

Watch

Extended reading notes

Core claim

On its own terms, the discovery is that a diffusion transformer trained exclusively on images at their native resolution learns a scale- and aspect-ratio-invariant visual distribution, so one model covers many output formats. NiT encodes each image with a 32x downsampling autoencoder, patches the latents into a token count set by the image's own height and width, packs all tokens in a batch up to a fixed token budget, and runs full attention inside each image through cumulative sequence lengths with a variable-length attention kernel. The 2D axial rotary position embedding gives tokens relative positional information along height and width separately, which is what lets the network keep spatial structure when the grid changes. The result is a single model with FID 2.03 at 256x256 and 1.45 at 512x512, and zero-shot FID 4.52 at 1024x1024 and 6.51 at 1536x1536, alongside strong scores on unseen aspect ratios; the authors read this as evidence that the model has internalized the intrinsic visual distribution rather than a fixed format.

Load-bearing premise

The claim rests on the assumption that native-resolution training data, rather than the new positional encoding and packing scheme, is what produces the model's ability to generalize to unseen resolutions.

Editorial extensions

If this is right

  • A single NiT model can replace two resolution-specific models on the standard ImageNet benchmarks, cutting cumulative training compute: the 131B-token run reports an mFID of 1.86 across 256x256 and 512x512 while using about 9.17% of the token budget of the DiT-XL/SiT-XL pair.
  • A model trained only on ImageNet's native resolutions generalizes to unseen 1024x1024 and 1536x1536 with FID 4.52 and 6.51, whereas resolution-expert baselines collapse (FID 40.74 for EDM2-L at 1024, and FiTv2 and SiT-REPA fail to produce valid 1024x1024 images).
  • Unseen aspect ratios from 1:3 to 3:1 generate with FID between 3.41 and 16.85, with the best scores near square ratios, indicating the model does not inherit the truncation bias that cropping to square training images induces.
  • The same architecture carries to text-to-image: NiT-T2I reports FID 9.18 and a CLIP score of 0.345 on zero-shot COCO, surpassing SD-v1.5 and SDXL-Turbo at a fraction of the model size.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial extension: the ablation in Table 4 changes the data mixture but keeps the NiT architecture fixed, so the paper does not isolate how much of the generalization comes from native-resolution data versus from 2D RoPE and packing; a factorial study with a fixed-resolution architecture on native data would separate the two.
  • Editorial extension: because the model packs to a token budget rather than a batch of fixed images, the natural scaling currency for this paradigm is total tokens; this suggests resolution diversity in the training set, not just dataset size, may be the resource that buys zero-shot generalization.
  • Editorial extension: the 32x downsampling autoencoder has a fixed receptive field per latent token, so the sharp FID degradation at 2048x2048 (24.76) may be an encoder or context limit rather than a fundamental limit of native-resolution training; testing with a higher-capacity encoder or sliding-window decoding would locate the bottleneck.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces native-resolution image synthesis, a paradigm for diffusion transformers that trains directly on images at their original resolutions and aspect ratios without resizing, cropping, or padding. The proposed architecture, NiT, tokenizes variable-size latents into packed sequences, uses FlashAttention with cumulative sequence lengths for per-instance attention, applies an axial 2D Rotary Position Embedding, and employs a packed adaptive layer normalization. The authors report that a single NiT-XL model achieves ImageNet FID 2.03 at 256x256 and 1.45 at 512x512, outperforming several resolution-expert baselines while using less total training compute, and that it generalizes zero-shot to unseen resolutions up to 1536x1536 and to non-square aspect ratios, with FID 4.52 at 1024x1024 and 4.11 at 432x768. Ablation studies compare different data mixtures and attribute the generalization to native-resolution training. An appendix additionally reports text-to-image experiments on COCO.

Significance. If the empirical claims hold, this is a practically valuable contribution: a single diffusion-transformer model that is competitive or state-of-the-art on two standard ImageNet benchmarks, with credible zero-shot extrapolation to higher resolutions and varied aspect ratios, would address a real limitation of fixed-resolution generative models. The paper's strengths include a transparent token-budget accounting, comparisons against several modern baselines, explicit descriptions of the packing and attention mechanisms, and an honest discussion of limitations. However, the central mechanistic conclusion, that training on native resolutions is what enables the generalization, rests on an ablation that does not isolate the contribution of native-resolution data from other variables. In addition, several evaluation details that are load-bearing for the quantitative claims, such as how FID reference statistics are constructed at novel resolutions and how classifier-free-guidance hyperparameters were chosen, are not specified. These gaps prevent the paper, in its current form, from fully establishing the advertised mechanism, though the empirical system itself appears promising.

major comments (4)
  1. [Section 4.3, Tables 4 and 5] The ablation study varies the data mixture while keeping the NiT architecture fixed, but the comparison between arm (b), 'Native Resolution + 256 + 512', and arm (c), '256 + 512', changes two things at once: the presence of native-resolution images and the number and diversity of resolutions seen during training. Moreover, no ablation removes or replaces the two architectural innovations, 2D RoPE and packed attention without padding, which are known to be favorable for positional extrapolation and for avoiding padding artifacts. Consequently, the large improvement of (b) over (c) at 768x768 (FID 12.42 vs. 33.50) and the gains in aspect-ratio generalization cannot be attributed specifically to 'learning intrinsic visual distributions from native resolutions.' The conclusion in the 'Insights' paragraph overstates what the experiments establish. Please add ablations that vary the data mixture while holding the architecture fixed and that also vary the architecture (e.g., with and without 2D RoPE, with and without packing) to disentangle these factors.
  2. [Section 4.1 and Tables 2-3] The paper reports FID values at resolutions and aspect ratios that do not exist natively in ImageNet (e.g., 1024x1024, 1536x1536, 320x960), but it never describes how the reference feature statistics are constructed for these evaluation settings. If the reference set is created by resizing or cropping ImageNet images to each target resolution, the resulting FID values depend on that preprocessing choice and are not directly comparable across resolutions or to the baseline numbers, which may have been computed on different reference sets. This is load-bearing for the zero-shot FID claims. Please describe the reference set construction explicitly, state the number of samples, and clarify whether the same reference set is used for all methods at each resolution and aspect ratio.
  3. [Table 7 and Section 4.2] Table 7 reports that the CFG scale and CFG interval are tuned separately for every evaluation setting: for example, 256x256 uses scale 2.25 and interval [0.0, 0.7], 1536x1536 uses scale 1.5 and interval [0.0, 1.0], and 960x320 uses scale 4.5 and interval [0.0, 0.9]. The headline comparisons in Tables 1-3 therefore mix model capability with per-setting guidance optimization. It is not stated whether the baseline models were tuned to the same degree for each resolution and aspect ratio. Please either report results with a fixed CFG policy across all settings or provide an explicit upper envelope over a grid of CFG values, and state the baseline tuning procedure.
  4. [Section 4.1 and Table 1] No confidence intervals, multiple evaluation runs, or different random seeds are reported for any FID value, and the code and checkpoints are not released. Given that some of the headline improvements are numerically small (e.g., 256x256 FID 2.03 vs. 2.00 for FlowDCN, and 512x512 FID 1.45 vs. 1.88 for EDM2-L in the same parameter class), the reader cannot assess whether the reported ordering is stable. Please report variance over at least a few evaluation runs or seeds and, if possible, release code and trained checkpoints to make the empirical claims reproducible.
minor comments (6)
  1. [Table 1] The column header line 'mFID↓FID↓sFID↓IS↑Prec.↑Rec.↑' is malformed, with the metric names and arrows run together; this should be split into separate columns for clarity.
  2. [Table 2] There are numerical formatting inconsistencies, such as '9.817' and '41.170' for FlowDCN, which contain an extra digit compared with the other entries; please standardize the number of decimal places.
  3. [Section 4.2] The text states that NiT-XL achieves an FID of 4.07 on 768x768 resolution, but Table 2 reports 4.05 for this setting; please correct the text to match the table.
  4. [Table 3] The footnote explains that SiT-REPA was evaluated at different resolutions (160x480, 216x384, etc.) because only the 256-resolution model is open-sourced, while all other methods are evaluated at 320x960, 432x768, etc.; this makes the comparison at those aspect ratios unmatched. Please state explicitly whether this difference is expected to bias the comparison and, if possible, obtain matched-resolution numbers for SiT-REPA.
  5. [Abstract and Section 1] The term 'zero-shot' is used for resolutions and aspect ratios that may partially overlap with the native-resolution training distribution of ImageNet; please define precisely what counts as an unseen resolution or aspect ratio relative to the training data and clarify the status of settings such as 768x768 and 4:3.
  6. [Table 5] Table 5 compares only arms (a) and (b), but the table does not include a row for arm (c) and the caption does not explain the omission; in the text, the comparison is described in Section 4.3, yet the table header 'Data' is less informative than the full data-mixture labels used in Table 4. Please align the notation and caption with the body text.

Circularity Check

0 steps flagged · score 1.0 of 10

No significant circularity: NiT's benchmark results are externally measured, and the paper's self-citations appear only as baselines or routine engineering precedents, not as premises that force the conclusions.

full rationale

NiT is an empirical system paper; there is no formal derivation whose inputs are equivalent to its outputs. The headline SOTA claims are evaluated against fixed external benchmarks (FID/sFID/IS on ImageNet 256x256 and 512x512), so they cannot reduce by construction to the method's own design choices. The ablation in Section 4.3 compares data mixtures (native-only, native+256+512, 256+512) and concludes that 'training in native-resolution to learn a resolution- and aspect-ratio invariant visual distribution' is the enabler of generalization. This experiment is confounded: the architecture, including 2D RoPE and packed attention, is held fixed while data diversity changes, so the mechanistic attribution is under-supported. That is, however, an experimental-design or correctness concern, not circularity, because the inputs (data mixture) and the measured outputs (FID at 768x768 and at varied aspect ratios) are distinct quantities. Self-citations to the authors' prior FiT and FiTv2 work appear only as baselines in Tables 1-3 and as background on padding artifacts; neither is invoked as an unverified premise that forces NiT's design or outcomes. The core architectural ingredients are grounded in external, non-overlapping prior work (FlashAttention-2 [15], histogram packing [39], RoPE [68], and native-resolution VLM processing [4, 70, 72, 75]). The paper's own limitation statement, 'its generalization ability on extremely high-resolution and aspect ratios is still not satisfactory,' further indicates that the generalization claim is presented as a bounded empirical result rather than as a definitional consequence. Score 1 reflects only the presence of minor, non-load-bearing self-citations.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

No new physical entities are postulated; the contribution is architectural and empirical. The free-parameter burden is moderate: per-evaluation CFG tuning and hand-chosen hyperparameters are inputs to all reported numbers, but they are not fitted constants of a theory. The axioms are standard background plus domain assumptions about evaluation validity and autoencoder reliability.

free parameters (3)
  • CFG scale and interval per evaluation setting = Per-setting values in Table 7 (e.g., 256x256: 2.25; 2048x2048: 1.5; 960x320: 4.5)
    Each resolution and aspect ratio uses its own tuned guidance schedule, and reported FID numbers depend on this per-setting selection.
  • Logit-Normal time distribution parameters (P_mean, P_std) = Not specified numerically; described as manually selected coefficients in Section 3.1
    These shape the noise schedule and affect training dynamics; they are inputs, not derived from first principles.
  • 2D RoPE base theta = 10000
    Section 3.2 follows common practice; this controls the frequency of positional encoding and influences extrapolation to unseen token counts.
assumptions (4)
  • standard math Flow matching with linear path and EDM logit-normal time distribution is a valid training objective for variable-resolution data.
    Section 3.1 adopts standard flow matching; this is established background, not derived in the paper.
  • domain assumption FID computed at each evaluated resolution and aspect ratio is a valid and comparable measure of generation quality.
    Section 4.1 and Tables 2 and 3 do not specify how reference statistics were constructed, which images, resizing method, or crop protocol; comparisons across resolutions may not be apples-to-apples.
  • domain assumption ImageNet native-resolution distribution is diverse enough to support extrapolation to unseen resolutions and aspect ratios.
    Figure 2 shows native resolutions concentrated between 200 and 600 pixels with sparse data beyond 800 pixels; the claim that the model learns a resolution-invariant distribution depends on sufficient diversity in the training set.
  • domain assumption The DC-AE autoencoder remains accurate at 32x downsampling for resolutions far outside its typical training range.
    Section 4.1 uses DC-AE with 32x downsampling; any decoder failure at 1536 or 2048 would cap achievable image quality independent of the diffusion transformer.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Native-Resolution Image Synthesis." pith.science (2026). https://pith.science/paper/NF4Z7NYG

@misc{pith2026250603131,
  author       = {Pith},
  title        = {Pith review of: Native-Resolution Image Synthesis},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/NF4Z7NYG}},
  note         = {Machine review of arXiv:2506.03131}
}
read the original abstract

We introduce native-resolution image synthesis, a novel generative modeling paradigm that enables the synthesis of images at arbitrary resolutions and aspect ratios. This approach overcomes the limitations of conventional fixed-resolution, square-image methods by natively handling variable-length visual tokens, a core challenge for traditional techniques. To this end, we introduce the Native-resolution diffusion Transformer (NiT), an architecture designed to explicitly model varying resolutions and aspect ratios within its denoising process. Free from the constraints of fixed formats, NiT learns intrinsic visual distributions from images spanning a broad range of resolutions and aspect ratios. Notably, a single NiT model simultaneously achieves the state-of-the-art performance on both ImageNet-256x256 and 512x512 benchmarks. Surprisingly, akin to the robust zero-shot capabilities seen in advanced large language models, NiT, trained solely on ImageNet, demonstrates excellent zero-shot generalization performance. It successfully generates high-fidelity images at previously unseen high resolutions (e.g., 1536 x 1536) and diverse aspect ratios (e.g., 16:9, 3:1, 4:3), as shown in Figure 1. These findings indicate the significant potential of native-resolution modeling as a bridge between visual generative modeling and advanced LLM methodologies.

Figures

Figures reproduced from arXiv: 2506.03131 by the authors.

Figure 1
Figure 1. Native-resolution image synthesis on ImageNet. A single Native-resolution diffusion Transformer (NiT) model, trained on ImageNet, generates images across diverse, arbitrary resolutions and aspect ratios (examples shown from 256 × 256 to 2048 × 2048, and aspect ratios from 1 : 5 to 3 : 1). This capability extends far beyond conventional fixed-resolution, square-image generation (e.g., 256 × 256), demonstrating strong… view at source ↗
Figure 2
Figure 2. NiT’s Superior Generalization Beyond ImageNet’s Typical Resolution Distribution. (a) ImageNet resolutions are mainly concentrated between 200 to 600 pixels (width/height), with sparse data beyond 800 pixels. Despite this, (b) shows our NiT model’s superior generalization to unseen high resolutions (e.g., 1024, 1536), evidenced by significantly lower FID scores. (c) further confirms NiT also exhibits the strongest ge… view at source ↗
Figure 3
Figure 3. Architecture Design of Native Resolution Diffusion Transformer (NiT). NiT takes noisy latent representations, tokenizes them into variable-length sequences based on the original image resolution. Each NiT block utilizes Packed Multi-Head-Self-Attention (MHSA) with 2D RoPE and incorporates timestep and class conditioning via adaptive layer normalization. Htarget × Wtarget discards this inherent scale diversity [57]. … view at source ↗
Figures from the paper (22 more)
Figure 4
Figure 4. Figure 4: Qualitative Comparison of Resolution and Aspect Ratio Generalization.We provide the visualization of NiT, EDM2 and FlowDCN, because FiTv2 and SiT-REPA demonstrate inferior generalization capability revealed by quantitative results. 8 [PITH_FULL_IMAGE:figures/full_fig_…
Figure 5
Figure 5. Figure 5: Illustration of NiT blocks used for text-to-image generation. [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison of resolution generalization on [PITH_FULL_IMAGE:figures/full_fig_p016_6.png]
Figure 7
Figure 7. Figure 7: Qualitative comparison of resolution generalization on [PITH_FULL_IMAGE:figures/full_fig_p016_7.png]
Figure 8
Figure 8. Figure 8: Qualitative comparison of resolution generalization on [PITH_FULL_IMAGE:figures/full_fig_p017_8.png]
Figure 9
Figure 9. Figure 9: Qualitative comparison of resolution generalization on [PITH_FULL_IMAGE:figures/full_fig_p017_9.png]
Figure 10
Figure 10. Figure 10: Qualitative comparison of aspect ratio generalization on [PITH_FULL_IMAGE:figures/full_fig_p018_10.png]
Figure 11
Figure 11. Figure 11: Qualitative comparison of aspect ratio generalization on [PITH_FULL_IMAGE:figures/full_fig_p018_11.png]
Figure 12
Figure 12. Figure 12: Qualitative comparison of aspect ratio generalization on [PITH_FULL_IMAGE:figures/full_fig_p019_12.png]
Figure 13
Figure 13. Figure 13: Qualitative comparison of aspect ratio generalization on [PITH_FULL_IMAGE:figures/full_fig_p019_13.png]
Figure 14
Figure 14. Figure 14: Qualitative comparison of aspect ratio generalization on [PITH_FULL_IMAGE:figures/full_fig_p020_14.png]
Figure 15
Figure 15. Figure 15: Qualitative comparison of aspect ratio generalization on [PITH_FULL_IMAGE:figures/full_fig_p020_15.png]
Figure 16
Figure 16. Figure 16: Uncurated generation results of NiT-XL. We use the class label as 33. [PITH_FULL_IMAGE:figures/full_fig_p021_16.png]
Figure 17
Figure 17. Figure 17: Uncurated generation results of NiT-XL. We use the class label as 88. [PITH_FULL_IMAGE:figures/full_fig_p021_17.png]
Figure 18
Figure 18. Figure 18: Uncurated generation results of NiT-XL. We use the class label as 250. [PITH_FULL_IMAGE:figures/full_fig_p022_18.png]
Figure 19
Figure 19. Figure 19: Uncurated generation results of NiT-XL. We use the class label as 279. [PITH_FULL_IMAGE:figures/full_fig_p022_19.png]
Figure 20
Figure 20. Figure 20: Uncurated generation results of NiT-XL. We use the class label as 388. [PITH_FULL_IMAGE:figures/full_fig_p023_20.png]
Figure 21
Figure 21. Figure 21: Uncurated generation results of NiT-XL. We use the class label as 417. [PITH_FULL_IMAGE:figures/full_fig_p023_21.png]
Figure 22
Figure 22. Figure 22: Uncurated generation results of NiT-XL. We use the class label as 437. [PITH_FULL_IMAGE:figures/full_fig_p024_22.png]
Figure 23
Figure 23. Figure 23: Uncurated generation results of NiT-XL. We use the class label as 812. [PITH_FULL_IMAGE:figures/full_fig_p024_23.png]
Figure 24
Figure 24. Figure 24: Uncurated generation results of NiT-XL. We use the class label as 980. [PITH_FULL_IMAGE:figures/full_fig_p025_24.png]
Figure 25
Figure 25. Figure 25: Uncurated generation results of NiT-XL. We use the class label as 985. [PITH_FULL_IMAGE:figures/full_fig_p025_25.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 3 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Transition Models: Rethinking the Generative Learning Objective

    cs.LG 2025-09 conditional novelty 6.0 of 10

    TiM trains a single diffusion-type model on arbitrary time-interval transitions, achieving strong one-step and multi-step text-to-image generation with 865M parameters.

  2. PixNerd: Pixel Neural Field Diffusion

    cs.CV 2025-07 conditional novelty 6.0 of 10

    PixNerd is a single-stage pixel-space diffusion transformer that uses predicted neural field weights to decode large patches, reaching 2.15 FID on ImageNet 256 without a VAE.

  3. Mage-Flow: An Efficient Native-Resolution Foundation Model for Image Generation and Editing

    cs.CV 2026-07 conditional novelty 5.0 of 10

    A compact 4B image generation/editing system with a fast one-step VAE, native-resolution packing, RL alignment, and 4-step distillation reports competitive benchmarks against 6B–80B open models.

Reference graph

Works this paper leans on

87 extracted references · 16 canonical work pages · cited by 3 Pith papers

  1. [1]

    Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXiv preprint arXiv:2303.08774, 2023

    arXiv 2023

  2. [2]

    Stochastic interpolants: A unifying framework for flows and diffusions.arXiv preprint arXiv:2303.08797, 2023

    Michael S Albergo, Nicholas M Boffi, and Eric Vanden-Eijnden. Stochastic interpolants: A unifying framework for flows and diffusions.arXiv preprint arXiv:2303.08797, 2023

    arXiv 2023

  3. [3]

    Building normalizing flows with stochastic interpolants

    Michael S Albergo and Eric Vanden-Eijnden. Building normalizing flows with stochastic interpolants. arXiv preprint arXiv:2209.15571, 2022

    arXiv 2022

  4. [4]

    Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, et al. Qwen2. 5-vl technical report.arXiv preprint arXiv:2502.13923, 2025

    arXiv 2025

  5. [5]

    ediff-i: Text-to-image diffusion models with an ensemble of expert denoisers.arXiv preprint arXiv:2211.01324, 2022

    Yogesh Balaji, Seungjun Nah, Xun Huang, Arash Vahdat, Jiaming Song, Qinsheng Zhang, Karsten Kreis, Miika Aittala, Timo Aila, Samuli Laine, et al. ediff-i: Text-to-image diffusion models with an ensemble of expert denoisers.arXiv preprint arXiv:2211.01324, 2022

    arXiv 2022

  6. [6]

    Language models are few-shot learners

    Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. NeurIPS, 2020

    2020

  7. [7]

    Muse: Text-to-image generation via masked generative transformers.arXiv preprint arXiv:2301.00704, 2023

    Huiwen Chang, Han Zhang, Jarred Barber, AJ Maschinot, Jose Lezama, Lu Jiang, Ming-Hsuan Yang, Kevin Murphy, William T Freeman, Michael Rubinstein, et al. Muse: Text-to-image generation via masked generative transformers.arXiv preprint arXiv:2301.00704, 2023

    arXiv 2023

  8. [8]

    Maskgit: Masked generative image transformer

    Huiwen Chang, Han Zhang, Lu Jiang, Ce Liu, and William T Freeman. Maskgit: Masked generative image transformer. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022

    2022

Show all 87 references
  1. [9]

    Clex: Continuous length extrapolation for large language models, 2024.URL https://arxiv

    Guanzheng Chen, Xin Li, Zaiqiao Meng, Shangsong Liang, and Lidong Bing. Clex: Continuous length extrapolation for large language models, 2024.URL https://arxiv. org/abs/2310.16450

    2024 arXiv

  2. [10]

    Pixart-sigma: Weak-to-strong training of diffusion transformer for 4k text-to-image generation

    Junsong Chen, Chongjian Ge, Enze Xie, Yue Wu, Lewei Yao, Xiaozhe Ren, Zhongdao Wang, Ping Luo, Huchuan Lu, and Zhenguo Li. Pixart-sigma: Weak-to-strong training of diffusion transformer for 4k text-to-image generation. InEuropean Conference on Computer Vision, pages 74–91. Spr...

    2024

  3. [11]

    Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis.arXiv preprint arXiv:2310.00426, 2023

    Junsong Chen, Jincheng Yu, Chongjian Ge, Lewei Yao, Enze Xie, Yue Wu, Zhongdao Wang, James Kwok, Ping Luo, Huchuan Lu, et al. Pixart-alpha: Fast training of diffusion transformer for photorealistic text-to-image synthesis.arXiv preprint arXiv:2310.00426, 2023

    arXiv

  4. [12]

    Deep compression autoencoder for efficient high-resolution diffusion models.arXiv preprint arXiv:2410.10733, 2024

    Junyu Chen, Han Cai, Junsong Chen, Enze Xie, Shang Yang, Haotian Tang, Muyang Li, Yao Lu, and Song Han. Deep compression autoencoder for efficient high-resolution diffusion models.arXiv preprint arXiv:2410.10733, 2024

    2024 arXiv

  5. [13]

    Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024

    Zhe Chen, Weiyun Wang, Yue Cao, Yangzhou Liu, Zhangwei Gao, Erfei Cui, Jinguo Zhu, Shenglong Ye, Hao Tian, Zhaoyang Liu, et al. Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling.arXiv preprint arXiv:2412.05271, 2024

    2024 arXiv

  6. [14]

    Ramadge, and Alexander Rudnicky

    Ta-Chung Chi, Ting-Han Fan, Peter J. Ramadge, and Alexander Rudnicky. Kerple: Kernelized relative positional embedding for length extrapolation. InNeurIPS, 2022

    2022

  7. [15]

    Flashattention-2: Faster attention with better parallelism and work partitioning.arXiv preprint arXiv:2307.08691, 2023

    Tri Dao. Flashattention-2: Faster attention with better parallelism and work partitioning.arXiv preprint arXiv:2307.08691, 2023

    2023 arXiv

  8. [16]

    Patch n’pack: Navit, a vision transformer for any aspect ratio and resolution.arXiv preprint arXiv:2307.06304, 2023

    Mostafa Dehghani, Basil Mustafa, Josip Djolonga, Jonathan Heek, Matthias Minderer, Mathilde Caron, Andreas Steiner, Joan Puigcerver, Robert Geirhos, Ibrahim Alabdulmohsin, et al. Patch n’pack: Navit, a vision transformer for any aspect ratio and resolution.arXiv preprint arXiv...

    2023 arXiv

  9. [17]

    Imagenet: A large-scale hierarchical image database

    Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. InCVPR, 2009

    2009

  10. [18]

    Diffusion models beat gans on image synthesis.NeurIPS, 2021

    Prafulla Dhariwal and Alexander Nichol. Diffusion models beat gans on image synthesis.NeurIPS, 2021

    2021

  11. [19]

    Cogview2: Faster and better text-to-image generation via hierarchical transformers.Advances in Neural Information Processing Systems, 35:16890–16902, 2022

    Ming Ding, Wendi Zheng, Wenyi Hong, and Jie Tang. Cogview2: Faster and better text-to-image generation via hierarchical transformers.Advances in Neural Information Processing Systems, 35:16890–16902, 2022. 26

    2022

  12. [20]

    Longrope: Extending llm context window beyond 2 million tokens.arXiv preprint arXiv:2402.13753, 2024

    Yiran Ding, Li Lyna Zhang, Chengruidong Zhang, Yuanyuan Xu, Ning Shang, Jiahang Xu, Fan Yang, and Mao Yang. Longrope: Extending llm context window beyond 2 million tokens.arXiv preprint arXiv:2402.13753, 2024

    2024 arXiv

  13. [21]

    An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:...

    2010 arXiv

  14. [22]

    The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models.arXiv preprint arXiv:2407.21783, 2024

    2024 arXiv

  15. [23]

    Scaling rectified flow transformers for high-resolution image synthesis

    Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas Müller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, et al. Scaling rectified flow transformers for high-resolution image synthesis. 2024

    2024

  16. [24]

    Make-a-scene: Scene-based text-to-image generation with human priors

    Oran Gafni, Adam Polyak, Oron Ashual, Shelly Sheynin, Devi Parikh, and Yaniv Taigman. Make-a-scene: Scene-based text-to-image generation with human priors. InEuropean Conference on Computer Vision, pages 89–106. Springer, 2022

    2022

  17. [25]

    Masked diffusion transformer is a strong image synthesizer.arXiv preprint arXiv:2303.14389, 2023

    Shanghua Gao, Pan Zhou, Ming-Ming Cheng, and Shuicheng Yan. Masked diffusion transformer is a strong image synthesizer.arXiv preprint arXiv:2303.14389, 2023

    2023 arXiv

  18. [26]

    Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprint arXiv:2501.12948, 2025

    2025 arXiv

  19. [27]

    Lm-infinite: Zero-shot extreme length generalization for large language models.arXiv preprint arXiv:2308.16137, 2023

    Chi Han, Qifan Wang, Hao Peng, Wenhan Xiong, Yu Chen, Heng Ji, and Sinong Wang. Lm-infinite: Zero-shot extreme length generalization for large language models.arXiv preprint arXiv:2308.16137, 2023

    2023 arXiv

  20. [28]

    Greg Heinrich, Mike Ranzinger, Yin Hongxu, Yao Lu, Jan Kautz, Andrew Tao, Bryan Catanzaro, and Pavlo Molchanov. Radiov2. 5: Improved baselines for agglomerative vision foundation models. InProc. CVPR, volume 2, page 6, 2025

    2025

  21. [29]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium.NeurIPS, 2017

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium.NeurIPS, 2017

    2017

  22. [30]

    Denoising diffusion probabilistic models.NeurIPS, 2020

    Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.NeurIPS, 2020

    2020

  23. [31]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. InICLR, 2022

    2022

  24. [32]

    Estimation of non-normalized statistical models by score matching

    Aapo Hyvärinen and Peter Dayan. Estimation of non-normalized statistical models by score matching. 2005

    2005

  25. [33]

    Springer Science & Business Media, 2005

    Bernd Jähne.Digital image processing. Springer Science & Business Media, 2005

    2005

  26. [34]

    Scaling laws for neural language models.arXiv preprint arXiv:2001.08361, 2020

    Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. Scaling laws for neural language models.arXiv preprint arXiv:2001.08361, 2020

    2001 arXiv

  27. [35]

    Elucidating the design space of diffusion-based generative models.NeurIPS, 2022

    Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models.NeurIPS, 2022

    2022

  28. [36]

    Analyzing and improving the training dynamics of diffusion models

    Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24174–24184, 2024

    2024

  29. [37]

    The impact of positional encoding on length generalization in transformers.Advances in Neural Information Processing Systems, 36:24892–24928, 2023

    Amirhossein Kazemnejad, Inkit Padhi, Karthikeyan Natesan Ramamurthy, Payel Das, and Siva Reddy. The impact of positional encoding on length generalization in transformers.Advances in Neural Information Processing Systems, 36:24892–24928, 2023

    2023

  30. [38]

    Segment anything

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. Segment anything. InProceedings of the IEEE/CVF international conference on computer vision, pages 4015–4026, 2023

    2023

  31. [39]

    Efficient sequence packing without cross-contamination: Accelerating large language models without impacting performance.arXiv preprint arXiv:2107.02027, 2021

    Mario Michael Krell, Matej Kosec, Sergio P Perez, and Andrew Fitzgibbon. Efficient sequence packing without cross-contamination: Accelerating large language models without impacting performance.arXiv preprint arXiv:2107.02027, 2021. 27

    2021 arXiv

  32. [40]

    Kynkäänniemi, T

    T. Kynkäänniemi, T. Karras, S. Laine, and T Lehtinen, J.and Aila. Improved precision and recall metric for assessing generative models.NeurIPS, 2019

    2019

  33. [41]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. InECCV, 2014

    2014

  34. [42]

    Flow matching for generative modeling.arXiv preprint arXiv:2210.02747, 2022

    Yaron Lipman, Ricky TQ Chen, Heli Ben-Hamu, Maximilian Nickel, and Matt Le. Flow matching for generative modeling.arXiv preprint arXiv:2210.02747, 2022

    2022 arXiv

  35. [43]

    Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report.arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  36. [44]

    Flow straight and fast: Learning to generate and transfer data with rectified flow

    Xingchao Liu, Chengyue Gong, and qiang liu. Flow straight and fast: Learning to generate and transfer data with rectified flow. InICLR, 2023

    2023

  37. [45]

    Ntk-aware scaled rope allows llama models to have extended (8k+) context size with- out any fine-tuning and minimal perplexity degradation

    LocalLLaMA. Ntk-aware scaled rope allows llama models to have extended (8k+) context size with- out any fine-tuning and minimal perplexity degradation. https://www.reddit.com/r/LocalLLaMA/ comments/14lz7j5/ntkaware_scaled_rope_allows_llama_models_to_have/. Accessed: 2024- 2-1

    2024

  38. [46]

    Fit: Flexible vision transformer for diffusion model

    Zeyu Lu, ZiDong Wang, Di Huang, Chengyue Wu, Xihui Liu, Wanli Ouyang, and LEI BAI. Fit: Flexible vision transformer for diffusion model. 2024

    2024

  39. [47]

    Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers.arXiv preprint arXiv:2401.08740, 2024

    Nanye Ma, Mark Goldstein, Michael S Albergo, Nicholas M Boffi, Eric Vanden-Eijnden, and Saining Xie. Sit: Exploring flow and diffusion-based generative models with scalable interpolant transformers.arXiv preprint arXiv:2401.08740, 2024

    2024 arXiv

  40. [48]

    Sdedit: Guided image synthesis and editing with stochastic differential equations.arXiv preprint arXiv:2108.01073, 2021

    Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon. Sdedit: Guided image synthesis and editing with stochastic differential equations.arXiv preprint arXiv:2108.01073, 2021

    2021 arXiv

  41. [49]

    The llama 4 herd: The beginning of a new era of natively multimodal ai innovation, april 2025, 2025

    AI Meta. The llama 4 herd: The beginning of a new era of natively multimodal ai innovation, april 2025, 2025

    2025

  42. [50]

    C. Nash, J. Menick, S. Dieleman, and P. W Battaglia. Generating images with sparse representations. arXiv preprint arXiv:2103.03841, 2021

    2021 arXiv

  43. [51]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    Alex Nichol, Prafulla Dhariwal, Aditya Ramesh, Pranav Shyam, Pamela Mishkin, Bob McGrew, Ilya Sutskever, and Mark Chen. Glide: Towards photorealistic image generation and editing with text-guided diffusion models.arXiv preprint arXiv:2112.10741, 2021

    2021 arXiv

  44. [52]

    Improved denoising diffusion probabilistic models

    Alexander Quinn Nichol and Prafulla Dhariwal. Improved denoising diffusion probabilistic models. In International conference on machine learning, pages 8162–8171. PMLR, 2021

    2021

  45. [53]

    Pearson Educación, 1997

    Alan V Oppenheim, Alan S Willsky, and Syed Hamid Nawab.Signals & systems. Pearson Educación, 1997

    1997

  46. [54]

    Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, et al. Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  47. [55]

    Scalable diffusion models with transformers

    William Peebles and Saining Xie. Scalable diffusion models with transformers. InICCV, 2023

    2023

  48. [56]

    Yarn: Efficient context window extension of large language models.arXiv preprint arXiv:2309.00071, 2023

    Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. Yarn: Efficient context window extension of large language models.arXiv preprint arXiv:2309.00071, 2023

    2023 arXiv

  49. [57]

    Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    2023 arXiv

  50. [58]

    Train short, test long: Attention with linear biases enables input length extrapolation

    Ofir Press, Smith Noah, and Lewis Mike. Train short, test long: Attention with linear biases enables input length extrapolation. InICLR, 2021

    2021

  51. [59]

    Hierarchical text-conditional image generation with clip latents.arXiv preprint arXiv:2204.06125, 1(2):3, 2022

    Aditya Ramesh, Prafulla Dhariwal, Alex Nichol, Casey Chu, and Mark Chen. Hierarchical text-conditional image generation with clip latents.arXiv preprint arXiv:2204.06125, 1(2):3, 2022

    2022 arXiv

  52. [60]

    Zero-shot text-to-image generation

    Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea V oss, Alec Radford, Mark Chen, and Ilya Sutskever. Zero-shot text-to-image generation. InInternational conference on machine learning, pages 8821–8831. Pmlr, 2021. 28

    2021

  53. [61]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer. High-resolution image synthesis with latent diffusion models. InCVPR, 2022

    2022

  54. [62]

    Salimans, I

    T. Salimans, I. Goodfellow, W. Zaremba, V . Cheung, A. Radford, and X Chen. Improved techniques for training gans.NeurIPS, 2016

    2016

  55. [63]

    Adversarial diffusion distillation

    Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation. InEuropean Conference on Computer Vision, pages 87–103. Springer, 2024

    2024

  56. [64]

    Generative modeling by estimating gradients of the data distribution

    Yang Song and Stefano Ermon. Generative modeling by estimating gradients of the data distribution. NeurIPS, 2019

    2019

  57. [65]

    Improved techniques for training score-based generative models.NeurIPS, 2020

    Yang Song and Stefano Ermon. Improved techniques for training score-based generative models.NeurIPS, 2020

    2020

  58. [66]

    Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456, 2020

    Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations.arXiv preprint arXiv:2011.13456, 2020

    2011 arXiv

  59. [67]

    Springer, 2013

    Milan Sonka, Vaclav Hlavac, and Roger Boyle.Image processing, analysis and machine vision. Springer, 2013

    2013

  60. [68]

    Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 2024

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding.Neurocomputing, 2024

    2024

  61. [69]

    Autore- gressive model beats diffusion: Llama for scalable image generation.arXiv preprint arXiv:2406.06525, 2024

    Peize Sun, Yi Jiang, Shoufa Chen, Shilong Zhang, Bingyue Peng, Ping Luo, and Zehuan Yuan. Autore- gressive model beats diffusion: Llama for scalable image generation.arXiv preprint arXiv:2406.06525, 2024

    2024 arXiv

  62. [70]

    Seed1.5-vl technical report.arXiv preprint arXiv:2505.07062, 2025

    ByteDance Seed Team. Seed1.5-vl technical report.arXiv preprint arXiv:2505.07062, 2025

    2025 arXiv

  63. [71]

    Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Yonghui Wu, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, et al. Gemini: a family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023

    2023 arXiv

  64. [72]

    Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.arXiv preprint arXiv:2403.05530, 2024

    Gemini Team, Petko Georgiev, Ving Ian Lei, Ryan Burnell, Libin Bai, Anmol Gulati, Garrett Tanzer, Damien Vincent, Zhufeng Pan, Shibo Wang, et al. Gemini 1.5: Unlocking multimodal understanding across millions of tokens of context.arXiv preprint arXiv:2403.05530, 2024

    2024 arXiv

  65. [73]

    Gemma 3 technical report.arXiv preprint arXiv:2503.19786, 2025

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, et al. Gemma 3 technical report.arXiv preprint arXiv:2503.19786, 2025

    2025 arXiv

  66. [74]

    Kimi Team, Angang Du, Bofei Gao, Bowei Xing, Changjiu Jiang, Cheng Chen, Cheng Li, Chenjun Xiao, Chenzhuang Du, Chonghua Liao, et al. Kimi k1. 5: Scaling reinforcement learning with llms.arXiv preprint arXiv:2501.12599, 2025

    2025 arXiv

  67. [75]

    Kimi-vl technical report.arXiv preprint arXiv:2504.07491, 2025

    Kimi Team, Angang Du, Bohong Yin, Bowei Xing, Bowen Qu, Bowen Wang, Cheng Chen, Chenlin Zhang, Chenzhuang Du, Chu Wei, et al. Kimi-vl technical report.arXiv preprint arXiv:2504.07491, 2025

    2025 arXiv

  68. [76]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction.Advances in neural information processing systems, 37:84839–84865, 2024

    Keyu Tian, Yi Jiang, Zehuan Yuan, Bingyue Peng, and Liwei Wang. Visual autoregressive modeling: Scalable image generation via next-scale prediction.Advances in neural information processing systems, 37:84839–84865, 2024

    2024

  69. [77]

    Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023a

    Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, and Baptiste Rozière et al. Llama: Open and efficient foundation language models.arXiv preprint arXiv:2302.13971, 2023a

    arXiv

  70. [78]

    Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023b

    Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, and Nikolay Bashlykov et al. Llama 2: Open foundation and fine-tuned chat models.arXiv preprint arXiv:2307.09288, 2023b

    arXiv

  71. [79]

    Flowdcn: Exploring dcn-like architectures for fast image generation with arbitrary resolution.arXiv preprint arXiv:2410.22655, 2024

    Shuai Wang, Zexian Li, Tianhui Song, Xubin Li, Tiezheng Ge, Bo Zheng, and Limin Wang. Flowdcn: Exploring dcn-like architectures for fast image generation with arbitrary resolution.arXiv preprint arXiv:2410.22655, 2024

    2024 arXiv

  72. [80]

    Fitv2: Scalable and improved flexible vision transformer for diffusion model.arXiv preprint arXiv:2410.13925, 2024

    ZiDong Wang, Zeyu Lu, Di Huang, Cai Zhou, Wanli Ouyang, and LEI BAI. Fitv2: Scalable and improved flexible vision transformer for diffusion model.arXiv preprint arXiv:2410.13925, 2024. 29

    2024 arXiv

  73. [81]

    Sana: Efficient high-resolution image synthesis with linear diffusion transformers.arXiv preprint arXiv:2410.10629, 2024

    Enze Xie, Junsong Chen, Junyu Chen, Han Cai, Haotian Tang, Yujun Lin, Zhekai Zhang, Muyang Li, Ligeng Zhu, Yao Lu, et al. Sana: Efficient high-resolution image synthesis with linear diffusion transformers.arXiv preprint arXiv:2410.10629, 2024

    arXiv

  74. [82]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report.arXiv preprint arXiv:2412.15115, 2024

    2024 arXiv

  75. [83]

    Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024

    Zhuoyi Yang, Jiayan Teng, Wendi Zheng, Ming Ding, Shiyu Huang, Jiazheng Xu, Yuanming Yang, Wenyi Hong, Xiaohan Zhang, Guanyu Feng, et al. Cogvideox: Text-to-video diffusion models with an expert transformer.arXiv preprint arXiv:2408.06072, 2024

    2024 arXiv

  76. [84]

    Minicpm-v: A gpt-4v level mllm on your phone.arXiv preprint arXiv:2408.01800, 2024

    Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, et al. Minicpm-v: A gpt-4v level mllm on your phone.arXiv preprint arXiv:2408.01800, 2024

    2024 arXiv

  77. [85]

    Scaling autoregressive models for content-rich text-to-image generation.arXiv preprint arXiv:2206.10789, 2(3):5, 2022

    Jiahui Yu, Yuanzhong Xu, Jing Yu Koh, Thang Luong, Gunjan Baid, Zirui Wang, Vijay Vasudevan, Alexander Ku, Yinfei Yang, Burcu Karagol Ayan, et al. Scaling autoregressive models for content-rich text-to-image generation.arXiv preprint arXiv:2206.10789, 2(3):5, 2022

    2022 arXiv

  78. [86]

    Representation alignment for generation: Training diffusion transformers is easier than you think

    Sihyun Yu, Sangkyung Kwak, Huiwon Jang, Jongheon Jeong, Jonathan Huang, Jinwoo Shin, and Saining Xie. Representation alignment for generation: Training diffusion transformers is easier than you think. arXiv preprint arXiv:2410.06940, 2024

    arXiv

  79. [87]

    Pose: Efficient context window extension of llms via positional skip-wise training.arXiv preprint arXiv:2309.10400, 2023

    Dawei Zhu, Nan Yang, Liang Wang, Yifan Song, Wenhao Wu, Furu Wei, and Sujian Li. Pose: Efficient context window extension of llms via positional skip-wise training.arXiv preprint arXiv:2309.10400, 2023. 30

    2023 arXiv

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.