Pith. sign in

REVIEW 3 major objections 4 minor 55 references

EarthMapper: Visual Autoregressive Models for Controllable Bidirectional Satellite-Map Translation

T0 review · 3 major / 4 minor · reviewed 2026-08-16 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read EarthMapper claims one autoregressive model translates satellite images to maps and back, outperforming GAN, diffusion, and prior AR baselines.

desk verdict CNSatMap is the real contribution; the SOTA margins are compromised by test-set tuning and a couple of questionable formulas, but the base architecture is strong enough that the paper deserves a serious referee. read the letter →

arxiv 2504.19432 v1 pith:V4D6CEK6 submitted 2025-04-28 cs.CV cs.AI

Zhe Dong , Yuzhe Sun , Tianzhu Liu , Wangmeng Zuo , Yanfeng Gu This is my paper · ORCID
classification cs.CVcs.AI
keywords bidirectionalsatellite-maptranslationvisualautoregressivemodelgeo-conditionedjointscaleautoregressionsemanticinfusionkeypointadaptiveguidanceCNSatMapdatasetcontrollableimagegenerationremotesensing
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

EarthMapper claims that one visual autoregressive model can translate satellite imagery into maps and maps into satellite imagery, and that this single model outperforms the compared GAN, diffusion, and prior autoregressive baselines in both translation directions. The paper attributes the gain to three mechanisms: geographic-coordinate embeddings that anchor generation to a place, a joint-scale autoregressive training objective that unifies both directions in one training cycle, and inference-time key-point guidance that balances fidelity with diversity. A semantic-infusion loss aligns generated features with a pretrained vision model's features during training. The paper also contributes CNSatMap, a dataset of 302,132 aligned satellite-map pairs across 38 Chinese cities, and shows the same model handles zero-shot in-painting, out-painting, and coordinate-only generation. The potential payoff is automated cartography, rapid urban simulation, and augmentation of satellite datasets with rare scenes.

What carries the argument

The load-bearing mechanism is the geo-conditioned joint scale autoregression (GJSA): both modalities are tokenized into multi-scale discrete token maps by hierarchical residual quantization, the satellite and map tokens are paired at each scale, and a transformer predicts each scale's token pair conditioned on all previous scales plus a sinusoidal embedding of latitude and longitude. That joint next-scale prediction gives the model a single training objective for both directions. At inference, the source modality's token maps are inserted into the joint sequence, and the key point force adds selected quantized indices of the condition to the sampled indices at high-importance positions; complexity guidance then modulates the conditioning strength as a function of resolution and image complexity. During training, a semantic infusion loss aligns the autoregressive hidden states with features from a frozen pretrained vision encoder, which the paper argues enforces feature-level consistency between generated and real images.

What would settle it

Run the map-to-satellite test set with the key-point index addition replaced by the identity mapping, keeping all other settings fixed, and recompute FID and KID; if the metrics do not materially worsen, the index-addition step is not the source of the reported conditional fidelity.

Watch

Extended reading notes

Core claim

The paper's central claim is that bidirectional satellite-map translation can be unified in a single multi-scale autoregressive framework, conditioned on geographic coordinates and on the source modality, so that map-to-satellite and satellite-to-map generation share one set of learned parameters. On the New York dataset it reports FID 36.54 and KID 0.99 for map-to-satellite translation, and SSIM 0.6534 with PSNR 25.04 for satellite-to-map; on the new CNSatMap dataset it reports FID 29.89 and KID 2.06 for map-to-satellite, and SSIM 0.7300 with PSNR 26.88 for satellite-to-map. In the paper's comparisons, these numbers beat the GAN, latent-diffusion, and autoregressive baselines included in the study on almost every reported metric, and the paper shows the same model producing coherent results for zero-shot in-painting, out-painting, and coordinate-only generation. The bidirectional capability follows from pairing the two modalities' token maps at each scale and treating a full scale's pair as one autoregressive step.

Load-bearing premise

The method assumes that adding the conditional image's quantized codebook indices to the generated image's indices at selected key points steers the autoregressive sampler toward the intended target, an assumption that is not proven and is not guaranteed because codebook indices are unordered labels.

Editorial extensions

If this is right

  • If the reported numbers hold, EarthMapper establishes a new published benchmark on both the New York and CNSatMap test sets, with map-to-satellite FID roughly half that of the strongest prior autoregressive baseline.
  • Because one model handles both directions, deployment for mapping and simulation pipelines could use a single checkpoint instead of separate translation networks for each direction.
  • The CNSatMap dataset becomes a large-scale shared resource: 302,132 aligned pairs across 38 Chinese cities could support training and evaluation of future cross-modal geographic models beyond this architecture.
  • The geographic-coordinate embedding may enable generating imagery for arbitrary locations without retraining, which would matter for scenario simulation after disasters and for augmenting satellite datasets with rare scene types.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A testable extension is to replace the key-point index addition with a learned attention-based conditioner: codebook indices are unordered labels, so index arithmetic is not semantically grounded, and a learned conditioner that matches or beats the reported metrics would show the index addition is not what carries conditional control.
  • The cross-dataset results, CNSatMap-to-New York generalizing better than New York-to-CNSatMap, suggest that scale and geographic diversity of the training corpus, not the architecture alone, drive transfer; this points to a practical recipe of pretraining on large multi-city datasets before fine-tuning on a target region.
  • Coordinate-only generation opens a quantitative test of the coordinate embedding itself: generating at held-out coordinates with known ground truth would separate how much the output is determined by location versus by learned visual priors, a separation the current aggregate metrics do not provide.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper presents EarthMapper, a visual autoregressive framework for bidirectional satellite-map translation (BSMT). The method combines geographic coordinate embeddings, multi-scale hierarchical residual quantization, a geo-conditioned joint scale autoregression (GJSA) transformer, a semantic infusion (SI) loss using a frozen DINOv2 encoder, and an inference-time key point adaptive guidance (KPAG) scheme consisting of key point force (KPF) and complexity guidance (CG). The authors also introduce CNSatMap, a large-scale dataset of 302,132 aligned satellite-map pairs across 38 Chinese cities. Experiments on CNSatMap and the New York dataset report state-of-the-art FID, KID, SSIM, PSNR, RMSE, and LPIPS numbers, along with ablations and zero-shot demonstrations for in-painting, out-painting, and coordinate-conditional generation.

Significance. If confirmed, the work would be a meaningful advance in remote-sensing image translation: it is one of the first autoregressive frameworks to unify bidirectional satellite-map translation in a single training cycle, and the CNSatMap dataset is a potentially valuable benchmark resource. The paper contains useful design ingredients, and the ablations in Table III show monotone improvements from each added component. The zero-shot in-painting/out-painting and coordinate-conditional experiments are promising, and the promised public release of code and data is commendable. However, the headline state-of-the-art claim is currently not well supported because the best guidance scales are selected using the test set, and the mathematical description of the inference-time guidance has unresolved issues.

major comments (3)
  1. [V.D.2 and Tables IV/V vs. Table I] The reported headline numbers in Table I are obtained by selecting the guidance scale on the New York test set: Table IV picks [8,8,8] for map-to-satellite and Table V picks [2,2,2] for satellite-to-map, and these exact configurations are then reported as EarthMapper's performance in Table I. This is test-set hyperparameter selection, not evaluation of a fixed configuration. The baselines in Table I are not given the same per-direction test-set tuning, so the comparison is not apples-to-apples. Furthermore, all metrics are reported as single point estimates with no error bars, multiple seeds, or statistical significance tests; margins such as SSIM 0.6534 vs. 0.6465 for ControlVAR may lie within run-to-run noise. The central claim of state-of-the-art performance should be re-established with validation-based hyperparameter selection, repeated seeds, and a fair tuning protocol for baselines.
  2. [IV.F, Eq. (19)] The key point force mechanism adds the conditional image's quantized codebook index to the generated index at selected key points. Codebook indices are categorical and unordered, so adding them with clipping (Eq. (19)) is not a semantically meaningful operation in the token space. The paper states that this 'integrates the information from the key points into the generated image', but no formal argument or dedicated experiment shows that this operation biases the autoregressive sampler toward the intended conditional target; the semantics of 'key point' selection in Eq. (17) is likewise based only on the magnitude of a normalized index. Since KPF is a core component of the claimed controllable bidirectional translation mechanism, the authors should either provide a principled justification for index addition or replace it with an operation that provably maps conditional information into the generative distribution.
  3. [IV.G, Eq. (23)] Equation (23) defines the complexity-guided CFG as pθ(xi|x<i,c) = pθ(xi|x<i) + s(xi,ϕ)·∇pθ(xi|x<i) log pφ(c|xi). As written, this is not a normalized probability distribution, and the gradient notation ∇pθ(xi|x<i) log pφ(c|xi) is undefined: the gradient is not taken with respect to the probability mass function, and no practical algorithm is given for computing this term for discrete token probabilities. This is not a cosmetic issue: Table III shows that complexity guidance contributes a meaningful improvement (FID 31.57→29.89, SSIM 0.7054→0.7300), so the mechanism is load-bearing. The authors should provide a precise, implementable definition of this guidance step, including how the gradient is computed, how normalization is handled, and how α(ri), β(C(xi)) and γ are set in practice.
minor comments (4)
  1. [IV.F, paragraph after Eq. (15)] The text says 'the map portion of the generated paired image is substituted with a vector-quantized real map' but the subsequent paragraph notes that the architecture prevents direct information exchange between generated paired images; the relationship between these two statements should be clarified.
  2. [V.C, paragraph 5] The claim that 'EarthMapper's Recall of 0.4890 (New York) ... exceeds ControlNet's 0.2110' is correct for New York, but the same sentence discusses CNSatMap where ControlNet's Precision (0.4886) exceeds EarthMapper's (0.4294); the text should acknowledge this asymmetry explicitly.
  3. [IV.C, Eq. (2) and Eq. (7)] The notation Q(f_m) in Eq. (2) is not defined until later, and Eq. (7) applies HRQ to each modality but the shared-codebook statement in the text should be made explicit in the equations.
  4. [General] There are minor typos and grammatical infelicities, e.g., 'we redefine' should be 'we define', 'a AR-based' should be 'an AR-based', and the phrase 'the optimisation achieved during training' in Section IV.F should be reworded.

Circularity Check

1 steps flagged · score 6.0 of 10

Test-set tuning of guidance scales in Tables IV/V directly produces the Table I SOTA numbers; no equation-level circularity in the core AR objective.

  1. fitted input called prediction [Section V.D.2 (Tables IV and V) feeding Section V.C Table I, New York dataset]
    "To elucidate the role of the guidance scale in EarthMapper’s complexity guidance mechanism, we conducted an ablation study on the New York test set... For map-to-satellite translation, a guidance scale of [8, 8, 8] achieves optimal performance, yielding an FID of 36.54... In contrast, for satellite-to-map translation, a guidance scale of [2, 2, 2] delivers the best performance, with an SSIM of 0.6534..."

    The guidance scale is a free inference hyperparameter. The paper sweeps it on the New York test set, selects the value that optimizes the headline metric on that exact split, and then reports those post-selection numbers as EarthMapper's row in the state-of-the-art comparison (Table I). Thus the claimed improvements (e.g., FID 36.54 vs ControlVAR 58.23; SSIM 0.6534 vs 0.6465) are the best case of a test-set search rather than the performance of a fixed, a priori configuration. Baselines receive no equivalent per-direction test-set tuning, so the margin is not an unbiased prediction; it is partly constructed by the selection procedure.

full rationale

The core derivation (Eqs. 10-14) is a standard joint next-scale autoregressive likelihood plus a feature-matching auxiliary loss; no variable is defined in terms of the target metric, and no load-bearing result is imported from a self-citation. The only genuinely circular element is the evaluation protocol: the guidance-scale ablation is run on the New York test set, the best scale is chosen per direction on that split, and the same post-selection numbers are then reported as EarthMapper's performance in Table I. Since FID/SSIM on the test set are both the selection criterion and the reported outcome, the headline NY margins over baselines are the best case of a small grid search rather than predictions of a fixed configuration; baselines were not given the same per-direction tuning. This is a partial, evaluation-level circularity rather than a mathematical derivation loop. The KPF index-addition heuristic in Eq. (19) is a correctness/interpretability concern, not a circularity: it does not reduce the output to the input by construction. CNSatMap results are less affected because no scale sweep on that test set is shown, though the same tuned scales may have been reused.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

The central claim rests mostly on empirical training and unverified architectural assumptions: the HRQ tokenizer and pretrained VAR weights are inherited from prior work, the KPF index-addition is an ad hoc manipulation, the semantic-infusion loss assumes DINOv2 features capture geospatial semantics, and dataset alignment is asserted without error measurements. Free parameters are numerous and several are tuned on the test sets.

free parameters (6)
  • sigma (semantic infusion weight) = 0.5
    Set in Section V-B without sensitivity analysis; it weights L_sem in the overall loss.
  • tau (key point selection threshold) = not reported
    Defined in Eq. (17) but the actual threshold value is never stated, so it is an unlisted hyperparameter.
  • guidance scale for complexity guidance = [8,8,8] for map-to-satellite, [2,2,2] for satellite-to-map on New York
    Selected from test-set ablation results in Tables IV and V, then used for the final reported numbers.
  • top-k and top-p sampling parameters = k=100, p=0.55
    Set in Section V-B for inference; no sensitivity analysis is provided.
  • map tile structural diversity threshold = color std < 10
    Used in Section III to filter CNSatMap tiles; affects dataset composition and benchmark difficulty.
  • alpha(r_i) and beta(C(x_i)) functions = unspecified
    The complexity guidance in Section IV-G depends on these functions but their explicit form or chosen values are never given.
assumptions (5)
  • ad hoc to paper Adding the conditional image's quantized codebook index to the generated index at key positions steers generation toward the conditional input.
    Eq. (19) is asserted without derivation; codebook indices are unordered so index addition is not a natural operation.
  • domain assumption DINOv2's feature space provides a valid semantic ground truth for the semantic-infusion loss.
    Section IV-E uses frozen DINOv2 features as supervision, assuming they align with geovisual semantics.
  • domain assumption The geometric alignment pipeline ensures pixel-level correspondence between satellite and map tiles.
    Section III claims pixel-level correspondence after registration, but no alignment error is measured and the paper itself notes the modalities lack strict pixel-wise correspondence.
  • ad hoc to paper Complexity guidance as formulated in Eq. (23) improves conditional generation.
    Eq. (23) is not derived from Eq. (20), and the update rule is not implemented or validated in sufficient detail.
  • domain assumption Joint scale autoregression p(r_k | r_{<k}, c_g) is a valid joint model of satellite and map modalities.
    This is the core modeling assumption; cross-modal dependencies are captured through paired tokens at each scale.

how reviews work

0 comments
Cite this review

Pith. "Pith review of EarthMapper: Visual Autoregressive Models for Controllable Bidirectional Satellite-Map Translation." pith.science (2026). https://pith.science/paper/V4D6CEK6

@misc{pith2026250419432,
  author       = {Pith},
  title        = {Pith review of: EarthMapper: Visual Autoregressive Models for Controllable Bidirectional Satellite-Map Translation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/V4D6CEK6}},
  note         = {Machine review of arXiv:2504.19432}
}
read the original abstract

Satellite imagery and maps, as two fundamental data modalities in remote sensing, offer direct observations of the Earth's surface and human-interpretable geographic abstractions, respectively. The task of bidirectional translation between satellite images and maps (BSMT) holds significant potential for applications in urban planning and disaster response. However, this task presents two major challenges: first, the absence of precise pixel-wise alignment between the two modalities substantially complicates the translation process; second, it requires achieving both high-level abstraction of geographic features and high-quality visual synthesis, which further elevates the technical complexity. To address these limitations, we introduce EarthMapper, a novel autoregressive framework for controllable bidirectional satellite-map translation. EarthMapper employs geographic coordinate embeddings to anchor generation, ensuring region-specific adaptability, and leverages multi-scale feature alignment within a geo-conditioned joint scale autoregression (GJSA) process to unify bidirectional translation in a single training cycle. A semantic infusion (SI) mechanism is introduced to enhance feature-level consistency, while a key point adaptive guidance (KPAG) mechanism is proposed to dynamically balance diversity and precision during inference. We further contribute CNSatMap, a large-scale dataset comprising 302,132 precisely aligned satellite-map pairs across 38 Chinese cities, enabling robust benchmarking. Extensive experiments on CNSatMap and the New York dataset demonstrate EarthMapper's superior performance, achieving significant improvements in visual realism, semantic consistency, and structural fidelity over state-of-the-art methods. Additionally, EarthMapper excels in zero-shot tasks like in-painting, out-painting and coordinate-conditional generation, underscoring its versatility.

Figures

Figures reproduced from arXiv: 2504.19432 by the authors.

Figure 1
Figure 1. Conceptual illustration of bidirectional satellite-map translation and [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the geographical distribution of satellite-map pairs sampled from the proposed CNSatMap dataset. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Overview of our proposed EarthMapper framework, with the upper section dedicated to training and the lower section to inference. During training, [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Schematic diagram of the inference section, with keypoint adaptive [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]
Figure 5
Figure 5. Figure 5: Qualitative comparison of bidirectional satellite-map translation results on the New York test set. The top five rows illustrate map-to-satellite translation, [PITH_FULL_IMAGE:figures/full_fig_p011_5.png]
Figure 6
Figure 6. Figure 6: Qualitative comparison of bidirectional satellite-map translation results on the CNSatMap test set. The top five rows illustrate map-to-satellite translation, [PITH_FULL_IMAGE:figures/full_fig_p012_6.png]
Figure 7
Figure 7. Figure 7: Illustration of generative diversity in EarthMapper. Given a single [PITH_FULL_IMAGE:figures/full_fig_p013_7.png]
Figure 9
Figure 9. Figure 9: Zero-shot performance of EarthMapper in in-painting and out-painting tasks. [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]
Figure 10
Figure 10. Figure 10: Zero-shot performance of EarthMapper in coordinate-conditional [PITH_FULL_IMAGE:figures/full_fig_p016_10.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

55 extracted references · 19 canonical work pages

  1. [1]

    Image to Image Translation : Generating maps from satellite images

    V . Ingale, R. Singh, and P. Patwal, “Image to image translation: Generating maps from satellite images,” arXiv preprint arXiv:2105.09253, 2021

    work page Pith review arXiv 2021

  2. [2]

    Generate your own scotland: Satellite image generation conditioned on maps,

    M. Espinosa and E. J. Crowley, “Generate your own scotland: Satellite image generation conditioned on maps,” arXiv preprint arXiv:2308.16648, 2023

    arXiv 2023

  3. [3]

    Generative adversarial networks,

    I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adversarial networks,” Communications of the ACM , vol. 63, no. 11, pp. 139–144, 2020

    2020

  4. [4]

    Conditional generative adversarial nets,

    M. Mirza and S. Osindero, “Conditional generative adversarial nets,” arXiv preprint arXiv:1411.1784 , 2014

    arXiv 2014

  5. [5]

    Image-to-image translation with conditional adversarial networks,

    P. Isola, J.-Y . Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 1125– 1134

    2017

  6. [6]

    Unpaired image-to-image translation using cycle-consistent adversarial networks,

    J.-Y . Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in Proceedings of the IEEE international conference on computer vision , 2017, pp. 2223–2232

    2017

  7. [7]

    Denoising diffusion probabilistic models,

    J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” Advances in neural information processing systems , vol. 33, pp. 6840– 6851, 2020

    2020

  8. [8]

    Diffusion models beat gans on image synthesis,

    P. Dhariwal and A. Nichol, “Diffusion models beat gans on image synthesis,” Advances in neural information processing systems , vol. 34, pp. 8780–8794, 2021

    2021

Show all 55 references
  1. [9]

    Generative pretraining from pixels,

    M. Chen, A. Radford, R. Child, J. Wu, H. Jun, D. Luan, and I. Sutskever, “Generative pretraining from pixels,” in International conference on machine learning . PMLR, 2020, pp. 1691–1703

    2020

  2. [10]

    Conditional image generation with pixelcnn decoders,

    A. Van den Oord, N. Kalchbrenner, L. Espeholt, O. Vinyals, A. Graves et al., “Conditional image generation with pixelcnn decoders,” Advances in neural information processing systems , vol. 29, 2016. 17

    2016

  3. [11]

    Visual autoregressive modeling: Scalable image generation via next-scale prediction,

    K. Tian, Y . Jiang, Z. Yuan, B. Peng, and L. Wang, “Visual autoregressive modeling: Scalable image generation via next-scale prediction,” Advances in neural information processing systems , vol. 37, pp. 84 839–84 865, 2024

    2024

  4. [12]

    Controlvar: Exploring controllable visual autoregressive modeling,

    X. Li, K. Qiu, H. Chen, J. Kuen, Z. Lin, R. Singh, and B. Raj, “Controlvar: Exploring controllable visual autoregressive modeling,” arXiv preprint arXiv:2406.09750 , 2024

    2024 arXiv

  5. [13]

    Car: Controllable autoregressive modeling for visual generation,

    Z. Yao, J. Li, Y . Zhou, Y . Liu, X. Jiang, C. Wang, F. Zheng, Y . Zou, and L. Li, “Car: Controllable autoregressive modeling for visual generation,” arXiv preprint arXiv:2410.04671 , 2024

    2024 arXiv

  6. [14]

    Adding conditional control to text-to-image diffusion models,

    L. Zhang, A. Rao, and M. Agrawala, “Adding conditional control to text-to-image diffusion models,” in Proceedings of the IEEE/CVF international conference on computer vision , 2023, pp. 3836–3847

    2023

  7. [15]

    High- resolution image synthesis with latent diffusion models,

    R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 10 684–10 695

    2022

  8. [16]

    Gligen: Open-set grounded text-to-image generation,

    Y . Li, H. Liu, Q. Wu, F. Mu, J. Yang, J. Gao, C. Li, and Y . J. Lee, “Gligen: Open-set grounded text-to-image generation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2023, pp. 22 511–22 521

    2023

  9. [17]

    A style-based generator architecture for generative adversarial networks,

    T. Karras, S. Laine, and T. Aila, “A style-based generator architecture for generative adversarial networks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2019, pp. 4401– 4410

    2019

  10. [18]

    Interfacegan: Interpreting the disentangled face representation learned by gans,

    Y . Shen, C. Yang, X. Tang, and B. Zhou, “Interfacegan: Interpreting the disentangled face representation learned by gans,” IEEE transactions on pattern analysis and machine intelligence , vol. 44, no. 4, pp. 2004–2018, 2020

    2004

  11. [19]

    High-resolution image reconstruction with latent diffusion models from human brain activity,

    Y . Takagi and S. Nishimoto, “High-resolution image reconstruction with latent diffusion models from human brain activity,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2023, pp. 14 453–14 463

    2023

  12. [20]

    An image is worth one word: Personalizing text-to-image generation using textual inversion,

    R. Gal, Y . Alaluf, Y . Atzmon, O. Patashnik, A. H. Bermano, G. Chechik, and D. Cohen-Or, “An image is worth one word: Personalizing text-to-image generation using textual inversion,” arXiv preprint arXiv:2208.01618, 2022

    2022 arXiv

  13. [21]

    Dreambooth: Fine tuning text-to-image diffusion models for subject- driven generation,

    N. Ruiz, Y . Li, V . Jampani, Y . Pritch, M. Rubinstein, and K. Aberman, “Dreambooth: Fine tuning text-to-image diffusion models for subject- driven generation,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2023, pp. 22 500–22 510

    2023

  14. [22]

    Glide: Towards photorealistic image generation and editing with text-guided diffusion models,

    A. Nichol, P. Dhariwal, A. Ramesh, P. Shyam, P. Mishkin, B. McGrew, I. Sutskever, and M. Chen, “Glide: Towards photorealistic image generation and editing with text-guided diffusion models,”arXiv preprint arXiv:2112.10741, 2021

    2021 arXiv

  15. [23]

    Hierarchical text-conditional image generation with clip latents,

    A. Ramesh, P. Dhariwal, A. Nichol, C. Chu, and M. Chen, “Hierarchical text-conditional image generation with clip latents,” arXiv preprint arXiv:2204.06125, vol. 1, no. 2, p. 3, 2022

    2022 arXiv

  16. [24]

    Learning transferable visual models from natural language supervision,

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al. , “Learning transferable visual models from natural language supervision,” in International conference on machine learning . PmLR, 2021, pp. 8748–8763

    2021

  17. [25]

    Classifier-free diffusion guidance,

    J. Ho and T. Salimans, “Classifier-free diffusion guidance,” arXiv preprint arXiv:2207.12598, 2022

    2022 arXiv

  18. [26]

    Any-to-any generation via composable diffusion,

    Z. Tang, Z. Yang, C. Zhu, M. Zeng, and M. Bansal, “Any-to-any generation via composable diffusion,” Advances in Neural Information Processing Systems, vol. 36, pp. 16 083–16 099, 2023

    2023

  19. [27]

    Language models are unsupervised multitask learners,

    A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever et al. , “Language models are unsupervised multitask learners,” OpenAI blog , vol. 1, no. 8, p. 9, 2019

    2019

  20. [28]

    Language models are few-shot learners,

    T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askellet al., “Language models are few-shot learners,” Advances in neural information processing systems, vol. 33, pp. 1877–1901, 2020

    1901

  21. [29]

    Image transformer,

    N. Parmar, A. Vaswani, J. Uszkoreit, L. Kaiser, N. Shazeer, A. Ku, and D. Tran, “Image transformer,” in International conference on machine learning. PMLR, 2018, pp. 4055–4064

    2018

  22. [30]

    Video super-resolution using an adaptive superpixel-guided auto-regressive model,

    K. Li, Y . Zhu, J. Yang, and J. Jiang, “Video super-resolution using an adaptive superpixel-guided auto-regressive model,” Pattern Recognition, vol. 51, pp. 59–71, 2016

    2016

  23. [31]

    Lar-sr: A local autoregressive model for image super-resolution,

    B. Guo, X. Zhang, H. Wu, Y . Wang, Y . Zhang, and Y .-F. Wang, “Lar-sr: A local autoregressive model for image super-resolution,” inProceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2022, pp. 1909–1918

    2022

  24. [32]

    Outpainting by queries,

    K. Yao, P. Gao, X. Yang, J. Sun, R. Zhang, and K. Huang, “Outpainting by queries,” in European conference on computer vision . Springer, 2022, pp. 153–169

    2022

  25. [33]

    Vqgan-clip: Open domain image generation and editing with natural language guidance,

    K. Crowson, S. Biderman, D. Kornis, D. Stander, E. Hallahan, L. Castricato, and E. Raff, “Vqgan-clip: Open domain image generation and editing with natural language guidance,” in European conference on computer vision . Springer, 2022, pp. 88–105

    2022

  26. [34]

    Controlar: Controllable image generation with autoregressive models,

    Z. Li, T. Cheng, S. Chen, P. Sun, H. Shen, L. Ran, X. Chen, W. Liu, and X. Wang, “Controlar: Controllable image generation with autoregressive models,” arXiv preprint arXiv:2410.02705 , 2024

    2024 arXiv

  27. [35]

    Pixel recurrent neural networks,

    A. Van Den Oord, N. Kalchbrenner, and K. Kavukcuoglu, “Pixel recurrent neural networks,” in International conference on machine learning. PMLR, 2016, pp. 1747–1756

    2016

  28. [36]

    Parallel multiscale autoregressive density estimation,

    S. Reed, A. Oord, N. Kalchbrenner, S. G. Colmenarejo, Z. Wang, Y . Chen, D. Belov, and N. Freitas, “Parallel multiscale autoregressive density estimation,” in International conference on machine learning . PMLR, 2017, pp. 2912–2921

    2017

  29. [37]

    Neural discrete representation learning,

    A. Van Den Oord, O. Vinyals et al. , “Neural discrete representation learning,” Advances in neural information processing systems , vol. 30, 2017

    2017

  30. [38]

    Generating diverse high-fidelity images with vq-vae-2,

    A. Razavi, A. Van den Oord, and O. Vinyals, “Generating diverse high-fidelity images with vq-vae-2,” Advances in neural information processing systems, vol. 32, 2019

    2019

  31. [39]

    Taming transformers for high- resolution image synthesis,

    P. Esser, R. Rombach, and B. Ommer, “Taming transformers for high- resolution image synthesis,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , 2021, pp. 12 873–12 883

    2021

  32. [40]

    Attention is all you need,

    A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems , vol. 30, 2017

    2017

  33. [41]

    Autoregressive image generation using residual quantization,

    D. Lee, C. Kim, S. Kim, M. Cho, and W.-S. Han, “Autoregressive image generation using residual quantization,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2022, pp. 11 523–11 532

    2022

  34. [42]

    Hsigene: A foundation model for hyperspectral image generation,

    L. Pang, X. Cao, D. Tang, S. Xu, X. Bai, F. Zhou, and D. Meng, “Hsigene: A foundation model for hyperspectral image generation,” arXiv preprint arXiv:2409.12470 , 2024

    2024 arXiv

  35. [43]

    Rsdiff: Remote sensing image generation from text using diffusion model,

    A. Sebaq and M. ElHelw, “Rsdiff: Remote sensing image generation from text using diffusion model,” Neural Computing and Applications , vol. 36, no. 36, pp. 23 103–23 111, 2024

    2024

  36. [44]

    Crs-diff: Controllable generative remote sensing foundation model,

    D. Tang, X. Cao, X. Hou, Z. Jiang, and D. Meng, “Crs-diff: Controllable generative remote sensing foundation model,” arXiv e-prints, pp. arXiv– 2403, 2024

    2024

  37. [45]

    Metaearth: A generative foundation model for global-scale remote sensing image generation,

    Z. Yu, C. Liu, L. Liu, Z. Shi, and Z. Zou, “Metaearth: A generative foundation model for global-scale remote sensing image generation,” IEEE Transactions on Pattern Analysis and Machine Intelligence , 2024

    2024

  38. [46]

    Diffusionsat: A generative foundation model for satellite imagery,

    S. Khanna, P. Liu, L. Zhou, C. Meng, R. Rombach, M. Burke, D. Lobell, and S. Ermon, “Diffusionsat: A generative foundation model for satellite imagery,” arXiv preprint arXiv:2312.03606 , 2023

    2023 arXiv

  39. [47]

    Gans trained by a two time-scale update rule converge to a local nash equilibrium,

    M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter, “Gans trained by a two time-scale update rule converge to a local nash equilibrium,” Advances in neural information processing systems , vol. 30, 2017

    2017

  40. [48]

    Demystifying mmd gans,

    M. Bi ´nkowski, D. J. Sutherland, M. Arbel, and A. Gretton, “Demystifying mmd gans,” arXiv preprint arXiv:1801.01401 , 2018

    2018 arXiv

  41. [49]

    Improved precision and recall metric for assessing generative models,

    T. Kynk ¨a¨anniemi, T. Karras, S. Laine, J. Lehtinen, and T. Aila, “Improved precision and recall metric for assessing generative models,” Advances in neural information processing systems , vol. 32, 2019

    2019

  42. [50]

    The unreasonable effectiveness of deep features as a perceptual metric,

    R. Zhang, P. Isola, A. A. Efros, E. Shechtman, and O. Wang, “The unreasonable effectiveness of deep features as a perceptual metric,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 586–595

    2018

  43. [51]

    Dinov2: Learning robust visual features without supervision,

    M. Oquab, T. Darcet, T. Moutakanni, H. V o, M. Szafraniec, V . Khalidov, P. Fernandez, D. Haziza, F. Massa, A. El-Nouby et al. , “Dinov2: Learning robust visual features without supervision,” arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  44. [52]

    Decoupled weight decay regularization,

    I. Loshchilov and F. Hutter, “Decoupled weight decay regularization,” arXiv preprint arXiv:1711.05101 , 2017

    2017 arXiv

  45. [53]

    Stegogan: Leveraging steganography for non- bijective image-to-image translation,

    S. Wu, Y . Chen, S. Mermet, L. Hurni, K. Schindler, N. Gonthier, and L. Landrieu, “Stegogan: Leveraging steganography for non- bijective image-to-image translation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition , 2024, pp. 7922–7931

    2024

  46. [54]

    Bbdm: Image-to-image translation with brownian bridge diffusion models,

    B. Li, K. Xue, B. Liu, and Y .-K. Lai, “Bbdm: Image-to-image translation with brownian bridge diffusion models,” in Proceedings of the IEEE/CVF conference on computer vision and pattern Recognition , 2023, pp. 1952–1961

    2023

  47. [55]

    Unicontrol: A unified diffusion model for controllable visual generation in the wild,

    C. Qin, S. Zhang, N. Yu, Y . Feng, X. Yang, Y . Zhou, H. Wang, J. C. Niebles, C. Xiong, S. Savarese et al. , “Unicontrol: A unified diffusion model for controllable visual generation in the wild,” arXiv preprint arXiv:2305.11147, 2023

    2023 arXiv

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.