{"id":"41ba6e66-939f-41ab-9902-87c716e38726","arxiv_id":"2506.23347","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"CycleVAR adapts a pretrained visual autoregressive model to unpaired image translation using softmax-relaxed quantization and source-token prefixes, achieving FID scores competitive with CycleGAN-Turbo.","lead":"The authors adapt a visual autoregressive image model, which normally generates images scale by scale, to translate images between domains without paired training examples. They replace the discrete codebook lookup with a soft probability mixture so gradients flow during training, and find that a single parallel pass produces better translations than a step-by-step autoregressive pass.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The SOTA claim is confounded by backbone scale and lacks uncertainty quantification: only the 2.3B Infinity version beats CycleGAN-Turbo, while the matched 310M version loses on one FID setting, and no seeds or error bars are reported.","rationale":"The reader's CONDITIONAL verdict is sensible, and I do not see a reason to move it. The framework is plausible, and the equations for both generation modes are internally coherent. I agree that the frozen decoder plus soft codebook mixtures is a genuine risk, but the paper's own temperature ablation shows that sharp one-hot decoding fails while moderate soft decoding works, which partially mitigates that concern. The less protected part of the argument is the external comparison: the headline SOTA claim depends on small, unrepeated metric differences and on the larger Infinity-2B backbone. The paper's own 310M row shows the method can lose on FID to CycleGAN-Turbo under a smaller backbone, suggesting a scale confound. The missing training details and code release noted by the reader further justify keeping the verdict conditional. The proposed test would decide whether the SOTA claim survives a controlled, uncertainty-aware comparison.","tokens_in":15682,"tokens_out":9515,"duration_ms":113216,"concrete_test":"Rerun the Table 1 comparison with CycleVAR w/ Infinity and CycleGAN-Turbo on the same validation split, using the same metric code, and run each method with at least 5 random seeds; report mean and standard deviation of FID and DINO, plus a paired bootstrap or Wilcoxon test on per-image DINO scores. Also run a backbone-matched CycleVAR variant (e.g., an Infinity checkpoint near 1B parameters) on horse-zebra and day-night. If the confidence intervals overlap, or the advantage disappears in the matched-backbone run, the 'surpasses CycleGAN-Turbo' claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that CycleVAR 'surpasses' CycleGAN-Turbo rests on Table 1, but that comparison does not establish superiority. The FID margins are 0.7-8.5 points (e.g., horse->zebra 40.3 vs 41.0, day->night 30.6 vs 31.3) and no seeds, confidence intervals, or significance tests are reported; margins of this size are commonly within run-to-run variance in unpaired translation. More importantly, the winning rows use CycleVAR w/ Infinity, a 2.3B text-conditional model, while CycleGAN-Turbo is built on the 1.1B SD-Turbo 2.1. The same CycleVAR framework on the 310M VAR backbone is worse than CycleGAN-Turbo on horse->zebra FID (47.8 vs 41.0), and day/night comparisons for that backbone are absent (footnote 3). Thus the apparent SOTA advantage may be an artifact of a larger pretrained backbone plus unreported noise, rather than of the prefilling and SRQ mechanism itself. The frozen-decoder/off-manifold concern is real, but the paper's temperature ablation (Table 5) gives some evidence that soft decoding is usable; the evaluation-support problem is the more immediate threat to the headline claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CycleVAR, an unsupervised image-to-image translation framework built on pre-trained visual autoregressive (VAR) models. The method tokenizes the source image into multi-scale residual maps, injects these as prefix context into a causal VAR transformer, and predicts target-domain tokens either serially across scales or in a parallel one-step mode. To enable end-to-end fine-tuning despite discrete quantization, the paper introduces Softmax Relaxed Quantization (SRQ), which replaces argmax codebook selection with a temperature-controlled softmax-weighted average of codebook vectors. Training uses cycle consistency, adversarial, and identity losses following CycleGAN-Turbo. Experiments on horse↔zebra, day↔night, and an anime scene dataset report FID and DINO-Structure metrics, and the paper claims state-of-the-art performance over prior unsupervised translation methods, especially CycleGAN-Turbo.","tokens_in":15937,"tokens_out":2542,"duration_ms":27479,"significance":"If the central claims hold, this is a meaningful contribution: it extends the autoregressive image generation paradigm to unsupervised translation with a single forward pass, potentially easing integration with LLM-style multimodal models. The paper's strengths include a clearly described framework, a concrete mechanism (SRQ) to address the gradient-truncation problem of VQ-based tokenizers, and ablations that isolate the benefits of the parallel one-step mode and multi-scale context/output (Tables 3 and 4). The temperature ablation (Table 5) gives direct evidence that soft codebook mixing is usable rather than catastrophically off-manifold. However, the headline comparison against CycleGAN-Turbo is confounded by backbone scale and lacks uncertainty quantification, which tempers the significance of the 'surpasses' claim.","major_comments":[{"comment":"The claim that CycleVAR surpasses CycleGAN-Turbo is not adequately supported. The winning rows all use CycleVAR w/ Infinity, a 2.3B text-conditional model, whereas CycleGAN-Turbo is built on SD-Turbo 2.1 with about 1.1B parameters. With the matched 310M VAR backbone, CycleVAR is worse on horse→zebra FID (47.8 vs 41.0) and is not evaluated on day↔night at all (footnote 3). FID margins in the winning rows are small (0.7–8.5 points), and no seeds, confidence intervals, or significance tests are reported. This comparison therefore does not demonstrate that the CycleVAR mechanism itself, rather than a larger backbone, is responsible for the apparent superiority.","section":"§4.2.1, Table 1"},{"comment":"The temperature τ in SRQ is tuned on the same FID and DINO-Structure metrics used for the headline comparisons, with no held-out validation or selection protocol. Since τ strongly controls translation quality (FID ranges from 228.2 at τ=0.01 to 47.82 at τ=2), hyperparameter selection on the evaluation metric can inflate the reported gains. The mixing weight a in Eqs. (6) and (9) is never ablated, leaving a second free parameter that could drive the parallel-mode improvement. At minimum, the paper should report sensitivity to a and clarify whether τ was chosen on a validation split.","section":"§4.3, Table 5"},{"comment":"The method assumes that the frozen VAE decoder, trained on one-hot codebook vectors, produces coherent images from soft convex combinations of codebook vectors (τ=2 in Table 5). The temperature ablation is helpful, but it reports only final image metrics; a more direct analysis of the decoded soft latents (e.g., reconstruction of soft-quantized features with the frozen decoder, or visual examples of decoded mixtures) would strengthen the claim that SRQ operates in a usable region of the decoder manifold and that gradients through the frozen decoder are not collapsed or degenerate.","section":"§3.3, Eq. (11) and Figure 6"}],"minor_comments":[{"comment":"The transformer is repeatedly referred to as the 'casual V AR Transformer' (e.g., in §3.1 and Figure 1 caption); this should be 'causal'.","section":"Throughout"},{"comment":"The text-conditional model name is misspelled as 'Infininty' in the footnote in the Related Works section; it should be 'Infinity'.","section":"§2.2, footnote"},{"comment":"The paper states that 'the discriminator uses the CLIP model as the feature extractor and an MLP as the decoder' but does not specify the loss function (hinge, least-squares, etc.) or the discriminator input resolution; this should be stated in the main text or supplementary.","section":"§4.1, Training details"},{"comment":"The notation in Eq. (5) uses K both as the number of scales in the sum and inside the summand, which is confusing; it should use k as the running index, e.g., sum_{k=1}^K up(R_k, (h,w)).","section":"Eq. (5)"}],"recommendation":"major_revision","confidential_remarks":"The core idea—repurposing a pre-trained VAR model for unpaired translation via prefilling and SRQ—is sound and interesting, and the ablations support the parallel mode. The main weakness is that the 'state-of-the-art' claim is not backed by a controlled comparison: backbone scale is entangled with the method, and no uncertainty quantification is provided. I would encourage the editor to require the authors to add matched-backbone comparisons, error bars, and an ablation of the mixing weight before considering acceptance."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear X,\n\nThe thing to know: CycleVAR is the first credible attempt I've seen to repurpose a pretrained visual autoregressive model (VAR/Infinity) for unpaired image translation. The core trick is multi-scale prefix conditioning plus a softmax relaxation of the codebook lookup, which lets gradients flow through the quantizer. The parallel one-step decoding mode is simple and clearly better than serial in their ablations. If the approach holds up, it gives the LLM-style image model family an unsupervised editing capability it didn't have.\n\nWhat's genuinely good: the framing is clean, the ablations isolate the important design choices (parallel vs serial, multi-scale context, temperature), and the experiments cover both a small class-conditional VAR and a larger text-conditional Infinity. The qualitative results look real, not cherry-picked. The paper is readable and the supplementary includes algorithm pseudocode and a small inference-time table.\n\nWhere it's soft: the headline comparison to CycleGAN-Turbo does not support the word 'surpasses.' The FID margins are 0.7–4 points in the winning rows, with no seeds or error bars; margins that size are usually within run-to-run noise in unpaired translation. More importantly, the winning rows use Infinity-2B (2.3B params), while CycleGAN-Turbo is built on SD-Turbo 2.1 (~1.1B). The matched smaller backbone, VAR-310M, is actually worse than CycleGAN-Turbo on horse→zebra FID (47.8 vs 41.0), and day/night results for that backbone are omitted because of class-conditional ImageNet limits. So the advantage may reflect scale, not the method. The speed advantage (0.03s) also applies only to the smaller variant; the Infinity version is slower than CycleGAN-Turbo (0.11s vs 0.08s).\n\nAlso, SRQ is essentially a temperature-scaled softmax, i.e., Gumbel-Softmax without the noise; presenting it as a novel contribution is a stretch. The paper should cite Jang et al. and maybe the VQ straight-through literature. The supplementary promises training details that aren't really there: no loss weights, learning rate, batch size, or iteration counts.\n\nThe frozen-decoder-off-manifold worry is less serious than the comparison problem. The temperature sweep (Table 5) shows soft decoding works across a wide range (τ=0.7–10), so the decoder seems to tolerate codebook mixtures.\n\nBottom line: it's a solid empirical paper with a real new capability, but the SOTA claim needs to be reworked with matched backbones, error bars, and an honest discussion of inference speed. I'd send it to review, not desk-reject it. The right referee can push the authors to nail down the comparison.","headline":"A plausible first use of pretrained AR image models for unpaired translation, but the SOTA claim is undercut by unmatched backbones and missing error bars.","tokens_in":16488,"tokens_out":4184,"would_cite":false,"duration_ms":37828,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper claims that an autoregressive next-scale image model can be fine-tuned without paired data to translate images across domains, and that a parallel one-step decoding mode beats both serial decoding and a leading one-step…","keywords":["unsupervised image translation","autoregressive image generation","visual autoregressive model","softmax relaxed quantization","vector quantization","cycle consistency","one-step inference","image-to-image translation"],"falsifier":"Train CycleVAR on horse↔zebra with SRQ replaced by straight-through hard quantization (or with $\\tau=0.01$) while keeping everything else fixed, and measure both FID and the gradient norm flowing through the frozen VAE decoder; if translation quality does not collapse and gradients remain healthy, the paper's central claim that soft relaxation is what enables end-to-end unsupervised training is not supported.","tokens_in":15447,"feed_emoji":"🎨","tokens_out":7785,"duration_ms":76756,"temperature":0.7,"pith_summary":"CycleVAR tries to show that a pretrained autoregressive image-generation model can do unpaired image-to-image translation—no paired source/target images—by treating the source image as a multi-scale token prefix and predicting the target image as its continuation. The paper identifies the hard vector-quantization argmax as the blocker, because it cuts gradients between the VAE decoder and the transformer; its fix, Softmax Relaxed Quantization, replaces codebook selection with a softmax-weighted average of codebook vectors so image-space losses can train the model end-to-end. Two decoding modes are proposed, serial multi-step and parallel one-step, and the paper reports that the one-step mode gives better translation quality and faster inference. If the paper is right, autoregressive visual models can be fine-tuned with the same kind of unpaired data used by CycleGAN-style methods while keeping a single forward pass at inference.","feed_headline":"Autoregressive model translates images without paired data","feed_subtitle":"Softmax-relaxed tokens let a next-scale predictor swap domains in a single pass, beating CycleGAN-Turbo.","key_machinery":"The load-bearing object is Softmax Relaxed Quantization (SRQ), a continuous relaxation of the VQ codebook lookup. Instead of computing $\\arg\\max$ over codebook logits and looking up a single code vector, SRQ applies $\\hat{g}_v = \\exp(\\tilde{g}_v/\\tau)/\\sum_q \\exp(\\tilde{g}_q/\\tau)$ and outputs $\\hat{f} = \\sum_v \\hat{g}_v Z_v$, so the quantized feature is differentiable in the logits. This gives the frozen VAE decoder smooth inputs and gives image-space losses a gradient path into the transformer, which is what makes adversarial training possible despite discrete tokenization. The second mechanism is multi-scale token prefilling: the source image's residual maps $F_1,\\dots,F_K$ are fed as contextual keys, so translation becomes next-scale prediction conditioned on the source, in the same spirit as a prefix in an autoregressive language model.","core_discovery":"The paper claims that image-to-image translation can be recast as image-conditional visual autoregressive generation, and that a pretrained next-scale transformer can be fine-tuned for it with no paired data. The source image is tokenized into multi-scale residual maps that are prefilled as contextual keys, like a prefix prompt in a language model, and the transformer predicts the target image's tokens at all scales. Because the usual vector-quantization argmax blocks gradients, the paper replaces it with Softmax Relaxed Quantization: logits over the codebook are turned into a softmax distribution at temperature $\\tau$, and the quantized feature is the convex combination $\\sum_v \\hat{g}_v Z_v$ of codebook vectors. With that differentiable path through the frozen VAE decoder, cycle-consistency, adversarial, and identity losses can be applied directly in image space. On horse↔zebra, day↔night, and an anime-scene dataset, the paper reports that the parallel one-step mode beats its own serial multi-step mode and, with the larger Infinity-2B model, matches or beats CycleGAN-Turbo on FID and DINO structure in all four translation directions.","pith_inferences":["If SRQ generalizes, the same soft relaxation should unlock image-space supervision for other discrete latent models, such as tokenizers based on finite scalar quantization or binary codes, without retraining the tokenizer.","The result that one parallel pass beats iterative scale-by-scale refinement suggests that when the source image already provides the content, the autoregressive prior may be bypassed entirely; this is worth testing on text-to-image and super-resolution tasks where conditioning is similarly content-rich.","Because temperature $\\tau=2$ outperformed both sharper and much softer distributions, a natural extension is to anneal $\\tau$ during training or make it per-scale, which the paper does not explore.","The reported speed advantage should grow at higher resolutions and larger models, since serial decoding cost scales with the number of scales while the parallel mode does not; measuring this on 512×512 or 1024×1024 translations would be a direct stress test."],"forward_implications":["A pretrained autoregressive image generator can be adapted to a new visual domain with unpaired images only, using cycle, adversarial, and identity losses, without ever supervising the predicted token indices.","The parallel one-step decoding mode is preferable for unsupervised translation: it produces better FID and structure scores than serial multi-step decoding while using roughly a third of the inference time.","The approach carries over to text-conditional and bitwise-quantized autoregressive models, suggesting it is not tied to VAR's specific residual quantizer.","Removing either the multi-scale context or the multi-scale output degrades translation, so the coarse-to-fine token structure is doing essential work in preserving source structure.","On the reported benchmarks, CycleVAR with Infinity-2B matches or beats CycleGAN-Turbo on both distribution matching and structure preservation across horse↔zebra and day↔night."],"supporting_citations":[{"why":"Supplies the pretrained next-scale autoregressive architecture (VAR) that CycleVAR fine-tunes.","marker":"[45]"},{"why":"Provides the larger text-conditional Infinity model used for the main comparisons and shows the approach extends to bitwise quantizers.","marker":"[8]"},{"why":"Defines the one-step diffusion translation baseline and the unpaired training losses (cycle, GAN, identity) that CycleVAR adopts.","marker":"[30]"},{"why":"Introduces VQ-VAE-style discrete quantization whose argmax gradient truncation is the problem SRQ is designed to solve.","marker":"[50]"},{"why":"Defines the horse↔zebra unpaired dataset protocol and cycle-consistency objective that later methods standardize.","marker":"[61]"},{"why":"Supplies the anime scene dataset and the user-study protocol used to evaluate style, content, and overall preference.","marker":"[11]"}],"fun_headline_variants":["Autoregressive model performs one-step unsupervised image translation","Softmax-relaxed tokens unlock unsupervised one-pass image translation","Pretrained next-scale transformer translates images without paired data","CycleVAR: single-pass translation surpasses CycleGAN-Turbo","CycleVAR: one-step unsupervised translation beats CycleGAN-Turbo"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole training loop assumes the frozen VAE decoder, trained on single one-hot codebook vectors, produces meaningful images and useful gradients when fed softmax-weighted averages of codebook vectors blended across scales; if those soft mixtures decode to blurry or degenerate latents, the translation and its learning signal both fail.","fun_headline_variants_meta":{"raw":{"variants":["Autoregressive model performs one-step unsupervised image translation","Softmax-relaxed tokens unlock unsupervised one-pass image translation","Pretrained next-scale transformer translates images without paired data","CycleVAR: single-pass translation surpasses CycleGAN-Turbo","CycleVAR: one-step unsupervised translation beats CycleGAN-Turbo"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000238,"raw_usage":{"total_tokens":1560,"prompt_tokens":1042,"completion_tokens":518,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":658,"completion_tokens_details":{"reasoning_tokens":434}},"tokens_in":658,"tokens_out":518,"duration_ms":5065,"temperature":1.0,"reasoning_tokens":434,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T21:44:48.816952+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train CycleVAR on horse↔zebra with SRQ replaced by straight-through hard quantization (or with $\\tau=0.01$) while keeping everything else fixed, and measure both FID and the gradient norm flowing through the frozen VAE decoder; if translation quality does not collapse and gradients remain healthy, the paper's central claim that soft relaxation is what enables end-to-end unsupervised training is not supported.","supporting_citations":[{"cited_title":"Unpaired image-to-image translation using cycle- consistent adversarial networks","cited_arxiv_id":null,"evidence_quote":"Defines the horse↔zebra unpaired dataset protocol and cycle-consistency objective that later methods standardize."},{"cited_title":"Scenimefy: Learning to craft anime scene via semi- supervised image-to-image translation","cited_arxiv_id":null,"evidence_quote":"Supplies the anime scene dataset and the user-study protocol used to evaluate style, content, and overall preference."}],"review_version":1}