Pith. sign in

REVIEW 3 major objections 5 minor 19 references

Paired and Unpaired Image to Image Translation using Generative Adversarial Networks

T0 review · 3 major / 5 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read The paper's central claim is that paired image-to-image translation with a conditional GAN and $L_1$ loss beats unpaired cycle-consistent translation on the facades, maps, cityscapes, and horse-to-zebra datasets.

desk verdict A clean but non-novel ablation of Pix2Pix and CycleGAN whose conclusion matches prior work, yet the evaluation is single-run and the precision/recall equation has a quantifier error. read the letter →

arxiv 2505.16310 v1 pith:F76UEYNK submitted 2025-05-22 cs.CV eess.IV

Gaurav Kumar , Soham Satyadharma , Harpreet Singh This is my paper · ORCID
classification cs.CVeess.IV
keywords image-to-imagetranslationgenerativeadversarialnetworksconditionalGANCycleconsistencyPatchFIDscorepairedvsunpaired
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper is trying to establish that when ground-truth image pairs are available, a conditional GAN (Pix2Pix) with an added $L_1$ loss produces better image-to-image translations than unpaired cycle-consistency training (CycleGAN), and that this ordering holds across the facades, maps, cityscapes, and horse-to-zebra datasets. It also claims that $L_1$ is the most reliable reconstruction loss, beating $L_2$ and a 50/50 mix, and that smaller PatchGAN discriminators yield sharper edges than larger ones. These results matter because they give practitioners a concrete default configuration for translation tasks, and they show that quantitative metrics (FID plus GAN precision and recall) line up with visual inspection.

What carries the argument

The load-bearing objects are the conditional GAN with a PatchGAN discriminator and the cycle-consistency loss. Pix2Pix pairs a U-Net generator with skip connections against a Markovian discriminator that classifies image patches as real or fake; its objective combines adversarial loss with a weighted $L_1$ loss between generated and ground-truth images. CycleGAN instead uses two such generator/discriminator pairs plus an $L_1$ cycle-consistency loss that requires each image to survive a round trip through both generators. The paper ablates this machinery by swapping $L_1$ for $L_2$ or a convex combination, changing PatchGAN size from 16 to 286, and removing skip connections, and attributes the observed quality differences to those components.

What would settle it

Recompute FID and precision and recall for each configuration from ten independently trained models per dataset; if the $L_1$ experiment does not beat the $L_2$ experiment on every dataset, or if paired and unpaired FID ranges overlap, the paper's ordering claim is falsified.

Watch

Extended reading notes

Core claim

The paper's central claim is that paired translation is more effective than unpaired translation, using Pix2Pix as the paired framework and CycleGAN as the unpaired framework, and that incorporating $L_1$ loss consistently improves image quality across domains. The evidence is a set of FID, precision, and recall tables: on every paired dataset the $L_1$ experiment had the lowest FID among the loss variants, while the skip-connection-free U-Net collapsed, with FID above 275 on all datasets. For unpaired translation, only FID is reported, and the $L_1$ cyclic loss again gives the lowest score on each dataset, matching the qualitative inspection that paired outputs generally look better than unpaired outputs.

Load-bearing premise

The comparison rests on the assumption that 256 generated images from a single training run, with hyperparameters such as $\lambda=100$ chosen without a separate validation split, are enough to tell genuine quality differences from run-to-run noise.

Editorial extensions

If this is right

  • For any paired translation task, a conditional GAN with $L_1$ loss is the configuration to try first; the paper's tables place it ahead of $L_2$ and mixed losses on all four datasets.
  • Unpaired translation should use $L_1$ cycle consistency rather than $L_2$; the reported FID scores favor it on maps, facades, and horse-to-zebra.
  • Removing skip connections from the generator is catastrophic, roughly tripling FID, so skip connections are doing essential work in this architecture.
  • Smaller PatchGAN discriminators (16) give sharper, higher-contrast outputs than larger ones (286), giving a cheap lever for visual quality.
  • Quantitative metrics and human inspection agree on the ranking, so FID plus precision and recall can serve as a screening tool for translation-model choices.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A reader who wants to act on this result should treat the ranking as provisional: every number comes from 256 generated images from a single run, so the ordering could shift with more samples or different seeds.
  • The paper never reports precision and recall for the unpaired experiments, so the paired-over-unpaired conclusion rests on FID comparisons; a full metric matrix would make the claim easier to trust.
  • One natural extension is to test whether the same $L_1$ advantage holds against perceptual losses or newer GAN losses, since the current metrics all use InceptionV3 features.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents an empirical comparison of paired image-to-image translation with Pix2Pix-style conditional GANs and unpaired translation with CycleGAN-style cycle-consistent GANs on four public datasets (facades, maps, cityscapes, horse-to-zebra). The experiments vary the reconstruction loss (L1, L2, and a convex combination), PatchGAN receptive-field size, and the presence of U-Net skip connections. Quantitative evaluation uses FID, precision, and recall computed on 256 generated images per configuration, supplemented by qualitative inspection. The central claims are that paired translation outperforms unpaired translation across the tested datasets and that incorporating L1 loss consistently improves image quality.

Significance. If the empirical claims are reliable, the paper would provide a useful benchmark-style confirmation of known results in image-to-image translation, with the practical value of comparing loss choices and PatchGAN sizes under a common training protocol. Strengths of the manuscript include the use of standard public datasets, standard quantitative metrics (FID, precision, recall), a clear description of training details, and complete result tables for all reported variants. However, the contribution is primarily a reproduction and comparison of existing methods rather than a new architecture or algorithm, and the absence of repeated runs, error bars, and a documented validation protocol substantially limits the strength of the conclusions that can be drawn from the tables. The paper is suitable for reconsideration after revision, but the central comparative claims currently rest on single-run, small-sample evaluations.

major comments (3)
  1. [§6, Tables 1–2] The central comparative conclusion in Section 8 — that paired translation is superior and that L1 loss consistently improves quality — is based on FID, precision, and recall computed on 256 generated images from a single training run per configuration, with no error bars, repeated seeds, or bootstrap confidence intervals. GAN training is highly stochastic, and 256 feature vectors are a small sample for FID and especially for k-NN-based precision/recall. For example, the Cityscapes L1-vs-L2 FID gap (100.52 vs 115.31) and the Facades L1-vs-Patch16 gap (110.68 vs 116.50) may be within run-to-run variation. Please run at least 3–5 seeds per configuration, report mean and standard deviation for every metric, and verify that the rankings asserted in Section 8 are stable across seeds.
  2. [§6.1, Eq. (7)] Equation (7) defines f(phi, Phi) = 1 only if the norm condition holds for all phi' in Phi, but the text immediately below states that a generated point is realistic if it falls inside the hypersphere of any phi' in Phi_r, and the standard Kynkaanniemi metric requires existence of at least one such neighbor. The universal-quantifier form would make precision and recall nearly zero on high-dimensional feature sets, which is inconsistent with the nonzero values reported in Table 1. This suggests the implemented form differs from the printed equation. Please correct Eq. (7) to use existential quantification (or state the implementation precisely) and confirm which form produced the reported numbers.
  3. [§5.1] The text says lambda=100 gave the 'best results' but does not describe a held-out validation split. Since Table 1 reports test-set metrics, selecting lambda on the same data could bias the reported advantage of the L1-loss configuration and make the comparison with other losses unfair. Please specify the validation procedure used for hyperparameter selection, state whether the final numbers in Tables 1–2 come from configurations selected without access to the test set, and provide the same training/evaluation protocol for all compared variants.
minor comments (5)
  1. [Table 1] In the Skip row, the Facades FID is reported as 275.91, identical to the Maps FID; this is likely a copy/paste error and should be verified.
  2. [§4.2, Eq. (6)] Equation (6) introduces lambda for the cycle-consistency loss but does not state its value for the unpaired experiments; please specify whether the same lambda=100 was used as in the paired task.
  3. [Figure 2 caption] The caption says the figure shows unpaired L1 cyclic loss experiments on facades and horse2zebra datasets, but the subcaptions list Facades and Maps; please align the caption with the displayed images.
  4. [§6 heading] The heading uses 'Frechet'; the correct spelling is 'Fréchet'.
  5. [Abstract] The phrase 'New quantitative metrics' is inaccurate, as FID, precision, and recall are established evaluation metrics; consider saying 'Standard quantitative metrics'.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: empirical benchmark evaluated with external FID, precision, and recall metrics.

full rationale

This paper is an empirical comparison of paired and unpaired image-to-image translation models using Pix2Pix and CycleGAN, evaluated with FID, precision, and recall. These metrics are external and standard: FID follows Heusel et al. and precision/recall follow Kynkaanniemi et al., both computed using a public pre-trained InceptionV3 network. No quantity is fitted to the reported outcome and then renamed as a prediction; the lambda=100 choice is conventional hyperparameter selection, not a fitted input called prediction. The paper does not derive any theoretical result from its own equations, and it makes no use of self-citations as load-bearing evidence. The comparative claims about L1 loss and paired versus unpaired translation could be false without affecting the definitions of the metrics, so the evaluation chain is not self-referential. Concerns such as the absence of repeated seeds, the small evaluation set, and the apparent universal quantifier typo in Eq. 7 are reproducibility or correctness issues, not circularity.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

No new theory or entity is introduced. The ledger records the hand-set hyperparameters and domain assumptions that the empirical comparison depends on. The central claims rely on standard GAN training heuristics, correct dataset labeling, and the validity of InceptionV3-based metrics; none of these are derived in the paper.

free parameters (6)
  • lambda loss weight = 100
    Section 5.1: authors tried different values of lambda and chose 100 because it gave "best results". This selection is not made on a separate validation set.
  • Optimizer learning rate = 2e-4
    Section 5.1: Adam learning rate, a standard GAN training choice, selected by hand.
  • Adam beta1 = 0.5
    Section 5.1: beta1 set to 0.5, a standard GAN training heuristic.
  • Batch size = 16
    Section 5.1 and 5.2: all models trained with batch size 16 for 150 epochs.
  • Training epochs = 150
    Section 5.1: all models trained for 150 epochs with no stated early stopping or convergence criterion.
  • Evaluation sample count = 256
    Section 6: "256 generated images for each experiment" chosen without justification or variance estimate.
assumptions (5)
  • domain assumption Standard GAN training stability heuristics are sufficient.
    Section 5.1 divides the discriminator loss by 2 and uses Adam with beta1=0.5; these are heuristics with no convergence proof.
  • domain assumption Dataset labels and domain memberships are correct.
    Section 3 assumes CMP facades, maps, cityscapes, and horse2zebra datasets are correctly curated for their translation tasks.
  • domain assumption InceptionV3 features capture perceptually meaningful image quality.
    Section 6 uses InceptionV3 embeddings for FID and precision/recall, assuming the pretrained network is a valid perceptual metric.
  • domain assumption 256 generated images are representative for metric estimation.
    Section 6 evaluates each model on 256 images with no bootstrap, repeated sampling, or confidence intervals.
  • domain assumption Random flip and jitter preprocessing improves generalization.
    Section 3.1 applies random flipping and jittering to all datasets; the paper assumes this helps rather than distorts the task.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Paired and Unpaired Image to Image Translation using Generative Adversarial Networks." pith.science (2026). https://pith.science/paper/F76UEYNK

@misc{pith2026250516310,
  author       = {Pith},
  title        = {Pith review of: Paired and Unpaired Image to Image Translation using Generative Adversarial Networks},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/F76UEYNK}},
  note         = {Machine review of arXiv:2505.16310}
}
read the original abstract

Image to image translation is an active area of research in the field of computer vision, enabling the generation of new images with different styles, textures, or resolutions while preserving their characteristic properties. Recent architectures leverage Generative Adversarial Networks (GANs) to transform input images from one domain to another. In this work, we focus on the study of both paired and unpaired image translation across multiple image domains. For the paired task, we used a conditional GAN model, and for the unpaired task, we trained it using cycle consistency loss. We experimented with different types of loss functions, multiple Patch-GAN sizes, and model architectures. New quantitative metrics - precision, recall, and FID score - were used for analysis. In addition, a qualitative study of the results of different experiments was conducted.

Figures

Figures reproduced from arXiv: 2505.16310 by the authors.

Figure 3
Figure 3. Loss curves for the L1 loss experiment on the maps [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figure 1
Figure 1. From top to bottom: Input, ground truth, and generated [PITH_FULL_IMAGE:figures/full_fig_p006_1.png] view at source ↗
Figure 4
Figure 4. Generation results using different methods. From left to [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

19 extracted references · 13 canonical work pages

  1. [1]

    The cityscapes dataset for semantic urban scene understanding

    Marius Cordts, Mohamed Omran, Sebastian Ramos, Timo Rehfeld, Markus Enzweiler, Rodrigo Benenson, Uwe Franke, Stefan Roth, and Bernt Schiele. The cityscapes dataset for semantic urban scene understanding. InProc. of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2016. 2

    work page 2016

  2. [2]

    Deblurgan-c: image restoration using gan and a correntropy based loss function in degraded visual environments

    Dennis Estrada, Susanne Lee, Fraser Dalgleish, Casey Den Ouden, Madison Young, Caitlin Smith, Joseph Des- jardins, and Bing Ouyang. Deblurgan-c: image restoration using gan and a correntropy based loss function in degraded visual environments. InBig Data II: Learning, Analytics, and Applications, page 1139507. International Society for Optics and Photonic...

    work page 2020

  3. [3]

    Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio

    Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial networks, 2014. 1

    2014

  4. [4]

    Image analogies

    Aaron Hertzmann, Charles E Jacobs, Nuria Oliver, Brian Curless, and David H Salesin. Image analogies. InProceed- ings of the 28th annual conference on Computer graphics and interactive techniques, pages 327–340, 2001. 1

    work page 2001

  5. [5]

    Gans trained by a two time-scale update rule converge to a local nash equilib- rium, 2018

    Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilib- rium, 2018. 4

    work page 2018

  6. [6]

    Image-to-image translation with conditional adver- sarial networks

    Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adver- sarial networks. InProceedings of the IEEE conference on computer vision and pattern recognition, pages 1125–1134,

  7. [7]

    Improved precision and recall metric for assessing generative models.arXiv preprint arXiv:1904.06991, 2019

    Tuomas Kynk ¨a¨anniemi, Tero Karras, Samuli Laine, Jaakko Lehtinen, and Timo Aila. Improved precision and recall metric for assessing generative models.arXiv preprint arXiv:1904.06991, 2019. 3

    arXiv 1904

  8. [8]

    Learning representations for automatic colorization

    Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. Learning representations for automatic colorization. InEuropean conference on computer vision, pages 577–593. Springer, 2016. 1

    work page 2016

Show all 19 references
  1. [9]

    Coupled generative adversar- ial networks.arXiv preprint arXiv:1606.07536, 2016

    Ming-Yu Liu and Oncel Tuzel. Coupled generative adversar- ial networks.arXiv preprint arXiv:1606.07536, 2016. 1

    2016 arXiv

  2. [10]

    Unsuper- vised image-to-image translation networks.arXiv preprint arXiv:1703.00848, 2017

    Ming-Yu Liu, Thomas Breuel, and Jan Kautz. Unsuper- vised image-to-image translation networks.arXiv preprint arXiv:1703.00848, 2017. 2

    2017 arXiv

  3. [11]

    Fully convolutional networks for semantic segmentation

    Jonathan Long, Evan Shelhamer, and Trevor Darrell. Fully convolutional networks for semantic segmentation. InPro- ceedings of the IEEE conference on computer vision and pat- tern recognition, pages 3431–3440, 2015. 1

    2015

  4. [12]

    Image colorization using generative adversarial networks.Lecture Notes in Computer Science, page 85–94, 2018

    Kamyar Nazeri, Eric Ng, and Mehran Ebrahimi. Image colorization using generative adversarial networks.Lecture Notes in Computer Science, page 85–94, 2018. 1

    2018

  5. [13]

    U- net: Convolutional networks for biomedical image segmen- tation

    Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U- net: Convolutional networks for biomedical image segmen- tation. InInternational Conference on Medical image com- puting and computer-assisted intervention, pages 234–241. Springer, 2015. 2, 3

    2015

  6. [14]

    Imagenet large scale visual recognition challenge.International journal of computer vision, 115(3):211–252, 2015

    Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, San- jeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael Bernstein, et al. Imagenet large scale visual recognition challenge.International journal of computer vision, 115(3):211–252, 2015. 2

    2015

  7. [15]

    Rethinking the inception archi- tecture for computer vision

    Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. Rethinking the inception archi- tecture for computer vision. InProceedings of the IEEE con- ference on computer vision and pattern recognition, pages 2818–2826, 2016. 4

    2016

  8. [16]

    Spatial pattern templates for recognition of objects with regular structure

    Radim Tyle ˇcek and Radim ˇS´ara. Spatial pattern templates for recognition of objects with regular structure. InProc. GCPR, Saarbrucken, Germany, 2013. 2

    2013

  9. [17]

    Holistically-nested edge de- tection

    Saining Xie and Zhuowen Tu. Holistically-nested edge de- tection. InProceedings of the IEEE international conference on computer vision, pages 1395–1403, 2015. 1

    2015

  10. [18]

    Colorful image colorization

    Richard Zhang, Phillip Isola, and Alexei A Efros. Colorful image colorization. InEuropean conference on computer vision, pages 649–666. Springer, 2016. 1

    2016

  11. [19]

    Unpaired image-to-image translation using cycle- consistent adversarial networks

    Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle- consistent adversarial networks. InProceedings of the IEEE international conference on computer vision, pages 2223– 2232, 2017. 2, 3 A. Appendix This section showcase...

    2017

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.