Pith. sign in

REVIEW 4 major objections 5 minor 27 references

Fast-Slow Co-advancing Optimizer: Toward Harmonious Adversarial Training of GAN

T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash

Pith's one-line read Reinforcement learning sets the discriminator's step size to keep GAN training stable

desk verdict A plausible but unsupported idea for RL-controlled GAN learning rates; no baselines, circular reward, and admitted failures make it not ready for review. read the letter →

arxiv 2504.15099 v1 pith:TX7I56OX submitted 2025-04-21 cs.LG cs.AI

classification cs.LGcs.AI
keywords generativeadversarialnetworksDDPGreinforcementlearningratecontroltrainingstabilitymodecollapsehyperparametersensitivitystep-sizeoptimization
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper argues that GAN training instability comes largely from fixed, manually chosen learning-rate step sizes, and proposes to replace the discriminator's fixed step size with a dynamically scaled one controlled by a reinforcement-learning agent. The new optimizer, FSCO, wraps the DDPG algorithm around the GAN: after each update the agent observes the training state and outputs a multiplier in [0,1] that shrinks the discriminator's base learning rate whenever the generator is falling behind. On MNIST, ANIME, and Ganyu datasets the authors report that the training process becomes a stable, self-correcting feedback loop and that the usable range of discriminator step sizes is enlarged. If true, this would reduce the manual hyperparameter debugging that currently dominates GAN practice.

What carries the argument

The central object is the FSCO control loop: a DDPG agent whose single continuous action scales the discriminator's learning rate. The loop is driven by the reward $-|G_{loss}(t)-D_{loss}(t)|$, which encodes the paper's notion of a harmonious game: when the discriminator's loss is high it speeds up, and when the generator's loss is high the agent suppresses the discriminator. This machinery converts the existing two-player gradient descent into a closed-loop system in which the discriminator 'waits' for the generator, and it is what the paper claims enlarges the tolerable range of step-size hyperparameters.

What would settle it

Train a GAN with FSCO while recording both the reward and the quality of generated images; if there exists a stage where the gap $|G_{loss}-D_{loss}|$ is driven to zero while the generator collapses or produces repetitive samples, the reward is not a sufficient surrogate. A cleaner test would hold all other hyperparameters fixed and compare the distribution of successful final models over a grid of discriminator base step sizes, with and without FSCO: if FSCO does not measurably widen the success region, the claim of reduced step-size sensitivity fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that adversarial training can be converted from a manually tuned game into a feedback-controlled process by letting a DDPG agent regulate the discriminator's step size. The action $u(t)\in(0,1)$ multiplies the fixed discriminator step size $\eta_D(t)$ to produce $\eta_{FSC\!O-D}(t)=\eta_D(t)u(t)$, and the reward $R(t)=-|G_{loss}(t)-D_{loss}(t)|$ pushes the two losses toward each other. The authors report that this prevents the discriminator from racing ahead, gives the generator time to learn, and keeps training stable across three datasets with different resolutions and variances. They also report limitations: later-stage overfitting appeared in all experiments, and a 512×512 setup failed because the discriminator could not learn features at all.

Load-bearing premise

The load-bearing premise is that matching the generator's and discriminator's losses measures healthy progress: if that reward is not aligned with generation quality, the entire control loop is optimizing the wrong objective.

Editorial extensions

If this is right

  • If FSCO works as claimed, GAN users could start from a wider range of discriminator step sizes and rely on the agent to adjust them, cutting down manual tuning time.
  • The same control loop could be applied to other two-player training setups, such as actor-critic pairs or adversarial defenses, where one network's learning rate needs to be throttled relative to the other's.
  • The paper's own results imply that FSCO does not remove overfitting; later training stages still degrade, so the controller postpones but does not fully solve collapse.
  • The reward design suggests that pairing the feedback signal of a controller with the balance of the two players' losses, rather than with a single network's loss, is a workable heuristic for stabilizing adversarial games.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the reward is a poor proxy, one testable extension is to replace it with a quality-aware signal such as a held-out inception score or a discriminator-confidence target, and to check whether the agent then learns a different policy.
  • The paper's observation that the learned step size concentrates near zero or in $[0.05\eta_D,0.9\eta_D]$ suggests the DDPG policy may be learning a hysteresis rule that could be distilled into a closed-form scheduler.
  • Because the state vector is never specified, a direct ablation of state features would reveal whether the agent actually uses loss information or is effectively acting on a nearly constant signal.
  • If the 512×512 failure reflects a limit of the control signal rather than the architecture, then FSCO's benefit may shrink as model capacity and data complexity grow, which a scaling study could test.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes Fast-Slow Co-advancing Optimizer (FSCO), which wraps a DCGAN training loop with a DDPG agent that modulates the discriminator learning rate. The action u(t) in [0,1] multiplies a fixed discriminator base learning rate (Eq. (2)), and the reward is the negative absolute difference between generator and discriminator losses (Eq. (3)). Experiments on MNIST, ANIME, and Ganyu report loss curves, learning-rate traces, and generated image samples. The paper claims that FSCO makes GAN training less sensitive to step-size hyperparameters and transforms the training process into a feedback-controlled, harmonious adversarial game. The manuscript also acknowledges severe overfitting, unstable image quality, and a failed 512x512 experiment.

Significance. If rigorously validated, an RL-based controller that widens the usable learning-rate range for GAN discriminators would be a useful contribution, particularly because existing learned optimizers mostly target single-model training. The paper's conceptual framing is clear, and it correctly identifies a practical pain point. However, the empirical validation is not at the standard required to support the central claim: there are no baselines, no quantitative image-quality metrics, no seed or compute-matched comparisons, and the one quantitative-looking measure (loss balancing) is the exact quantity the reward optimizes. The undefined state representation compounds the reproducibility problem. As presented, the evidence does not support the claimed enlargement of the usable hyperparameter range.

major comments (4)
  1. [§3.3.2, Eq. (3)] The reward Reward(t) = -|G_loss(t) - D_loss(t)| is the very quantity used to evaluate success in Figures 6, 8, and 10, where the discriminator and generator losses approach each other. Because the action u(t) in Eq. (2) lies in [0,1], the agent can trivially drive this reward to zero by setting u(t) approximately 0 whenever G_loss exceeds D_loss, and the near-zero discriminator learning rates in the figures are consistent with that trivial behavior. The paper provides no evidence that minimizing the raw loss gap is a valid proxy for generation quality, training stability, or mode-collapse avoidance, so the central mechanism is not established.
  2. [Tables 1–3] All three hyperparameter tables declare 'State dimension 6' for the DDPG agent, but the six components of the state vector are never defined anywhere in the manuscript. Since the learned control policy is the paper's contribution, an undefined state representation makes the method irreproducible and prevents the reader from assessing whether the agent has access to sufficient information.
  3. [§4.1–§4.3] The experimental section contains no fixed-learning-rate baseline, no comparison to standard GAN training without FSCO, no quantitative metric such as FID or Inception Score, no repeated runs, and no standard deviation or error bars. The only quantitative evidence is the loss-gap behavior, which is the optimized reward. Consequently, the claim that FSCO enlarges the usable range of discriminator/generator step sizes is not supported by the presented experiments.
  4. [§4.4] The discussion explicitly states that image quality is unstable, that generator effectiveness in later training is uncertain, that overfitting is 'quite severe', and that a 512x512 experiment failed because the discriminator could not learn features. These admissions directly undercut the central claims of improved stability and reduced sensitivity to step sizes, and the manuscript does not provide a systematic hyperparameter sweep or a quantitative failure analysis that would clarify the conditions under which FSCO helps.
minor comments (5)
  1. [Title/front matter] The title contains clear typos ('FAST-S LOW', 'T OWARD'), and Figure 3 contains 'Enviorment' and 'updata'; the manuscript needs a careful proofreading pass.
  2. [§3.3.1] The first two paragraphs of Section 3.3.1 are duplicated verbatim; one copy should be removed.
  3. [§3.3.2] The text refers to 'Section 2.3.3' when describing the gaming process, but this section does not exist; the correct reference appears to be Section 3.3.3.
  4. [§4.4] The sentence 'the step size in the FSCO method should be mostly distributed in the range [0, η_FSCO−D(t)]' appears to contain a self-referential notation error; it should presumably refer to [0, η_D(t)].
  5. [§2.1] The citation labeled 'Zhang et al.' is attributed to a reference by Tang et al.; the in-text name and the reference entry should be made consistent.

Circularity Check

1 steps flagged · score 6.0 of 10

The primary evidence for FSCO is the reward objective itself: Eq. (3) rewards |G_loss − D_loss| → 0, and Figures 6, 8, and 10 show that gap closing, so the 'harmonious' result is achieved by construction.

  1. self definitional [Section 3.3.2, Eq. (3); Section 4.1.2, Fig. 6]
    "As depicted in the right side of Figure 3, the DDPG reward encourages closer losses of the discriminator and the generator through using the min-max loss function. Reward(t) =−|Gloss(t)−Dloss(t))| (3) This reward Equation in (3) serves two purposes: first, if the discriminator’s loss(Dloss) is large, it can increase discriminator’s step size to accelerate discriminator learning; second, if the generator’s loss(Gloss) is too large, it can reduce discriminator’s step size to give the generator sufficient steps for learning."

    The paper's success criterion—a 'harmonious adversarial game'—is operationalized as the generator and discriminator losses being close, which is exactly the quantity maximized by the reward in Eq. (3). The DDPG agent is trained to minimize |G_loss − D_loss|, and the experimental evidence in Figures 6, 8, and 10 is that this gap shrinks while the discriminator learning rate is driven toward zero via Eq. (2). Thus the central evidence of effectiveness reduces to the reward definition itself, not to an independent measure of generation quality or of an enlarged usable step-size range.

full rationale

The paper uses no self-citation chain or imported uniqueness theorem; the GAN and DDPG background is standard external material. The circularity is concentrated in the reward/evaluation loop. Eq. (3) defines the reward as the negative absolute difference between generator and discriminator losses, and the paper's stated goal is harmonious adversarial training in which these losses stay close. The main experimental evidence (Figures 6, 8, and 10) is that exactly this gap shrinks while Eq. (2) suppresses the discriminator learning rate toward zero. Because the DDPG agent is trained to maximize that reward, the observed loss convergence is the optimization objective itself, not an independent test of generation quality or of an enlarged usable learning-rate range. The paper does show generated images and candidly reports instability, overfitting, and failure at 512×512 in Section 4.4, so there is some independent signal outside the reward loop; nevertheless, the load-bearing 'harmonious' evidence reduces to the reward definition. This warrants a 6 rather than a higher score: the method is not fully circular because the images and admitted failure modes are outside the reward loop, and no self-citation is load-bearing.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The method rests on the assumption that loss difference is a usable control signal for DDPG in a non-stationary GAN environment, plus hand-set learning-rate bounds; these are not derived or validated independently.

free parameters (5)
  • Discriminator base learning rate eta_D = 0.002 (MNIST), 0.005 (ANIME/Ganyu)
    Sets the upper bound of the controlled step size in Eq. (2); the paper admits this value must be chosen carefully (Section 4.4), so the method does not remove sensitivity to this hyperparameter.
  • Generator learning rate eta_G = 0.0002
    Kept fixed in all experiments; the balance with eta_D is part of the manual setup that FSCO is supposed to reduce.
  • DDPG state representation = 6 dimensions, contents not specified
    State is the input to the control policy but its components are never listed, making the controller underspecified and the choice a hand-selected design.
  • Discriminator LR minimum multiplier = 0.001 (64x64 and 128x128)
    Introduced in Tables 2 and 3 to clamp the action; affects behavior but is not derived.
  • DDPG exploration noise = 0.1
    Chosen by hand with no sensitivity analysis; controls how much the agent explores during GAN training.
assumptions (5)
  • domain assumption The difference between generator and discriminator losses is a meaningful and stable control signal.
    Equations (2)-(3) use |G_loss - D_loss| as reward; no scaling or stationarity argument is given, and GAN loss scales can differ between networks.
  • domain assumption DDPG can learn an effective policy in a non-stationary environment where the GAN it controls is also changing.
    The DDPG replay buffer stores transitions from many GAN states; no proof or adaptation mechanism is offered (Section 3.3).
  • domain assumption Minimizing loss difference prevents mode collapse and improves generation quality.
    Section 3.3.2 asserts this motivation; no theoretical or empirical causal evidence is given.
  • ad hoc to paper The state vector of dimension 6 contains sufficient information for the agent.
    State dimension is listed in Tables 1-3 but the components are never defined; the claim depends on this unspecified input.
  • standard math Standard min-max GAN objective and gradient descent updates (Eqs 4-7).
    Background from Goodfellow et al.; used as substrate for the method.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fast-Slow Co-advancing Optimizer: Toward Harmonious Adversarial Training of GAN." pith.science (2026). https://pith.science/paper/TX7I56OX

@misc{pith2026250415099,
  author       = {Pith},
  title        = {Pith review of: Fast-Slow Co-advancing Optimizer: Toward Harmonious Adversarial Training of GAN},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/TX7I56OX}},
  note         = {Machine review of arXiv:2504.15099}
}
read the original abstract

Up to now, the training processes of typical Generative Adversarial Networks (GANs) are still particularly sensitive to data properties and hyperparameters, which may lead to severe oscillations, difficulties in convergence, or even failures to converge, especially when the overall variances of the training sets are large. These phenomena are often attributed to the training characteristics of such networks. Aiming at the problem, this paper develops a new intelligent optimizer, Fast-Slow Co-advancing Optimizer (FSCO), which employs reinforcement learning in the training process of GANs to make training easier. Specifically, this paper allows the training step size to be controlled by an agent to improve training stability, and makes the training process more intelligent with variable learning rates, making GANs less sensitive to step size. Experiments have been conducted on three benchmark datasets to verify the effectiveness of the developed FSCO.

Figures

Figures reproduced from arXiv: 2504.15099 by the authors.

Figure 1
Figure 1. Dialectical thinking guided diagram and even due to the interaction between penalty terms and regularization techniques, excessive regularization may also limit model expressiveness. To this end, this paper develops a novel training method, namely Fast-Slow Co-advancing Optimizer (FSCO), which aims to solve the problem of excessive sensitivity to step size hyperparameters in GAN training. As indicated in [PITH_FULL… view at source ↗
Figure 2
Figure 2. The Structure of GAN intelligent models. Therefore, it is necessary to research reinforcement learning-based step size optimization methods specifically designed for GANs. 3 Algorithm Introduction Since the developed FSCO integrates DDPG as an embedded module of GAN to optimize the training process, this section introduces the basics of GAN and DDPG at first, and then elaborates the details of the developed FSCO. 3.… view at source ↗
Figure 3
Figure 3. The Structure of DDPG [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (8 more)
Figure 4
Figure 4. Figure 4: Harmonious adversarial training diagram in terms of step size [PITH_FULL_IMAGE:figures/full_fig_p006_4.png]
Figure 5
Figure 5. Figure 5: Algorithm diagram of the developed FSCO method [PITH_FULL_IMAGE:figures/full_fig_p008_5.png]
Figure 6
Figure 6. Figure 6: Step sizes and losses along the training process on the MNIST [PITH_FULL_IMAGE:figures/full_fig_p009_6.png]
Figure 7
Figure 7. Figure 7: Generated images in size of 28×28 [PITH_FULL_IMAGE:figures/full_fig_p010_7.png]
Figure 8
Figure 8. Figure 8: Step sizes and losses along the training process on the Anime [PITH_FULL_IMAGE:figures/full_fig_p011_8.png]
Figure 9
Figure 9. Figure 9: Generated images in size of 64×64 11 [PITH_FULL_IMAGE:figures/full_fig_p011_9.png]
Figure 10
Figure 10. Figure 10: Step sizes and losses along the training process on the Ganyu [PITH_FULL_IMAGE:figures/full_fig_p012_10.png]
Figure 11
Figure 11. Figure 11: Generated images in size of 128×128 4.4 Experimental Discussion The experiments conducted were far more than just the above presented, but the differences among these experiments were mostly small. To be specific, through adjusting the discriminator’s fixed learning r…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

27 extracted references · 17 canonical work pages

  1. [1]

    2016, in Advances in Neural Information Processing Systems, 3981--3989

    Andrychowicz, M., Denil, M., Gomez, S., et al. 2016, in Advances in Neural Information Processing Systems, 3981--3989

  2. [2]

    2017, Proceedings of the 34th International Conference on Machine Learning (ICML), 70, 214

    Arjovsky, M., Chintala, S., & Bottou, L. 2017, Proceedings of the 34th International Conference on Machine Learning (ICML), 70, 214. http://proceedings.mlr.press/v70/arjovsky17a.html

  3. [3]

    Bellman, R. E. 1957, Dynamic Programming (Princeton University Press)

  4. [4]

    v., & Meger, D

    Fujimoto, S., Hoof, H. v., & Meger, D. 2018, in Proceedings of Machine Learning Research, Vol. 80, Proceedings of the 35th International Conference on Machine Learning (ICML) (PMLR), 1587--1596. http://proceedings.mlr.press/v80/fujimoto18a.html

  5. [5]

    J., Pouget-Abadie, J., Mirza, M., et al

    Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., et al. 2014, in Advances in Neural Information Processing Systems 27 (NIPS 2014), 2672--2680. https://proceedings.neurips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf

  6. [6]

    Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. C. 2017, in Advances in Neural Information Processing Systems 30 (NIPS 2017), 5767--5777. https://proceedings.neurips.cc/paper/2017/file/892c3b1c6dccd52936e27cbd0ff683d6-Paper.pdf

  7. [7]

    2018, in Proceedings of Machine Learning Research, Vol

    Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. 2018, in Proceedings of Machine Learning Research, Vol. 80, Proceedings of the 35th International Conference on Machine Learning (ICML) (PMLR), 1861--1870. http://proceedings.mlr.press/v80/haarnoja18b.html

  8. [8]

    2019, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 4401--4410, 10.1109/CVPR.2019.00453

    Karras, T., Laine, S., & Aila, T. 2019, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 4401--4410, 10.1109/CVPR.2019.00453

Show all 27 references
  1. [9]

    2020, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 8110--8119, 10.1109/CVPR42600.2020.00813

    Karras, T., Laine, S., Aittala, M., et al. 2020, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 8110--8119, 10.1109/CVPR42600.2020.00813

  2. [10]

    2024, arXiv preprint arXiv:2402.01582

    Lan, Q., Zhang, C., Zhang, W., et al. 2024, arXiv preprint arXiv:2402.01582

  3. [11]

    P., Hunt, J

    Lillicrap, T. P., Hunt, J. J., Pritzel, A., et al. 2015, arXiv preprint arXiv:1509.02971

  4. [12]

    P., Mirza, M., et al

    Mnih, V., Badia, A. P., Mirza, M., et al. 2016, in Proceedings of Machine Learning Research, Vol. 48, Proceedings of the 33rd International Conference on Machine Learning (ICML) (PMLR), 1928--1937. http://proceedings.mlr.press/v48/mnih16.html

  5. [13]

    2013, arXiv preprint arXiv:1312.5602

    Mnih, V., Kavukcuoglu, K., Silver, D., et al. 2013, arXiv preprint arXiv:1312.5602. 1312.5602

  6. [14]

    2015, Nature, 518, 529, 10.1038/nature14236

    ---. 2015, Nature, 518, 529, 10.1038/nature14236

  7. [15]

    Pavlov, I. P. 1927, Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex (Oxford University Press)

  8. [16]

    2015, arXiv preprint arXiv:1511.06434

    Radford, A., Metz, L., & Chintala, S. 2015, arXiv preprint arXiv:1511.06434. 1511.06434

  9. [17]

    2020, Nature, 588, 604, 10.1038/s41586-020-03051-4

    Schrittwieser, J., Antonoglou, I., Hubert, T., et al. 2020, Nature, 588, 604, 10.1038/s41586-020-03051-4

  10. [18]

    I., & Moritz, P

    Schulman, J., Levine, S., Abbeel, P., Jordan, M. I., & Moritz, P. 2015, in Proceedings of Machine Learning Research, Vol. 37, Proceedings of the 32nd International Conference on Machine Learning (ICML) (PMLR), 1889--1897. http://proceedings.mlr.press/v37/schulman15.html

  11. [19]

    2017, arXiv preprint arXiv:1707.06347

    Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. 2017, arXiv preprint arXiv:1707.06347. 1707.06347

  12. [20]

    2014, in Proceedings of Machine Learning Research, Vol

    Silver, D., Lever, G., Heess, N., et al. 2014, in Proceedings of Machine Learning Research, Vol. 32, Proceedings of the 31st International Conference on Machine Learning (ICML) (PMLR), 387--395. http://proceedings.mlr.press/v32/silver14.html

  13. [21]

    P., Pervez, R., et al

    Subramanian, A., Mudumba, S. P., Pervez, R., et al. 2023, arXiv preprint arXiv:2305.15076

  14. [22]

    S., & Barto, A

    Sutton, R. S., & Barto, A. G. 1998, Reinforcement Learning: An Introduction (MIT Press)

  15. [23]

    2020, in International Conference on Learning Representations (ICLR)

    Tang, Z., Bowles, C., Kastaniotis, D., et al. 2020, in International Conference on Learning Representations (ICLR). https://openreview.net/forum?id=S1x8psC9Ym

  16. [24]

    Thorndike, E. L. 1911, Animal Intelligence: Experimental Studies (Macmillan)

  17. [25]

    W., et al

    Wichrowska, O., Maheswaranathan, N., Hoffman, M. W., et al. 2017, in International Conference on Machine Learning, PMLR, 3751--3760

  18. [26]

    2019, in International Conference on Learning Representations

    Xu, X., Hospedales, T., & Gong, S. 2019, in International Conference on Learning Representations

  19. [27]

    Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. 2017, in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2223--2232, 10.1109/ICCV.2017.244

Pith tools

Reviewed August 16, 2026 · model on record in the stance chip above.