REVIEW 4 major objections 5 minor 27 references
Fast-Slow Co-advancing Optimizer: Toward Harmonious Adversarial Training of GAN
T0 review · 4 major / 5 minor · reviewed 2026-08-16 · deepseek-v4-flash
Pith's one-line read Reinforcement learning sets the discriminator's step size to keep GAN training stable
desk verdict A plausible but unsupported idea for RL-controlled GAN learning rates; no baselines, circular reward, and admitted failures make it not ready for review. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the FSCO control loop: a DDPG agent whose single continuous action scales the discriminator's learning rate. The loop is driven by the reward $-|G_{loss}(t)-D_{loss}(t)|$, which encodes the paper's notion of a harmonious game: when the discriminator's loss is high it speeds up, and when the generator's loss is high the agent suppresses the discriminator. This machinery converts the existing two-player gradient descent into a closed-loop system in which the discriminator 'waits' for the generator, and it is what the paper claims enlarges the tolerable range of step-size hyperparameters.
What would settle it
Train a GAN with FSCO while recording both the reward and the quality of generated images; if there exists a stage where the gap $|G_{loss}-D_{loss}|$ is driven to zero while the generator collapses or produces repetitive samples, the reward is not a sufficient surrogate. A cleaner test would hold all other hyperparameters fixed and compare the distribution of successful final models over a grid of discriminator base step sizes, with and without FSCO: if FSCO does not measurably widen the success region, the claim of reduced step-size sensitivity fails.
Extended reading notes
Core claim
The paper's central claim is that adversarial training can be converted from a manually tuned game into a feedback-controlled process by letting a DDPG agent regulate the discriminator's step size. The action $u(t)\in(0,1)$ multiplies the fixed discriminator step size $\eta_D(t)$ to produce $\eta_{FSC\!O-D}(t)=\eta_D(t)u(t)$, and the reward $R(t)=-|G_{loss}(t)-D_{loss}(t)|$ pushes the two losses toward each other. The authors report that this prevents the discriminator from racing ahead, gives the generator time to learn, and keeps training stable across three datasets with different resolutions and variances. They also report limitations: later-stage overfitting appeared in all experiments, and a 512×512 setup failed because the discriminator could not learn features at all.
Load-bearing premise
The load-bearing premise is that matching the generator's and discriminator's losses measures healthy progress: if that reward is not aligned with generation quality, the entire control loop is optimizing the wrong objective.
Editorial extensions
If this is right
- If FSCO works as claimed, GAN users could start from a wider range of discriminator step sizes and rely on the agent to adjust them, cutting down manual tuning time.
- The same control loop could be applied to other two-player training setups, such as actor-critic pairs or adversarial defenses, where one network's learning rate needs to be throttled relative to the other's.
- The paper's own results imply that FSCO does not remove overfitting; later training stages still degrade, so the controller postpones but does not fully solve collapse.
- The reward design suggests that pairing the feedback signal of a controller with the balance of the two players' losses, rather than with a single network's loss, is a workable heuristic for stabilizing adversarial games.
Reading between the lines
- If the reward is a poor proxy, one testable extension is to replace it with a quality-aware signal such as a held-out inception score or a discriminator-confidence target, and to check whether the agent then learns a different policy.
- The paper's observation that the learned step size concentrates near zero or in $[0.05\eta_D,0.9\eta_D]$ suggests the DDPG policy may be learning a hysteresis rule that could be distilled into a closed-form scheduler.
- Because the state vector is never specified, a direct ablation of state features would reveal whether the agent actually uses loss information or is effectively acting on a nearly constant signal.
- If the 512×512 failure reflects a limit of the control signal rather than the architecture, then FSCO's benefit may shrink as model capacity and data complexity grow, which a scaling study could test.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes Fast-Slow Co-advancing Optimizer (FSCO), which wraps a DCGAN training loop with a DDPG agent that modulates the discriminator learning rate. The action u(t) in [0,1] multiplies a fixed discriminator base learning rate (Eq. (2)), and the reward is the negative absolute difference between generator and discriminator losses (Eq. (3)). Experiments on MNIST, ANIME, and Ganyu report loss curves, learning-rate traces, and generated image samples. The paper claims that FSCO makes GAN training less sensitive to step-size hyperparameters and transforms the training process into a feedback-controlled, harmonious adversarial game. The manuscript also acknowledges severe overfitting, unstable image quality, and a failed 512x512 experiment.
Significance. If rigorously validated, an RL-based controller that widens the usable learning-rate range for GAN discriminators would be a useful contribution, particularly because existing learned optimizers mostly target single-model training. The paper's conceptual framing is clear, and it correctly identifies a practical pain point. However, the empirical validation is not at the standard required to support the central claim: there are no baselines, no quantitative image-quality metrics, no seed or compute-matched comparisons, and the one quantitative-looking measure (loss balancing) is the exact quantity the reward optimizes. The undefined state representation compounds the reproducibility problem. As presented, the evidence does not support the claimed enlargement of the usable hyperparameter range.
major comments (4)
- [§3.3.2, Eq. (3)] The reward Reward(t) = -|G_loss(t) - D_loss(t)| is the very quantity used to evaluate success in Figures 6, 8, and 10, where the discriminator and generator losses approach each other. Because the action u(t) in Eq. (2) lies in [0,1], the agent can trivially drive this reward to zero by setting u(t) approximately 0 whenever G_loss exceeds D_loss, and the near-zero discriminator learning rates in the figures are consistent with that trivial behavior. The paper provides no evidence that minimizing the raw loss gap is a valid proxy for generation quality, training stability, or mode-collapse avoidance, so the central mechanism is not established.
- [Tables 1–3] All three hyperparameter tables declare 'State dimension 6' for the DDPG agent, but the six components of the state vector are never defined anywhere in the manuscript. Since the learned control policy is the paper's contribution, an undefined state representation makes the method irreproducible and prevents the reader from assessing whether the agent has access to sufficient information.
- [§4.1–§4.3] The experimental section contains no fixed-learning-rate baseline, no comparison to standard GAN training without FSCO, no quantitative metric such as FID or Inception Score, no repeated runs, and no standard deviation or error bars. The only quantitative evidence is the loss-gap behavior, which is the optimized reward. Consequently, the claim that FSCO enlarges the usable range of discriminator/generator step sizes is not supported by the presented experiments.
- [§4.4] The discussion explicitly states that image quality is unstable, that generator effectiveness in later training is uncertain, that overfitting is 'quite severe', and that a 512x512 experiment failed because the discriminator could not learn features. These admissions directly undercut the central claims of improved stability and reduced sensitivity to step sizes, and the manuscript does not provide a systematic hyperparameter sweep or a quantitative failure analysis that would clarify the conditions under which FSCO helps.
minor comments (5)
- [Title/front matter] The title contains clear typos ('FAST-S LOW', 'T OWARD'), and Figure 3 contains 'Enviorment' and 'updata'; the manuscript needs a careful proofreading pass.
- [§3.3.1] The first two paragraphs of Section 3.3.1 are duplicated verbatim; one copy should be removed.
- [§3.3.2] The text refers to 'Section 2.3.3' when describing the gaming process, but this section does not exist; the correct reference appears to be Section 3.3.3.
- [§4.4] The sentence 'the step size in the FSCO method should be mostly distributed in the range [0, η_FSCO−D(t)]' appears to contain a self-referential notation error; it should presumably refer to [0, η_D(t)].
- [§2.1] The citation labeled 'Zhang et al.' is attributed to a reference by Tang et al.; the in-text name and the reference entry should be made consistent.
Circularity Check
The primary evidence for FSCO is the reward objective itself: Eq. (3) rewards |G_loss − D_loss| → 0, and Figures 6, 8, and 10 show that gap closing, so the 'harmonious' result is achieved by construction.
-
self definitional
[Section 3.3.2, Eq. (3); Section 4.1.2, Fig. 6]
"As depicted in the right side of Figure 3, the DDPG reward encourages closer losses of the discriminator and the generator through using the min-max loss function. Reward(t) =−|Gloss(t)−Dloss(t))| (3) This reward Equation in (3) serves two purposes: first, if the discriminator’s loss(Dloss) is large, it can increase discriminator’s step size to accelerate discriminator learning; second, if the generator’s loss(Gloss) is too large, it can reduce discriminator’s step size to give the generator sufficient steps for learning."
The paper's success criterion—a 'harmonious adversarial game'—is operationalized as the generator and discriminator losses being close, which is exactly the quantity maximized by the reward in Eq. (3). The DDPG agent is trained to minimize |G_loss − D_loss|, and the experimental evidence in Figures 6, 8, and 10 is that this gap shrinks while the discriminator learning rate is driven toward zero via Eq. (2). Thus the central evidence of effectiveness reduces to the reward definition itself, not to an independent measure of generation quality or of an enlarged usable step-size range.
full rationale
The paper uses no self-citation chain or imported uniqueness theorem; the GAN and DDPG background is standard external material. The circularity is concentrated in the reward/evaluation loop. Eq. (3) defines the reward as the negative absolute difference between generator and discriminator losses, and the paper's stated goal is harmonious adversarial training in which these losses stay close. The main experimental evidence (Figures 6, 8, and 10) is that exactly this gap shrinks while Eq. (2) suppresses the discriminator learning rate toward zero. Because the DDPG agent is trained to maximize that reward, the observed loss convergence is the optimization objective itself, not an independent test of generation quality or of an enlarged usable learning-rate range. The paper does show generated images and candidly reports instability, overfitting, and failure at 512×512 in Section 4.4, so there is some independent signal outside the reward loop; nevertheless, the load-bearing 'harmonious' evidence reduces to the reward definition. This warrants a 6 rather than a higher score: the method is not fully circular because the images and admitted failure modes are outside the reward loop, and no self-citation is load-bearing.
Assumptions & free parameters
free parameters (5)
- Discriminator base learning rate eta_D =
0.002 (MNIST), 0.005 (ANIME/Ganyu)
- Generator learning rate eta_G =
0.0002
- DDPG state representation =
6 dimensions, contents not specified
- Discriminator LR minimum multiplier =
0.001 (64x64 and 128x128)
- DDPG exploration noise =
0.1
assumptions (5)
- domain assumption The difference between generator and discriminator losses is a meaningful and stable control signal.
- domain assumption DDPG can learn an effective policy in a non-stationary environment where the GAN it controls is also changing.
- domain assumption Minimizing loss difference prevents mode collapse and improves generation quality.
- ad hoc to paper The state vector of dimension 6 contains sufficient information for the agent.
- standard math Standard min-max GAN objective and gradient descent updates (Eqs 4-7).
Cite this review
Pith. "Pith review of Fast-Slow Co-advancing Optimizer: Toward Harmonious Adversarial Training of GAN." pith.science (2026). https://pith.science/paper/TX7I56OX
@misc{pith2026250415099,
author = {Pith},
title = {Pith review of: Fast-Slow Co-advancing Optimizer: Toward Harmonious Adversarial Training of GAN},
year = {2026},
howpublished = {\url{https://pith.science/paper/TX7I56OX}},
note = {Machine review of arXiv:2504.15099}
}
read the original abstract
Up to now, the training processes of typical Generative Adversarial Networks (GANs) are still particularly sensitive to data properties and hyperparameters, which may lead to severe oscillations, difficulties in convergence, or even failures to converge, especially when the overall variances of the training sets are large. These phenomena are often attributed to the training characteristics of such networks. Aiming at the problem, this paper develops a new intelligent optimizer, Fast-Slow Co-advancing Optimizer (FSCO), which employs reinforcement learning in the training process of GANs to make training easier. Specifically, this paper allows the training step size to be controlled by an agent to improve training stability, and makes the training process more intelligent with variable learning rates, making GANs less sensitive to step size. Experiments have been conducted on three benchmark datasets to verify the effectiveness of the developed FSCO.
Figures
Figures from the paper (8 more)
Reference graph
Works this paper leans on
-
[1]
2016, in Advances in Neural Information Processing Systems, 3981--3989
Andrychowicz, M., Denil, M., Gomez, S., et al. 2016, in Advances in Neural Information Processing Systems, 3981--3989
work page 2016
-
[2]
2017, Proceedings of the 34th International Conference on Machine Learning (ICML), 70, 214
Arjovsky, M., Chintala, S., & Bottou, L. 2017, Proceedings of the 34th International Conference on Machine Learning (ICML), 70, 214. http://proceedings.mlr.press/v70/arjovsky17a.html
work page 2017
-
[3]
Bellman, R. E. 1957, Dynamic Programming (Princeton University Press)
work page 1957
-
[4]
Fujimoto, S., Hoof, H. v., & Meger, D. 2018, in Proceedings of Machine Learning Research, Vol. 80, Proceedings of the 35th International Conference on Machine Learning (ICML) (PMLR), 1587--1596. http://proceedings.mlr.press/v80/fujimoto18a.html
work page 2018
-
[5]
J., Pouget-Abadie, J., Mirza, M., et al
Goodfellow, I. J., Pouget-Abadie, J., Mirza, M., et al. 2014, in Advances in Neural Information Processing Systems 27 (NIPS 2014), 2672--2680. https://proceedings.neurips.cc/paper/2014/file/5ca3e9b122f61f8f06494c97b1afccf3-Paper.pdf
work page 2014
-
[6]
Gulrajani, I., Ahmed, F., Arjovsky, M., Dumoulin, V., & Courville, A. C. 2017, in Advances in Neural Information Processing Systems 30 (NIPS 2017), 5767--5777. https://proceedings.neurips.cc/paper/2017/file/892c3b1c6dccd52936e27cbd0ff683d6-Paper.pdf
work page 2017
-
[7]
2018, in Proceedings of Machine Learning Research, Vol
Haarnoja, T., Zhou, A., Abbeel, P., & Levine, S. 2018, in Proceedings of Machine Learning Research, Vol. 80, Proceedings of the 35th International Conference on Machine Learning (ICML) (PMLR), 1861--1870. http://proceedings.mlr.press/v80/haarnoja18b.html
work page 2018
-
[8]
Karras, T., Laine, S., & Aila, T. 2019, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 4401--4410, 10.1109/CVPR.2019.00453
arXiv 2019
Show all 27 references
-
[9]
2020, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 8110--8119, 10.1109/CVPR42600.2020.00813
Karras, T., Laine, S., Aittala, M., et al. 2020, in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 8110--8119, 10.1109/CVPR42600.2020.00813
2020
-
[10]
2024, arXiv preprint arXiv:2402.01582
Lan, Q., Zhang, C., Zhang, W., et al. 2024, arXiv preprint arXiv:2402.01582
2024 arXiv
-
[11]
P., Hunt, J
Lillicrap, T. P., Hunt, J. J., Pritzel, A., et al. 2015, arXiv preprint arXiv:1509.02971
2015 arXiv
-
[12]
P., Mirza, M., et al
Mnih, V., Badia, A. P., Mirza, M., et al. 2016, in Proceedings of Machine Learning Research, Vol. 48, Proceedings of the 33rd International Conference on Machine Learning (ICML) (PMLR), 1928--1937. http://proceedings.mlr.press/v48/mnih16.html
2016
-
[13]
2013, arXiv preprint arXiv:1312.5602
Mnih, V., Kavukcuoglu, K., Silver, D., et al. 2013, arXiv preprint arXiv:1312.5602. 1312.5602
2013 arXiv
-
[14]
2015, Nature, 518, 529, 10.1038/nature14236
---. 2015, Nature, 518, 529, 10.1038/nature14236
2015 doi
-
[15]
Pavlov, I. P. 1927, Conditioned Reflexes: An Investigation of the Physiological Activity of the Cerebral Cortex (Oxford University Press)
1927
-
[16]
2015, arXiv preprint arXiv:1511.06434
Radford, A., Metz, L., & Chintala, S. 2015, arXiv preprint arXiv:1511.06434. 1511.06434
2015 arXiv
-
[17]
2020, Nature, 588, 604, 10.1038/s41586-020-03051-4
Schrittwieser, J., Antonoglou, I., Hubert, T., et al. 2020, Nature, 588, 604, 10.1038/s41586-020-03051-4
2020 doi
-
[18]
I., & Moritz, P
Schulman, J., Levine, S., Abbeel, P., Jordan, M. I., & Moritz, P. 2015, in Proceedings of Machine Learning Research, Vol. 37, Proceedings of the 32nd International Conference on Machine Learning (ICML) (PMLR), 1889--1897. http://proceedings.mlr.press/v37/schulman15.html
2015
-
[19]
2017, arXiv preprint arXiv:1707.06347
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., & Klimov, O. 2017, arXiv preprint arXiv:1707.06347. 1707.06347
2017 arXiv
-
[20]
2014, in Proceedings of Machine Learning Research, Vol
Silver, D., Lever, G., Heess, N., et al. 2014, in Proceedings of Machine Learning Research, Vol. 32, Proceedings of the 31st International Conference on Machine Learning (ICML) (PMLR), 387--395. http://proceedings.mlr.press/v32/silver14.html
2014
-
[21]
P., Pervez, R., et al
Subramanian, A., Mudumba, S. P., Pervez, R., et al. 2023, arXiv preprint arXiv:2305.15076
2023 arXiv
-
[22]
S., & Barto, A
Sutton, R. S., & Barto, A. G. 1998, Reinforcement Learning: An Introduction (MIT Press)
1998
-
[23]
2020, in International Conference on Learning Representations (ICLR)
Tang, Z., Bowles, C., Kastaniotis, D., et al. 2020, in International Conference on Learning Representations (ICLR). https://openreview.net/forum?id=S1x8psC9Ym
2020
-
[24]
Thorndike, E. L. 1911, Animal Intelligence: Experimental Studies (Macmillan)
1911
-
[25]
W., et al
Wichrowska, O., Maheswaranathan, N., Hoffman, M. W., et al. 2017, in International Conference on Machine Learning, PMLR, 3751--3760
2017
-
[26]
2019, in International Conference on Learning Representations
Xu, X., Hospedales, T., & Gong, S. 2019, in International Conference on Learning Representations
2019
-
[27]
Zhu, J.-Y., Park, T., Isola, P., & Efros, A. A. 2017, in Proceedings of the IEEE International Conference on Computer Vision (ICCV), 2223--2232, 10.1109/ICCV.2017.244
2017 doi
Reviewed August 16, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.