{"id":"e354d106-0f9b-44c0-90a7-68700f5f3879","arxiv_id":"2504.15099","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"FSCO multiplies the discriminator learning rate by a DDPG-chosen factor in [0,1], using reward -|G_loss - D_loss|, and reports more stable-looking GAN training on three image datasets.","lead":"This paper proposes FSCO, a GAN optimizer that uses a DDPG reinforcement learning agent to scale the discriminator's learning rate down based on the gap between generator and discriminator losses. A smart generalist might read it to see a concrete RL-based alternative to manual GAN hyperparameter tuning, though the current evidence is qualitative and lacks baseline comparisons.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The reward in Eq. (3) rewards matching G_loss to D_loss, and the DDPG agent can trivially maximize it by driving the discriminator learning rate toward zero; no evidence links this reward to generation quality or to the claimed enlargement of the usable learning-rate range.","rationale":"The reader's REJECT verdict is appropriate, and my read does not change it. However, I would sharpen the load-bearing weakness: the reward in Eq. (3) can be trivially maximized by setting the discriminator learning rate to zero, so the observed loss-balancing behavior is not by itself evidence of a successful learned controller. This is more fundamental than the undefined 6-dimensional state vector: even with a fully specified state, the reward could be gamed. The paper's own Section 4.4 admits severe overfitting, unstable image quality, and a failed 512x512 run, so the abstract's claim of decreased sensitivity to step sizes is unsupported. A controlled experiment with a hand-coded wait policy and quantitative image-quality metrics would distinguish genuine feedback control from a trivial policy that simply stops the discriminator whenever the generator lags.","tokens_in":11285,"tokens_out":4038,"duration_ms":36608,"concrete_test":"Run a controlled comparison on MNIST and ANIME: (a) FSCO with the full DDPG, (b) a fixed-learning-rate DCGAN baseline, and (c) a hand-coded 'wait' policy that sets u(t)=0 whenever G_loss(t) > D_loss(t), all matched for total generator updates and compute, reporting FID/IS and a sweep of discriminator base learning rates. In the same runs, compute the Spearman correlation between the Eq. (3) reward and FID across checkpoints. If the wait policy matches FSCO's loss curves and image quality, or if reward correlates weakly or negatively with FID, the paper's central claim is not supported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that FSCO enlarges the usable range of discriminator/generator step sizes rests entirely on the DDPG reward in Eq. (3), Reward(t) = -|G_loss(t) - D_loss(t)|. Nothing in the paper shows that minimizing this absolute difference is a valid proxy for generation quality or training stability. The action in Eq. (2) multiplies a fixed discriminator learning rate by u(t) in [0,1], so the agent can achieve a perfect reward by setting u(t)=0 whenever G_loss exceeds D_loss. That is exactly the behavior shown in Figures 6, 8, and 10, where the discriminator learning rate is driven to near zero. This may reduce the loss gap, but the paper provides no quantitative image-quality metric such as FID or IS, no fixed-learning-rate baseline, no seed or compute-matched comparison, and the authors state in Section 4.4 that image quality is unstable, overfitting is severe, and the 512x512 run fails. The 6-dimensional state vector is never defined in Tables 1-3, so the policy's inputs are not reproducible. Consequently, the observed 'harmonious' loss curves could be an artifact of a trivial wait policy rather than evidence of a robust feedback-controlled optimizer.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Fast-Slow Co-advancing Optimizer (FSCO), which wraps a DCGAN training loop with a DDPG agent that modulates the discriminator learning rate. The action u(t) in [0,1] multiplies a fixed discriminator base learning rate (Eq. (2)), and the reward is the negative absolute difference between generator and discriminator losses (Eq. (3)). Experiments on MNIST, ANIME, and Ganyu report loss curves, learning-rate traces, and generated image samples. The paper claims that FSCO makes GAN training less sensitive to step-size hyperparameters and transforms the training process into a feedback-controlled, harmonious adversarial game. The manuscript also acknowledges severe overfitting, unstable image quality, and a failed 512x512 experiment.","tokens_in":11659,"tokens_out":2865,"duration_ms":27952,"significance":"If rigorously validated, an RL-based controller that widens the usable learning-rate range for GAN discriminators would be a useful contribution, particularly because existing learned optimizers mostly target single-model training. The paper's conceptual framing is clear, and it correctly identifies a practical pain point. However, the empirical validation is not at the standard required to support the central claim: there are no baselines, no quantitative image-quality metrics, no seed or compute-matched comparisons, and the one quantitative-looking measure (loss balancing) is the exact quantity the reward optimizes. The undefined state representation compounds the reproducibility problem. As presented, the evidence does not support the claimed enlargement of the usable hyperparameter range.","major_comments":[{"comment":"The reward Reward(t) = -|G_loss(t) - D_loss(t)| is the very quantity used to evaluate success in Figures 6, 8, and 10, where the discriminator and generator losses approach each other. Because the action u(t) in Eq. (2) lies in [0,1], the agent can trivially drive this reward to zero by setting u(t) approximately 0 whenever G_loss exceeds D_loss, and the near-zero discriminator learning rates in the figures are consistent with that trivial behavior. The paper provides no evidence that minimizing the raw loss gap is a valid proxy for generation quality, training stability, or mode-collapse avoidance, so the central mechanism is not established.","section":"§3.3.2, Eq. (3)"},{"comment":"All three hyperparameter tables declare 'State dimension 6' for the DDPG agent, but the six components of the state vector are never defined anywhere in the manuscript. Since the learned control policy is the paper's contribution, an undefined state representation makes the method irreproducible and prevents the reader from assessing whether the agent has access to sufficient information.","section":"Tables 1–3"},{"comment":"The experimental section contains no fixed-learning-rate baseline, no comparison to standard GAN training without FSCO, no quantitative metric such as FID or Inception Score, no repeated runs, and no standard deviation or error bars. The only quantitative evidence is the loss-gap behavior, which is the optimized reward. Consequently, the claim that FSCO enlarges the usable range of discriminator/generator step sizes is not supported by the presented experiments.","section":"§4.1–§4.3"},{"comment":"The discussion explicitly states that image quality is unstable, that generator effectiveness in later training is uncertain, that overfitting is 'quite severe', and that a 512x512 experiment failed because the discriminator could not learn features. These admissions directly undercut the central claims of improved stability and reduced sensitivity to step sizes, and the manuscript does not provide a systematic hyperparameter sweep or a quantitative failure analysis that would clarify the conditions under which FSCO helps.","section":"§4.4"}],"minor_comments":[{"comment":"The title contains clear typos ('FAST-S LOW', 'T OWARD'), and Figure 3 contains 'Enviorment' and 'updata'; the manuscript needs a careful proofreading pass.","section":"Title/front matter"},{"comment":"The first two paragraphs of Section 3.3.1 are duplicated verbatim; one copy should be removed.","section":"§3.3.1"},{"comment":"The text refers to 'Section 2.3.3' when describing the gaming process, but this section does not exist; the correct reference appears to be Section 3.3.3.","section":"§3.3.2"},{"comment":"The sentence 'the step size in the FSCO method should be mostly distributed in the range [0, η_FSCO−D(t)]' appears to contain a self-referential notation error; it should presumably refer to [0, η_D(t)].","section":"§4.4"},{"comment":"The citation labeled 'Zhang et al.' is attributed to a reference by Tang et al.; the in-text name and the reference entry should be made consistent.","section":"§2.1"}],"recommendation":"reject","confidential_remarks":"The manuscript reads as an early draft and does not meet the experimental standards expected for a journal submission. The lack of any baseline, any quantitative metric, and any definition of the state vector makes the central claim unverifiable. Even a substantial revision would require new experiments and a reworked reward analysis, so rejection is appropriate."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"You should know this paper proposes using DDPG to control the discriminator's learning rate in GAN training, with the reward being how close the generator and discriminator losses are. That combination is new as far as the cited literature goes, and the authors deserve credit for a clean statement of the idea and for candidly reporting overfitting, unstable image quality, and a failed 512×512 run.\n\nThe problem is that the evidence does not support the central claim that FSCO enlarges the usable range of step sizes. There are no baselines, no FID or IS, no repeated runs, no seeds, no code. The main behavioral evidence—the \"harmonious\" loss curves in Figures 6, 8, and 10—is exactly what the reward in Eq. (3) optimizes, so it is circular. The stress-test note is on target: the agent can trivially maximize the reward by driving the discriminator learning rate to zero, which is precisely what the figures show. Nothing links that loss-matching behavior to generation quality or to a broader usable learning-rate range. The six-dimensional state vector is never defined, so even the method as described is not reproducible.\n\nThe paper reads like a preliminary report, not a completed study. The idea might be worth exploring properly, but as submitted it is not a serious empirical contribution. I would not send it to reviewers; I would tell the authors to run a real comparison against fixed learning rates and standard stabilizers, report standard image-quality metrics, define the state, and test whether the reward actually tracks generation quality.\n\nFor the record: the authors are not being dishonest; they list their own limitations. But the load-bearing claim is unsupported. This is a desk reject, not because the idea is worthless, but because the evidence is far too thin.","headline":"A plausible but unsupported idea for RL-controlled GAN learning rates; no baselines, circular reward, and admitted failures make it not ready for review.","tokens_in":12099,"tokens_out":1998,"would_cite":false,"duration_ms":19930,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Reinforcement learning sets the discriminator's step size to keep GAN training stable","keywords":["generative adversarial networks","DDPG","reinforcement learning","learning rate control","adversarial training stability","mode collapse","hyperparameter sensitivity","step-size optimization"],"falsifier":"Train a GAN with FSCO while recording both the reward and the quality of generated images; if there exists a stage where the gap $|G_{loss}-D_{loss}|$ is driven to zero while the generator collapses or produces repetitive samples, the reward is not a sufficient surrogate. A cleaner test would hold all other hyperparameters fixed and compare the distribution of successful final models over a grid of discriminator base step sizes, with and without FSCO: if FSCO does not measurably widen the success region, the claim of reduced step-size sensitivity fails.","tokens_in":11045,"feed_emoji":"🎛️","tokens_out":4081,"duration_ms":33368,"temperature":0.7,"pith_summary":"This paper argues that GAN training instability comes largely from fixed, manually chosen learning-rate step sizes, and proposes to replace the discriminator's fixed step size with a dynamically scaled one controlled by a reinforcement-learning agent. The new optimizer, FSCO, wraps the DDPG algorithm around the GAN: after each update the agent observes the training state and outputs a multiplier in [0,1] that shrinks the discriminator's base learning rate whenever the generator is falling behind. On MNIST, ANIME, and Ganyu datasets the authors report that the training process becomes a stable, self-correcting feedback loop and that the usable range of discriminator step sizes is enlarged. If true, this would reduce the manual hyperparameter debugging that currently dominates GAN practice.","feed_headline":"An RL agent now tunes the GAN learning steps","feed_subtitle":"A DDPG controller scales the discriminator's step to keep generator and discriminator losses in balance.","key_machinery":"The central object is the FSCO control loop: a DDPG agent whose single continuous action scales the discriminator's learning rate. The loop is driven by the reward $-|G_{loss}(t)-D_{loss}(t)|$, which encodes the paper's notion of a harmonious game: when the discriminator's loss is high it speeds up, and when the generator's loss is high the agent suppresses the discriminator. This machinery converts the existing two-player gradient descent into a closed-loop system in which the discriminator 'waits' for the generator, and it is what the paper claims enlarges the tolerable range of step-size hyperparameters.","core_discovery":"The paper's central claim is that adversarial training can be converted from a manually tuned game into a feedback-controlled process by letting a DDPG agent regulate the discriminator's step size. The action $u(t)\\in(0,1)$ multiplies the fixed discriminator step size $\\eta_D(t)$ to produce $\\eta_{FSC\\!O-D}(t)=\\eta_D(t)u(t)$, and the reward $R(t)=-|G_{loss}(t)-D_{loss}(t)|$ pushes the two losses toward each other. The authors report that this prevents the discriminator from racing ahead, gives the generator time to learn, and keeps training stable across three datasets with different resolutions and variances. They also report limitations: later-stage overfitting appeared in all experiments, and a 512×512 setup failed because the discriminator could not learn features at all.","pith_inferences":["If the reward is a poor proxy, one testable extension is to replace it with a quality-aware signal such as a held-out inception score or a discriminator-confidence target, and to check whether the agent then learns a different policy.","The paper's observation that the learned step size concentrates near zero or in $[0.05\\eta_D,0.9\\eta_D]$ suggests the DDPG policy may be learning a hysteresis rule that could be distilled into a closed-form scheduler.","Because the state vector is never specified, a direct ablation of state features would reveal whether the agent actually uses loss information or is effectively acting on a nearly constant signal.","If the 512×512 failure reflects a limit of the control signal rather than the architecture, then FSCO's benefit may shrink as model capacity and data complexity grow, which a scaling study could test."],"forward_implications":["If FSCO works as claimed, GAN users could start from a wider range of discriminator step sizes and rely on the agent to adjust them, cutting down manual tuning time.","The same control loop could be applied to other two-player training setups, such as actor-critic pairs or adversarial defenses, where one network's learning rate needs to be throttled relative to the other's.","The paper's own results imply that FSCO does not remove overfitting; later training stages still degrade, so the controller postpones but does not fully solve collapse.","The reward design suggests that pairing the feedback signal of a controller with the balance of the two players' losses, rather than with a single network's loss, is a workable heuristic for stabilizing adversarial games."],"supporting_citations":[{"why":"Provides the DDPG algorithm that FSCO embeds as the step-size controller.","marker":"Lillicrap et al. 2015"},{"why":"Defines the GAN min-max objective whose training instability FSCO addresses.","marker":"Goodfellow et al. 2014"},{"why":"Establishes the learning-to-optimize paradigm of using a learned model as a step-size optimizer, which FSCO extends to GANs.","marker":"Andrychowicz et al. 2016"},{"why":"Proposes using reinforcement learning as a step-size optimizer, the direct predecessor of FSCO's approach.","marker":"Xu et al. 2019"}],"fun_headline_variants":["RL agent tunes GAN step sizes for stable training","DDPG controller balances GAN losses to prevent divergence","Harness RL to keep GAN training from oscillating","AI agent adjusts discriminator steps for steady GANs","Reinforcement learning stabilizes GAN training dynamics"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that matching the generator's and discriminator's losses measures healthy progress: if that reward is not aligned with generation quality, the entire control loop is optimizing the wrong objective.","fun_headline_variants_meta":{"raw":{"variants":["RL agent tunes GAN step sizes for stable training","DDPG controller balances GAN losses to prevent divergence","Harness RL to keep GAN training from oscillating","AI agent adjusts discriminator steps for steady GANs","Reinforcement learning stabilizes GAN training dynamics"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000581,"raw_usage":{"total_tokens":2698,"prompt_tokens":867,"completion_tokens":1831,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":483,"completion_tokens_details":{"reasoning_tokens":1752}},"tokens_in":483,"tokens_out":1831,"duration_ms":9794,"temperature":1.0,"reasoning_tokens":1752,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T11:33:07.383930+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a GAN with FSCO while recording both the reward and the quality of generated images; if there exists a stage where the gap $|G_{loss}-D_{loss}|$ is driven to zero while the generator collapses or produces repetitive samples, the reward is not a sufficient surrogate. A cleaner test would hold all other hyperparameters fixed and compare the distribution of successful final models over a grid of discriminator base step sizes, with and without FSCO: if FSCO does not measurably widen the success region, the claim of reduced step-size sensitivity fails.","supporting_citations":[{"cited_title":"J., Pouget-Abadie, J., Mirza, M., et al","cited_arxiv_id":null,"evidence_quote":"Defines the GAN min-max objective whose training instability FSCO addresses."},{"cited_title":"2016, in Advances in Neural Information Processing Systems, 3981--3989","cited_arxiv_id":null,"evidence_quote":"Establishes the learning-to-optimize paradigm of using a learned model as a step-size optimizer, which FSCO extends to GANs."}],"review_version":1}