REVIEW 3 major objections 5 minor 17 references
Generalized Dual Discriminator GANs
T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read Dual-discriminator GANs with alpha-loss reduce to minimizing a forward and a reverse f-divergence.
desk verdict A clean but incremental generalization of dual-discriminator GANs to alpha-loss and arbitrary losses, with a small algebraic error that does not affect the main conclusion, and experiments that are too thin to lean on. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The carrying object is the pointwise optimal-discriminator ratio. For a fixed generator, the inner supremum separates into one-dimensional maximizations of $h(t)=-a\,\ell_{\alpha_1}(t)+b\,(\ell_{\alpha_2}(t)-1)$ over positive scores, whose unique maximum is $t^*=(a/b)^{\alpha_1\alpha_2/(\alpha_2-\alpha_1)}$; substituting $a=c_1P_d(x), b=P_g(x)$ and $a=P_d(x), b=c_2P_g(x)$ gives the two closed-form discriminators. In the generalized version, the same substitution is performed by the convex function $$f_c(u)=\sup_{t>0}[-u\,\ell_1(t)+\ell_2(t)/c]$$, a Legendre-type transform, since the supremum is automatically convex in $u$, that manufactures an f-divergence from any pair of loss functions on the positive reals. That transform is what converts the three-player game into a sum of a forward and a reverse f-divergence.
What would settle it
On the 8-Gaussian ring benchmark used in the paper, train dual-discriminator alpha-GANs with several discriminator widths and compare the achieved value with the supremum over all measurable discriminators for the same generator. If the gap does not shrink as the networks grow wider, or if supplying the exact closed-form discriminators does not drive the generator to the true mixture, the reduction is not realized and should be treated as an ideal-capacity statement rather than a description of the trained system.
Extended reading notes
Core claim
On the paper's own terms, the central discovery is an identity rather than an architecture tweak: under the stated ideal-capacity assumption, once each discriminator is optimal for a fixed generator, the dual-discriminator $\alpha$-GAN value function becomes $$c_1 D_f(P_d\|P_g) + c_2 D_f(P_g\|P_d)$$ for an explicit convex $f$, so the whole three-player game is exactly a weighted two-direction divergence minimization. The optimal discriminators are $$D_1^*(x) = (c_1 P_d(x)/P_g(x))^{\alpha_1\alpha_2/(\alpha_2-\alpha_1)}$$ and $$D_2^*(x) = (c_2 P_g(x)/P_d(x))^{\alpha_1\alpha_2/(\alpha_2-\alpha_1)}$$, which are valid maximizers when $\alpha_2>\alpha_1$, and substituting them yields the divergence objective. The second theorem generalizes the collapse to arbitrary $\ell_1,\ell_2:\mathbb{R}_+\to\mathbb{R}$, with $$f_c(u)=\sup_{t>0}[-u\,\ell_1(t)+\ell_2(t)/c]$$, which is convex by construction. The identities hold only up to additive constants that do not affect the minimizing generator, and the minimizer is $P_g=P_d$.
Load-bearing premise
The whole reduction assumes each discriminator can output, at every input location, the exact closed-form function of the density ratio $P_d/P_g$; with finite neural networks this is not achieved, and if the two distributions are not mutually absolutely continuous the ratio itself is undefined.
Editorial extensions
If this is right
- If the theorems are correct, the D2 alpha-GAN objective has the true data distribution as its generator equilibrium, which directly addresses mode collapse by penalizing both missing modes and off-manifold samples.
- The known D2 GAN result, reduction to $c_1 D_{KL}(P_d\|P_g)+c_2 D_{KL}(P_g\|P_d)$, is recovered as the limit $\alpha_1\to 1$, $\alpha_2\to\infty$, making the new framework a strict generalization.
- The tunable parameters $\alpha_1,\alpha_2,c_1,c_2$ interpolate between exponential loss, binary cross-entropy, and soft 0-1 loss without leaving the divergence-minimization interpretation.
- Because $f_c$ is convex for every pair of losses, the generalized construction always yields a mathematically well-defined f-divergence objective, not merely an ad hoc game.
- By Proposition 1, a single-discriminator GAN cannot carry two different alpha parameters, so the dual-discriminator architecture is necessary for this tunable objective.
Reading between the lines
- The variational form of Theorem 2 invites an inverse-design route the paper does not explore: choose a target f-divergence pair and solve for $\ell_1,\ell_2$ whose Legendre-type transform realizes it, then train toward that target.
- A capacity-scaling experiment would clarify the practical status of the theorems: if wider discriminators progressively approach the closed-form value, the reduction is a limiting statement; the paper's fixed-architecture experiments alone do not establish that.
- The condition $\alpha_2>\alpha_1$ suggests a concrete design heuristic not present in D2 GANs: the discriminator rewarding real data should carry a larger alpha than the one rewarding generated data; the paper does not study this rule separately.
- The same supremum construction may extend to more than two discriminators grouped into complementary directions, producing sums of paired f-divergences; that generalization remains implicit.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes two related GAN variants: dual discriminator α-GANs (D2 α-GANs), which replace the logarithmic and linear rewards of D2 GANs with α-loss terms controlled by parameters α1 and α2, and generalized dual discriminator GANs, which allow arbitrary functions ℓ1 and ℓ2 on R+ in the two discriminator rewards. The central theoretical claim is that, under a sufficient capacity assumption, the three-player min-max game reduces to minimizing a linear combination of an f-divergence and a reverse f-divergence. Lemma 1 gives the closed-form optimal discriminators; Theorem 1 states the reduction for α-loss with an explicit f_c; Theorem 2 states a general reduction with f_c(u)=sup_t[-u ℓ1(t)+ℓ2(t)/c]. Experiments on a 2D mixture of Gaussians compare D2 α-GANs with D2 GANs and vanilla GANs using symmetric KL divergence and Wasserstein distance.
Significance. If the reduction is correct, the paper gives a clean unification: D2 α-GANs interpolate between exponential loss, binary cross-entropy, and soft 0-1 loss, and Theorem 2 shows that any dual-discriminator objective of the proposed form reduces to divergence minimization. The paper explicitly states the infinite-capacity idealization, which is standard for this type of GAN analysis. The main weakness is that the stated equalities are not exact: the '−1' terms in the ℓ2 rewards are dropped in the proofs of both theorems, so Eqs. (13), (15), and (18) are off by G-independent additive constants. Since the constants do not depend on G, the minimizer and the qualitative conclusion survive, but the theorems as printed are not mathematically correct. The empirical section is illustrative rather than conclusive, with a single seed and no error bars.
major comments (3)
- [Section III, Theorem 1 (Eqs. (13)–(15)) and Appendix B] The proof drops the '−1' in the terms Pg(x)(ℓα2(D1(x))−1) and Pd(x)(ℓα2(D2(x))−1) when substituting the optimal discriminators from Lemma 1. In Appendix B, these terms are treated as Pg(x)ℓα2(D1(x)) and Pd(x)ℓα2(D2(x)); compare the expression in Eq. (21) with the substitution in Eq. (24). Since ℓα2(t)−1 = ℓα2(t) + 1/(α2−1), the omitted contribution is (Pg(x)+Pd(x))/(α2−1), which integrates to 2/(α2−1). Consequently Eq. (13) should include an additive term 2/(α2−1), Eq. (14) should include the constant 1/(c(α2−1)) in f_c, and Eq. (15) is missing the same constant. The minimizer is unaffected because the constant is independent of G, but the equalities as stated are not exact.
- [Section IV, Theorem 2 (Eqs. (18)–(19)) and Appendix D] The same '−1' omission appears in the proof of Theorem 2. The value function in Eq. (17) contains Pg(ℓ2(D1)−1) and Pd(ℓ2(D2)−1), but the first displayed computation in Appendix D replaces these with Pgℓ2(D1) and Pdℓ2(D2). The correct definition should be fc(u) = sup_t [ −u·ℓ1(t) + (ℓ2(t)−1)/c ], not Eq. (19). With the printed definition, the reduction holds only modulo an additive constant. Moreover, Remark 4's claim that the generalized framework recovers D2 α-GANs is inconsistent with Eqs. (14) and (19) unless both are corrected in the same way.
- [Section IV, Theorem 2] Theorem 2 is stated for arbitrary functions ℓ1,ℓ2 : R+ → R, but no conditions are given for the supremum in Eq. (19) to be finite or for the pointwise supremum and the integral to be interchangeable. For arbitrary ℓ1 and ℓ2, fc(u) can be infinite, making the divergence in Eq. (18) ill-defined. The theorem should either restrict the class of functions or state explicitly that the analysis is in the extended reals under a well-posedness assumption. This is a generality gap rather than a fatal flaw, but it should be fixed in revision.
minor comments (5)
- [Appendix A, Eq. (21)] The expression in Eq. (21) uses D1(G(z)) inside an integral over x with Pg(x); this appears to be a typo and should read D1(x).
- [Section V] The empirical claims of 'faster convergence' and 'more stable learning' are based on a single seed with no error bars or repeated runs; the plots in Figure 1 should be accompanied by statistical evidence or explicitly labeled as illustrative.
- [Equations (9)–(10)] The notation for the value function alternates between Vα(G,D1,D2) in Eq. (9) and Vα(D1,D2,G) in Eq. (10); please choose one consistent ordering.
- [Appendix D] The proof writes sums over x where integrals are intended; this informal notation should be made rigorous, especially since the interchange of sup and integral is part of the argument.
- [Figure 1] The axis labels appear duplicated in the caption ('Symmetric KL-divergence' and 'Wasserstein Distance' are repeated); the figure and caption should be cleaned up.
Circularity Check
No load-bearing circularity: the min-max reductions are derived by direct substitution and pointwise optimization, not by fitting or by imported uniqueness.
full rationale
The central derivation is self-contained. Theorem 1 substitutes the closed-form discriminators from Lemma 1 into the value function (10) and obtains the f-divergence objective algebraically; Theorem 2 obtains f_c as the pointwise supremum over t of -u*l1(t)+l2(t)/c in Eq. (19), which is a variational representation, not a fitted parameter relabeled as a prediction. No parameter is fitted to data and then predicted back: c1, c2, alpha1, alpha2 are explicit model hyperparameters, and the experiment section is illustrative rather than used to derive the theorem. The only author-overlapping citation is [4], cited for the background CPE-loss and alpha-loss GAN framework; the alpha-loss formula is also attributed to [12]-[14], and no uniqueness theorem or ansatz is imported from that citation into the main proof. The stated finite-capacity idealization in Lemma 1 is an acknowledged limitation, not a circular step. The skeptic's point about Appendix B dropping G-independent 1/(alpha2-1) constants in l_alpha2(t)-1 is a real correctness defect in the exact value at Eq. (15), but since the missing constant is independent of G, it does not make the minimizer or the divergence reduction circular. Therefore no circular step is exhibited; score 0.
Assumptions & free parameters
free parameters (4)
- alpha1 =
0.6 in the reported experiment
- alpha2 =
0.9 in the reported experiment
- c1 =
0.01 in the reported experiment
- c2 =
1.5 in the reported experiment
assumptions (4)
- domain assumption Generator and discriminators have sufficient parametric capacity to realize the pointwise optimal discriminators in Lemma 1.
- domain assumption The densities Pd and Pg are mutually absolutely continuous so that ratios Pd/Pg and Pg/Pd are well defined.
- standard math Standard convexity and Jensen's inequality properties of f-divergences.
- domain assumption The alpha-loss can be extended from [0,1] to positive reals.
Cite this review
Pith. "Pith review of Generalized Dual Discriminator GANs." pith.science (2026). https://pith.science/paper/N5J33F5C
@misc{pith2026250717684,
author = {Pith},
title = {Pith review of: Generalized Dual Discriminator GANs},
year = {2026},
howpublished = {\url{https://pith.science/paper/N5J33F5C}},
note = {Machine review of arXiv:2507.17684}
}
abstract
Dual discriminator generative adversarial networks (D2 GANs) were introduced to mitigate the problem of mode collapse in generative adversarial networks. In D2 GANs, two discriminators are employed alongside a generator: one discriminator rewards high scores for samples from the true data distribution, while the other favors samples from the generator. In this work, we first introduce dual discriminator $\alpha$-GANs (D2 $\alpha$-GANs), which combines the strengths of dual discriminators with the flexibility of a tunable loss function, $\alpha$-loss. We further generalize this approach to arbitrary functions defined on positive reals, leading to a broader class of models we refer to as generalized dual discriminator generative adversarial networks. For each of these proposed models, we provide theoretical analysis and show that the associated min-max optimization reduces to the minimization of a linear combination of an $f$-divergence and a reverse $f$-divergence. This generalizes the known simplification for D2-GANs, where the objective reduces to a linear combination of the KL-divergence and the reverse KL-divergence. Finally, we perform experiments on 2D synthetic data and use multiple performance metrics to capture various advantages of our GANs.
Figures
Reference graph
Works this paper leans on
-
[4]
Addressing GAN training instabilities via tunable classification losses,
M. Welfert, G. R. Kurri, K. Otstot, and L. Sankar, “Addressing GAN training instabilities via tunable classification losses,” IEEE Journal on Selected Areas in Information Theory , 2024
work page 2024
-
[1]
I. J. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adversarial nets,” in Advances in Neural Information Processing Systems , Z. Ghahramani, M. Welling, C. Cortes, N. Lawrence, and K. Weinberger, Eds., vol. 27. Curran Associates, Inc., 2014
work page 2014
-
[2]
Wasserstein generative adversarial networks,
M. Arjovsky, S. Chintala, and L. Bottou, “Wasserstein generative adversarial networks,” in International conference on machine learning. PMLR, 2017, pp. 214–223
2017
-
[3]
Unrolled generative adversarial networks,
L. Metz, B. Poole, D. Pfau, and J. Sohl-Dickstein, “Unrolled generative adversarial networks,” arXiv preprint arXiv:1611.02163 , 2016
arXiv 2016
-
[5]
f-GAN: Training generative neural samplers using variational divergence minimization,
S. Nowozin, B. Cseke, and R. Tomioka, “f-GAN: Training generative neural samplers using variational divergence minimization,” Advances in neural information processing systems , vol. 29, 2016
work page 2016
-
[6]
On the empirical estimation of integral probability metrics,
B. Sriperumbudur, K. Fukumizu, A. Gretton, B. Schölkopf, and G. Lanckriet, “On the empirical estimation of integral probability metrics,” Electronic Journal of Statistics , vol. 6, 01 2012
work page 2012
-
[7]
How well generative adversarial networks learn distributions,
T. Liang, “How well generative adversarial networks learn distributions,” Journal of Machine Learning Research, vol. 22, no. 228, pp. 1–41, 2021
2021
-
[8]
Y . Pantazis, D. Paul, M. Fasoulakis, Y . Stylianou, and M. A. Katsoulakis, “Cumulant GAN,” IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 11, pp. 9439–9450, 2022
work page 2022
Show all 17 references
-
[9]
Least k th-order and Rényi generative adversarial networks,
H. Bhatia, W. Paul, F. Alajaji, B. Gharesifard, and P. Burlina, “Least k th-order and Rényi generative adversarial networks,” Neural Computa- tion, vol. 33, no. 9, pp. 2473–2510, 2021
2021
-
[10]
A unifying generator loss function for generative adversarial networks,
J. Veiner, F. Alajaji, and B. Gharesifard, “A unifying generator loss function for generative adversarial networks,” Entropy, vol. 26, no. 4, p. 290, 2024
2024
-
[11]
Dual discriminator generative adversarial nets,
T. Nguyen, T. Le, H. Vu, and D. Phung, “Dual discriminator generative adversarial nets,” Advances in neural information processing systems , vol. 30, 2017
2017
-
[12]
Information-theoretical considerations on estimation prob- lems,
S. Arimoto, “Information-theoretical considerations on estimation prob- lems,” Information and control , vol. 19, no. 3, pp. 181–194, 1971
1971
-
[13]
Tunable measures for information leakage and applications to privacy-utility tradeoffs,
J. Liao, O. Kosut, L. Sankar, and F. du Pin Calmon, “Tunable measures for information leakage and applications to privacy-utility tradeoffs,” IEEE Transactions on Information Theory , vol. 65, no. 12, pp. 8043– 8066, 2019
2019
-
[14]
A tunable loss function for robust classification: Calibra- tion, landscape, and generalization,
T. Sypherd, M. Diaz, J. K. Cava, G. Dasarathy, P. Kairouz, and L. Sankar, “A tunable loss function for robust classification: Calibra- tion, landscape, and generalization,” IEEE Transactions on Information Theory, vol. 68, no. 9, pp. 6021–6051, 2022
2022
-
[15]
NIPS 2016 tutorial: Generative adversarial networks,
I. Goodfellow, “NIPS 2016 tutorial: Generative adversarial networks,” arXiv preprint arXiv:1701.00160 , 2016
2016 arXiv
-
[16]
Symmetric KL-divergence by Stein’s method,
L.-Q. Yao and S.-H. Liu, “Symmetric KL-divergence by Stein’s method,” Stochastic Processes and their Applications , p. 104635, 2025
2025
-
[17]
On surrogate loss functions and f-divergences,
X. Nguyen, M. J. Wainwright, and M. I. Jordan, “On surrogate loss functions and f-divergences,” The Annals of Statistics , vol. 37, no. 2, Apr. 2009. APPENDIX A. Proof of Lemma 1 It is easy to see that the value function in equation (3) can be re-written by expanding the expec...
2009
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.