REVIEW 3 major objections 5 minor 25 references
Domain Intersection and Domain Difference
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Given two unpaired image sets, three losses—zero, adversarial, and reconstruction—provably separate each image into shared content and domain-specific content, matching the ground-truth factors and enabling guided translation…
desk verdict The empirical method is a real step forward; the formal theorem overclaims, because assumption (12) does the work and no loss enforces it. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the encoder–decoder decomposition $E(x) := (E_c(x), E_{s_A}(x), E_{s_B}(x))$ with decoder $G$, trained by $L = L_{zero} + \lambda_1 L_{adv} + \lambda_2 L_{recon}$. The mechanism is the information-theoretic argument that these three loss families are both sufficient and necessary: the zero and adversarial losses cap what the separate and shared encoders may store, the reconstruction loss forces the encoding pair to retain all the information of the input, and the intersection assumption then forces the learned $E_c$ to coincide informationally with the ground-truth common factor $e_c$, with the separate encoder following by symmetry. The proof hinges on the data-processing inequality and the entropy identity $H(a) = H(e_c(a)) + H(e_{s_A}(a))$, which convert the three losses into the bound $I(E_c(a); E_{s_A}(a)) \leq \epsilon$.
What would settle it
Run the method on a synthetic pair of domains with known generative factors $z_c$, $z_a$, $z_b$, then compute $I(E_c(a); E_{s_A}(a))$ and check whether decoding $E_{s_A}(a)$ alone recovers $z_c$. If the mutual information exceeds $\epsilon$ or the shared factor can be read out of the separate code, the claimed disentanglement is falsified because it violates the paper's Assumption (12), which no loss term enforces.
Extended reading notes
Core claim
The central discovery is that disentangling shared from domain-specific content does not require paired data, architectural bottlenecks, or a subset relation between domains; three losses enforce it. The zero loss caps the separate encoders, the adversarial loss caps the shared encoder by matching the shared-code distributions, and the reconstruction loss guarantees no information needed for the input is lost. The formal result (Theorem 2 in the appendix) states that if the ground-truth representation is an intersection (the shared ground-truth code carries the maximum possible common information among all invertible splits), the learned separate code satisfies $H(E_{s_A}(a)) \leq H(e_{s_A}(a)) + \epsilon$, and the reconstruction and adversarial losses are minimized, then $I(E_c(a); E_{s_A}(a)) \leq \epsilon$, $E_c(a)$ is a function of $e_c(a)$, and $H(E_c(a)) \geq H(e_c(a)) - \epsilon$. Hence the learned shared and separate encoders capture the same information as the ground-truth shared and separate factors, which is exactly what makes guided translation, intersection images $G(E_c(x), 0, 0)$, and union images $G(E_c(x), E_{s_A}(a), E_{s_B}(b))$ possible.
Load-bearing premise
The load-bearing premise is that the learned domain-specific encoder never carries more information about a source image than the true, unobserved domain-specific factor does; if training produces a separate code that also stores shared content, the theorem's independence and content-recovery conclusions collapse.
Editorial extensions
If this is right
- Guided translation works in both directions: $G(E_c(a), 0, E_{s_B}(b))$ removes A's specific attribute from $a$ and imports B's specific attribute from $b$.
- Images from the intersection of the two domains, never seen during training, can be produced as $G(E_c(x), 0, 0)$.
- Images from the union of the two domains can be produced as $G(E_c(x), E_{s_A}(a), E_{s_B}(b))$, overlaying both specific attributes on one shared identity.
- The common and separate latent codes support independent linear interpolation, so identity and attributes can be blended separately.
- The formulation is symmetric: the same trained model maps either domain to the other, and the framework does not assume that one domain is a subset of the other or that the domain-specific part is only style.
Reading between the lines
- One consequence not drawn in the paper is that the same three-loss recipe could be scaled to more than two domains by adding one separate encoder per new domain while keeping a single shared encoder, which would make the construction a general primitive for domain arithmetic.
- The theorem singles out a testable failure mode: if a trained model fails to remove the source attribute, the likely culprit is the separate-code entropy bound rather than the data-generation model, so measuring the entropy of $E_{s_A}(a)$ on real activations could predict when the method will fail.
- A direct experimental probe would be to run the method on synthetic data with known generative factors and measure the mutual information between learned shared and separate codes; the theorem predicts this is near zero exactly when the separate-code entropy bound holds.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes an unsupervised method for separating the shared content between two visual domains from the content specific to each domain. The model uses three encoders (a common encoder Ec and two separate encoders EsA, EsB) and one decoder G, trained with three losses: a zero loss that forces each separate encoder to vanish on the other domain, an adversarial loss that matches the distributions of the common encodings of the two domains, and reconstruction losses. The central theoretical claim, stated in the abstract and in Theorem 1/2, is that these constraints are sufficient and necessary to make Ec capture the ground-truth shared factor ec and the separate encoders capture the domain-specific factors esA and esB, enabling guided translation and generation of the domain intersection and union. The experimental section evaluates the method on CelebA attribute transfer, latent-space interpolation, intersection and union generation, MNIST-to-SVHN domain adaptation, and ablations, with comparisons to several baselines.
Significance. If the theoretical claim were valid, the paper would make a significant contribution: it would provide a symmetric, unsupervised solution to guided image translation, support generation of domain intersection and union without training samples, and offer a principled alternative to style-only transfer methods. The empirical study is extensive and credible: it includes quantitative comparisons with several baselines, a user study, an ablation study, and a domain-adaptation benchmark, and the authors release their code. However, the formal guarantee is currently not established; the proof relies on an unenforced assumption about the learned encoder and contains an invalid invertibility step. As a result, the paper's contribution is at present empirical rather than theoretical, and the advertised 'sufficient and necessary constraints' claim is not supported.
major comments (3)
- [Appendix C, Theorem 2, Eq. (12)] The assumption H(EsA(a)) ≤ H(esA(a)) + ε is not implied by any of the proposed losses. The zero loss only forces EsA(b)=0 on B, the adversarial loss only forces PEc(A)=PEc(B), and the reconstruction loss only forces (Ec(a), EsA(a)) to decode to a. In the generative model of Sec. 2, define Ec(x)≡c0, EsA(a)=(esA(a), ec(a)) on A, EsA(b)=0 on B, EsB(b)=(esB(b), ec(b)) on B, EsB(a)=0 on A, and let G be the inverse of g on the corresponding slots. Then Lzero=0, Lrecon=0, and the adversarial objective is minimized because PEc(A) and PEc(B) are both point masses at c0; yet H(EsA(a))=H(esA(a))+H(ec(a)) violates Eq. (12), and the conclusions I(Ec(a);EsA(a))≤ε and H(Ec(a))≥H(ec(a))−ε fail since Ec carries no shared content. Thus the advertised sufficiency of the proposed losses does not follow unless Eq. (12) is enforced by an additional constraint, a regularizer, or an architectural restriction.
- [Appendix D, construction of g′] The proof defines g′(êc(a), êsA(a), 0)=g(ec(a), esA(a)) and then asserts that invertibility of g implies invertibility of g′. This is invalid: g′ ignores the added components Ec(a) and EsA(a), so it is not injective. For example, two tuples that differ only in the Ec component but have the same ec map to the same image. Consequently Definition 1 cannot be applied to g′, and the step H((ec(a), Ec(a)))≤H(ec(a)) is unjustified. This invalidates the proof that Ec(a) is a function of ec(a) and hence the second conclusion of Theorem 2.
- [Abstract and Section 4] The phrase 'sufficient and necessary constraints' is stronger than what is shown. Theorem 2 is a conditional sufficiency statement under assumptions that include the maximal-entropy intersection condition (Definition 1), exact loss minimization, and the unenforced entropy bound Eq. (12). No necessity direction is proved, and the ablation study in Sec. 5.4 even suggests that the zero loss is not necessary. The claims in the abstract and Section 4 should be softened to describe a conditional guarantee, and the status of Eq. (12) should be stated explicitly as an additional assumption on the optimization outcome rather than a consequence of the losses.
minor comments (5)
- [Theorem 2 statement] The second expected reconstruction term is written as Eb‖G(Ec(b),0,EsB(b))−a‖1=0; the subtracted term should be b, not a.
- [Section 1.1] There is a typo 'structur' in 'based on the structur of the data', and the text contains the stray fragment 'bf Guided Translation' before the guided-translation paragraph.
- [Section 5.1] In the user-study description, 'identify' should be 'identity' in 'Is the identify of a’s image preserved'.
- [Table 3] The entry '64,5%' uses a decimal comma while all other entries use decimal points; please make the formatting consistent.
- [Equation (2)] The displayed formulas contain an extra closing parenthesis; please normalize the notation so that the arguments of G are unambiguous.
Circularity Check
The paper's central theorem is a conditional identifiability argument, and its main weakness is an unenforced entropy assumption rather than a circular derivation.
full rationale
We find no circular step in which a stated prediction or first-principles result reduces by construction to its own inputs. The load-bearing claim is Theorem 2, which is a conditional identifiability statement: under the generative model of Sec. 2, the maximal-entropy intersection assumption of Def. 1, exact expected reconstruction, the matched-marginal condition PEc(A)=PEc(B), and the entropy bound H(EsA(a)) ≤ H(esA(a)) + ε from Eq. (12), the proof derives approximate independence of Ec and EsA, functionhood of Ec with respect to ec, and the entropy lower bound H(Ec(a)) ≥ H(ec(a)) − ε. The conclusion is not identical to the assumptions by definition, because ec is fixed by the data model while Ec is a learned encoder; Def. 1 supplies only the upper bound H(Ec) ≤ H(ec), and the lower bound is obtained from reconstruction together with (12). The paper does contain self-citations (e.g., [6] and [20]), but none is load-bearing for the theoretical claim, and the proof in Appendix C-D is self-contained rather than resting on an external uniqueness theorem. The genuine weakness is a correctness gap, not circularity: Assumption (12) is imposed on the optimization solution without being implied by any of the proposed losses, and the proof in Appendix D asserts invertibility of the augmented map g′ without adequate justification. These are important validity concerns, but under the strict circularity rules they do not amount to a derivation that is equivalent to its input by construction, so the circularity score is 0.
Assumptions & free parameters
free parameters (3)
- loss weight λ1 =
0.001
- loss weight λ2 =
1
- separate-encoder dimension sep =
25
assumptions (5)
- domain assumption Samples factor as a = g(ec(a), esA(a), 0) and b = g(ec(b), 0, esB(b)) with ec independent of esA, ec independent of esB, and g invertible.
- ad hoc to paper The ground-truth factorization is an intersection: H(ec(a)) is maximal among all invertible factorizations with matching common-code marginals.
- ad hoc to paper The learned separate encoder satisfies H(EsA(a)) ≤ H(esA(a)) + ε.
- domain assumption At the solution, the reconstruction losses vanish exactly and PEc(A) = PEc(B).
- standard math Standard entropy identities and the data processing inequality hold for the image and latent variables.
Cite this review
Pith. "Pith review of Domain Intersection and Domain Difference." pith.science (2026). https://pith.science/paper/KAQP5YZH
@misc{pith2026190811628,
author = {Pith},
title = {Pith review of: Domain Intersection and Domain Difference},
year = {2026},
howpublished = {\url{https://pith.science/paper/KAQP5YZH}},
note = {Machine review of arXiv:1908.11628}
}
read the original abstract
We present a method for recovering the shared content between two visual domains as well as the content that is unique to each domain. This allows us to map from one domain to the other, in a way in which the content that is specific for the first domain is removed and the content that is specific for the second is imported from any image in the second domain. In addition, our method enables generation of images from the intersection of the two domains as well as their union, despite having no such samples during training. The method is shown analytically to contain all the sufficient and necessary constraints. It also outperforms the literature methods in an extensive set of experiments. Our code is available at https://github.com/sagiebenaim/DomainIntersectionDifference.
Figures
Figures from the paper (10 more)
Reference graph
Works this paper leans on
-
[20]
Emerging disentanglement in auto-encoder based unsuper- vised image content transfer
Ori Press, Tomer Galanti, Sagie Benaim, and Lior Wolf. Emerging disentanglement in auto-encoder based unsuper- vised image content transfer. In International Conference on Learning Representations, 2019. 1, 2, 6
work page 2019
-
[1]
Augmented Cy- cleGAN: Learning many-to-many mappings from unpaired data
Amjad Almahairi, Sai Rajeshwar, Alessandro Sordoni, Philip Bachman, and Aaron Courville. Augmented Cy- cleGAN: Learning many-to-many mappings from unpaired data. In ICML, 2018. 2
work page 2018
-
[2]
Cvae-gan: Fine-grained image generation through asymmetric training
Jianmin Bao, Dong Chen, Fang Wen, Houqiang Li, and Gang Hua. Cvae-gan: Fine-grained image generation through asymmetric training. In 2017 IEEE International Confer- ence on Computer Vision (ICCV) , pages 2764–2773. IEEE,
work page 2017
-
[3]
One-sided unsupervised do- main mapping
Sagie Benaim and Lior Wolf. One-sided unsupervised do- main mapping. In NIPS, 2017. 8
work page 2017
-
[4]
Xi Chen, Xi Chen, Yan Duan, Rein Houthooft, John Schul- man, Ilya Sutskever, and Pieter Abbeel. InfoGAN: Inter- pretable representation learning by information maximizing generative adversarial nets. In NIPS. 2016. 2
work page 2016
-
[5]
Thomas M. Cover and Joy A. Thomas. Elements of Informa- tion Theory (Wiley Series in Telecommunications and Signal Processing). Wiley-Interscience, New York, NY , USA, 2006. 11
work page 2006
-
[6]
The role of minimal complexity functions in unsupervised learning of semantic mappings
Tomer Galanti, Lior Wolf, and Sagie Benaim. The role of minimal complexity functions in unsupervised learning of semantic mappings. In International Conference on Learn- ing Representations, 2018. 1
work page 2018
-
[7]
Image-to-image translation for cross-domain disentan- glement
Abel Gonzalez-Garcia, Joost van de Weijer, and Yoshua Ben- gio. Image-to-image translation for cross-domain disentan- glement. In NIPS, 2018. 2
work page 2018
Show all 25 references
-
[8]
Generative adversarial nets
Ian Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. Generative adversarial nets. In NIPS. 2014. 12
2014
-
[9]
A two-step disentanglement method
Naama Hadad, Lior Wolf, and Moni Shahar. A two-step disentanglement method. In Proceedings of the IEEE Con- ference on Computer Vision and Pattern Recognition, pages 772–780, 2018. 2
2018
-
[10]
Non-adversarial mapping with vaes
Yedid Hoshen. Non-adversarial mapping with vaes. In Pro- ceedings of the 32Nd International Conference on Neural In- formation Processing Systems, NIPS’18, pages 7539–7548, USA, 2018. Curran Associates Inc. 8
2018
-
[11]
NAM - unsupervised cross- domain image mapping without cycles or GANs
Yedid Hoshen and Lior Wolf. NAM - unsupervised cross- domain image mapping without cycles or GANs. In ICLR workshop, 2018. 2, 8
2018
-
[12]
Multimodal unsupervised image-to-image translation
Xun Huang, Ming-Yu Liu, Serge Belongie, and Jan Kautz. Multimodal unsupervised image-to-image translation. In ECCV, 2018. 1, 2, 6
2018
-
[13]
Image-to-image translation with conditional adver- sarial networks
Phillip Isola, Jun-Yan Zhu, Tinghui Zhou, and Alexei A Efros. Image-to-image translation with conditional adver- sarial networks. In CVPR, 2017. 2
2017
-
[14]
Learning to discover cross-domain rela- tions with generative adversarial networks
Taeksoo Kim, Moonsu Cha, Hyunsoo Kim, Jungkwon Lee, and Jiwon Kim. Learning to discover cross-domain rela- tions with generative adversarial networks. arXiv preprint arXiv:1703.05192, 2017. 2
2017 arXiv
-
[15]
Fader networks: Ma- nipulating images by sliding attributes
Guillaume Lample, Neil Zeghidour, Nicolas Usunier, An- toine Bordes, Ludovic Denoyer, et al. Fader networks: Ma- nipulating images by sliding attributes. InNIPS, pages 5967– 5976, 2017. 2, 6
2017
-
[16]
Diverse image-to-image translation via disentangled representations
Hsin-Ying Lee, Hung-Yu Tseng, Jia-Bin Huang, Maneesh Singh, and Ming-Hsuan Yang. Diverse image-to-image translation via disentangled representations. In The Euro- pean Conference on Computer Vision (ECCV) , September
-
[17]
Unsupervised image-to-image translation networks
Ming-Yu Liu, Thomas Breuel, and Jan Kautz. Unsupervised image-to-image translation networks. In NIPS. 2017. 2
2017
-
[18]
Coupled generative adversar- ial networks
Ming-Yu Liu and Oncel Tuzel. Coupled generative adversar- ial networks. In NIPS, pages 469–477. 2016. 2
2016
-
[19]
Exemplar guided unsupervised image-to- image translation
Liqian Ma, Xu Jia, Stamatios Georgoulis, Tinne Tuytelaars, and Luc Van Gool. Exemplar guided unsupervised image-to- image translation. arXiv preprint arXiv:1805.11145, 2018. 2
2018 arXiv
-
[21]
From facial parts responses to face detection: A deep learn- ing approach
Shuo Yang, Ping Luo, Chen Change Loy, and Xiaoou Tang. From facial parts responses to face detection: A deep learn- ing approach. In ICCV, pages 3676–3684, 2015. 5
2015
-
[22]
Dual- GAN: Unsupervised dual learning for image-to-image trans- lation
Zili Yi, Hao Zhang, Ping Tan, and Minglun Gong. Dual- GAN: Unsupervised dual learning for image-to-image trans- lation. arXiv preprint arXiv:1704.02510, 2017. 2
2017 arXiv
-
[23]
Unpaired image-to-image translation using cycle-consistent adversarial networkss
Jun-Yan Zhu, Taesung Park, Phillip Isola, and Alexei A Efros. Unpaired image-to-image translation using cycle-consistent adversarial networkss. arXiv preprint arXiv:1703.10593, 2017. 2, 8
2017 arXiv
-
[24]
To- ward multimodal image-to-image translation
Jun-Yan Zhu, Richard Zhang, Deepak Pathak, Trevor Dar- rell, Alexei A Efros, Oliver Wang, and Eli Shechtman. To- ward multimodal image-to-image translation. InNIPS, 2017. 2 A. Additional Guided Translation Results We provide the reverse translation to that given in Fig. 2 of t...
2017
-
[25]
differs in that it doesn’t contain a spectral or instance normalization and that Tanh activation is applied instead of ReLU.sep is the dimension of the sep- arate encoders, set to be 25 for all datasets. The latent discriminator d consists of a fully connected layer of 512 filt...
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.