{"id":"c245d0ca-c7a6-4a8a-8ea9-94f196874cbe","arxiv_id":"2502.03123","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"DiD uses a GAN with a difference encoder that maximizes distances between image-change vectors from different latent axes, reporting improved disentanglement metrics on dSprites and 3DShapes.","lead":"A new neural network approach aims to separate the meaningful factors in images, such as shape and color, by making the network push apart the changes caused by different factors. The authors report top scores on two standard synthetic datasets, though the core loss is unbounded and may not truly implement the stated goal.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The DiD objective (Eq. 3) only maximizes separation between difference vectors of different axes; it has no term forcing same-axis differences to be compact or bounded, so it can be maximized while the generator's axes remain semantically entangled.","rationale":"The reader's weakest assumption is essentially correct, but the problem is sharper than an unenforced assumption: Eq. (3) optimizes an objective that is satisfiable by non-disentangled solutions. A WGAN-GP alone can fit the data distribution under any invertible reparameterization of the latent code, and Lenc (Eq. 4) only asks the encoder to invert the generator's own code; neither ties code axes to semantic factors. Thus the model's success, if any, would be an emergent property of the dataset and architecture rather than a consequence of the proposed direct semantic supervision. The paper's central conceptual claim, that DiD 'directly' learns semantic disentanglement by maximizing inter-factor differences, is not supported by the stated loss. The reader also identifies the same structural gap, so the verdict of REJECT remains appropriate; no verdict adjustment is needed.","tokens_in":9992,"tokens_out":5757,"duration_ms":57664,"concrete_test":"After training DiD on dSprites with the reported settings, freeze all networks and compute, over at least 1000 random base codes c per axis a, the intra-axis variance σ_a = Var_c[H(G(c), G(c+δe_a))] and the inter-axis distance μ_ab = E_c[||H(G(c), G(c+δe_a)) − H(G(c), G(c+δe_b))||]. If max_a σ_a is of the same order as min_{a≠b} μ_ab, or if ||v1−v2|| grew without bound during training, then Eq. (3) has not induced same-factor compactness alongside cross-factor separation, and the central semantic-difference mechanism is not what the loss implements.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 3.4, the only training signal for semantic differences is LH = −||v1 − v2||2 (Eq. 3), with v1 = H(G(c), G(c′)) where c′ is c perturbed along axis a1, and v2 = H(G(c), G(c′′)) where c′′ is c perturbed along axis a2. This loss maximizes the distance between one pair from axis a1 and one pair from axis a2. It contains no term that compresses H(G(c), G(c+δe_a)) across different base codes c, no term relating v1 to the actual semantic content of the generated images, and no Lipschitz or norm bound on H. Because ||v1−v2|| is unbounded, H can also trivially increase the loss by increasing its output scale, independent of G. Consequently, the objective can be driven up by H assigning widely separated outputs to the two input pairs while G keeps every latent direction entangled in image space. The assumption in Section 3.2 that the two samples 'differ only in one latent space dimension' holds for the input code by construction, but the semantic factors of the generated images are never constrained to follow the code axes. Maximizing Eq. 3 therefore does not implement the paper's stated prior that same-factor variations are compact and different-factor variations are separated; it only pushes two arbitrary pair-encodings apart. The reported MIG, DCI-D, and SAP values, even if reproducible, would not show that the difference objective causes semantic axis alignment.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Disentanglement in Difference (DiD), a GAN-based method for learning semantically disentangled representations. DiD trains a generator with WGAN-GP, a Difference Encoder that maps pairs of generated images to difference vectors, and a Sample Encoder that reconstructs the latent code. The central idea is to contrast sample pairs generated by perturbing different latent axes, using the loss LH = -||v1-v2||2 (Eq. 3) to maximize the distance between difference vectors of different axes. Experiments on dSprites and 3DShapes report higher MIG, DCI-D, and SAP scores than several VAE- and GAN-based baselines. The paper also presents an analysis showing that total correlation and MIG are not monotonically related in baseline models, and an ablation of the difference module.","tokens_in":10332,"tokens_out":3984,"duration_ms":34468,"significance":"If the proposed objective genuinely enforced the stated prior that same-factor variations are compact and different-factor variations are separated, the idea of directly optimizing semantic differences rather than statistical independence would be a meaningful contribution to unsupervised disentanglement. The paper has some strengths: it evaluates on standard benchmarks, includes an ablation of the difference module, and makes the interesting empirical observation that lower TC does not consistently improve MIG. However, the central objective as formulated does not implement its stated prior, and the quantitative claims in Table 1 are overstated, so the significance of the reported results is not established. The paper does not ship code or provide sufficient implementation details to reproduce the experiments, which further limits its impact.","major_comments":[{"comment":"The loss LH = -||v1 - v2||2 is an unbounded negative Euclidean distance with no margin, no normalization, no Lipschitz constraint on H, and no within-factor compactness term. Minimizing this loss (as implied by Eq. 5) can be trivially driven to -infinity by scaling the output of H, independent of the generator G. Moreover, the objective only separates one pair of difference vectors (one from axis a1 and one from axis a2) at a time; it contains no term that compresses H(G(c), G(c+δe_a)) across different base codes c, and no term relating the difference vectors to the actual semantic content of the generated images. Consequently, maximizing Eq. (3) does not implement the paper's stated principle that same-factor variations are small and different-factor variations are large; it only pushes two arbitrary pair encodings apart. The reported disentanglement metrics, even if reproducible, cannot be attributed to the contrastive objective as written.","section":"Section 3.4, Eq. (3)"},{"comment":"The paper states in Section 3.2 that 'it is assumed that within each sample pair, the two samples differ only in one latent space dimension, while remaining consistent across other factors.' This assumption holds by construction for the input code c, but it is not enforced for the generated images: the loss does not align the generator's axes with the semantic factors of the dataset. If a single input axis mixes multiple semantic factors, the Difference Encoder can maximize inter-axis distances without achieving factor disentanglement. The paper provides no mechanism, such as a reconstruction or classification term on semantic factors, that would force the generator to map each latent axis to a single factor. The central claim therefore relies on an unenforced and untested assumption.","section":"Section 3.2 and Section 3.4"},{"comment":"The claim that DiD 'consistently achieves significantly higher SAP, MIG, and DCI-D scores compared to all baseline methods' is contradicted by Table 1. On dSprites, DiD's SAP is 0.17 ± 0.02, which is lower than FactorVAE's 0.19 ± 0.03 and only equal to InfoGAN-CR within error; on DCI-D, DiD's 0.58 ± 0.06 overlaps with FactorVAE's 0.55 ± 0.05. No statistical significance tests are reported, so 'significantly higher' is unsupported. The claim should be corrected to reflect the actual numbers, and the discussion should acknowledge where DiD does not dominate.","section":"Section 4.4, Table 1"},{"comment":"The Sample Encoder E is trained exclusively on generated samples G(c) with the loss Lenc = ||c - E(G(c))||2_2. The paper does not specify how E is applied to real dataset images when computing the disentanglement metrics (MIG, DCI-D, SAP), which are typically computed on the encoder's representations of the training data. If real images are encoded, there is a domain shift because E has only seen generated images; if generated images are used, the metrics may be biased because the codes c are known. The experimental protocol for metric computation is not described, making the quantitative results non-reproducible.","section":"Section 3.5 and Section 4.3"}],"minor_comments":[{"comment":"The terminology is inconsistent: the module is called 'Difference Encoder' in Section 3.4 and the abstract, but 'Difference Estimator' in Figure 1 and Section 4.6. Please unify.","section":"Throughout"},{"comment":"The sign of LH is confusing: the text says 'Our objective is to maximize the distance, d(v1, v2)', but Eq. (3) defines LH as the negative distance. Since Eq. (5) sums LH into a loss to be minimized, the intended effect is to maximize ||v1-v2||; this should be stated explicitly to avoid ambiguity.","section":"Section 3.4, Eq. (3)"},{"comment":"Several free parameters are not specified: the perturbation step size δ along latent axes, the number of dimensions used for comparison in the Difference Encoder, and the relative weights of LH and Lenc in Eq. (5). Without these, the experiments cannot be reproduced.","section":"Section 4.3"},{"comment":"The paper refers to 'Differentiable Information Disentanglement (DiD)' in the first sentence of Section 4.1, but the acronym was defined as 'Disentanglement in Difference' in the abstract and introduction. This appears to be a typo.","section":"Section 4.1"},{"comment":"The ablation study varies the number of comparison dimensions, but the x-axis label is not readable in the figure, and the text does not state the default number used in Table 1. Please clarify the experimental setup.","section":"Section 4.6, Figure 3"}],"recommendation":"reject","confidential_remarks":"The core objective in Eq. (3) does not implement the paper's stated disentanglement prior, and the quantitative claims in Table 1 are not supported by the reported numbers. These are load-bearing issues that cannot be fixed by local revisions; the method would need to be redesigned. Additionally, the lack of implementation details and code makes the results non-reproducible. I recommend rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Worth a look mainly as a case study. The paper's conceptual starting point is correct: statistical independence of latent variables is not the same as semantic disentanglement, and most existing methods optimize the former while claiming the latter. The DiD design—a WGAN-GP with a Difference Encoder that maps paired generated images to difference vectors and maximizes the distance between vectors coming from different latent axes—is not in the cited literature, and the authors deserve credit for trying to move from indirect priors to an explicit difference-based objective. The ablation showing that the difference comparison matters is also useful, and the reported gains on dSprites and 3DShapes across MIG, DCI-D, and SAP are at least internally consistent.\n\nThe soft spot is load-bearing. Equation (3) is LH = −||v1 − v2||2, with no margin, no normalization, no same-factor compactness term, and no Lipschitz bound. The paper's own Section 3.1 and 3.2 state the intended prior: variations from the same factor should be compact and variations from different factors should be separated. But the loss only pushes one pair-difference vector away from another pair-difference vector. Nothing forces the same-axis differences to be similar across different base codes, and nothing forces the generator's axes to align with semantic factors. An encoder with large output scale can inflate the loss without the generator doing anything disentangled. The assumption in Section 3.2 that each generated pair differs in exactly one semantic factor is simply asserted, never enforced; if a latent axis mixes two factors, the model can still maximize the loss while remaining entangled.\n\nThere are also reproducibility gaps: no released code, no stated latent dimension, perturbation step, number of compared dimensions, or metric protocol details. That matters because the empirical claim depends on these choices, and the reported SAP numbers on dSprites are not actually higher than FactorVAE's within error bars, even though the abstract claims superiority. The circularity of training a Sample Encoder to invert the generator's own code is minor by comparison, but it does not help the semantic-interpretation argument.\n\nThe reader's stress-test holds up: the objective can be maximized while the generator stays entangled. This is not a manufactured flaw; it is the central equation failing to match the central claim. The paper is still a serious attempt with a real idea, and a referee could push the authors toward a proper margin, bounded or normalized encodings, same-factor cluster constraints, and more transparent evaluation. I would not cite it as a method, but I would not desk-reject it either.\n\nRecommendation: send it to peer review, but expect the reviewers to demand either a reworked objective or much more evidence that the difference loss actually drives axis-factor alignment.","headline":"The novelty is real but the core objective doesn't implement the paper's stated same-factor/different-factor principle; a genuinely interesting idea undermined by a loss that can be maximized without semantic alignment.","tokens_in":10864,"tokens_out":1530,"would_cite":false,"duration_ms":17711,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A contrastive loss on inter-sample differences, rather than statistical independence, is the direct driver of semantic disentanglement in the proposed DiD method.","keywords":["disentangled representation learning","semantic disentanglement","difference encoder","contrastive loss","WGAN-GP","latent factor separation","dSprites","3DShapes"],"falsifier":"Train DiD on a benchmark where one ground-truth factor is deliberately split across two latent axes or two factors always change together, then measure per-axis factor purity (e.g., SAP per dimension). If the model maintains high MIG while no single axis predicts a single factor better than chance, then maximizing inter-axis difference does not by itself yield semantic disentanglement.","tokens_in":1685,"feed_emoji":"🧩","tokens_out":2559,"duration_ms":74156,"temperature":0.7,"pith_summary":"This paper sets out to show that statistical independence of latent variables is the wrong optimization target for disentanglement: reducing total correlation does not reliably improve semantic factor separation. It proposes Disentanglement in Difference (DiD), a WGAN-GP whose extra loss terms make the model directly contrast variations caused by different latent axes. A Difference Encoder maps pairs of generated images to difference vectors, and a contrastive term maximizes the Euclidean distance between difference vectors coming from different axes; a Sample Encoder reconstructs the latent code from each generated image. Together these terms are meant to organize the latent space so that each axis controls one semantic factor, and the reported experiments on dSprites and 3DShapes give DiD higher MIG, DCI-D, and SAP scores than the VAE- and InfoGAN-based baselines tested.","feed_headline":"Pairwise difference loss beats VAE disentanglement baselines","feed_subtitle":"DiD’s contrastive encoder separates semantic factors directly, topping MIG, DCI-D, and SAP on dSprites and 3DShapes.","key_machinery":"The load-bearing mechanism is the Difference Encoder $H$, realized as an MLP, which turns an image pair into one difference vector. The contrastive loss $L_H=-\\lVert H(g_1)-H(g_2)\\rVert_2$ amplifies the separation between variations caused by different latent axes; the WGAN-GP generator and discriminator provide a high-quality sample distribution over a bounded uniform latent space $c\\sim U[-1,1]^n$; and the Sample Encoder $E$ with MSE reconstruction loss $\\lVert c-E(G(c))\\rVert_2^2$ ties generated samples back to latent codes. The whole system is optimized so that the generator's axes become directions of single-factor variation.","core_discovery":"The central discovery claimed here is that semantic disentanglement can be learned directly from inter-sample differences, without any penalty on total correlation or mutual information. Define three latent codes $c$, $c'$, $c''$ where $c'$ moves a small step from $c$ along axis $a_1$ and $c''$ moves the same step along axis $a_2$; generate image pairs $g_1=[G(c),G(c')]$ and $g_2=[G(c),G(c'')]$; then train a Difference Encoder $H$ to maximize $-\\lVert H(g_1)-H(g_2)\\rVert_2$ while a Sample Encoder $E$ is trained with $\\lVert c-E(G(c))\\rVert_2^2$. The paper argues that this contrastive difference objective directly encodes the prior that different semantic factors should be far apart in representation space, and reports that on dSprites and 3DShapes the method consistently achieves significantly higher SAP, MIG, and DCI-D scores compared to all baseline methods.","pith_inferences":["A natural next test is whether a learned or normalized metric on difference vectors behaves better than raw Euclidean distance; when factors have very different variability, raw Euclidean distance can be dominated by the highest-variance axis, a case the paper does not examine.","The same difference-contrastive prior could be applied to video or multi-view sequences, where temporal or viewpoint changes provide natural same-factor pairs, extending the method beyond synthetic generative benchmarks.","The paper's TC-versus-MIG scatter suggests a mathematical gap between statistical independence and semantic disentanglement, but it does not formalize it; a formal condition stating when inter-axis difference maximization implies factor-wise alignment would settle the theoretical status of the method."],"forward_implications":["If the contrastive difference objective is what produces disentanglement, then disentanglement can be optimized as a first-class objective rather than as a side effect of a KL or TC penalty.","The method extends to any generative model with a continuous latent space and a differentiable generator, since the difference encoder operates on image pairs rather than on dataset labels.","DiD also gives an operational definition of semantic difference: two factors are disentangled when their induced difference vectors are far apart, which could be used as a training signal in settings where ground-truth factors are unknown.","The ablation result implies that without the difference comparison the model reduces to an ordinary WGAN-GP and disentanglement scores drop, so the contrastive term, not the GAN backbone, is the source of the reported gains."],"supporting_citations":[{"why":"Supplies the WGAN-GP sample generation paradigm that produces the images used for difference comparison.","marker":"Gulrajani et al., 2017"},{"why":"Provides the dSprites dataset used to evaluate disentanglement.","marker":"Matthey et al., 2017"},{"why":"Provides the 3DShapes dataset used to evaluate disentanglement.","marker":"Burgess & Kim, 2018"},{"why":"Defines the MIG metric used for quantitative disentanglement evaluation.","marker":"Chen et al., 2018"},{"why":"Defines the DCI disentanglement metric used for evaluation.","marker":"Eastwood & Williams, 2018"},{"why":"Defines the SAP metric used for evaluation.","marker":"Kumar et al., 2017a"},{"why":"FactorVAE baseline whose total-correlation penalty represents the statistical-independence approach DiD contrasts with.","marker":"Kim & Mnih, 2018"},{"why":"Beta-VAE baseline whose KL weighting represents the statistical-independence approach DiD contrasts with.","marker":"Higgins et al., 2017"},{"why":"InfoGAN-CR baseline, the closest contrastive-disentanglement GAN alternative.","marker":"Lin et al., 2020"}],"fun_headline_variants":["DiD: contrastive objective for semantic disentanglement","No more statistical independence: learn semantics directly","Inter-factor differences beat VAE disentanglement baselines","Semantic disentanglement via pairwise differences","Learn disentanglement by maximizing inter-factor differences"],"cache_read_input_tokens":12928,"weakest_assumption_plain":"The method assumes that nudging one raw coordinate of the sampled latent code produces an image pair that differs in exactly one semantic factor; the training loss never checks this, so a latent axis that mixes two factors can still satisfy the contrastive objective without any factor being cleanly separated.","fun_headline_variants_meta":{"raw":{"variants":["DiD: contrastive objective for semantic disentanglement","No more statistical independence: learn semantics directly","Inter-factor differences beat VAE disentanglement baselines","Semantic disentanglement via pairwise differences","Learn disentanglement by maximizing inter-factor differences"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000267,"raw_usage":{"total_tokens":1620,"prompt_tokens":957,"completion_tokens":663,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":573,"completion_tokens_details":{"reasoning_tokens":594}},"tokens_in":573,"tokens_out":663,"duration_ms":5612,"temperature":1.0,"reasoning_tokens":594,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T05:50:20.764534+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train DiD on a benchmark where one ground-truth factor is deliberately split across two latent axes or two factors always change together, then measure per-axis factor purity (e.g., SAP per dimension). If the model maintains high MIG while no single axis predicts a single factor better than chance, then maximizing inter-axis difference does not by itself yield semantic disentanglement.","supporting_citations":[{"cited_title":"and Williams, C","cited_arxiv_id":null,"evidence_quote":"Defines the DCI disentanglement metric used for evaluation."},{"cited_title":"P., Glorot, X., Botvinick, M","cited_arxiv_id":null,"evidence_quote":"Beta-VAE baseline whose KL weighting represents the statistical-independence approach DiD contrasts with."},{"cited_title":"Infogan-cr and modelcentrality: Self-supervised model training and selection for disentangling gans","cited_arxiv_id":null,"evidence_quote":"InfoGAN-CR baseline, the closest contrastive-disentanglement GAN alternative."}],"review_version":1}