{"id":"71424641-2710-4572-ab1e-954ce1130865","arxiv_id":"2507.18794","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"CLEAR adds a pair-switching anti-contrastive loss to a VAE that disentangles content from style using only content labels and improves OOD classification.","lead":"CLEAR is a new training objective that splits data into task-relevant 'content' and task-irrelevant 'style' using only content labels, by maximizing mutual information between content and labels while minimizing it for style. On several image datasets, CLEAR-VAE improved classification on unseen style-content combinations and enabled controllable style and content swapping.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The claimed MI-minimization guarantee for PS-SNN is not established: the Appx. A.4 proof assumes E_y[h(y,z)]=1, which the implemented cosine-similarity form of h does not satisfy, so the upper bound does not apply to the actual loss.","rationale":"The reader's weakest_assumption targeted the content/style independence and a posteriori independence in the graphical model (Appx. A.1, Fig. 2a). That is a valid real-data limitation, especially for demographic attributes entangled with pathology. However, the most load-bearing concern for the paper's central claim is internal to the proof: the claimed MI-minimization guarantee for PS-SNN does not hold for the cosine-similarity implementation because the derivation assumes E_y[h]=1, which is true for the log-bilinear density-ratio model but not for exp(cosine_sim/tau). The paper's own Sec. 5 acknowledges the unresolved relationship between supervised contrastive losses and MI, and this gap directly invalidates the 'prove that our anti-contrastive penalty minimizes MI' statement. I agree with the reader's CONDITIONAL verdict: the empirical evidence (swapping, interpolation, gMIG, OOD classification, ablations) is suggestive and the method is easy to implement, so the paper should not be rejected, but the theoretical claim must be either fixed or softened. My concrete test directly checks whether the inequality holds on the paper's own simulation; if it fails, the proof is invalid for the implementation as written.","tokens_in":18618,"tokens_out":2756,"duration_ms":23996,"concrete_test":"Reproduce the Appx. A.4 derivation for the implemented h(z_i,z_j)=exp(cos_sim(z_i,z_j)/tau) and test the claimed inequality on the simulation in Sec. 3.1/Appx. B. At each step with N=1500, compute the KNN estimate of I(y;z) and the PS-SNN loss for the same z samples; if L_PS-SNN < I - log(N) occurs at any step, the bound fails for the implemented loss. Also measure E_y[h(y,z)] for z drawn from the trained encoder on Styled-MNIST; if it differs from 1 by more than 10%, the A.19 normalization is empirically violated and the proof does not carry through.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"Sec. 3.3 and Appx. A.4 claim that L_PS-SNN is an upper bound on I(y; z(s)) - log(N), and that minimizing it therefore minimizes I(y; z(s)). But the derivation in Appx. A.4 (Eqs. A.17-A.23) relies on the replacement E_y[h(y,z)] = 1 (line A.19), which is correct for the log-bilinear density-ratio estimate h = f(y|z)/p(y), but not for the implemented h(z_i,z_j) = exp(sim(z_i,z_j)/tau) with cosine similarity on normalized latents. For that h, E_y[exp(cos_sim(z,y)/tau)] is generally not 1, so L_PS-SNN as implemented is not an upper bound of I(y; z(s)) - log(N), and the theoretical guarantee stated in the abstract and Sec. 3.3 does not apply to the actual objective. Moreover, the final step of Appx. A.4 goes from an MC-1 empirical average to the population mutual information without justifying convergence under q_phi and p(y); the surrogate's structure (a ratio of sums over batch negatives) is not the log-ratio log p(y|z)/p(y) averaged under the joint that defines MI. The paper itself concedes in Sec. 5 that the connection between SNN-style losses and MI 'has not been rigorously established' and that 'a closed-form relationship still remains unclear'; the PS bound inherits exactly this gap at its critical step. Thus the central theoretical claim is unsupported for the implemented loss, even though the empirical results may still stand on their own.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript proposes CLEAR, a weakly supervised framework for learning content and style representations in a VAE when only content labels are available. The method augments a β-VAE objective with a supervised contrastive loss on the content latent subspace and a 'pair-switched' anti-contrastive loss on the style latent subspace, claiming that the pair-switched loss minimizes the mutual information between style latents and content labels. The authors instantiate the framework as CLEAR-VAE and evaluate it on Styled/Colored-MNIST, PACS, CelebA, and Camelyon17-WILDS, reporting qualitative content-style swaps and interpolations, ablations, and improved downstream classification on out-of-distribution style-content combinations.","tokens_in":1595,"tokens_out":1661,"duration_ms":75106,"significance":"If the theoretical claim held, the paper would contribute a simple and practically attractive way to unlearn spurious style-content associations using only content labels. The empirical component is substantial: multiple datasets, comparisons with several MI-minimization baselines, an OOD evaluation protocol, and qualitative swap/interpolation results. However, the proof that the implemented PS loss upper-bounds I(y; z(s)) − log N contains a load-bearing gap, and the manuscript itself concedes in Sec. 5 that the SNN-to-MI connection 'has not been rigorously established.' The practical value of the method may still stand on the OOD classification evidence, but the theoretical contribution, as stated in the abstract and Sec. 3.3, is not supported by the current derivation.","major_comments":[{"comment":"The proof that L_PS-SNN upper-bounds I(y; z(s)) − log N replaces the denominator Σ_j h(y_j, z_i^{(s)}) with (N − 1) E_y[h(y, z_i^{(s)})] and marks this expectation as equal to 1. That normalization holds only if h(y,z) is a valid density-ratio estimate h(y,z) = f(y|z)/p(y). In the implemented loss, Eq. (7) defines h through cosine similarity on normalized latents, h(z_i,z_j) = exp(sim(z_i,z_j)/τ), for which E_y[exp(cos_sim(y,z)/τ)] has no reason to equal 1. Therefore the inequality L_PS-SNN ≥ I(y; z(s)) − log N is not established for the objective actually optimized, and the theorem claimed in Sec. 3.3 does not apply to the implemented PS-SNN loss.","section":"Appx. A.4, Eq. (A.19)"},{"comment":"Even if h were a valid density ratio, the step from the batch average (1/N) Σ_i log h(y_i, z_i^{(s)}) to the population mutual information I(y; z(s)) requires that h is exactly f(y|z)/p(y) and that the samples are drawn i.i.d. from the joint distribution; no such convergence argument is provided. The PS-SNN loss is a ratio of sums over batch negatives, and its expectation is not obviously the log-ratio whose expectation defines MI. The paper's own Sec. 5 states that the SNN/InfoNCE-to-MI connection 'has not been rigorously established,' which directly undermines the proof's final step.","section":"Appx. A.4, Eqs. (A.20)-(A.21)"},{"comment":"The manuscript acknowledges, as a limitation, that supervised contrastive losses such as SupCon and the modified SNN do not have a rigorously established connection to MI, and that 'a closed-form relationship still remains unclear.' Since the PS upper-bound proof in Appx. A.4 depends exactly on this connection, the abstract's claim that PS 'minimizes the Mutual Information between the style attributes and content labels' is stronger than what the manuscript itself supports. The claims should be revised to describe PS as an empirical/approximate regularizer, or the proof must be repaired by using a properly normalized density-ratio critic.","section":"Sec. 5"},{"comment":"The ELBO decomposition and the content/style objective assume that z(c) and z(s) are independent a priori and a posteriori and that style is independent of the content label y. In real medical data, demographic attributes can be causally associated with pathology, so this separation may fail and the unlearning target becomes ill-defined; optimizing the proposed objective could then remove content-relevant information. The authors should explicitly scope the method's validity to settings where the independence assumption is plausible and discuss how to detect or handle violations.","section":"Appx. A.1 and Fig. 2(a)"}],"minor_comments":[{"comment":"The gMIG metric measures exactly the quantities that L(c)_SNN maximizes and L(s)_PS-SNN minimizes, namely the MI between z(c) and y and between z(s) and y. High gMIG therefore partly reflects alignment with the training objective rather than an independent measure of disentanglement; it should be presented as a diagnostic, with the OOD classification results serving as the independent evidence.","section":"Appx. E, Eq. (E.35)"},{"comment":"The derivation switches from an inner product with a class-embedding vector e_k to the sample-sample cosine similarity used in Eq. (7). This transition should be made explicit, including whether e_k is a batch-dependent average and how the final loss uses pairwise sample similarities rather than class embeddings.","section":"Appx. A.3.2, Eqs. (A.13)-(A.14)"},{"comment":"There are several presentation issues: inconsistent spacing in 'V AE', inconsistent dataset naming ('Camelyon17-WILD' vs 'Camelyon17-WILDS'), and an apparent typo in Eq. (A.19) where h(y_j, z_i) should likely be h(y_i, z_i).","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The empirical work is extensive and the central practical claim—that CLEAR-VAE improves OOD classification on unseen content-style combinations—is credible. The main obstacle is the theoretical guarantee: the proof in Appx. A.4 relies on an expectation normalization that the implemented cosine-similarity critic does not satisfy, and the paper itself concedes the MI link is unestablished. I would ask the authors to either repair the proof with a properly normalized density-ratio estimator or remove the theorem and reframe the contribution as empirical; a refusal to do either would make the current abstract misleading. The independence assumption also deserves a more prominent limitation discussion, particularly for the healthcare motivation."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Short version: CLEAR is worth a look for the pair-switching trick and the unseen-combination evaluation, but the theoretical gloss in the abstract and Sec. 3.3 doesn't survive contact with the implemented loss. Don't reject the work over it; ask the authors to demote the guarantee and lean on the empirical story.\n\nWhat's new and good: The PS loss is literally flipping positives and negatives in a supervised contrastive loss, which is a neat, easy-to-implement way to encourage z(s) to be uninformative about y without style labels. The OOD evaluation protocol—train on k styles, test on held-out styles, report relative gains over a CNN baseline—is genuinely useful and makes the empirical claims concrete. The results across Styled/Colored-MNIST, PACS, CelebA, and Camelyon17 are consistent, and the swapping/interpolation figures show real disentanglement. Code link is provided. Also to their credit, Sec. 5 concedes that the SNN-to-MI connection is not rigorously established; the authors know where the weak spot is.\n\nWhere it's soft: The proof. Appx. A.4's upper bound relies on E_y[h(y,z)] = 1, and that holds for a log-bilinear density ratio, not for exp(cos_sim(z_i,z_j)/tau) with normalized latents. So the implemented L_PS-SNN is not shown to upper-bound I(y; z(s)) - log N. The MC-1 estimate to population MI step is also hand-wavy. This is not a nitpick: the abstract says 'we prove', and for the actual objective that proof does not go through. The paper's own limitation statement basically admits this, so the fix is to reframe the theoretical contribution or repair the bound. Also, the disentanglement model assumes z(c) and z(s) are independent a priori and a posteriori, with y determining content and style independent of y. That is fine for MNIST, but for CelebA or Camelyon17, race/gender can be causally entangled with pathology; then the 'style' target is ill-defined. The gMIG metric uses the same content-label signal that guides training, so it is partially circular; the held-out OOD splits are the stronger evidence and should carry the weight.\n\nWho it's for: researchers working on domain generalization, fairness, or robust representation learning. They'll get a useful baseline and a clean evaluation template. It deserves a serious referee, provided the authors either fix the MI bound or explicitly replace it with an empirical claim.","headline":"Pair-switching is a genuinely simple idea and the OOD evaluation is useful, but the paper's central MI-minimization proof does not apply to the implemented cosine-similarity loss.","tokens_in":19518,"tokens_out":2057,"would_cite":true,"duration_ms":23920,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A weakly supervised VAE can separate content from style using only content labels, via a pair-switched anti-contrastive loss that cuts style-label mutual information.","keywords":["representation disentanglement","contrastive learning","mutual information minimization","variational autoencoder","out-of-distribution generalization","weakly supervised learning","domain generalization","unlearning spurious correlations"],"falsifier":"Construct a synthetic dataset in which disease status changes both the content pattern and the style distribution, so no style subspace is truly independent of the label. If CLEAR-VAE can still drive an estimate of $I(y; z^{(s)})$ to zero while preserving reconstruction, the independence assumption is unnecessary; if it cannot, the method's success is limited to separable content-style data.","tokens_in":18427,"feed_emoji":"🧩","tokens_out":8201,"duration_ms":83633,"temperature":0.7,"pith_summary":"The paper proposes CLEAR, a weakly supervised method that separates task-relevant content from task-irrelevant style in a variational autoencoder using only content-label supervision. Its anti-contrastive Pair-Switching (PS) term reverses the roles of positive and negative pairs in the supervised contrastive loss, and the authors prove that minimizing this term minimizes an upper bound on the mutual information $I(y; z^{(s)})$ between style features and content labels. In experiments on styled and colored MNIST, PACS, CelebA, and Camelyon17, CLEAR-VAE swaps and interpolates content and style between samples and improves downstream classification on content-style combinations not seen during training. If correct, a single labeling signal suffices to regularize both latent subspaces, which matters for equitable and generalizable prediction when demographic or hospital-specific style shifts at test time.","feed_headline":"Flipping contrastive pairs makes VAEs unlearn spurious style","feed_subtitle":"One content label powers a loss that cuts style-label mutual information and boosts OOD classification.","key_machinery":"The load-bearing object is the Pair-Switching (PS) loss, an anti-contrastive regularization term built from the same soft-nearest-neighbour contrast used for content. For style vectors it is $L_{\\mathrm{PS-SNN}} = \\frac{1}{N}\\sum_i -\\log \\frac{\\mathrm{neg}_i^{(s)}}{\\mathrm{pos}_i^{(s)}+\\mathrm{neg}_i^{(s)}}$, where same-label style vectors act as positives and different-label style vectors as negatives, so the objective rewards a style vector that looks unrelated to its own label. The proof that this loss upper-bounds $I(y; z^{(s)}) - \\log N$ is what turns a simple pair-flip into a principled unlearning mechanism, and the same contrastive machinery simultaneously maximizes $I(y; z^{(c)})$ for the content subspace.","core_discovery":"The central claim is that a pair-switched contrastive loss on the style subspace drives the style representation to carry no information about the content label. Starting from the graphical model in which $z^{(c)}$ is determined by $y$, $z^{(s)}$ is independent of $y$, and $x$ is generated from both, the authors derive an ELBO whose last term is equivalent to a supervised InfoNCE/soft-nearest-neighbour objective on content. Flipping positive and negative pairs in that objective gives $L_{\\mathrm{PS-SNN}}$, and the paper proves $L_{\\mathrm{PS-SNN}} > I(y; z^{(s)}) - \\log N$, so minimizing the loss pushes the mutual information down. Empirically, the resulting CLEAR-VAE achieves higher group Mutual Information Gap scores and better out-of-distribution classification than VAEs without this term and than variants using other mutual-information minimizers.","pith_inferences":["If the independence assumption holds only approximately, unlearning style could also remove predictive signal when a protected attribute genuinely causes the outcome; a fairness-oriented variant might control the amount of style information removed rather than minimize it unconditionally.","The bound depends on batch size through $\\log N$, so the degree of unlearning should improve with larger batches or memory banks; varying batch size while holding the model fixed would test this directly.","The ELBO derivation is VAE-specific, but the pair-switching loss is defined on any paired latent representation, so the same idea could be grafted onto other generative or contrastive models, though the MI upper-bound proof would need to be re-established.","On clinical data, the learned style subspace could be checked against known demographic or hospital labels to see whether CLEAR removes demographic signal only when it is spurious and preserves it when it is clinically meaningful."],"forward_implications":["Using only content labels, a VAE can learn content and style subspaces that support semantically correct swapping and interpolation between any pair of samples.","Classifiers that read only the content subspace retain accuracy on test data whose content-style combinations were never seen during training, as shown on Camelyon17 tumor detection and other datasets.","The pair-switching loss is non-negative and requires no auxiliary discriminator or alternating adversarial training, so it is a simpler drop-in replacement for other mutual-information minimizers.","Because the loss is built from the same pairs as the contrastive term, adding it to a VAE objective introduces only one extra hyperparameter, with the paper setting the two contrastive weights equal."],"supporting_citations":[{"why":"Supplies the InfoNCE objective and the link between maximizing log-density ratios and maximizing mutual information that CLEAR's contrastive term builds on.","marker":"[36]"},{"why":"Provides the soft nearest neighbor loss that CLEAR modifies into its multi-positive supervised contrastive term and then flips into the PS loss.","marker":"[13]"},{"why":"Contributes the supervised contrastive learning formulation with class-label positive pairs that underlies $L_{SNN}^{(c)}$.","marker":"[21]"},{"why":"Establishes the VAE and ELBO machinery, including the reparameterization trick, on which CLEAR-VAE's objective is built.","marker":"[23]"},{"why":"Supplies the swapping and interpolation evaluation protocol used to demonstrate semantic disentanglement.","marker":"[34]"},{"why":"Provides the multi-level VAE grouping idea and the group-level content/style evaluation that CLEAR extends.","marker":"[5]"},{"why":"Gives the KNN mutual-information estimator used for numerical verification of the PS loss and for computing gMIG.","marker":"[38]"},{"why":"Supplies the total-correlation adversarial MI-minimization baseline that CLEAR-PS is compared against.","marker":"[22]"},{"why":"Defines the mutual-information-gap metric whose group-level adaptation (gMIG) measures content-style disentanglement.","marker":"[7]"}],"fun_headline_variants":["CLEAR anti-contrastive loss unlearns spurious style","Pair-switched contrastive loss cuts style-label mutual info","VAE unlearns style via anti-contrastive penalty","Minimizing content-style mutual info for OOD-robust VAEs","Flipped contrastive pairs make VAEs style-agnostic"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that content and style are independent in the data-generating process, with only content determining the label; if style and content are causally entangled in real data, the ELBO decomposition and the unlearning target become ill-defined.","fun_headline_variants_meta":{"raw":{"variants":["CLEAR anti-contrastive loss unlearns spurious style","Pair-switched contrastive loss cuts style-label mutual info","VAE unlearns style via anti-contrastive penalty","Minimizing content-style mutual info for OOD-robust VAEs","Flipped contrastive pairs make VAEs style-agnostic"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000278,"raw_usage":{"total_tokens":1705,"prompt_tokens":1047,"completion_tokens":658,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":663,"completion_tokens_details":{"reasoning_tokens":565}},"tokens_in":663,"tokens_out":658,"duration_ms":6574,"temperature":1.0,"reasoning_tokens":565,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T14:31:29.623234+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Construct a synthetic dataset in which disease status changes both the content pattern and the style distribution, so no style subspace is truly independent of the label. If CLEAR-VAE can still drive an estimate of $I(y; z^{(s)})$ to zero while preserving reconstruction, the independence assumption is unnecessary; if it cannot, the method's success is limited to separable content-style data.","supporting_citations":[{"cited_title":"Analyzing and improving representations with the soft nearest neighbor loss","cited_arxiv_id":null,"evidence_quote":"Provides the soft nearest neighbor loss that CLEAR modifies into its multi-positive supervised contrastive term and then flips into the PS loss."},{"cited_title":"Disentangling factors of variation in deep representation using adversarial training","cited_arxiv_id":null,"evidence_quote":"Supplies the swapping and interpolation evaluation protocol used to demonstrate semantic disentanglement."},{"cited_title":"Multi-level variational autoencoder: Learning disentangled representations from grouped observations","cited_arxiv_id":null,"evidence_quote":"Provides the multi-level VAE grouping idea and the group-level content/style evaluation that CLEAR extends."},{"cited_title":"Mutual information between discrete and continuous data sets","cited_arxiv_id":null,"evidence_quote":"Gives the KNN mutual-information estimator used for numerical verification of the PS loss and for computing gMIG."}],"review_version":1}