{"id":"3e6c8ccf-5088-440a-b666-09193d77601f","arxiv_id":"2502.07281","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"SCBD uses supervised contrastive losses to disentangle target-correlated features from environment-correlated features, improving out-of-distribution accuracy on Camelyon17 and batch correction on a 26M-cell optical pooled screen.","lead":"Researchers introduce Supervised Contrastive Block Disentanglement (SCBD), a contrastive-learning method that learns two separate representations: one for the signal of interest and one for spurious variations like hospital or well effects. The method is tested on medical images and 26 million cell images, where it improves out-of-distribution accuracy and removes batch effects, though the headline results rely on choosing a key setting using the test set.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Table 1's claimed superiority relies on test-set-selected α; under the paper's own validation selection rule α=0 is chosen and SCBD falls below ERM, so the central domain-generalization claim is not yet established.","rationale":"I read the paper in good faith. The algorithm is clearly specified, the invariance loss is novel and simple, and the code in Appendix B is a useful independent aid. There is no formal verification, but that is not the decisive issue. The decisive issue is exactly the one the reader identified: the comparison in Table 1 is protocol-unfair. The paper itself concedes that selection by validation picks α=0, which makes the α=192 row an oracle-selected result rather than a practical model-selection outcome. This does not impugn the method's internal consistency or the honesty of the authors, who explicitly disclose the limitation, but it does mean the headline 'SCBD significantly outperforms all baselines' is conditional on access to the target distribution. The batch-correction experiment is more robust because its evaluation is in-distribution, though it still uses a hand-picked α and does not compare with the closely related Tu et al. (2024); I would not change the reader's conditional verdict on that basis. My proposed test directly settles whether the protocol concern changes the table: if validation-based selection of α recovers α=0, the central domain-generalization claim fails as stated.","tokens_in":21411,"tokens_out":6038,"duration_ms":56406,"concrete_test":"Reproduce Table 1 under a single model-selection rule: include α in the same validation-based selection used for baselines (tune α ∈ {0,64,128,192} together with learning rate and weight decay on the validation split) on CMNIST and Camelyon17-WILDS with ten seeds. If the selected α is 0 and SCBD's test accuracy is near Table 1's α=0 row (25.5 and 61.9), below ERM (37.8 and 65.8), the concern lands. As a complementary check, give baselines the same oracle treatment (select their hyperparameters by test accuracy) and see whether any overtakes SCBD at α=192.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing weakness is the evaluation protocol in Section 4.1.6 / Table 1. The paper states 'We do not optimize α during model selection, since this would result in choosing α = 0' and then reports the α=192 row as the headline. All baseline hyperparameters, by contrast, are selected on the in-distribution validation set. So the table compares SCBD whose α is chosen with access to the test distribution against baselines chosen without such access. This is not an internal inconsistency—the paper explicitly flags that tuning α is an open problem—but it makes the central empirical claim 'SCBD significantly outperforms all baselines' conditional on an oracle choice. Following the paper's own selection protocol selects α=0, and Table 1 then shows SCBD (25.5±3.0 CMNIST, 61.9±3.8 Camelyon17) below ERM (37.8±2.6, 65.8±4.9). The invariance mechanism itself may be sound, and the batch-correction experiment is less affected because its evaluation is in-distribution, but the claimed domain-generalization superiority is not established by the reported comparison.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Supervised Contrastive Block Disentanglement (SCBD), a method that learns two embeddings, zc and zs, from observations x. A supervised contrastive loss clusters zc by target y, a second supervised contrastive loss clusters zs by environment e, and a new invariance loss is added to make zc invariant to e; an optional additive decoder provides reconstruction for qualitative inspection. The method is evaluated on Colored MNIST and Camelyon17-WILDS for domain generalization and on a 26-million-cell Optical Pooled Screening dataset for batch correction. The headline empirical claims are that SCBD with alpha=192 outperforms all baselines on both domain-generalization datasets and that SCBD removes inter-well batch effects while preserving biological signal.","tokens_in":21583,"tokens_out":7139,"duration_ms":64541,"significance":"Block disentanglement with a purely contrastive objective is a potentially useful contribution, and the paper is transparent about the central difficulty of tuning alpha: it explicitly states that validation-based model selection would choose alpha=0. The algorithm is simple, the code is provided in Appendix B, and the Optical Pooled Screening experiments are on a large real-world dataset. If the empirical claims held under a protocol-fair comparison, the method would be practically valuable. However, the main quantitative support for domain generalization is not protocol-fair, and the batch-effect metric largely measures the property that the loss already optimizes, so the strength of the stated claims is currently not commensurate with the evidence.","major_comments":[{"comment":"The central claim that 'SCBD with alpha=192 significantly outperforms all baselines on both CMNIST and Camelyon17-WILDS' is not established by a protocol-fair comparison. The paper states that alpha is not optimized during model selection because that would choose alpha=0, and then reports the alpha=192 row as the headline result; thus the reported SCBD test accuracy is selected with access to the test distribution, while all baseline hyperparameters are selected on the in-distribution validation set. Under the paper's own validation-selection rule, alpha=0 is selected and SCBD achieves 25.5±3.0 on CMNIST and 61.9±3.8 on Camelyon17, below ERM (37.8±2.6 and 65.8±4.9, respectively). In addition, Appendix Table 5 restricts all non-SCBD baselines to batch size 32, while SCBD uses batch size 2048, which is a further confound and is particularly relevant for contrastive methods. The comparison should be rerun under a common model-selection protocol, or the claims should be explicitly reframed as an oracle-alpha analysis.","section":"Section 4.1.6 / Table 1 / Appendix Table 5"},{"comment":"The well-F1 metric trains a linear classifier on zc to predict the well label e, which is essentially the same property that the invariance loss Linv is explicitly designed to eliminate. A low well-F1 for SCBD is therefore partly by construction and does not provide independent evidence of batch correction; the CORUM prediction result is the independent part of the evaluation. The authors should state this caveat and, ideally, add a metric that is not directly aligned with the training objective, such as the degree of well mixing in a nearest-neighbor graph or the preservation of within-well perturbation structure.","section":"Section 4.2.3 / Figure 4"}],"minor_comments":[{"comment":"The sentence 'Our approach differs from theirs because we do not use contrastive learning' contradicts the paper's central claim that SCBD is based purely on supervised contrastive learning; presumably the intended contrast is that SCBD does not use a variational autoencoder framework.","section":"Section 3"},{"comment":"Equation (2) defines the invariance loss with sums over positive and negative pairs, but the code in Appendix B computes means over these sets and then averages over the batch; the notation should be aligned with the implementation.","section":"Section 2 / Appendix B"},{"comment":"The word 'significantly' is used without a statistical significance test; given the large standard deviation on CMNIST (82.9 ± 12.1), the authors should report confidence intervals or a paired test across the ten seeds.","section":"Table 1"},{"comment":"The text describes the relationship as 'monotonic', but only four values of alpha are tested; it would be clearer to say that the trade-off is observed over the tested values, rather than claiming monotonicity in general.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The main issue is protocol fairness in Table 1. The authors are transparent about the alpha selection problem, but the framing overclaims: the validation-selected alpha=0 result is below ERM, and the alpha=192 result is test-selected. I would ask for a protocol-fair comparison (e.g., validation-selected alpha as the primary result, with the oracle-alpha curve presented as an upper bound) and for a clearer caveat on the batch-correction well-F1 metric. The Section 3 sentence about contrastive learning appears to be a typo and should be corrected. The core idea is interesting and the OPS experiments are substantive, so the paper is worth a revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The key thing to know: the invariance loss in Eq. 3 is genuinely new, and the idea of using the difference between positive and negative SCL log-probabilities to make zc uninformative about e is clean and avoids adversarial training. The two-block pipeline is simple and well motivated. The paper also does something right that many don't: it states plainly that ID and OOD performance often positively correlate, that SCBD only helps in the negative trade-off regime, and that tuning alpha is an open problem. The qualitative counterfactual swap on CMNIST is a nice sanity check.\n\nThe soft spot is exactly where the stress-test note lands. Table 1 compares SCBD at alpha=192 against baselines selected by validation. The paper itself says that if alpha were selected by validation, you'd get alpha=0, at which point SCBD is below ERM on both CMNIST (25.5 vs 37.8) and Camelyon17 (61.9 vs 65.8). So the headline 'SCBD significantly outperforms all baselines' holds only under an oracle choice of alpha with access to the test distribution. The authors are transparent about this, but the presentation still leads with the alpha=192 row. That's a load-bearing flaw in the domain-generalization claim as stated.\n\nThe batch-correction experiment is more convincing because the evaluation is in-distribution and the comparison to CellProfiler is meaningful. But it also uses a hand-picked alpha=1, and the well-F1 metric is a linear classifier on the exact embedding that the loss was trained to make uninformative about e -- standard in this literature, but you should read it with that in mind. The missing comparison to Tu et al. (2024), a supervised-contrastive disentanglement method on related cell data, is a gap, not a fatal one.\n\nIs the paper serious? Yes. The math is straightforward, the loss is novel, the limitations are acknowledged, and the batch-correction result is a real empirical contribution. The central domain-generalization claim, however, is not yet established by the reported protocol. A fair revision would either provide a validation-based or nested procedure for selecting alpha, or reframe the contribution as 'invariance control' rather than 'outperforms baselines'.\n\nWho gets value: anyone working on contrastive invariance, batch correction, or block disentanglement. I'd send it to review, with the protocol issue as the main revision request. I'd probably cite it for the loss and the batch-correction result.","headline":"Genuinely new contrastive invariance loss, but the domain-generalization superiority claim needs a protocol-fair alpha selection; the batch-correction result is the stronger evidence.","tokens_in":22265,"tokens_out":2113,"would_cite":true,"duration_ms":17356,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that a single contrastive invariance loss, scaled by one hyperparameter α, can block-disentangle content from environment and beat domain-generalization baselines on CMNIST and Camelyon17-WILDS while removing batch…","keywords":["block disentanglement","supervised contrastive learning","invariance loss","domain generalization","batch correction","spurious correlations","single-cell perturbation","out-of-distribution generalization"],"falsifier":"A direct check would be to select α by validation accuracy on a held-out environment (or on the training hospital split) and then compare that SCBD checkpoint with ERM under the same selection rule; if SCBD at the selected α no longer beats ERM on Camelyon17-WILDS, the reported superiority depends on the test-informed choice α=192 rather than on the invariance loss itself.","tokens_in":21084,"feed_emoji":"🔬","tokens_out":8760,"duration_ms":74673,"temperature":0.7,"pith_summary":"SCBD learns two embeddings from the same images: one that tracks the target variable y, and one that absorbs environment-specific variation e. Its novel piece is a supervised-contrastive invariance loss that makes the content embedding unable to distinguish same-environment from different-environment examples, so spurious correlations have nowhere to hide. A single scalar α sets how hard that invariance is enforced; increasing α improves out-of-distribution accuracy at the cost of in-distribution accuracy. On CMNIST and Camelyon17-WILDS, SCBD at α=192 beats ERM, CORAL, DANN, IRM, Fish, and Group DRO, and on a 26-million-cell optical pooled screen it preserves more biological signal while being less sensitive to inter-well batch effects than the strongest morphology baseline. If these results hold, practitioners get a simple contrastive alternative to adversarial or variational invariance methods.","feed_headline":"Contrastive loss beats all baselines on two hard domain shifts","feed_subtitle":"At α=192 it beats ERM and six others on two benchmarks, then cleans 26M-cell batch effects.","key_machinery":"The load-bearing object is the invariance loss $L^{\\mathrm{inv}}_{z_c,e}$, built from the same softmax matching probabilities as supervised contrastive learning. For each anchor, it compares the summed log-probabilities of matching zc to same-environment pairs versus different-environment pairs and takes the absolute difference; minimizing it drives the two sums together, which prevents any linear readout of e from zc. It shares the spirit of adversarial domain discrimination but removes the minimax optimization, and it is combined with two supervised-contrastive terms, one clustering zc by y and one clustering zs by e (or by the pair (y,e)), plus an optional additive decoder that lets the user swap zc or zs to inspect counterfactual images.","core_discovery":"The central claim is that the invariance of the content embedding zc to the environment e can be enforced by a purely supervised-contrastive term rather than by adversarial training or variational regularization. The loss treats same-environment pairs of zc as positives and different-environment pairs as negatives, then minimizes the absolute gap between their summed matching log-probabilities; this makes the environment unpredictable from zc while the two supervised contrastive terms keep zc informative about y and zs informative about e. The paper reports that α=192 achieves 82.9±12.1% test accuracy on CMNIST and 72.7±3.0% on Camelyon17-WILDS, above all listed baselines, and that α=1 on the batch-correction task outperforms the baselines on both measures of biological content and of residual sensitivity to wells.","pith_inferences":["Beyond the paper: the same α-controlled invariance loss could be applied to a single contrastive embedding (with y as the target) as a drop-in debiasing regularizer for pretrained features, though the paper only studies the two-block setting.","A testable extension the paper leaves implicit is to choose α by leave-one-environment-out validation, for example training on four hospitals and picking α on a fifth, to check whether SCBD still beats ERM under a fully target-free selection rule.","Because the invariance loss is symmetric in the pair labels, it may be reusable to enforce invariance to any categorical nuisance variable such as donor, scanner, or stain whenever such labels exist, not only the well or hospital labels used here."],"forward_implications":["Domain generalization gains a single-knob method: raising α monotonically trades in-distribution accuracy for out-of-distribution accuracy on datasets where the two are negatively correlated.","Batch correction can reuse the same objective: zc preserves perturbation-related biology while the ability to predict the well from zc drops, which is what the paper demonstrates on 26 million cell images.","The absence of a discriminator and of variational bounds removes two common failure modes, minimax instability and posterior collapse, making invariance training more stable with standard image encoders.","On datasets where in-distribution and out-of-distribution performance are positively correlated, the paper's own experiments on PACS and VLCS show SCBD has no advantage over ERM, so the method's applicability is conditional.","The optional reconstruction view, by swapping zc and zs, gives a direct check that the two blocks carry distinct information."],"supporting_citations":[{"why":"Supplies the supervised contrastive learning objective, the temperature setting, and the batch-size scaling that SCBD reuses for clustering zc by y and zs by e.","marker":"Khosla et al. (2020)"},{"why":"Introduces Colored MNIST and the IRM baseline; SCBD extends the CMNIST data-generating process with a graded color-digit correlation.","marker":"Arjovsky et al. (2019)"},{"why":"Provides the Camelyon17-WILDS benchmark and the leaderboard baseline numbers the paper compares against.","marker":"Koh et al. (2021)"},{"why":"Establishes ERM as the hard-to-beat baseline and frames the model-selection problem that motivates the α-tuning discussion.","marker":"Gulrajani & Lopez-Paz (2021)"},{"why":"Documents the negative in-distribution and out-of-distribution correlation on Camelyon17-WILDS that SCBD relies on.","marker":"Teney et al. (2024)"},{"why":"Defines the adversarial domain-discrimination approach whose fool-the-discriminator strategy the invariance loss replaces without minimax optimization.","marker":"Ganin et al. (2016)"},{"why":"Provides the hand-engineered morphology feature baseline that SCBD must beat on batch correction.","marker":"Carpenter et al. (2006)"},{"why":"Contributes the 26-million-image Optical Pooled Screen dataset used for the batch-correction experiments.","marker":"Funk et al. (2022)"},{"why":"Underlies the iVAE baseline that SCBD is compared against as a variational block-disentanglement approach.","marker":"Khemakhem et al. (2020)"},{"why":"Supplies the multi-contrastive VAE baseline and the protein-complex evaluation protocol for perturbation embeddings.","marker":"Wang et al. (2023)"}],"fun_headline_variants":["Supervised contrastive block disentangles spurious env, boosts OOD","SCBD: contrastive loss enforces env invariance, beats baselines on both tasks","No adversarial: SCBD uses contrastive pairs to kill env bias, wins on two benchmarks","α=192: contrastive block beats ERM on CMNIST and Camelyon17","Block disentanglement via supervised contrastive: cleans 26M-cell batches"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported gains assume α can be picked without seeing the target distribution, yet the chosen α=192 is exactly what a validation-set protocol would not select, since the paper states that optimizing α during model selection would pick α=0.","fun_headline_variants_meta":{"raw":{"variants":["Supervised contrastive block disentangles spurious env, boosts OOD","SCBD: contrastive loss enforces env invariance, beats baselines on both tasks","No adversarial: SCBD uses contrastive pairs to kill env bias, wins on two benchmarks","α=192: contrastive block beats ERM on CMNIST and Camelyon17","Block disentanglement via supervised contrastive: cleans 26M-cell batches"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000265,"raw_usage":{"total_tokens":1619,"prompt_tokens":967,"completion_tokens":652,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":583,"completion_tokens_details":{"reasoning_tokens":544}},"tokens_in":583,"tokens_out":652,"duration_ms":6560,"temperature":1.0,"reasoning_tokens":544,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-08T13:14:06.621975+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A direct check would be to select α by validation accuracy on a held-out environment (or on the training hospital split) and then compare that SCBD checkpoint with ERM under the same selection rule; if SCBD at the selected α no longer beats ERM on Camelyon17-WILDS, the reported superiority depends on the test-informed choice α=192 rather than on the invariance loss itself.","supporting_citations":[],"review_version":1}