{"id":"824e992c-fca0-4cc0-922f-194302be8801","arxiv_id":"2412.02896","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"GUESS replaces the zero off-diagonal target in Barlow-Twins with a correlation matrix from autoencoder latents, reporting improved self-supervised accuracy on several benchmarks.","lead":"A new self-supervised visual learning method, GUESS, adds autoencoder-derived uncertainty to the whitening loss and uses an ensemble of blocks. It reports accuracy improvements over several standard baselines, but the paper contains internal inconsistencies and does not release code or error bars.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Loss equation and Algorithm 1 define different objectives, so the reported 'new baseline' claim rests on an ambiguous loss and is not reproducible as stated.","rationale":"The paper's central claim is empirical: GUESS-1 and GUESS-1-E set a new baseline on linear and transfer learning. For that claim to hold, the trained objective must be the one described; otherwise the comparisons do not test the proposed method. The mismatch between Eq. (1) and Algorithm 1 is not a cosmetic notation issue. It changes the fixed point of the off-diagonal term by a factor of β (0.01), so under the pseudocode the 'pseudo-whitening' component is nearly a strict whitener, whereas under Eq. (1) it is a genuinely data-dependent target. Since no code, seeds, or error bars are provided, a reader cannot determine which loss produced the table. This is more fundamental than the reader's weakest assumption about whether the autoencoder cross-correlation is a meaningful uncertainty signal: even if that signal is meaningful, the paper does not currently specify how it enters the loss. The Appendix C claim is also unproven, and the initial motivating experiment reports only a 0.47% gain, but these are secondary. A clear resolution would be to run both objectives and release code. I therefore keep the reader's CONDITIONAL verdict.","tokens_in":17166,"tokens_out":5805,"duration_ms":52888,"concrete_test":"Run the reported GUESS-1 configuration (ResNet50, ImageNet or CIFAR-100 protocol, β = 0.01, α = 0.2) under both objectives with identical hyperparameters and at least 3 seeds. If the Algorithm 1 version reproduces the reported accuracy while the Eq. (1) version does not, then the paper's stated loss is wrong and the headline result corresponds to an unnamed variant; if both reproduce within noise, the mismatch is benign. Additionally, release the training code so the exact loss and the pretraining/freezing schedule of the autoencoders can be inspected.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing issue is that Section III-B's loss and Algorithm 1 are not the same objective. Equation (1) defines Lw = Σ_i(1−C_ii)^2 + βΣ_{i≠j}(C_ij − C1_ij)^2, with C from projector outputs and C1 from autoencoder latents. Algorithm 1 instead computes C1 from the network outputs, C2 from the autoencoder latents, zeroes diag(C2), sets C = eye(D) + βC2, and minimizes (C1 − C)^2 + Lr. Expanded, the pseudocode minimizes Σ_i(1−C1_ii)^2 + Σ_{i≠j}(C1_ij − βC2_ij)^2, while Eq. (1) minimizes Σ_i(1−C_ii)^2 + βΣ_{i≠j}(C_ij − C1_ij)^2. These differ by more than notation: with β = 0.01, the pseudocode drives network off-diagonal correlations toward 0.01·C2_ij, whereas Eq. (1) drives them toward C1_ij with a small weight. The pseudo-whitening mechanism is therefore qualitatively different depending on which expression was actually trained. Since no code or seeds are provided, Table I's claimed one-block gains over Barlow-Twins (e.g., 75.6 vs 73.6 on ImageNet) cannot be assigned to the stated loss, and the central 'new baseline' claim is not checkable. The Appendix C assertion that the surrogate 'maintains the same global optimum' is also unsupported, but the equation/pseudocode mismatch is the first-order blocker: it makes the method under test ambiguous.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes GUESS, a self-supervised learning framework that replaces the strict whitening target of Barlow-Twins with a data-dependent \"pseudo-whitening\" target derived from the cross-correlation of a pair of autoencoder latents, and that combines this loss with an ensemble of blocks, each trained on its own augmented views. The authors report linear-evaluation and transfer-learning results on CIFAR10/100, Tiny ImageNet, ImageNet, VOC0712, and COCO, and claim that GUESS, even with one block, sets a new baseline on several benchmarks. An efficient variant replaces the cross-correlation with an auto-correlation to halve computational cost.","tokens_in":17531,"tokens_out":4851,"duration_ms":83821,"significance":"If the empirical claims hold, GUESS would be a meaningful contribution to non-contrastive SSL: it introduces a concrete mechanism for data-dependent invariance enforcement, evaluates it across six datasets and three downstream tasks, and provides an efficient variant with comparable cost to Barlow-Twins. The paper also ships a substantial ablation study, including sensitivity to beta, robustness to heavy augmentation, and the effect of autoencoder pre-training. However, the central 'new baseline' claim is currently undermined by an inconsistency between the stated loss and the provided pseudocode, by overstatements in the text relative to the reported tables, and by the absence of error bars or multiple seeds. These issues must be resolved before the empirical claims can be accepted.","major_comments":[{"comment":"The loss defined in Eq. (1) and the loss minimized in Algorithm 1 are not the same objective. In Eq. (1), the network cross-correlation matrix C is pulled toward the autoencoder cross-correlation C1, with the off-diagonal mismatch weighted by beta. In Algorithm 1, C1 denotes the network correlation, C2 denotes the autoencoder correlation, and the loss is (C1 - (eye(D) + beta*C2))^2, which expands to sum_i (1 - C1_ii)^2 + sum_{i != j} (C1_ij - beta*C2_ij)^2. These differ in which matrix is the target, in the placement of beta, and in the treatment of diagonal entries. Consequently, the reported 'new baseline' results cannot be attributed to the loss stated in Eq. (1), and the method is not reproducible as written. Please align the pseudocode with Eq. (1), or vice versa, and state explicitly which objective was actually optimized in the experiments.","section":"Section III-B, Eq. (1) and Algorithm 1"},{"comment":"The text claims that 'GUESS-5 outperforms the state-of-the-art' on CIFAR10 when pre-training on Tiny ImageNet, but Table II lists BYOL at 93.17 and GUESS-5 at 92.93 for exactly this setting. Similarly, Section IV-D.1 states that GUESS-1 and GUESS-1-E 'consistently improved over all baselines' on CIFAR100 under linear evaluation, yet Table I shows W-MSE4 at 67.64 and GUESS-1-E at 67.53. These overstatements contradict the paper's own tables and should be corrected, with the narrative aligned to the reported numbers.","section":"Section IV-D.2 and Table II"},{"comment":"No error bars, standard deviations, or multiple seeds are reported for any experiment. Many of the claimed improvements are small (for example, CIFAR100 GUESS-1 67.67 vs W-MSE4 67.64, and Tiny ImageNet GUESS-5 52.33 vs SimSiam 51.66), so single-run numbers are insufficient to support the 'new baseline' claims. Please report mean and standard deviation over at least three independent runs, and state how many seeds were used for the reproduced baselines.","section":"Section IV-C and Tables I-III"},{"comment":"The assertion that the surrogate objective 'maintains the same global optimum' as Eq. (15) is made without proof. This is a nontrivial statement relating a Frobenius-norm surrogate to a log-determinant information-bottleneck objective, and it is especially doubtful because the surrogate uses a non-identity off-diagonal target while the IB objective involves covariance determinants. Either provide a derivation or explicitly label this as a conjecture.","section":"Appendix C, after Eq. (15)"}],"minor_comments":[{"comment":"The motivating experiment on pseudo-whitening reports only a 0.47% gain (50.51% vs 50.04%) over Barlow-Twins, and the Gaussian-noise variant gives 50.11%; given these small margins, the language describing this as supporting 'pseudo-whitening' should be tempered or supplemented with additional evidence.","section":"Section IV-B"},{"comment":"The symbol C is used both for the pseudo-whitening matrix in Algorithm 1 and for the network cross-correlation matrix in Eq. (1); this notation conflict makes the inconsistency between the two formulations harder to detect and should be fixed.","section":"Algorithm 1 and Eq. (1)"},{"comment":"The abbreviation 'NP' in 'GUESS-1 (NP)' is not defined in the table or text; it should be expanded (e.g., 'no autoencoder pre-training').","section":"Table III"},{"comment":"There is a typo 'GUERSS-1' instead of 'GUESS-1'.","section":"Section IV-D.2"},{"comment":"Eq. (2) and Appendix A's Eq. (6) are identical; one of the two should be removed to avoid duplication.","section":"Appendix A and Eq. (2)"},{"comment":"Reference [25] is a podcast rather than a peer-reviewed source; please cite the underlying literature instead. Also, 'Lecunn' should be 'LeCun' in Appendix B.","section":"References and Appendix B"}],"recommendation":"major_revision","confidential_remarks":"The core issue is not the idea but its verification: the loss-function inconsistency means the empirical results cannot currently be attributed to a precisely specified method. If the authors align the pseudocode with the actual trained loss and provide seed-averaged numbers, the paper could become acceptable; as written, the benchmark claims are not checkable. I would also encourage the editor to request the code or at least precise hyperparameter and seed details, since the reported margins are small."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe one genuinely new thing here is the pseudo-whitening target: instead of forcing off-diagonal cross-correlations to zero as Barlow-Twins does, GUESS derives a target from the cross-correlation of a pair of autoencoders run on the same views. That is a plausible way to relax strict decorrelation and let data influence the invariance constraint. The paper also runs a fair number of experiments across classification, detection, and segmentation, and includes ablations on the loss, the beta hyperparameter, and the ensemble size. The consistent improvement of the one-block variant over Barlow-Twins, if real, is the kind of result the SSL community would care about.\n\nBut the paper as written is not reproducible. The loss defined in Eq. (1) and the loss in Algorithm 1 are different objectives. Eq. (1) matches the projector cross-correlation C to the autoencoder target C1 with beta weighting; Algorithm 1 instead matches C1 to eye(D) + beta*C2, which expands to a different expression. With beta = 0.01, the two behave quite differently. Since no code or seeds are provided, Table I's claimed gains cannot be assigned to either objective. The transfer-learning text also overclaims: it says GUESS-5 beats all baselines on CIFAR10 under Tiny ImageNet pre-training, but Table II shows BYOL at 93.17 vs GUESS-5 at 92.93. Appendix C's claim that the surrogate objective 'maintains the same global optimum' as the information bottleneck objective is asserted without proof, and the appendix contains an unfinished placeholder. These are not minor quibbles: the central claim of a new baseline rests on empirical numbers, and those numbers are tied to an ambiguous training objective.\n\nStill, I do not think this should be desk-rejected. The underlying idea is worth testing, and a revised version that fixes the equation/pseudocode mismatch, releases code and seeds, corrects the transfer-learning overclaim, and either proves or removes the IB claim would be a legitimate submission. The initial motivating experiment (relaxed vs strict whitening) is a good sanity check. The paper deserves a serious referee, but as it stands it needs major revision.\n\nRecommendation: send to review, and flag the objective ambiguity as the top-priority issue. I would not cite it in its current form.","headline":"GUESS has a plausible pseudo-whitening idea, but the loss in Eq. (1) differs from the pseudocode's, so the reported 'new baseline' is not tied to a well-defined method.","tokens_in":18063,"tokens_out":3101,"would_cite":false,"duration_ms":28208,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that replacing the identity target in whitening-based self-supervised learning with a data-derived target from autoencoder correlations improves representations and sets new baselines on several benchmarks.","keywords":["self-supervised learning","pseudo-whitening","redundancy reduction","uncertainty representation","ensemble learning","generative-discriminative loss","representation learning","data augmentation invariance"],"falsifier":"Train GUESS-1 on ImageNet with the autoencoder-derived target replaced by a fixed random matrix with the same off-diagonal distribution, keeping all other settings identical; if accuracy stays near GUESS-1's 75.6, the data-dependence of the target is not the cause of the gain, while a drop toward Barlow-Twins' 73.6 would confirm the autoencoder signal is the active ingredient.","tokens_in":16931,"feed_emoji":"🧠","tokens_out":11239,"duration_ms":99565,"temperature":0.7,"pith_summary":"Self-supervised learning usually trains a network to map two augmented views of the same image to similar representations, and recent whitening methods go further by forcing the cross-correlation of the two views' embeddings to be the identity matrix. This paper argues that such strict, blind invariance enforcement discards useful information and can hurt downstream tasks. It proposes GUESS, which instead learns a pseudo-whitening target: the diagonal of the target is still 1, but the off-diagonal entries come from the cross-correlation of two autoencoders fed the same augmented views, making the loss data-dependent and preserving view-specific structure. An ensemble of such blocks, each trained on its own distorted views, outperforms or stays competitive with Barlow-Twins, Whitening-MSE, SimCLR, BYOL, SimSiam, and SwAV across four classification benchmarks, with the largest gains at ensemble sizes three and five. If the claim holds, uncertainty-aware invariance enforcement is a practical and general alternative to aggressive whitening in self-supervised learning.","feed_headline":"Data-derived targets beat strict whitening in self-supervised learning","feed_subtitle":"Autoencoder-driven pseudo-whitening pushes ImageNet linear accuracy to 75.6, above Barlow-Twins' 73.6.","key_machinery":"The generative uncertainty block is the unit that carries the argument. It couples a twin encoder-projector pair with a twin autoencoder pair: the encoders produce the representation to be learned, while the autoencoders, trained for reconstruction on the same augmented views, produce a data-dependent uncertainty signal. The cross-correlation of the autoencoder latents, with its diagonal set to 1, replaces the identity matrix as the target in the whitening loss, so the loss no longer forces off-diagonal correlations to zero and instead lets the data decide what is redundant. The second machinery piece is the ensemble: $M$ such blocks, each with its own augmentation stream and its own reconstruction-trained autoencoders, whose classifiers vote at test time. The efficient variant reuses one branch and an auto-correlation target to halve the compute, turning a fixed decorrelation constraint into a per-sample, per-distortion constraint.","core_discovery":"On the paper's own terms, the central discovery is that the optimal whitening target in redundancy-reduction self-supervised learning is not the identity matrix but a data-dependent matrix shaped by the input distortions. For each block, GUESS computes two cross-correlation matrices: one from the two encoder-projector branches and one from the latent spaces of two autoencoders. The loss $L_w = \\sum_i (1-C_{ii})^2 + \\beta \\sum_{i \\neq j} (C_{ij} - C_{1,ij})^2$ pulls the encoder cross-correlation toward a target whose off-diagonal entries come from the autoencoders rather than from zero, so the optimizer is uncertain about which correlations are redundant. The architecture wraps this block in an ensemble of $M$ such blocks, each fed its own augmented views, and combines their classifiers by majority vote at test time; an efficient variant replaces cross-correlation with auto-correlation, halving the compute. The paper reports that even the one-block version outperforms all baselines on ImageNet linear evaluation with 75.6 top-1 accuracy, and that three- and five-block ensembles push accuracy further on the linear-evaluation benchmarks.","pith_inferences":["A direct extension the paper leaves implicit: the same autoencoder-derived target could be plugged into other redundancy-reduction or contrastive losses, and testing that transfer would reveal whether the mechanism is tied to the Barlow-Twins-style loss or is a general property of data-dependent whitening.","The Appendix C claim that the surrogate objective \"maintains the same global optimum\" as the information-bottleneck objective is asserted without proof; a formal derivation or a counterexample would settle whether the gains come from a principled objective change or from optimization-side effects.","If the autoencoder target is truly measuring augmentation-induced uncertainty, then its quality should degrade smoothly with autoencoder capacity and pre-training duration, which would make the framework a candidate for online or adaptive uncertainty estimates rather than fixed pre-trained targets."],"forward_implications":["If GUESS is correct, the identity matrix should not be treated as the universal whitening target; a data-derived pseudo-whitening target is a direct drop-in replacement for the decorrelation term in whitening-based SSL losses.","The ensemble scaling results imply that adding independently augmented blocks generally improves linear-evaluation accuracy, at least up to the tested size of five.","The efficient ensemble results imply that the improvement does not require the doubled compute of the full architecture: GUESS-1-E stays within about 0.1 to 0.2 accuracy points of GUESS-1 while adding roughly no computational overhead over Barlow-Twins.","The heavy-augmentation ablation implies that the data-dependent target is more forgiving than deterministic invariance: under the heavy augmentation protocol GUESS-3 keeps 50.11 top-1 accuracy on Tiny ImageNet, where SimSiam falls to 44.11.","The ablation without autoencoder pre-training implies that the uncertainty target must be learned before it is used: skipping the 250-epoch pre-training drops Tiny ImageNet accuracy from 50.91 to 48.16 and slows convergence."],"supporting_citations":[{"why":"Defines the soft-whitening baseline whose identity-matrix objective GUESS replaces, and supplies the cross-correlation formulation the loss builds on.","marker":"[7]"},{"why":"Hard-whitening baseline; provides the whitening-MSE loss and the evaluation protocol GUESS follows on all benchmarks.","marker":"[19]"},{"why":"Non-contrastive baseline (BYOL) that GUESS compares against in linear, KNN, and transfer evaluations.","marker":"[16]"},{"why":"Non-contrastive baseline (SimSiam) used for comparison, for the convergence table, and for the heavy-augmentation robustness check.","marker":"[17]"},{"why":"Contrastive baseline (SimCLR) whose augmentation and hyperparameter protocol GUESS adopts for training.","marker":"[14]"},{"why":"Clustering-based baseline (SwAV) that provides a strong state-of-the-art comparison across datasets.","marker":"[32]"},{"why":"Open-source reproduction library used to generate the baseline numbers reported in the comparison tables.","marker":"[41]"},{"why":"Heavy augmentation protocol used in the robustness ablation showing GUESS degrades less than SimSiam.","marker":"[42]"},{"why":"Information-bottleneck formulation that the theory appendix invokes to position pseudo-whitening as an IB instantiation.","marker":"[49]"}],"fun_headline_variants":["Uncertainty injection beats strict whitening in self-supervised learning","Ensemble of uncertain blocks lifts SSL linear accuracy to 75.6","Data-driven whitening targets outperform identity in SSL","GUESS: letting SSL doubt its targets boosts ImageNet accuracy","Pseudo-whitening with autoencoder targets sets new SSL baseline"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the cross-correlation of two autoencoders' latent vectors is a meaningful and stable measure of augmentation-induced uncertainty; if that matrix mostly reflects reconstruction artifacts or training noise, the pseudo-whitening target carries no useful signal and the reported gains would come from the ensemble or from regularization alone.","fun_headline_variants_meta":{"raw":{"variants":["Uncertainty injection beats strict whitening in self-supervised learning","Ensemble of uncertain blocks lifts SSL linear accuracy to 75.6","Data-driven whitening targets outperform identity in SSL","GUESS: letting SSL doubt its targets boosts ImageNet accuracy","Pseudo-whitening with autoencoder targets sets new SSL baseline"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000746,"raw_usage":{"total_tokens":3369,"prompt_tokens":1033,"completion_tokens":2336,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":649,"completion_tokens_details":{"reasoning_tokens":2262}},"tokens_in":649,"tokens_out":2336,"duration_ms":16887,"temperature":1.0,"reasoning_tokens":2262,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T22:58:45.967133+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train GUESS-1 on ImageNet with the autoencoder-derived target replaced by a fixed random matrix with the same off-diagonal distribution, keeping all other settings identical; if accuracy stays near GUESS-1's 75.6, the data-dependence of the target is not the cause of the gain, while a drop toward Barlow-Twins' 73.6 would confirm the autoencoder signal is the active ingredient.","supporting_citations":[{"cited_title":"Whitening for self-supervised representation learning,","cited_arxiv_id":null,"evidence_quote":"Hard-whitening baseline; provides the whitening-MSE loss and the evaluation protocol GUESS follows on all benchmarks."},{"cited_title":"solo-learn: A library of self-supervised methods for visual representation learning","cited_arxiv_id":null,"evidence_quote":"Open-source reproduction library used to generate the baseline numbers reported in the comparison tables."},{"cited_title":"Directional self-supervised learning for heavy image augmentations,","cited_arxiv_id":null,"evidence_quote":"Heavy augmentation protocol used in the robustness ablation showing GUESS degrades less than SimSiam."}],"review_version":1}