{"id":"77d4e3b2-1b2a-4b09-b6f4-e192a1a33246","arxiv_id":"2501.02860","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"CO-SSL, which aligns pre-pooling local representations with the global image embedding, reaches 71.5% ImageNet-1K top-1 at 100 epochs and improves robustness over BYOL.","lead":"This paper introduces CO-SSL, a self-supervised learning recipe that trains a network to align small image-region representations with the whole-image representation, and reports 71.5% top-1 accuracy on ImageNet-1K after 100 pretraining epochs. It matters because this simple local-global alignment improves robustness and may make unsupervised visual representation learning more sample-efficient.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Headline ImageNet-1K gain may be confounded by unstated fine-tuned hyperparameters inherited from the authors' BYOL reproduction, not just the CO loss.","rationale":"The reader identified the attribution assumption as the weakest link, and my stress-test converges on the same point but sharpens it: the paper's own Section 3.3 admits hyperparameter changes to the BYOL baseline (two-hidden-layer projection head) and per-method learning-rate tuning, and Section 3.3 admits ImageNet-100-based selection of RF size and other CO-SSL hyperparameters. The strongest evidence for the mechanism is Table 6's ablation on ImageNet-100, which is internally controlled (BYOL vs CO-BYOL with identical backbone), but the headline is ImageNet-1K where the only CO-SSL R50 comparison is 71.4 vs 70.1 and the RF99 comparison is 71.5 vs an untuned-BYOL-with-R50 baseline. The paper's own conclusion that 'CO-BYOL is essential' rests on the ImageNet-100 ablation; the ImageNet-1K result is consistent but not independently verified because configs are withheld and runs are single-seed. I agree with CONDITIONAL because the mechanism is plausible, clearly described, and supported by ablations, but the exact magnitude and even the direction of the headline gain could shift with seed variance or fair hyperparameter matching. The concrete test directly targets that gap: vary only the CO loss weight while holding all other choices fixed, and repeat for seeds. This is feasible because the authors already have the training pipeline in solo-learn. No mathematical error was found in the loss definition or RF-ResNet construction; the concern is purely empirical attribution and reproducibility, which is exactly what a CONDITIONAL verdict should require.","tokens_in":19231,"tokens_out":1945,"duration_ms":37866,"concrete_test":"Release the exact configuration files and training logs for Table 1, then run each ImageNet-1K condition at least three times with different seeds, keeping all hyperparameters fixed except the CO loss (ws=0 vs. ws=0.2), using the same BYOL reproduction (two-hidden-layer projection, tuned LR) as the baseline. If the mean CO-BYOL minus BYOL gap on ImageNet-1K drops below ~0.5 points or overlaps with seed noise, the headline attribution to the CO loss is not supported. Additionally, rerun the ImageNet-100 ablations (Table 6) with the same projection-head change applied to BYOL to verify the 2.8-point gain is not an artifact of the head modification.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim is that the CO-SSL local-global loss drives the ImageNet-1K gains. The headline comparison is CO-BYOL (RF99-R50) at 71.5% vs. 'BYOL (ours)' at 70.1%, yet Section 3.3 reports that the authors modified the BYOL projection head to two hidden layers because it 'works better' and fine-tuned initial learning rate per method. These changes are part of the baseline; Table 1 labels 'BYOL (pub)' as 69.3 but the actual controlled baseline is the authors' 70.1 reproduction, which already embeds possibly nonstandard hyperparameters. More critically, the hyperparameters for CO-BYOL (RF size, ws, cmin, projection architecture) were selected on ImageNet-100, while the baseline BYOL's hyperparameters were not re-tuned on ImageNet-100 in the same way; this creates a selection-asymmetry confound. The ablations in Table 6 (ImageNet-100) show CO-BYOL boosts BYOL by 2.8 points on R50 and 4.0 on RF99-R50, but the ImageNet-1K R50 gain is only 1.3 points (71.4 vs 70.1), suggesting the ImageNet-100-tuned settings do not transfer uniformly. Without releasing configs, exact loss weights, learning-rate schedules, and the BYOL+ reproduction details, the attribution of the remaining 1.4-point gain specifically to co-occurrence learning (rather than to the extra projection heads, the RF99 architecture, or the fine-tuned BYOL baseline) is not established. This is a verification concern, not an accusation of fabrication: the paper explicitly states configs will be released upon acceptance, and single-run results without error bars make the 1.4-point gap fragile.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes CO-SSL, a family of self-supervised representation learning methods that add a local-to-global alignment loss to existing SSL objectives such as BYOL, DINO, and MoCoV3. The local representations are taken before the final pooling layer and are aligned with the global image representation through a separate projection head. The paper also introduces RF-ResNet, a ResNet variant with controlled receptive field sizes, and reports that CO-BYOL with RF99-ResNet50 reaches 71.5% top-1 linear-probe accuracy on ImageNet-1K after 100 pre-training epochs, compared with 70.1% for the authors' BYOL reproduction. Additional experiments address robustness to ImageNet-C corruptions, internal masking, and small PGD attacks, together with ablations on ImageNet-100 and analyses of effective receptive fields.","tokens_in":19598,"tokens_out":6209,"duration_ms":59364,"significance":"If the central attribution holds, the contribution is significant: a simple and general local-global alignment term improves both accuracy and robustness of convolutional SSL, and the RF-ResNet construction provides a useful tool for studying receptive field effects. The paper has clear strengths: it tests three SSL bases, includes ablation studies, analyzes receptive fields explicitly, and offers a mechanistic explanation in terms of redundant local representations. However, the evidence is currently single-run, partly confounded by hyperparameter selection, and the robustness claims are contradicted for the default RF99 architecture in several tables. The significance is therefore conditional on resolving these empirical issues.","major_comments":[{"comment":"The abstract and introduction state that CO-SSL is more robust to noise corruption, internal corruption, and small adversarial attacks, but this is not supported for the default RF99-ResNet50 model. In Table 3, CO-BYOL (RF99-R50) obtains 19.6/18.9/8.0 on Gaussian, Shot, and Impulse noise versus 28.5/26.7/19.2 for BYOL, and is also worse on Defocus, Glass, Motion, and Zoom blur; it is better only on the Mask corruption. In Table 4, CO-BYOL (RF99-R50) is substantially worse than BYOL under several PGD settings, e.g. 18.6 versus 51.7 at epsilon=0.003 with gamma=epsilon/10 and 3.8 versus 27.4 at epsilon=0.01. Similarly, CO-DINO in Table 12 is worse than DINO on internal masking (37.6 versus 30.4). The robustness claims should be restricted to the specific variants and corruption types for which they hold, or the paper should provide an explanation for the RF99-R50 failures; as written, the headline robustness claim is overclaimed.","section":"Section 4.2, Tables 3 and 4, Abstract"},{"comment":"The attribution of the ImageNet-1K gains to the co-occurrence loss is not fully established. Section 3.3 states that the RF size was hyperparameterized on ImageNet-100, that ws was selected in {0.2, 0.5}, that cmin was set to 0.2 for CO-SSL, that a two-hidden-layer projection head was adopted because it 'works better' for BYOL-based methods, and that the initial learning rate was fine-tuned per method; the precise configuration files are withheld until acceptance. The baseline BYOL was not re-tuned on ImageNet-100 in the same way, and the headline 71.5 versus 70.1 comparison includes architecture changes (RF99-R50 versus R50) as well as the extra local heads. The CO-BYOL (R50) result of 71.4 partially addresses the architecture confound, but the selection-asymmetry confound remains. The paper should release the exact configurations, run a baseline tuned under the same protocol, and report multiple seeds with error bars so the 1.3-1.4 point differences can be assessed.","section":"Section 3.3, Tables 1 and 6"},{"comment":"All headline results are single runs without error bars or seed information, and several comparisons mix published numbers with the authors' reproductions under different epochs and training protocols. For example, Table 1 compares 'BYOL (pub)' at 69.3, 'BYOL+' at 70.1, and 'BYOL (ours)' at 70.1, while Table 7 lists 'BYOL (ours)' at 72.4 and 'BYOL (Grill et al., 2020)' at 72.5; the differences among these baselines are not explained beyond the '+ denotes improved reproduction' note. Given that the central claim rests on margins of about one point, the paper should report at least three seeds for the main comparisons and specify the exact protocol for each reproduced baseline. This is a reproducibility and statistical-support issue, not an allegation of fabrication.","section":"Tables 1, 2 and 7; general protocol"}],"minor_comments":[{"comment":"The sentence 'For ImageNet-1K, Tiny-ImageNet, ImageNet-100, we apply hyperparameters provided in the original papers for ImageNet-1K' is ambiguous because it lists ImageNet-100 twice and does not clarify which original paper provides the hyperparameters for each dataset.","section":"Section 3.3"},{"comment":"The table footnote says 'CO-BYOL uses a RF99-ResNet5°'; the degree sign should be a zero, i.e. RF99-ResNet50.","section":"Table 5"},{"comment":"The column header 'RF99-50 (1 head)' should read 'RF99-R50 (1 head)' for consistency with the rest of the paper.","section":"Table 6"},{"comment":"The description of memory calculation is internally inconsistent: it says memory is 'the number of parameters multiplied by the number of input images and four' but then states 'a float32 is encoded with eight memory bytes.' A float32 is four bytes, so the accounting should be corrected and clarified.","section":"Appendix A.4"},{"comment":"The abbreviation 'C/R' is used without definition; it should be defined as crop/resize at first use. Also, there is a missing space in 'cmin' in the Figure 3A caption.","section":"Section 4.3 and Figure 3A"},{"comment":"The claim that CO-BYOL is 'three times more sample-efficient' than BYOL-mc is misleading: BYOL-mc uses six augmented views but those views come from the same two original images, whereas CO-BYOL uses two original images. Sample efficiency is typically measured per unique training image, not per augmented view, and the table does not report unique-image counts.","section":"Appendix A.4 and Table 11"}],"recommendation":"major_revision","confidential_remarks":"The paper is a plausible empirical contribution, and I do not see evidence of fabrication or deliberate concealment. The main concern is that the headline robustness and attribution claims go beyond what the current tables support, and the withheld configurations make the comparisons difficult to verify. With released configs, matched hyperparameter tuning, error bars, and qualified robustness statements, the paper could be suitable for publication. The robustness contradictions in Tables 3, 4, and 12 should be addressed before any accept decision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: this is a solid, honest empirical SSL paper. The concrete contribution—an SSL loss between pre-pooling local representations and the global embedding, plus the RF-ResNet architecture with controlled receptive fields—is well executed, and the ablations show the local loss is doing real work. But the headline ImageNet-1K numbers are more fragile than the prose suggests. Single runs with no error bars, configs withheld until acceptance, and a selection asymmetry (CO-SSL's RF size, ws, and cmin were tuned on ImageNet-100 while the BYOL baseline wasn't) mean the 1.4-point gain over their own BYOL+ reproduction is a plausible effect, not an established one. The DINO baseline in Table 12 also looks oddly weak on corruption robustness, so I'd trust the CO-BYOL comparisons more than the CO-DINO ones.\n\nWhat's genuinely good: the method is simple and clearly described; the loss is explicit; they test it on three SSL bases (BYOL, DINO, MoCoV3); the robustness evaluation (ImageNet-C, internal masking, small PGD) is thorough; and the 1-head ablation in Table 6 partly addresses the extra-capacity confound. The RF-ResNet is a useful controlled way to vary receptive field while matching parameter count. They also handle prior work honestly, citing AMDIM, DetCo, DenseCL, and VICRegL as the closest precedent rather than overclaiming novelty.\n\nThe main soft spot is verification. Releasing configs only on acceptance means no one can currently reproduce the numbers; without multiple seeds, the 1.4-point gap falls within plausible run-to-run variance in SSL. That's a fixable issue, but it should be a condition of acceptance. The selection asymmetry is second-order: it doesn't invalidate the ImageNet-100 ablations, but it makes the headline comparison less clean than a 'BYOL vs CO-BYOL' label implies. Section 4.4's redundancy explanation is correlational; the authors present it as a hypothesis, and that's fine.\n\nWho this is for: SSL researchers working on local-global alignment, dense self-supervision, or robustness. It deserves a serious referee. I'd send it out, asking for code, at least three seeds, and a closer look at the DINO baseline before the numbers enter the literature.","headline":"A solid, honest empirical SSL paper on local-global alignment; the headline ImageNet gain is plausible but the 1.4-point gap over their own BYOL baseline is not yet pinned to the CO loss (single runs, withheld configs, selection asymmetry), so treat the headline numbers as provisional.","tokens_in":20192,"tokens_out":5766,"would_cite":true,"duration_ms":55173,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A local-global alignment objective improves self-supervised category learning and robustness.","keywords":["self-supervised learning","representation learning","spatial co-occurrence","local-global alignment","receptive field","robustness","linear probing","ImageNet-1K"],"falsifier":"A controlled run that adds CO-BYOL's local projection heads and loss weight to BYOL but aligns each local representation with the global representation of a randomly sampled different image should show no gain over BYOL; if it still shows the ImageNet-1K gain, then the improvement is not caused by learning co-occurrence statistics within an image.","tokens_in":19008,"feed_emoji":"🧩","tokens_out":5646,"duration_ms":49633,"temperature":0.7,"pith_summary":"The paper tries to establish that modeling spatial co-occurrences, by aligning the local representations of an image with its global representation, improves self-supervised visual learning. If true, this gives a simple, general principle: make co-occurring visual features similar, and category recognition becomes less sensitive to which features are present. The paper implements this as CO-SSL, instantiates CO-BYOL and CO-DINO, and reports that CO-BYOL reaches 71.5% top-1 linear-probe accuracy on ImageNet-1K after 100 pretraining epochs, 1.4 points above their BYOL reproduction. It also reports improved robustness to ImageNet-C noise, internal masking, and small PGD attacks, explained by the high redundancy of the learned local representations.","feed_headline":"Local-global alignment lifts self-supervised accuracy to 71.5%","feed_subtitle":"CO-BYOL beats its BYOL baseline by 1.4 points on ImageNet-1K and is more robust to noise, masking, and small attacks.","key_machinery":"The load-bearing object is the local-global alignment loss: the global representation of an image is used as a target for the local representations of the same image taken before pooling, with the SSL loss averaged over local positions. RF-ResNet is the complementary architectural mechanism: it bounds the receptive field of local representations (for example, 99 by 99 pixels on 224 by 224 images) so that aligning them with the global view requires learning co-occurrence statistics rather than merging features that already cover the image. The loss weight $w_s$ trades off the standard global loss against the local-global loss.","core_discovery":"The central claim is that aligning local representations of an image with its global representation teaches a model the spatial co-occurrence statistics of visual features, and that this improves category recognition and robustness. The authors introduce CO-SSL, which adds a self-supervised loss between each local representation taken right before average pooling and the global embedding of a differently augmented view. They also introduce RF-ResNet, a convolutional architecture that bounds the receptive field of local representations so that alignment requires learning true co-occurrence statistics. With 100 epochs of pretraining, CO-BYOL on the RF99-ResNet50 reaches 71.5% top-1 on ImageNet-1K; even with a standard ResNet50 it reaches 71.4%, and an averaged bag of patch representations alone reaches 71.2%. Robustness experiments show CO-BYOL with a ResNet50 consistently surpassing BYOL under ImageNet-C noise, internal masking, and PGD attacks, and the paper attributes this to the high similarity among local representations within an image.","pith_inferences":["Our inference: if the co-occurrence principle is general, the same local-global alignment could be applied to masked-image modeling and vision transformers, potentially improving their robustness at small patch sizes.","Our inference: the redundancy explanation suggests a testable prediction that adversarial robustness should scale with the number of local representations aligned per image, up to the point where receptive fields become too large.","Our inference: the biological motivation points to a concrete experiment, measuring whether representations trained with local-global alignment show the same tolerance to partial occlusion as human object recognition."],"forward_implications":["If the central claim holds, any instance-discrimination SSL method can be upgraded with a local-global alignment term, and the CO-MoCoV3 and CO-DINO results suggest the gain is not specific to BYOL.","Category recognition improves with modest receptive fields: the paper finds RF sizes between 67 and 163 pixels work best on 224 by 224 images, so bounded receptive fields are part of the recipe.","Robustness to masking follows directly from redundancy: because many local representations are pulled toward the same global representation, corrupting or removing some of them leaves the global feature mostly intact.","The approach is sample-efficient compared with multicrop methods: CO-BYOL processes two images per iteration yet matches or beats multicrop baselines that consume more images.","A simple averaged bag of small patch representations, with no global integration beyond pooling, already produces strong category accuracy, suggesting that co-occurrence statistics can be learned locally."],"supporting_citations":[{"why":"Provides the BYOL loss and architecture that CO-BYOL extends with the local alignment term.","marker":"Grill et al., 2020"},{"why":"Provides DINO and the multicrop baseline that CO-DINO and the CO-SSL comparisons are built against.","marker":"Caron et al., 2021"},{"why":"Supplies the earlier intra-network contrastive loss approach, AMDIM, that CO-SSL compares to and improves on.","marker":"Bachman et al., 2019"},{"why":"Supplies the receptive-field formula used to design the RF-ResNet family.","marker":"Araujo et al., 2019"},{"why":"Defines the ImageNet-C corruption benchmark used for the robustness evaluation.","marker":"Hendrycks & Dietterich, 2018"},{"why":"Defines the PGD attack used to measure adversarial robustness.","marker":"Kurakin et al., 2018"},{"why":"Provides the BagNet local-feature model used to compare very small receptive fields.","marker":"Brendel & Bethge, 2018"},{"why":"Provides the psychological evidence that humans learn spatial co-occurrences, motivating the objective.","marker":"Fiser & Aslin, 2001"}],"fun_headline_variants":["Local-global alignment lifts SSL to 71.5% on ImageNet","CO-SSL aligns patches with global view, beats baselines","Self-supervised learning via local-global co-occurrence","Aligning local and global features boosts SSL robustness","Patch-to-global contrastive learning reaches 71.5% top-1"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The results depend on attributing the accuracy and robustness gains to the local-global alignment objective itself, rather than to the changed receptive-field size, the extra projection heads, or the hyperparameters that were tuned before the main comparison.","fun_headline_variants_meta":{"raw":{"variants":["Local-global alignment lifts SSL to 71.5% on ImageNet","CO-SSL aligns patches with global view, beats baselines","Self-supervised learning via local-global co-occurrence","Aligning local and global features boosts SSL robustness","Patch-to-global contrastive learning reaches 71.5% top-1"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000161,"raw_usage":{"total_tokens":1217,"prompt_tokens":905,"completion_tokens":312,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":521,"completion_tokens_details":{"reasoning_tokens":223}},"tokens_in":521,"tokens_out":312,"duration_ms":3381,"temperature":1.0,"reasoning_tokens":223,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:01:24.497116+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled run that adds CO-BYOL's local projection heads and loss weight to BYOL but aligns each local representation with the global representation of a randomly sampled different image should show no gain over BYOL; if it still shows the ImageNet-1K gain, then the improvement is not caused by learning co-occurrence statistics within an image.","supporting_citations":[{"cited_title":"Bootstrap your own latent: A new approach to self-supervised learning","cited_arxiv_id":null,"evidence_quote":"Provides the BYOL loss and architecture that CO-BYOL extends with the local alignment term."},{"cited_title":"Adversarial examples in the physical world","cited_arxiv_id":null,"evidence_quote":"Defines the PGD attack used to measure adversarial robustness."},{"cited_title":"Approximating cnns with bag-of-local-features models works surprisingly well on imagenet","cited_arxiv_id":null,"evidence_quote":"Provides the BagNet local-feature model used to compare very small receptive fields."},{"cited_title":"Unsupervised statistical learning of higher-order spatial structures from visual scenes","cited_arxiv_id":null,"evidence_quote":"Provides the psychological evidence that humans learn spatial co-occurrences, motivating the objective."}],"review_version":1}