{"id":"7cc06521-1d44-4df2-870a-e9e31897c628","arxiv_id":"2412.07979","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":4,"one_line_summary":"AmCLR and xAmCLR combine SogCLR's global contrastive objective with image and text augmentations, reporting small accuracy gains over SogCLR on retrieval and zero-shot tasks.","lead":"A new pair of contrastive loss functions for vision-language models, built on SogCLR, adds image and text augmentations to improve retrieval and zero-shot scores on a 100k subset of CC3M. The claimed gains over SogCLR are 1 to 3 points, but the method details and code are not disclosed.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing augmentation-controlled baseline: SogCLR and iSogCLR are trained without AmCLR's P1/P2 augmentations, so the reported gains may stem from the augmentation distribution rather than the proposed loss.","rationale":"To substantiate the headline claim that AmCLR and xAmCLR outperform SogCLR and iSogCLR, it must be shown that the improvement comes from the proposed objective rather than from trivially adding more augmented views. The paper's own theory does not bridge this gap: Eqs. 16-19 assert that the augmented batch loss has the same expectation as a global loss, and Eq. 20 reuses SogCLR's per-sample u variables without deriving unbiased estimators for anchors that are freshly sampled augmentations. The empirical tables therefore do not isolate the loss. This is a correctable omission: an augmentation-controlled baseline and a precise definition of P2 would settle the question. The reader's conditional verdict is appropriate because the observed numbers could survive the control; the concern strengthens the conditions (release code, specify augmentations, add controls) rather than moving to reject. I therefore leave the verdict unchanged but emphasize that the central empirical claim is currently under-determined.","tokens_in":13545,"tokens_out":12126,"duration_ms":132677,"concrete_test":"Rerun Tables 1-3 with SogCLR and iSogCLR trained on the exact augmented pairs used by AmCLR/xAmCLR: same image transformation pipeline P1, same text paraphrase mechanism P2, same omega=1, and same batch size, epochs, and optimizers, but with the original SogCLR/iSogCLR loss. If these augmented baselines match or exceed the AmCLR/xAmCLR numbers, the claimed advantage of the proposed loss functions is not supported. This requires releasing P1, P2, and training code.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The most load-bearing gap is the absence of any augmentation-controlled comparison. AmCLR and xAmCLR are defined by adding omega=1 random image transformations and text paraphrases (P1/P2) to every sample in Algorithm 1, and the loss sums all kappa combinations (Eqs. 24-31 for AmCLR; Eqs. 50-61 for xAmCLR). The baselines SogCLR and iSogCLR are trained under the same optimizer and batch settings, but nowhere does the paper state that they received the same P1/P2 augmented views; the method description reserves those augmentations for the proposed losses. Thus the comparison in Tables 1-3 changes two things at once: the loss structure and the training data distribution. The reported improvements (e.g., +1.54 Top-1 text retrieval and +2.86 zero-shot Top-1 over SogCLR) could be produced by a standard SogCLR trained on the same augmented pairs, in which case the new loss adds nothing. The paper never specifies P2 (the paraphrase generator) and releases no code, so the augmentation distribution cannot be reproduced or checked. The theoretical claim that Eq. 20 retains SogCLR's unbiased stochastic estimator is also asserted via Eqs. 16-19 without a derivation for augmented anchors, but the missing control is the more direct threat to the central 'outperform' claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes two new contrastive losses for bimodal vision-language representation learning, AmCLR and xAmCLR, built on the SogCLR stochastic global-contrastive framework. AmCLR adds one image augmentation and one text paraphrase per sample and computes cross-modal contrastive losses over all original/augmented combinations. xAmCLR additionally includes intra-modal losses between original and augmented images/texts. The authors train ResNet-50 and DistilBERT encoders on a 100k subset of CC3M with batch size 128 and 30 epochs, and report retrieval and zero-shot results on MSCOCO and ImageNet subsets. The central claim is that AmCLR and xAmCLR outperform SogCLR and iSogCLR when paired with AdamW and AdamP optimizers.","tokens_in":13897,"tokens_out":2697,"duration_ms":27084,"significance":"The proposed loss formulation is a natural extension of SogCLR and could be practically useful if the reported gains are shown to come from the loss itself rather than from the additional augmentations. The paper's strength is that it evaluates with a small batch size (128), which is an important practical direction. However, the evidence is currently incomplete: there is no augmentation-controlled baseline, the theoretical unbiasedness claim is asserted without proof, and the lack of error bars and code hinders reproducibility. These issues are load-bearing for the paper's central 'outperform' claim, so the significance is conditional on additional experiments and analysis.","major_comments":[{"comment":"The comparison is not augmentation-controlled. AmCLR and xAmCLR are defined by applying omega=1 image transformations and text paraphrases (P1/P2) to every sample, and the baselines SogCLR and iSogCLR are trained without these augmented views, as described in Algorithm 1 and Section 3.1. Thus the reported gains (e.g., +1.54 Top-1 text retrieval, +2.86 zero-shot Top-1) could be due entirely to the augmented training distribution rather than the proposed loss structure. To support the claim that the losses are superior, the authors should include SogCLR and iSogCLR trained with the same P1/P2 augmented pairs, holding the data distribution fixed. They should also specify the text paraphrase generator P2, which is never defined, and release the exact augmentation pipeline.","section":"Section 5, Tables 1-3"},{"comment":"The theoretical justification that the stochastic estimator remains unbiased is asserted without proof. Eq. (19) claims F(w) equals an expectation over batches and augmentations, but the augmented data points (x_hat, t_hat) are not elements of the original dataset D, and the expectation over P1/P2 does not trivially reduce to the global loss over D. The sentence 'This holds because the sampling strategy ensures that all data points in D are equally likely to appear' only addresses batch sampling, not the distribution of augmented views. A rigorous derivation is needed, or explicit assumptions under which the equality holds, before the gradient estimator in Eq. (20) can be claimed to retain SogCLR's unbiasedness.","section":"Section 3.1, Eqs. (16)-(19)"},{"comment":"The optimizer selection is post-hoc and introduces selection bias. The paper states that RAdam, NAdam, NvNovograd, and SGDP performed poorly with SogCLR and iSogCLR and were therefore not tested with AmCLR and xAmCLR. Since the authors had already observed the baseline results, this is not a pre-registered experimental plan. The claim 'outperform SogCLR and iSogCLR across all tasks' is thus restricted to optimizers chosen after seeing the baselines. Please report results for all optimizers for the proposed losses, or clearly frame the claim as optimizer-specific rather than general.","section":"Section 4.1 and Section 5"},{"comment":"No error bars or multiple-seed runs are reported. The differences between the proposed methods and the baselines are often small (e.g., Top-1 image retrieval 11.08 vs 10.06; zero-shot 25.61 vs 25.82 for different variants), and without variance estimates it is unclear whether these differences are statistically significant. The paper should include mean and standard deviation over at least three random seeds, or state the number of runs used to produce the tables.","section":"Section 5, Tables 1-3"}],"minor_comments":[{"comment":"Notation is inconsistent: D is used both for the dataset and as the index set in sums, and n is the full dataset size in Eq. (5) but later m is the batch size; please clarify the index sets in each expression.","section":"Section 3.1, Eq. (5) and surrounding text"},{"comment":"Eq. (13) ends with an ellipsis and Eq. (14) writes F_kappa(w; beta) where the argument should be B; this typo appears again in Section 3.2, Eq. (39).","section":"Section 3.1, Eqs. (13)-(14)"},{"comment":"The loop variable n in 'for n = 1...num _batches' conflicts with the dataset size n defined earlier; use a different variable such as t or step.","section":"Algorithms 1 and 2"},{"comment":"The text paraphrase set P2 is never specified. Please describe the generator (e.g., a specific model, back-translation, or rule-based method) and how paraphrases are sampled; otherwise the method cannot be reproduced.","section":"Section 3.1, paragraph on P2"},{"comment":"The link 'https://github.com/AmCLR' in the abstract is not a specific repository and appears to be a placeholder; no code is provided. The paper should include a direct link to the actual code or state that it will be released.","section":"Availability statement"},{"comment":"References [7] and [14] are about dry beans and brain tumor classification and are not relevant to contrastive learning; please replace them with appropriate citations or remove them.","section":"References"},{"comment":"The caption says 'The plot shows the zero-shot learning capabilities', but the figure as rendered appears to be an image file without visible data; please ensure the figure is correctly embedded and readable.","section":"Figure 2"}],"recommendation":"major_revision","confidential_remarks":"The paper has a promising core idea and the experiments are cheap to extend, but the missing augmentation-controlled baseline and the unproven unbiasedness claim are central defects. If the authors can add controlled baselines, specify the augmentation distributions, and provide error bars, the paper could become publishable. I recommend major revision rather than rejection because these issues appear fixable within the manuscript's scope. The author selection and relevance of some references also need attention."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"AmCLR/xAmCLR is a straightforward extension of SogCLR: add one image transform and one text paraphrase per sample, sum the resulting cross-modal contrastive losses, and optionally add intra-modal terms. The exact combination is new as written, and the writing on SogCLR's global objective is clear enough to follow. On the evidence side, the paper shows consistent 1-3 point gains over SogCLR/iSogCLR on a 100k CC3M subset at batch size 128 across retrieval and zero-shot, with two optimizers. That is a useful data point for people who need small-batch contrastive training.\n\nBut the central comparison is under-controlled. SogCLR and iSogCLR are trained without the image transformations and text paraphrases that define AmCLR/xAmCLR. So the tables change two things at once: the loss and the training distribution. The +1.5 to +2.9 point improvements could in principle come entirely from the augmentations, with the extra loss terms adding nothing. The paper needs the obvious baseline: SogCLR trained on the same augmented pairs. That missing control is the load-bearing issue. If it fails, the reported 'outperform' claim is an artifact of augmentation, not the objective.\n\nThe theory section is also asserted rather than proven. Eqs. (16) and (19) claim the expectation of the augmented batch loss equals the dataset loss, but the argument assumes the stochastic estimator stays unbiased when the anchors are randomly transformed; no derivation is given. The paraphrase distribution P2 is never specified at all. No code, no seeds, no error bars. The optimizer story is a little too convenient: RAdam, NAdam, NvNovograd, SGDP are dropped after they look bad on the baselines. That is a minor concern but worth noting. Some references are odd (a dry-beans paper and a brain-tumor paper in a contrastive-learning context), which suggests a loose citation pass, not a substantive issue.\n\nNone of this is fatal to the practical recipe. If you want a small-batch CLIP-like trainer, adding augmentations and summing the losses will likely help, and the paper gives a decent template. But as a scientific claim that these losses improve on SogCLR, the evidence is not yet there.","headline":"A useful small-batch recipe that overclaims its novelty; the missing augmentation-controlled baseline means the loss itself may add nothing beyond standard SogCLR trained on the same augmented pairs.","tokens_in":14374,"tokens_out":2137,"would_cite":false,"duration_ms":22549,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper claims that adding image and text augmentations to the SogCLR contrastive objective improves cross-modal retrieval and zero-shot classification, and that an extended variant with intra-modal alignment does the same.","keywords":["contrastive learning","vision-language pretraining","global contrastive loss","SogCLR","data augmentation","text paraphrasing","zero-shot classification","cross-modal retrieval"],"falsifier":"Run the exact AmCLR training but replace the text-paraphrase generator with random word substitution: if the reported gains disappear while the loss values stay the same, the paraphrase distribution, not the objective, drives the result. More directly, on a fixed validation batch compute the gradient of the augmented loss and the gradient of SogCLR's original global loss; if their angle is consistently large, the augmented objective is not the same global loss the estimator is supposed to optimize.","tokens_in":13383,"feed_emoji":"🖼️","tokens_out":5551,"duration_ms":50393,"temperature":0.7,"pith_summary":"The paper proposes two bimodal contrastive objectives, AmCLR and xAmCLR, built on SogCLR's memory-efficient stochastic optimization of a global contrastive loss. AmCLR adds randomly sampled image transformations and text paraphrases to every image-text pair and sums contrastive losses over all original-augmented combinations. xAmCLR adds intra-modal contrastive terms that align original images with augmented images and original texts with paraphrased texts. The authors claim that both objectives, paired with AdamW or AdamP, outperform SogCLR and iSogCLR on text retrieval, image retrieval, and zero-shot classification, using a batch size of 128 on a 100k subset of CC3M. If the claim holds, it points toward CLIP-like alignment without CLIP's dependence on very large batches.","feed_headline":"Augmented losses beat SogCLR on every tested task","feed_subtitle":"Adding image transforms and text paraphrases lifts Top-1 retrieval and zero-shot accuracy at batch size 128.","key_machinery":"The carrying object is a loss that enumerates all $\\kappa$ combinations of original and augmented image-text pairs; for $\\omega=1$ augmentation per modality, this gives $\\kappa=8$ cross-modal terms in AmCLR and $\\kappa=12$ in xAmCLR once intra-modal terms are added. Each term is an InfoNCE-style contrastive loss in one direction, and the total is optimized with SogCLR's stochastic gradient estimator, where per-sample moving averages $u^I_{i,t}$ and $u^T_{i,t}$ estimate the global partition sums. The paper argues that because the original pairs are sampled uniformly and the augmentations are applied independently, the expectation of the batch loss over batches and augmentations equals the global dataset loss, so the SogCLR estimator remains applicable.","core_discovery":"On the paper's own terms, the central discovery is that augmenting each batch's images and captions, and for xAmCLR also contrasting each modality with its own augmented version, lets a small-batch global-contrastive learner capture more of what a huge-batch contrastive model sees. Concretely, AmCLR with AdamW reaches 14.64% Top-1 text retrieval versus 13.1% for SogCLR, and 25.87% versus 24.28% for zero-shot classification; AmCLR with AdamP reaches 11.46% versus 9.32% for image retrieval. The paper presents these as consistent wins across all tasks and metrics, obtained with the same batch size, epochs, and encoders as the baselines.","pith_inferences":["The paper does not isolate whether the gain comes from the extra loss terms or from the augmentation distribution; a control that applies the same image and text augmentations to SogCLR's original loss would separate those two effects.","The unbiasedness argument in Eqs. (16)-(19) is asserted rather than proved, so if augmentations shift the objective, the reported gains could be a property of the paraphrase generator rather than of contrastive learning.","The authors state they plan to combine their augmentations with iSogCLR's distributionally robust individual temperatures; if the underlying assumption holds, that combination should compound the gains.","Because the experiments use a 100k subset and pretrained encoders, the 1-3 point margins over SogCLR may shrink or invert on full CC3M or when encoders are trained from scratch; the claim should be read as conditional on this small-scale setup."],"forward_implications":["With batch size 128 and 30 epochs on a 100k CC3M subset, AmCLR with AdamW improves top-1 text retrieval by 1.54 points (14.64% vs 13.1%) and zero-shot top-1 by 1.59 points (25.87% vs 24.28%) over SogCLR.","With AdamP, AmCLR improves top-1 image retrieval by 2.14 points (11.46% vs 9.32%) and zero-shot top-1 by 2.86 points (25.08% vs 22.22%) over SogCLR.","xAmCLR's intra-modal terms yield smaller retrieval gains than AmCLR's cross-modal terms but nearly match AmCLR on zero-shot classification, which the paper reads as a sign that intra-modal alignment mainly aids generalization.","Because the method inherits SogCLR's per-sample moving-average estimators, the added loss terms do not require the batch size to grow; the memory overhead stays linear in batch size.","The reported improvements use pretrained ResNet-50 and DistilBERT encoders, so the gain is in fine-tuning cross-modal alignment rather than in learning encoders from scratch."],"supporting_citations":[{"why":"Supplies the stochastic global-contrastive optimization and moving-average estimator that AmCLR and xAmCLR build on.","marker":"[5]"},{"why":"Defines the bimodal contrastive loss and the large-batch baseline the paper positions against.","marker":"[2]"},{"why":"Provides the iSogCLR baseline with individual temperatures that the proposed losses are compared against.","marker":"[6]"},{"why":"Provides the 100k Conceptual Captions subset used for all training runs.","marker":"[8]"},{"why":"The pretrained ResNet-50 image encoder used in every configuration.","marker":"[9]"},{"why":"The pretrained DistilBERT text encoder used in every configuration.","marker":"[10]"},{"why":"One of the two optimizers paired with the proposed losses in the main comparisons.","marker":"[11]"},{"why":"The other optimizer used in the main comparisons.","marker":"[12]"}],"fun_headline_variants":["AmCLR beats SogCLR with augmented contrastive losses","Small batch size, big gains: AmCLR tops SogCLR","Paraphrases and image flips improve cross-modal learning","Augmented contrastive learning wins at batch size 128"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The argument depends on the assumption that augmenting images and paraphrasing texts preserves the global contrastive objective, so SogCLR's small-batch estimator still estimates the intended full-dataset loss; the paper asserts this without proof and does not specify the paraphrase generator.","fun_headline_variants_meta":{"raw":{"variants":["AmCLR beats SogCLR with augmented contrastive losses","Small batch size, big gains: AmCLR tops SogCLR","Paraphrases and image flips improve cross-modal learning","Augmented contrastive learning wins at batch size 128"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000334,"raw_usage":{"total_tokens":1841,"prompt_tokens":917,"completion_tokens":924,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":533,"completion_tokens_details":{"reasoning_tokens":850}},"tokens_in":533,"tokens_out":924,"duration_ms":9156,"temperature":1.0,"reasoning_tokens":850,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T18:19:50.185888+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the exact AmCLR training but replace the text-paraphrase generator with random word substitution: if the reported gains disappear while the loss values stay the same, the paraphrase distribution, not the objective, drives the result. More directly, on a fixed validation batch compute the gradient of the augmented loss and the gradient of SogCLR's original global loss; if their angle is consistently large, the augmented objective is not the same global loss the estimator is supposed to optimize.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the stochastic global-contrastive optimization and moving-average estimator that AmCLR and xAmCLR build on."},{"cited_title":"W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., & others","cited_arxiv_id":null,"evidence_quote":"Defines the bimodal contrastive loss and the large-batch baseline the paper positions against."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the 100k Conceptual Captions subset used for all training runs."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"The pretrained ResNet-50 image encoder used in every configuration."}],"review_version":1}