{"id":"9fc2741b-1835-4003-83b7-c4bbd7782020","arxiv_id":"2501.16360","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":2.0,"correctness_risk":"high","formal_verification":"none","parameter_count":5,"one_line_summary":"A proposed MoCo variant with a symmetric dual-view loss and 'hard negative' filtering by farthest cosine distance, whose reported superiority is contradicted by its own experimental table.","lead":"This paper modifies the MoCo contrastive learning method with a symmetric query-key loss and a negative sampling rule that keeps only the most distant memory-bank samples as negatives. The reported accuracy gains are contradicted by the paper's own comparison table, which shows several existing methods outperforming the proposed one.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper's own Table II contradicts its central superiority claim: MoHN is 86.32% while BYOL, FASTSIAM, and DCL are 91.0%, 90.2%, and 87.9%, so the claimed 'highest Top-1 Accuracy' is numerically false in the same table.","rationale":"I read the paper as making an empirical claim: an enhanced MoCo with dual-view loss and farthest-negative filtering beats prior self-supervised methods on CIFAR-10/100. The load-bearing condition is that the reported comparison is internally consistent and that the proposed variant outperforms its controls. Table II violates that condition on its face: the proposed method is ranked below BYOL, FASTSIAM, and DCL in the same table, and the surrounding text gives three different numbers for the same CIFAR-10 result. The paper's other innovation, farthest-negative selection, is described as 'hard negative filtering' but is the opposite of the hard-negative definition in the cited work (Robinson et al., closest negatives), and no ablation shows it helps. None of this requires assuming bad faith; it is visible from the manuscript's own equations and tables. The reader's REJECT verdict is therefore unchanged. I mark agreement as partial because the reader's weakest_assumption emphasizes the theoretical inversion of hard negatives, whereas my primary load-bearing concern is the self-contradictory evidence for the headline result; the two points reinforce each other but are not identical.","tokens_in":6006,"tokens_out":4571,"duration_ms":43845,"concrete_test":"Re-run the CIFAR-10 protocol described in Sections III.B and III.E with a fixed ResNet-18 and the stated augmentations, and evaluate three variants under identical epochs and seeds: (a) standard MoCo v2 InfoNCE with the full memory bank, (b) the dual-view loss of Eq. (3) with the full bank, and (c) the dual-view loss with the farthest-20% subset of Eq. (4). Also report BYOL and MoHN from the same shared evaluation script with the same memory-bank size, batch size, and epoch count. The concern lands if variant (c) does not beat (b), or if (b) does not beat (a); the internal contradiction is settled if the reproduced BYOL and MoHN numbers differ from 91.0 and 86.32, which would show the table mixed incompatible settings.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim in the Abstract and Section III.E.1 is that the proposed MoHN framework achieves superior downstream performance. That claim stands or falls on the CIFAR-10 linear-evaluation numbers in Table II. The table lists MoHN (OUR) at 86.32%, while BYOL is 91.0%, FASTSIAM is 90.2%, and DCL is 87.9%; the sentence immediately below the table, 'we achieve the highest Top-1 Accuracy of 86.32%,' is numerically inconsistent with the same table. The inconsistency is not limited to wording: Table I reports 87.56% for MoHN on CIFAR-10, Table II reports 86.32%, and a later paragraph reports 86.23% against a BYOL figure of 86.8% that does not match Table II's 91.0%. No single consistent set of numbers supports the superiority claim. The second contribution is also conceptually inverted: Section II.B.2 correctly describes Robinson et al. [8] as selecting the closest negatives to the anchor as hard negatives, but Eq. (4) then filters for features 'farthest from q' and calls this hard-negative filtering. No ablation compares farthest filtering to closest filtering or to no filtering; Table III only gives a single 20%-subset configuration on CIFAR-10/100. Thus the empirical base for the central claim is self-contradictory, and the one novel mechanism is not supported by any control comparison.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes MoHN, an extension of MoCo that combines a dual-view InfoNCE loss, weighting query and key terms, with a 'hard negative filtering' strategy that selects a subset of memory-bank features farthest from the query in cosine similarity. The authors report linear-evaluation accuracies on CIFAR-10 and CIFAR-100 and claim superior downstream performance with lower GPU memory use than existing self-supervised methods.","tokens_in":6328,"tokens_out":4069,"duration_ms":39927,"significance":"If the empirical claims were consistent and the negative-selection mechanism were validated, the dual-view loss and selective negative sampling could be a modest but useful contribution to MoCo-style contrastive learning. The paper provides pseudocode, a small subset-fraction experiment, and GPU-memory measurements. However, the central claims are contradicted by the paper's own tables, the accuracy numbers are internally inconsistent, and the proposed hard-negative rule is not tested against any control, so the significance of the contribution cannot be assessed from the manuscript as written.","major_comments":[{"comment":"Table II reports MoHN at 86.32%, while BYOL, FASTSIAM, and DCL are listed at 91.0%, 90.2%, and 87.9%. The sentence immediately below the table, 'we achieve the highest Top-1 Accuracy of 86.32%,' is therefore false within the same table. The abstract's and Section III.E.1's central claim of superior performance is not supported by the paper's own data.","section":"III.E.1, Table II"},{"comment":"The proposed 'hard negative filtering' selects features farthest from q, but the text correctly summarizes Robinson et al. [8] as selecting negatives closest to the anchor, which is the opposite strategy. No experiment compares farthest, closest, and unfiltered negatives; Table III only reports a single 20%-subset configuration. The second contribution is therefore neither conceptually aligned with its stated motivation nor empirically validated.","section":"II.B.2, Eq. (4)"},{"comment":"The reported CIFAR-10 accuracy for MoHN appears as 87.56 in Table I, 86.32 in Tables II and III, and 86.23 in the text, while the text compares MoHN with a BYOL figure of 86.8% that does not match Table II's 91.0%. CIFAR-100 results also differ: 60.1 in Table I versus 58.24 in Table III. No consistent set of numbers supports any stated accuracy claim.","section":"Tables I-III and Section III.E.1"},{"comment":"The pseudocode computes negative_query_logits by calling select_top_negatives(k, memory_queue), i.e., using the key feature k to select negatives, whereas Eq. (4) states that the filtered set FN consists of features farthest from q. If the implementation follows the pseudocode, the implemented loss is not the one derived in Section II.B.2; if it follows Eq. (4), the pseudocode is wrong. Either way, the method as written is not reproducible.","section":"Algorithm 1, Step 6"}],"minor_comments":[{"comment":"The sentence 'typically set close to 1 (e.g., 0.99) to ensure slow u' is cut off and incomplete, and the text contains stray artifacts such as 'kjThis paper' and 'jk # Step 2' that should be removed.","section":"II.A.1"},{"comment":"The sentence 'Empirical studies suggest that mm performs best within the range [0.1, 0.01]' contains a typo ('mm'), presents the interval in descending order, and no ablation for the loss weight m is reported anywhere in the experiments.","section":"II.B.1, Eq. (3)"},{"comment":"MoCo is listed as 87.33 in Table I and as 84.7 in Table II under MoCo*; the asterisk indicates a different memory-bank setting, but the relationship between the two entries should be stated explicitly so readers can assess comparability.","section":"Tables I-II"},{"comment":"The comments 'Compute loss for keys' and 'Compute query logits and loss' appear to be swapped relative to Eq. (3): Step 5 computes the standard q-view term and Step 6 computes the k-view term. The naming should match the equations.","section":"Algorithm 1"},{"comment":"The fine-tuning description says the model is initialized with pretrained weights from ImageNet; since the self-supervised pretraining is on CIFAR, it should be clarified whether ImageNet initialization is used and whether this affects the comparison with other methods.","section":"III.D"},{"comment":"The abstract says the strategy 'emphasizes the most challenging negatives,' but the method in Eq. (4) selects the farthest negatives, which is the opposite of the closest-negative definition of hard negatives used by Robinson et al. The terminology should be aligned or corrected.","section":"Abstract and II.B"}],"recommendation":"reject","confidential_remarks":"The manuscript has the form of an early preprint with substantial presentation issues. The central numerical claim is contradicted by Table II, and the accuracy values are inconsistent across tables and text; these are load-bearing problems that cannot be fixed by local editing. A rejection is appropriate unless the authors re-run the experiments and provide a consistent set of results plus a proper control for the negative-selection rule."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This one has a load-bearing problem: the numbers do not support the central claim, and the one novel mechanism is inverted relative to the literature it cites. The paper combines two known ideas—a symmetric dual-view InfoNCE loss (standard in SimCLR) and selecting a subset of negatives. The only twist is choosing the farthest negatives, which is the opposite of Robinson et al.'s hard negatives. That is not a new useful technique; it is a misreading of the prior work.\n\nCredit where due: the paper gives a clean pseudocode algorithm, reports GPU memory usage, and runs standard CIFAR benchmarks. The writing is clear in places, and the authors did not hide their hyperparameters.\n\nThe soft spots are fatal. Table II lists BYOL at 91.0, FASTSIAM at 90.2, DCL at 87.9, and the proposed MoHN at 86.32, yet the sentence below the table claims \"we achieve the highest Top-1 Accuracy of 86.32%.\" That is false in the same table. The reported CIFAR-10 accuracy changes three times: 87.56 (Table I), 86.32 (Table II/III), 86.23 (text). The text's comparison with BYOL (86.23 vs 86.8) does not match Table II's BYOL 91.0. No single consistent set of numbers supports the superiority claim. Second, the paper correctly describes Robinson et al. as selecting closest negatives as hard, but Eq. (4) filters for features farthest from the query. No ablation compares farthest vs closest vs none; only a single 20%-subset experiment on CIFAR-10/100 is reported. The second contribution is therefore unsupported and likely counterproductive. The loss weight and subset fraction are tuned on the evaluation datasets, and no code or data are provided. There are also garbled passages—\"kjThis\" in the intro, \"mm performs best\" in the method—that signal editorial sloppiness.\n\nThis is not a paper for a serious referee round. The central empirical claim is refuted by the paper's own table, and the conceptual contribution is a step backward. The authors could fix the negative-selection direction and run proper ablations, but as it stands the paper's internal inconsistency is too deep. Recommendation: desk reject.","headline":"The paper's own Table II contradicts its headline accuracy claim, and its 'hard negative filtering' selects the farthest negatives, the opposite of the hard-negative literature it cites.","tokens_in":6904,"tokens_out":2130,"would_cite":false,"duration_ms":20199,"reading_group":"no","serious_thinker":"no","would_accept_peer_review":false},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A dual-view extension of MoCo with farthest-negative filtering reports 86.32% top-1 accuracy on CIFAR-10 at 4.3 GB GPU memory","keywords":["contrastive learning","Momentum Contrast (MoCo)","dual-view loss","negative sampling","hard negative filtering","memory bank","self-supervised representation learning","CIFAR-10"],"falsifier":"Run the same dual-view loss on CIFAR-10 with identical hyperparameters but replace the farthest-negative filter with a random subset of the same size; if accuracy does not drop, the filtering step contributes nothing beyond the dual-view loss.","tokens_in":5774,"feed_emoji":"🧠","tokens_out":8873,"duration_ms":80157,"temperature":0.7,"pith_summary":"The paper argues that the standard MoCo contrastive framework underuses the key encoder's embeddings and lets noisy or mislabeled negatives in its memory bank hurt learning. To fix this, it extends InfoNCE into a dual-view loss that optimizes both query and key embeddings, and adds a negative-sampling filter that keeps only the memory-bank samples farthest from the query in cosine similarity. On CIFAR-10 and CIFAR-100 the authors report that the modified model, called MoHN, reaches 86.32% and 60.1% top-1 accuracy respectively, matching or exceeding many prior self-supervised methods while using 4.3 GB to 4.2 GB of GPU memory. The pitch is that contrastive representations can be made both more accurate and cheaper to train, widening the practical reach of unsupervised learning.","feed_headline":"86.32% top-1: dual-view MoCo with farthest-negative filtering","feed_subtitle":"The method balances query and key views while using less GPU memory than common baselines.","key_machinery":"The two load-bearing components are the dual-view extended InfoNCE loss and the farthest-negative filter. The dual-view loss adds a second cross-entropy term in which the key embedding $k$ plays the role of the anchor, with $q_+$ as its positive and a subset of memory-bank features as negatives; the scalar $m\\in[0.1,0.01]$ controls the balance between the query and key terms. The filter defines $\\mathcal{FN}$ as the set of memory-bank features farthest from $q$ in cosine similarity, so only a fraction of the queue (20% in the reported runs) contributes to the key-view term. Together, these pieces are what the paper credits for reducing sensitivity to mislabeled negatives and for making both encoders learn discriminative features.","core_discovery":"The central claim, stated on the paper's terms, is that balancing the contrastive objective across both views and filtering negatives by cosine distance yields better representations than MoCo's query-only InfoNCE. The proposed loss is $L_q = -(1-m)\\log\\frac{\\exp(q\\cdot k_+/\\tau)}{\\sum_i\\exp(q\\cdot k_i/\\tau)} - m\\log\\frac{\\exp(k\\cdot q_+/\\tau)}{\\sum_{i\\in\\mathcal{FN}}\\exp(k\\cdot k_i/\\tau)}$, where the first term is the standard query-view objective and the second optimizes the key view against a filtered set $\\mathcal{FN}$ of negatives that are most distant from $q$. With this loss and a momentum encoder whose memory bank is updated as a circular queue, the authors report top-1 accuracies of 86.32% on CIFAR-10 and 60.1% on CIFAR-100 under linear evaluation, and they argue that the model delivers these results at lower GPU memory than comparison methods such as MoCo, BarlowTwins, and DINO.","pith_inferences":["The paper calls its filter \"hard negative,\" but it keeps the farthest negatives; if the accuracy gain is real, the active mechanism may be removal of same-class false negatives rather than emphasis of difficult examples.","An ablation the paper does not report, holding the dual-view loss fixed and comparing closest, random, and farthest negative selection, would isolate whether the filtering step or the balanced loss is responsible for the improvement.","The dual-view term could be tested outside MoCo by using a second augmented view's embedding as the key anchor in SimCLR-style training, giving a cheap check of whether key-view gradients are the active ingredient.","If farthest negatives help chiefly by discarding near-duplicate entries, memory banks could be shrunk substantially; the paper's use of only 20% of the bank for the key-view term is a partial demonstration."],"forward_implications":["If the dual-view loss transfers as the authors expect, other query-only contrastive objectives can be strengthened by adding the same key-view term.","The 20% filtering result implies that only a fraction of the memory bank must be scored for the key-view loss, so bigger effective negative pools become affordable on fixed hardware.","The reported GPU-memory figures (4.3 GB on CIFAR-10, 4.2 GB on CIFAR-100 with ResNet-18) put self-supervised pretraining within reach of smaller GPUs than many cited baselines require.","The authors expect the balanced loss and filtered negatives to generalize beyond images, including to natural language processing tasks that use contrastive objectives."],"supporting_citations":[{"why":"Supplies the base MoCo memory-bank and momentum-encoder framework that the paper modifies.","marker":"[5]"},{"why":"Defines the InfoNCE loss that the paper extends into the dual-view formulation.","marker":"[6]"},{"why":"Provides the MoCo v2 augmentation pipeline used in all training runs.","marker":"[3]"},{"why":"Motivates hard negative sampling by closest examples; the paper adapts this idea in the opposite direction.","marker":"[8]"},{"why":"Provides the SimCLR contrastive baseline and the general paradigm the paper builds on.","marker":"[1]"},{"why":"BYOL is the strongest accuracy baseline the paper compares against in the results table.","marker":"[4]"}],"fun_headline_variants":["Balanced MoCo hits 86.32% with hardest negatives","Key-view loss + farthest negatives: 86.32% top-1","Dual-view loss and filtered negatives achieve 86.32% on CIFAR-10","Less GPU memory, higher accuracy: balanced contrastive views","MoCo with key-view loss and farthest-negative sampling beats baselines"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The assumption that carries the negative-sampling contribution is that the memory-bank samples farthest from the query in cosine similarity are the ones whose filtering improves learning; if that choice is wrong, the hard-negative filtering claim has no support.","fun_headline_variants_meta":{"raw":{"variants":["Balanced MoCo hits 86.32% with hardest negatives","Key-view loss + farthest negatives: 86.32% top-1","Dual-view loss and filtered negatives achieve 86.32% on CIFAR-10","Less GPU memory, higher accuracy: balanced contrastive views","MoCo with key-view loss and farthest-negative sampling beats baselines"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001157,"raw_usage":{"total_tokens":4781,"prompt_tokens":921,"completion_tokens":3860,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":537,"completion_tokens_details":{"reasoning_tokens":3763}},"tokens_in":537,"tokens_out":3860,"duration_ms":29994,"temperature":1.0,"reasoning_tokens":3763,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T17:53:18.400570+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the same dual-view loss on CIFAR-10 with identical hyperparameters but replace the farthest-negative filter with a random subset of the same size; if accuracy does not drop, the filtering step contributes nothing beyond the dual-view loss.","supporting_citations":[{"cited_title":"A simple framework for contrastive learning of visual representations","cited_arxiv_id":null,"evidence_quote":"Provides the SimCLR contrastive baseline and the general paradigm the paper builds on."},{"cited_title":"Bootstrap your own latent-a new approach to self-supervised learning","cited_arxiv_id":null,"evidence_quote":"BYOL is the strongest accuracy baseline the paper compares against in the results table."}],"review_version":1}