{"id":"2e947fc0-44ee-43c5-bab2-1420dd23fd41","arxiv_id":"2411.09453","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":9,"one_line_summary":"A detection pre-training framework that dynamically rebalances rare classes and adds dual reconstruction improves tail-class AP on COCO and LVIS by about 0.5 to 1.5 points in most configurations.","lead":"Researchers propose a pre-training stage for object detectors that combines contrastive learning, rare-class rebalancing, and image reconstruction, and they evaluate it on COCO and LVIS. The method reports small but steady gains on rare object classes in most settings, with higher compute cost as the main trade-off.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The LVIS headline gains may be confounded: the manuscript never states the pre-training dataset for 2DRCL or defines the detection loss Ldet, so the central comparison may be extra supervised training on the target split rather than transferable pre-training.","rationale":"I read the paper as claiming that 2DRCL is a pre-training stage that improves long-tailed detection, particularly tail classes. For that claim to hold, the LVIS experiments must compare like with like: a transferable pre-training method should be pre-trained on a source split and fine-tuned on a target split, with the same data/label exposure for all baselines. The manuscript does not establish this. Equation (8) introduces Ldet but never defines it; Section 4.2 says 'we first use our 2DRCL for pre-training' immediately in the context of LVIS, and no table lists the pre-training dataset/split for each method. If the authors pre-trained on LVIS v1.0 train and used ground-truth detection loss, the method is essentially extra supervised training on the target split, which would explain the gains without any transferable pre-training story. This is more fundamental than the missing proposal-matching rule in Section 3.1.2: that rule affects one component's fidelity, while an unspecified pre-training protocol affects the validity of every reported LVIS comparison. I also note Table 4b's same-epoch control is helpful but does not control for source data or label usage. The paper has positive signals, such as ablations consistent with DRB/DRC helping and same-epoch improvements, so if the protocol turns out to be COCO-based, the verdict could return to CONDITIONAL or ACCEPT. As written, the evidence is not yet verifiable.","tokens_in":17858,"tokens_out":20545,"duration_ms":188421,"concrete_test":"Obtain the authors' training logs or code and extract, for every LVIS row in Tables 2, 3, and A.1: (i) the pre-training dataset and split, (ii) the definition of Ldet and whether it consumes ground-truth boxes/labels, (iii) pre-training epochs. If 2DRCL was pre-trained on LVIS v1.0 train, rerun the Table 3 comparison with 2DRCL pre-trained on COCO train2017 (the same source used for the MoCo/SoCo baselines) under the same total-epoch schedule, and check whether the APbb and APr gains over ECM persist. If they do not, the headline claim is confounded.","verdict_should_be":"UNVERDICTED","load_bearing_attack":"Section 3's final objective (Eq. 8) includes Ldet, 'the loss of object detection', but Ldet is never defined anywhere in the paper or appendix. Section 4.2 introduces the LVIS experiments by saying 'we first use our 2DRCL for pre-training, followed by existing long-tailed methods for fine-tuning', and no table or paragraph states which dataset/split is used for that pre-training. The natural reading is that 2DRCL is pre-trained on LVIS v1.0 train and then fine-tuned on the same split. If that is what was done, Tables 2 and 3 compare methods pre-trained on a source dataset (e.g., COCO) against 2DRCL that has already seen the target LVIS training images, and if Ldet uses LVIS ground-truth labels, the pre-training is supervised target-domain training. The reported gains (APbb 27.3 vs 26.7; APr 18.6 vs 17.5 in Table 3a) would then reflect extra exposure to the target data, not a transferable pre-training mechanism. The same-epoch control in Table 4b does not resolve this: it equalizes total epochs only within the same (unspecified) protocol, and it does not say whether the +Ours pre-training used LVIS images. LCL proposal matching (the reader's concern) is a real reproducibility gap, but it is secondary: even with perfect matching, the central comparison is uninterpretable without the pre-training protocol.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes 2DRCL, a pre-training framework for object detection that combines holistic contrastive learning (HCL), local contrastive learning (LCL) on object proposals, a dynamic rebalancing sampler, and a dual reconstruction mechanism (appearance and semantic), together with an unspecified detection loss Ldet. Experiments on COCO and LVIS v1.0 with Faster/Mask R-CNN report gains in overall and rare-class AP over several baselines, including a same-epoch comparison in Table 4b and component ablations in Table 5. The main claim is that 2DRCL provides a transferable pre-training stage that improves long-tailed object detection, especially for tail classes.","tokens_in":18206,"tokens_out":7400,"duration_ms":63741,"significance":"If the protocol issues are resolved, the proposed framework would be a useful pre-training addition to existing long-tailed detectors, with a plausible mechanism (dynamic rebalancing plus reconstruction) and broad empirical coverage: COCO and LVIS, two backbones, two detector heads, plus an ATSS variant and a COCO-LT experiment. The paper also includes component ablations, error analyses, and efficiency measurements. The same-epoch control in Table 4b and the ablation in Table 5 are useful empirical practices. However, the significance is conditional because the central claim depends on the pre-training protocol and the definition of Ldet, which are not specified in the manuscript.","major_comments":[{"comment":"The pre-training data source for the LVIS experiments is never stated. Section 4.2 says only 'we first use our 2DRCL for pre-training' before fine-tuning, and Appendix A.1 describes the architecture but not the dataset. Moreover, Ldet in Eq. (8) is introduced as 'the loss of object detection' but is never defined anywhere in the paper. If pre-training is performed on LVIS v1.0 train with a supervised detection loss, then the comparisons in Tables 2, 3, A.1, and A.2 pit methods that have not seen target labels during pre-training against a method that has. The same-epoch control in Table 4b equalizes total epochs but not data or label exposure, so it does not remove this confound. Please specify for every experiment: (i) the exact pre-training dataset, (ii) whether ground-truth labels are used, (iii) a precise definition of Ldet, and (iv) the pre-training/fine-tuning epoch split for each table.","section":"Section 4.1/4.2 and Eq. (8)"},{"comment":"The positive proposal zbb+ is said to be 'a corresponding positive object proposal within another augmented input image x+', but the matching rule between the proposals generated independently for x and x+ is never described. Since the class-agnostic detector may output different numbers and locations of boxes per view, the loss is not reproducible without an explicit matching algorithm (e.g., IoU after known geometric transforms, or feature similarity). Please provide the matching procedure and any threshold parameters.","section":"Section 3.1.2, Eq. (2)"},{"comment":"The claimed rare-class benefit reverses on ResNet-101: APr drops from 21.9 (ECM) to 21.1 (Ours), a 0.8-point decrease, while the overall AP gain is only 0.1. This is inconsistent with the abstract's claim of 'particularly improving the mAP/AP scores for tail classes.' Please discuss this failure case and, more generally, report results over multiple seeds with standard deviations, since all tables report single runs.","section":"Table 3(b), R101-FPN row"}],"minor_comments":[{"comment":"The quantity f_c is called the harmonic mean of f_im and f_in, but the denominator is a weighted arithmetic mean, not the standard harmonic mean denominator; please correct the terminology or the formula and clarify the intended weighting.","section":"Section 3.2, Eq. (4)"},{"comment":"The statement 'the weights of all losses in L are set to 1' conflicts with the α_c, β_c, and α_r weights defined in Eqs. (3) and (7); please reconcile these statements.","section":"Section 3.3.3, after Eq. (8)"},{"comment":"The 'From scratch' baseline methods use 200-800 pre-training epochs, while Ours uses ImageNet-initialized 12-epoch pre-training; the comparison would be clearer if the total pre-training compute or epochs for each method were listed.","section":"Table 1"},{"comment":"It is not clear which dataset the competing pre-trained methods (MoCo v2, SimCLR, BYOL, SoCo, AlignDet) were pre-trained on before LVIS fine-tuning; please add a column or footnote specifying the pre-training source for each method.","section":"Table 2"},{"comment":"The generator architecture is described, but the encoder architecture used for the reconstruction path is not specified; please clarify how f(x) is obtained from the detection backbone.","section":"Appendix A.1"}],"recommendation":"major_revision","confidential_remarks":"The central risk is the unspecified pre-training protocol for LVIS: if 2DRCL is pre-trained on the target LVIS split with a supervised detection loss, the headline comparisons become uninterpretable. The undefined Ldet and missing proposal-matching rule are concrete reproducibility gaps that must be fixed. The R101 rare-class reversal also weakens the 'tail classes' claim. These issues are fixable with additional experiments and clarifications, so I recommend major revision rather than rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper proposes 2DRCL, a pre-training framework for long-tailed detection that combines MoCo-style holistic contrastive learning, object-level contrastive learning on proposals, a dynamic rebalancing sampler (harmonic mean of image/instance frequency with time-varying alpha), and dual reconstruction (pixel and masked-feature). The individual ingredients are known, but the combination and the time-varying sampler are new, and the authors test across COCO, LVIS, COCO-LT, two backbones, and both two-stage and one-stage detectors. The ablation table is informative and shows each piece contributing. Credit where due: the same-epoch controls in Table 4b and the efficiency table are the right instinct, and the paper is candid about compute costs.\n\nBut there is a load-bearing problem. The final loss in Eq. 8 includes Ldet, \"the loss of object detection,\" which is never defined anywhere in the paper or appendix. More seriously, for the LVIS experiments the paper never states which dataset/split is used for 2DRCL pre-training. The natural reading is that it pre-trains on LVIS v1.0 train and then fine-tunes on the same split; Table 4b's protocol (6 epochs pre-training + 6 epochs fine-tuning on LVIS) strongly suggests this. If so, the comparison in Tables 2 and 3 is between methods that see target images only during fine-tuning and a method that also trains on those images for 6 epochs beforehand, possibly with supervised detection labels via Ldet. That makes the headline gains (APbb 27.3 vs 26.7, APr 18.6 vs 17.5) uninterpretable as evidence for a transferable pre-training mechanism.\n\nOther soft spots are secondary. No error bars or multiple seeds. The rare-class advantage reverses on ResNet-101 for Mask R-CNN (APr 21.9 to 21.1 in Table 3b). The LCL positive-pair matching between proposals across augmented views is never specified, so that part is not reproducible. The simplicity-bias claim rests on attention maps and weight-norm plots, not a direct measurement. The free hyperparameters are hand-set without sensitivity analysis.\n\nIf the authors clarify the pre-training protocol, define Ldet, and add properly matched baselines (e.g., pre-training on COCO only, or equal data exposure), the paper could be a useful contribution. As written, the central comparison is confounded. I'd send it to review with a clear request for major revision on those points, not desk reject — the method is new enough and the ablation is reasonably careful.","headline":"Useful combination of known ideas, but the LVIS comparison is confounded by an unspecified pre-training protocol and an undefined Ldet; needs major revision, not desk reject.","tokens_in":18744,"tokens_out":2758,"would_cite":false,"duration_ms":24848,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper proposes 2DRCL, a detection-specific pre-training method that combines holistic-local contrastive learning, dynamic rebalancing, and dual reconstruction to substantially improve rare-class accuracy in long-tailed object…","keywords":["long-tailed object detection","pre-training","contrastive learning","dynamic rebalancing","dual reconstruction","simplicity bias","LVIS","rare-class accuracy"],"falsifier":"A controlled comparison in which the local contrastive positives are formed by matching proposals across the two augmented views using Intersection-over-Union, versus pairing them at random, would settle the mechanism: if random pairing does not lower rare-class box AP on LVIS v1.0 by at least the reported margin over the no-LCL baseline, then the LCL component is not carrying the improvement that the paper attributes to it.","tokens_in":17636,"feed_emoji":"🎯","tokens_out":9335,"duration_ms":78824,"temperature":0.7,"pith_summary":"Long-tailed object detection fails mostly on rare classes, and the paper argues that the failure starts before fine-tuning: standard pre-training ignores the imbalance and lets models settle for simple patterns. 2DRCL is a pre-training stage that attacks both problems at once, using contrastive learning over whole images and over object proposals, a sampling schedule that progressively favors rare classes at the instance level, and a dual reconstruction task that forces the model to keep fine detail. On LVIS v1.0, the paper reports the highest box average precision among the compared methods for both Faster R-CNN and Mask R-CNN, with the largest gains on rare classes (box $\\mathrm{AP}^{bb}_{r}$ rising from 17.5 to 18.6 against the ECM baseline). On balanced COCO the same pre-training also improves box AP to 41.4, so the benefit is not achieved by sacrificing common classes. The claim matters because it makes pre-training, rather than only loss design, a direct and reusable tool for fixing tail-class detection.","feed_headline":"Pre-training that rebalances rare classes lifts long-tail detection","feed_subtitle":"A pre-training step fixes tail-class failure on LVIS and COCO without changing the loss.","key_machinery":"The load-bearing object is the 2DRCL loss, a sum of three terms: the Holistic-Local Contrastive Learning loss (InfoNCE on global views plus InfoNCE on object proposals, with a large queue-based dictionary for negative samples), the Dynamic Rebalancing sampler (a per-class repeat factor built from the harmonic mean of image-level and instance-level occurrence frequencies, with the mixing coefficient $\\alpha_d = T/T_{\\max}$ growing across epochs), and the Dual Reconstruction loss (pixel-level appearance reconstruction $L_{\\mathrm{AR}} = \\|x - g(f(x))\\|_2^2$ and semantic reconstruction $L_{\\mathrm{SR}}$ comparing features of the original image with features of a masked reconstruction). The local contrastive branch is what aligns pre-training with detection; the dynamic rebalancing is what shifts attention to tail classes; the dual reconstruction is what prevents the encoder from discarding the fine details that tail classes need.","core_discovery":"The central claim is that a detection-oriented pre-training stage can carry most of the burden in long-tailed object detection, provided it couples three mechanisms. Holistic-Local Contrastive Learning runs an InfoNCE objective both on full augmented views (holistic) and on object proposals produced by a pretrained class-agnostic detector (local), so the detection head, not only the backbone, is pre-trained on object-level features. Dynamic Rebalancing computes, per class, an image-level imbalance score and an instance-level (proposal) imbalance score, folds them into a harmonic mean whose mixing weight shifts from image-level to instance-level as training proceeds, and turns the resulting score into a repeat factor that oversamples rare classes. Dual Reconstruction adds appearance reconstruction (pixel-wise MSE) and semantic reconstruction (consistency between features of the original and of a masked reconstruction), which the authors say counters simplicity bias. The paper's reported evidence is that on LVIS v1.0 this recipe beats both general pre-training baselines and state-of-the-art long-tail losses: with Faster R-CNN R50, box average precision is 27.3 versus 26.7 for the effective class-margins (ECM) loss, and rare-class box AP is 18.6 versus 17.5; with Mask R-CNN R50, mask AP is 27.7 versus 27.4 and rare-class mask AP is 20.4 versus 19.7.","pith_inferences":["A testable extension would make the local contrastive matching explicit: proposals could be paired across views by IoU, by feature similarity, or by attention, and the choice would likely change rare-class AP; the paper's silence on this point leaves the mechanism underspecified.","The dynamic rebalancing idea could transfer to other self-supervised objectives, such as masked image modeling, where sampling of patches or instances is usually static; adapting the repeat factor to training stage might yield similar tail-class gains outside detection.","If LCL's effect is to pre-train the detection head, then pushing the same proposals through the regression head (not only the classification head) could extend the gains to localization quality, which the paper reports only indirectly.","The simplicity-bias framing suggests an adversarial variant in which the masking ratio for semantic reconstruction grows for rare classes; the paper uses a fixed 25% mask, so adaptivity is an untested natural extension."],"forward_implications":["If pre-training can shift rare-class accuracy this way, then long-tail methods that redesign the fine-tuning loss are not the only lever; a carefully built pre-training stage can be combined with them for extra gains, as the paper shows with eight existing methods.","The dynamic rebalancing schedule implies that the right sampling weight for a class depends on training stage, so fixed resampling ratios that are optimal at the start may be suboptimal later in pre-training.","Because the gains appear in Faster R-CNN, Mask R-CNN, and ATSS detectors, the pre-training effect transfers across detector designs rather than fitting one architecture.","The equal-epoch comparison (same total training time) indicates that the improvement is a property of the pre-training objective, not merely of longer training, so the method offers a path to better tail performance under a fixed compute budget.","The dual reconstruction's two terms each contribute in ablations, suggesting that both pixel fidelity and semantic invariance under masking are needed to defeat simplicity bias for rare classes."],"supporting_citations":[{"why":"Supplies the LVIS v1.0 dataset, its rare/common/frequent evaluation split, and the Repeat Factor Sampling baseline that the dynamic rebalancing strategy extends.","marker":"[11]"},{"why":"The effective class-margins (ECM) loss is the fine-tuning partner that, combined with 2DRCL pre-training, produces the paper's reported 'Ours' results on LVIS.","marker":"[22]"},{"why":"The pretrained class-agnostic detector generates the object proposals that Local Contrastive Learning uses as local positives.","marker":"[24]"},{"why":"Provides the momentum-contrast formulation and InfoNCE objective that the Holistic Contrastive Learning branch is built on.","marker":"[13]"},{"why":"The improved momentum-contrast baseline whose augmentation pipeline and exponential moving average recipe the holistic branch adopts.","marker":"[5]"},{"why":"AlignDet is the strongest detection pre-training baseline on COCO and LVIS that 2DRCL must beat in Tables 1 and 2.","marker":"[25]"}],"fun_headline_variants":["Dynamic rebalancing pre-training boosts rare-class detection","Pre-training with dual reconstruction lifts rare object AP","Rare-class pre-training: dynamic rebalancing beats long-tail losses","Contrastive pre-training with dynamic rebalancing improves tail AP"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The local contrastive branch assumes that object proposals produced by a pretrained class-agnostic detector on two differently augmented views of the same image can be paired into trustworthy positive pairs, but the paper never specifies the matching rule, so noisy pairing could break the alignment that the method credits for its gains.","fun_headline_variants_meta":{"raw":{"variants":["Dynamic rebalancing pre-training boosts rare-class detection","Pre-training with dual reconstruction lifts rare object AP","Rare-class pre-training: dynamic rebalancing beats long-tail losses","Contrastive pre-training with dynamic rebalancing improves tail AP"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000285,"raw_usage":{"total_tokens":1727,"prompt_tokens":1042,"completion_tokens":685,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":658,"completion_tokens_details":{"reasoning_tokens":616}},"tokens_in":658,"tokens_out":685,"duration_ms":5831,"temperature":1.0,"reasoning_tokens":616,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T20:38:18.058937+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"A controlled comparison in which the local contrastive positives are formed by matching proposals across the two augmented views using Intersection-over-Union, versus pairing them at random, would settle the mechanism: if random pairing does not lower rare-class box AP on LVIS v1.0 by at least the reported margin over the no-LCL baseline, then the LCL component is not carrying the improvement that the paper attributes to it.","supporting_citations":[{"cited_title":"LVIS: A dataset for large vocabulary instance segmentation","cited_arxiv_id":null,"evidence_quote":"Supplies the LVIS v1.0 dataset, its rare/common/frequent evaluation split, and the Repeat Factor Sampling baseline that the dynamic rebalancing strategy extends."},{"cited_title":"Long-tail detection with effective class-margins","cited_arxiv_id":null,"evidence_quote":"The effective class-margins (ECM) loss is the fine-tuning partner that, combined with 2DRCL pre-training, produces the paper's reported 'Ours' results on LVIS."},{"cited_title":"Learning open-world object proposals without learning to classify","cited_arxiv_id":null,"evidence_quote":"The pretrained class-agnostic detector generates the object proposals that Local Contrastive Learning uses as local positives."},{"cited_title":"Momentum contrast for unsupervised visual representation learning","cited_arxiv_id":null,"evidence_quote":"Provides the momentum-contrast formulation and InfoNCE objective that the Holistic Contrastive Learning branch is built on."},{"cited_title":"AlignDet: Aligning pre-training and fine-tuning in object detection","cited_arxiv_id":null,"evidence_quote":"AlignDet is the strongest detection pre-training baseline on COCO and LVIS that 2DRCL must beat in Tables 1 and 2."}],"review_version":1}