{"id":"ecd8725d-ddf0-4239-b628-00b4c4f8deb9","arxiv_id":"2412.05611","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":6,"one_line_summary":"Detectors trained only on medium/large COCO instances, aided by test-time upscaling, detect small instances about as well as a detector trained on all sizes.","lead":"This paper asks whether object detectors really need annotations for small objects, which are costly and error prone to label. On COCO, training without small-object boxes and then enlarging test images matches the accuracy of a fully annotated detector.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Central claim overreaches: aggregate APS parity in Table 2 hides per-class failures (Appendix A: traffic lights, remotes) for classes that are predominantly small, undercutting the conclusion that small-instance annotation is unnecessary.","rationale":"The reader's weakest_assumption is that upscaling plus resolution-matching augmentation fully closes the distribution gap between upscaled small objects and native medium objects, with Appendix A showing failures on context-dependent classes. My analysis agrees and makes the consequence explicit: the central claim is about aggregate parity, but the paper's own appendix shows the mechanism fails for classes that predominantly appear small. This is a load-bearing concern because the practical conclusion ('rethinking annotation') depends on the method working for the objects whose annotation is being saved, not just on a benchmark average. A per-class analysis would settle whether the aggregate surplus masks losses on traffic lights, remotes, and similar small-context classes. The reader's CONDITIONAL verdict already accounts for this limitation (and for the additional issues of missing error bars and validation-set hyperparameter selection), so no change to the verdict is needed. The paper is honest and provides substantial evidence for the aggregate phenomenon; the issue is the strength of the generalizing conclusion, which should be tempered or backed by per-class evidence.","tokens_in":12762,"tokens_out":10466,"duration_ms":98009,"concrete_test":"Compute per-class small-instance APS on COCO val for Up@Test-scale (gamma=3, alpha=2.0 for Faster R-CNN and 2.2 for FCOS) and Baseline, focusing on classes whose COCO train instances are predominantly small (e.g., traffic light, remote, stop sign, sports ball, clock). If the per-class APS deficit for these classes is comparable to or larger than the aggregate APS gain, then the claim 'it suffices to only use test-time upscaling' fails for the very categories most relevant to the annotation-cost question, and the conclusion should be restricted to classes with sufficient medium/large supervision.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim (Sec. 5.4, observation i) is that a detector trained without small-instance annotations, using test-time upscaling plus resolution-matching augmentation, achieves 'at least comparable performance' to a fully annotated baseline, so small-instance annotation is not necessary. The supporting evidence is the aggregate APS on COCO val (Table 2: 13.7 vs 11.9 for Faster R-CNN; 23.6 vs 22.3 for FCOS). The load-bearing problem is that aggregate parity can conceal systematic failure on exactly the classes where the cost-benefit claim matters most. Appendix A explicitly states: 'for objects that have only small-size instances in the training data (e.g., traffic light and remote), Baseline performs better.' This is not an edge case; traffic lights and remotes are exactly the kind of small, context-dependent objects whose annotation the paper suggests skipping. Section 4.3 acknowledges an irreducible domain gap from context and perspective after scaling, and the appendix confirms that gap has practical consequences. Without a per-class APS breakdown, the reader cannot tell whether the aggregate surplus is driven by classes with ample medium/large supervision while classes that predominantly appear small lose accuracy. For a cost-benefit question, the relevant metric is per-class, not aggregate: if skipping small-instance annotation sacrifices detection of precisely those classes, the headline conclusion 'small-size annotation is not worth its cost' is not supported. The paper provides no analysis to rule this out.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper poses the cost-benefit question of whether annotating small-size object instances is worth its cost, and reformulates it as a falsifiable question: can a detector trained on data free of small-size instance annotations still detect small instances? The authors propose two methods: Up@Test, which upscales the input image at test time while training only on medium/large instances, optionally with blur or downscale-upscale augmentation to narrow the domain gap; and Down@Train, which downscales training images to create synthetic small instances. Experiments on COCO show that Up@Test with scale augmentation (γ=3) reaches small-instance AP slightly above the baseline trained on all annotations (Faster RCNN 13.7 vs 11.9; FCOS 23.6 vs 22.3), while Down@Train performs much worse. A distillation step yields a single-path student model with AP-small 11.0 vs 11.9 (Faster RCNN) and 22.4 vs 22.3 (FCOS), which the paper describes as comparable to the full-annotation baseline. The paper concludes that small-instance annotation may be unnecessary, and that this result should prompt rethinking of annotation practices.","tokens_in":13048,"tokens_out":4660,"duration_ms":43398,"significance":"The paper addresses a practically important and underexplored question: whether the high cost of annotating small objects is justified. The central claim, if it holds, would have direct implications for dataset construction and annotation cost reduction. The study is framed as a clear, testable hypothesis, and the proposed methods are simple, detector-agnostic, and include a distillation step that addresses the computational cost of running the detector twice. The paper is honest about the residual domain gap and shows failure cases. However, the evidence is limited to two detectors on a single dataset, and the headline conclusion relies on aggregate AP-small values that may conceal systematic per-class degradation, particularly for classes that only appear at small sizes in the training data. A per-class breakdown and a measure of variance across runs are needed to make the central claim robust.","major_comments":[{"comment":"The central claim that Up@Test achieves 'at least comparable performance' to Baseline is supported only at the aggregate AP-small level (Table 2: 13.7 vs 11.9 for Faster RCNN; 23.6 vs 22.3 for FCOS). Appendix A explicitly states that 'for objects that have only small-size instances in the training data (e.g., traffic light and remote), Baseline performs better.' This is not an edge case: traffic lights and remotes are exactly the kinds of small, context-dependent objects whose annotation the paper argues could be skipped. Without a per-class AP breakdown, the aggregate surplus could be driven by classes that have ample medium/large supervision while classes that predominantly appear small lose detection accuracy. Since the cost-benefit question is fundamentally per-class, the paper should report AP-small per class, or at least for classes with a high fraction of small instances, to support the headline conclusion.","section":"Sec. 5.4, observation (i) and Appendix A"},{"comment":"The distilled single-path model for Faster RCNN achieves APS 11.0 vs Baseline 11.9, a deficit of 0.9, while for FCOS the values are 22.4 vs 22.3. The paper calls these 'comparable' but provides no error bars or multiple-seed results, so it is impossible to tell whether the Faster RCNN gap is within noise or a real degradation. The same issue applies to Table 2 (13.7 vs 11.9; 23.6 vs 22.3). The claim of 'at least comparable' would be much stronger with standard deviations over at least three runs.","section":"Sec. 5.5, Table 3"},{"comment":"The method rests on the assumption that the downscale-upscale augmentation (γ=3) or Gaussian blur closes the resolution part of the domain gap, and that the remaining context/perspective gap is small enough. Appendix A shows concrete failures on context-dependent cases (e.g., 'sports ball' on a cupcake, 'clock' in an eye) and on small-predominant classes. The paper acknowledges these failures but does not quantify how often they occur or how much they contribute to the aggregate result. A quantitative analysis of the residual domain gap, for instance by reporting AP-small separately for classes with a majority of small instances versus those with substantial medium/large supervision, would clarify the scope of the claim and the conditions under which the method can be expected to transfer to new datasets.","section":"Sec. 4.3 and Appendix A"},{"comment":"All hyperparameters (α, γ, β, blur kernel and σ, confidence thresholds, mixing ratio) are selected based on COCO validation, and the final results are reported on that same validation split. This is effectively tuning on the test set, which can inflate the reported numbers and make the 'at least comparable' claim less reliable. The authors should either evaluate on a held-out test split (e.g., COCO test-dev) or, failing that, explicitly discuss the risk of overfitting to the validation set and report the sensitivity of the main results to the chosen hyperparameters.","section":"Sec. 5.1"}],"minor_comments":[{"comment":"The abbreviation 'DAT' is used in the opening paragraph of the appendix without ever being defined; it should be defined or replaced with 'Down@Train' for consistency.","section":"Appendix A"},{"comment":"The sentence 'We do not employ any kind of multi-scale test or multi-scale training other than ours' is confusing because Up@Test itself is a form of multi-scale testing; please rephrase to clarify that no standard multi-scale augmentation or test-time augmentation is used beyond the proposed methods.","section":"Sec. 5.1"},{"comment":"The mixing ratio of 0.414 : 0.586 for blur/downscale augmentations is motivated only by the instance count ratio in COCO; the rationale for using this ratio for mixing training samples should be explained, since a ratio based on instance counts does not obviously translate to a ratio for image-level mixing.","section":"Sec. 5.1"},{"comment":"The caption should state more explicitly that the 'human subject' is one of the authors and that the 500 images were chosen randomly, to avoid any impression of a formal human-study methodology.","section":"Figure 3"},{"comment":"The phrase 'at least comparable' is too strong given that the distilled Faster RCNN result (11.0 APS) is 0.9 below the baseline (11.9); consider softening to 'comparable within uncertainty' or reporting a confidence interval.","section":"Abstract and Sec. 5.5"},{"comment":"The legends and axis labels in Figures 5-7 are very small and hard to read; also, the figure captions should explicitly note that the broken lines are the Baseline values for each AP metric, not a single reference line.","section":"Figures 5-7"}],"recommendation":"major_revision","confidential_remarks":"This paper asks a fresh and important question, and the simple approach plus distillation is a nice contribution. The aggregate results do suggest that small-instance annotations may be less critical than commonly believed, but the per-class failure on small-predominant classes (traffic light, remote) is a real concern for the general 'not worth its cost' conclusion. I would like to see the authors add a per-class APS breakdown and variance estimates; those are feasible additions. The human-annotation experiment is anecdotal and should be labelled as such, not used as strong evidence. Overall, the manuscript is within scope for CV and could be accepted after a major revision that addresses the per-class and statistical robustness issues."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: the paper asks a genuinely new question — can you skip small-instance annotations and recover accuracy by upscaling at test time? The evidence says mostly yes on COCO aggregate, with a per-class caveat that undercuts the headline.\n\nWhat's new: nobody has previously asked whether small-instance annotations can simply be dropped if you upscale at test. SNIP/SNIPER and Touvron et al. deal with scale mismatch, not annotation value. The experiment is direct: train on medium/large instances only, upscale at test time, compare against COCO val. Table 2 shows Up@Test-scale beating the fully-annotated baseline on small-AP for both detectors (13.7 vs 11.9 Faster R-CNN; 23.6 vs 22.3 FCOS), and the distilled single-path model lands near baseline (11.0 vs 11.9; 22.4 vs 22.3), which answers the obvious \"twice the compute\" objection. The appendix deserves credit for showing failure cases instead of hiding them.\n\nWhere it's soft: the per-class issue is the load-bearing one, and it is confirmed by the paper's own text. Appendix A says plainly that for classes with only small instances in the training data — traffic lights, remotes — Baseline performs better. Those are exactly the classes where the cost-benefit claim matters most: if you skip small annotations to save money, you lose accuracy on the classes that predominantly appear small. The aggregate surplus could be carried by classes with ample medium/large supervision (fire hydrants, skateboards). There is no per-class AP table, so the reader cannot see how much damage there is. That makes the conclusion \"small annotation is not worth its cost\" overreach the evidence.\n\nThe secondary softs are ordinary: no error bars or multiple seeds (the margins are small enough to care), two detectors on COCO only, and the hyperparameters (α, γ, blur kernel, distillation threshold) selected on the validation split rather than a held-out set. Each is fixable in revision. The Down@Train failure fits the context/perspective gap the paper describes, which actually supports the method's logic rather than undermining it.\n\nWho it's for: dataset builders and anyone budgeting annotation effort, plus people working on scale-robust detection. The paper is worth a serious referee. The gaps are addressable — per-class AP, multi-seed runs, a tempered claim — without a redesign.","headline":"Genuinely new cost-benefit question, clean COCO experiment, honest appendix — but the aggregate parity hides per-class losses on small-only classes like traffic lights, so the 'small annotation isn't worth it' headline overreaches the evidence.","tokens_in":13641,"tokens_out":4854,"would_cite":true,"duration_ms":42672,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper argues that detectors can match fully annotated baselines on small objects using only test-time upscaling and resolution-matching augmentation.","keywords":["small object detection","annotation cost","test-time upscaling","domain gap","knowledge distillation","COCO","object detection"],"falsifier":"Compute per-class average precision for small instances on the COCO validation split for context-dependent classes whose training instances are predominantly small (for example, traffic light and remote); if the upscaling method falls clearly below the fully annotated baseline on those classes, the claimed parity does not hold for all objects.","tokens_in":12512,"feed_emoji":"🔍","tokens_out":7397,"duration_ms":66496,"temperature":0.7,"pith_summary":"This paper asks whether the expensive, error-prone work of annotating small object instances is necessary for training an object detector. It restates the question as a testable one: can a detector trained with no small-instance annotations still detect small objects? On COCO, using Faster R-CNN and FCOS, the authors show that upscaling the test image and feeding it through the same detector, together with a training-time augmentation that makes medium-sized training instances look as blurry as upscaled small ones, reaches at least the accuracy of a baseline trained with all annotations. Downscaling training images to create synthetic small instances does not work as well, which the authors attribute to a context and perspective gap. The result suggests that skipping or coarsening small-instance annotation could cut annotation cost without sacrificing comparable detection accuracy.","feed_headline":"Skip tiny-object labels; test-time upscaling keeps detection accuracy.","feed_subtitle":"On COCO, Faster R-CNN and FCOS with test-time upscaling matched or beat full-data baselines.","key_machinery":"The mechanism is test-time upscaling: the detector is applied twice to the same image, once at native resolution to catch medium and large objects and once enlarged by a factor α (around 2.0–2.2) to make small objects appear medium-sized, with detected boxes rescaled back and merged. Because upscaled small objects are blurrier than native medium objects, the training set is augmented by downscaling training images by 1/γ and upscaling them back by γ (γ = 3) — or by Gaussian blurring — so that the detector learns to map blurry, medium-sized appearances to the correct classes. A contrastive method, downscaling training images so medium objects become small synthetic instances, fails, isolating the domain gap in context and perspective as the limiting factor. Finally, knowledge distillation turns the two-pass teacher into a single-path student trained on pseudo labels for small instances plus the original medium and large annotations, restoring the practical computational cost of the baseline.","core_discovery":"The central claim is that detectors can learn to detect small-size instances without any real small-size annotation, provided the input is upscaled at test time and the resolution gap is narrowed during training. In the COCO validation evaluation, the two-pass Up@Test-scale method reaches an average precision for small instances of 13.7 for Faster R-CNN against a baseline of 11.9, and 23.6 for FCOS against 22.3. Distilling the two-pass model into a single-path detector preserves the result, giving 11.0 for Faster R-CNN and 22.4 for FCOS, both comparable to their full-data baselines. The same experiments show that synthesizing small instances by downscaling training images does not match this performance, supporting the authors' conclusion that test-time upscaling, not small-instance annotation, is the effective ingredient.","pith_inferences":["Editorial inference: the result raises the possibility of revising annotation guidelines so that crowd-like clusters of small instances are labeled with a single coarse box or an iscrowd flag, with test-time upscaling recovering the lost individual detections.","Editorial inference: a testable extension is to apply the same recipe to datasets captured with different camera focal lengths or to video frames; if the context and perspective gap widens, parity with fully annotated baselines should degrade.","Editorial inference: combining test-time upscaling with a super-resolution step could push small-object accuracy beyond the baseline, since the remaining gap the authors document for very small instances is partly a resolution effect.","Editorial inference: the per-class analysis in the appendix suggests that classes whose training examples are mostly small, such as traffic lights and remotes, may still need annotations; a per-class cost-benefit rule could identify exactly which classes justify small-instance labeling."],"forward_implications":["If the claim holds, an annotation pipeline can omit or roughly label small instances and still obtain a detector whose small-object accuracy is comparable to one trained on complete data.","The annotation budget saved on tiny objects could be redirected to harder medium and large annotations or to rare, context-dependent classes.","Because the method is detector-agnostic, the same upscaling-plus-augmentation recipe may transfer to other anchor-based and anchor-free detectors.","The failure of Down@Train predicts that synthetic small instances made by resizing will not substitute for real small annotations, so annotation decisions should account for context and perspective, not resolution alone."],"supporting_citations":[{"why":"Supplies the COCO dataset, its small/medium/large size split, and the evaluation protocol used for every comparison.","marker":"[19]"},{"why":"Provides Faster R-CNN, one of the two base detectors used to test Up@Test and distillation.","marker":"[34]"},{"why":"Provides FCOS, the anchor-free detector used to confirm the result is not specific to one detector family.","marker":"[42]"},{"why":"Provides the feature pyramid backbone used by both detectors, which the scale-aware detection relies on.","marker":"[20]"},{"why":"Supplies the finding that train-test resolution discrepancy harms recognition, motivating the resolution-matching augmentation at training time.","marker":"[43]"},{"why":"Defines the scale-normalized training idea (SNIP) that the paper's resizing-based approach builds on and compares with.","marker":"[37]"},{"why":"Extends SNIP with efficient sub-image sampling, a related resizing-based alternative for scale handling.","marker":"[38]"}],"fun_headline_variants":["Tiny-object labels unnecessary: test-time upscaling preserves accuracy","Rethink annotation: test-time upscaling matches small-instance detection","Costly small-instance labels? Test-time upscaling makes them redundant","Drop small-instance annotations: upscaling at test time keeps detector performance","Is annotating small objects worth it? Test-time upscaling says no"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that an upscaled small object, after the resolution-matching augmentation, looks enough like a native medium-sized object to the detector, so the remaining differences in context and perspective do not matter for the classes being detected.","fun_headline_variants_meta":{"raw":{"variants":["Tiny-object labels unnecessary: test-time upscaling preserves accuracy","Rethink annotation: test-time upscaling matches small-instance detection","Costly small-instance labels? Test-time upscaling makes them redundant","Drop small-instance annotations: upscaling at test time keeps detector performance","Is annotating small objects worth it? Test-time upscaling says no"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000402,"raw_usage":{"total_tokens":2078,"prompt_tokens":908,"completion_tokens":1170,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":524,"completion_tokens_details":{"reasoning_tokens":1071}},"tokens_in":524,"tokens_out":1170,"duration_ms":9476,"temperature":1.0,"reasoning_tokens":1071,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T20:32:32.093093+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Compute per-class average precision for small instances on the COCO validation split for context-dependent classes whose training instances are predominantly small (for example, traffic light and remote); if the upscaling method falls clearly below the fully annotated baseline on those classes, the claimed parity does not hold for all objects.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the COCO dataset, its small/medium/large size split, and the evaluation protocol used for every comparison."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides Faster R-CNN, one of the two base detectors used to test Up@Test and distillation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides FCOS, the anchor-free detector used to confirm the result is not specific to one detector family."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the feature pyramid backbone used by both detectors, which the scale-aware detection relies on."},{"cited_title":"Touvron, A","cited_arxiv_id":null,"evidence_quote":"Supplies the finding that train-test resolution discrepancy harms recognition, motivating the resolution-matching augmentation at training time."},{"cited_title":"Singh and L","cited_arxiv_id":null,"evidence_quote":"Defines the scale-normalized training idea (SNIP) that the paper's resizing-based approach builds on and compares with."},{"cited_title":"Singh, M","cited_arxiv_id":null,"evidence_quote":"Extends SNIP with efficient sub-image sampling, a related resizing-based alternative for scale handling."}],"review_version":1}