{"id":"63b81142-98c8-46a1-8ea0-a99bbabf78d0","arxiv_id":"1908.05217","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A semi-supervised framework with coarse-to-fine attention and dual-level memory detects fine-grained classes at close to fully supervised mAP using only coarse boxes and image labels.","lead":"This paper presents a detector that learns to localize thousands of fine-grained classes using only coarse-class bounding boxes and fine-class image labels. The method reaches within about 7 mAP points of fully supervised detection on OpenImages and ImageNet, with much lower annotation cost.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The 11K-class headline is unsupported: the paper reports no fine-grained mAP or proposal recall for Ours-11K, so 'nearly fully-supervised accuracy' is asserted, not shown.","rationale":"I read the paper as making a specific, testable claim: coarse-box supervision plus image-level labels for fine classes can reach close to fully-supervised detection accuracy, including at 11K classes. The OpenImages ablation is internally consistent: soft attention and DLM-FA raise FG mAP from 34.0 to 51.9, and the components are described with enough detail to be plausible. The central unsupported step is ImageNet-11K: Table 4 reports no FG mAP, the text explicitly says only qualitative results are shown, and the shared RPN was trained only on 200 coarse classes. Because the weakly supervised stream re-ranks existing proposals (Eqs. 7-8), proposal recall is a necessary condition for the claim; it is not measured. This does not mean the method is wrong, and it is not a reason to reject the OpenImages or 3K contributions. It does mean the title and abstract overreach. A conditional acceptance with a required 11K evaluation or a tempered claim is the right outcome; my read therefore does not change the reader's verdict. I partially agree with the reader's weakest-assumption: the RPN recall issue is real and part of the same gap, but I would locate the load-bearing weakness in the absence of any quantitative fine-grained evaluation for the flagship 11K model, rather than in the RPN assumption alone.","tokens_in":12712,"tokens_out":8742,"duration_ms":86221,"concrete_test":"Run the published Ours-11K checkpoint on the existing ImageNet-3K validation split and report (i) fine-grained mAP at IoU 0.5 and 0.5:0.95 for the 3K classes and (ii) shared-RPN recall at IoU 0.5 using the 3K ground-truth boxes as proposal targets. If mAP is near the Ours-3K value (35.1) and recall is high, the 11K generalization concern is substantially mitigated; if mAP is far below or recall is low, the 'nearly fully-supervised' claim fails for classes outside the coarse hierarchy. Ideally, also sample and annotate a random subset of 11K classes outside the 3K set and evaluate on it.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim is that coarse-box plus image-label training detects all fine-grained classes at nearly fully-supervised accuracy, with the ImageNet-11K model as the flagship result. The single most load-bearing weakness is that this claim is never quantitatively tested for the 11K model. In Table 4, Fine-grained-11K has mAP-FG '-' and the text explicitly cites only qualitative results (Figure 4). The only numeric fine-grained ImageNet result is for 3K classes, where the gap to full supervision is 35.1 vs 41.6 mAP@0.5. The mechanism itself depends on the coarse-trained shared RPN (Section 3.3.1) proposing the fine-grained objects before the soft-attention re-ranking (Eqs. 7-8) can act; a proposal that never exists cannot be recovered. On ImageNet the coarse mAP degrades from 54.0 (fully supervised) to 50.7/49.1, and no proposal-recall or per-class breakdown is given. If many of the 11K synsets are visually distant from the 200 ILSVRC detection classes, the transfer assumed by the abstract could fail silently because the evaluation is absent. This is an evidentiary gap rather than an observed contradiction, but it directly affects the paper's title and abstract.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a semi-supervised object detection framework for large-scale fine-grained detection. It assumes bounding-box annotations are available only for a set of coarse-grained classes while fine-grained classes have image-level labels only. The method combines a fully-supervised detection stream (R-FCN/SNIPER-style) with a weakly supervised fine-grained stream, connected by a shared backbone and RPN, a soft-attention proposal re-ranking mechanism built from coarse-to-fine class correlations, and a dual-level memory module. Experiments on OpenImages (34 coarse + 462 fine-grained classes) and ImageNet (200 coarse + 3K/11K fine-grained classes) report mAP@0.5 numbers, with OpenImages fine-grained mAP at 51.9 versus 59.1 for full supervision, ImageNet-3K fine-grained mAP at 35.1 versus 41.6, and a claim of near-parity in mAP@0.5:0.95 supported only by text statements. The 11K ImageNet model is evaluated only qualitatively.","tokens_in":12964,"tokens_out":4122,"duration_ms":41310,"significance":"The problem is practically important and the proposed architecture is a sensible way to combine coarse detection labels with fine-grained image-level labels. The OpenImages ablations are internally consistent and show that soft-attention (34.0 to 49.2 mAP-FG) and the dual-level memory (49.2 to 51.9) each contribute substantial gains, which is a meaningful technical result. If the parity claims were fully substantiated with quantitative results, the paper would be a strong contribution to large-scale detection. However, the headline claim of 'nearly fully-supervised accuracy' is overstated by the reported numbers (6-7 point gaps at mAP@0.5), and the flagship 11K ImageNet result is not quantitatively evaluated at all, so the significance as currently demonstrated is lower than the abstract suggests.","major_comments":[{"comment":"The central claim of detecting 11K classes at nearly fully-supervised accuracy is unsupported by the evidence. In Table 4, Ours-11K reports mAP-FG as '-', and the only evidence for fine-grained 11K detection is the qualitative Figure 4. Since the title and abstract emphasize the 11K-class setting, the paper needs either a quantitative evaluation on a labeled subset of the 11K classes (even a small one) or a substantial softening of the claim. The only numeric fine-grained result, on 3K classes, shows a 6.5-point mAP@0.5 gap (35.1 vs 41.6), which is not 'nearly fully-supervised' in the usual sense.","section":"Section 4.3, Table 4"},{"comment":"The mAP@0.5:0.95 parity claims are load-bearing for the 'as accurately as fully-supervised' statements but are not tabulated. The text says OpenImages results are 'both at around 36' and ImageNet results are 'our model at 25 and SNIPER-FG-Fully at 22,' yet no table reports these numbers or the evaluation protocol (which classes, which IoU range, how the fine-grained stream was evaluated on the 11K model). These numbers should be added to the tables with sufficient detail to be verifiable.","section":"Sections 4.2 and 4.3, after Tables 2 and 4"},{"comment":"The shared RPN is trained only on coarse-grained bounding boxes, and the weakly supervised fine-grained stream reuses it to generate proposals; soft-attention re-ranking can only re-rank proposals that already exist. The paper never measures proposal recall on fine-grained classes. If a fine-grained object is visually distant from all coarse classes, the RPN may fail to propose it, and neither the attention nor the memory component can recover it. The paper should report proposal recall on a fine-grained set with box annotations (e.g., OpenImages FG validation or ImageNet-3K) and, ideally, break it down by class or by semantic/visual distance to the coarse classes.","section":"Sections 3.3.1 and 3.3.2"}],"minor_comments":[{"comment":"The notation is inconsistent: the text introduces 'the fine-grained attention map Ww(p)' but Eq. (7) defines 'Aw(p)'; please unify the symbol.","section":"Section 3.3.2, Eq. (7)"},{"comment":"The final weakly supervised loss uses a multi-label label vector y, but the classification loss Lcls is not formally defined; please state whether it is a sigmoid cross-entropy or a softmax-based loss and how it handles multiple positive labels.","section":"Section 3.3.2, Eq. (8)"},{"comment":"The YOLO-9000 and R-FCN-3000 comparisons are explicitly not trained on the same data as the proposed method; the sentence claiming the method 'outperforms YOLO-9000 and R-FCN-3000 by a large margin on coarse-grained classes' should be removed or clearly labeled as a non-apples-to-apples reference.","section":"Table 4"},{"comment":"The description 'randomly sample 1/8 images from all classes' does not specify the resulting training set size or whether the sampling is stratified per class; this matters for interpreting the 11K result and should be stated.","section":"Section 4.3"},{"comment":"The foreground attention pooling sums features only for proposals whose argmax score equals class c; if no proposal satisfies this for a class present in the image, the image-level feature is empty or zero, and the handling of this case is not described.","section":"Section 3.3.3, Eq. (10)"}],"recommendation":"major_revision","confidential_remarks":"The paper has a solid technical core in the OpenImages results, but the abstract and title overclaim the 11K result. The missing quantitative evaluation of the 11K model and the untested proposal-recall assumption are the main obstacles. I would encourage the editor to request a revision that either supplies the missing evidence or substantially reframes the claims; the paper is not ready for acceptance in its current form."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Two things to know. First, the core recipe works: shared backbone plus attention-based proposal re-ranking from coarse detector scores, plus a dual-level memory, lifts fine-grained mAP on OpenImages from 34.0 to 51.9 at IoU 0.5, and on 3K-class ImageNet to 35.1. Those ablations are coherent and the gains are large. Second, the paper's headline claim—detecting 11K classes at nearly fully-supervised accuracy—is not actually tested. The 11K row in Table 4 has no fine-grained mAP; the only evidence is a few qualitative images.\n\nWhat's genuinely new: unlike YOLO9000's naive shared-backbone combination, this work explicitly maps coarse detection scores to fine-grained classes via a precomputed correlation matrix (Eq. 7) and uses that as attention to re-weight weak-stream proposals. The dual-level memory with foreground attention pooling is also novel. The OpenImages setup is well-designed: 34 coarse classes from the semantic tree, 462 fine leaves, disjoint supervision. The fact that coarse mAP also improves with the addition of fine-grained data suggests the shared backbone is genuinely benefiting from the diversity.\n\nWhere it falls short. The 'nearly fully-supervised' phrase oversells the numbers: at IoU 0.5, the gap to full supervision is 7.2 points on OpenImages and 6.5 on ImageNet-3K. The mAP@0.5:0.95 parity claims (36 vs 36, and 25 vs 22) appear only in prose with no table, no error bars, and no explanation of why a method that is 7 points lower at IoU 0.5 would match at the stricter metric; that deserves scrutiny. The 11K model has no quantitative evaluation at all. The shared RPN is trained only on coarse boxes, and the paper never measures proposal recall on fine-grained classes, so the failure mode where a fine-grained object is never proposed—and the re-ranking cannot recover it—is real and unaddressed. No code, no supplement, and several free parameters (lambda, beta, theta, top-k) with no sensitivity analysis. Those are all fixable in revision.\n\nWho it's for: anyone working on annotation-efficient or large-scale detection. The idea is worth engaging with, and the OpenImages result set is a solid starting point even if the 11K claim is not yet substantiated. A serious referee should ask for the 11K evaluation—even a sampled subset with manual annotation—and a full metric table with variance.\n\nI'd send it to peer review. It's not a desk reject. The paper needs revision, but the core method is sound and the contribution is distinct from prior semi-supervised detection work.","headline":"The core semi-supervised detection recipe is real and the OpenImages/3K results are decent, but the 11K headline and 'nearly fully-supervised' claim are not backed by the reported numbers.","tokens_in":13539,"tokens_out":4273,"would_cite":true,"duration_ms":39626,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A shared detector can learn fine-grained classes from image-level labels alone.","keywords":["semi-supervised object detection","weakly supervised detection","fine-grained detection","proposal re-ranking","dual-level memory","large-scale detection","ImageNet 11K classes","OpenImages"],"falsifier":"Train the same framework with coarse classes chosen to be visually distant from the fine-grained set, for example coarse classes from one domain and fine-grained classes from another, and measure fine-grained mAP; if it collapses while fully supervised fine-grained mAP stays high, the proposal-recall assumption is the load-bearing one. Concretely, compute the recall of the shared RPN on held-out fine-grained boxes and check whether fine-grained mAP correlates with that recall.","tokens_in":12458,"feed_emoji":"🎯","tokens_out":4223,"duration_ms":34022,"temperature":0.7,"pith_summary":"This paper claims that a detector can be trained to recognize a huge number of fine-grained classes, up to 11K, without a single fine-grained bounding box, by combining a small set of fully annotated coarse classes with abundant image-level labels. On OpenImages and ImageNet the method reaches detection accuracy close to fully supervised training on the same fine-grained classes, and even matches fully supervised accuracy under a stricter localization metric. The practical stake is annotation cost: building a detector for tens of thousands of classes would only require bounding boxes for a few hundred coarse categories, plus cheap image labels for the rest.","feed_headline":"Detect 11K classes without fine-grained boxes","feed_subtitle":"A semi-supervised detector matches fully supervised accuracy using only coarse-class boxes and cheap image-level labels.","key_machinery":"The mechanism is soft-attention based proposal re-ranking, defined by Eq. 7-8, coupled with a dual-level memory module with foreground attention pooling, Eq. 9-12. The re-ranking computes a coarse attention map from RPN proposals via softmax over classes and proposals, maps it to fine-grained labels through an encoding function $M(j)$ built from semantic or visual correlations, and multiplies it into the weakly supervised score map $S^w$ before pooling. The memory module applies a semi-supervised clustering-style loss at both the coarse proposal level and the fine image level, aggregating features by foreground attention pooling. A shared RPN, trained only on coarse boxes, generates proposals for both streams.","core_discovery":"The central claim is that correlations between coarse-grained and fine-grained classes are sufficient to transfer accurate localization knowledge. The paper's two-stream architecture shares a backbone and a region proposal network between a fully supervised detection stream, trained on coarse boxes, and a weakly supervised classification stream, trained on fine-grained image labels. A soft-attention re-ranking uses coarse proposal scores as an attention map, mapped through semantic or visual encodings, to re-rank fine-grained proposals, and a dual-level memory module regularizes both proposal-level and image-level distributions. Together these components lift fine-grained mAP from 34.0 to 51.9 on OpenImages, and yield a model that detects ImageNet's 11K classes with reasonable qualitative performance.","pith_inferences":["If the claim holds, annotation pipelines could shift from exhaustive box labeling to a coarse taxonomy plus image labels, potentially making web-scale detection economically feasible.","A testable extension is replacing the fixed semantic or visual mapping with a learned correlation matrix updated during training, which might adapt to classes whose visual similarity differs from semantic distance.","The method's success depends on proposal recall from the shared RPN; explicitly measuring fine-grained recall would reveal whether the approach extends to visually distant fine-grained classes."],"forward_implications":["Fine-grained classes can be detected without fine-grained bounding boxes at accuracy close to fully supervised training, dramatically reducing annotation cost.","The shared backbone benefits from diverse fine-grained images: the naive joint baseline already improves coarse-grained mAP by 7 points on OpenImages.","The method outperforms pure weakly supervised detection by a large margin on OpenImages fine-grained classes, 51.9 versus 20.2 mAP.","On ImageNet, the model detects 11K classes while keeping coarse-grained mAP above 49, outperforming YOLO-9000 and R-FCN-3000 on coarse classes.","The framework is end-to-end trainable and can be transferred to any two-stage RPN-based detector."],"supporting_citations":[{"why":"Provides the SNIPER implementation and multiscale training baseline that the framework is built on, including the fully supervised comparisons.","marker":"[28]"},{"why":"Supplies the proposal-level ranking softmax formulation that the soft-attention re-ranking extends.","marker":"[2]"},{"why":"Provides the memory module structure and loss used for the dual-level memory.","marker":"[4]"},{"why":"Is the R-FCN detection backbone used for the fully supervised stream.","marker":"[5]"},{"why":"Defines the 3K fine-grained ImageNet setting and the R-FCN-3000 baseline.","marker":"[27]"},{"why":"Is the YOLO-9000 semi-supervised baseline compared on coarse-grained classes.","marker":"[24]"}],"fun_headline_variants":["11K classes, zero fine-grained boxes","Semi-supervised detection hits 11K classes","Coarse boxes only: detect 11K fine-grained classes","Detect 11K classes with only coarse boxes"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The shared region proposal network, trained only on coarse-class bounding boxes, must propose the fine-grained objects with reliably high recall; if a fine-grained object never appears in the proposal list, neither attention re-ranking nor memory can recover it.","fun_headline_variants_meta":{"raw":{"variants":["11K classes, zero fine-grained boxes","Semi-supervised detection hits 11K classes","Coarse boxes only: detect 11K fine-grained classes","Detect 11K classes with only coarse boxes"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000879,"raw_usage":{"total_tokens":3772,"prompt_tokens":889,"completion_tokens":2883,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":505,"completion_tokens_details":{"reasoning_tokens":2820}},"tokens_in":505,"tokens_out":2883,"duration_ms":20109,"temperature":1.0,"reasoning_tokens":2820,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T13:20:12.123170+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the same framework with coarse classes chosen to be visually distant from the fine-grained set, for example coarse classes from one domain and fine-grained classes from another, and measure fine-grained mAP; if it collapses while fully supervised fine-grained mAP stays high, the proposal-recall assumption is the load-bearing one. Concretely, compute the recall of the shared RPN on held-out fine-grained boxes and check whether fine-grained mAP correlates with that recall.","supporting_citations":[{"cited_title":"SNIPER: Efficient multi-scale training","cited_arxiv_id":null,"evidence_quote":"Provides the SNIPER implementation and multiscale training baseline that the framework is built on, including the fully supervised comparisons."},{"cited_title":"Weakly supervised deep detection networks","cited_arxiv_id":null,"evidence_quote":"Supplies the proposal-level ranking softmax formulation that the soft-attention re-ranking extends."},{"cited_title":"Semi-supervised deep learning with memory","cited_arxiv_id":null,"evidence_quote":"Provides the memory module structure and loss used for the dual-level memory."},{"cited_title":"R-FCN: object detec- tion via region-based fully convolutional networks","cited_arxiv_id":null,"evidence_quote":"Is the R-FCN detection backbone used for the fully supervised stream."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the 3K fine-grained ImageNet setting and the R-FCN-3000 baseline."},{"cited_title":"YOLO9000: better, faster, stronger","cited_arxiv_id":null,"evidence_quote":"Is the YOLO-9000 semi-supervised baseline compared on coarse-grained classes."}],"review_version":1}