{"id":"57446fd4-376e-4d71-8fff-16a92f60a0b4","arxiv_id":"1909.02680","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Coarse2Fine adds a learnable deconvolutional path from attention maps to the input image and a second fine-grained classifier, reporting top-1 accuracy gains of 0.1-1.6 points over WS-DAN on five fine-grained benchmarks.","lead":"This paper presents Coarse2Fine, a two-stage training method that uses attention maps from one network to highlight informative image regions for a second, fine-grained classifier. It reports small accuracy gains over prior state-of-the-art on five fine-grained visual classification benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Core feedback mechanism never isolated: reported gains over WS-DAN may stem from orthogonal initialization, two-head averaging, or dataset errors rather than the proposed differentiable up-sampler.","rationale":"The most load-bearing assumption in the paper is that the differentiable up-sampling path is what drives the observed accuracy gains. The experiments do not isolate this component: WS-DAN already uses bilinear up-sampling of attention maps to highlight images, and Coarse2Fine introduces multiple simultaneous changes over WS-DAN, including orthogonal initialization, shared backbone, averaging of two prediction heads, and modified center loss. The only ablation present (Table 8) evaluates orthogonal initialization, and that alone accounts for most of the gain on iNaturalist (0.8 of 1.6 points). No ablation tests the up-sampler or the feedback loop itself. Without such evidence, the 0.1–1.6 point improvements could be due to the other factors or to random variation, especially since no error bars or multiple runs are reported. The iNaturalist training-set misreport in Table 1 (12,000 vs. the actual ~579K images) further weakens confidence in that particular result. Because this is a preprint with no code release, the missing ablations cannot be checked post hoc. The reader's CONDITIONAL verdict correctly captures this situation: the paper is promising, but the central mechanism is unverified. Our stress-test agrees with the reader's weakest assumption and does not warrant a change to a stronger or weaker verdict; however, if the authors cannot supply the missing ablations and correct the dataset table, the state-of-the-art claim should not be accepted as established.","tokens_in":11677,"tokens_out":5927,"duration_ms":56550,"concrete_test":"Retrain the Coarse2Fine model on CUB-200-2011 and iNaturalist 2017 with the deconvolutional up-sampler replaced by a fixed bilinear up-sampler, keeping all other hyperparameters, orthogonal initialization, and prediction averaging identical. If the top-1 accuracy remains within 0.2 points of the deconv-based model, the learnable inverse mapping is not responsible for the reported gains; if it drops by more than 0.5 points, the mechanism likely contributes. As a complementary check, also train a variant where the fine network receives the original image (i.e., element-wise multiplication by an all-ones mask) to test whether the masking operation itself is necessary for the feedback to help.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim is that the deconvolutional up-sampler creates a differentiable path from attention maps to the raw input, and that the fine-grained classifier's gradient improves attention learning, yielding state-of-the-art accuracy. This claim is not supported by the experiments as presented. Compared to WS-DAN, Coarse2Fine changes several factors simultaneously: orthogonal initialization of attention weights (the only ablation, Table 8, contributes 0.2–0.8 points), sharing the backbone between coarse and fine heads, averaging coarse and fine predictions at inference, applying center loss to both feature sets, and randomly selecting one attention map per training step. No ablation replaces the learned up-sampler with a fixed bilinear or nearest-neighbor up-sampler, and no ablation removes the fine-grained feedback path (e.g., training the fine network on original or randomly masked images). The paper asserts that a bilinear up-sampler degrades accuracy (citing Hu et al. 2019), but this assertion is never tested within the proposed architecture. Consequently, the 0.1–1.6-point gains over WS-DAN cannot be attributed to the proposed mechanism. Additionally, Table 1 lists the iNaturalist 2017 training set as 12,000 images, identical to Stanford Dogs and far below the actual ~579K images; this reporting error undermines the credibility of the iNaturalist result and the overall reproducibility of the experiments.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Coarse2Fine, a two-stage training method for fine-grained visual classification built on bilinear attention pooling. In the coarse stage, an InceptionV3 backbone produces attention maps and a coarse classification prediction. One randomly selected attention map is passed through a deconvolutional up-sampler and multiplied with the raw input to create a highlighted image; this image is classified by a second, fine-grained network. The fine-grained classification loss backpropagates through the up-sampler to the attention maps, creating a differentiable feedback path from the input space to the attended features. The method also includes orthogonal initialization of the 1x1 convolution that produces attention maps and a center loss applied to the pooled part features. The authors report state-of-the-art top-1 accuracy on CUB-200-2011, FGVC-Aircraft, Stanford Cars, Stanford Dogs, and iNaturalist 2017, improved weakly supervised object localization, and an attribute-classification experiment on CelebA.","tokens_in":11946,"tokens_out":5692,"duration_ms":59190,"significance":"If the central claim holds, the paper offers a simple, parameter-efficient way to improve attention learning in fine-grained recognition: a learnable deconvolutional up-sampler that lets fine-grained classification error supervise the attention maps directly. The reported gains over the strong WS-DAN baseline are modest but consistent across five datasets (0.1 to 1.6 accuracy points), and the orthogonal-initialization ablation (Table 8) is a useful, controlled comparison. However, the evidence for the proposed mechanism is incomplete: the experiments do not isolate the differentiable up-sampler from the many other changes relative to WS-DAN, and the reported iNaturalist dataset statistics are inconsistent with the public dataset. With the missing ablations and corrected dataset description, the paper could be a solid empirical contribution; as presented, the attribution of the gains to the Coarse2Fine mechanism is not established.","major_comments":[{"comment":"The central claim that the deconvolutional up-sampler creates a differentiable feedback path that improves attention learning is not tested by any ablation. Compared with WS-DAN, Coarse2Fine changes several factors at once: the learnable up-sampler, the random selection of one attention map per training step, the averaging of coarse and fine predictions at inference, the sharing of the backbone, the application of center loss to both feature sets, and orthogonal initialization. The only ablation provided (Table 8) addresses orthogonal initialization, not the feedback path. In particular, the paper asserts in the Approach section that a non-trainable up-sampler (e.g., bilinear) should cause accuracy degradation, citing Hu et al. 2019, but this is never tested within the proposed architecture. A reader cannot tell whether the reported improvements over WS-DAN come from the differentiable up-sampler, from two-head averaging, from random attention selection, or from orthogonal initialization. I request an ablation that replaces the learned deconvolutional up-sampler with a fixed bilinear or nearest-neighbor up-sampler, and an ablation that removes the fine-grained feedback path (for example, training the fine network on original or randomly masked images). Without these controls, the central claim is not supported.","section":"Coarse2Fine Training; Tables 2-6"},{"comment":"The iNaturalist 2017 row in Table 1 lists Training=12000 and Test=8580, which are exactly the Stanford Dogs numbers. The public iNaturalist 2017 dataset has a training set of roughly 579,000 images and a test set of roughly 96,000 images. This is not a minor typo, because it affects the interpretation of the 70.5% result in Table 6: if the model was trained on a 12,000-image subset, the comparison with WS-DAN (68.9%) may not be on the same training data. The authors must correct the dataset statistics and explicitly state which training and test splits were used for the iNaturalist experiments, including any random subsetting if that was done.","section":"Table 1"},{"comment":"All accuracy numbers are reported as single runs without error bars or multiple-seed statistics. Several of the headline improvements over WS-DAN are very small (e.g., 0.1 points on CUB-200-2011 in Table 2 and 0.2 points on Stanford Cars in Table 4), and such differences are typically within run-to-run variance for CNN training. To support a state-of-the-art claim, the authors should provide the mean and standard deviation over at least three runs, or otherwise demonstrate that the differences are statistically meaningful. This concern applies equally to the weakly supervised localization results in Table 7.","section":"Tables 2-8"},{"comment":"The paper states that averaging the coarse and fine predictions gives the best accuracy, but no experiment or table is shown to support this choice. Since inference-time averaging is one of the differences from WS-DAN and directly affects the reported numbers, this is a load-bearing design decision that needs an ablation. Without it, the reader cannot separate the contribution of the two-head averaging from the contribution of the proposed feedback mechanism.","section":"Approach, Inference paragraph"}],"minor_comments":[{"comment":"The text refers to 'Figure 9' when describing the CelebA results, but the manuscript contains no Figure 9; the figure numbering is incomplete or the reference is wrong.","section":"Discussion"},{"comment":"The claim that the final model contains only 6144 extra parameters appears to count only the attention 1x1 convolution (8 x 768 = 6144), but the deconvolutional up-sampler also adds parameters. Please clarify the total parameter count attributable to the proposed components.","section":"Coarse2Fine Training, 'only 6144 extra parameters'"},{"comment":"The pre-training procedure is underspecified: what images and resolutions were used for the 'synthetic dataset of pair of 60000 input and its up-sampled version,' and what exactly is the up-sampled target? It is also unclear whether the up-sampler remains trainable after pre-training. Please provide enough detail to reproduce this step.","section":"Setup, 'Pre-training of the deconvolutional network'"},{"comment":"The notation 'CoarseFeats+FineFeats' in the center-loss line is ambiguous: it could mean addition, concatenation, or separate losses on both feature sets. Please clarify how the center loss is applied to the two feature matrices.","section":"Algorithm 1, line 12"},{"comment":"The description of orthogonal initialization is mathematically unclear. If the attention weights are an 8x768 matrix X, the SVD gives X = U Sigma V^T with V of size 768x768, so 'taking V as the initial weights' cannot directly initialize a layer with 8 output channels. Please specify the exact initialization procedure (e.g., initializing with the first singular vectors or with a reshaped orthogonal matrix).","section":"Orthogonal initialization"},{"comment":"The paper says the localization evaluation follows Zhang et al. 2018, but it does not describe how the predicted bounding box is extracted from the attention maps (e.g., thresholding and connected components). Adding this detail would improve reproducibility.","section":"Object Localization Results"}],"recommendation":"major_revision","confidential_remarks":"The missing ablation of the core mechanism is the main technical obstacle, and the iNaturalist dataset-size error is a serious reproducibility issue. If the authors can provide the requested ablations (bilinear versus learned up-sampler; fine network on unhighlighted images; removal of two-head averaging), correct the dataset statistics, and report variance, the paper would be substantially stronger. The novelty is incremental but the idea is clean and the comparison with WS-DAN is a useful baseline."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nBottom line: Coarse2Fine is a reasonable extension of WS-DAN that replaces the fixed bilinear up-sampler with a learned deconvolutional one and adds a second fine-grained head on the highlighted image. The writing is clear, the architecture is sensible, and the reported numbers are in a plausible range—89.5% on CUB, 93.4% on Aircraft, etc. I believe the results are real, but the paper does not actually demonstrate that its core idea is what produces them.\n\nWhat is new: the differentiable path from attention maps to raw input via a pretrained deconv up-sampler, plus orthogonal initialization of the attention weights. The orthogonal init ablation (Table 8) is the one clean experiment, and it shows a small but consistent gain. That is a real contribution, and the deconv up-sampler idea is worth testing. The paper is honest about the extra parameter count and gives a fair comparison to the WS-DAN baseline.\n\nThe soft spots are the usual ones for this type of paper, and they matter more because the claimed gains are small (0.1–1.6 points). No error bars or multiple seeds; no code released; and, most importantly, no ablation that isolates the deconv up-sampler. The authors assert that a bilinear up-sampler would degrade accuracy, citing Hu et al. 2019, but they never run that comparison inside their own architecture. They also never train the fine network on original images or randomly masked ones, so the feedback path itself is never tested. On top of that, Table 1 lists the iNaturalist 2017 training set as 12,000 images, which is off by a factor of about fifty; that is a factual error, not a nit, and it casts doubt on the iNaturalist result and on the care in reporting.\n\nThe stress-test note gets it right: compared to WS-DAN, the method changes multiple things at once—orthogonal init, shared backbone, two-head averaging, random attention selection, center loss on both feature sets. The 0.1–1.6 point gains cannot be attributed to the proposed up-sampler mechanism on the present evidence.\n\nThat said, this is not a crackpot paper. The architecture is grounded in the WS-DAN baseline, the experiments cover five standard datasets, and the localization results are consistent. If the authors add a proper ablation (bilinear vs. deconv up-sampler, and a no-feedback control), the paper would be a solid FGVC contribution. As it stands, it is a promising technical report that overclaims its own mechanism.\n\nFor peer review: I would send it to referees, but with the expectation that the ablation gap and the iNaturalist number get fixed. It is above the desk-reject line; it just needs to be sharpened.","headline":"Plausible WS-DAN extension with a learnable up-sampler, but the reported SOTA gains are not pinned to the proposed mechanism; the paper needs an ablation and a fix for a dataset table error before I would trust the claims.","tokens_in":12494,"tokens_out":3256,"would_cite":false,"duration_ms":29534,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Coarse2Fine, a two-stage training method that feeds attention-highlighted images back through a second classifier, surpasses the previous state of the art on five fine-grained visual classification benchmarks.","keywords":["fine-grained visual classification","visual attention","bilinear attention pooling","deconvolutional up-sampler","attention feedback","orthogonal initialization","weakly supervised object localization","image classification"],"falsifier":"Train the identical Coarse2Fine pipeline on CUB-200-2011 with a fixed bilinear up-sampler in place of the learned deconvolutional one, keeping all other settings the same; if top-1 accuracy stays at or above 89.5%, the claim that learning an inverse mapping to the input drives the improvement is falsified. A second check would feed the fine-grained network the original, unmasked image; matching accuracy would mean the highlighted-image path is not the source of the gain.","tokens_in":11480,"feed_emoji":"🐦","tokens_out":9874,"duration_ms":94444,"temperature":0.7,"pith_summary":"Coarse2Fine is a two-stage training method that makes a fine-grained classifier's attention maps answer for what they highlight in the raw image. The first network produces attention maps and a coarse prediction; one attention map is up-sampled by a learned deconvolutional network and multiplied with the input to form a highlighted image. A second classifier must recognize the class from that highlighted image, and its error gradient flows back to sharpen the attention maps. The paper reports that this feedback, plus orthogonal initialization of the attention weights, beats the previous state of the art on all five fine-grained benchmarks tested, including 89.5 percent on CUB-200-2011 birds and 70.5 percent on iNaturalist 2017. If the result holds, it offers a cheap way, only 6144 extra parameters, to improve part-level attention without any extra annotation.","feed_headline":"Attention feedback loop tops five fine-grained benchmarks","feed_subtitle":"A learnable up-sampler sends highlighted bird parts back through training, lifting CUB to 89.5 percent.","key_machinery":"The load-bearing object is the differentiable feedback path from attention maps to raw image, implemented by a deconvolutional up-sampler inserted between a Bilinear Attention Pooling (BAP) block and a second classifier. BAP forms one feature vector per attention map by pooling the element-wise product of feature maps and attention maps; each vector is meant to correspond to an object part such as a beak or headlight. In Coarse2Fine, one attention map is up-sampled into a mask, element-wise multiplied with the input, and passed to the fine-grained network, so the fine-grained cross-entropy gradient reaches the attention weights through real pixel values. A center loss on the part-feature rows and orthogonal SVD-based initialization of the attention convolution are auxiliary mechanisms that encourage each attention map to specialize on a distinct part. The up-sampler is pre-trained on synthetic image and up-sampled pairs so it starts as a reasonable interpolation before end-to-end training.","core_discovery":"The paper's central claim is that creating a differentiable path from the attended feature maps back to the input image improves fine-grained attention learning. Coarse2Fine cascades two classifiers that share a backbone: a coarse network produces bilinear attention maps, a deconvolutional up-sampler turns one randomly chosen attention map into a soft mask, and the mask is multiplied with the original image to feed a fine-grained classifier. Because the fine-grained loss is a function of the attention maps through this path, the attention maps are trained not only to separate classes at the feature level but also to isolate regions that let a second network make a fine distinction. The paper reports state-of-the-art top-1 accuracy on CUB-200-2011 (89.5%), FGVC-Aircraft (93.4%), Stanford Cars (94.7%), Stanford Dogs (93.0%), and iNaturalist 2017 (70.5%), and lower weakly supervised localization error than prior methods. It also argues that initializing the 1x1-convolution attention weights from the SVD of those weights, so attention maps start nearly orthogonal, contributes to the gain.","pith_inferences":["Because the fine-grained network is trained only on highlighted images, the method can be viewed as a learned, task-driven augmentation; an extension would be to apply the same feedback loop to one-stage detectors or part-segmentation heads, where the mask supervision could come from dense labels instead of a second classifier.","The random selection of one attention map per iteration resembles dropout over object parts, which suggests a testable variant: always sum all attention maps into the mask and measure whether specializing on a single part is what forces sharper attention.","The deconvolutional up-sampler is pre-trained on synthetic interpolation pairs, so an alternative explanation of the gain is that a warm-started, higher-capacity up-sampler regularizes the attention maps; comparing the learned up-sampler against a fixed bilinear one within this exact pipeline would settle which part of the path matters.","The improvement on the face-attribute task, where categories are not visually fine-grained in the usual sense, hints that the feedback path may help any task in which local appearance differences matter, not only species-level classification."],"forward_implications":["On the five tested benchmarks, Coarse2Fine with orthogonal initialization reports top-1 accuracy above every listed prior model: 89.5% on CUB-200-2011, 93.4% on FGVC-Aircraft, 94.7% on Stanford Cars, 93.0% on Stanford Dogs, and 70.5% on iNaturalist 2017.","The same attention maps improve weakly supervised object localization, with localization error falling to 17.2% on CUB-200-2011 and 17.9% on Stanford Dogs, below the prior WS-DAN numbers.","The full method adds only 6144 parameters over the InceptionV3 backbone, so the attention feedback path is cheap in model size.","Averaging the coarse and fine prediction heads gives the best accuracy, and the coarse head alone already beats a plain CNN, giving a practical inference trade-off.","The attention module transfers to a face-attribute task: on CelebA, adding the attention layers lifts average accuracy over the selected six attributes by about 2.1 percentage points."],"supporting_citations":[{"why":"Introduces Bilinear Attention Pooling, the attention mechanism the Coarse2Fine pipeline builds on.","marker":"Hu et al. 2018"},{"why":"WS-DAN, the strongest prior baseline; it uses a linear up-sampler that Coarse2Fine replaces with a learnable one.","marker":"Hu et al. 2019"},{"why":"Defines bilinear pooling, the foundation of BAP's part-feature extraction.","marker":"Lin et al. 2015b"},{"why":"Center loss used to pull each part feature row toward a per-class center during attention learning.","marker":"Wen et al. 2016"},{"why":"InceptionV3 backbone whose Mix6e features feed the attention maps.","marker":"Szegedy et al. 2016b"},{"why":"CUB-200-2011 dataset, the main bird benchmark where the method reports 89.5%.","marker":"Wah et al. 2011"},{"why":"iNaturalist 2017 dataset, the large species benchmark where Coarse2Fine reaches 70.5%.","marker":"Horn et al. 2017"},{"why":"Supplies the weakly supervised localization metric and ACoL baseline used for localization comparison.","marker":"Zhang et al. 2018"}],"fun_headline_variants":["Coarse2Fine feedback path tops five fine-grained benchmarks","Differentiable attention feedback sharpens fine-grained accuracy","Coarse2Fine's return path trains attention on five benchmarks","Coarse2Fine: feedback loop guides attention to fine details","Coarse2Fine and SVD-initialized attention top five benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that multiplying an up-sampled attention map with the raw image yields a highlighted image whose fine-grained classification error trains the attention maps to find more discriminative parts; the paper asserts this correlation between feature-map activation and input location but does not isolate it with an ablation, so the accuracy gains could in principle come from the extra classifier, the up-sampler capacity, or the auxiliary losses rather than from the attention feedback itself.","fun_headline_variants_meta":{"raw":{"variants":["Coarse2Fine feedback path tops five fine-grained benchmarks","Differentiable attention feedback sharpens fine-grained accuracy","Coarse2Fine's return path trains attention on five benchmarks","Coarse2Fine: feedback loop guides attention to fine details","Coarse2Fine and SVD-initialized attention top five benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000717,"raw_usage":{"total_tokens":3217,"prompt_tokens":938,"completion_tokens":2279,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":554,"completion_tokens_details":{"reasoning_tokens":2195}},"tokens_in":554,"tokens_out":2279,"duration_ms":19218,"temperature":1.0,"reasoning_tokens":2195,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T04:42:46.760925+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the identical Coarse2Fine pipeline on CUB-200-2011 with a fixed bilinear up-sampler in place of the learned deconvolutional one, keeping all other settings the same; if top-1 accuracy stays at or above 89.5%, the claim that learning an inverse mapping to the input drives the improvement is falsified. A second check would feed the fine-grained network the original, unmasked image; matching accuracy would mean the highlighted-image path is not the source of the gain.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Center loss used to pull each part feature row toward a per-class center during attention learning."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"CUB-200-2011 dataset, the main bird benchmark where the method reports 89.5%."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the weakly supervised localization metric and ACoL baseline used for localization comparison."}],"review_version":1}