REVIEW 4 major objections 6 minor 40 references
REAPS: Towards Better Recognition of Fine-grained Images by Region Attending and Part Sequencing
T0 review · 4 major / 6 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read A single network, trained with only image-level labels, reaches state-of-the-art fine-grained recognition by attending to the object region and modeling its parts as a sequence.
desk verdict REAPS is a plausible incremental combination of CAM cropping and sequence learning, but its state-of-the-art claim does not survive a direct comparison with the omitted NTS-Net baseline. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing mechanism is the 'soft-part' sequence model. After the CAM-based region attending network crops and zooms the attended region, the backbone's feature map $X\in\mathbb{R}^{H\times W\times C}$ is decomposed by a pooling operation into a sequence $Y=[Y_1,\dots,Y_N]$ of $C$-dimensional vectors, each describing a rectangular region of the raw image. A bi-directional LSTM then maps this sequence into $M$ implicit part representations, and the concatenated hidden states serve as the part-level feature; this replaces hard part localization with a learned ordering over spatial descriptors. The second component, the class activation map from Eq. (1), is what suppresses background and defines the attended region that the sequence model sees.
What would settle it
A concrete check: on CUB-200-2011, compute the CAM bounding box at $\tau=0.1$ for every test image and measure how often ground-truth part keypoints such as head, wing, or tail fall outside the crop; then retrain the pipeline under several values of $\tau$. If a substantial fraction of diagnostic parts fall outside at $\tau=0.1$, or if accuracy increases when the crop is enlarged, the central premise that the attended region preserves all discriminative details is disproved.
Extended reading notes
Core claim
The central discovery is that two previously separate ideas, region-level attention and part-level modeling, can be fused into one end-to-end, weakly supervised pipeline that matches or exceeds stronger-supervised methods. On Stanford Cars, FGVC-Aircraft, and CUB-200-2011, the proposed REAPS+ reaches 93.5%, 92.6%, and 86.8% respectively. The largest margin over the prior state of the art appears on FGVC-Aircraft, where REAPS+ surpasses MA-CNN by a relative 2.7% gain; the authors attribute this to the sequence model exploiting the strong spatial and sequential structure of aircraft. The paper also reports that adding the part-sequence branch to a plain backbone gives a relative performance gain of about 1.0% on Stanford Cars, and that the region-attending component outperforms FCAN and RA-CNN in the single-scale comparison.
Load-bearing premise
The method's accuracy rests on the assumption that the CAM-based attended region, cropped with threshold $\tau=0.1$, contains every detail needed to tell the classes apart; if the crop leaves out a diagnostic part, the part-sequence branch never gets to see it.
Editorial extensions
If this is right
- If REAPS is right, fine-grained recognition no longer needs bounding boxes or part keypoints at any stage; image-level labels are enough, which lowers annotation cost dramatically.
- Because parts are implicit and their count is not fixed, the method should tolerate occlusion and incomplete objects better than approaches that detect a preset number of semantic parts.
- Modeling parts as an ordered sequence lets the network exploit spatial context among parts, so categories with strong geometric ordering, such as aircraft, cars, and birds, stand to benefit most.
- The framework is stackable: feeding the attended region from one PSN into a second PSN yields further gains of 93.5%, 92.6%, and 86.8%, suggesting that accuracy improves with depth of region refinement.
Reading between the lines
- My inference: a testable extension the paper leaves open is to vary the CAM crop threshold $\tau$ and measure accuracy; if performance is sensitive to $\tau$, the method's robustness claim depends on a single unablated hyperparameter.
- My inference: the same sequence mapping could be applied to other fine-grained domains where parts are naturally ordered, such as text lines, handwritten characters, or fingerprint ridges.
- My inference: the paper's results on FGVC-Aircraft suggest, though the authors do not test it explicitly, that the sequence model's benefit grows with the regularity of part order, so categories with highly variable part arrangements might show smaller gains.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes REAPS, a weakly supervised fine-grained recognition framework that combines a CAM-based region attending network (RAN) with a part sequence-learning network (PSN). RAN computes a class activation map (Eq. 1), thresholds it to crop and zoom in on an attended region (Eq. 2), and PSN serializes the backbone features into N vectors, maps them to M implicit part representations with a bi-directional LSTM, and classifies with three combined softmax losses (Eq. 5). Experiments on Stanford Cars, FGVC-Aircraft, and CUB-200-2011 report REAPS+ accuracies of 93.5%, 92.6%, and 86.8%, respectively, and the paper claims state-of-the-art performance with only image-level labels.
Significance. The conceptual contribution is attractive: replacing fixed hard part detectors with an implicit soft-part sequence model on a CAM-cropped object region is simple, end-to-end trainable, and requires no part annotations. The paper evaluates on three standard benchmarks with a consistent backbone, and the qualitative visualizations support the claim that the part branch produces more compact feature maps. However, the central state-of-the-art claim is not supported by the published comparison: the directly comparable weakly supervised method NTS-Net (ECCV 2018) reports 93.9% on Stanford Cars, 91.4% on FGVC-Aircraft, and 87.5% on CUB-200-2011 with image-level labels, so REAPS+ is higher only on Aircraft. In addition, key hyperparameters needed for reproduction are not specified. The paper contains no machine-checked proofs or released code, so its value rests on the completeness and correctness of the empirical comparison, and that comparison is currently incomplete.
major comments (4)
- [Table 3 / Sec. 4.2] Table 3 and the discussion in Section 4.2 omit NTS-Net (Yang et al., ECCV 2018), a directly comparable weakly supervised fine-grained recognition method that reports 93.9% on Stanford Cars, 91.4% on FGVC-Aircraft, and 87.5% on CUB-200-2011 using only image-level labels. Under this published comparison, REAPS+ (93.5%, 92.6%, 86.8%) is not state of the art on two of the three benchmarks, and the sentence in Section 4.2 that REAPS+ "obtains the best performance on three datasets" is not supportable. The authors must add NTS-Net and any other comparable published weakly supervised baselines, and revise the state-of-the-art claims accordingly.
- [Sec. 3.3 / Sec. 4.1] Key hyperparameters of the proposed architecture are never specified: the sequence length N in Eq. (3), the implicit part count M in Eq. (4), the LSTM hidden size U, the number of LSTM layers, and the scaling factor used for the second PSN in REAPS+. Section 4.1 states only the loss weights lambda_1=lambda_2=lambda_3=1, the threshold tau=0.1, and the learning-rate schedule. Without these values the experiments cannot be reproduced, and the sensitivity of the reported margins to these choices cannot be assessed.
- [Eq. (2) / Sec. 4.2] The central design assumption is that the thresholded CAM crop in Eq. (2) preserves all diagnostic fine-grained parts before the part sequence model sees them, but the paper does not ablate tau or quantify the information loss caused by cropping. If the class activation map highlights only the most salient patch and drops other diagnostic regions, the crop removes information before PSN can model it; an ablation over tau values and a comparison of REAPS with and without cropping would be needed to substantiate the claim in Section 4.2 that RAN "effectively depress[es] background noise" without sacrificing fine-grained details.
- [Table 3 / Sec. 4.2] The reported differences are small on two of the three benchmarks (93.5 vs. 92.8 for MA-CNN and 93.9 for NTS-Net on Cars; 86.8 vs. 86.5 for MA-CNN and 87.5 for NTS-Net on CUB), yet no standard deviations, confidence intervals, or significance tests are reported. Given the missing baseline and unspecified hyperparameters, the conclusion that REAPS "outperforms the existing methods" is not robust; at minimum the authors should report the variance over multiple runs or justify single-run reporting.
minor comments (6)
- [Sec. 4.2] The statement that REAPS+ surpasses MA-CNN "by a relative 2.7% gain" on FGVC-Aircraft is arithmetically inaccurate: 92.6% vs. 89.9% is a 2.7 percentage-point improvement and a 3.0% relative improvement.
- [Sec. 4.2 / Table 2] The text says the part branch gives a "relative performance gain of 1.0%" for PSN on Stanford Cars, but the numbers 91.3% and 92.3% differ by 1.0 percentage point; the wording should be corrected.
- [Sec. 3.3 / Eq. (4)] There is an apparent inconsistency between the notation in Eq. (4), where the mapped part representation is [P_1,...,P_M] with M <= N, and the later sentence stating that the concatenated LSTM hidden states give P_P in R^{N x U}; the authors should clarify whether M=N in their implementation or describe the additional reduction step.
- [Table 3] The baseline named "REAPS wo PSN" is not clearly defined in the text; it should be stated explicitly which branches are removed and how this differs from the "PSN wo part" model in Table 2.
- [Abstract / Sec. 5] The abstract and conclusion claim "state of the art performance" without qualification; given the omitted NTS-Net comparison, this wording should be revised to a more defensible claim such as "competitive with current methods" unless the comparison is updated.
- [Sec. 3.3 / Fig. 3] The notation U and M appears in Fig. 3 without definition in the caption or main text; please define all symbols at first use.
Circularity Check
No circular derivation: REAPS is an empirical pipeline with fixed hyperparameters and external baselines.
full rationale
The paper's pipeline (CAM-based region attending, cropping, part-sequence LSTM, multi-loss training) contains no step in which an output quantity is defined in terms of itself, nor any 'prediction' that reduces to a fitted parameter. The only thresholds and weights (tau=0.1, lambda_j=1) are fixed manually and not tuned on the test set. The attention map in Eq. (1) is a standard weighted sum of convolutional activations using softmax weights; the crop in Eq. (2) uses a fixed threshold; the sequence mapping in Eq. (4) is a learned bi-LSTM. Accuracy is reported on held-out test sets from standard benchmarks. All cited prior works are external; there are no self-citations or imported uniqueness theorems that carry the argument. The omission of NTS-Net from Table 3 is a potential correctness/comparison gap, not a circularity. Therefore the derivation chain is self-contained and non-circular.
Assumptions & free parameters
free parameters (5)
- CAM bounding-box threshold tau =
0.1
- sequence length N =
not reported
- implicit part count M =
not reported
- LSTM hidden size U =
not reported
- loss weights lambda_1, lambda_2, lambda_3 =
1, 1, 1
assumptions (4)
- domain assumption ImageNet-pretrained VGG19 features transfer to the three fine-grained datasets, and fine-tuning the same backbone is sufficient for fair comparison.
- domain assumption The CAM-based region attention from Eq. (1), thresholded at tau, localizes the full object region rather than only a few discriminative spots.
- domain assumption Serializing the feature map into N rectangular blocks and pooling each block preserves the fine-grained visual details needed for classification.
- domain assumption A bidirectional LSTM can map the source sequence of N feature vectors to M implicit part representations without alignment or part-level supervision.
Cite this review
Pith. "Pith review of REAPS: Towards Better Recognition of Fine-grained Images by Region Attending and Part Sequencing." pith.science (2026). https://pith.science/paper/SFHST6I6
@misc{pith2026190801962,
author = {Pith},
title = {Pith review of: REAPS: Towards Better Recognition of Fine-grained Images by Region Attending and Part Sequencing},
year = {2026},
howpublished = {\url{https://pith.science/paper/SFHST6I6}},
note = {Machine review of arXiv:1908.01962}
}
read the original abstract
Fine-grained image recognition has been a hot research topic in computer vision due to its various applications. The-state-of-the-art is the part/region-based approaches that first localize discriminative parts/regions, and then learn their fine-grained features. However, these approaches have some inherent drawbacks: 1) the discriminative feature representation of an object is prone to be disturbed by complicated background; 2) it is unreasonable and inflexible to fix the number of salient parts, because the intended parts may be unavailable under certain circumstances due to occlusion or incompleteness, and 3) the spatial correlation among different salient parts has not been thoroughly exploited (if not completely neglected). To overcome these drawbacks, in this paper we propose a new, simple yet robust method by building part sequence model on the attended object region. Concretely, we first try to alleviate the background effect by using a region attention mechanism to generate the attended region from the original image. Then, instead of localizing different salient parts and extracting their features separately, we learn the part representation implicitly by applying a mapping function on the serialized features of the object. Finally, we combine the region attending network and the part sequence learning network into a unified framework that can be trained end-to-end with only image-level labels. Our extensive experiments on three fine-grained benchmarks show that the proposed method achieves the state of the art performance.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
- [1]
- [2]
- [3]
-
[4]
Branson, S., Horn, G.V., Belongie, S.J., Perona, P.: Bird Species Categorization Using Pose Normalized Deep Convolutional Nets. CoRR abs/1406.2952 (2014)
arXiv 2014
- [5]
- [6]
- [7]
- [8]
Show all 40 references
-
[9]
In: CVPR
Fu, J., Zheng, H., Mei, T.: Look Closer to See Better: Recurrent Attention Con- volutional Neural Network for Fine-Grained Image Recognition. In: CVPR. pp. 4476–4484 (2017)
2017
-
[10]
In: CVPR
Gao, Y., Beijbom, O., Zhang, N., Darrell, T.: Compact Bilinear Pooling. In: CVPR. pp. 317–326 (2016)
2016
-
[11]
In: ICCV
Gavves, E., Fernando, B., Snoek, C.G.M., Smeulders, A.W.M., Tuytelaars, T.: Fine-Grained Categorization by Alignments. In: ICCV. pp. 1713–1720 (2013)
2013
-
[12]
IEEE Trans
He, X., Peng, Y., Zhao, J.: Fast Fine-Grained Image Classification via Weakly Supervised Discriminative Localization. IEEE Trans. Circuits Syst. Video Techn. 29(5), 1394–1407 (2019)
2019
-
[13]
Neural Computation 9(8), 1735–1780 (1997)
Hochreiter, S., Schmidhuber, J.: Long Short-Term Memory. Neural Computation 9(8), 1735–1780 (1997)
1997
-
[14]
In: CVPR
Huang, S., Xu, Z., Tao, D., Zhang, Y.: Part-Stacked CNN for Fine-Grained Visual Categorization. In: CVPR. pp. 1173–1182 (2016)
2016
-
[15]
In: NIPS
Jaderberg, M., Simonyan, K., Zisserman, A., Kavukcuoglu, K.: Spatial Transformer Networks. In: NIPS. pp. 2017–2025 (2015)
2015
-
[16]
In: CVPR Workshop on FGVC
Khosla, A., Jayadevaprakash, N., Yao, B., Li, F.F.: Novel dataset for fine-grained image categorization: Stanford dogs. In: CVPR Workshop on FGVC. vol. 2 (2011)
2011
-
[17]
In: CVPR
Kong, S., Fowlkes, C.C.: Low-Rank Bilinear Pooling for Fine-Grained Classifica- tion. In: CVPR. pp. 7025–7034 (2017)
2017
-
[18]
In: CVPR
Krause, J., Jin, H., Yang, J., Li, F.: Fine-grained recognition without part anno- tations. In: CVPR. pp. 5546–5555 (2015)
2015
-
[19]
In: CVPR
Lin, D., Shen, X., Lu, C., Jia, J.: Deep LAC: deep localization, alignment and classification for fine-grained recognition. In: CVPR. pp. 1666–1674 (2015)
2015
-
[20]
In: BMVC (2017)
Lin, T., Maji, S.: Improved Bilinear Pooling with CNNs. In: BMVC (2017)
2017
-
[21]
In: ICCV
Lin, T., Roy Chowdhury, A., Maji, S.: Bilinear CNN Models for Fine-Grained Visual Recognition. In: ICCV. pp. 1449–1457 (2015) 12 Peng et al
2015
-
[22]
In: ECCV
Liu, J., Kanazawa, A., Jacobs, D.W., Belhumeur, P.N.: Dog Breed Classification Using Part Localization. In: ECCV. pp. 172–185 (2012)
2012
-
[23]
CoRR abs/1603.06765 (2016)
Liu, X., Xia, T., Wang, J., Lin, Y.: Fully Convolutional Attention Localization Networks: Efficient Attention Localization for Fine-Grained Recognition. CoRR abs/1603.06765 (2016)
2016 arXiv
-
[24]
HAL - INRIA (2013)
Maji, S., Rahtu, E., Kannala, J., Blaschko, M.B., Vedaldi, A.: Fine-Grained Visual Classification of Aircraft. HAL - INRIA (2013)
2013
-
[25]
In: BMVC (2016)
Moghimi, M., Belongie, S.J., Saberian, M.J., Yang, J., Vasconcelos, N., Li, L.: Boosted Convolutional Neural Networks. In: BMVC (2016)
2016
-
[26]
In: Sixth Indian Conference on Computer Vision, Graphics & Image Processing, Bhubaneswar, India
Nilsback, M., Zisserman, A.: Automated Flower Classification over a Large Number of Classes. In: Sixth Indian Conference on Computer Vision, Graphics & Image Processing, Bhubaneswar, India. pp. 722–729 (2008)
2008
-
[27]
In: ECCV
Perronnin, F., S´ anchez, J., Mensink, T.: Improving the Fisher Kernel for Large- Scale Image Classification. In: ECCV. pp. 143–156 (2010)
2010
-
[28]
In: CVPR
Qian, Q., Jin, R., Zhu, S., Lin, Y.: Fine-grained visual categorization via multi- stage metric learning. In: CVPR. pp. 3716–3724 (2015)
2015
-
[29]
Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The Caltech-UCSD Birds-200-2011 Dataset. Tech. Rep. CNS-TR-2011-001, California Institute of Technology (2011)
2011
-
[30]
In: ICCV
Wang, D., Shen, Z., Shao, J., Zhang, W., Xue, X., Zhang, Z.: Multiple Granularity Descriptors for Fine-Grained Categorization. In: ICCV. pp. 2399–2406 (2015)
2015
-
[31]
In: CVPR
Wang, Y., Choi, J., Morariu, V.I., Davis, L.S.: Mining Discriminative Triplets of Patches for Fine-Grained Classification. In: CVPR. pp. 1163–1172 (2016)
2016
-
[32]
Pattern Recognition 76, 704–714 (2018)
Wei, X., Xie, C., Wu, J., Shen, C.: Mask-CNN: Localizing parts and selecting descriptors for fine-grained bird species categorization. Pattern Recognition 76, 704–714 (2018)
2018
-
[33]
In: CVPR
Xiao, T., Xu, Y., Yang, K., Zhang, J., Peng, Y., Zhang, Z.: The application of two-level attention models in deep convolutional neural network for fine-grained image classification. In: CVPR. pp. 842–850 (2015)
2015
-
[34]
In: ICML
Xu, K., Ba, J., Kiros, R., Cho, K., Courville, A.C., Salakhutdinov, R., Zemel, R.S., Bengio, Y.: Show, Attend and Tell: Neural Image Caption Generation with Visual Attention. In: ICML. pp. 2048–2057 (2015)
2015
-
[35]
In: CVPR
Zhang, N., Farrell, R., Darrell, T.: Pose pooling kernels for sub-category recogni- tion. In: CVPR. pp. 3665–3672 (2012)
2012
-
[36]
In: ICCV
Zhang, N., Farrell, R., Iandola, F.N., Darrell, T.: Deformable Part Descriptors for Fine-Grained Recognition and Attribute Prediction. In: ICCV. pp. 729–736 (2013)
2013
-
[37]
In: CVPR
Zhang, X., Zhou, F., Lin, Y., Zhang, S.: Embedding Label Structures for Fine- Grained Feature Representation. In: CVPR. pp. 1114–1123 (2016)
2016
-
[38]
In: CVPR
Zhang, X., Xiong, H., Zhou, W., Lin, W., Tian, Q.: Picking Deep Filter Responses for Fine-Grained Image Recognition. In: CVPR. pp. 1134–1142 (2016)
2016
-
[39]
In: ICCV
Zheng, H., Fu, J., Mei, T., Luo, J.: Learning Multi-attention Convolutional Neural Network for Fine-Grained Image Recognition. In: ICCV. pp. 5219–5227 (2017)
2017
-
[40]
In: CVPR
Zhou, B., Khosla, A., Lapedriza, `A., Oliva, A., Torralba, A.: Learning Deep Fea- tures for Discriminative Localization. In: CVPR. pp. 2921–2929 (2016)
2016
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.