REVIEW 5 major objections 5 minor 30 references
Enhancing Fine-grained Image Classification through Attentive Batch Training
T0 review · 5 major / 5 minor · reviewed 2026-08-11 · deepseek-v4-flash
Pith's one-line read A small plug-in module that makes image classifiers attend to relationships between images in the same training batch raises fine-grained classification accuracy by 2–6 points and sets a state-of-the-art 95.79% on Stanford Dogs.
desk verdict A plausible plug-in with a parameter-count confound; the cross-image mechanism is interesting but not yet disentangled from added capacity. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing component is the Residual Relationship Attention (RRA) module, a batch-level attention layer. It duplicates each image's feature embedding into query, key, and value tensors of shape $B\times B\times D$, so that every ordered pair $(i,j)$ has its own interaction slot. The relationship position encoding (RPE) supplies a per-pair scalar $s_{ij}$ computed as normalized PSNR between the raw input images, and this scalar is added to the keys and values inside the attention computation. A gating parameter $\beta$, learned from the concatenation of the attention embedding and the original features, blends the two via batch-normalized residual summation. The effect is that each image's final representation is a weighted mix of its own features and features borrowed from all other images in the batch, with the mix controlled by visual similarity and by the classification loss.
What would settle it
A direct falsifier is to take any reported baseline/backbone pair, for example ConvNeXtLarge on Stanford Dogs, and train it with the authors' exact uniform configuration both with and without RBI, using the same random seeds and hyperparameters, then measure the accuracy difference over multiple runs; if the average gain does not reach the reported +2.08 points (or if the baseline reproduces the published literature accuracy rather than the authors' lower number), the central claim of consistent improvement is not supported.
Extended reading notes
Core claim
The central claim is that in fine-grained classification, a classifier trained to explicitly integrate features across the images of a batch learns more discriminative features than one that sees each image in isolation. The RBI framework adds two ingredients to any backbone: RPE computes a pairwise similarity matrix between input images using PSNR, and RRA uses that matrix to produce attention embeddings that combine each image's features with those of the other batch members, with a residual gate that balances original and integrated features. The authors report that this consistently improves accuracy across nine model/dataset combinations, with larger gains on CNN backbones (3–6 points) than on transformer backbones (1–2 points), and that a smaller backbone plus RBI can beat a larger backbone without it.
Load-bearing premise
The claimed gains depend on the fairness of the authors' baseline protocol: every baseline is retrained with the authors' uniform configuration, which they acknowledge produces accuracies that diverge from published numbers, so the reported deltas and the 95.79% state-of-the-art claim could be inflated if that configuration handicaps the baselines.
Editorial extensions
If this is right
- RBI behaves as a plug-in: adding it to existing CNN and transformer classifiers raises accuracy under the authors' training protocol.
- Smaller backbones with RBI can match or beat larger backbones without it; for example, SwinT-Small-RBI beats SwinT-Big and ConvNeXtBase-RBI beats ConvNeXtLarge, with little extra inference time.
- The gains are larger for CNN backbones than for transformer backbones, suggesting the module's cross-image attention is most additive when the backbone lacks built-in self-attention.
- With batch size 1 at test time, accuracy is largely retained, so inference does not strictly require a fixed batch.
- The method also transfers to general (non-fine-grained) classification, with a claimed 93.71% on Tiny-ImageNet.
Reading between the lines
- The success of RPE suggests that raw pixel similarity (PSNR) is a useful prior for which images should exchange features; an ablation replacing RPE with learnable pairwise similarities could isolate how much of the gain comes from that prior vs. the attention mechanism itself.
- If the gains are real and protocol-independent, cross-image feature exchange could be applied to other instance- or pair-based tasks like retrieval or few-shot learning, where batch composition also matters.
- The authors' uniform re-training of baselines is both the strength (fair internal comparison) and the risk (literature numbers differ); a fair external check would be to run RBI on top of released checkpoints and exact training recipes of the original baselines.
- The batch-size sensitivity plot suggests that training with larger batches may yield more stable gains; the method's benefit might interact with batch composition (e.g., class balance within a batch), which is not studied.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The manuscript proposes a plug-in module, Relationship Batch Integration (RBI), which combines Residual Relationship Attention (RRA) and Relationship Position Encoding (RPE). RPE computes a raw-pixel PSNR-based similarity matrix between images in a batch, and RRA uses that matrix in attention operations (Eqs. 7--13) to mix DNN feature embeddings across batch images. The authors retrain ten baseline models with and without RBI on CUB-200-2011, Stanford Dogs, and NABirds, report consistent accuracy improvements, claim an average gain of roughly +2.5 to +3.8 points, and claim a state-of-the-art 95.79% on Stanford Dogs and 93.71% on Tiny-ImageNet. The central claim is that RBI is a general cross-image attentive batch integration mechanism that improves fine-grained classifiers without changing the backbone.
Significance. If the central claim were established, RBI would be a useful and simple plug-in module: the paper evaluates a broad set of backbone/dataset combinations, reports inference times and parameter counts, and the direction of the accuracy changes is consistently positive across the table. The module targets a real problem in fine-grained classification, and the idea of using raw-pixel similarity as a relationship prior is clearly stated. However, the current evidence does not establish that the cross-image relationship mechanism is the cause of the gains, the baseline protocol is explicitly acknowledged to produce accuracies that diverge from the literature, and several headline numbers are internally inconsistent. The paper also makes a Tiny-ImageNet claim that is not backed by any experiment in the manuscript. These are fixable with additional experiments and rewriting, but they are load-bearing for the paper's main claims.
major comments (5)
- [Proposed Approach / Residual Relationship Attention (RRA); Experiments / The Impact of Batch Configurations] The central claim that cross-image relationship integration causes the observed gains is not established. Under test-time batch size B=1, Eqs. (10)--(11) reduce to per-sample transformations because S and A are 1x1, yet Figure 3 reports that most RBI variants maintain accuracy within 1% (and MobileNetV3-S-RBI shows the largest variability, not a collapse). This invariance is consistent with the gains arising from the large added parameter budget (e.g., MobileNetV3-S 1.6M to 17.4M; DenseNet201 18.3M to 73.7M in Table 1) rather than from the relationship signal. A capacity-matched control that keeps the RRA/RPE architecture but disables cross-image mixing during training (e.g., identity similarity matrix) is needed to attribute the improvement to the proposed mechanism.
- [Experiments / Comparison results] The fairness of the baseline protocol is load-bearing and currently acknowledged as a limitation: the authors state that the reported accuracies "diverge from those in the literature" because of their uniform training configurations. Since every baseline is retrained under this protocol, the bracketed gains, the "smaller variant beats larger variant" comparisons, and the claimed state-of-the-art 95.79% on Stanford Dogs are all relative to potentially handicapped baselines. Please report the original implementations' results under the authors' protocol, or provide evidence that the uniform configuration does not systematically disadvantage the baselines; without this, the state-of-the-art claim is not supported.
- [Abstract and Experiments / Datasets] The abstract claims a state-of-the-art result of 93.71% on Tiny-ImageNet, but the Experiments section states that experiments are conducted on only three fine-grained datasets (CUB-200-2011, Stanford Dogs, NABirds) and no Tiny-ImageNet protocol, table, or result appears anywhere in the manuscript. This unsupported claim must either be removed or substantiated with the corresponding experimental details.
- [Experiments / Comparison results / Table 1] The quantitative claims are internally inconsistent. The abstract attributes +2.78% to CUB-200-2011 and +3.83% to Stanford Dogs; the text attributes +2.78% to Stanford Dogs and +3.83% to CUB-200-2011; Table 1's "Avg. Improvement" row lists +2.51/+3.46/+3.04; and direct arithmetic means of the bracketed per-row deltas are approximately +2.67/+3.33/+3.03. These discrepancies need to be reconciled before the average-improvement claims can be evaluated.
- [Experiments / Implementation details and Table 1] No repeated runs or error bars are reported. Several of the claimed improvements are small (+0.28 on Stanford Dogs for HERB-SwinT-RBI, +0.47 for TransFG-RBI on CUB-200-2011, +0.55 on NABirds), so without run-to-run variance it is not possible to determine whether these differences are significant. Report at least three seeds and show the mean and standard deviation for each model-dataset combination.
minor comments (5)
- [Throughout] Several section cross-references are empty: "described in details in Section ." in the Proposed Approach, "in section ." in the Experiments / Datasets subsection, and "section ." in the Batch size experiment. These need to be filled in or removed.
- [Proposed Approach / Relationship Position Encoding] Equation (1) is typographically garbled: the summation limits are rendered as "2X x=0", "H-1X y=0", and "W -1X z=0". The formula should clearly sum over the three color channels and the spatial dimensions.
- [Experiments / Datasets] The dataset name "CUB-200-201" should be "CUB-200-2011"; this typo appears in the Datasets paragraph.
- [Conclusion] The Conclusion states that the architectural innovation "significantly reduces model parameters" compared to conventional DNN architectures, but Table 1 shows that every RBI variant increases parameter count (e.g., P2P-Net 63.4M to 110.6M). Please qualify this sentence to refer to the smaller-variant-versus-larger-baseline comparison rather than to the same backbone with and without RBI.
- [Conclusion] The sentence "In this study, we has proposed" contains a grammatical error and should be corrected.
Circularity Check
No circularity found: the RPE/RRA module is defined from raw inputs, the claimed gains are measured on held-out test sets, and no fitted parameter is relabeled as a prediction.
full rationale
The proposed framework's only fixed component, RPE, computes a normalized PSNR from raw pixel values (Eqs. 1-2), so it is an input-derived encoding rather than a parameter fitted to the outcome. RRA's attention (Eqs. 7-13) is trained end-to-end and its contribution is evaluated on held-out test images, not fitted to the test labels. The accuracy deltas in Table 1 are comparisons against separately retrained baselines; even though the baselines use the authors' uniform configuration and diverge from literature numbers, this affects fairness, not circularity. The batch-size experiment showing test accuracy persists at batch size 1 undermines the mechanistic claim that cross-image relationships drive the gain, but that is an evidentiary limitation rather than a definitional reduction of a prediction to its input. There are no load-bearing self-citations: the references are standard prior work and are not invoked to license the method's assumptions. The abstract/table average mismatch (+2.78/+3.83/+3.29 vs +2.51/+3.46/+3.04) is an internal inconsistency, not circularity. Therefore circularity score is 0.
Assumptions & free parameters
free parameters (5)
- Batch size =
32
- Embedding dimension =
1024
- Learning rate =
1e-5
- Training epochs =
50
- RPE epsilon =
unspecified small value
assumptions (3)
- domain assumption Pretrained ImageNet1K weights plus 50 epochs of fine-tuning are sufficient for all backbones and datasets.
- ad hoc to paper Raw-pixel PSNR between two images is a useful prior for fusing their feature vectors.
- domain assumption The uniform training configuration is a fair protocol for every baseline.
invented entities (2)
-
Residual Relationship Attention (RRA)
-
Relationship Position Encoding (RPE)
Cite this review
Pith. "Pith review of Enhancing Fine-grained Image Classification through Attentive Batch Training." pith.science (2026). https://pith.science/paper/AWFND2FL
@misc{pith2026241219606,
author = {Pith},
title = {Pith review of: Enhancing Fine-grained Image Classification through Attentive Batch Training},
year = {2026},
howpublished = {\url{https://pith.science/paper/AWFND2FL}},
note = {Machine review of arXiv:2412.19606}
}
abstract
Fine-grained image classification, which is a challenging task in computer vision, requires precise differentiation among visually similar object categories. In this paper, we propose 1) a novel module called Residual Relationship Attention (RRA) that leverages the relationships between images within each training batch to effectively integrate visual feature vectors of batch images and 2) a novel technique called Relationship Position Encoding (RPE), which encodes the positions of relationships between original images in a batch and effectively preserves the relationship information between images within the batch. Additionally, we design a novel framework, namely Relationship Batch Integration (RBI), which utilizes RRA in conjunction with RPE, allowing the discernment of vital visual features that may remain elusive when examining a singular image representative of a particular class. Through extensive experiments, our proposed method demonstrates significant improvements in the accuracy of different fine-grained classifiers, with an average increase of $(+2.78\%)$ and $(+3.83\%)$ on the CUB200-2011 and Stanford Dog datasets, respectively, while achieving a state-of-the-art results $(95.79\%)$ on the Stanford Dog dataset. Despite not achieving the same level of improvement as in fine-grained image classification, our method still demonstrates its prowess in leveraging general image classification by attaining a state-of-the-art result of $(93.71\%)$ on the Tiny-Imagenet dataset. Furthermore, our method serves as a plug-in refinement module and can be easily integrated into different networks.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
, " * write output.state after.block = add.period write newline
ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Chou, P.-Y.; Kao, Y.-Y.; and Lin, C.-H. 2023. Fine-grained Visual Classification with High-temperature Refinement and Background Suppression. arXiv:2303.06442
arXiv 2023
-
[4]
Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; Uszkoreit, J.; and Houlsby, N. 2021. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. arXiv:2010.11929
arXiv 2021
-
[5]
He, J.; Chen, J.-N.; Liu, S.; Kortylewski, A.; Yang, C.; Bai, Y.; and Wang, C. 2022. Transfg: A transformer architecture for fine-grained recognition. In Proceedings of the AAAI conference on artificial intelligence, volume 36, 852--860
2022
-
[6]
He, X.; and Peng, Y. 2017. Fine-Grained Image Classification via Combining Vision and Language. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). IEEE
work page 2017
-
[7]
V.; Branson, S.; Farrell, R.; Haber, S.; Barry, J.; andS
Horn, G. V.; Branson, S.; Farrell, R.; Haber, S.; Barry, J.; andS. Belongie, P. I.; and Perona, P. 2015 a . Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection. In 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 595--604. Los Alamitos, CA, USA: IEEE Comp...
work page 2015
-
[8]
V.; Branson, S.; Farrell, R.; Haber, S.; Barry, J.; Ipeirotis, P.; Perona, P.; and Belongie, S
Horn, G. V.; Branson, S.; Farrell, R.; Haber, S.; Barry, J.; Ipeirotis, P.; Perona, P.; and Belongie, S. 2017. The iNaturalist species classification and detection dataset,. In CVPR, 8769–8778
work page 2017
Show all 30 references
-
[9]
V.; Cole, E.; Beery, S.; Wilber, K.; Belongie, S.; and Aodha, O
Horn, G. V.; Cole, E.; Beery, S.; Wilber, K.; Belongie, S.; and Aodha, O. M. 2015 b . Benchmarking representation learning for natural world image collections,. In CVPR
2015
-
[10]
G.; Zhu, M.; Chen, B.; Kalenichenko, D.; Wang, W.; Weyand, T.; Andreetto, M.; and Adam, H
Howard, A. G.; Zhu, M.; Chen, B.; Kalenichenko, D.; Wang, W.; Weyand, T.; Andreetto, M.; and Adam, H. 2017. MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications
2017
-
[11]
Hu, J.; Shen, L.; and Sun, G. 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, 7132--7141
2018
-
[12]
Huang, G.; Liu, Z.; Van Der Maaten, L.; and Weinberger, K. Q. 2017. Densely Connected Convolutional Networks. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2261--2269
2017
-
[13]
Khosla, A.; Jayadevaprakash, N.; Yao, B.; and Fei-Fei, L. 2011. Novel Dataset for Fine-Grained Image Categorization. In First Workshop on Fine-Grained Visual Categorization (FGVC)
2011
-
[14]
Kim, S.; Nam, J.; and Ko, B. C. 2022. V i T - N e T : Interpretable Vision Transformers with Neural Tree Decoder. In Chaudhuri, K.; Jegelka, S.; Song, L.; Szepesvari, C.; Niu, G.; and Sabato, S., eds., Proceedings of the 39th International Conference on Machine Learning, volum...
2022
-
[15]
Liu, X.; Xia, T.; Wang, J.; Yang, Y.; Zhou, F.; and Lin, Y. 2017. Fully Convolutional Attention Networks for Fine-Grained Recognition. arXiv:1603.06765
2017 arXiv
-
[16]
Liu, Z.; Lin, Y.; Cao, Y.; Hu, H.; Wei, Y.; Zhang, Z.; Lin, S.; and Guo, B. 2021. Swin Transformer: Hierarchical Vision Transformer using Shifted Windows. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), 9992--10002
2021
-
[17]
Liu, Z.; Mao, H.; Wu, C.-Y.; Feichtenhofer, C.; Darrell, T.; and Xie, S. 2022. A ConvNet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)
2022
-
[18]
Lu, F.; Li, W.; Li, C.; Liu, S.; Wu, D.; Fang, M.; Zou, X.; Li, M.; Zheng, R.; Ren, Y.; Liao, X.; Jin, H.; and Zomaya, A. Y. 2023. Fine-grained Lesion Classification Framework for Early Auxiliary Diagnosis. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 1--13
2023
-
[19]
Mnih, V.; Heess, N.; Graves, A.; et al. 2014. Recurrent models of visual attention. Advances in neural information processing systems, 27
2014
-
[20]
Pan, Y.; Xia, Y.; and Shen, D. 2019. Foreground Fisher Vector: Encoding Class-Relevant Foreground to Improve Image Classification. IEEE Transactions on Image Processing, 28(10): 4716--4729
2019
-
[21]
R.; Cogswell, M.; Das, A.; Vedantam, R.; Parikh, D.; and Batra, D
Selvaraju, R. R.; Cogswell, M.; Das, A.; Vedantam, R.; Parikh, D.; and Batra, D. 2019. Grad- CAM : Visual Explanations from Deep Networks via Gradient-Based Localization. International Journal of Computer Vision, 128(2): 336--359
2019
-
[22]
Van Horn, G.; Branson, S.; Farrell, R.; Haber, S.; Barry, J.; Ipeirotis, P.; Perona, P.; and Belongie, S. 2015. Building a bird recognition app and large scale dataset with citizen scientists: The fine print in fine-grained dataset collection. In 2015 IEEE Conference on Comput...
2015
-
[23]
N.; Kaiser, .; and Polosukhin, I
Vaswani, A.; Shazeer, N.; Parmar, N.; Uszkoreit, J.; Jones, L.; Gomez, A. N.; Kaiser, .; and Polosukhin, I. 2017. Attention is all you need. Advances in neural information processing systems, 30
2017
-
[24]
Wah, C.; Branson, S.; Welinder, P.; Perona, P.; and Belongie., S. 2011. The Caltech-UCSD Birds-200-2011 Dataset, Computation & Neural Systems, Technical Report, CNS-TR, California Institute of Technology, USA
2011
-
[25]
Wen, S.; Chen, Y.; Guo, S.; Ma, Y.; Gu, Y.; and Chan, P. 2023. Discriminative Domain Adaptation Network for Fine-grained Disease Severity Classification. In 2023 International Joint Conference on Neural Networks (IJCNN), 1--8
2023
-
[26]
Woo, S.; Park, J.; Lee, J.-Y.; and Kweon, I. S. 2018. Cbam: Convolutional block attention module. In Proceedings of the European conference on computer vision (ECCV), 3--19
2018
-
[27]
Xiao, T.; Xu, Y.; Yang, K.; Zhang, J.; Peng, Y.; and Zhang, Z. 2014. The Application of Two-level Attention Models in Deep Convolutional Neural Network for Fine-grained Image Classification. arXiv:1411.6447
2014 arXiv
-
[28]
Xu, K.; Ba, J.; Kiros, R.; Cho, K.; Courville, A.; Salakhudinov, R.; Zemel, R.; and Bengio, Y. 2015. Show, attend and tell: Neural image caption generation with visual attention. In International conference on machine learning, 2048--2057. PMLR
2015
-
[29]
Yang, X.; Wang, Y.; Chen, K.; Xu, Y.; and Tian, Y. 2022. Fine-grained object classification via self-supervised pose alignment. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 7399--7408
2022
-
[30]
Zhang, C.; He, J.; and Shang, L. 2023. An X-ray image classification method with fine-grained features for explainable diagnosis of pneumoconiosis. Personal and Ubiquitous Computing, 1--13
2023
Reviewed August 11, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.