REVIEW 2 major objections 5 minor 32 references
FlexiGrad: Adaptive Gradient Modulation for Hierarchical Fine-Grained Classification
T0 review · 2 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash
Pith's one-line read FlexiGrad, a parameter-free gradient-modulation rule, removes only the conflicting coarse–fine component and reinforces partial agreement, lifting multi-granularity accuracy on three fine-grained benchmarks.
desk verdict PCGrad plus a cosine-weighted cooperation branch; consistent but modest gains, yet the update rule is ambiguous and the theory is a non sequitur. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central mechanism is the pairwise cosine similarity cosθ_ij between the gradients of two granularity tasks. When the cosine is negative, FlexiGrad subtracts the projection of one gradient onto the other, removing only the conflicting component; when it is positive, it blends the two gradients with a smooth linear weight (1+cos θ_ij)/2, reinforcing the shared direction while preserving each task's individual component. The corrected gradients are summed to form the final parameter update, so the method intervenes purely at backpropagation time and needs no architecture changes or extra parameters.
What would settle it
On the three reported benchmarks, train with the same schedule but replace the cosine-based weights with a fixed constant w=0.5 (or with randomly shuffled pairwise cosines). If the accuracy gain over vanilla joint training matches FlexiGrad's, the alignment-sensitivity of the rule is not what drives the improvement. A second check: train with plain SGD instead of Adam; the theory only certifies descent for the aggregate gradient, so if the gain vanishes under SGD, the mechanism's causal claim would be in doubt.
Extended reading notes
Core claim
The paper's central claim is that the naive sum of coarse and fine task gradients is suboptimal because it treats all cross-level interactions as equally compatible. FlexiGrad replaces it with a hierarchy-aware update: for each pair of task gradients, on conflict (negative cosine) the projection of one onto the other is removed; on partial agreement the gradient is re-weighted as a convex combination of itself and its projection, using w_ij = (1+cos θ_ij)/2. The paper shows each corrected gradient keeps a non-negative inner product with its original, so the sum remains a descent direction for the total loss. Empirically, on three benchmarks and across four base models, this yields higher mul
Load-bearing premise
The load-bearing premise is that coarse and fine supervision are best treated as two separate tasks whose harmful interactions can be repaired by pairwise gradient projections; if the reported gains actually come from Adam-specific dynamics or from an accidental regularisation effect of the reweighting, the 'hierarchical gradient conflict' explanation would not be the cause.
Editorial extensions
If this is right
- Joint training with FlexiGrad reports higher accuracy at every granularity than vanilla joint training, with the largest absolute gains at the finest label level (species or model).
- FlexiGrad is a drop-in backpropagation module: no architectural change, no extra parameters, and only a small training-time overhead (about 7% in the reported CUB setting).
- The gains persist across backbone families: the paper reports improvements for both CNN-based and transformer-based fine-grained models when FlexiGrad is plugged in.
- The improvement is largest precisely where coarse-to-fine interference is strongest: hard families and subtrees show the biggest species-level gains, up to about 19 percentage points on CUB in the paper's analysis.
- Because the corrected update is a surrogate descent direction for the total loss, the method can be combined with any optimiser and any hierarchical head setup without changing the objective.
Reading between the lines
- The same pairwise alignment rule should extend to hierarchies deeper than two levels and to any ordered set of tasks, since the procedure is already sequential and pairwise; a testable variant would apply it to kingdom–phylum–class–order taxonomies and check whether per-level gains compound.
- The paper's descent-direction guarantee covers only the aggregate gradient, so the method's benefit under adaptive optimisers could partly be a learning-rate interaction; running FlexiGrad with plain SGD under the same schedule would isolate the mechanism's contribution.
- Because the weight w=(1+cos θ)/2 assumes gradients are on comparable scales, per-task loss scaling could change the outcome; a whitened-gradient variant would test whether normalisation is needed for very different loss magnitudes.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes FlexiGrad, a parameter-free gradient-modulation method for hierarchical fine-grained classification. Given task gradients from coarse and fine classifiers, FlexiGrad measures pairwise cosine similarity, removes the conflicting component when gradients disagree, and mixes the gradient with a projected component using weight w_ij=(1+cosθ_ij)/2 when they partially agree. The corrected per-task gradients are summed and used in place of the raw multi-task gradient. Experiments on CUB-200-2011, FGVC-Aircraft, and Stanford Cars with ResNet-50, Swin-B, ViT-B/16, and several FGVC models report consistent accuracy gains over Vanilla single, PCGrad, and FGoN, with ablations and visualizations supporting the claimed behavior.
Significance. If the exact update is specified and the empirical results are reproducible, FlexiGrad would be a useful, simple, and architecture-agnostic contribution to hierarchical FGVC. It is genuinely parameter-free apart from a small epsilon, it does not alter the network structure, and the reported gains are consistent across datasets and backbones. The core idea of treating coarse-to-fine gradient interactions asymmetrically and continuously, rather than symmetrically projecting or blocking them, is well motivated. The paper also gives credit for reporting standard deviations over three runs and for ablating the two components. However, two load-bearing issues currently prevent acceptance: the algorithm is not uniquely specified, and the theoretical descent-direction claim is not proven.
major comments (2)
- [§3.5 / Algorithm 1] Algorithm 1 (lines 3–15) defines a different update from the one described in §3.5. In the pseudocode the inner loop is over tasks with k > i, so the last task's gradient is never modified (g_FG^K = g_K) and the ordering of tasks matters. §3.5 states that each task 'sequentially applies the conflict or cooperation rule with all other tasks j ≠ i', which is symmetric. These two rules are not equivalent. The numbers in Tables 1–4 do not reveal which rule was used, and even the sequential version is ambiguous about whether the numerator in lines 8 and 12 uses the raw g_i or the already-updated g_FG^i. This makes the empirical claim unreproducible as stated. Please correct the pseudocode or the text, specify the exact rule, justify the ordering, and release code so the reported gains can be attributed to a single method.
- [§3.6, Eqs. (9)–(10)] The proof does not establish the stated conclusion. Eqs. (9)–(10) only show g_FG^i · g_i ≥ 0. To conclude that g_FG = Σ_i g_FG^i is a descent direction for L_total, one needs Σ_i g_FG^i · g_i + Σ_{i≠j} g_FG^i · g_j ≥ 0. The cross terms are not examined and can be negative. As a logical illustration, take g1=(1,0), g2=(-2,0), g_FG^1=g1, g_FG^2=0; then both per-task inner products are nonnegative but g_FG·(g1+g2) = -1. Therefore the surrogate-descent claim is not established. Provide a proof that controls the cross terms using the actual projection rule, or remove the theoretical claim and present §3.6 as intuition.
minor comments (5)
- [§3.4 / Algorithm 1] When cos(θ_ij)=0, Eq. (7) gives g_coop_i = 0.5 g_i because the projection term vanishes. Orthogonal gradients are therefore halved, not 'reinforced'. This magnitude shrinkage is not discussed and may be part of why the method works; please analyze or at least acknowledge it.
- [§4.1 / Fig. 2] The experiments are described as using SGD for 100 epochs, but the Figure 2 caption refers to 'the standard Adam optimizer' and 'Adam with PCGrad / FlexiGrad'. Please clarify which optimizer is used in the actual experiments and in the visualization.
- [Algorithm 1] The epsilon in the denominator of the projection is never specified or ablated. Since the paper emphasizes that FlexiGrad is parameter-free, please state the value of epsilon and explain why it does not affect the qualitative behavior.
- [§4.1] For reproducibility, please report random seeds, exact augmentation details, dataset split conventions, and learning-rate schedules. The statement 'standard augmentations' is too vague for a method whose reported gains are 0.5–1.0%.
- [Fig. 5] The accuracy-vs-epoch figure lacks axis labels and a legend entry for 'Vanilla single'. Please improve the figure captions so the visualization can be interpreted independently.
Circularity Check
No significant circularity: FlexiGrad's improvements are externally tested, not derived from fitted parameters or self-citations.
full rationale
The paper's central empirical claim is that the FlexiGrad gradient-modulation rule improves multi-granularity accuracy on CUB-200-2011, FGVC-Aircraft, and Stanford Cars. This claim is evaluated against external benchmarks and baselines (PCGrad, FGoN, etc.), and the method is parameter-free: 'FlexiGrad introduces no parameters, requires no modifications to the architecture and integrates seamlessly into any optimiser by replacing the raw gradient with g_FG.' The modulation formulas (Eqs. 5–7) are defined directly on raw task gradients and involve no constants fitted to the reported accuracies. The weighting function w_ij=(1+cos θ_ij)/2 is chosen, ablated against alternatives (Table 3), and not derived from the target results, so it is not a fitted input renamed as a prediction. Self-citations appear in related work and baselines (e.g., FGoN [3] is the authors' prior architecture-level gradient-blocking method), but they are not used to justify FlexiGrad's central claim; the claim is supported by the reported experiments. The theoretical interpretation in §3.6 is logically questionable—the step from 'each corrected gradient remains non-ascending for its own task' to 'their sum g_FG constitutes a principled surrogate descent for L_total' does not follow, as cross-terms with other tasks' gradients are unexamined. However, an invalid proof is not circularity: the empirical results do not depend on that proof, and the proof does not reduce the method to its inputs. Similarly, the inconsistency between Algorithm 1 (which updates only against tasks with k>i) and §3.5 (which says 'all other tasks j≠i') is a reproducibility/correctness problem, not a circularity problem. No step in the derivation chain exhibits self-definitional reasoning, fitted-input-called-prediction, load-bearing self-citation, imported uniqueness, ansatz-smuggling via citation, or renaming of a known result as a new derivation. The honest finding is no significant circularity.
Assumptions & free parameters
free parameters (2)
- epsilon in Algorithm 1 =
not reported
- training hyperparameters (learning rate, epochs, schedule) =
not reported in detail
assumptions (3)
- domain assumption Pairwise cosine similarity between gradients at two levels is a sufficient signal to distinguish beneficial from harmful gradient interactions in a hierarchy.
- domain assumption The shared backbone is trained jointly with K task-specific heads whose losses are summed unweighted (§3.1).
- domain assumption Pre-trained ImageNet features are the right starting point and fine-tuning for 100 epochs with SGD is a representative protocol.
invented entities (1)
-
Hierarchical gradient conflict (as an entity to be measured by cosine similarity)
Cite this review
Pith. "Pith review of FlexiGrad: Adaptive Gradient Modulation for Hierarchical Fine-Grained Classification." pith.science (2026). https://pith.science/paper/C6K6H5XX
@misc{pith2026260717563,
author = {Pith},
title = {Pith review of: FlexiGrad: Adaptive Gradient Modulation for Hierarchical Fine-Grained Classification},
year = {2026},
howpublished = {\url{https://pith.science/paper/C6K6H5XX}},
note = {Machine review of arXiv:2607.17563}
}
read the original abstract
Many fine-grained recognition tasks contain hierarchical labels such as order, family and species. Although this supervision should be beneficial, jointly optimising all levels often leads to unstable training because coarse and fine classifiers impose inconsistent gradients on the shared backbone. This hierarchical gradient conflict prevents the model from learning a coherent coarse-to-fine representation. In this paper, we propose FlexiGrad, a simple and parameter-free method that regulates gradient interactions during backpropagation. FlexiGrad removes only the harmful conflicting component when tasks disagree and reinforces the shared direction when they partially agree through a smooth hierarchy-aware weighting function. This produces stable optimisation and preserves both global structure and fine-grained discriminative cues. FlexiGrad integrates into existing architectures without modification while improves multi-granularity accuracy on CUB-200-2011, FGVC-Aircraft and Stanford Cars. The code will be available at PRIS-CV/FlexiGrad.
Figures
Figures from the paper (4 more)
Reference graph
Works this paper leans on
-
[1]
International Journal of Data Science and Analytics21(1), 77 (2026) 4
Abdelsamie, M.M., Azab, S.S., Hefny, H.A.: Deep multi-task learning: a review of concepts, methods, and cross-domain applications. International Journal of Data Science and Analytics21(1), 77 (2026) 4
2026
-
[2]
IEEE Transactions on Image Processing29, 4683–4695 (2020) 2
Chang, D., Ding, Y., Xie, J., Bhunia, A.K., Li, X., Ma, Z., Wu, M., Guo, J., Song, Y.Z.: The devil is in the channels: Mutual-channel loss for fine-grained image classification. IEEE Transactions on Image Processing29, 4683–4695 (2020) 2
2020
-
[3]
In: CVPR (2021) 2, 4, 8, 9
Chang, D., Pang, K., Zheng, Y., Ma, Z., Song, Y.Z., Guo, J.: Your” flamingo” is my” bird”: Fine-grained, or not. In: CVPR (2021) 2, 4, 8, 9
2021
-
[4]
In: VCIP (2022) 4
Chen, J., Chang, D., Xie, J., Du, R., Ma, Z.: Cross-layer feature based multi- granularity visual classification. In: VCIP (2022) 4
2022
-
[5]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition
Chen, J., Zhou, Z., Tong, Y., Chang, D., Luo, Y., Ma, Z.: Seeing as experts do: A knowledge-augmented agent for open-set fine-grained visual understanding. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recog- nition. pp. 41446–41455 (2026) 4
2026
-
[6]
In: CVPR (2009) 8
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: A large-scale hierarchical image database. In: CVPR (2009) 8
2009
-
[7]
In: ECCV (2020) 8
Du, R., Chang, D., Bhunia, A.K., Xie, J., Ma, Z., Song, Y.Z., Guo, J.: Fine-grained visual classification via progressive multi-granularity training of jigsaw patches. In: ECCV (2020) 8
2020
-
[8]
IEEE Transactions on Pattern Analysis and Machine Intelligence44(12), 9521– 9535 (2021) 1
Du, R., Xie, J., Ma, Z., Chang, D., Song, Y.Z., Guo, J.: Progressive learning of category-consistent multi-granularity features for fine-grained visual classification. IEEE Transactions on Pattern Analysis and Machine Intelligence44(12), 9521– 9535 (2021) 1
2021
Show all 32 references
-
[9]
arXiv preprint arXiv:2504.13608 (2025) 4
Gao, P., Liang, Y., Song, Y., Yang, Z.: Cross-hierarchical bidirectional consistency learning for fine-grained visual classification. arXiv preprint arXiv:2504.13608 (2025) 4
2025 arXiv
-
[10]
IEEE Transactions on Image Processing35, 3395–3410 (2026) 4
Gao, Y., Chang, D., Yu, B., Qin, H., Diao, M., Chen, L., Liang, K., Ma, Z.: Toward generalizable forgery detection and reasoning. IEEE Transactions on Image Processing35, 3395–3410 (2026) 4
2026
-
[11]
In: BMVC
Gao, Y., Lin, W., Xu, J., Xu, W., Chen, P.: Self-supervised adversarial training for robust face forgery detection. In: BMVC. p. 718 (2023) 4
2023
-
[12]
kirchdorfer et al
Kirchdorfer, L., Sesterhenn, T., Bartelt, C., Stuckenschmidt, H., Schott, L., K¨ ohler, J.M.: Investigating uncertainty weighting for multi-task learning: Insights and an- alytical alternative: L. kirchdorfer et al. International Journal of Computer Vision 134(1), 8 (2026) 4
2026
-
[13]
In: ICCV Workshops (2013) 3, 8
Krause, J., Stark, M., Deng, J., Fei-Fei, L.: 3d object representations for fine- grained categorization. In: ICCV Workshops (2013) 3, 8
2013
-
[14]
Engineering Applications of Artificial Intelligence163, 113123 (2026) 2
Kuang, W., Li, Z.: Spatial-aware feature enhancement network for fine-grained visual classification. Engineering Applications of Artificial Intelligence163, 113123 (2026) 2
2026
-
[15]
IEEE Transactions on Vehicular Tech- nology68(5), 4204–4212 (2019) 1
Li, X., Yu, L., Chang, D., Ma, Z., Cao, J.: Dual cross-entropy loss for small- sample fine-grained vehicle classification. IEEE Transactions on Vehicular Tech- nology68(5), 4204–4212 (2019) 1
2019
-
[16]
NeurIPS (2021) 2, 4
Liu, B., Liu, X., Jin, X., Stone, P., Liu, Q.: Conflict-averse gradient descent for multi-task learning. NeurIPS (2021) 2, 4
2021
-
[17]
In: AAAI (2025) 4 FlexiGrad for Hierarchical FGVC 15
Liu, E., Wu, Y.C., Huang, X., Gao, C., Wang, R.J., Xue, K., Qian, C.: Pareto set learning for multi-objective reinforcement learning. In: AAAI (2025) 4 FlexiGrad for Hierarchical FGVC 15
2025
-
[18]
In: AAAI (2025) 4
Liu, Y., Miao, Y., Xia, L.: Direct routing gradient (drgrad): A personalized infor- mation surgery for multi-task learning (mtl) recommendations. In: AAAI (2025) 4
2025
-
[19]
In: ICCV (2025) 4
Liu, Y., Yang, L., Wang, Y.: Long-tailed classification with multi-granularity se- mantics. In: ICCV (2025) 4
2025
-
[20]
arXiv preprint arXiv:1306.5151 (2013) 1, 3, 8
Maji, S., Rahtu, E., Kannala, J., Blaschko, M., Vedaldi, A.: Fine-grained visual classification of aircraft. arXiv preprint arXiv:1306.5151 (2013) 1, 3, 8
2013 arXiv
-
[21]
In: W ACV (2025) 4
Matsubara, Y., Mendula, M., Levorato, M.: A multi-task supervised compression model for split computing. In: W ACV (2025) 4
2025
-
[22]
In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition
Qin, H., Chang, D., Gao, Y., Tan, Y., Chen, L., Ma, Z.: Increfa: Breaking the static wall of generative model attribution. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. pp. 35405–35415 (2026) 4
2026
-
[23]
arXiv preprint arXiv:2505.15217 (2025) 4
Qin, H., Chang, D., Gao, Y., Yu, B., Chen, L., Ma, Z.: Multimodal conditional in- formation bottleneck for generalizable ai-generated image detection. arXiv preprint arXiv:2505.15217 (2025) 4
2025 arXiv
-
[24]
In: ICCV (2017) 10
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad- cam: Visual explanations from deep networks via gradient-based localization. In: ICCV (2017) 10
2017
-
[25]
Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The caltech-ucsd birds-200-2011 dataset (2011) 1, 3, 8
2011
-
[26]
Transactions on Image Processing (2024) 8
Wang, J., Xu, Q., Jiang, B., Luo, B., Tang, J.: Multi-granularity part sampling attention for fine-grained visual classification. Transactions on Image Processing (2024) 8
2024
-
[27]
Advances in Neural Information Processing Systems38(2026) 4
Wang, X., Xu, S., Xiangxuan, S., Zhang, Y., Diao, M., Duan, X., Liang, K., Ma, Z., et al.: Cinetechbench: A benchmark for cinematographic technique understanding and generation. Advances in Neural Information Processing Systems38(2026) 4
2026
-
[28]
Wang, X., Zhang, Y., Zhang, X., Yan, H., Diao, M., Xu, S., Yan, Z., Li, H., Liang, K., Ma, Z.: Detailverifybench: A benchmark for dense hallucination localization in long image captions (2026) 4
2026
-
[29]
In: ECCV (2018) 8
Yang, Z., Luo, T., Wang, D., Hu, Z., Gao, J., Wang, L.: Learning to navigate for fine-grained classification. In: ECCV (2018) 8
2018
-
[30]
NeurIPS (2020) 2, 4, 8
Yu, T., Kumar, S., Gupta, A., Levine, S., Hausman, K., Finn, C.: Gradient surgery for multi-task learning. NeurIPS (2020) 2, 4, 8
2020
-
[31]
Pattern Recognition (2024) 8
Zhang, Z.C., Chen, Z.D., Wang, Y., Luo, X., Xu, X.S.: A vision transformer for fine- grained classification by reducing noise and enhancing discriminative information. Pattern Recognition (2024) 8
2024
-
[32]
In: ICCV (2025) 4
Zheng, H., Yang, S., He, Z., Yang, J., Huang, Z.: Hierarchical cross-modal prompt learning for vision-language models. In: ICCV (2025) 4
2025
Reviewed August 1, 2026 · model on record in the stance chip above.
Discussion (0). Sign in to comment.