REVIEW 3 major objections 5 minor 1 cited by
Hierarchical Text Classification with Reinforced Label Assignment
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read Hierarchical text classification is treated as a Markov decision process in which a learned policy places each object in the label hierarchy and decides when to stop, producing consistent multi-path label sets.
desk verdict Genuinely new RL-for-HTC idea with a learned stop action, but the headline 33.4% Macro-F1 gain over flat classifiers is inflated by untuned flat baselines. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is the label-assignment Markov decision process and its policy network. At time $t$, the state is formed by concatenating the object embedding $e_d$ with the embedding $l_t$ of the most recently assigned label and projecting the result through two ReLU layers to $s_t$; the action space holds the embeddings of all unvisited children of every label already selected, plus a trainable stop embedding $e_{\mathrm{stop}}$, and the policy is a softmax over the matrix product $A_t s_t$. The reward at each step is the difference between the current and previous example-based F1 scores, so the cumulative reward measures whether the whole assignment improved. A supervised pre-training phase walks only ground-truth paths using local per-parent sigmoid losses and a flat loss over all labels, and the final objective mixes the reinforcement loss with that supervised loss.
What would settle it
Run HiLAP with a state that also aggregates all assigned labels (for example, the mean or attention-weighted sum of their embeddings) on a business-review or gene-function dataset; a clear improvement in Macro-F1 would show the original Markov state is insufficient, while a tie would support it. Separately, retrain the flat baselines with per-label threshold tuning and check whether the 33.4% average gain persists.
Extended reading notes
Core claim
The central claim is that label dependencies in hierarchical text classification should be modeled by one policy that interacts with the hierarchy directly. Starting at the root, the policy repeatedly selects either an unvisited child of any label already assigned to the object or a learned stop action, and the labels visited when it stops are the prediction. The reward at each step is the change in example-based F1, and the policy is trained with a policy-gradient objective plus a greedy self-critical baseline, so the agent learns to look ahead rather than optimize each placement in isolation. This formulation guarantees label consistency by construction, supports multi-path and non-leaf predictions, and removes the mismatch between hierarchical training and flat inference. The paper further claims that the framework is encoder-agnostic: swapping the base encoder only changes the object embedding, and the same reinforcement setup improves all four tested encoders.
Load-bearing premise
The policy's state contains only the embedding of the most recently assigned label, so the learned decisions implicitly assume that labels assigned earlier do not matter for where to go next or when to stop.
Editorial extensions
If this is right
- Predictions are consistent by construction: because the policy only adds children of labels already chosen, contradictory label sets cannot occur and no post-processing is needed; the paper reports 0% inconsistent predictions on RCV1.
- Sparse, deep labels benefit most: reported gains concentrate on lower-level labels and labels with few training examples, which flat classifiers tend to ignore.
- The framework inherits improvements from any differentiable encoder: better base models should transfer directly to the policy without redesign.
- Training and inference use the same traversal, reducing the exposure bias of local top-down classifiers, and the learned stop action removes the need for thresholding or mandatory leaf-node prediction.
Reading between the lines
- Not tested in the paper: comparing HiLAP against flat classifiers that tune a threshold per label would likely narrow the 33.4% headline gain, since the paper notes such tuned flat baselines are stronger.
- If the Markov state is the bottleneck, augmenting it to pool embeddings of all assigned labels rather than only the most recent one should help on datasets with many labels per object, such as gene-function prediction; this is a direct experiment the paper does not run.
- The same MDP formulation should transfer beyond text to any domain with a label taxonomy and a differentiable object encoder, such as product categorization or image classification, because the policy consumes only the object embedding and label embeddings.
- The stop action effectively learns a data-dependent threshold on when further placement is not worth the F1 reward; reading its probability along a traversal could provide a confidence score and a controllable precision-recall trade-off, which the paper does not explore.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes HiLAP, a reinforcement-learning framework for hierarchical text classification. Label assignment is cast as an MDP: starting from the root, the agent repeatedly selects either a child of one of the already assigned labels or a special stop action; the reward is the change in example-based F1. The state is a projection of the object embedding and the last assigned label embedding, and the policy is trained with self-critical policy gradient after a supervised pre-training stage that combines flat and local losses. The framework is evaluated on RCV1, NYT, Yelp, FunCat, and GO with TextCNN, HAN, bow-CNN, and a feed-forward base model. The paper reports consistent Macro-F1 gains, including an average 33.4% improvement over flat classifiers, and ablations showing benefits for sparse bottom-level labels and 0% label inconsistency.
Significance. If the results hold, HiLAP is a useful and general recipe for HTC: it is encoder-agnostic, guarantees hierarchy-consistent predictions without post-processing, supports multi-path and non-mandatory-leaf prediction, and the released code and data support reproducibility. The reward-shaping and self-critical training details are standard, and the main value is in demonstrating that a learned stop action and hierarchical exploration can improve sparse-label Macro-F1 across text and non-text domains. However, the headline comparison to flat classifiers is weakened by threshold calibration, and the MDP state definition needs clarification before the central claims can be accepted as stated.
major comments (3)
- [Section 4.4, Appendix B] The abstract's headline claim of a 33.4% average Macro-F1 improvement over flat classifiers is not supported as stated because the flat baselines use an uncalibrated decision threshold whereas HiLAP replaces thresholding with a learned stop action. Macro-F1 in multi-label classification is highly sensitive to the decision threshold, and the paper itself notes in Appendix B that per-label scutFBR threshold tuning (as used by Johnson and Zhang 2014) is infeasible in its setting. The comparison to the same base models therefore conflates hierarchy modeling with an adaptive-threshold effect. Please either calibrate the flat baselines (for example, by tuning a global threshold or per-label thresholds on the validation set) or explicitly qualify the 33.4% claim as including the thresholding effect.
- [Section 2.2, "States and Policy Network"] The MDP formulation is incomplete because the state st = ReLU(W1_l ReLU(W2_l [ed; lt])) contains only the last assigned label, while the action space At contains the unvisited children of every label assigned so far and the example-F1 reward depends on the entire set of assigned labels. For example, after a leaf label is assigned it no longer appears in At but must still be counted in the reward. No Markovianity or sufficiency argument is provided, so the process as defined is a POMDP rather than an MDP. Please include an aggregate of the assigned-label history in the state or provide an empirical ablation showing that the last-label-only state is sufficient.
- [Tables 2-4 and Figure 4] The empirical comparison reports single-run scores with no error bars, multiple seeds, or statistical significance tests, although the text describes improvements as "significant" and the datasets are very imbalanced. Given that the headline gain is large and Macro-F1 values for rare labels are unstable, please report at least the mean and standard deviation over several random seeds and, where feasible, a paired significance test over examples or folds.
minor comments (5)
- [Table 1 and Appendix A] The Yelp test-set size is reported as 37,265 in Table 1 but as 37,517 in Appendix A; please reconcile the discrepancy.
- [Appendix D] Please report the exact values of the mixing ratio lambda, the scaling factor alpha, the discount factor gamma, and the maximum episode steps; Appendix D gives network sizes and batch size but not these hyperparameters, which are needed for reproducibility.
- [Section 2.2, "Actions"] For DAG hierarchies such as Yelp and GO, the paper does not specify how duplicate children reachable through multiple parents are handled when constructing the action space; please clarify this in the action-space definition.
- [Figure 5] The popularity bins (P1-P3) and level definitions are not described in the text; the caption only says that labels are divided into three equal-sized categories, so please add the exact construction used for the popularity-based and level-based gains.
- [Author affiliations] The affiliation numbering in the header lists affiliation 3 twice rather than providing unique affiliation numbers for the two universities; this typo should be fixed.
Circularity Check
No significant circularity: HiLAP's RL formulation and reported gains are self-contained, and the self-cited reward-shaping component is not load-bearing.
full rationale
We walked the paper's derivation chain. HiLAP defines an MDP with states as concatenations of object and current-label embeddings, actions as unvisited children plus a stop action, and rewards as the difference in example-based F1. This is a genuine learning objective, not a fitted parameter disguised as a prediction. The headline metric, Macro-F1, is not the training reward, and the reported numbers are measured on held-out test sets against external baselines, so the 33.4% improvement claim is an empirical result rather than a tautology. The self-citation to Mao et al. (2018) is used only to motivate intermediate reward shaping, a standard technique; the central claim does not depend on any uniqueness theorem or unverified self-cited result. The paper's guarantee of 0% label inconsistency follows directly from the construction of the action space (only children of already assigned labels are available), so it is a structural property, not a circularly derived empirical finding. The potential concern that flat baselines are not threshold-tuned (Appendix B) bears on fair comparison and correctness, not on circularity. No equation or claim in the paper reduces to its own inputs by construction. Therefore no significant circularity is present.
Assumptions & free parameters
free parameters (6)
- Loss mixing ratio lambda =
not reported
- Scaling factor alpha =
not reported
- Discount factor gamma =
not reported
- Maximum episode steps =
not reported
- Batch size =
32
- Policy network hidden sizes =
W2_l size 500 on text, 1000 on genomics; W1_l and label embeddings 50 on text, 1000 on genomics
assumptions (5)
- standard math Gradient estimates from the policy gradient theorem (Williams 1992) and the self-critical baseline (Rennie et al. 2017) are unbiased for the stated objective Og.
- domain assumption Each object's ground-truth labels form a consistent sub-tree of the hierarchy, and labels may be non-leaf nodes.
- ad hoc to paper The state st, built only from the object embedding and the last assigned label embedding, contains enough information for optimal action selection.
- ad hoc to paper Greedy inference using the highest-probability action at each step gives a good approximation of the learned policy.
- domain assumption Example-based F1 is an appropriate reward and its per-step differences provide useful shaped feedback.
Cite this review
Pith. "Pith review of Hierarchical Text Classification with Reinforced Label Assignment." pith.science (2026). https://pith.science/paper/BBPX25MQ
@misc{pith2026190810419,
author = {Pith},
title = {Pith review of: Hierarchical Text Classification with Reinforced Label Assignment},
year = {2026},
howpublished = {\url{https://pith.science/paper/BBPX25MQ}},
note = {Machine review of arXiv:1908.10419}
}
read the original abstract
While existing hierarchical text classification (HTC) methods attempt to capture label hierarchies for model training, they either make local decisions regarding each label or completely ignore the hierarchy information during inference. To solve the mismatch between training and inference as well as modeling label dependencies in a more principled way, we formulate HTC as a Markov decision process and propose to learn a Label Assignment Policy via deep reinforcement learning to determine where to place an object and when to stop the assignment process. The proposed method, HiLAP, explores the hierarchy during both training and inference time in a consistent manner and makes inter-dependent decisions. As a general framework, HiLAP can incorporate different neural encoders as base models for end-to-end training. Experiments on five public datasets and four base models show that HiLAP yields an average improvement of 33.4% in Macro-F1 over flat classifiers and outperforms state-of-the-art HTC methods by a large margin. Data and code can be found at https://github.com/morningmoni/HiLAP.
Figures
Figures from the paper (3 more)
Forward citations
Cited by 1 Pith paper
-
Enhancing Text-Based Hierarchical Multilabel Classification for Mobile Applications via Contrastive Learning
HMCL, a level-wise negative-sampling contrastive pretraining scheme, improves HMCN's hierarchical multilabel classification for mobile apps and achieved a reported 10.70% KS improvement in a downstream credit-risk task.
Reference graph
Works this paper leans on
-
[1]
URL: " 'urlintro :=
ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block STRINGS urlintro eprinturl eprintpr...
-
[2]
write newline
" write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...
-
[3]
Rahul Agrawal, Archit Gupta, Yashoteja Prabhu, and Manik Varma. 2013. Multi-label learning with millions of labels: Recommending advertiser bid phrases for web pages. In WWW, pages 13--24. ACM
work page 2013
-
[4]
Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473
arXiv 2014
-
[5]
Wei Bi and Jame T Kwok. 2015. Bayes-optimal hierarchical multilabel classification. TKDE, 27(11):2907--2918
work page 2015
-
[6]
Wei Bi and James T Kwok. 2011. Multi-label classification on tree-and dag-structured hierarchies. In ICML-11, pages 17--24
work page 2011
-
[7]
Lijuan Cai and Thomas Hofmann. 2004. Hierarchical document categorization with support vector machines. In CIKM, pages 78--87. ACM
work page 2004
-
[8]
Asli Celikyilmaz, Antoine Bosselut, Xiaodong He, and Yejin Choi. 2018. Deep communicating agents for abstractive summarization. In NAACL, pages 1662--1675
work page 2018
Show all 41 references
-
[9]
Ricardo Cerri, Rodrigo C Barros, Andr\' e CPLF de Carvalho, and Yaochu Jin. 2016. Reduction strategies for hierarchical multi-label classification in protein function prediction. BMC bioinformatics, 17(1):373
2016
-
[10]
Nicol \`o Cesa-Bianchi, Claudio Gentile, and Luca Zaniboni. 2006. Hierarchical classification: combining bayes with svm. In ICML, pages 177--184. ACM
2006
-
[11]
Amanda Clare and Ross D King. 2003. Predicting gene function in saccharomyces cerevisiae. Bioinformatics, 19(suppl\_2):ii42--ii49
2003
-
[12]
Siddharth Gopal and Yiming Yang. 2013. Recursive regularization for large-scale classification with hierarchical and graphical dependencies. In KDD, pages 257--265. ACM
2013
-
[13]
Boris Hayete and Jadwiga R Bienkowska. 2005. Gotrees: predicting go associations from protein domain composition using decision trees. In Biocomputing 2005, pages 127--138. World Scientific
2005
-
[14]
Rie Johnson and Tong Zhang. 2014. Effective use of word order for text categorization with convolutional neural networks. arXiv preprint arXiv:1412.1058
2014 arXiv
-
[15]
Yoon Kim. 2014. Convolutional neural networks for sentence classification. arXiv preprint arXiv:1408.5882
2014 arXiv
-
[16]
Daphne Koller and Mehran Sahami. 1997. Hierarchically classifying documents using very few words. In ICML, pages 170--178. Morgan Kaufmann Publishers Inc
1997
-
[17]
Siwei Lai, Liheng Xu, Kang Liu, and Jun Zhao. 2015. Recurrent convolutional neural networks for text classification. In AAAI, volume 333, pages 2267--2273
2015
-
[18]
David D Lewis, Yiming Yang, Tony G Rose, and Fan Li. 2004. Rcv1: A new benchmark collection for text categorization research. Journal of machine learning research, 5(Apr):361--397
2004
-
[19]
Jingzhou Liu, Wei-Cheng Chang, Yuexin Wu, and Yiming Yang. 2017. Deep learning for extreme multi-label text classification. In Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 115--124. ACM
2017
-
[20]
Tie-Yan Liu, Yiming Yang, Hao Wan, Hua-Jun Zeng, Zheng Chen, and Wei-Ying Ma. 2005. Support vector machines classification with a very large-scale taxonomy. Acm Sigkdd Explorations Newsletter, 7(1):36--43
2005
-
[21]
Yuning Mao, Xiang Ren, Jiaming Shen, Xiaotao Gu, and Jiawei Han. 2018. End-to-end reinforcement learning for automatic taxonomy induction. In ACL, pages 2462--2472. Association for Computational Linguistics
2018
-
[22]
Yu Meng, Jiaming Shen, Chao Zhang, and Jiawei Han. 2018. Weakly-supervised neural text classification. In CIKM
2018
-
[23]
Ioannis Partalas, Aris Kosmopoulos, Nicolas Baskiotis, Thierry Arti \` e res, George Paliouras, \' E ric Gaussier, Ion Androutsopoulos, Massih - Reza Amini, and Patrick Gallinari. 2015. LSHTC: A benchmark for large-scale text classification. CoRR, abs/1503.08581
2015 arXiv
-
[24]
Hao Peng, Jianxin Li, Yu He, Yaopeng Liu, Mengjiao Bao, Lihong Wang, Yangqiu Song, and Qiang Yang. 2018. Large-scale hierarchical text classification with recursively regularized deep graph-cnn. In WWW, pages 1063--1072
2018
-
[25]
Shengwen Peng, Ronghui You, Hongning Wang, Chengxiang Zhai, Hiroshi Mamitsuka, and Shanfeng Zhu. 2016. Deepmesh: deep semantic representation for improving large-scale mesh indexing. Bioinformatics, 32(12):i70--i79
2016
-
[26]
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In EMNLP, pages 1532--1543
2014
-
[27]
Xipeng Qiu, Wenjun Gao, and Xuanjing Huang. 2009. Hierarchical multi-class text categorization with global margin maximization. In acl-ijcnlp 2009, pages 165--168
2009
-
[28]
Bo Qu, Gao Cong, Cuiping Li, Aixin Sun, and Hong Chen. 2012. An evaluation of classification models for question topic categorization. JASIST, 63:889--903
2012
-
[29]
Steven J Rennie, Etienne Marcheret, Youssef Mroueh, Jarret Ross, and Vaibhava Goel. 2017. Self-critical sequence training for image captioning. In CVPR, page 3
2017
-
[30]
Juho Rousu, Craig Saunders, Sandor Szedmak, and John Shawe-Taylor. 2005. Learning hierarchical multi-category text classification models. In ICML, pages 744--751. ACM
2005
-
[31]
Evan Sandhaus. 2008. The new york times annotated corpus. Linguistic Data Consortium, Philadelphia, 6(12):e26752
2008
-
[32]
Carlos N Silla and Alex A Freitas. 2011. A survey of hierarchical classification across different application domains. Data Mining and Knowledge Discovery, 22(1-2):31--72
2011
-
[33]
Carlos N Silla Jr and Alex A Freitas. 2009. A global-model naive bayes approach to the hierarchical prediction of protein functions. In ICDM'09, pages 992--997. IEEE
2009
-
[34]
Aixin Sun and Ee-Peng Lim. 2001. Hierarchical text classification and evaluation. In ICDM
2001
-
[35]
Ioannis Tsochantaridis, Thorsten Joachims, Thomas Hofmann, and Yasemin Altun. 2005. Large margin methods for structured and interdependent output variables. Journal of machine learning research, 6(Sep):1453--1484
2005
-
[36]
Celine Vens, Jan Struyf, Leander Schietgat, Sa s o D z eroski, and Hendrik Blockeel. 2008. Decision trees for hierarchical multi-label classification. Machine Learning, 73(2):185
2008
-
[37]
Ke Wang, Senqiang Zhou, and Yu He. 2001. Hierarchical classification of real life documents. In SDM, pages 1--16. SIAM
2001
-
[38]
Jonatas Wehrmann, Ricardo Cerri, and Rodrigo Barros. 2018. Hierarchical multi-label classification networks. In ICML, pages 5225--5234
2018
-
[39]
Ronald J Williams. 1992. Simple statistical gradient-following algorithms for connectionist reinforcement learning. Machine learning, 8(3-4):229--256
1992
-
[40]
IPC WIPO. 2014. International patent classification (ipc). World Intellectual Property Organization, Geneve
2014
-
[41]
Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alex Smola, and Eduard Hovy. 2016. Hierarchical attention networks for document classification. In NAACL, pages 1480--1489
2016
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.