Trie-Constrained Token Prediction with Hierarchy-Aware Semantic Alignment for HS Code Prediction

Hyerim Bae; Kikun Park; Minseop Kim; Taekhyun Park

REVIEW 3 major objections 7 minor 5 references

Trie-Constrained Token Prediction with Hierarchy-Aware Semantic Alignment for HS Code Prediction

T0 review · 3 major / 7 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read TRIE-HSA makes sub-10-billion-parameter language models accurate, structurally valid HS code predictors.

desk verdict Useful engineering with a serious measurement flaw: the headline accuracy numbers are agreement with Gemini-generated labels, not verified customs ground truth. read the letter →

arxiv 2608.04464 v1 pith:7Y35F2WD submitted 2026-08-05 cs.CE

Minseop Kim , Taekhyun Park , Kikun Park , Hyerim Bae This is my paper · ORCID

classification cs.CE

keywords HScodepredictionHarmonizedSystemsmalllanguagemodelstrie-constrainedtokenhierarchy-awaresemanticalignmentprototype-contrastivelearningportlogisticscommoditytextclassification

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper tries to establish that accurate, structurally valid Harmonized System (HS) code prediction can be done by small language models under 10 billion parameters, not only by large ones. The proposed method, TRIE-HSA, constrains each digit the model outputs to the set of valid continuations under the HS taxonomy, and trains the model to pull commodity-text representations toward learned chapter and heading prototypes that preserve the HS hierarchy. On 23,355 commodity texts from an operating container terminal, the framework lifts average six-digit (HS6) accuracy from 14.02% under zero-shot inference to 63.98%, and its best configuration beats the strongest task-specific benchmark by 11.94 percentage points. If the result holds, port logistics facilities that cannot run large language models can still deploy accurate HS classification.

What carries the argument

The two load-bearing mechanisms are a digit-level trie built from the HS taxonomy and learnable prototypes at chapter and heading levels. The trie maps any generated prefix to the set of valid next digits, and the resulting mask is added to the model's logits so invalid tokens receive negative infinity; teacher forcing stabilizes fine-tuning. The prototypes are initialized by mean-pooling backbone representations of training texts, with each heading prototype parameterized as its parent chapter prototype plus a learned offset, so chapter updates propagate while headings stay locally adjustable. Two triplet losses separate same-level prototypes while a containment loss classifies each heading into its parent chapter, jointly producing within-level discrimination and cross-level hierarchy.

What would settle it

Have a customs expert or official customs records independently determine true HS codes for a random sample of the 20% test split, then recompute TRIE-HSA's HS6 accuracy against those verified labels. If accuracy drops toward or below the 59.67% SSCNN baseline, the central claim fails and the reported gains were an artifact of label-model agreement.

Watch

Extended reading notes

Core claim

The central claim is that two complementary mechanisms—trie-constrained token prediction and hierarchy-aware semantic alignment—are sufficient to make small language models competitive at HS code assignment. The trie restricts every generation step to digits that continue a valid code in the HS taxonomy, eliminating invalid-code hallucinations; the alignment loss organizes the model's representation space so chapter prototypes separate from each other, heading prototypes sit close to their parent chapters, and commodity text clusters near the right prototypes. Fine-tuned with low-rank adapters, this combination exceeds both flat softmax classification and zero-shot prompting on every tested model, with the best configuration reaching 90.52% HS2, 83.41% HS4, and 71.61% HS6 accuracy. The paper reads these results as evidence that hierarchy plus validity constraints, rather than raw scale, is what small language models need for HS code prediction.

Load-bearing premise

The load-bearing premise is that the Gemini 2.5 Flash-generated HS codes used as ground truth are correct; if those labels are systematically wrong, the reported accuracy gains measure agreement with Gemini rather than true classification skill.

Editorial extensions

If this is right

Every predicted HS code is structurally valid by construction, so post-hoc validation or repair modules become unnecessary for backbones that tokenize digits individually.
Hierarchy alignment matters most at fine granularity: HSA improved HS6 accuracy by 1.33 percentage points over plain trie-constrained fine-tuning, roughly 2.3 times its HS2 gain.
Because fewer than 0.1% of backbone parameters are updated and the best model has fewer than 10 billion parameters, HS code prediction can run on a single-GPU workstation rather than requiring large-language-model infrastructure.
High-confidence predictions are reliable, with samples at or above 99% digit confidence keeping roughly 99% HS6 accuracy at the first digit and 90% at the sixth, which could support selective human review or automated downstream decisions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

If the generated ground-truth labels are themselves biased, the reported accuracy measures agreement with that labeling model, so verifying a sample against customs records or expert annotation is the natural pre-deployment check.
The framework's dependence on single-digit tokenization suggests a testable extension: retokenizing or fine-tuning multi-digit-token backbones to emit single digits before applying the trie could close the architecture-specific gap observed under zero-shot constrained decoding.
The same recipe—trie-constrained output plus prototype-based hierarchy alignment—could transfer to other multi-level taxonomies such as ten-digit national HS extensions, SITC codes, or ICD coding, where output validity and parent-child semantics both matter.
The evaluation covers one terminal and only 35.9% of HS6 subheadings, so confirming generalization would require multilingual commodity text and data from additional ports.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

The paper is a solid engineering contribution with a serious measurement problem. The method—trie-constrained autoregressive decoding plus hierarchy-aware prototype alignment for HS code prediction—is sensible, and the experiments are broad. The authors test eleven SLMs, include ablations, and show that fine-tuning with the trie mask turns structurally invalid output into valid codes. The observation that tokenization (single-digit vs multi-digit) explains why zero-shot constrained decoding helps some architectures and hurts LLaMA/Phi is genuinely useful. The HSA gain over plain trie is small (about 1.3 pp HS6 accuracy) but consistent and larger at deeper levels, which is a plausible signature of hierarchical alignment.

The soft spot is not a detail. Section 4.1 says the terminal does not record HS codes, so the authors used Gemini 2.5 Flash to generate labels for every commodity text and treated those as ground truth for training and evaluation. There is no check against customs records, no manually verified sample, nothing. That means every accuracy number in Tables 2 and 3—including the 49.96 percentage point improvement and the 11.94 point edge over SSCNN—is actually agreement with Gemini, not correctness relative to the true tariff classification. If Gemini's labels are biased—say, toward frequent chapters or plausible-sounding subheadings—the absolute claims are inflated and the benchmark comparisons partly reward models that share Gemini's inductive biases. The paper is transparent about this, which I respect, but transparency does not fix the inference.

Secondary issues: a single data split, no error bars, no release of code or data. Those are fixable. The label problem is not.

I would still not desk-reject. The framework is real, the ablation design is careful, and the tokenization result is worth knowing. A serious referee should ask for verification on a sample of real customs data, or a reframed claim as 'agreement with an LLM labeler,' plus code and data release. With that, the paper could be a useful contribution to the applied HSP literature.

Recommendation: send to peer review, but expect major revision.

Referee Report

3 major / 7 minor

Summary. The paper introduces TRIE-HSA, an SLM-based framework for HS code prediction that combines trie-constrained autoregressive digit decoding with hierarchy-aware semantic alignment (HSA) through learnable chapter and heading prototypes, triplet losses, and a containment loss. The method is evaluated on 23,355 commodity-text records from a container terminal at the Port of Busan, using Gemini 2.5 Flash to generate the ground-truth HS codes. Experiments compare 11 small language models under zero-shot, softmax fine-tuning, trie-constrained fine-tuning, and the complete TRIE-HSA configuration, plus seven existing HSP benchmarks. The headline results are an average HS6 accuracy of 63.98% (49.96 percentage points over zero-shot) and 71.61% HS6 accuracy with Gemma-2-9B, which exceeds the adapted SSCNN benchmark by 11.94 percentage points. The paper concludes that accurate and structurally valid HSP is achievable with models under 10 billion parameters.

Significance. The core technical idea is sound and well motivated: HS codes form a fixed six-digit hierarchy, so a trie mask is a natural way to guarantee structural validity, and prototype-based hierarchy alignment directly targets the chapter, heading, and subheading structure. The paper is unusually thorough in its architecture coverage and reports detailed hyperparameters and prompts, which supports reproducibility. I found no equation-level circularity: the method is not tuned on the test set and the reported numbers are not fitted to it. However, the significance of the result hinges on the validity of the evaluation labels. All ground-truth codes are produced by Gemini 2.5 Flash and explicitly regarded as correct without customs or expert verification, so the reported accuracies and F1 scores are, strictly speaking, agreement rates with another LLM rather than verified classification accuracy. If the label issue is resolved through external validation or by reframing the claims, this would be a practically valuable contribution to applied trade-logistics NLP.

major comments (3)

[Section 4.1] The ground-truth HS codes are generated by Gemini 2.5 Flash and 'regarded as correct' with no external verification against customs records or expert annotation. This assumption is load-bearing: the headline improvements (49.96 percentage points over zero-shot, 11.94 percentage points over SSCNN) and all F1 numbers measure agreement with Gemini's labels, not true classification accuracy. If the generator is systematically biased, for example toward frequent chapters or plausible-but-wrong subheadings, the reported gains overstate real correctness and the margin over other methods may partly reflect shared inductive biases with the labeler. The authors should verify labels on a representative sample using customs declarations or domain-expert annotation and report accuracy on that verified subset, or explicitly reframe all claims as agreement with the Gemini labeler and temper the practical conclusions.
[Section 4.3] SSCNN is evaluated after omitting its auxiliary network because the dataset contains only a single commodity-text field. The paper nevertheless calls SSCNN 'the strongest task-specific benchmark' and claims an 11.94 percentage point margin over it. Since the auxiliary network is part of the published SSCNN method, the comparison is against a modified variant; if that component contributes substantially to SSCNN's performance, the margin is overstated. Please implement the full SSCNN on a compatible dataset, or clearly qualify the benchmark as a single-field adaptation and adjust the conclusion accordingly.
[Section 4.2] All results come from a single 60/20/20 split with one run per configuration, so no error bars, confidence intervals, or significance tests are reported. This matters particularly for the HSA contribution: averaged over models, TRIE-HSA improves HS6 accuracy by only 1.33 percentage points over trie-only fine-tuning (63.98 vs 62.65), and at HS2 the gain is 0.57 percentage points. Without variance estimates, the claim that HSA reliably improves over the trie constraint alone is not statistically supported. Please report multiple seeds or bootstrapped confidence intervals, and where feasible paired significance tests.

minor comments (7)

[Tables 2-4] The table numbering is inconsistent: two tables are labeled Table 2 (taxonomy coverage and inference-mode results), the benchmark comparison is captioned Table 3, but the text refers to it as Table 4. All tables should be renumbered.
[Section 4.1] The F1 score is never defined. Please specify whether macro, micro, or weighted averaging is used and how it is computed for the HS6 label space.
[Eq. (19)] The denominator in the containment loss is ambiguous: for a fixed heading j, the sum should be over chapter prototypes i in C only, but the current notation appears to sum over both i and j. Please rewrite the denominator explicitly.
[Sections 3.3.1 and 4.4.2] Initial chapter and heading prototypes are constructed from all non-test data, which includes the validation set, while Section 4.4.2 presents validation samples as evidence of generalization. This overlap should be acknowledged or the prototypes should be built from training data only.
[Section 4.4.1] The confidence-binning procedure is not specified. Please give the bin edges and the number of test samples in each bin so the monotonicity claim can be checked.
[Appendices 2 and 3] Appendix 2 searches dropout over {0.1, 0.2, 0.3}, while Appendix 3 lists LoRA Dropout as {0.05}. Please clarify which dropout value is used for TRIE-HSA and whether the same grid applies to all backbones.
[Section 4.2] The text refers to 'DSR1-Qwen-7B' while the results table lists 'DeepSeek-DT-7B'; please unify the model naming. Also, 'gained approximately 10 % over (a-1)' should read '10 percentage points'.

Circularity Check

0 steps flagged · score 0.0 of 10

No equation-level circularity; Gemini-generated ground-truth labels are an external-validity caveat, not a circular reduction.

full rationale

Section 4.1 states: "The HS code of a container is unavailable at the terminal. Therefore, following Kim et al. (2026) and Marra De Artiñano et al. (2023), we generated an HS code for each commodity text with Gemini 2.5 Flash, an LLM of a class reported to achieve HSP accuracy of up to 90%, and regarded the generated codes as the ground truth for training and evaluation." This is a genuine limitation: reported "accuracy" measures agreement with an unverified LLM labeler rather than verified customs codes, so absolute accuracy could be overstated. However, it is not a circular step under the required standard. The labels are fixed before training, the SLM never sees Gemini's logits, and every baseline (zero-shot SLMs, softmax fine-tuning, and task-specific benchmarks) is scored against the same label set, so the relative claims are self-contained empirical results. The trie constraint (Eqs. 7-10) is defined from the public HS taxonomy and only restricts the output space; the HSA losses (Eqs. 11-21) are a prototype-contrastive regularizer. No equation reduces to its own input, and no fitted test-set parameter is relabeled as a prediction. The citation to the authors' own Kim et al. (2026) for the labeling practice is a minor self-citation, but it is paired with an independent external citation (Marra De Artiñano et al., 2023) and is not the load-bearing justification for any derived number. Therefore, no circular step is established.

Assumptions & free parameters 6 free parameters · 5 assumptions · 0 invented entities

No new physical or theoretical entities are postulated. The learnable prototypes are model parameters, not independent entities with falsifiable handles. The free parameters are the hyperparameters of the method, and the key axiom is the unverified LLM-generated ground truth.

free parameters (6)

lambda (weight for HSA losses) = 0.5
Balances within-level triplet losses and containment loss in Eq. (20). Chosen by hand; no sensitivity analysis reported.
m1 (chapter triplet margin) = 1
Margin in Eq. (16) enforcing chapter-level separation. Chosen by hand; no sensitivity analysis reported.
m2 (heading triplet margin) = 0.5
Margin in Eq. (17) enforcing heading-level separation. Chosen by hand; no sensitivity analysis reported.
LoRA rank r and alpha = r=8, alpha=16
PEFT hyperparameters from Appendix 3, fixed across all models.
Learning rate = selected from {0.001, 0.0001}; exact per-model choice not reported
Appendix 2 lists the search space but not the chosen values per backbone, hindering replication.
Dropout = selected from {0.1, 0.2, 0.3}; exact per-model choice not reported
Appendix 2 lists the search space but not the chosen values per backbone, hindering replication.

assumptions (5)

domain assumption Gemini 2.5 Flash-generated HS codes are correct enough to serve as ground truth for training and evaluation.
Section 4.1 states this explicitly. If the labels contain systematic errors, all accuracy numbers are inflated and comparisons are biased toward agreement with Gemini.
domain assumption The HS taxonomy is fixed and complete: 96 chapters, 1,228 headings, and 5,612 subheadings.
Used to build the trie and prototypes in Sections 3.2 and 3.3. The paper later acknowledges taxonomy revisions would require retraining.
domain assumption Single-digit tokenization is usable for the target SLMs after fine-tuning.
The trie mask only permits single-digit tokens, and LLaMA and Phi initially use multi-digit tokens, which degrades zero-shot performance (Section 4.2). Fine-tuning is assumed to shift probability mass to single-digit tokens.
domain assumption Prototype initialization on all non-test data, including validation, does not introduce leakage that materially inflates results.
Section 3.3.1 uses 'every commodity text x except the test data' to initialize prototypes, which includes validation data; early stopping uses the same validation set.
standard math Standard cross-entropy and triplet loss optimization assumptions hold.
Losses in Eqs. (10), (16), (17), and (19) are standard and well-posed under conventional supervised learning assumptions.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Trie-Constrained Token Prediction with Hierarchy-Aware Semantic Alignment for HS Code Prediction." pith.science (2026). https://pith.science/paper/7Y35F2WD

@misc{pith2026260804464,
  author       = {Pith},
  title        = {Pith review of: Trie-Constrained Token Prediction with Hierarchy-Aware Semantic Alignment for HS Code Prediction},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/7Y35F2WD}},
  note         = {Machine review of arXiv:2608.04464}
}

read the original abstract

Harmonized System (HS) code prediction (HSP) from commodity text is essential to international trade, and its importance continues to grow in port logistics. For the purposes of such prediction, recently, large language models (LLMs) have been actively investigated, owing especially to their strong language-understanding capabilities. However, their high computational cost limits deployment in constrained environments such as container terminals. Small language models (SLMs) offer a practical alternative, but their smaller scale makes them prone to generating invalid HS codes and to overlooking the hierarchical semantics between commodity text and HS codes. To address these limitations, this study proposes TRIE-HSA, which combines trie-constrained token prediction with hierarchy-aware semantic alignment (HSA). This method constrains the SLM to predict only valid digits under the HS taxonomy and aligns commodity text representations with the hierarchical structure of HS codes. In extensive experiments on data collected from an operational container terminal, TRIE-HSA improved average HS6 accuracy by 49.96 %p over zero-shot inference and exceeded the strongest task-specific benchmark by 11.94 %p. These results demonstrate that accurate and structurally valid HSP is achievable with fewer than 10 billion parameters. Therefore, TRIE-HSA offers a practical basis for deployment of HSP in port logistics operations that cannot support largescale LLMs.

Review history (2 revisions) →

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

5 extracted references · 2 canonical work pages

[1]

Altaheri, F., & Shaalan, K. (2020). Exploring machine learning models to predict harmonized system code. In M. Themistocleous & M. Papadaki (Eds.), Information systems (pp. 291–303). Cham: Springer International Publishing. Amel, O., Stassin, S., Mahmoudi, S. A., & Siebert, X. (2023). Multimodal approach for harmonized system code prediction. ESANN 2023 P...

arXiv 2020
[140]

Qi, L., Zhang, Q., Lin, X., Zhang, J., & Liao, M. (2025). Attribute knowledge and KBGAT for predicting the accuracy of the harmonized system code for classifying import and export commodities. Scientific Reports, 15(1), Article 43504. Qwen Team, Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., … Qiu, Z. (2025). Qwen2.5 technical report. arXiv preprint a...

arXiv 2025
[631]

Yu, S. C. L., He, J., Gutiérrez-Basulto, V ., & Pan, J. Z. (2023). Instances and labels: Hierarchy -aware joint supervised contrastive learning for hierarchical multi-label text classification. Findings of the Association for Computational Linguistics: EMNLP 2023, 8858–8875. Yuvraj, P., & Devarakonda, S. (2025). ATLAS: Benchmarking and adapting LLMs for g...

work page arXiv 2023
[964]

Hokamp, C., & Liu, Q. (2017). Lexically constrained decoding for sequence generation using grid beam search. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 1535–1546. Hu, E. J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., & Chen, W. (2022). LoRA: Low -rank adaptation of large language models...

work page arXiv 2017
[9901]

Schroff, F., Kalenichenko, D., & Philbin, J. (2015). FaceNet: A unified embedding for face recognition and clustering. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 815–823. Shubham, Arya, A., Roy, S., & Jonnala, S. (2023). An ensemble-based approach for assigning text to correct harmonized system code. 2023 International Confere...

arXiv 2015

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.

[1] [1]

Altaheri, F., & Shaalan, K. (2020). Exploring machine learning models to predict harmonized system code. In M. Themistocleous & M. Papadaki (Eds.), Information systems (pp. 291–303). Cham: Springer International Publishing. Amel, O., Stassin, S., Mahmoudi, S. A., & Siebert, X. (2023). Multimodal approach for harmonized system code prediction. ESANN 2023 P...

arXiv 2020

[2] [140]

Qi, L., Zhang, Q., Lin, X., Zhang, J., & Liao, M. (2025). Attribute knowledge and KBGAT for predicting the accuracy of the harmonized system code for classifying import and export commodities. Scientific Reports, 15(1), Article 43504. Qwen Team, Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., … Qiu, Z. (2025). Qwen2.5 technical report. arXiv preprint a...

arXiv 2025

[3] [631]

Yu, S. C. L., He, J., Gutiérrez-Basulto, V ., & Pan, J. Z. (2023). Instances and labels: Hierarchy -aware joint supervised contrastive learning for hierarchical multi-label text classification. Findings of the Association for Computational Linguistics: EMNLP 2023, 8858–8875. Yuvraj, P., & Devarakonda, S. (2025). ATLAS: Benchmarking and adapting LLMs for g...

work page arXiv 2023

[4] [964]

Hokamp, C., & Liu, Q. (2017). Lexically constrained decoding for sequence generation using grid beam search. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 1535–1546. Hu, E. J., Shen, Y ., Wallis, P., Allen-Zhu, Z., Li, Y ., Wang, S., Wang, L., & Chen, W. (2022). LoRA: Low -rank adaptation of large language models...

work page arXiv 2017

[5] [9901]

Schroff, F., Kalenichenko, D., & Philbin, J. (2015). FaceNet: A unified embedding for face recognition and clustering. 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 815–823. Shubham, Arya, A., Roy, S., & Jonnala, S. (2023). An ensemble-based approach for assigning text to correct harmonized system code. 2023 International Confere...

arXiv 2015