OMP sparse coding of donor token embeddings, with coefficients transferred to the base embedding space, preserves LLM performance after tokenizer replacement better than published zero-shot baselines, though simple heuristics tie it when token overlap is high.
Orthogonal Matching Pursuit for Text Classification
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
In text classification, the problem of overfitting arises due to the high dimensionality, making regularization essential. Although classic regularizers provide sparsity, they fail to return highly accurate models. On the contrary, state-of-the-art group-lasso regularizers provide better results at the expense of low sparsity. In this paper, we apply a greedy variable selection algorithm, called Orthogonal Matching Pursuit, for the text classification task. We also extend standard group OMP by introducing overlapping Group OMP to handle overlapping groups of features. Empirical analysis verifies that both OMP and overlapping GOMP constitute powerful regularizers, able to produce effective and very sparse models. Code and data are available online: https://github.com/y3nk0/OMP-for-Text-Classification .
citation-role summary
other 1
citation-polarity summary
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1roles
other 1polarities
unclear 1representative citing papers
citing papers explorer
-
Training-Free Tokenizer Transplantation via Orthogonal Matching Pursuit
OMP sparse coding of donor token embeddings, with coefficients transferred to the base embedding space, preserves LLM performance after tokenizer replacement better than published zero-shot baselines, though simple heuristics tie it when token overlap is high.