LPFS: Learnable Polarizing Feature Selection for Click-Through Rate Prediction
read the original abstract
In industry, feature selection is a standard but necessary step to search for an optimal set of informative feature fields for efficient and effective training of deep Click-Through Rate (CTR) models. Most previous works measure the importance of feature fields by using their corresponding continuous weights from the model, then remove the feature fields with small weight values. However, removing many features that correspond to small but not exact zero weights will inevitably hurt model performance and not be friendly to hot-start model training. There is also no theoretical guarantee that the magnitude of weights can represent the importance, thus possibly leading to sub-optimal results if using these methods. To tackle this problem, we propose a novel Learnable Polarizing Feature Selection (LPFS) method using a smoothed-$\ell^0$ function in literature. Furthermore, we extend LPFS to LPFS++ by our newly designed smoothed-$\ell^0$-liked function to select a more informative subset of features. LPFS and LPFS++ can be used as gates inserted at the input of the deep network to control the active and inactive state of each feature. When training is finished, some gates are exact zero, while others are around one, which is particularly favored by the practical hot-start training in the industry, due to no damage to the model performance before and after removing the features corresponding to exact-zero gates. Experiments show that our methods outperform others by a clear margin, and have achieved great A/B test results in KuaiShou Technology.
This paper has not been read by Pith yet.
Forward citations
Cited by 5 Pith papers
-
Beyond Distributive Justice: Hermeneutical Fairness in Ad Delivery
Ad allocation should add constraints against hermeneutical deprivation and distortion to prevent epistemic harms from concentrating in disadvantaged groups.
-
LeAP: Learnable Adaptive Permutation for Feature Selection in Heterogeneous and Sparse Recommender Systems
LeAP is a model-agnostic plug-in that learns to permute and rank heterogeneous sparse features for selection, achieving SOTA on public datasets and removing over 3600 redundant features in a production system with 12k...
-
Performance-Driven QUBO for Recommender Systems on Quantum Annealers
PDQUBO is a new performance-driven QUBO method for feature selection in recommender systems that incorporates counterfactual performance impacts of features and pairs, is model-agnostic, and outperforms prior quantum ...
-
Light-FMP: Lightweight Feature and Model Pruning for Enhanced Deep Recommender Systems
Light-FMP prunes features and model parameters in deep recommender systems by pretraining a hard-concrete masking layer on data subsets, then retraining the reduced model to improve both efficiency and accuracy over p...
-
ShuffleGate: Scalable Feature Optimization for Recommender Systems via Batch-wise Sensitivity Learning
ShuffleGate learns polarized importance gates by measuring model sensitivity to random component shuffling, unifying feature selection, dimension selection, and embedding compression with SOTA results on four recommen...
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.