A from-scratch model that tokenizes items into hierarchical codes and predicts next-item codes reaches higher average zero-shot AUC on 8 datasets than LLM recommenders up to 7B parameters.
Evaluating Recabilities of Foundation Models: A Multi-Domain, Multi-Dataset Benchmark
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Comprehensive evaluation of the recommendation capabilities of existing foundation models across diverse datasets and domains is essential for advancing the development of recommendation foundation models. In this study, we introduce RecBench-MD, a novel and comprehensive benchmark designed to assess the recommendation abilities of foundation models from a zero-resource, multi-dataset, and multi-domain perspective. Through extensive evaluations of 19 foundation models across 15 datasets spanning 10 diverse domains -- including e-commerce, entertainment, and social media -- we identify key characteristics of these models in recommendation tasks. Our findings suggest that in-domain fine-tuning achieves optimal performance, while cross-dataset transfer learning provides effective practical support for new recommendation scenarios. Additionally, we observe that multi-domain training significantly enhances the adaptability of foundation models. All code and data have been publicly released to facilitate future research.
fields
cs.IR 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
RecBase: Generative Foundation Model Pretraining for Zero-Shot Recommendation
A from-scratch model that tokenizes items into hierarchical codes and predicts next-item codes reaches higher average zero-shot AUC on 8 datasets than LLM recommenders up to 7B parameters.