SeqLLM injects behavioral-sequence modeling into a pretrained LLM through a discrete field-level vocabulary, a text-grounded residual projector, and task-prefixed SFT instead of continual pretraining, preserving language ability while enabling joint text-behavior reasoning.
OneReason Technical Report
1 Pith paper cite this work. Polarity classification is still indexing.
abstract
Generative recommendation models in the OneRec family have been widely deployed in many real-world services, such as short-video, live-streaming, advertising, and e-commerce. However, these generative models can only benefit from the scaling advantage, while their reasoning ability is hard to activate, since we cannot construct meaningful Chain-of-Thought (CoT) sequences consisting of itemic tokens only. Inspired by the success of the reasoning-style ``think before answer'' paradigm in the LLM field, we conduct preliminary studies (i.e., OneRec-Think, OpenOneRec) to explore reasoning capability in generative recommendation. Nevertheless, we notice an unexpected phenomenon: the thinking mode does not show advantages over the non-thinking mode. Drawing insights from recent findings on CoT robustness in multi-modal language models, we argue that effective reasoning in recommendation rests on two factors: perception, the ability to ground itemic tokens in their underlying language semantics, and cognition, the ability to reorganize a user's behavior sequence into coherent latent interest points. We therefore propose OneReason, which includes: (1) strong itemic token perception in pre-training, (2) a three-level cognition-enhanced CoT format for recommendation tasks in SFT, and (3) a specialize-then-unify training recipe in RL to enhance the thinking ability.
citation-role summary
citation-polarity summary
fields
cs.CL 1years
2026 1verdicts
CONDITIONAL 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
SeqLLM: Augmenting LLMs with Behavioral-Sequence Modeling for High-Stakes Decisions at WeChat Pay
SeqLLM injects behavioral-sequence modeling into a pretrained LLM through a discrete field-level vocabulary, a text-grounded residual projector, and task-prefixed SFT instead of continual pretraining, preserving language ability while enabling joint text-behavior reasoning.