REVIEW 11 cited by
Beyond Unimodal Boundaries: Generative Recommendation with Multimodal Semantics
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Beyond Unimodal Boundaries: Generative Recommendation with Multimodal Semantics
read the original abstract
Generative recommendation (GR) has become a powerful paradigm in recommendation systems that implicitly links modality and semantics to item representation, in contrast to previous methods that relied on non-semantic item identifiers in autoregressive models. However, previous research has predominantly treated modalities in isolation, typically assuming item content is unimodal (usually text). We argue that this is a significant limitation given the rich, multimodal nature of real-world data and the potential sensitivity of GR models to modality choices and usage. Our work aims to explore the critical problem of Multimodal Generative Recommendation (MGR), highlighting the importance of modality choices in GR nframeworks. We reveal that GR models are particularly sensitive to different modalities and examine the challenges in achieving effective GR when multiple modalities are available. By evaluating design strategies for effectively leveraging multiple modalities, we identify key challenges and introduce MGR-LF++, an enhanced late fusion framework that employs contrastive modality alignment and special tokens to denote different modalities, achieving a performance improvement of over 20% compared to single-modality alternatives.
Forward citations
Cited by 11 Pith papers
-
From Raw IDs to Semantic Planning: How Recommender Systems Utilize Information at Scale
Recommender systems are moving from raw IDs to semantic IDs, and the authors argue the next stage is 'semantic planning'—predicting an exposure's goal before choosing the item.
-
MLPs are Efficient Distilled Generative Recommenders
SID-MLP distills autoregressive generative recommenders into efficient position-specific MLP heads for Semantic ID tasks, achieving 8.74x faster inference with matching accuracy.
-
Expressiveness Limits of Autoregressive Semantic ID Generation in Generative Recommendation
Autoregressive semantic ID generation creates tree-induced probability correlations that prevent generative recommenders from capturing simple patterns; Latte adds latent tokens to relax these correlations.
-
SynGR: Unleashing the Potential of Cross-Modal Synergy for Generative Recommendation
SynGR is a synergistic generative recommendation method that masks dominant-modal tokens and uses contrastive learning to capture cross-modal synergies, outperforming prior methods on Amazon Arts, Games, and Instruments.
-
Efficient Item ID Generation for Large-Scale LLM-based Recommendation
LLM-based recommenders can treat item IDs as single direct embeddings and decode in one step, with a two-level softmax for efficiency and quality matching or beating multi-token models.
-
LLM-Based Generative Retrieval for Snapchat Content Recommendation
SnapLGR, a production LLM-based generative retrieval system for Snapchat short video, lifted View Time 0.37% and related engagement metrics in a 7-day A/B test, with offline ablation attributing most of the gain to de...
-
Stream-aware Side Adaptation for Large Pre-trained Multimodal Embedding Models in Sequential Recommendation
Stream-aware fusion (SHAF) and residual stream adapters (ReSA) stabilize deep side adaptation of frozen multimodal embedding models and improve sequential recommendation over standard side adapters.
-
From Raw IDs to Semantic Planning: How Recommender Systems Utilize Information at Scale
Recommender systems are moving from raw IDs to semantic IDs, and the next step should be semantic planning that first predicts an exposure's purpose before choosing or generating content.
-
Semantic IDs for Recommender Systems at Snapchat: Use Cases, Technical Challenges, and Design Choices
Semantic IDs with STE-trained multi-modal RQ-VAE and heuristic collision resolution improve Snapchat ranking and generative retrieval offline and in online A/B tests.
-
From Raw IDs to Semantic Planning: How Recommender Systems Utilize Information at Scale
Industrial recommenders are evolving from raw IDs through semantic IDs toward semantic planning, where the system predicts a semantic next-exposure target before choosing or generating a concrete item.
-
SynGR: Unleashing the Potential of Cross-Modal Synergy for Generative Recommendation
SynGR is a new framework for generative recommendation that constrains overreliance on single modalities to exploit synergistic cross-modal information for better item semantics and user preference modeling.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.