CL-Bench is the first expert-validated benchmark for continual learning in frontier LLMs across six real-world domains, showing limited gains and that naive in-context learning outperforms dedicated memory systems.
hub Mixed citations
Bridging Language and Items for Retrieval and Recommendation: Benchmarking LLMs as Semantic Encoders
Mixed citation behavior. Most common role is background (44%).
61
Pith papers citing it
4
external citations · Pith
Background
44% of classified citations
abstract
Feature engineering has long been central to recommender systems, yet effectively leveraging textual item features remains challenging. Recent advances in large language models (LLMs) have enabled their use as semantic encoders for recommendation, but their roles and behaviors in this setting are still not well understood. Prior studies often rely on general-purpose embedding benchmarks (e.g., MTEB) when selecting LLMs, overlooking the unique characteristics of recommendation tasks. To address this gap, we introduce BLaIR, a comprehensive benchmark for evaluating LLMs as semantic encoders in recommendation scenarios. We contribute (1) a new large-scale Amazon Reviews 2023 dataset with over 570 million reviews and 48 million items, (2) a unified benchmark covering sequential recommendation, collaborative filtering, and product search, and (3) a new complex-query product search task featuring both semi-synthetic and real-world evaluation datasets. Experiments with 11 leading LLMs show that their rankings on BLaIR show little correlation with MTEB, highlighting the unique challenges of semantic encoding in recommendation.
hub tools
citation-role summary
dataset 4
background 3
method 2
citation-polarity summary
representative citing papers
Retrieval coverage limits LLM rerankers in cold-start recommendation; a learned hybrid fusion improves pool quality but LLM reranking often degrades end-to-end performance while simpler rankers exploit the pool.
SemCEB is the first benchmark for cardinality estimation over semantic operators, evaluating sampling methods and Semantic Histograms on accuracy, cost, latency, and memory using 102 queries on a real-world dataset.
SPHERE uses LLM-generated semantic personas and a dual-tower architecture to transfer recommendation knowledge across disjoint domains without shared users or items, showing consistent gains over NCF, SVD++, and LightGCN on Amazon Books, Goodreads, and Steam while linking transfer success to target
FedMPO recovers missing modalities via topology-aware generation, filters noisy recoveries with missing-aware routing, and uses reliability-aware aggregation to achieve up to 5.65% gains over baselines in high-missing and non-IID federated graph settings.
RecoAtlas is a benchmark that evaluates LLM recommendation agents on behavior-grounded metrics for relevance, complementarity, and diversity in addition to semantic coherence.
fmxcoders improve cross-layer feature recovery in transformers via factorized weights and layer masking, delivering 10-30 point probing F1 gains, 25-50% lower MSE, doubled functional coherence, and 3-13x more coherent latents than standard crosscoders on GPT2-Small, Pythia, and Gemma2 models.
FraudBench shows that current multimodal LLMs and specialized AI-image detectors often fail to spot AI-generated fake damage in refund evidence, with true positive rates frequently below 50% on synthetic subsets while producing false positives on real damage.
On-policy distillation has an extrapolation cliff at closed-form lambda*(p,b,c) set by teacher modal probability, warm-start mass, and clip strength, past which training shifts from format-preserving to format-collapsing.
Autoregressive semantic ID generation creates tree-induced probability correlations that prevent generative recommenders from capturing simple patterns; Latte adds latent tokens to relax these correlations.
InvariRank achieves permutation-invariant listwise reranking for LLM-based recommendations via a structured attention mask that blocks cross-candidate interactions and shared positional framing under RoPE, enabling stable rankings in one forward pass.
Hyper-Parallel Decoding enables parallel generation of independent sequences in LLMs via position ID manipulation, delivering up to 13.8X speedup for attribute value extraction.
HORIZON creates a cross-domain, long-horizon user modeling benchmark from Amazon Reviews that tests generalization across time, domains, and unseen users, exposing gaps in sequential and LLM-based recommendation models.
DynLP is a parallel dynamic batch update algorithm for label propagation that achieves significant speedups by updating only relevant parts of the graph on GPUs.
GenRecEdit injects cold-start items into generative recommendation models via context-aware token editing and interference-reducing triggers, boosting cold-start accuracy while using only 9.5% of retraining time.
ItemRAG augments LLM recommendation prompts with item-level retrievals that blend semantic and co-purchase signals, outperforming user-history RAG in both standard and cold-start settings.
VoteGCL augments graph-based recommendation systems with high-confidence synthetic interactions generated via majority-voting LLM reranks and integrates them into graph contrastive learning to improve accuracy and reduce popularity bias.
CoPersona introduces a multiplex persona graph for facet-level peer alignment and a dual-branch retrieval-plus-reasoning architecture to improve LLM personalization under sparse and biased user interaction data.
FedLAB organizes multimodal graph knowledge into typed hierarchical codebooks for modality evidence, node semantics, and topology context via federated semantic barycenter pre-training, improving performance by up to 7.53% on benchmarks while enabling semantic traceability.
CEB and TIDE are two-layer append-only B+-tree indexes for intervals under the increasing ending time assumption that claim smaller size, faster insertions, and superior query performance over prior art.
PRISMR replaces in-context list processing with a hypernetwork-generated instance-specific LoRA adapter to reduce parse collapse and improve multimodal listwise ranking performance.
GraphMNL applies negative learning as cross-branch guidance in multimodal graphs to mitigate semantic imbalance without propagating bias from dominant branches.
GenAIR generates LLM-derived archetype embeddings for items and applies behavioral calibration to close the semantic-behavioral gap, yielding performance gains on three real-world datasets when integrated with existing sequential models.
Introduces ShopTrajQA long-context benchmark and an RLVR-trained tool-augmented agent that bypasses LLM context limits by external file storage and code-based retrieval for shopping trajectories.
citing papers explorer
-
Continual Learning Bench: Evaluating Frontier AI Systems in Real-World Stateful Environments
CL-Bench is the first expert-validated benchmark for continual learning in frontier LLMs across six real-world domains, showing limited gains and that naive in-context learning outperforms dedicated memory systems.
-
Diagnosing and Mitigating Retrieval Bottlenecks in LLM-Based Cold-Start Recommendation
Retrieval coverage limits LLM rerankers in cold-start recommendation; a learned hybrid fusion improves pool quality but LLM reranking often degrades end-to-end performance while simpler rankers exploit the pool.
-
SemCEB: A Cardinality Estimation Benchmark for Semantic Operators
SemCEB is the first benchmark for cardinality estimation over semantic operators, evaluating sampling methods and Semantic Histograms on accuracy, cost, latency, and memory using 102 queries on a real-world dataset.
-
Breaking the Information Silo: Semantic Personas for Cross-Domain Recommendation
SPHERE uses LLM-generated semantic personas and a dual-tower architecture to transfer recommendation knowledge across disjoint domains without shared users or items, showing consistent gains over NCF, SVD++, and LightGCN on Amazon Books, Goodreads, and Steam while linking transfer success to target
-
Towards Robust Federated Multimodal Graph Learning under Modality Heterogeneity
FedMPO recovers missing modalities via topology-aware generation, filters noisy recoveries with missing-aware routing, and uses reliability-aware aggregation to achieve up to 5.65% gains over baselines in high-missing and non-IID federated graph settings.
-
RecoAtlas: From Semantic Plausibility to Set-Level Utility in LLM Recommendation Agents
RecoAtlas is a benchmark that evaluates LLM recommendation agents on behavior-grounded metrics for relevance, complementarity, and diversity in addition to semantic coherence.
-
fmxcoders: Factorized Masked Crosscoders for Cross-Layer Feature Discovery
fmxcoders improve cross-layer feature recovery in transformers via factorized weights and layer masking, delivering 10-30 point probing F1 gains, 25-50% lower MSE, doubled functional coherence, and 3-13x more coherent latents than standard crosscoders on GPT2-Small, Pythia, and Gemma2 models.
-
FraudBench: A Multimodal Benchmark for Detecting AI-Generated Fraudulent Refund Evidence
FraudBench shows that current multimodal LLMs and specialized AI-image detectors often fail to spot AI-generated fake damage in refund evidence, with true positive rates frequently below 50% on synthetic subsets while producing false positives on real damage.
-
The Extrapolation Cliff in On-Policy Distillation of Near-Deterministic Structured Outputs
On-policy distillation has an extrapolation cliff at closed-form lambda*(p,b,c) set by teacher modal probability, warm-start mass, and clip strength, past which training shifts from format-preserving to format-collapsing.
-
Expressiveness Limits of Autoregressive Semantic ID Generation in Generative Recommendation
Autoregressive semantic ID generation creates tree-induced probability correlations that prevent generative recommenders from capturing simple patterns; Latte adds latent tokens to relax these correlations.
-
One Pass, Any Order: Position-Invariant Listwise Reranking for LLM-Based Recommendation
InvariRank achieves permutation-invariant listwise reranking for LLM-based recommendations via a structured attention mask that blocks cross-candidate interactions and shared positional framing under RoPE, enabling stable rankings in one forward pass.
-
Breaking the Autoregressive Chain: Hyper-Parallel Decoding for Efficient LLM-Based Attribute Value Extraction
Hyper-Parallel Decoding enables parallel generation of independent sequences in LLMs via position ID manipulation, delivering up to 13.8X speedup for attribute value extraction.
-
HORIZON: A Benchmark for In-the-wild User Behaviour Modeling
HORIZON creates a cross-domain, long-horizon user modeling benchmark from Amazon Reviews that tests generalization across time, domains, and unseen users, exposing gaps in sequential and LLM-based recommendation models.
-
DynLP: Parallel Dynamic Batch Update for Label Propagation in Semi-Supervised Learning
DynLP is a parallel dynamic batch update algorithm for label propagation that achieves significant speedups by updating only relevant parts of the graph on GPUs.
-
GenRecEdit: Adapting Model Editing for Generative Recommendation with Cold-Start Items
GenRecEdit injects cold-start items into generative recommendation models via context-aware token editing and interference-reducing triggers, boosting cold-start accuracy while using only 9.5% of retraining time.
-
ItemRAG: Item-Based Retrieval-Augmented Generation for LLM-Based Recommendation
ItemRAG augments LLM recommendation prompts with item-level retrievals that blend semantic and co-purchase signals, outperforming user-history RAG in both standard and cold-start settings.
-
VoteGCL: Enhancing Graph-based Recommendations with Majority-Voting LLM-Rerank Augmentation
VoteGCL augments graph-based recommendation systems with high-confidence synthetic interactions generated via majority-voting LLM reranks and integrates them into graph contrastive learning to improve accuracy and reduce popularity bias.
-
CoPersona: Collaborative Persona Graphs for Robust LLM Personalization
CoPersona introduces a multiplex persona graph for facet-level peer alignment and a dual-branch retrieval-plus-reasoning architecture to improve LLM personalization under sparse and biased user interaction data.
-
FedLAB: Traceable Semantic Codebooks for Federated Multimodal Graph Foundation Learning
FedLAB organizes multimodal graph knowledge into typed hierarchical codebooks for modality evidence, node semantics, and topology context via federated semantic barycenter pre-training, improving performance by up to 7.53% on benchmarks while enabling semantic traceability.
-
Disk-Based Interval Indexes Under the Increasing Ending Time Assumption
CEB and TIDE are two-layer append-only B+-tree indexes for intervals under the increasing ending time assumption that claim smaller size, faster insertions, and superior query performance over prior art.
-
PRISMR: Overcoming Parse Collapse in Multimodal Listwise Ranking via Parameterized Representation Internalization
PRISMR replaces in-context list processing with a hypernetwork-generated instance-specific LoRA adapter to reduce parse collapse and improve multimodal listwise ranking performance.
-
Multimodal Graph Negative Learning
GraphMNL applies negative learning as cross-branch guidance in multimodal graphs to mitigate semantic imbalance without propagating bias from dominant branches.
-
Generative Archetype-Grounded Item Representations for Sequential Recommendation
GenAIR generates LLM-derived archetype embeddings for items and applies behavioral calibration to close the semantic-behavioral gap, yielding performance gains on three real-world datasets when integrated with existing sequential models.
-
Customer-Agent: Overcoming Context Limitations in Ultra-Long Shopping Trajectories via Tool-Augmented Agents and RLVR
Introduces ShopTrajQA long-context benchmark and an RLVR-trained tool-augmented agent that bypasses LLM context limits by external file storage and code-based retrieval for shopping trajectories.
-
Preference-Aware Rubric Learning for Personalized Evaluation
PARL formulates personalized LLM evaluation as a learning problem that induces preference-aware rubrics from raw user histories via discriminative RL and self-validation.
-
LATTE: Forecasting Peer Anchored Preference Trajectories for Personalized LLM Generation
LATTE improves personalized LLM generation by forecasting peer-anchored relative preference trajectories and injecting the forecast via a State to Token Bridge, raising ROUGE-L from 0.219-0.245 to 0.259 on Amazon Reviews 2023 over static and compression baselines.
-
PipeANN-Filter: An Efficient Filtered Vector Search System on SSD
PipeANN-Filter improves filtered vector search latency and throughput on SSD by exploring a superset of valid vectors identified via probabilistic filters and verifying attributes only after selecting top-k candidates.
-
Conditional Attribute Estimation with Autoregressive Sequence Models
Conditional Attribute Transformers jointly estimate next-token probabilities and conditional attribute values for autoregressive sequence models, enabling credit assignment, counterfactuals, and steerable generation in one pass.
-
Task-Aware Automated User Profile Generation for Recommendation Simulation Using Large Language Models
APG4RecSim automatically generates realistic user profiles for LLM-based recommendation simulations, outperforming manual baselines by up to 7% in nDCG@10 and 8% in JSD on three benchmark datasets.
-
CAMPA: Efficient and Aligned Multimodal Graph Learning via Decoupled Propagation and Aggregation
CAMPA resolves modal conflicts in decoupled multimodal GNNs via cross-modal aligned propagation and trajectory aligned aggregation, outperforming coupled and decoupled baselines on benchmarks while retaining efficiency.
-
LLM Agents Enable User-Governed Personalization Beyond Platform Boundaries
LLM agents enable users to integrate cross-platform and offline data for personalization that outperforms single-platform baselines in proof-of-concept tests.
-
Bridging Textual Profiles and Latent User Embeddings for Personalization
BLUE aligns LLM-generated textual user profiles with embedding-based recommendation objectives via reinforcement learning and next-item text supervision, yielding better zero-shot performance and cross-domain transfer than baselines.
-
PREFER: Personalized Review Summarization with Online Preference Learning
PREFER is an online preference learning system that generates personalized review summaries and improves alignment with user interests in simulations on Amazon review data.
-
One Pool, Two Caches: Adaptive HBM Partitioning for Accelerating Generative Recommender Serving
HELM adaptively partitions HBM between EMB and KV caches via a three-layer PPO controller and EMB-KV-aware scheduling, reducing P99 latency by 24-38% while achieving 93.5-99.6% SLO satisfaction on production workloads.
-
Decision-aware User Simulation Agent for Evaluating Conversational Recommender Systems
Hesitator is a theory-grounded simulator that separates utility-based item selection from overload-aware commitment decisions to reduce unrealistic high acceptance rates in conversational recommender evaluations.
-
From Top-1 to Top-K: A Reproducibility Study and Benchmarking of Counterfactual Explanations for Recommender Systems
A unified benchmark of eleven CE methods shows effectiveness-sparsity trade-offs vary by method and format, performance is consistent from item to list level, and graph-based explainers face scalability limits on large graphs.
-
Self-Distilled Reinforcement Learning for Co-Evolving Agentic Recommender Systems
CoARS enables co-evolving recommender and user agents by using interaction-derived rewards and self-distilled credit assignment to internalize multi-turn feedback into model parameters, outperforming prior agentic baselines.
-
PeReGrINE: Evaluating Personalized Review Fidelity with User Item Graph Context
PeReGrINE is a graph-based benchmark that restructures Amazon Reviews 2023 with temporal cutoffs and introduces dissonance analysis to measure how well retrieval-conditioned models match user style and product consensus.
-
TRU: Targeted Reverse Update for Efficient Multimodal Recommendation Unlearning
TRU is a plug-and-play unlearning method for multimodal recommenders that applies ranking fusion, modality scaling, and layer isolation to achieve better retain-forget trade-offs than uniform baselines.
-
Detecting LLM-Generated Spam Reviews by Integrating Language Model Embeddings and Graph Neural Network
Introduces FraudSquad, a hybrid model using language model embeddings and a gated graph transformer that outperforms baselines on newly created LLM-generated spam review datasets.
-
SessionIntentBench: A Multi-task Inter-session Intention-shift Modeling Benchmark for E-commerce Customer Behavior Understanding
SessionIntentBench is a large-scale multimodal benchmark for inter-session intention-shift modeling in e-commerce, with 1.95M intention entries and human-annotated gold labels showing current L(V)LMs struggle but improve when intention is injected.
-
Drift Happens: An Empirical Study of Neural Architecture Robustness to Temporal Distribution Shift
Architectures with stronger inductive biases attain higher in-distribution accuracy but degrade faster under temporal distribution shift, while frozen pretrained encoders trade accuracy for stability.
-
Dialogue to Discovery: Attribute-Aware Preference Elicitation for Conversational Product Search Assistants
D2D adaptively prioritizes informative attribute queries and times recommendations in conversational search, yielding 22-30% higher target accuracy and shorter conversations than baselines in simulations.
-
URecJPQ: Memory-efficient Multimodal Recommendation Models through RecJPQ in Large-Scale Scenarios
URecJPQ compresses user and item embeddings via joint product quantization for multimodal top-k recommendation, cutting checkpoint size 86-98% and parameters 98-99% with average 8.5% recall drop across three datasets.
-
LRanker: LLM Ranker for Massive Candidates
LRanker combines K-means candidate aggregation with graph-partitioned ensemble of query embeddings to improve LLM ranking accuracy and scalability on massive candidate pools, reporting 3-30% gains on RBench tasks up to 6.8M candidates.
-
Causal Risk Minimization for High-Dimensional Treatments
Proposes causal risk minimization via higher-order moment-balancing error decomposition and attribute projection for high-dimensional treatments, with experiments on continuous, discrete, and text data.
-
Ocean4Rec: Offline LLM-Derived OCEAN Profiles for Request-Time VOD Reranking
Ocean4Rec uses offline LLM to create OCEAN profiles for items and time-decayed user profiles for request-time numeric reranking, improving NDCG@20 by 7.6% and 61.5% over base+recency in offline VOD evaluations.
-
Don't Let Bandit Feedback Pull Continual LLM-Recommender Updates Off Target
ABPO combines group-relative policy optimization with anchored exposure correction and asymmetric feedback handling to enable effective continual updates for LLM recommenders under bandit feedback constraints.
-
RcLLM: Accelerating Generative Recommendation via Beyond-Prefix KV Caching
RcLLM accelerates generative recommendation inference by 1.31x-9.51x in TTFT through beyond-prefix KV caching, replicated user caches, sharded item caches, affinity scheduling, and selective attention with negligible accuracy loss.
-
Prompt Optimization for User Simulation in Conversational Recommender Systems: A Multi-Objective Framework
A multi-objective prompt optimization framework for LLM user simulators in conversational recommender systems improves behavioral alignment with human patterns over baselines.