REVIEW 5 cited by
LoRe: Personalizing LLMs via Low-Rank Reward Modeling
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
LoRe: Personalizing LLMs via Low-Rank Reward Modeling
read the original abstract
Personalizing large language models (LLMs) to accommodate diverse user preferences is essential for enhancing alignment and user satisfaction. Traditional reinforcement learning from human feedback (RLHF) approaches often rely on monolithic value representations, limiting their ability to adapt to individual preferences. We introduce a novel framework that leverages low-rank preference modeling to efficiently learn and generalize user-specific reward functions. By representing reward functions in a low-dimensional subspace and modeling individual preferences as weighted combinations of shared basis functions, our approach avoids rigid user categorization while enabling scalability and few-shot adaptation. We validate our method on multiple preference datasets, demonstrating superior generalization to unseen users and improved accuracy in preference prediction tasks.
Forward citations
Cited by 5 Pith papers
-
LLM Evaluation as Tensor Completion: Low Rank Structure and Semiparametric Efficiency
LLM pairwise evaluation is recast as low-rank tensor completion, yielding semiparametric efficient estimators and asymptotic normality for ability functionals via a score-whitening correction for anisotropic operators.
-
PAFO: Pareto Fairness Optimization for Personalized Reward Modeling
PAFO applies Pareto fairness optimization and group-specialized distillation to produce a single personalized reward model that improves accuracy for both majority and minority preference groups without requiring grou...
-
Beyond Isolated Behaviors: Hierarchical User Modeling for LLM Personalization
PHF applies Bourdieu's Theory of Practice to create hierarchical user models for LLM personalization and reports consistent gains on the LaMP benchmark.
-
POPI: Personalizing LLMs via Optimized Natural Language Preference Inference
POPI distills user preferences into reusable natural-language summaries via a shared inference model and conditions a generator on them, trained jointly with RL to improve personalization quality while cutting context...
-
Preference Learning for AI Alignment: a Causal Perspective
Advocates applying causal inference to preference learning for LLM alignment to diagnose generalization failures and guide better data practices.
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.