RLHF should decompose annotations into dimensions each matched to one of three models—extension, evidence, or authority—instead of applying a single unified pipeline.
Group preference optimization: Few-shot alignment of large language models
8 Pith papers cite this work. Polarity classification is still indexing.
8
Pith papers citing it
representative citing papers
A dataset revealing high inter-designer disagreement on UI preferences motivates a sample-efficient method that personalizes generative interfaces by embedding new users in the space of prior designers, outperforming baselines in both modeling and user preference.
Bidirectional LLM-GNN co-teaching with round-based pseudo-label preference optimization outperforms golden-teacher baselines on few-shot TAG benchmarks by 3-8% absolute gains.
A new personalization benchmark shows reward models are only ~76% accurate at identifying user-specific preference, and its scores track downstream BoN/PPO quality better than prior benchmarks.
WIMHF uses sparse autoencoders on seven preference datasets to identify a handful of human-interpretable features that capture the majority of the signal in black-box preference models.
The paper formalizes three types of pluralistic AI models and three benchmark classes, arguing that current alignment techniques may reduce rather than increase distributional pluralism.
PAFO applies Pareto fairness optimization and group-specialized distillation to produce a single personalized reward model that improves accuracy for both majority and minority preference groups without requiring group labels at inference.
POPI distills user preferences into reusable natural-language summaries via a shared inference model and conditions a generator on them, trained jointly with RL to improve personalization quality while cutting context length by up to 10x on benchmarks.
citing papers explorer
-
Three Models of RLHF Annotation: Extension, Evidence, and Authority
RLHF should decompose annotations into dimensions each matched to one of three models—extension, evidence, or authority—instead of applying a single unified pipeline.
-
Efficient Personalization of Generative User Interfaces
A dataset revealing high inter-designer disagreement on UI preferences motivates a sample-efficient method that personalizes generative interfaces by embedding new users in the space of prior designers, outperforming baselines in both modeling and user preference.
-
Beyond the Golden Teacher: Enhancing Graph Learning through LLM-GNN Co-teaching
Bidirectional LLM-GNN co-teaching with round-based pseudo-label preference optimization outperforms golden-teacher baselines on few-shot TAG benchmarks by 3-8% absolute gains.
-
Personalized RewardBench: Evaluating Reward Models with Human Aligned Personalization
A new personalization benchmark shows reward models are only ~76% accurate at identifying user-specific preference, and its scores track downstream BoN/PPO quality better than prior benchmarks.
-
What's In My Human Feedback? Learning Interpretable Descriptions of Preference Data
WIMHF uses sparse autoencoders on seven preference datasets to identify a handful of human-interpretable features that capture the majority of the signal in black-box preference models.
-
A Roadmap to Pluralistic Alignment
The paper formalizes three types of pluralistic AI models and three benchmark classes, arguing that current alignment techniques may reduce rather than increase distributional pluralism.
-
PAFO: Pareto Fairness Optimization for Personalized Reward Modeling
PAFO applies Pareto fairness optimization and group-specialized distillation to produce a single personalized reward model that improves accuracy for both majority and minority preference groups without requiring group labels at inference.
-
POPI: Personalizing LLMs via Optimized Natural Language Preference Inference
POPI distills user preferences into reusable natural-language summaries via a shared inference model and conditions a generator on them, trained jointly with RL to improve personalization quality while cutting context length by up to 10x on benchmarks.