UQ4CT integrates functional-level uncertainty calibration into mixture-of-experts LoRA fine-tuning via a dedicated loss, cutting expected calibration error by over 25% on multiple-choice and generative QA tasks.
Uncertainty-penalized reinforcement learning from human feedback with diverse reward lora ensembles.arXiv preprint arXiv:2401.00243
5 Pith papers cite this work, alongside 1 external citations. Polarity classification is still indexing.
5
Pith papers citing it
1
external citations · external index
representative citing papers
A distributional reward model p(r|x,y) yields the closed-form effective reward ilde r(x,y) = eta ext{log} ext{E}_p[e^{r/eta}] (pessimistic branch) that unifies prior RLHF aggregation heuristics under Bayesian or KL-DRO views.
P4IR applies supervised fine-tuning followed by GRPO reinforcement learning to reduce tree edit distance by up to 23.8% and Levenshtein distance by up to 38.6% versus SFT baselines while outperforming several frontier LLMs on code structure and semantics for automated building code compliance.
Proxy RL produces a staged proxy-internalization capability that emerges before and predicts reward hacking in coding environments.
citing papers explorer
-
Functional-level Uncertainty Quantification for Calibrated Fine-tuning on LLMs
UQ4CT integrates functional-level uncertainty calibration into mixture-of-experts LoRA fine-tuning via a dedicated loss, cutting expected calibration error by over 25% on multiple-choice and generative QA tasks.
-
A Unifying Lens on Reward Uncertainty in RLHF
A distributional reward model p(r|x,y) yields the closed-form effective reward ilde r(x,y) = eta ext{log} ext{E}_p[e^{r/eta}] (pessimistic branch) that unifies prior RLHF aggregation heuristics under Bayesian or KL-DRO views.
-
Reinforcement learning to improve large language model-based automated code compliance systems
P4IR applies supervised fine-tuning followed by GRPO reinforcement learning to reduce tree edit distance by up to 23.8% and Levenshtein distance by up to 38.6% versus SFT baselines while outperforming several frontier LLMs on code structure and semantics for automated building code compliance.
-
Proxy Reward Internalization and Mechanistic Exploitation: A Learned Precursor to Reward Hacking and Its Generalization
Proxy RL produces a staged proxy-internalization capability that emerges before and predicts reward hacking in coding environments.
- Wasserstein Distributionally Robust Regret Optimization for Reinforcement Learning from Human Feedback