REVIEW 13 cited by
OffsetBias: Leveraging Debiased Data for Tuning Evaluators
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Employing Large Language Models (LLMs) to assess the quality of generated responses, such as prompting instruct-tuned models or fine-tuning judge models, has become a widely adopted evaluation method. It is also known that such evaluators are vulnerable to biases, such as favoring longer responses. While it is important to overcome this problem, the specifics of these biases remain under-explored. In this work, we qualitatively identify six types of biases inherent in various judge models. We propose EvalBiasBench as a meta-evaluation collection of hand-crafted test cases for each bias type. Additionally, we present de-biasing dataset construction methods and the associated preference dataset OffsetBias. Experimental results demonstrate that fine-tuning on our dataset significantly enhances the robustness of judge models against biases and improves performance across most evaluation scenarios. We release our datasets and the fine-tuned judge model to public.
Forward citations
Cited by 13 Pith papers
-
Sandcastles in the Storm: Revisiting the (Im)possibility of Strong Watermarking
Random-walk attacks against text watermarks fail to remove the mark in most cases once human-rated text quality is preserved, calling the practical force of the WITS impossibility result into question.
-
Evaluating Judges as Evaluators: The JETTS Benchmark of LLM-as-Judges as Test-Time Scaling Evaluators
JETTS, a new benchmark, shows LLM-as-judges are competitive in response reranking, worse than process reward models in beam search, and ineffective as critique providers for refinement.
-
RewardAnything: Generalizable Principle-Following Reward Models
RewardAnything follows natural-language reward principles at inference time and, with the new RABench benchmark, demonstrates that principle-conditioned listwise training beats fixed-preference reward models on held-o...
-
Helpful Agent Meets Deceptive Judge: Understanding Vulnerabilities in Agentic Workflows
LLM agents frequently switch correct answers after one round of misleading feedback, and the new WAFER-QA benchmark measures this with web-backed critiques.
-
J1: Exploring Simple Test-Time Scaling for LLM-as-a-Judge
J1-7B, a judge LLM trained with supervised fine-tuning and reinforcement learning, improves when forced to reflect with 'wait' tokens, and the scaling ability emerges during the RL phase.
-
WorldPM: Scaling Human Preference Modeling
Preference modeling exhibits scaling laws for objective and adversarial tasks, with up to 5-14% gains when used as initialization for fine-tuning on smaller human-preference datasets.
-
An Empirical Study of LLM-as-a-Judge: How Design Choices Impact Evaluation Reliability
The reliability of LLM-as-a-Judge depends strongly on scoring rubrics and reference answers; sampling with averaging outperforms greedy decoding, and chain-of-thought reasoning adds little when rubrics are clear.
-
A Survey of Automatic Evaluation Methods on Text, Visual and Speech Generations
A unified taxonomy and comparative meta-evaluation of automatic evaluation methods across text, vision, and speech generation, concluding that LLM-based evaluators dominate current practice.
-
Immune: Improving Safety Against Jailbreaks in Multi-modal LLMs via Inference-Time Alignment
An inference-time alignment method using a safety reward model and controlled decoding that reduces jailbreak success rates in multimodal LLMs while preserving utility.
-
Interpreting Language Reward Models via Contrastive Explanations
Reward model preferences can be explained by generating counterfactual and semifactual answer variations along 15 hand-picked evaluation attributes and measuring which attribute changes flip the model's preference.
-
Reward Modeling for Reinforcement Learning-Based LLM Reasoning: Design, Challenges, and Evaluation
A taxonomy-driven survey arguing that reward design is the central mechanism shaping reliable LLM reasoning, with maps of reward paradigms, reward-hacking failure modes, and benchmark pitfalls.
-
Efficient MAP Estimation of LLM Judgment Performance with Prior Transfer
BetaConform estimates LLM ensemble judgment accuracy from few labeled samples via a mixture of Beta-Binomial distributions, conformal-style adaptive stopping, and text-similarity prior transfer, but its theoretical gu...
-
Reusing Embeddings: Reproducible Reward Model Research in Large Language Model Alignment without GPUs
Embedding-based reward models reproduce key alignment research findings on CPU-only hardware, lowering cost and improving reproducibility.
Discussion (0). Continue with ORCID to comment.