Pith. sign in

REVIEW 8 cited by

CodeUltraFeedback: An LLM-as-a-Judge Dataset for Aligning Large Language Models to Coding Preferences

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2403.09032 v3 pith:EZVLPNDG submitted 2024-03-14 cs.SE cs.CLcs.LG

Martin Weyssow , Aton Kamanda , Xin Zhou , Houari Sahraoui This is my paper · ORCID
classification cs.SEcs.CLcs.LG
keywords llmsalignmentcodingcodeultrafeedbackpreferencesfeedbackmodelmodels
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Evaluating the alignment of large language models (LLMs) with user-defined coding preferences is a challenging endeavour that requires a deep assessment of LLMs' outputs. Existing methods and benchmarks rely primarily on automated metrics and static analysis tools, which often fail to capture the nuances of user instructions and LLM outputs. To address this gap, we propose using the LLM-as-a-Judge methodology to evaluate the alignment of LLMs with coding preferences. Based on this approach, we present CodeUltraFeedback, a comprehensive dataset designed to facilitate the evaluation and improvement of LLM alignment. CodeUltraFeedback consists of 10,000 coding instructions, each annotated with four responses generated from a diverse pool of 14 LLMs. These responses are ranked based on five distinct coding preferences using GPT-3.5 as a judge, providing both numerical scores and detailed textual feedback. Our analysis of CodeUltraFeedback reveals that responses from GPT-3.5 and GPT-4 are generally preferred over those from open-weight LLMs, highlighting significant differences in alignment between closed and open-weight models. In turn, we explore the usage of CodeUltraFeedback as feedback data to fine-tune and align CodeLlama-7B-Instruct using supervised fine-tuning (SFT) and reinforcement learning from AI feedback (RLAIF) with direct preference optimization (DPO). The resulting aligned CodeLlama-7B-Instruct model outperforms larger LLMs in terms of alignment with coding preferences and shows improved functional correctness on the HumanEval+ benchmark compared to the original instruct model. Therefore, our contributions bridge the gap in preference tuning of LLMs for code and set the stage for further advancements in model alignment and RLAIF in automated software engineering.

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 8 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. How Developers Experience Debugging Unfamiliar Codebases with Code Tours Generated and Evaluated by Local LLMs

    cs.SE 2026-07 conditional novelty 6.0 of 10

    Developers prefer scannable, length-scaled, guiding LLM code tours for unfamiliar Java bugs, distrust AI-looking text, and find open-weight LLM judges unreliable due to sycophancy and confabulation.

  2. Balancing Usefulness and Naturalness: An LLM-based Curation Pipeline for Code Review Comments

    cs.SE 2026-07 conditional novelty 6.0 of 10

    Selective LLM reformulation guided by high-quality exemplars yields cleaner, more diverse code-review datasets that improve downstream comment generation and code refinement.

  3. VIS-Shepherd: Constructing Critic for LLM-based Data Visualization Generation

    cs.CV 2025-06 conditional novelty 6.0 of 10

    A 7-billion-parameter multimodal model fine-tuned on 2,500 expert critiques of data visualizations matches or beats much larger models at identifying visualization defects.

  4. Leveraging Reward Models for Guiding Code Review Comment Generation

    cs.SE 2025-06 conditional novelty 6.0 of 10

    CoRAL fine-tunes a language model with reinforcement learning using reward models derived from comment semantics and code refinement success, and reports improved code review comment quality.

  5. A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data

    cs.AI 2026-01 conditional novelty 5.0 of 10

    A metric-oriented survey that classifies intrinsic quality and trustworthiness metrics for LLM-generated data across six modalities and documents systematic evaluation gaps in the current literature.

  6. Towards Reliable Generative AI-Driven Scaffolding: Reducing Hallucinations and Enhancing Quality in Self-Regulated Learning Support

    cs.CY 2025-08 unverdicted novelty 5.0 of 10

    A multi-agent reliability check and an LLM-as-a-judge quality check both reduce hallucinations in AI-generated study scaffolds, with the multi-agent check matching human expert judgments almost perfectly.

  7. AutoMixAlign: Adaptive Data Mixing for Multi-Task Preference Optimization in LLMs

    cs.LG 2025-05 conditional novelty 5.0 of 10

    AutoMixAlign adaptively reweights or resamples task data during DPO training to match specialist-model losses, improving average performance on helpfulness, coding, and safety benchmarks compared to standard DPO and m...

  8. Combining Large Language Models with Static Analyzers for Code Review Generation

    cs.SE 2025-02 conditional novelty 5.0 of 10

    Injecting static analyzer output into LLM prompts (RAG) improves code review accuracy and coverage over the LLM alone, while data-augmented training improves coverage only.

Pith tools