Pith. sign in

REVIEW 5 cited by

Context-DPO: Aligning Language Models for Context-Faithfulness

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2412.15280 v1 pith:5GYVTVTA submitted 2024-12-18 cs.CL cs.AIcs.IR

Baolong Bi , Shaohan Huang , Yiwei Wang , Tianchi Yang , Zihan Zhang , Haizhen Huang , Lingrui Mei , Junfeng Fang
show 6 more authors
Zehao Li Furu Wei Weiwei Deng Feng Sun Qi Zhang Shenghua Liu
This is my paper
classification cs.CLcs.AIcs.IR
keywords context-dpocontext-faithfulnessllmsalignmentmodelsconfiqacontextlanguage
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
abstract

Reliable responses from large language models (LLMs) require adherence to user instructions and retrieved information. While alignment techniques help LLMs align with human intentions and values, improving context-faithfulness through alignment remains underexplored. To address this, we propose $\textbf{Context-DPO}$, the first alignment method specifically designed to enhance LLMs' context-faithfulness. We introduce $\textbf{ConFiQA}$, a benchmark that simulates Retrieval-Augmented Generation (RAG) scenarios with knowledge conflicts to evaluate context-faithfulness. By leveraging faithful and stubborn responses to questions with provided context from ConFiQA, our Context-DPO aligns LLMs through direct preference optimization. Extensive experiments demonstrate that our Context-DPO significantly improves context-faithfulness, achieving 35% to 280% improvements on popular open-source models. Further analysis demonstrates that Context-DPO preserves LLMs' generative capabilities while providing interpretable insights into context utilization. Our code and data are released at https://github.com/byronBBL/Context-DPO

Discussion (0). Sign in to comment.

Forward citations

Cited by 5 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Beyond Correctness: Rewarding Faithful Reasoning in Retrieval-Augmented Generation

    cs.CL 2025-10 reject novelty 6.0 of 10

    A turn-level faithfulness reward improves a Search-R1-style agent's Information-Think and Think-Answer faithfulness as judged by the same reward model used for training, while task accuracy is roughly unchanged.

  2. FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generation

    cs.CL 2025-06 conditional novelty 5.0 of 10

    FaithfulRAG resolves knowledge conflicts in RAG by extracting the model's parametric facts, aligning them with context, and reasoning through discrepancies before generating an answer.

  3. Who is in the Spotlight: The Hidden Bias Undermining Multimodal Retrieval-Augmented Generation

    cs.CL 2025-05 conditional novelty 4.0 of 10

    Multimodal RAG models show a U-shaped position bias in evidence ordering, and the bias grows logarithmically with the number of retrieved items, according to a new metric.

  4. RACE-Align: Retrieval-Augmented and Chain-of-Thought Enhanced Preference Alignment for Large Language Models

    cs.CL 2025-06 reject novelty 3.0 of 10

    RACE-Align generates preference pairs from RAG-grounded chain-of-thought answers and applies DPO to align a 1.7B model, showing improved reasoning scores in TCM QA but lacking statistical support.

  5. Large Language Models as Computable Approximations to Solomonoff Induction

    cs.LG 2025-05 reject novelty 2.0 of 10

    The paper argues LLMs are computable approximations of Solomonoff induction, but its central derivation recovers the model's own probabilities by construction.

Pith tools