REVIEW 13 cited by
Alignment for Honesty
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Recent research has made significant strides in aligning large language models (LLMs) with helpfulness and harmlessness. In this paper, we argue for the importance of alignment for \emph{honesty}, ensuring that LLMs proactively refuse to answer questions when they lack knowledge, while still not being overly conservative. However, a pivotal aspect of alignment for honesty involves discerning an LLM's knowledge boundaries, which demands comprehensive solutions in terms of metric development, benchmark creation, and training methodologies. We address these challenges by first establishing a precise problem definition and defining ``honesty'' inspired by the Analects of Confucius. This serves as a cornerstone for developing metrics that effectively measure an LLM's honesty by quantifying its progress post-alignment. Furthermore, we introduce a flexible training framework which is further instantiated by several efficient fine-tuning techniques that emphasize honesty without sacrificing performance on other tasks. Our extensive experiments reveal that these aligned models show a marked increase in honesty, as indicated by our proposed metrics. We open-source all relevant resources to facilitate future research at \url{https://github.com/GAIR-NLP/alignment-for-honesty}.
Forward citations
Cited by 13 Pith papers
-
Prompt Compression via Activation Aggregation
A learned weighted sum of intermediate-layer activations compresses an instruction prompt into a single patch vector that, injected at an early layer, recovers task accuracy within ~2% of the full prompt.
-
The Curious Case of Factuality Finetuning: Models' Internal Beliefs Can Improve Factuality
Fine-tuning on model-generated text filtered by internal probes improves factual accuracy more than fine-tuning on gold documents across three long-form generation domains.
-
Can Video LLMs Refuse to Answer? Alignment for Answerability in Video Large Language Models
Video-LLMs can be trained, via SFT or DPO on a new synthetic dataset UVQA, to refuse questions that cannot be answered from the video content, with modest cost to answerable QA performance.
-
CausalAbstain: Enhancing Multilingual LLMs with Causal Reasoning for Trustworthy Abstention
CausalAbstain filters multilingual self-feedback by comparing how much it changes the model's abstention decision, improving abstention accuracy over baselines on two benchmarks.
-
Writing Like the Best: Exemplar-Based Expository Text Generation
Exemplar-based expository generation: a recurrent plan-then-adapt LLM framework converts a source topic's exemplar into target-topic text by question transfer, retrieval, and confidence-gated answering.
-
Structural Entropy Guided Agent for Detecting and Repairing Knowledge Deficiencies in LLMs
SENATOR guides a language model through a knowledge graph, measures its uncertainty with structural entropy, and fine-tunes it on synthetic data chosen to fix its weak spots, gaining up to 12 percent average relative ...
-
GRAIT: Gradient-Driven Refusal-Aware Instruction Tuning for Effective Hallucination Mitigation
GRAIT selects and reweights refusal-training examples using gradient influence, reporting lower hallucination rates and better helpfulness scores than prior refusal-aware tuning baselines.
-
UAlign: Leveraging Uncertainty Estimations for Factuality Alignment on Large Language Models
UAlign improves LLM factuality alignment by adding predicted confidence and semantic entropy as input features to prompts and the reward model, helping the model answer known questions and refuse unknown ones.
-
Improve Decoding Factuality by Token-wise Cross Layer Entropy of Large Language Models
A new decoding-time method, END, uses per-token cross-layer entropy of prediction growth to boost factual tokens, improving truthfulness and informativeness on hallucination benchmarks.
-
Aligning Large Language Models for Faithful Integrity Against Opposing Argument
An LLM is fine-tuned with DPO to make the strength of its stance in conversation match its self-estimated confidence, improving resistance to misleading arguments and receptiveness to corrections.
-
How Knowledge Popularity Influences and Enhances LLM Knowledge Boundary Perception
Entity popularity and entity co-occurrence in Wikipedia correlate with LLM QA accuracy, confidence, and calibration, and combining them with confidence improves answer-correctness prediction by 5.24% on average.
-
A Survey on Uncertainty Quantification of Large Language Models: Taxonomy, Open Research Challenges, and Future Directions
A review that organizes LLM uncertainty quantification into token-level, self-verbalized, semantic-similarity, and mechanistic interpretability categories.
-
Building Task Bots with Self-learning for Enhanced Adaptability, Extensibility, and Factuality
A thesis that combines self-learning from dialog logs, schema-guided prompting, and self-aligned factuality to build task bots with minimal human intervention.
Discussion (0). Continue with ORCID to comment.