TriMix dynamically fuses logits from three model sources to outperform baselines and Proxy Tuning on eight low-resource languages across four model families.
Contrastive decoding improves reasoning in large language models
6 Pith papers cite this work. Polarity classification is still indexing.
6
Pith papers citing it
citation-role summary
background 2
citation-polarity summary
fields
cs.CL 6roles
background 2polarities
background 2representative citing papers
HSPD detoxifies pretraining corpora via hierarchical semantic-preserving rewriting with Soft Contrastive Decoding, cutting toxicity probability from 0.42 to 0.18 and expected maximum toxicity from 0.43 to 0.20 on GPT2-XL with consistent gains on other models.
LightReasoner distills supervision signals from SLM-LLM behavioral divergence to improve LLM reasoning on math benchmarks with up to 28.1% accuracy gains and 90-99% reductions in resources.
DoLa reduces hallucinations in LLMs by contrasting logits from later versus earlier layers during decoding, improving truthfulness on TruthfulQA by 12-17 absolute points without fine-tuning or retrieval.
DCRD uses attention-map analysis to detect context-memory conflicts in LLMs and conditionally applies either greedy or fidelity-based dynamic decoding, achieving SOTA results on QA tasks across four models and six datasets.
The paper surveys hallucination in LLMs with an innovative taxonomy, factors, detection methods, benchmarks, mitigation strategies, and open research directions.
citing papers explorer
-
Efficient Low-Resource Language Adaptation via Multi-Source Dynamic Logit Fusion
TriMix dynamically fuses logits from three model sources to outperform baselines and Proxy Tuning on eight low-resource languages across four model families.
-
Detoxification for LLM: From Dataset Itself
HSPD detoxifies pretraining corpora via hierarchical semantic-preserving rewriting with Soft Contrastive Decoding, cutting toxicity probability from 0.42 to 0.18 and expected maximum toxicity from 0.43 to 0.20 on GPT2-XL with consistent gains on other models.
-
LightReasoner: Can Small Language Models Teach Large Language Models Reasoning?
LightReasoner distills supervision signals from SLM-LLM behavioral divergence to improve LLM reasoning on math benchmarks with up to 28.1% accuracy gains and 90-99% reductions in resources.
-
DoLa: Decoding by Contrasting Layers Improves Factuality in Large Language Models
DoLa reduces hallucinations in LLMs by contrasting logits from later versus earlier layers during decoding, improving truthfulness on TruthfulQA by 12-17 absolute points without fine-tuning or retrieval.
-
Mitigating Context-Memory Conflicts in LLMs through Dynamic Cognitive Reconciliation Decoding
DCRD uses attention-map analysis to detect context-memory conflicts in LLMs and conditionally applies either greedy or fidelity-based dynamic decoding, achieving SOTA results on QA tasks across four models and six datasets.
-
A Survey on Hallucination in Large Language Models: Principles, Taxonomy, Challenges, and Open Questions
The paper surveys hallucination in LLMs with an innovative taxonomy, factors, detection methods, benchmarks, mitigation strategies, and open research directions.