A new Russian poetry dataset and benchmark show that supervised classifiers trained on synthetic distortions detect grammatical defects (F0.5=0.86) while unsupervised and zero-shot methods perform near random.
Pre-trained Language Models Return Distinguishable Probability Distributions to Unfaithfully Hallucinated Texts
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
In this work, we show the pre-trained language models return distinguishable generation probability and uncertainty distribution to unfaithfully hallucinated texts, regardless of their size and structure. By examining 24 models on 6 data sets, we find out that 88-98% of cases return statistically significantly distinguishable generation probability and uncertainty distributions. Using this general phenomenon, we showcase a hallucination-reducing training algorithm. Our algorithm outperforms other baselines by achieving higher faithfulness metrics while maintaining sound general text quality measures.
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Detecting Spelling and Grammatical Anomalies in Russian Poetry Texts
A new Russian poetry dataset and benchmark show that supervised classifiers trained on synthetic distortions detect grammatical defects (F0.5=0.86) while unsupervised and zero-shot methods perform near random.