Prompting LLMs with test-taking strategies for true/false factuality checks reduces tokens by over 80%, matches strong baselines on two benchmarks with SOTA on one, and enables fine-tuned SLMs to perform similarly at low cost with rationales.
Fangrui Lv, Kaixiong Gong, Jian Liang, Xinyu Pang, and Changshui Zhang
10 Pith papers cite this work. Polarity classification is still indexing.
10
Pith papers citing it
fields
cs.CL 10representative citing papers
This systematic survey organizes prompt engineering into a taxonomy of 58 LLM techniques and 40 others, supplies a shared vocabulary, and offers guidelines for state-of-the-art models.
Authors propose a new framework for automated LLM creativity evaluation that separates measurement from the task, using semantic entropy and multi-agent judges, validated on problem-solving, research ideation, and creative writing domains.
Aggregating imperfect factuality metrics into preference data from lexically similar summaries yields consistent factuality gains across model sizes, allowing smaller models to approach larger ones.
LLM-ReSum uses LLM self-evaluation in a closed feedback loop to refine summaries, improving factual accuracy by up to 33% and coverage by 39% with 89% human preference.
A proposed pipeline shows LLMs introduce detectable race and gender biases when summarizing life narratives, creating potential for representational harm in research.
The method aggregates multiple hallucination evaluation scores via conformal p-values to enable calibrated detection with controlled false alarm rates across LLMs and datasets.
LaMSUM is a novel multi-level LLM framework with voting methods for extractive summarization of large incident report collections that outperforms prior extractive methods.
SelfCheckGPT detects hallucinations by checking consistency across multiple sampled responses from black-box LLMs on WikiBio biography generation tasks.
Authors create LLM-Fake Theory integrating social psychology, then use a prompt engineering pipeline to build the MegaFake dataset of LLM-generated fake news for advancing detection methods.
citing papers explorer
-
Teaching Language Models to Check Grounded Claim Factuality with Human Test-Taking Strategies
Prompting LLMs with test-taking strategies for true/false factuality checks reduces tokens by over 80%, matches strong baselines on two benchmarks with SOTA on one, and enables fine-tuned SLMs to perform similarly at low cost with rationales.
-
The Prompt Report: A Systematic Survey of Prompt Engineering Techniques
This systematic survey organizes prompt engineering into a taxonomy of 58 LLM techniques and 40 others, supplies a shared vocabulary, and offers guidelines for state-of-the-art models.
-
Automated Creativity Evaluation of Language Models Across Open-Ended Tasks
Authors propose a new framework for automated LLM creativity evaluation that separates measurement from the task, using semantic entropy and multi-agent judges, validated on problem-solving, research ideation, and creative writing domains.
-
Optimising Factual Consistency in Summarisation via Preference Learning from Multiple Imperfect Metrics
Aggregating imperfect factuality metrics into preference data from lexically similar summaries yields consistent factuality gains across model sizes, allowing smaller models to approach larger ones.
-
LLM-ReSum: A Framework for LLM Reflective Summarization through Self-Evaluation
LLM-ReSum uses LLM self-evaluation in a closed feedback loop to refine summaries, improving factual accuracy by up to 33% and coverage by 39% with 89% human preference.
-
Whose Story Gets Told? Positionality and Bias in LLM Summaries of Life Narratives
A proposed pipeline shows LLMs introduce detectable race and gender biases when summarizing life narratives, creating potential for representational harm in research.
-
Principled Detection of Hallucinations in Large Language Models via Multiple Testing
The method aggregates multiple hallucination evaluation scores via conformal p-values to enable calibrated detection with controlled false alarm rates across LLMs and datasets.
-
LaMSUM: Amplifying Voices Against Harassment through LLM Guided Extractive Summarization of User Incident Reports
LaMSUM is a novel multi-level LLM framework with voting methods for extractive summarization of large incident report collections that outperforms prior extractive methods.
-
SelfCheckGPT: Zero-Resource Black-Box Hallucination Detection for Generative Large Language Models
SelfCheckGPT detects hallucinations by checking consistency across multiple sampled responses from black-box LLMs on WikiBio biography generation tasks.
-
MegaFake: A Theory-Driven Dataset of Fake News Generated by Large Language Models
Authors create LLM-Fake Theory integrating social psychology, then use a prompt engineering pipeline to build the MegaFake dataset of LLM-generated fake news for advancing detection methods.