REVIEW 2 cited by
Towards a Framework for Evaluating Explanations in Automated Fact Verification
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
read the original abstract
As deep neural models in NLP become more complex, and as a consequence opaque, the necessity to interpret them becomes greater. A burgeoning interest has emerged in rationalizing explanations to provide short and coherent justifications for predictions. In this position paper, we advocate for a formal framework for key concepts and properties about rationalizing explanations to support their evaluation systematically. We also outline one such formal framework, tailored to rationalizing explanations of increasingly complex structures, from free-form explanations to deductive explanations, to argumentative explanations (with the richest structure). Focusing on the automated fact verification task, we provide illustrations of the use and usefulness of our formalization for evaluating explanations, tailored to their varying structures.
Forward citations
Cited by 2 Pith papers
-
Retrieval Augmented Decision-Making: A Requirements-Driven, Multi-Criteria Framework for Structured Decision Support
RAD automatically extracts weighted, hierarchical decision criteria from documents and uses LLMs to generate structured decision reports, but its evaluation is largely self-referential.
-
Towards Automated Fact-Checking of Real-World Claims: Exploring Task Formulation and Assessment with LLMs
In a benchmark of 17,856 PolitiFact claims, larger Llama-3 models and retrieved web evidence improve automated fact-checking accuracy and justification quality, though fine-grained labels remain difficult.
Discussion (0). Continue with ORCID to comment.