HAMLET introduces a three-level key-fact hierarchy with automated query-focused summarization to evaluate LLM recall and faithfulness in book-length contexts, achieving over 90% agreement with experts.
1-2 / 1" range • 3 points= A = YES but exactly one of B-D in the
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.CL 1years
2025 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
Towards a Holistic and Automated Evaluation Framework for Multi-Level Comprehension of LLMs in Book-Length Contexts
HAMLET introduces a three-level key-fact hierarchy with automated query-focused summarization to evaluate LLM recall and faithfulness in book-length contexts, achieving over 90% agreement with experts.