Pith. sign in

REVIEW 4 cited by

QuALITY: Question Answering with Long Input Texts, Yes!

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2112.08608 v2 pith:PT3W64FS submitted 2021-12-16 cs.CL

Richard Yuanzhe Pang , Alicia Parrish , Nitish Joshi , Nikita Nangia , Jason Phang , Angelica Chen , Vishakh Padmakumar , Johnny Ma
show 3 more authors
Jana Thompson He He Samuel R. Bowman
This is my paper
classification cs.CL
keywords modelspassagesperformqualityquestionsadditionannotatorsanswerable
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

To enable building and testing models on long-document comprehension, we introduce QuALITY, a multiple-choice QA dataset with context passages in English that have an average length of about 5,000 tokens, much longer than typical current models can process. Unlike in prior work with passages, our questions are written and validated by contributors who have read the entire passage, rather than relying on summaries or excerpts. In addition, only half of the questions are answerable by annotators working under tight time constraints, indicating that skimming and simple search are not enough to consistently perform well. Our baseline models perform poorly on this task (55.4%) and significantly lag behind human performance (93.5%).

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. OpenAlex reports about 10 citations worldwide. Full citation record

  1. EraRAG: Efficient and Incremental Retrieval Augmented Generation for Growing Corpora

    cs.IR 2025-06 conditional novelty 6.0 of 10

    EraRAG uses hyperplane-based locality-sensitive hashing to build a hierarchical retrieval graph whose affected regions only are re-summarized when new documents arrive, cutting update cost by up to an order of magnitude.

  2. HoPE: Hyperbolic Rotary Positional Encoding for Stable Long-Range Dependency Modeling in Large Language Models

    cs.CL 2025-09 reject novelty 4.0 of 10

    HoPE replaces RoPE's sine/cosine rotations with hyperbolic functions plus an exponential damping term to enforce monotonic attention decay, but the claimed consistent superiority and the 'RoPE as special case' theorem...

  3. A Comprehensive Graph Framework for Question Answering with Mode-Seeking Preference Alignment

    cs.CL 2025-06 conditional novelty 4.0 of 10

    GraphMPA combines an embedding-similarity hierarchical graph with mode-seeking preference optimization to improve RAG question answering on six datasets.

  4. Evolutionary Perspectives on the Evaluation of LLM-Based AI Agents: A Comprehensive Survey

    cs.CL 2025-06 conditional novelty 4.0 of 10

    A survey that classifies AI agent evaluation benchmarks along environment and capability axes, and proposes five traits that distinguish agents from chatbots.

Pith tools