REVIEW 22 cited by
MuSiQue: Multihop Questions via Single-hop Question Composition
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
abstract
Multihop reasoning remains an elusive goal as existing multihop benchmarks are known to be largely solvable via shortcuts. Can we create a question answering (QA) dataset that, by construction, \emph{requires} proper multihop reasoning? To this end, we introduce a bottom-up approach that systematically selects composable pairs of single-hop questions that are connected, i.e., where one reasoning step critically relies on information from another. This bottom-up methodology lets us explore a vast space of questions and add stringent filters as well as other mechanisms targeting connected reasoning. It provides fine-grained control over the construction process and the properties of the resulting $k$-hop questions. We use this methodology to create MuSiQue-Ans, a new multihop QA dataset with 25K 2-4 hop questions. Relative to existing datasets, MuSiQue-Ans is more difficult overall (3x increase in human-machine gap), and harder to cheat via disconnected reasoning (e.g., a single-hop model has a 30 point drop in F1). We further add unanswerable contrast questions to produce a more stringent dataset, MuSiQue-Full. We hope our datasets will help the NLP community develop models that perform genuine multihop reasoning.
Forward citations
Cited by 22 Pith papers
-
Do Large Language Models Perform Latent Multi-Hop Reasoning without Exploiting Shortcuts?
A shortcut-filtered benchmark shows LLMs genuinely compose facts internally for country-bridge queries (over 80% for the best models) but almost never for year-bridge queries (about 5-6%).
-
MemOPD: On-Policy Distillation through Memory State Alignment for Long-Horizon Agents
Reconstructing the exact model invocation state for each sampled action, instead of flattening the whole interaction, lets teacher distillation guide compact-memory agents without state mismatch.
-
HopRefusalBench: Diagnosing Refusal Failures in Search-Augmented Agents for Multi-Hop Reasoning
A new benchmark shows search-augmented LLMs correctly refuse only up to 42.9% of unanswerable multi-hop questions, with failures split between hallucinated answers and search exhaustion.
-
EviSD: Evidence-Conditioned Self-Distillation for Search-Augmented Agents
EviSD improves search-agent RL by re-scoring each sampled action under a context containing supporting evidence or the golden answer, and using that detached gap to modulate GRPO credit only on action tokens.
-
Fishing Out Free Riders: Shapley-Based Reward Attribution for Parallel Reasoning via Reinforcement Learning
Rewarding each parallel reasoning path by Monte-Carlo-Shapley marginal contribution, scored by a generative reward model, lifts Pass@16 on AIME24/AIME25/AMC23 by 4-90% relative over Parallel-R1 with a fifth of the tra...
-
Tool-Adaptive LLM Reranker
Pointwise LLM reranking cast as an agentic MDP with language-preserving warm-up and asymmetric cost-aware RL yields adaptive tool use, SOTA NDCG, and high throughput.
-
Rethinking Uncertainty Evaluation in Large Language Models
A three-axis 'C1' evaluation claims to show that well-calibrated LLM confidence estimates still violate probability axioms such as entailment monotonicity 31% of the time.
-
Shapley Context Pruning: A Cooperative Game Perspective for Context Reranking and Pruning
SCP ranks and prunes context sentences using Shapley values from a learned Deep Sets value function, matching or beating baselines on several multi-hop QA datasets at 50% compression.
-
EvolKV: Evolutionary KV Cache Compression for LLM Inference
CMA-ES search over per-layer KV cache budgets beats uniform and pyramidal compression heuristics on LongBench, NIAH, RULER, and GSM8K, and edges past the full cache on one code dataset at 1.5% of the budget.
-
Hybrid Deep Searcher: Scalable Parallel and Sequential Search Reasoning
Fine-tuning Qwen3-8B on a synthetic hybrid-hop dataset with parallel and sequential search trajectory supervision yields a search agent that outperforms several agentic RAG baselines on five QA benchmarks.
-
SSRL: Self-Search Reinforcement Learning
SSRL, a training pipeline that uses an LLM's own repeated sampling as a search environment for RL, improves question answering without external tools and transfers to real search engines.
-
RAVine: Reality-Aligned Evaluation for Agentic Search
RAVine is an attributable nugget-based benchmark with process metrics that shows current agentic search models have low citation recall and rely heavily on internal knowledge.
-
HIRAG: Hierarchical-Thought Instruction-Tuning Retrieval-Augmented Generation
A hierarchical chain-of-thought instruction-tuning curriculum for filtering, combination, and reasoning improves zero-shot retrieval-augmented QA.
-
MDBench: A Synthetic Multi-Document Reasoning Benchmark Generated with Knowledge Guidance
MDBench is a synthetically generated, knowledge-guided benchmark for multi-document QA on which frontier LLMs achieve only about 60% exact match.
-
NovelHopQA: Diagnosing Multi-Hop Reasoning Failures in Long Narrative Contexts
NovelHopQA is a new benchmark that pairs long novel excerpts with 1-4 hop questions and shows LLM accuracy drops consistently with both context length and reasoning depth.
-
SiReRAG: Indexing Similar and Related Information for Multihop Reasoning
SiReRAG indexes a corpus with both a similarity tree and an entity-based relatedness tree, improving average multihop QA F1 by about 1.9 points over prior RAG indexing methods.
-
Agent-as-Tool: A Study on the Hierarchical Decision Making with Reinforcement Learning
A two-level agent that separates planning from tool execution, fine-tuned with GRPO on 180 samples, beats a modified Search-R1 on the Bamboogle QA benchmark.
-
FaithfulRAG: Fact-Level Conflict Modeling for Context-Faithful Retrieval-Augmented Generation
FaithfulRAG resolves knowledge conflicts in RAG by extracting the model's parametric facts, aligning them with context, and reasoning through discrepancies before generating an answer.
-
WebDancer: Towards Autonomous Information Seeking Agency
A four-stage data-and-training pipeline (synthetic QA, trajectory rejection sampling, SFT, DAPO reinforcement learning) produces WebDancer, a web agent that outperforms vanilla ReAct and several open-source agentic fr...
-
Even Small Reasoners Should Quote Their Sources: Introducing the Pleias-RAG Model Family
Two mid-trained small language models (350M and 1B) are claimed to achieve state-of-the-art RAG accuracy in their size class while generating native literal-quote citations.
-
Reducing Hallucinations in Complex Question Answering using Simple Graph-based Retrieval-Augmented Generation (long version)
A graph-augmented RAG system with vector and graph query tools halves hallucinations and raises factual correctness scores on the MoNaCo complex QA benchmark.
-
Jan-nano Technical Report
A 4B model trained with reinforcement learning and no standard next-token fine-tuning reportedly reaches 83.2% on SimpleQA when given web-search tools, beating the 78.2% reported for DeepSeek-671B with the same tool setup.
Discussion (0). Continue with ORCID to comment.