VAKRA is a benchmark combining executable database-backed APIs, document retrieval, and tool-use policy constraints to evaluate multi-hop reasoning in AI agents, and it finds large performance drops on compositional and policy-constrained tasks.
SEL and SLOT) already leads to a relatively more difficult task the queries used for this split of the dataset could be overlapping with other settings
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
fields
cs.AI 1years
2026 1verdicts
CONDITIONAL 1representative citing papers
citing papers explorer
-
VAKRA: Evaluating Multi-Hop Reasoning Across APIs and Retrieval Under Tool-Use Policies
VAKRA is a benchmark combining executable database-backed APIs, document retrieval, and tool-use policy constraints to evaluate multi-hop reasoning in AI agents, and it finds large performance drops on compositional and policy-constrained tasks.