A new automated benchmark of 2D physics questions finds vision-language models are stronger on formulaic tasks than on spatial reasoning, and parameter count does not fully explain performance.
A Comprehensive Survey on Multi-hop Machine Reading Comprehension Datasets and Metrics
1 Pith paper cite this work. Polarity classification is still indexing.
1
Pith paper citing it
abstract
Multi-hop Machine reading comprehension is a challenging task with aim of answering a question based on disjoint pieces of information across the different passages. The evaluation metrics and datasets are a vital part of multi-hop MRC because it is not possible to train and evaluate models without them, also, the proposed challenges by datasets often are an important motivation for improving the existing models. Due to increasing attention to this field, it is necessary and worth reviewing them in detail. This study aims to present a comprehensive survey on recent advances in multi-hop MRC evaluation metrics and datasets. In this regard, first, the multi-hop MRC problem definition will be presented, then the evaluation metrics based on their multi-hop aspect will be investigated. Also, 15 multi-hop datasets have been reviewed in detail from 2017 to 2022, and a comprehensive analysis has been prepared at the end. Finally, open issues in this field have been discussed.
citation-role summary
background 1
citation-polarity summary
fields
cs.LG 1years
2025 1verdicts
REJECT 1roles
background 1polarities
unclear 1representative citing papers
citing papers explorer
-
Interpretable Physics Reasoning and Performance Taxonomy in Vision-Language Models
A new automated benchmark of 2D physics questions finds vision-language models are stronger on formulaic tasks than on spatial reasoning, and parameter count does not fully explain performance.