Paper Citation Record · LEDGER

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks

As of 9 August 2026, this Paper Citation Record lists 49 of 49 outbound references and 0 inbound Pith citation observations for arXiv:2607.10240.

A citation records a reference. It does not transfer a finding from one paper to another.

pith.paper-citation-record.v1
2607.10240 v1

Coverage vector

measured 49 of 49 reference resolution

Typed states for the displayed outbound observations.

Source: paper_references, paper_reference_links, observed 2026-07-14T13:17:55.700527Z

measured 49 of 49 standing notices

One-hop event checks from named stored sources.

Source: scholarly_work_events, retraction_status_cache, observed 2026-08-08T06:32:00.761636+00:00

measured 0 of 0 inbound itemization

Pith citing papers itemized under the disclosed page cap.

Source: paper_references, paper_reference_links

measured 0 of 1 external citation measurements

A source-named dated measurement, never combined with another source.

Source: cited_works

Reference resolution

49 of 49 outbound references displayed

verified exact0
verified fuzzy0
unresolved47
parse uncertain0
malformed identifier2
metadata mismatch0

External citation measurements

No source-named external measurement is stored.

Outbound references

Observation 26b102c3-a43b-4ec0-8008-0eedf637e57a · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 1

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2a441827-94cb-4111-8217-e72426d2217a · outbound

This paper cites Lawrence Zitnick, and Devi Parikh.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Lawrence Zitnick, and Devi Parikh

Reference 2

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation a8fe680c-579b-4ba4-9c6c-de4ee052b74d · outbound

This paper cites InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV).

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

Reference 3

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 174b0ce4-7f06-4366-a379-ea746e09bf3c · outbound

This paper cites Qwen2.5-VL Technical Report.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Qwen2.5-VL Technical Report

Reference 4

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 978781ae-ec38-4684-8736-ef0e8b22037e · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 5

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation cead67fd-ee82-42c8-b0e2-61de3fb884f2 · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 6

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1158f281-8989-4d84-8274-fa8c3a7ed59a · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 7

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1dd1c501-ba03-440e-b893-ced9f776338a · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 8

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 42020f16-b031-47c0-929b-3d1dd330a494 · outbound

This paper cites In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

Reference 9

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation af2a2ebd-ed25-44e5-9521-cf5b8abcbb19 · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 10

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5793357f-1a46-499b-baf2-d75f22d36c3f · outbound

This paper cites Gemma 3 Technical Report.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Gemma 3 Technical Report

Reference 11

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation b0f029c4-825f-48b4-a21b-d377385af2f8 · outbound

This paper cites Open-ended VQA benchmarking of Vision-Language models by exploiting Classification datasets and their semantic hierarchy.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Open-ended VQA benchmarking of Vision-Language models by exploiting Classification datasets and their semantic hierarchy

Reference 12

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation bf5de91a-d61c-46bf-b96d-82e8b784efdf · outbound

This paper cites The Llama 3 Herd of Models.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks The Llama 3 Herd of Models

Reference 13

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 1ca64184-b7b6-487a-bd76-4307e054e4e5 · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 14

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 797f9128-dc06-4d91-8294-3e45b0f70872 · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 15

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8f6d4e81-69c7-4d8f-b77f-7f7249b814fc · outbound

This paper cites InFind- ings of the Association for Computational Linguistics: EMNLP.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks InFind- ings of the Association for Computational Linguistics: EMNLP

Reference 16

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 09c494c6-cb8f-44ad-ad6b-52d062b925d3 · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 17

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ea7664fd-f28a-41ba-afc8-d2e8353bafab · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 18

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9f08e2be-a55b-4834-b447-9c027527f1e1 · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 19

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 9819454d-ae5e-4800-b61d-6c7df4bda32f · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 20

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3aecee85-bf54-4213-8cf2-59105151aa42 · outbound

This paper cites Gonzalez, Hao Zhang, and Ion Stoica.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Gonzalez, Hao Zhang, and Ion Stoica

Reference 21

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c5df8434-33d6-46e0-843b-502f1b615588 · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 22

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 05614de5-8417-40e8-bdae-395a180d34b2 · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 23

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation eb0f4217-f738-4f4d-be83-00b99dc2619e · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 24

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation cd3131db-2d22-4da2-821a-034e5b8c8f5a · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 25

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c62d589b-bdb4-43d0-bbee-1b4e4fc9d59a · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 26

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6086f200-4f9d-4d24-a3b4-46876332ce3e · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 27

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 0de7136f-b488-49f6-abbb-cd4c04b32719 · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 28

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6ab9deb0-9a4b-4c98-b7cd-9c8155674bfe · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 29

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 6d1722f6-70d4-4508-b0fd-39718cd21532 · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 30

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3401c0ef-217c-40bb-a986-7f33841017bf · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 31

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 868d5bfd-0ceb-44ce-8c61-866da95c08d4 · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 32

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation d2292abf-2aca-4673-ade5-5e44490ebcb8 · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 33

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 733539bd-7344-42e5-bdfb-3d6a08b28efc · outbound

This paper cites ANLS* -- A Universal Document Processing Metric for Generative Large Language Models.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks ANLS* -- A Universal Document Processing Metric for Generative Large Language Models

Reference 34

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8624e63b-b5ef-40bd-8832-895beea429ba · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 35

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8000e368-7a7b-4285-8c2a-1dd293331a3a · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 36

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2a0b000b-fd5c-40cf-82c9-914acbfca9df · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 37

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2ec11b24-cda5-46bb-9021-32a9bbafc844 · outbound

This paper cites Le, Ed H.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Le, Ed H

Reference 38

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation da78a49b-cfd7-4d62-b0f8-ee25765bdefb · outbound

This paper cites InProceedings of the International Conference on Learning Representations (ICLR).

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks InProceedings of the International Conference on Learning Representations (ICLR)

Reference 39

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 8c253bc3-e8ad-46cf-acb1-07b3173a5ed9 · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 40

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation cb3fcd5e-5456-40ce-8d82-dd65fe8e264d · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 41

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 3d09161f-6065-4973-905d-f8c4b8e4e035 · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 42

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 2ec93f55-ccb3-43d6-90dd-c4e8e7199fd0 · outbound

This paper cites Weld, and Jonathan Berant.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Weld, and Jonathan Berant

Reference 43

Resolution: malformed identifier; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 190bb2dc-22b6-4241-bf9e-166d1a6817ea · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 44

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 178c6799-d934-45f7-a11f-291314743a04 · outbound

This paper cites Are Large Language Models Suitable for Graph Computation? Progress and Prospects.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Are Large Language Models Suitable for Graph Computation? Progress and Prospects

Reference 45

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation ee4cec9d-a0bb-4d81-978f-b47215c88469 · outbound

This paper cites Xing, et al.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Xing, et al

Reference 46

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 73a301a1-6b82-4671-9999-27aa813222c7 · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 47

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation 5bed46e7-fdf3-4f39-a509-fc3466cd52fc · outbound

This paper cites an unresolved cited work.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks Unresolved cited work

Reference 48

Resolution: unresolved; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Observation c177fff9-953b-4a7a-a737-af2d8a944b0a · outbound

This paper cites InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models.

What Does Your Short-Answer VQA Score Actually Measure? Evaluator-Dependent Instability in Multimodal Short-Answer Benchmarks InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models

Reference 49

Resolution: malformed identifier; no resolver link, observed 2026-07-14T13:17:55.700527Z

Source-reported events for the cited work

Unavailable: canonical work link unavailable.

Pith citing papers

No inbound Pith citation observations are available.