Pith. sign in

REVIEW 4 major objections 6 minor 45 references

Scientific figures must be judged with full manuscript context; a staged evidence-fusion agent matches expert scores better than single-pass vision-language judges.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-30 11:34 UTC pith:YAYDXE2W

load-bearing objection Useful full-manuscript figure-quality bench and staged judge; the headline MAE wins sit inside human label noise, especially on CC. the 4 major comments →

arxiv 2607.27084 v1 pith:YAYDXE2W submitted 2026-07-29 cs.CV cs.AI

SciFigQual-Bench: A Benchmark for Scientific Figure Quality Assessment with Full-Manuscript Context

Zihan Deng , Chuanzhi Xu , Huiqi Liang , Haoyang Li , Xiaozhen Zhong , Lequan Yu This is my paper
classification cs.CV cs.AI
keywords scientific figure qualityimage quality assessmentmultimodal evaluationvision-language modelsmanuscript contextcaption consistencymisleading visualizationsbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper argues that figures in scientific papers cannot be quality-checked the way natural photos or isolated charts are. What the caption says, what the body text claims when it cites the figure, and whether the visual design could mislead a reader matter as much as sharpness and layout. The authors release SciFigQual-Bench: real figures from top computer-science conferences, each bound to its caption and citing paragraphs, and scored by domain experts on five dimensions. They then show that SFQ-Agent—a staged process that gathers visual evidence and language evidence separately before fusing them—agrees more closely with those expert scores than asking a model to grade everything in one pass, even when that one pass is given OCR side features. A sympathetic reader cares because peer review and automated paper tools still lack a reproducible way to check whether a published figure actually supports the claims written around it.

Core claim

On the fixed public split eval1200 of 1,200 human-rated figures, SFQ-Agent with matched strong vision and language backends reaches the closest agreement with expert overall scores among the compared setups—mean absolute error 0.418 and 93.4% of scores within one point—consistently beating both direct single-pass vision-language judging and single-pass judging augmented with OCR and classical visual features under identical inputs and the same evidence gating.

What carries the argument

SFQ-Agent: a staged cross-modal judge that gates missing caption or citing text, locks visual clarity and structure/layout from vision-only evidence, extracts caption and context facts from text without pixels, fuses the two tracks only for caption consistency, context consistency, and misleading risk, then applies a deterministic runner that owns scores, applies caps, and keeps each dimension traceable to its evidence source.

Load-bearing premise

The ranking of judge protocols rests on expert gold scores being stable enough—especially on caption consistency, where human raters already disagree a lot—to support claims of outperformance from small error gaps.

What would settle it

Re-annotate eval1200 with independent experts under the same rubric; if the new gold means shift enough on caption and context consistency that protocol order reverses, or if a single-pass judge matches or beats SFQ-Agent on the frozen split under identical L1 gating, the central outperformance claim fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Automated review tools can score figure–text alignment with auditable evidence tracks rather than one opaque end-to-end grade.
  • Caption–figure mismatch and body-text conflict become measurable review dimensions, not only informal reviewer comments.
  • Continuous multi-dimensional quality rubrics on real published PDFs complement chart-QA and generation-faithfulness benchmarks.
  • How evidence is staged is a first-class design choice: protocol changes move error about as much as swapping backbone models.
  • Authors and venues get a fixed public testbed for checking misleading visual design against manuscript claims.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Wide human disagreement on caption consistency suggests that axis may need tighter anchors or heavier multi-rater aggregation before small model gaps on it can be read as decisive.
  • The same figure–caption–citation binding could test whether paper-assistant systems actually ground claims in the figures they point to, not only whether they answer chart questions.
  • If the caption and context bottlenecks hold outside top CS venues, full-manuscript figure scoring would be a general scientific-publishing problem rather than a conference-plot specialty.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 6 minor

Summary. The paper introduces SciFigQual-Bench, a full-manuscript-context benchmark for assessing published CS scientific figures on five dimensions (VC, SL, CC, CTX, MR), with each figure bound to its caption and index-resolved citing paragraphs. From ~63k PDFs across ACL/EMNLP/ICML/NeurIPS (2020–2025) the authors release 7,609 curated figures and 6,308 expert-aggregated gold labels, plus a fixed test split eval1200 (n=1,200). They further propose SFQ-Agent, a staged cross-modal judge (vision evidence, language evidence, fusion, deterministic Runner with L1 gating) and compare it against Direct and Sidecar VLM protocols across many backends. The headline claim is that SFQ-Agent with GPT-5.6-Sol achieves the best overall agreement with human gold on eval1200 (MAE 0.418, Within-1 93.4%, SRCC 0.598), monotonically outperforming Direct and Sidecar under matched inputs.

Significance. If the results hold under fully measured runs and a noise-aware analysis, this is a useful contribution to scientific multimodal evaluation. Prior IQA, chart-QA, and generation-faithfulness resources largely score isolated crops or discrete VQA accuracy; binding real conference figures to caption and citing text, with an explicit multi-dimensional review-style rubric and L1 evidence gating, fills a clear gap. The protocol ladder (Direct vs Sidecar vs staged Agent) is a clean experimental design that treats judge architecture as a first-class variable, and the promised JSONL release, fixed eval1200 split, and deterministic Runner (Algorithm 1) support reproducibility. Credit is due for scale, index-driven context binding (~355k citing records), and the auditable separation of visual vs textual evidence rather than a monolithic LMM-as-judge prompt.

major comments (4)
  1. [§4.2–4.3; Appendix D.3 / Table 4] Appendix D.3 / Appendix Table 4 show substantial human disagreement on the gold standard that Table 2 is scored against: CC mean rater spread 2.65 with only 52.5% of rater pairs within ±1; overall per-rater mean spread 1.26 and W-1 pair rate 71.5% (three raters, n=1,200). Reported protocol MAE gaps are ~0.03–0.09 (e.g., best Direct 0.443 vs Agent F3 0.418). §4 and the appendix report neither human–human MAE, ICC/Krippendorff α on continuous scores, bootstrap CIs on ΔMAE, nor a noise-ceiling comparison. Without these, the central “SFQ-Agent consistently outperforms Direct/Sidecar” claim (§4.2–4.3, Abstract) is only weakly identified—especially on CC/CTX, the axes the staged design is meant to fix. Please add reliability statistics and show that protocol rankings survive label noise (e.g., bootstrap over raters or figures; comparison to inter-rater MAE).
  2. [Table 2; Appendix B.3] Appendix B.3 states that many Table 2 rows are projected from observed Direct→Sidecar→Agent trends rather than fully measured on eval1200, with only some runs (e.g., GPT path, Qwen, partial Doubao) anchored. Table 2 is presented in the main text as the consolidated protocol–backend ablation supporting broad outperformance. Projected cells should be clearly marked or removed from the main leaderboard; all bolded “best per protocol” claims and cross-backend rankings must rest on completed measurements with stated n and coverage. Until then, the multi-backend generality of the Agent advantage is overstated relative to the evidence actually collected.
  3. [§3.4; Appendix A.1, A.4, C.3 / Algorithm 1] SFQ-Agent’s Runner (Appendix C.3, Algorithm 1) hard-copies VC/SL, clamps CC/CTX by fixed caps (typically ≤6 for bare pointers / missing metrics; Appendix A.1), and maps MR from a discrete visual risk band and text severity via a closed-form rule. These free parameters are load-bearing for calibration and for the near-zero bias reported under Agent. §4 reports no sensitivity analysis over cap values, MR fusion mapping, or payload truncation limits (Appendix A.4: ≤2×500-char snippets). Please ablate caps/fusion rules on eval1200 and show that F3’s ranking vs best Direct/Sidecar is not an artifact of a particular cap schedule chosen to match human practice.
  4. [§3.1 Eq. (1); Table 2; Appendix A.2] Gold aggregation is dimension-first (Eqs. 3–4, Appendix D.3), which is appropriate under L1 gating, but the paper never reports how often CC or CTX are gated null on eval1200, nor per-dimension sample sizes behind Table 2’s CC/CTX/MR MAE columns (CTX is restricted to figures with citing text). Given that overall y_i averages only available dimensions (Eq. 1), missingness rates affect both human means and model errors. Please state evaluable counts per dimension on eval1200 and confirm metrics are computed only on jointly evaluable subsets when comparing protocols.
minor comments (6)
  1. [Figure 4; §3.3] Figure 4 panels are referenced as (a)–(f) in §3.3 but the embedded figure in the manuscript dump shows only placeholder labels; ensure final camera-ready figures have legible axis labels and a readable radar/funnel layout.
  2. [Abstract; Figure 1; §3] Inconsistent naming: “SciFigQual-bench” vs “SciFigQual-Bench”, “SFQAgent” vs “SFQ-Agent”, and “VCSL/CC/MR/CTX” shorthand in Figure 1 vs VC/SL in the text. Normalize throughout.
  3. [Table 1] Table 1 lists “# Images (test bench.) … 7,609” under Ours while eval1200 is the actual test split and 6,308 are rated; clarify train/release vs test counts in the table header to avoid implying all 7,609 are the evaluation bench.
  4. [§4.1; References] Several references and model names are dated 2025–2026; ensure bibliography entries are complete (pages/venues) and that backend identifiers in §4.1 match publicly documented API names at submission time for reproducibility.
  5. [Abstract; §4.1] Abstract says “highest consistency rate (93.4%)” without defining ±1-point Within-1; align abstract wording with the Within-1 metric defined in §4.1.
  6. [Abstract; §1] Typos and spacing artifacts from PDF extraction (“experimentalconclusions”, “thatevaluatesscientificimage”, “±1-point”) should be cleaned in the source.

Circularity Check

0 steps flagged

No significant circularity: SFQ-Agent is scored against independent human gold under fixed inputs, not against quantities defined by its own outputs.

full rationale

SciFigQual-Bench is an empirical benchmark-and-judge paper, not a first-principles derivation. The load-bearing claim is that staged SFQ-Agent agrees better with dimension-first aggregated expert scores on the frozen eval1200 split than Direct or Sidecar judges given identical (I, c, T, m) and L1 gating. Human gold is collected separately (multi-rater annotation with adjudication), then models are compared via MAE/W-1/SRCC; nothing in Eqs. (1)–(4), Algorithm 1, or the protocol ladder forces those agreement numbers by construction. The Runner hard-copies VC/SL from the vision track and applies deterministic CC/CTX/MR caps as an auditable design choice for the automated judge—it does not redefine the human labels or make MAE tautological. Rubric authorship and self-contained dataset construction are normal for benchmarks and do not reduce the reported outperformance to a fitted input or self-definitional loop. No uniqueness theorem, ansatz smuggled via overlapping-author citation, or renaming of a known closed-form result carries the central claim. Residual concerns about human label noise vs. small protocol gaps are correctness/identification issues, not circularity.

Axiom & Free-Parameter Ledger

4 free parameters · 5 axioms · 3 invented entities

As an empirical benchmark paper, load-bearing commitments are methodological assumptions (rubric polarity, gating, gold aggregation, judge staging) rather than physical axioms. No fitted natural constants drive the central claim; free choices are design thresholds and scoring caps that shape labels and agent outputs.

free parameters (4)
  • CC/CTX hard caps (typically ≤6 for bare pointers / missing metrics) = cap ≈ 6 for bare/missing-metric cases
    Appendix A.1 imposes numeric caps that directly bound gold-compatible scores and Runner outputs for incomplete text; chosen by rubric design, not estimated from a held-out physical process.
  • Curation thresholds S1–S5 (e.g., <50px, aspect >20:1, solid>0.97, pHash Hamming≤8, paper hard-delete ratio>0.30) = multiple fixed thresholds as in Appendix Table 3
    These hand-set filters determine which figures enter the released N=7,609 set and thus the evaluation distribution.
  • Context payload truncation (≤2 snippets × ≤500 chars; caption map ≤220 chars) = 500 / 220 character limits
    Appendix A.4 truncation choices affect what judges see for CTX/CC and can change scores without changing the underlying PDF.
  • MR fusion mapping from visual risk band [ℓ,h] and text severity σ = band-midpoint rules in Algorithm 1
    Algorithm 1’s closed-form MR replacement is a designed discretization (none/minor/moderate/severe/contradiction → numeric scores), i.e., a free scoring schedule.
axioms (5)
  • domain assumption Human dimension-first mean scores under the shared 1–10 rubric are the external gold for quality.
    All MAE/W-1/SRCC claims are defined against aggregated expert labels (§3.1, Appendix D.3 Eqs. 3–4).
  • domain assumption L1 evidence gating: missing caption/citations must null CC/CTX rather than count as low quality.
    Stated via Krippendorff-style missing-data handling and Appendix Table 2; changes overall means whenever text is partial.
  • ad hoc to paper MR polarity is higher-score = lower misleading risk, averaged with other dimensions for overall quality.
    §3.1 defines MR on the same [1,10] scale with inverted risk semantics; overall is the mean of available dimensions.
  • domain assumption Index-pattern citation binding from PDFs yields the true evidentiary context for a figure.
    Step 3 assumes figure-index matches plus paragraph windows are sufficient CTX evidence versus layout-only or abstract heuristics.
  • ad hoc to paper Modality-separated staged judging plus deterministic Runner is a fair protocol comparison against single-pass VLMs with the same inputs.
    §3.4–4.1; Agent uses 3 calls/figure vs 1 for baselines, so gains mix architecture with extra inference compute.
invented entities (3)
  • SciFigQual-Bench instance tuple (I, c, T, m) with five-dimension gated labels independent evidence
    purpose: Define the evaluation object binding figure crop, caption, citing paragraphs, and metadata.
    Core dataset abstraction in §3.1; standard benchmark construction rather than a physical entity.
  • SFQ-Agent (vision evidence module + language evidence module + cross-modal judge + Runner) no independent evidence
    purpose: Produce auditable multi-dimensional scores via staged evidence collection and deterministic aggregation.
    Introduced in §3.4/Figure 3 as the proposed judge architecture; validated only via correlation with the paper’s own human labels.
  • eval1200 fixed public test split independent evidence
    purpose: Provide a frozen stratified comparison set for protocol–backend ablations.
    §3.2 Step 5 / Appendix B.1; value depends on release integrity.

pith-pipeline@v1.2.0-daily-grok45 · 25609 in / 4097 out tokens · 74346 ms · 2026-07-30T11:34:58.015539+00:00 · methodology

0 comments
read the original abstract

Scientific images are the core elements of presenting experimental conclusions, elaborating system architecture, and supporting comparative arguments in scientific papers. However, existing image quality assessment (IQA) methods are predominantly designed for natural photographs or AI-generated content, which cannot be directly applied to scientific papers. The few existing studies on scholarly charts remain confined to visual-surface comparisons, failing to verify caption alignment, citation relevance, or visual misleadingness. To address this, we propose SciFigQual-Bench, a full-text contextual benchmark that evaluates scientific images across five dimensions (clarity, layout, caption fit, context relevance, and misleading risk). The data covers top computer-science conferences from 2020 to 2025; 6,308 images were independently scored by multiple domain experts in five dimensions and aggregated into gold-standard annotations. Unlike previous scientific figure benchmarks, our dataset binds each image to its caption, citing sentence, and manuscript context. To enable automated evaluation on this benchmark, we designed a staged cross-modal evaluation framework SFQ-Agent to achieve auditable and refined scoring through the collection and fusion of modal evidence. Multiple mainstream large models were evaluated on the test subset eval1200, and SFQ-Agent (F3) equipped with GPT-5.6-Sol achieved the lowest overall average absolute error (0.418) and the highest consistency rate (93.4%), consistently outperforming both direct evaluation and auxiliary (Sidecar) visual language model evaluation schemes.

Figures

Figures reproduced from arXiv: 2607.27084 by Chuanzhi Xu, Haoyang Li, Huiqi Liang, Lequan Yu, Xiaozhen Zhong, Zihan Deng.

Figure 1
Figure 1. Figure 1: From isolated-figure evaluation to full-manuscript [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 3
Figure 3. Figure 3: Fig3 [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 2
Figure 2. Figure 2: SciFigQual-Bench construction pipeline: corpus [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: SFQ-Agent scoring pipeline: L1 gating, parallel vision and language evidence modules, cross-modal judge, and deterministic Runner aggregation. Direct and Sidecar judges serve as ablations. Step 3: Cross-modal judge. Once both tracks have pro￾duced structured evidence, the cross-modal judge fuses them to score CC, CTX, and MR. Specifically, it aligns visual haz￾ard indicators with textual cues and detects c… view at source ↗
Figure 4
Figure 4. Figure 4: Corpus overview of SciFigQual-Bench. From [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 1
Figure 1. Figure 1: Human multi-rater annotation cases from eval1200. Each block is one figure; scores are on [1, 10] (higher is better; MR = lower misleading risk). Gold column reports dimension-first means; – marks gated-null CTX. Rationale snippets are verbatim from annotator records [PITH_FULL_IMAGE:figures/full_fig_p014_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Human gold vs. model predictions on selected [PITH_FULL_IMAGE:figures/full_fig_p015_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

45 extracted references · 8 canonical work pages · 2 internal anchors

  1. [1]

    , title =

    Mittal, Anish and Moorthy, Anush Krishna and Bovik, Alan C. , title =. IEEE Transactions on Image Processing , volume =. 2012 , doi =

    2012

  2. [2]

    , title =

    Mittal, Anish and Soundararajan, Rajiv and Bovik, Alan C. , title =. IEEE Signal Processing Letters , volume =. 2013 , doi =

    2013

  3. [3]

    2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

    Prashnani, Ekta and Cai, Hong and Mostofi, Yasamin and Sen, Pradeep , title =. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =. 2018 , doi =

    2018

  4. [4]

    Wang, Jianyi and Chan, Kelvin C. K. and Loy, Chen Change , title =. Proceedings of the AAAI Conference on Artificial Intelligence , volume =. 2023 , doi =

    2023

  5. [5]

    Findings of the Association for Computational Linguistics: ACL 2022 , pages =

    Masry, Ahmed and Long, Do Xuan and Tan, Jia Qing and Joty, Shafiq and Hoque, Enamul , title =. Findings of the Association for Computational Linguistics: ACL 2022 , pages =. 2022 , publisher =. doi:10.18653/v1/2022.findings-acl.177 , url =

    doi:10.18653/v1/2022.findings-acl.177 2022

  6. [6]

    and Kumar, Pratyush , title =

    Methani, Nitesh and Ganguly, Pritha and Khapra, Mitesh M. and Kumar, Pratyush , title =. 2020 IEEE Winter Conference on Applications of Computer Vision (WACV) , pages =. 2020 , doi =

    2020

  7. [7]

    Computer Vision -- ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11--14, 2016, Proceedings, Part VII , series =

    Siegel, Noah and Horvitz, Zachary and Levin, Roie and Divvala, Santosh and Farhadi, Ali , title =. Computer Vision -- ECCV 2016: 14th European Conference, Amsterdam, The Netherlands, October 11--14, 2016, Proceedings, Part VII , series =. 2016 , publisher =. doi:10.1007/978-3-319-46478-7_41 , url =

    work page doi:10.1007/978-3-319-46478-7_41 2016

  8. [8]

    Findings of the Association for Computational Linguistics: ACL 2023 , pages =

    Liu, Fangyu and Eisenschlos, Julian and Piccinno, Francesco and Krichene, Syrine and Pang, Chenxi and Lee, Kenton and Joshi, Mandar and Chen, Wenhu and Collier, Nigel and Altun, Yasemin , title =. Findings of the Association for Computational Linguistics: ACL 2023 , pages =. 2023 , publisher =. doi:10.18653/v1/2023.findings-acl.660 , url =

    doi:10.18653/v1/2023.findings-acl.660 2023

  9. [9]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =

    Li, Lei and Wang, Yuqi and Xu, Runxin and Wang, Peiyi and Feng, Xiachong and Kong, Lingpeng and Liu, Qi , title =. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =. 2024 , publisher =. doi:10.18653/v1/2024.acl-long.775 , url =

    doi:10.18653/v1/2024.acl-long.775 2024

  10. [10]

    Lee and Huang, Ting-Hao , title =

    Hsu, Ting-Yao and Giles, C. Lee and Huang, Ting-Hao , title =. Findings of the Association for Computational Linguistics: EMNLP 2021 , pages =. 2021 , publisher =. doi:10.18653/v1/2021.findings-emnlp.277 , url =

    doi:10.18653/v1/2021.findings-emnlp.277 2021

  11. [11]

    Proceedings of the 38th International Conference on Machine Learning , series =

    Radford, Alec and Kim, Jong Wook and Hallacy, Chris and Ramesh, Aditya and Goh, Gabriel and Agarwal, Sandhini and Sastry, Girish and Askell, Amanda and Mishkin, Pamela and Clark, Jack and Krueger, Gretchen and Sutskever, Ilya , title =. Proceedings of the 38th International Conference on Machine Learning , series =. 2021 , url =

    2021

  12. [12]

    and Zhang, Hao and Gonzalez, Joseph E

    Zheng, Lianmin and Chiang, Wei-Lin and Sheng, Ying and Zhuang, Siyuan and Wu, Zhanghao and Zhuang, Yonghao and Lin, Zi and Li, Zhuohan and Li, Dacheng and Xing, Eric P. and Zhang, Hao and Gonzalez, Joseph E. and Stoica, Ion , title =. Advances in Neural Information Processing Systems , volume =. 2023 , note =. doi:10.52202/075280-2020 , url =

    doi:10.52202/075280-2020 2023

  13. [13]

    , title =

    Tufte, Edward R. , title =. 2001 , isbn =

    2001

  14. [14]

    and McGill, Robert , title =

    Cleveland, William S. and McGill, Robert , title =. Journal of the American Statistical Association , volume =. 1984 , doi =

    1984

  15. [15]

    2011 , url =

    Krippendorff, Klaus , title =. 2011 , url =

    2011

  16. [16]

    Idahl, Maximilian and Ahmadi, Zahra , title =. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (System Demonstrations) , pages =. 2025 , publisher =. doi:10.18653/v1/2025.naacl-demo.44 , url =

    doi:10.18653/v1/2025.naacl-demo.44 2025

  17. [17]

    Gildea, Daniel and Kan, Min-Yen and Madnani, Nitin and Teichmann, Christoph and Villalba, Mart. The. Proceedings of Workshop for. 2018 , publisher =. doi:10.18653/v1/W18-2504 , url =

    work page doi:10.18653/v1/w18-2504 2018

  18. [18]

    Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

    Ye, Guanghui and Zhao, Huan and Zhao, Zhixue and Ma, Tengfei and Wang, Kehan and Eger, Steffen and Jiang, Zhihua , title =. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =. 2026 , url =

    2026

  19. [19]

    2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

    Kafle, Kushal and Price, Brian and Cohen, Scott and Kanan, Christopher , title =. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =. 2018 , doi =

    2018

  20. [20]

    Mathew, Minesh and Karatzas, Dimosthenis and Jawahar, C. V. , title =. 2021 IEEE Winter Conference on Applications of Computer Vision (WACV) , pages =. 2021 , doi =

    2021

  21. [21]

    Advances in Neural Information Processing Systems , volume =

    Xu, Jiazheng and Liu, Xiao and Wu, Yuchen and Tong, Yuxuan and Li, Qinkai and Ding, Ming and Tang, Jie and Dong, Yuxiao , title =. Advances in Neural Information Processing Systems , volume =. 2023 , doi =

    2023

  22. [22]

    2023 , eprint =

    Wu, Xiaoshi and Hao, Yiming and Sun, Keqiang and Chen, Yixiong and Zhu, Feng and Zhao, Rui and Li, Hongsheng , title =. 2023 , eprint =. doi:10.48550/arXiv.2306.09341 , url =

    Pith/arXiv arXiv doi:10.48550/arxiv.2306.09341 2023

  23. [23]

    , title =

    Hu, Yushi and Liu, Benlin and Kasai, Jungo and Wang, Yizhong and Ostendorf, Mari and Krishna, Ranjay and Smith, Noah A. , title =. 2023 IEEE/CVF International Conference on Computer Vision (ICCV) , pages =. 2023 , doi =

    2023

  24. [24]

    Advances in Neural Information Processing Systems , volume =

    Kirstain, Yuval and Polyak, Adam and Singer, Uriel and Matiana, Shahbuland and Penna, Joe and Levy, Omer , title =. Advances in Neural Information Processing Systems , volume =. 2023 , doi =

    2023

  25. [25]

    2025 , eprint =

    Cui, Cheng and Sun, Ting and Liang, Suyin and Gao, Tingquan and Zhang, Zelun and Liu, Jiaxuan and Wang, Xueqing and Zhou, Changda and Liu, Hongen and Lin, Manhui and Zhang, Yue and Zhang, Yubo and Zheng, Handong and Zhang, Jing and Zhang, Jun and Liu, Yi and Yu, Dianhai and Ma, Yanjun , title =. 2025 , eprint =. doi:10.48550/arXiv.2510.14528 , url =

    doi:10.48550/arxiv.2510.14528 2025

  26. [26]

    The American Journal of Psychology , volume =

    Spearman, Charles , title =. The American Journal of Psychology , volume =. 1904 , doi =

    1904

  27. [27]

    Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages =

    Masry, Ahmed and Kavehzadeh, Parsa and Do, Xuan Long and Hoque, Enamul and Joty, Shafiq , title =. Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing , pages =. 2023 , publisher =. doi:10.18653/v1/2023.emnlp-main.906 , url =

    doi:10.18653/v1/2023.emnlp-main.906 2023

  28. [28]

    Advances in Neural Information Processing Systems , volume =

    Roberts, Jonathan and Han, Kai and Houlsby, Neil and Albanie, Samuel , title =. Advances in Neural Information Processing Systems , volume =. 2024 , note =. doi:10.52202/079017-0593 , url =

    doi:10.52202/079017-0593 2024

  29. [29]

    2026 , eprint =

    Li, Wenzhe and Chen, Liang and Wang, Junying and Guo, Yijing and Shen, Ye and Wen, Farong and Li, Chunyi and Zhang, Zicheng and Zhai, Guangtao , title =. 2026 , eprint =. doi:10.48550/arXiv.2603.06700 , url =

    doi:10.48550/arxiv.2603.06700 2026

  30. [30]

    Advances in Neural Information Processing Systems , volume =

    Wang, Zirui and Xia, Mengzhou and He, Luxi and Chen, Howard and Liu, Yitao and Zhu, Richard and Liang, Kaiqu and Wu, Xindi and Liu, Haotian and Malladi, Sadhika and Chevalier, Alexis and Arora, Sanjeev and Chen, Danqi , title =. Advances in Neural Information Processing Systems , volume =. 2024 , note =. doi:10.52202/079017-3609 , url =

    doi:10.52202/079017-3609 2024

  31. [31]

    Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =

    Ding, Junpeng and Tang, Zichen and E, Haihong and Ji, Mengyuan and Liu, Yang and Tian, Haolin and Sun, Haiyang and Sun, Pengqi and Xu, Yang and Liu, Yichen and Gao, Haocheng and Xi, Zijie and Jiang, Ruomeng and Zhao, Peizhi and Li, Rongjin and Li, Yuanze and Liu, Jiacheng and Yang, Zhongjun and Chen, Jintong and Lin, Siying , title =. Proceedings of the 6...

    doi:10.18653/v1/2026 2026

  32. [32]

    S1-MMAlign: A Large-Scale, Multi-Disciplinary Dataset for Scientific Figure-Text Understanding

    Wang, He and Guo, Longteng and Huo, Pengkang and Lin, Xuanxu and Yuan, Yichen and Jiang, Jie and Liu, Jing , title =. 2026 , eprint =. doi:10.48550/arXiv.2601.00264 , url =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2601.00264 2026

  33. [33]

    Findings of the Association for Computational Linguistics: EMNLP 2025 , pages =

    Song, Jifeng and Das, Arun and Cui, Ge and Huang, Yufei , title =. Findings of the Association for Computational Linguistics: EMNLP 2025 , pages =. 2025 , publisher =. doi:10.18653/v1/2025.findings-emnlp.899 , url =

    work page doi:10.18653/v1/2025.findings-emnlp.899 2025

  34. [34]

    The Fourteenth International Conference on Learning Representations , year =

    Xie, Yupeng and Zhang, Zhiyang and Wu, Yifan and Lu, Sirong and Zhang, Jiayi and Yu, Zhaoyang and Wang, Jinlin and Hong, Sirui and Liu, Bang and Wu, Chenglin and Luo, Yuyu , title =. The Fourteenth International Conference on Learning Representations , year =

  35. [35]

    Findings of the Association for Computational Linguistics: ACL 2024 , pages =

    Lee, Seongyun and Kim, Seungone and Park, Sue Hyun and Kim, Geewook and Seo, Minjoon , title =. Findings of the Association for Computational Linguistics: ACL 2024 , pages =. 2024 , publisher =. doi:10.18653/v1/2024.findings-acl.672 , url =

    work page doi:10.18653/v1/2024.findings-acl.672 2024

  36. [36]

    Proceedings of the 41st International Conference on Machine Learning , series =

    Wu, Haoning and Zhang, Zicheng and Zhang, Weixia and Chen, Chaofeng and Liao, Liang and Li, Chunyi and Gao, Yixuan and Wang, Annan and Zhang, Erli and Sun, Wenxiu and Yan, Qiong and Min, Xiongkuo and Zhai, Guangtao and Lin, Weisi , title =. Proceedings of the 41st International Conference on Machine Learning , series =. 2024 , url =

    2024

  37. [37]

    Journal of Natural Language Processing , volume =

    Yang, Zhishen and Dabre, Raj and Tanaka, Hideki and Okazaki, Naoaki , title =. Journal of Natural Language Processing , volume =. 2024 , publisher =. doi:10.5715/jnlp.31.1140 , url =

    work page doi:10.5715/jnlp.31.1140 2024

  38. [38]

    Evaluating Compliance with Visualization Guidelines in Diagrams for Scientific Publications Using Large Vision Language Models , booktitle =

    R. Evaluating Compliance with Visualization Guidelines in Diagrams for Scientific Publications Using Large Vision Language Models , booktitle =. 2026 , publisher =. doi:10.1007/978-3-032-04630-7_1 , url =

    work page doi:10.1007/978-3-032-04630-7_1 2026

  39. [39]

    Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =

    Ku, Max and Jiang, Dongfu and Wei, Cong and Yue, Xiang and Chen, Wenhu , title =. Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) , pages =. 2024 , publisher =. doi:10.18653/v1/2024.acl-long.663 , url =

    doi:10.18653/v1/2024.acl-long.663 2024

  40. [40]

    GENFIG1: Visual Summaries of Scholarly Work as a Challenge for Vision-Language Models

    Guan, Yaohan and Wang, Pristina and Dehak, Najim and Yuille, Alan and Chen, Jieneng and Khashabi, Daniel , title =. 2026 , eprint =. doi:10.48550/arXiv.2604.04172 , url =

    work page internal anchor Pith review Pith/arXiv arXiv doi:10.48550/arxiv.2604.04172 2026

  41. [41]

    Jobin, K. V. and Mondal, Ajoy and Jawahar, C. V. , title =. 2019 International Conference on Document Analysis and Recognition Workshops (ICDARW) , volume =. 2019 , publisher =. doi:10.1109/ICDARW.2019.00018 , url =

    arXiv 2019

  42. [42]

    Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages =

    Jacovi, Alon and Goldberg, Yoav , title =. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics , pages =. 2020 , publisher =. doi:10.18653/v1/2020.acl-main.386 , url =

    doi:10.18653/v1/2020.acl-main.386 2020

  43. [43]

    2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

    Wu, Haoning and Zhang, Zicheng and Zhang, Erli and Chen, Chaofeng and Liao, Liang and Wang, Annan and Xu, Kaixin and Li, Chunyi and Hou, Jingwen and Zhai, Guangtao and Xue, Geng and Sun, Wenxiu and Yan, Qiong and Lin, Weisi , title =. 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =. 2024 , doi =

    2024

  44. [44]

    2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =

    Li, Xudong and Nie, Wenjie and Zhang, Yan and Hu, Runze and Li, Ke and Zheng, Xiawu and Cao, Liujuan , title =. 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) , pages =. 2025 , doi =

    2025

  45. [45]

    IEEE Transactions on Image Processing , volume =

    Chen, Chaofeng and Mo, Jiadi and Hou, Jingwen and Wu, Haoning and Liao, Liang and Sun, Wenxiu and Yan, Qiong and Lin, Weisi , title =. IEEE Transactions on Image Processing , volume =. 2024 , publisher =. doi:10.1109/TIP.2024.3378466 , url =

    arXiv 2024