REVIEW 12 cited by
SciGraphQA: A Large-Scale Synthetic Multi-Turn Question-Answering Dataset for Scientific Graphs
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
In this work, we present SciGraphQA, a synthetic multi-turn question-answer dataset related to academic graphs. SciGraphQA is 13 times larger than ChartVQA, the previously largest chart-visual question-answering dataset. It is also the largest open-sourced chart VQA dataset with non-synthetic charts. To build our dataset, we selected 290,000 Computer Science or Machine Learning ArXiv papers published between 2010 and 2020, and then used Palm-2 to generate 295K samples of open-vocabulary multi-turn question-answering dialogues about the graphs. As context, we provided the text-only Palm-2 with paper title, abstract, paragraph mentioning the graph, and rich text contextual data from the graph itself, obtaining dialogues with an average 2.23 question-answer turns for each graph. We asked GPT-4 to assess the matching quality of our question-answer turns given the paper's context, obtaining an average rating of 8.7/10 on our 3K test set. We evaluated the 0-shot capability of the most popular MLLM models such as LLaVa, mPLUGowl, BLIP-2, and openFlamingo's on our dataset, finding LLaVA-13B being the most performant with a CIDEr score of 0.08. We further enriched the question prompts for LLAVA by including the serialized data tables extracted from the graphs using the DePlot model, boosting LLaVA's 0-shot CIDEr to 0.15. To verify the validity of our dataset, we also fine-tuned LLaVa using our dataset, reaching a substantially higher CIDEr score of 0.26. We anticipate further accuracy improvement by including segmentation mask tokens and leveraging larger LLM backbones coupled with emergent prompting techniques. Our code and data are open-sourced.
Forward citations
Cited by 12 Pith papers
-
SciFigAlign: Scoring Scientific Figures by Fine-tuned Alignment of Visuals with Manuscript Evidence
A fine-tuned CLIP+SciBERT multimodal scorer with manuscript-bound inputs and within-paper ranking reaches MAE 0.35 and 82% pairwise accuracy on held-out conference figures, far above LLM judges.
-
FinChart-Bench: Benchmarking Financial Chart Comprehension in Vision-Language Models
A new benchmark of real-world financial charts shows current vision-language models lag badly on questions that require reading values from chart axes.
-
Do Large Multimodal Models Solve Caption Generation for Scientific Figures? Lessons Learned from SciCap Challenge 2023
In human evaluations by three professional editors, GPT-4V captions for scientific figures were preferred over author-written captions and over captions from challenge-winning models.
-
Evaluating and Improving Graph to Text Generation with Large Language Models
Introducing PlanGTG, a 29k-pair instruction dataset with reordering and attribution subtasks, and fine-tuning 7B LLMs on it improves graph-to-text generation on WebNLG and DART over untuned and dataset-tuned baselines.
-
Rethinking Comprehensive Benchmark for Chart Understanding: A Perspective from Scientific Literature
A new benchmark built from real scientific paper charts, including flowcharts and context-dependent questions, shows large multimodal models perform far below human level on chart understanding.
-
Heterogeneous Graph Prompt Learning via Adaptive Weight Pruning
GPAWP prunes low-importance graph prompts using gradient-based importance masks and retunes the remaining prompts, improving few-shot node classification over HGPrompt on ACM, DBLP, and Freebase.
-
Argus Inspection: Do Multimodal Large Language Models Possess the Eye of Panoptes?
A new 1,430-item multimodal benchmark shows that leading multimodal LLMs rarely notice small visual traps needed for commonsense safety reasoning, with top scores near 0.46 on a scale whose maximum is about 0.97.
-
Flash-VL 2B: Optimizing Vision-Language Model Performance for Ultra-Low Latency and High Throughput
Flash-VL 2B is a 2-billion-parameter vision-language model with higher measured throughput than similar 2B models and slightly better average benchmark scores, thanks to a new image-tiling method.
-
Patent Figure Classification using Large Vision-language Models
A new dataset pair and a tournament-style multiple-choice query strategy let a fine-tuned InstructBLIP model classify patent figures by type, projection, object, and USPC class, beating CNN baselines on type and USPC.
-
Coling-UniA at SciVQA 2025: Few-Shot Example Retrieval and Confidence-Informed Ensembling for Multimodal Large Language Models
An ensemble of InternVL3-78B and Pixtral-Large with retrievable few-shot examples and a confidence threshold beats individual models on scientific figure question answering.
-
Evaluating LLMs for Visualization Generation and Understanding
GPT-4o led on most chart generation and understanding tests in this sample, but all four models struggled with complex charts, dotted lines, and close bar lengths.
-
BlueLM-V-3B: Algorithm and System Co-Design for Multimodal Large Language Models on Mobile Devices
BlueLM-V-3B is a 3B multimodal language model that runs on a phone NPU at 24.4 token/s and reports strong visual benchmark scores, enabled by a relaxed image-resolution rule and hardware-aware inference.
Discussion (0). Continue with ORCID to comment.