REVIEW 15 cited by
Visual Chain of Thought: Bridging Logical Gaps with Multimodal Infillings
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
Recent advances in large language models elicit reasoning in a chain-of-thought that allows models to decompose problems in a human-like fashion. Though this paradigm improves multi-step reasoning ability in language models, it is limited by being unimodal and applied mainly to question-answering tasks. We claim that incorporating visual augmentation into reasoning is essential, especially for complex, imaginative tasks. Consequently, we introduce VCoT, a novel method that leverages chain-of-thought prompting with vision-language grounding to recursively bridge the logical gaps within sequential data. Our method uses visual guidance to generate synthetic multimodal infillings that add consistent and novel information to reduce the logical gaps for downstream tasks that can benefit from temporal reasoning, as well as provide interpretability into models' multi-step reasoning. We apply VCoT to the Visual Storytelling and WikiHow summarization datasets and demonstrate through human evaluation that VCoT offers novel and consistent synthetic data augmentation beating chain-of-thought baselines, which can be used to enhance downstream performance.
Forward citations
Cited by 15 Pith papers
-
Point-RFT: Improving Multimodal Reasoning with Visually Grounded Reinforcement Finetuning
Point-RFT uses point-grounded chain-of-thought with GRPO reinforcement to improve chart reasoning, reporting 90.04% on ChartQA, though internal tables and the OOD setup weaken the claim.
-
VLM-R$^3$: Region Recognition, Reasoning, and Refinement for Enhanced Multimodal Chain-of-Thought
VLM-R3 uses reinforcement learning to teach a multimodal model when and where to crop an image during its chain-of-thought, improving scores on several visual reasoning benchmarks.
-
SketchAgent: Language-Driven Sequential Sketch Generation
SketchAgent uses a multimodal LLM prompted with a numbered-grid sketching language to generate, edit, and collaboratively draw sequential vector sketches without any training.
-
MagiC: Evaluating Multimodal Cognition Toward Grounded Visual Reasoning
MagiC evaluates answer correctness, reasoning validity, grounding fidelity, and self-correction on about 900 hand-annotated visual questions across 15 vision-language models.
-
ViCrit: A Verifiable Reinforcement Learning Proxy Task for Visual Perception in VLMs
A verifiable RL proxy task that asks VLMs to locate a single injected hallucination in a 200-word caption improves visual perception and transfers to math and abstract reasoning benchmarks.
-
Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery Simulation
APC, a training-free pipeline combining detection, depth, and orientation modules, improves VLM accuracy on allocentric spatial reasoning benchmarks by converting perspective questions into egocentric prompts.
-
Argus Inspection: Do Multimodal Large Language Models Possess the Eye of Panoptes?
A new 1,430-item multimodal benchmark shows that leading multimodal LLMs rarely notice small visual traps needed for commonsense safety reasoning, with top scores near 0.46 on a scale whose maximum is about 0.97.
-
Argus: Vision-Centric Reasoning with Grounded Chain-of-Thought
Argus adds explicit language-guided visual attention to multimodal LLMs by grounding questions to bounding boxes and re-engaging those regions, improving vision-centric reasoning and grounding accuracy.
-
ReFineVLA: Reasoning-Aware Teacher-Guided Transfer Fine-Tuning
Fine-tuning a vision-language-action robot model on teacher-generated reasoning rationales raises average simulated manipulation success by up to 8.6 percentage points over the SpatialVLA baseline.
-
Paint Outside the Box: Synthesizing and Selecting Training Data for Visual Grounding
POBF paints new backgrounds around preserved objects to synthesize visual-grounding training data and filters those samples with teacher-model scores, improving accuracy by 5.83% over real-only training.
-
Less is More Tokens: Efficient Math Reasoning via Difficulty-Aware Chain-of-Thought Distillation
Difficulty-aware compression of CoT traces plus SFT and DPO lets LLMs shorten reasoning on easy math problems, cutting tokens by up to 30% with mixed accuracy effects.
-
Explainability for Vision Foundation Models: A Survey
A structured review of 122 papers on explainability for vision foundation models, with a taxonomy and the finding that quantitative evaluation is rare (36%).
-
A Review of Multimodal Explainable Artificial Intelligence: Past, Present and Future
A historical review that organizes multimodal explainability methods into four chronological eras and three explainability types, extending coverage to generative LLMs.
-
How Vision-Language Tasks Benefit from Large Pre-trained Models: A Survey
A survey that categorizes pre-trained-model-based vision-language methods into four challenge-driven paradigms, with performance tables and a discussion of risks.
-
Nature's Insight: A Novel Framework and Comprehensive Analysis of Agentic Reasoning Through the Lens of Neuroscience
A survey and taxonomy that organizes AI agentic reasoning into four neuroscience-inspired categories without introducing new empirical results.
Discussion (0). Continue with ORCID to comment.