REVIEW 10 cited by
OmniCaptioner: One Captioner to Rule Them All
Not yet reviewed by Pith; the record is open.
This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.
SPECIMEN: schema-true, not a live event
T0 review · schema-true
One-sentence machine reading of the paper's core claim.
pith:XXXXXXXX · record.json · timestamp
Signed reviews
read the original abstract
We propose OmniCaptioner, a versatile visual captioning framework for generating fine-grained textual descriptions across a wide variety of visual domains. Unlike prior methods limited to specific image types (e.g., natural images or geometric visuals), our framework provides a unified solution for captioning natural images, visual text (e.g., posters, UIs, textbooks), and structured visuals (e.g., documents, tables, charts). By converting low-level pixel information into semantically rich textual representations, our framework bridges the gap between visual and textual modalities. Our results highlight three key advantages: (i) Enhanced Visual Reasoning with LLMs, where long-context captions of visual modalities empower LLMs, particularly the DeepSeek-R1 series, to reason effectively in multimodal scenarios; (ii) Improved Image Generation, where detailed captions improve tasks like text-to-image generation and image transformation; and (iii) Efficient Supervised Fine-Tuning (SFT), which enables faster convergence with less data. We believe the versatility and adaptability of OmniCaptioner can offer a new perspective for bridging the gap between language and visual modalities.
Forward citations
Cited by 10 Pith papers
-
CodePercept: Code-Grounded Visual STEM Perception for MLLMs
Perception, not reasoning, is the main bottleneck for MLLM STEM visual reasoning, and training on executable reconstruction code measurably fixes it.
-
AVCap: Reinforcing Audio-Video Joint Caption with Detail-Aware Reward
An audio-video captioning model trained on a new 100K dataset with a detail-aware GRPO reward, plus a matching atomic QA benchmark, reports gains over open-source baselines and competitive results with Gemini-2.5-Pro ...
-
Scene Graph Thinking: Reinforcing Structured Visual Reasoning for Multimodal Large Language Models
Scene Graph Thinking (SaGe) converts images into hierarchical graphs, samples 120K node-articulated CoTs, and uses node-as-proxy GRPO rewards to improve MLLM fine-grained and relational visual reasoning.
-
Z-Image: An Efficient Image Generation Foundation Model with Single-Stream Diffusion Transformer
Z-Image is an efficient 6B-parameter foundation model for image generation that rivals larger commercial systems in photorealism and bilingual text rendering through a new single-stream diffusion transformer and strea...
-
EvoLMM: Self-Evolving Large Multimodal Models with Continuous Rewards
A self-evolving multimodal model using continuous self-consistency rewards improves math reasoning by about 2–3% using only raw images, without labels or external reward models.
-
SC-Captioner: Improving Image Captioning with Self-Correction by Reinforcement Learning
An RL framework that trains vision-language models to self-correct captions via a scene-graph-based reward outperforms SFT and DPO on caption quality.
-
MME-Reasoning: A Comprehensive Benchmark for Logical Reasoning in MLLMs
A new 1,188-question multimodal benchmark covering deductive, inductive, and abductive reasoning shows that leading MLLMs score around 60% and are especially weak at abductive reasoning.
-
Chimera: Improving Generalist Model with Domain-Specific Experts
Chimera fuses frozen domain-expert encoders into a generalist multimodal LLM via routing and a 30% masking of general tokens, lifting InternVL2-8B from 61.6 to 64.9 on MathVista and from 31.3 to 32.4 on MathVerse.
-
Lumina-mGPT 2.0: Stand-Alone AutoRegressive Image Modeling
A from-scratch, decoder-only autoregressive model reports text-to-image quality comparable to diffusion models and handles several image tasks in one framework.
-
Infi-MMR: Curriculum-based Unlocking Multimodal Reasoning via Phased Reinforcement Learning in Multimodal Small Language Models
Infi-MMR-3B trains a 3B visual language model with text-first, caption-augmented, then caption-free reinforcement learning, reaching 43.68% on MathVerse testmini.
Discussion (0). Continue with ORCID to comment.