pith. sign in

arxiv: 2502.09927 · v1 · pith:6YGP4SWUnew · submitted 2025-02-14 · 💻 cs.CV · cs.AI

Granite Vision: a lightweight, open-source multimodal model for enterprise Intelligence

Granite Vision Team: Leonid Karlinsky , Assaf Arbelle , Abraham Daniels , Ahmed Nassar , Amit Alfassi , Bo Wu , Eli Schwartz , Dhiraj Joshi
show 54 more authors
Jovana Kondic Nimrod Shabtay Pengyuan Li Roei Herzig Shafiq Abedin Shaked Perek Sivan Harary Udi Barzelay Adi Raz Goldfarb Aude Oliva Ben Wieles Bishwaranjan Bhattacharjee Brandon Huang Christoph Auer Dan Gutfreund David Beymer David Wood Hilde Kuehne Jacob Hansen Joseph Shtok Ken Wong Luis Angel Bathen Mayank Mishra Maksym Lysak Michele Dolfi Mikhail Yurochkin Nikolaos Livathinos Nimrod Harel Ophir Azulai Oshri Naparstek Rafael Teixeira de Lima Rameswar Panda Sivan Doveh Shubham Gupta Subhro Das Syed Zawad Yusik Kim Zexue He Alexander Brooks Gabe Goodhart Anita Govindjee Derek Leist Ibrahim Ibrahim Aya Soffer David Cox Kate Soule Luis Lastras Nirmit Desai Shila Ofek-koifman Sriram Raghavan Tanveer Syeda-Mahmood Peter Staar Tal Drory Rogerio Feris
This is my paper
classification 💻 cs.CV cs.AI
keywords modelgranitevisionlightweightvisualarchitecturedesigneddocument
0
0 comments X
read the original abstract

We introduce Granite Vision, a lightweight large language model with vision capabilities, specifically designed to excel in enterprise use cases, particularly in visual document understanding. Our model is trained on a comprehensive instruction-following dataset, including document-related tasks, such as content extraction from tables, charts, diagrams, sketches, and infographics, as well as general image tasks. The architecture of Granite Vision is centered around visual modality alignment with a decoder-only, 2 billion parameter Granite large language model. Additionally, we introduce a dedicated safety classification approach in test-time that leverages a sparse set of attention vectors to identify potential harmful inputs. Despite its lightweight architecture, Granite Vision achieves strong results in standard benchmarks related to visual document understanding, as well as on the LiveXiv benchmark, which is designed to avoid test set contamination by using a constantly updated corpus of recently published Arxiv papers. We are releasing the model under the Apache-2 license, allowing for both research and commercial use, while offering complete visibility into the training data and other relevant details. See https://huggingface.co/ibm-granite/ for model weights.

This paper has not been read by Pith yet.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 6 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. WikiVQABench: A Knowledge-Grounded Visual Question Answering Benchmark from Wikipedia and Wikidata

    cs.CV 2026-05 conditional novelty 7.0

    WikiVQABench is a human-curated collection of Wikipedia-based VQA items that require both visual evidence and external knowledge from Wikidata to answer correctly.

  2. Beyond Bag-of-Patches: Learning Global Layout via Textual Supervision for Late-Interaction Visual Document Retrieval

    cs.CV 2026-05 unverdicted novelty 7.0

    A text-supervised global layout embedding augments local patch representations in late-interaction VDR, yielding +2.4 nDCG@5 and +2.3 MAP@5 gains over ColPali/ColQwen baselines on ViDoRe-v2.

  3. ParseBench: A Document Parsing Benchmark for AI Agents

    cs.CV 2026-04 accept novelty 7.0

    ParseBench is a new benchmark for document parsing in AI agents that reveals fragmented performance across five semantic dimensions with LlamaParse Agentic scoring highest at 84.9%.

  4. ChartNet: A Million-Scale, High-Quality Multimodal Dataset for Robust Chart Understanding

    cs.CV 2026-03 unverdicted novelty 7.0

    ChartNet is a million-scale multimodal dataset for chart understanding created via code-guided synthesis spanning 24 chart types with five aligned modalities per sample.

  5. POTATR: A Lightweight Image-to-Graph Model for Page-Level Table Extraction

    cs.CV 2026-06 unverdicted novelty 6.0

    POTATR extends TATR into a 29M-parameter image-to-graph model for contextual page-level table extraction, reporting 0.964 GriTS_Con on PubTables-v2 Single Pages while running 130x faster and 300x cheaper than tested a...

  6. Building a Precise Video Language with Human-AI Oversight

    cs.CV 2026-04 unverdicted novelty 6.0

    CHAI framework pairs AI pre-captions with expert human critiques to produce precise video descriptions, enabling open models to outperform closed ones like Gemini-3.1-Pro and improve fine-grained control in video gene...