Pith. sign in

REVIEW 18 cited by

TextSquare: Scaling up Text-Centric Visual Instruction Tuning

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2404.12803 v3 pith:NDY2FZUN submitted 2024-04-19 cs.CV cs.LG

Jingqun Tang , Chunhui Lin , Zhen Zhao , Shu Wei , Binghong Wu , Qi Liu , Yangfan He , Kuan Lu
show 10 more authors
Hao Feng Yang Li Siqi Wang Lei Liao Wei Shi Yuliang Liu Hao Liu Yuan Xie Xiang Bai Can Huang
This is my paper
classification cs.CVcs.LG
keywords modelstext-centricdatainstructionmllmssquare-10mtextsquaretuning
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

Text-centric visual question answering (VQA) has made great strides with the development of Multimodal Large Language Models (MLLMs), yet open-source models still fall short of leading models like GPT4V and Gemini, partly due to a lack of extensive, high-quality instruction tuning data. To this end, we introduce a new approach for creating a massive, high-quality instruction-tuning dataset, Square-10M, which is generated using closed-source MLLMs. The data construction process, termed Square, consists of four steps: Self-Questioning, Answering, Reasoning, and Evaluation. Our experiments with Square-10M led to three key findings: 1) Our model, TextSquare, considerably surpasses open-source previous state-of-the-art Text-centric MLLMs and sets a new standard on OCRBench(62.2%). It even outperforms top-tier models like GPT4V and Gemini in 6 of 10 text-centric benchmarks. 2) Additionally, we demonstrate the critical role of VQA reasoning data in offering comprehensive contextual insights for specific questions. This not only improves accuracy but also significantly mitigates hallucinations. Specifically, TextSquare scores an average of 75.1% across four general VQA and hallucination evaluation datasets, outperforming previous state-of-the-art models. 3) Notably, the phenomenon observed in scaling text-centric VQA datasets reveals a vivid pattern: the exponential increase of instruction tuning data volume is directly proportional to the improvement in model performance, thereby validating the necessity of the dataset scale and the high quality of Square-10M.

Discussion (0). Sign in to comment.

Forward citations

Cited by 18 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Local Spatiotemporal Convolutional Network for Robust Gait Recognition

    cs.CV 2026-05 unverdicted novelty 7.0 of 10

    LSTCN is a dual-branch CNN that extracts temporal gait features by pooling spatial data into strips and applying local spatiotemporal convolutions with asymmetric kernels.

  2. Adaptive Slicing-Assisted Hyper Inference for Enhanced Small Object Detection in High-Resolution Imagery

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    ASAHI adaptively slices high-res images into 6 or 12 patches, adds slicing-assisted fine-tuning, and uses Cluster-DIoU-NMS to hit 56.8% mAP on VisDrone2019 and 22.7% on xView while running 20-25% faster than fixed sli...

  3. DocPO: Advancing Document Policy Optimization via Tailored Step-Aware Rewards

    cs.CV 2026-08 conditional novelty 6.0 of 10

    A reward-annealing trick that progressively sharpens edit-distance rewards improves GRPO-style RL for document parsing across text, tables, and formulas.

  4. DAIN: Dynamic Agent-Based Interaction Network for Efficient and Collaborative Multimodal Reasoning

    cs.CL 2026-06 unverdicted novelty 6.0 of 10

    DAIN reframes multimodal fusion as dynamic agent collaboration with sparse activation, claiming SOTA results including 2.6% accuracy gain on ADNI across five benchmarks.

  5. Clearer Sight, Fewer Lies: Oriented Pickup Preference Optimization for Multimodal Hallucination Mitigation

    cs.CV 2026-06 unverdicted novelty 6.0 of 10

    OPPO is an evidence-aware preference optimization objective that contrasts faithful responses under varying visual evidence strengths to reduce hallucinations in MLLMs.

  6. Do You Need Text Rectification? Soft Attention Mask Embedding for Rectification-Free Scene Text Spotting

    cs.CV 2026-05 unverdicted novelty 6.0 of 10

    SAME-Net adds a differentiable soft attention mask embedding module to achieve rectification-free end-to-end scene text spotting with 84.02% H-mean on Total-Text.

  7. ProWAFT: A ROMA-LPD Instance for Workload-Aware and Dynamic Fault Tolerance in FPGA-Based CNN Accelerators

    cs.CL 2026-07 unverdicted novelty 5.0 of 10

    ProWAFT proposes a workload-aware dynamic fault-tolerance method for FPGA CNN accelerators via selective TMR and partial reconfiguration, reporting lower composite cost than static TMR or reactive approaches on ResNet...

  8. Clearer Sight, Fewer Lies: Oriented Pickup Preference Optimization for Multimodal Hallucination Mitigation

    cs.CV 2026-06 unverdicted novelty 5.0 of 10

    OPPO is an evidence-aware preference optimization that contrasts faithful responses under varying visual evidence strengths to reduce hallucinations in MLLMs.

  9. DocOCR-Eval: A Correction-Based Framework for OCR Tool Selection Without Ground Truth

    cs.LG 2026-05 conditional novelty 5.0 of 10

    OCR tools can be ranked without ground-truth labels by measuring how much a multimodal LLM must correct each tool's output.

  10. Lightweight Real-Time Rendering Parameter Optimization via XGBoost-Driven Lookup Tables

    cs.CV 2026-04 unverdicted novelty 5.0 of 10

    LUT-Opt distills XGBoost regressors into lookup tables to enable sub-millisecond adaptive optimization of rendering parameters such as subsurface scattering and ambient occlusion.

  11. Multi-Branch Non-Homogeneous Image Dehazing via Concentration Partitioning and Image Fusion

    cs.CV 2026-04 unverdicted novelty 5.0 of 10

    CPIFNet decomposes non-homogeneous dehazing into multiple homogeneous sub-problems via specialized IENet branches trained on different haze concentrations, then uses IFNet to fuse advantageous regions through deep fea...

  12. Feature Perturbation Pool-based Fusion Network for Unified Multi-Class Industrial Defect Detection

    cs.CV 2026-04 unverdicted novelty 5.0 of 10

    FPFNet reports state-of-the-art AUROC scores on MVTec-AD and VisA for unified multi-class defect detection by adding feature perturbation and hierarchical fusion to UniAD with no extra parameters.

  13. Hierarchical Awareness Adapters with Hybrid Pyramid Feature Fusion for Dense Depth Prediction

    cs.CV 2026-04 unverdicted novelty 5.0 of 10

    A multilevel perceptual CRF model using Swin Transformer, HPF fusion, HA adapters, and dynamic scaling attention achieves state-of-the-art monocular depth estimation on NYU Depth v2, KITTI, and MatterPort3D with reduc...

  14. A Survey on Evaluating Quality and Trustworthiness in LLM-Generated Data

    cs.AI 2026-01 conditional novelty 5.0 of 10

    A metric-oriented survey that classifies intrinsic quality and trustworthiness metrics for LLM-generated data across six modalities and documents systematic evaluation gaps in the current literature.

  15. LPCAN: Lightweight Pyramid Cross-Attention Network for Rail Surface Defect Detection Using RGB-D Data

    cs.CV 2026-01 reject novelty 4.0 of 10

    A lightweight RGB-D cross-attention network is proposed for rail defect detection, but the SOTA accuracy and generalization claims are internally inconsistent and the implementation is not public.

  16. Gait Recognition via Deep Residual Networks and Multi-Branch Feature Fusion

    cs.CV 2026-04 unverdicted novelty 3.0 of 10

    A multi-branch ResNet architecture with HRNet pose estimation and channel-attention fusion reaches 94.52% Rank-1 gait recognition accuracy on CASIA-B normal walking and leads skeleton-based methods on coat-wearing cases.

  17. Image Classification via Random Dilated Convolution with Multi-Branch Feature Extraction and Context Excitation

    cs.CV 2026-04 unverdicted novelty 3.0 of 10

    RDCNet reports state-of-the-art accuracy on CIFAR-10, CIFAR-100, SVHN, Imagenette, and Imagewoof by combining random dilated convolutions with multi-branch and attention modules.

  18. Knowledge-Embedded and Hypernetwork-Guided Few-Shot Substation Meter Defect Image Generation Method

    cs.CV 2026-01 reject novelty 3.0 of 10

    Fine-tuning Stable Diffusion with DreamBooth-style knowledge and hypernetwork-guided crack control maps can synthesize substation meter defect images that boost a YOLOv8 defect detector's mAP when added to the training set.

Pith tools