Pith. sign in

REVIEW 4 cited by

InternGPT: Solving Vision-Centric Tasks by Interacting with ChatGPT Beyond Language

Not yet reviewed by Pith; the record is open.

Re-run · record.json Download PDF Read on arXiv ↗

This paper has not been read by Pith yet. Machine review is queued; the pith claim, tier, and objections will appear here once it completes.

SPECIMEN: schema-true, not a live event

T0 review · schema-true

One-sentence machine reading of the paper's core claim.

pith:XXXXXXXX · record.json · timestamp

arxiv 2305.05662 v4 pith:YPZQFC5W submitted 2023-05-09 cs.CV

Zhaoyang Liu , Yinan He , Wenhai Wang , Weiyun Wang , Yi Wang , Shoufa Chen , Qinglong Zhang , Zeqiang Lai
show 12 more authors
Yang Yang Qingyun Li Jiashuo Yu Kunchang Li Zhe Chen Xue Yang Xizhou Zhu Yali Wang Limin Wang Ping Luo Jifeng Dai Yu Qiao
This is my paper
classification cs.CV
keywords interngptvisualchatbotscontroligptinteractivepointingtasks
verification ladder T0 review T1 audit T2 compute T3 formal
0 comments
read the original abstract

We present an interactive visual framework named InternGPT, or iGPT for short. The framework integrates chatbots that have planning and reasoning capabilities, such as ChatGPT, with non-verbal instructions like pointing movements that enable users to directly manipulate images or videos on the screen. Pointing (including gestures, cursors, etc.) movements can provide more flexibility and precision in performing vision-centric tasks that require fine-grained control, editing, and generation of visual content. The name InternGPT stands for \textbf{inter}action, \textbf{n}onverbal, and \textbf{chat}bots. Different from existing interactive systems that rely on pure language, by incorporating pointing instructions, the proposed iGPT significantly improves the efficiency of communication between users and chatbots, as well as the accuracy of chatbots in vision-centric tasks, especially in complicated visual scenarios where the number of objects is greater than 2. Additionally, in iGPT, an auxiliary control mechanism is used to improve the control capability of LLM, and a large vision-language model termed Husky is fine-tuned for high-quality multi-modal dialogue (impressing ChatGPT-3.5-turbo with 93.89\% GPT-4 Quality). We hope this work can spark new ideas and directions for future interactive visual systems. Welcome to watch the code at https://github.com/OpenGVLab/InternGPT.

Discussion (0). Sign in to comment.

Forward citations

Cited by 4 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. PixDLM: A Dual-Path Multimodal Language Model for UAV Reasoning Segmentation

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    The work introduces the UAV Reasoning Segmentation task, the DRSeg benchmark dataset, and PixDLM as a baseline dual-path multimodal language model for reasoning-based segmentation in aerial imagery.

  2. GM-PRM: A Generative Multimodal Process Reward Model for Multimodal Mathematical Reasoning

    cs.CL 2025-08 conditional novelty 6.0 of 10

    A generative multimodal process reward model that produces step-level critiques and corrections improves average math accuracy for six multimodal LLMs by 2.9 to 5.9 points under a refinement-based Best-of-N strategy.

  3. LIRA: Inferring Segmentation in Large Multi-modal Models with Local Interleaved Region Assistance

    cs.CV 2025-07 conditional novelty 6.0 of 10

    LIRA improves referring segmentation and reduces hallucination in multimodal LLMs by fusing semantic and pixel features and interleaving local image regions with text descriptions.

  4. Docopilot: Improving Multimodal Models for Document-Level Understanding

    cs.CV 2025-07 conditional novelty 5.0 of 10

    A new academic-paper dataset and a retrieval-free fine-tuned InternVL2 model improve multi-page document QA accuracy and latency on several benchmarks.

Pith tools