{"work":{"id":"45b0563f-7563-4f00-8da5-8be70ff803fb","openalex_id":null,"doi":null,"arxiv_id":"2205.14100","raw_key":null,"title":"GIT: A Generative Image-to-text Transformer for Vision and Language","authors":null,"authors_text":"Jianfeng Wang, Zhengyuan Yang, Xiaowei Hu, Linjie Li, Kevin Lin, Zhe Gan","year":2022,"venue":"cs.CV","abstract":"In this paper, we design and train a Generative Image-to-text Transformer, GIT, to unify vision-language tasks such as image/video captioning and question answering. While generative models provide a consistent network architecture between pre-training and fine-tuning, existing work typically contains complex structures (uni/multi-modal encoder/decoder) and depends on external modules such as object detectors/taggers and optical character recognition (OCR). In GIT, we simplify the architecture as one image encoder and one text decoder under a single language modeling task. We also scale up the pre-training data and the model size to boost the model performance. Without bells and whistles, our GIT establishes new state of the arts on 12 challenging benchmarks with a large margin. For instance, our model surpasses the human performance for the first time on TextCaps (138.2 vs. 125.5 in CIDEr). Furthermore, we present a new scheme of generation-based image classification and scene text recognition, achieving decent performance on standard benchmarks. Codes are released at \\url{https://github.com/microsoft/GenerativeImage2Text}.","external_url":"https://arxiv.org/abs/2205.14100","cited_by_count":null,"metadata_source":"pith","metadata_fetched_at":"2026-05-25T04:56:38.168256+00:00","pith_arxiv_id":"2205.14100","created_at":"2026-05-10T05:20:54.813250+00:00","updated_at":"2026-06-05T21:23:00.469572+00:00","title_quality_ok":true,"display_title":"GIT: A Generative Image-to-text Transformer for Vision and Language","render_title":"GIT: A Generative Image-to-text Transformer for Vision and Language"},"hub":{"state":{"work_id":"45b0563f-7563-4f00-8da5-8be70ff803fb","tier":"hub","tier_reason":"10+ Pith inbound or 1,000+ external citations","pith_inbound_count":28,"external_cited_by_count":null,"distinct_field_count":5,"first_pith_cited_at":"2022-09-14T17:24:07+00:00","last_pith_cited_at":"2026-05-22T12:22:20+00:00","author_build_status":"not_needed","summary_status":"needed","contexts_status":"needed","graph_status":"needed","ask_index_status":"not_needed","reader_status":"not_needed","recognition_status":"not_needed","updated_at":"2026-06-11T05:37:44.423591+00:00","tier_text":"hub"},"tier":"hub","role_counts":[{"context_role":"background","n":7},{"context_role":"baseline","n":1}],"polarity_counts":[{"context_polarity":"background","n":7},{"context_polarity":"baseline","n":1}],"runs":{},"summary":{},"graph":{},"authors":[]}}