Pith. sign in

REVIEW 5 major objections 7 minor 48 references

Fast or Slow? Integrating Fast Intuition and Deliberate Thinking for Enhancing Visual Question Answering

T0 review · 5 major / 7 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read FOCUS, a training-free method that routes each VQA question to fast zero-shot answering or to deliberate visual-marking, reports consistent gains across four benchmarks and roughly 44% faster inference than the all-object-marking baseline.

desk verdict FOCUS is a sensible plug-and-play VQA wrapper with consistent gains, but its answerability router is validated on only 100 VizWiz samples and needs per-sample evidence before the accuracy and efficiency claims can be fully trusted. read the letter →

arxiv 2506.00806 v1 pith:B7DBEDT6 submitted 2025-06-01 cs.CL

Songtao Jiang , Chenyi Zhou , Yan Zhang , Yeying Jin , Zuozhu Liu This is my paper
classification cs.CL
keywords FOCUSvisualquestionansweringmultimodallargelanguagemodelspromptingcomplexityroutingdualprocesstheoryself-consistencyGrounded-SAM
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

FOCUS is a training-free wrapper for multimodal large language models (MLLMs) that treats visual prompting as a scarce resource to be spent only when needed. It first asks the model to judge, at high temperature and three times, whether a given visual question is answerable from the raw image; if all three answers are 'Answerable', the model answers immediately, while otherwise FOCUS extracts key objects mentioned in the question, segments them with Grounded-SAM, and feeds the highlighted image back to the model. The paper's central claim is that this complexity-adaptive strategy beats indiscriminately annotating all objects on ScienceQA, TextVQA, VizWiz, and MME, achieving state-of-the-art scores with LLaVA-1.5-13B (74.4, 63.6, 58.5, 1551.0) and cutting inference time by nearly 44% compared with Set-of-Mark prompting. If true, it would mean that the bottleneck in MLLM visual reasoning is not the visual encoder alone but attention misallocation, and that routing questions by self-reported confidence is a viable way to fix it.

What carries the argument

The load-bearing mechanism is the question-complexity router built on self-consistency: the MLLM is prompted at high temperature to answer 'Answerable' or 'Unanswerable', three samples are drawn, and the question is treated as simple only if all three agree on 'Answerable'. Complex questions then enter the conceptualizing-before-observation pipeline: GPT-3.5 extracts K key elements from the question text, Grounded-SAM produces bounding-box-conditioned segmentations for those elements, and the resulting cropped image replaces the original input. The design choice that carries the argument is the contrast with Set-of-Mark prompting, which segments all objects for every question; FOCUS's gains are attributed to spending visual markers only where the model's own uncertainty indicates attention is needed.

What would settle it

Human-annotate a few thousand questions from the four benchmarks as needing fine-grained visual perception or not, compare FOCUS's router decisions to these labels, and then run FOCUS with an oracle router: if the oracle yields substantially larger gains than the self-consistency router, the confidence-check premise is the weak link.

Watch

Extended reading notes

Core claim

The paper's central discovery is that the dual-process distinction from cognitive psychology transfers to VQA: some questions are answered correctly by the model's fast zero-shot response, while others need a deliberate second pass in which the model's attention is pointed at the right image regions. FOCUS operationalizes this by routing every query through a lightweight confidence check and, only for low-confidence queries, substituting a 'conceptualizing before observation' pass—the language model names the key objects from the question, a segmentation model crops exactly those regions, and the MLLM re-answers on the refined image. The paper reports that this consistently improves five MLLMs and achieves SoTA on four benchmarks with LLaVA-1.5-13B, while reducing inference time by nearly 44% versus Set-of-Mark, because simple questions skip visual marking entirely and complex questions mark only a few key elements rather than the whole image.

Load-bearing premise

The effectiveness of FOCUS rests on the assumption that three high-temperature 'Answerable' responses from the MLLM reliably distinguish questions that need no visual refinement from those that do; if that confidence proxy is miscalibrated, FOCUS either skips deliberate thinking on hard questions or wastes computation on easy ones.

Editorial extensions

If this is right

  • Visual-prompting methods for VQA should route by question difficulty rather than annotate all objects for every question.
  • FOCUS with a 7-billion-parameter LLaVA reaches accuracy close to vanilla 13-billion-parameter LLaVA, suggesting selective routing can substitute for part of the benefit of larger models.
  • The near-44% inference-time reduction over all-object marking makes selective prompting practical for latency-sensitive VQA applications.
  • The method is plug-and-play and improved three open-source and two black-box MLLMs, so the routing principle transfers across architectures and scales.
  • Attention visualizations indicate the mechanism operates by shifting the model's attention to question-relevant image regions.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the confidence router could be strengthened by replacing the all-or-nothing majority vote with a semantic-entropy threshold; the paper's own Table 6 shows entropy-based variants correct more challenging cases than the original strategy.
  • Editorial inference: the reported gains on GPT-4V and Gemini Pro come from only 250 sampled ScienceQA questions, so larger black-box evaluations are needed before the plug-and-play claim for black-box models is settled.
  • Editorial inference: because keyword extraction and cropping depend on GPT-3.5 and Grounded-SAM, the method inherits their failure modes; a single end-to-end module that learns which visual regions matter could be more robust, though it would sacrifice the training-free property.
  • Editorial inference: the same complexity-routing idea could be transferred to other multimodal tasks such as referring expression comprehension or OCR-heavy document QA, where attention misallocation is also a known bottleneck.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 7 minor

Summary. The paper proposes FOCUS, a plug-and-play inference-time method for visual question answering that routes each question to either a fast zero-shot response (fast intuition) or a slower 'conceptualizing before observation' pipeline (deliberate thinking) that extracts key phrases with GPT-3.5 and highlights corresponding regions with Grounded-SAM. The router is a self-consistency check over three high-temperature 'Answerable/Unanswerable' responses from the MLLM. Experiments on ScienceQA, TextVQA, VizWiz, and MME with three open-source and two black-box MLLMs report consistent accuracy gains, state-of-the-art results with LLaVA-1.5-13B, and a roughly 44% inference-time reduction versus Set-of-Mark prompting.

Significance. If the routing mechanism is reliable, FOCUS is a practical, model-agnostic contribution: it is training-free, improves both accuracy and inference efficiency, and is evaluated across diverse MLLMs and benchmarks. The paper includes ablations, attention visualizations, analysis of failure modes, and a cost comparison, and the authors promise code release. However, the central empirical claims currently rest on the router, whose validation is thin, and on single-run accuracy numbers without error bars; the significance of the result therefore depends on whether the requested additional analyses confirm that the routing policy, rather than the overall pipeline, drives the observed gains.

major comments (5)
  1. [§2.2, Appendix B.1] The question-complexity router is the core mechanism of FOCUS, yet it is validated only on 100 VizWiz samples against a GPT-4V proxy, with agreement 81% for N=3. This does not establish that the questions routed to deliberate thinking are actually those for which deliberate thinking improves accuracy on the four benchmarks. The aggregate ablation in Table 4 cannot distinguish an adaptive router from a trivial policy that sends most questions to one branch. Please provide a per-benchmark analysis that partitions questions by router decision and reports accuracy under FOCUS, Only FI, and Only DT in each partition, together with router precision/recall relative to an oracle that knows which branch yields the correct answer.
  2. [Tables 1–5] No error bars, confidence intervals, or significance tests are reported anywhere. Several headline gains are small (for example, InstructBLIP +1.8 on ScienceQA in Table 2, and the +2.1 TextVQA gain in Table 1), and the black-box results in Table 3 are based on only 250 samples. Without repeated runs or bootstrap intervals, the claim that FOCUS 'consistently improves' all models across all benchmarks is not formally supported. At minimum, report standard errors or bootstrap confidence intervals for the main comparisons, especially those versus SoM.
  3. [Appendix C, Tables 1–3] The authors' closely related method JVTP (Jiang et al., 2024b) is cited in Related Work as a prompt-optimization approach but is absent from all experimental tables. Because JVTP is also an object-centric visual and text prompting method and is part of the same line of work, omitting it from comparisons undermines the claim of state-of-the-art performance. Please add JVTP (or an explicit and justified statement of why it is not comparable) to Table 1 and the open-source model comparisons.
  4. [Figure 4, §3.2 'Cost Analysis'] The efficiency claim of 'nearly 44%' faster inference than SoM is load-bearing for the paper's contribution, but the manuscript provides neither the measurement protocol nor the breakdown of time spent in each component (router, GPT-3.5 keyword extraction, Grounded-SAM segmentation, and final MLLM inference). The percentage depends directly on the router's false-positive rate, which is validated only on 100 VizWiz samples. Please specify the hardware, the number of queries, how the 44% figure is computed, and report the time breakdown separately for the FI branch and the DT branch.
  5. [§2.2, Appendices B.1, B.2] Hyperparameters N=3 and the grounded-SAM confidence threshold 0.7 are selected using 100 VizWiz samples and then applied to all four benchmarks. If those 100 samples are part of the VizWiz test set, the method's hyperparameters have been tuned on a test set, which would inflate the reported gains. Please clarify whether these samples are a held-out validation set, and report sensitivity of the main results to N and the threshold across benchmarks.
minor comments (7)
  1. [Section 3 heading] The section title 'Experiement' contains a typo; it should read 'Experiment'.
  2. [Section 4, 'Analysis of Answerable and Unanswerable Proportions'] The text mentions 'LLaV A-1.6' and 'LLaV A-1.5' with inconsistent spacing; presumably both refer to LLaVA-1.5. Please standardize the model name.
  3. [Table 4 caption] The caption states 'we used LLaV A-1.5 as the MLLM' but does not specify the 7B or 13B variant. The reported numbers are identical to the FOCUS + LLaVA-1.5-13B row in Table 1; please clarify which variant was used.
  4. [Figure 4] The bars are labeled with values such as 100%, 53%, 47%, 68%, and 57% without definition in the caption, and the caption contains the typo 'comparasion'. Please explain what these percentages represent and how relative inference time was measured.
  5. [Table 6] The column headers 'DT (Count)', 'FI (Count)', and 'Corrected (DT/FI)' are not defined in the caption. Please explain the experimental setup for the 100 challenging questions and what the counts refer to.
  6. [Section 2.3, Eq. (3)] Equation (3) defines the processed image as a sum over segmented regions, but Appendix B.3 states that when keyword extraction fails, the original image is preserved. Please clarify how a failed extraction (K=0) is handled in the notation of Eq. (3).
  7. [Throughout] The benchmark name 'TextVQA' is inconsistently abbreviated as 'TextQA' or 'TVQA' in Tables and text. Please use a single consistent abbreviation.

Circularity Check

0 steps flagged · score 0.0 of 10

No material circularity: FOCUS's routing and deliberate-thinking pipeline are validated against external full-benchmark comparisons, and the self-citations are contextual rather than load-bearing.

full rationale

The claimed derivation chain is empirical rather than formal: FOCUS routes each question by self-reported 'Answerable/Unanswerable' responses, and for 'complex' questions applies key-element extraction and segmentation before calling the MLLM. The load-bearing evidence is the accuracy comparison on ScienceQA, TextVQA, VizWiz, and MME (Tables 1-3) against the unmodified MLLM and SoM, which are external benchmarks not constructed from the router's validation data. The router's hyperparameters (N=3, segmentation threshold 0.7) are tuned using small 100-sample VizWiz subsets (Appendices B.1-B.3), but the final accuracy and efficiency claims are measured on full benchmarks and are not algebraically forced by those subsets. No equation in the paper defines a predicted quantity as equal to a fitted input; the 'self-consistency' used is the standard LLM self-consistency heuristic of Wang et al. (2022), and it is not used to construct the target answers. The paper cites prior work by overlapping authors (e.g., Jiang et al. 2024b in Related Work), but these citations are contextual and are not invoked as a uniqueness theorem or as the justification for the main experimental claim. The 100-sample GPT-4V validation of the router is a proxy and could be criticized for limited scale or construct validity, but that is a correctness and robustness concern, not a circularity concern. Therefore the derivation is self-contained with respect to the circularity patterns considered.

Assumptions & free parameters 3 free parameters · 3 assumptions · 0 invented entities

No physically new entities are introduced. The method's load-bearing assumptions are the router reliability and external tool accuracy; both are validated on 100-sample subsets rather than full benchmarks, and the main hyperparameters (N=3, threshold 0.7) are selected on those same subsets.

free parameters (3)
  • N (number of self-consistency responses) = 3
    N=3 was chosen after measuring complexity-evaluation accuracy of 59%, 81%, 83% for N=1, 3, 5 on 100 VizWiz samples (Appendix B.1).
  • Grounded-SAM confidence threshold = 0.7
    Threshold for accepting segmentation masks set to 0.7 in Appendix B.2 to avoid modifying images when segmentation is uncertain; no sensitivity analysis is reported.
  • Temperature for complexity evaluation = high (exact value not reported)
    Section 2.2 says a high temperature is used to activate the MLLM, but the exact value is not disclosed, making the router difficult to reproduce.
assumptions (3)
  • ad hoc to paper Three high-temperature "Answerable/Unanswerable" responses reliably separate easy from hard VQA questions.
    This is the router in Section 2.2; the paper validates it on 100 VizWiz samples with GPT-4V labels (Appendix B.1), not on the four benchmarks.
  • domain assumption GPT-3.5-turbo extracts the visual keywords that matter, and Grounded-SAM segments them accurately.
    The paper reports 87% keyword extraction and 85% segmentation accuracy on 100 VizWiz samples (Appendices B.2, B.3), and thresholds are set so failures leave the image unmodified.
  • domain assumption Replacing or augmenting the full image with aggregated segmented regions (Eq. 3) preserves the information necessary to answer.
    The method discards or de-emphasizes unmarked background and objects; this assumes those regions are irrelevant for the question.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Fast or Slow? Integrating Fast Intuition and Deliberate Thinking for Enhancing Visual Question Answering." pith.science (2026). https://pith.science/paper/B7DBEDT6

@misc{pith2026250600806,
  author       = {Pith},
  title        = {Pith review of: Fast or Slow? Integrating Fast Intuition and Deliberate Thinking for Enhancing Visual Question Answering},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/B7DBEDT6}},
  note         = {Machine review of arXiv:2506.00806}
}
read the original abstract

Multimodal large language models (MLLMs) still struggle with complex reasoning tasks in Visual Question Answering (VQA). While current methods have advanced by incorporating visual prompts, our study uncovers critical limitations: these approaches indiscriminately annotate all detected objects for every visual question, generating excessive visual markers that degrade task performance. This issue stems primarily from a lack of focus on key visual elements, raising two important questions: Are all objects equally important, and do all questions require visual prompts? Motivated by Dual Process Theory, which distinguishes between instinctive and deliberate cognitive modes in human reasoning, we propose FOCUS, a plug-and-play approach that dynamically adapts to the complexity of questions, combining fast intuitive judgments with deliberate analytical reasoning to enhance the vision-language reasoning capability of the MLLM. For straightforward questions, FOCUS supports efficient zero-shot reasoning. For more complex tasks, it employs the conceptualizing before observation strategy to highlight critical elements. Extensive experiments on four benchmarks, ScienceQA, TextQA, VizWiz, and MME, demonstrate that FOCUS consistently improves the performance of both open-source and black-box MLLMs, achieving significant gains across all datasets. Ablation studies further validate the importance of combining diverse cognitive strategies with refined visual information for superior performance. Code will be released.

Figures

Figures reproduced from arXiv: 2506.00806 by the authors.

Figure 1
Figure 1. Comparison of methods: baseline (original [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Overview of our model pipeline. (1) Question complexity evaluation for MLLM-based behavior to [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figure 4
Figure 4. Relative inference time comparasion Cost Analysis. As illustrated in [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗
Figures from the paper (2 more)
Figure 5
Figure 5. Figure 5: More cases on GPT-4V What college does the batter have on her helmet? Answer question using a single word or phrase. UCSB What college does the batter have on her helmet? Answer question using a single word or phrase. UCSD What college does the batter have on her helme…
Figure 6
Figure 6. Figure 6: More cases on GPT-4V [PITH_FULL_IMAGE:figures/full_fig_p010_6.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

48 extracted references · 9 canonical work pages

  1. [1]

    Shengnan An, Yifei Li, Zeqi Lin, Qian Liu, Bei Chen, Qiang Fu, Weizhu Chen, Nanning Zheng, and Jian-Guang Lou. 2022. Input-tuning: Adapting unfamiliar inputs to frozen pretrained models. arXiv preprint arXiv:2203.03131

    arXiv 2022

  2. [2]

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966

    arXiv 2023

  3. [3]

    Mu Cai, Haotian Liu, Siva Karthik Mustikovela, Gregory P Meyer, Yuning Chai, Dennis Park, and Yong Jae Lee. 2024. Vip-llava: Making large multimodal models understand arbitrary visual prompts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 12914--12923

    work page 2024

  4. [4]

    Dongping Chen, Ruoxi Chen, Shilin Zhang, Yinuo Liu, Yaochen Wang, Huichi Zhou, Qihui Zhang, Yao Wan, Pan Zhou, and Lichao Sun. 2024. Mllm-as-a-judge: Assessing multimodal llm-as-a-judge with vision-language benchmark. arXiv preprint arXiv:2402.04788

    arXiv 2024

  5. [5]

    Jun Chen, Deyao Zhu, Xiaoqian Shen, Xiang Li, Zechun Liu, Pengchuan Zhang, Raghuraman Krishnamoorthi, Vikas Chandra, Yunyang Xiong, and Mohamed Elhoseiny. 2023. Minigpt-v2: large language model as a unified interface for vision-language multi-task learning. arXiv preprint arXiv:2310.09478

    arXiv 2023

  6. [6]

    Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. 2023. http://arxiv.org/abs/2305.06500 Instructblip: Towards general-purpose vision-language models with instruction tuning

    arXiv 2023

  7. [7]

    Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric P Xing, and Zhiting Hu. 2022. Rlprompt: Optimizing discrete text prompts with reinforcement learning. arXiv preprint arXiv:2205.12548

    arXiv 2022

  8. [8]

    Alexey Dosovitskiy. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929

    arXiv 2020

Show all 48 references
  1. [9]

    Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Zhenyu Qiu, Wei Lin, Jinrui Yang, Xiawu Zheng, et al. 2023. Mme: A comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306.13394

    2023 arXiv

  2. [10]

    Gemini Team, Google . 2023. https://storage.googleapis.com/deepmind-media/gemini/gemini_1_report.pdf Gemini 1 report . Google

    2023

  3. [11]

    Yuxian Gu, Xu Han, Zhiyuan Liu, and Minlie Huang. 2021. Ppt: Pre-trained prompt tuning for few-shot learning. arXiv preprint arXiv:2109.04332

    2021 arXiv

  4. [12]

    Danna Gurari, Qing Li, Abigale J Stangl, Anhong Guo, Chi Lin, Kristen Grauman, Jiebo Luo, and Jeffrey P Bigham. 2018. Vizwiz grand challenge: Answering visual questions from blind people. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 3...

    2018

  5. [13]

    Zixi Jia, Jiqiang Liu, Hexiao Li, Qinghua Liu, and Hongbin Gao. 2024. Dcot: Dual chain-of-thought prompting for large multimodal models. In The 16th Asian Conference on Machine Learning (Conference Track)

    2024

  6. [14]

    Songtao Jiang, Yuan Wang, Sibo Song, Yan Zhang, Zijie Meng, Bohan Lei, Jian Wu, Jimeng Sun, and Zuozhu Liu. 2025. Omniv-med: Scaling medical vision-language model for universal visual understanding. arXiv preprint arXiv:2504.14692

    2025 arXiv

  7. [15]

    Songtao Jiang, Yan Zhang, Ruizhe Chen, Yeying Jin, and Zuozhu Liu. 2024 a . Modality-fair preference optimization for trustworthy mllm alignment. arXiv preprint arXiv:2410.15334

    2024 arXiv

  8. [16]

    Songtao Jiang, Yan Zhang, Chenyi Zhou, Yeying Jin, Yang Feng, Jian Wu, and Zuozhu Liu. 2024 b . Joint visual and text prompting for improved object-centric perception with multimodal large language models. arXiv preprint arXiv:2404.04514

    2024 arXiv

  9. [17]

    Songtao Jiang, Tuo Zheng, Yan Zhang, Yeying Jin, Li Yuan, and Zuozhu Liu. 2024 c . Med-moe: Mixture of domain-specific experts for lightweight medical vision-language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 3843--3860

    2024

  10. [18]

    Daniel Kahneman. 2011. Thinking, fast and slow. Farrar, Straus and Giroux

    2011

  11. [19]

    S KAZEMZADE, V Ordonez, M MATTENV, et al. 2014. Referring to objects in photographs of natural scenes. In Empirical Methods in Natural Language Processing, volume 28, pages 787--789

    2014

  12. [20]

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al. 2023. Segment anything. arXiv preprint arXiv:2304.02643

    2023 arXiv

  13. [21]

    Hugo Lauren c on, Lucile Saulnier, L \'e o Tronchon, Stas Bekman, Amanpreet Singh, Anton Lozhkov, Thomas Wang, Siddharth Karamcheti, Alexander Rush, Douwe Kiela, et al. 2024. Obelics: An open web-scale filtered dataset of interleaved image-text documents. Advances in Neural In...

    2024

  14. [22]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. arXiv preprint arXiv:2301.12597

    2023 arXiv

  15. [23]

    Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190

    2021 arXiv

  16. [24]

    Lawrence Zitnick, and Piotr Dollár

    Tsung-Yi Lin, Michael Maire, Serge Belongie, Lubomir Bourdev, Ross Girshick, James Hays, Pietro Perona, Deva Ramanan, C. Lawrence Zitnick, and Piotr Dollár. 2015. http://arxiv.org/abs/1405.0312 Microsoft coco: Common objects in context

    2015 arXiv

  17. [25]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2023 a . http://arxiv.org/abs/2310.03744 Improved baselines with visual instruction tuning

    2023 arXiv

  18. [26]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023 b . Visual instruction tuning. arXiv preprint arXiv:2304.08485

    2023 arXiv

  19. [27]

    Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang. 2023 c . Gpt understands, too. AI Open

    2023

  20. [28]

    Pan Lu, Swaroop Mishra, Tanglin Xia, Liang Qiu, Kai-Wei Chang, Song-Chun Zhu, Oyvind Tafjord, Peter Clark, and Ashwin Kalyan. 2022. Learn to explain: Multimodal reasoning via thought chains for science question answering. Advances in Neural Information Processing Systems, 35:2...

    2022

  21. [29]

    Chancharik Mitra, Brandon Huang, Trevor Darrell, and Roei Herzig. 2024. Compositional chain-of-thought prompting for large multimodal models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14420--14431

    2024

  22. [30]

    Ron Mokady, Amir Hertz, and Amit H Bermano. 2021. Clipcap: Clip prefix for image captioning. arXiv preprint arXiv:2111.09734

    2021 arXiv

  23. [31]

    OpenAI. 2023 a . Gpt-3.5 turbo fine-tuning and api updates. https://openai.com/blog/gpt-3-5-turbo-fine-tuning-and-api-updates

    2023

  24. [32]

    OpenAI. 2023 b . Gpt-4v(ision) system card. https://cdn.openai.com/papers/GPTV_System_Card.pdf. 2023

    2023

  25. [33]

    R OpenAI. 2023 c . Gpt-4 technical report. arxiv 2303.08774. View in Article, 2

    2023 arXiv

  26. [34]

    Jing Qian, Li Dong, Yelong Shen, Furu Wei, and Weizhu Chen. 2022. Controllable natural language generation with contrastive prefixes. arXiv preprint arXiv:2202.13257

    2022 arXiv

  27. [35]

    Tianhe Ren, Shilong Liu, Ailing Zeng, Jing Lin, Kunchang Li, He Cao, Jiayu Chen, Xinyu Huang, Yukang Chen, Feng Yan, et al. 2024. Grounded sam: Assembling open-world models for diverse visual tasks. arXiv preprint arXiv:2401.14159

    2024 arXiv

  28. [36]

    Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. 2019. Towards vqa models that can read. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8317--8326

    2019

  29. [37]

    Hao Sun. 2023. Offline prompt evaluation and optimization with inverse reinforcement learning. arXiv preprint arXiv:2309.06553

    2023 arXiv

  30. [38]

    Tu Vu, Brian Lester, Noah Constant, Rami Al-Rfou, and Daniel Cer. 2021. Spot: Better frozen model adaptation through soft prompt transfer. arXiv preprint arXiv:2110.07904

    2021 arXiv

  31. [39]

    Junke Wang, Lingchen Meng, Zejia Weng, Bo He, Zuxuan Wu, and Yu-Gang Jiang. 2023. To see is to believe: Prompting gpt-4v for better visual instruction tuning. arXiv preprint arXiv:2311.07574

    2023 arXiv

  32. [40]

    Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171

    2022 arXiv

  33. [41]

    Xiang Wei, Xingyu Cui, Ning Cheng, Xiaobin Wang, Xin Zhang, Shen Huang, Pengjun Xie, Jinan Xu, Yufeng Chen, Meishan Zhang, et al. 2023. Zero-shot information extraction via chatting with chatgpt. arXiv preprint arXiv:2302.10205

    2023 arXiv

  34. [42]

    Miao Xiong, Zhiyuan Hu, Xinyang Lu, Yifei Li, Jie Fu, Junxian He, and Bryan Hooi. 2023. Can llms express their uncertainty? an empirical evaluation of confidence elicitation in llms. arXiv preprint arXiv:2306.13063

    2023 arXiv

  35. [43]

    Qinwei Xu, Xingkun Xu, Chenyi Zhou, Zuozhu Liu, Feiyue Huang, Shaoxin Li, Lifeng Zhu, Zhian Bai, Yuchen Xu, and Weiguo Hu. 2025. https://doi.org/https://doi.org/10.1016/j.eswa.2025.126585 Towards normalized clinical information extraction in chinese radiology report with large...

    2025

  36. [44]

    Jianwei Yang, Hao Zhang, Feng Li, Xueyan Zou, Chunyuan Li, and Jianfeng Gao. 2023. Set-of-mark prompting unleashes extraordinary visual grounding in gpt-4v. arXiv preprint arXiv:2310.11441

    2023 arXiv

  37. [45]

    Karypis, and Alexander J

    Zhuosheng Zhang, Aston Zhang, Mu Li, Hai Zhao, G. Karypis, and Alexander J. Smola. 2023. https://dx.doi.org/10.48550/arXiv.2302.00923 Multimodal chain-of-thought reasoning in language models . arXiv preprint arXiv:2302.00923

    2023 doi arXiv

  38. [46]

    Ge Zheng, Bin Yang, Jiajin Tang, Hong-Yu Zhou, and Sibei Yang. 2023. Ddcot: Duty-distinct chain-of-thought prompting for multimodal reasoning in language models. Advances in Neural Information Processing Systems, 36:5168--5191

    2023

  39. [47]

    URL: " 'urlintro :=

    ENTRY address author booktitle chapter edition editor howpublished institution journal key month note number organization pages publisher school series title type volume year eprint doi pubmed url lastchecked label extra.label sort.label short.list INTEGERS output.state before...

  40. [48]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.