Pith. sign in

REVIEW 3 major objections 5 minor 1 cited by

ReGraP-LLaVA: Reasoning enabled Graph-based Personalized Large Language and Vision Assistant

T0 review · 3 major / 5 minor · reviewed 2026-08-15 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A personalized multimodal assistant trained on knowledge graphs and chain-of-thought QA pairs can reason over relations among a user's concepts, not just recognize or caption them.

desk verdict Useful new dataset and graph-prompting method for personalized MLLMs, but the GPT-4o closed evaluation loop undermines the reasoning claim until the graph facts are independently checked. read the letter →

arxiv 2505.03654 v2 pith:LVVZIA52 submitted 2025-05-06 cs.CV cs.AI

Yifan Xiang , Zhenxi Zhang , Bin Li , Yixuan Weng , Shoujun Zhou , Yangfan He , Keqin Li This is my paper · ORCID
classification cs.CVcs.AI
keywords personalizedmultimodallargelanguagemodelsknowledgegraphreasoningchain-of-thoughtQApromptingrelationalinMLLMspersonalizationbenchmarkvisualinstructiontuningalignment
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ReGraP-LLaVA tries to establish that a personalized multimodal assistant can reason over the relations among a user's concepts, not merely recognize or caption them. The paper builds the ReGraP dataset of 120 personalized-knowledge sets, each combining images, a knowledge graph, and chain-of-thought QA pairs, then trains a 7B LLaVA model on these triples with soft and hard graph prompting. On a new benchmark spanning multiple-choice, fill-in-the-blank, true/false, and descriptive questions, the model reports the highest accuracy among prompt-based and fine-tuned baselines, including larger models. If correct, user-facing assistants could answer multi-step contextual queries such as "Who can help Bocchi with lyric writing?" by connecting attributes and relations across several concepts.

What carries the argument

The carrying object is the personalized knowledge graph $G$: nodes are personalized concepts and their attributes, edges are relations, and each CoT QA pair is attached to the reasoning subgraph $R \subseteq G$ that justifies the answer. Soft prompting encodes $R$ with a graph neural network and projects it with an MLP into the LLM's embedding space as $\hat{H} = \mathrm{MLP}(\mathrm{GNN}(R))$, then concatenates it with instruction embeddings. Hard prompting serializes $R$ into triples, introduces new entity and relation tokens into the tokenizer, and extends the classification head by $2N+1$ tokens. Both routes make the graph structure part of the model's input, and the CoT QA pairs act as a natural-language extraction of that same structure; the hard-prompt route is the configuration with the best accuracy in the paper.

What would settle it

Show the model a counterfactual image in which a learned relation is visually reversed (e.g., two band members' roles swapped) and ask the same relational question; if the model answers according to the stored graph rather than the visible evidence, then the reported reasoning is retrieval of generated relations, not reasoning grounded in the image.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a 7B multimodal LLM can internalize a user's knowledge graph and chain-of-thought reasoning from QA pairs, and then use that structure to answer questions that join facts across two or more concepts. The authors construct 120 personalized-knowledge sets, each with images, a knowledge graph, and CoT QA pairs whose answers are tied to the reasoning subgraph, and train LLaVA on them with soft and/or hard graph prompting. Across close-ended tasks, ReGraP-LLaVA reaches 0.942 simple and 0.892 difficult multiple-choice accuracy, and improves open-ended descriptive answers, while also transferring to the Yo'LLaVA and MyVLM recognition/captioning benchmarks. The conclusion is that graph-structured personal knowledge plus CoT supervision is sufficient for an MLLM to move from recognizing concepts to reasoning over their relations.

Load-bearing premise

The load-bearing premise is that the GPT-4o-constructed knowledge graphs and CoT QA pairs, together with the benchmark questions and reference answers derived from them, correctly and completely capture true personalized knowledge and the answers a human would give; if the generated graphs contain invented attributes or relations, both training and evaluation are measuring fidelity to the generator's fiction rather than genuine reasoning about the world.

Editorial extensions

If this is right

  • Personalized assistants can answer multi-step relational queries (e.g., "Who can help Bocchi with lyrics?") instead of only recognizing or captioning a user's concepts.
  • Training MLLMs on image-KG-CoT triples with graph prompting can yield large gains on relation-based questions without scaling the base model beyond 7B.
  • The ReGraP benchmark provides a reusable test for evaluating knowledge connection and relational reasoning in personalized MLLMs across MC, fill-in-the-blank, true/false, and descriptive tasks.
  • The hard graph-prompting variant, which adds entity and relation tokens, is the highest-accuracy configuration in the paper and is the recommended default.
  • The model also transfers to prior personalization datasets (Yo'LLaVA and MyVLM), preserving recognition and captioning while adding relational ability.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the same soft/hard graph-prompting recipe may transfer to any structured knowledge injection into MLLMs, such as temporal or causal graphs, since the mechanism only assumes a graph can be serialized or embedded.
  • Editorial inference: the benchmark's difficult questions are all derivable from the same generated graphs; a stronger generalization test would hold out entire relation types or ask questions requiring graph knowledge to be combined with visual details not present in the graph.
  • Editorial inference: because the paper reports no negative or counterfactual training examples, the model's confidence in denying an object's presence may degrade; adding such examples could improve both negative recognition and robustness.
  • Editorial inference: replacing the GPT-4o-generated graphs with human-verified graphs would clarify whether the reported relational reasoning transfers beyond the generator's internal consistency; this is a testable extension the paper does not report.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper introduces ReGraP, a dataset of 120 personalized-knowledge sets, each containing images, a GPT-4o-constructed knowledge graph (KG), and chain-of-thought QA pairs derived from that graph. On top of this, the authors propose ReGraP-LLaVA, a LLaVA-based MLLM trained with soft and/or hard graph prompts, and a benchmark with multiple-choice, fill-in-the-blank, true/false, and descriptive questions in closed- and open-ended settings. Experiments report that ReGraP-LLaVA outperforms prompt-based and finetuning-based baselines on most relational-reasoning tasks, with additional ablations, human-preference evaluation, and qualitative evidence.

Significance. If the central claims hold, the paper makes a useful contribution to personalized MLLMs: it is among the first to move beyond single-concept recognition to multi-concept relational reasoning, it releases code and data, and it includes careful comparisons against both prompt-based and finetuning-based baselines. The proposed benchmark, despite the concerns below, is a plausible template for evaluating personalized relational reasoning. However, the empirical case rests on GPT-4o-generated training and evaluation content, and the paper currently does not establish that the model is reasoning over human-grounded personalized knowledge rather than reproducing the generator's synthetic graph; this is the load-bearing issue for the claimed significance.

major comments (3)
  1. [Section 3, Eq. (3); Section 5 (Benchmark); Tables 18-19; Section J] The training and evaluation are closed-loop with respect to GPT-4o-generated content. The KGs and CoT QA pairs are produced by GPT-4o (Section 3), and the benchmark's reference answers for open-ended questions are also GPT-4o-generated (Table 18) and scored by GPT-4o (Table 19). Section C evaluates the quality of CoT QA pairs, but not the fidelity of the graph facts to the real objects or human-provided knowledge. Section E states that responses are manually reviewed, but this is not a systematic fact-level verification of the triplets. Section J itself admits that the model may generate fabricated information. As a result, the gains on 'difficult' reasoning questions (e.g., Table 2, Fill-in-the-Blank Difficult 0.858 vs 0.817) could reflect the model learning the generator's synthetic consistency rather than performing human-grounded relational reasoning. The authors should either independently verify graph triplets (e.g., by human judges or cross-referencing the images and user-provided text), reserve a held-out set of human-written benchmark questions, or analyze which errors are attributable to graph hallucinations.
  2. [Section 6, Tables 2-5; Section I] No error bars, confidence intervals, or significance tests are reported for any of the main quantitative results. The differences between top methods are often small; for example, in Table 2, GPT-4o (Prompt) and Qwen2.5-VL-72B (Prompt) differ by 0.021 on difficult multiple-choice, and in Table 5 the three graph-prompting variants differ by at most about 0.4%. Section I checks semantic consistency of outputs across repeated runs for five sets, but it does not report the variance of the benchmark metrics used in Tables 2-5. Without this information, the 'SoTA' claim and the head-to-head comparisons are not statistically grounded. Reporting repeated-seed standard deviations or bootstrap confidence intervals would make the claims commensurate with the evidence.
  3. [Section 5 (Benchmark); Tables 25, 26-31] The benchmark questions appear to be near-duplicates of the training QA facts. For instance, the training example for girls_band_cry in Table 25 states 'Rupa is the bassist of Togenashi Togeari,' and the benchmark in Table 31 asks 'Rupa plays the ___ in TOGENASHI' with the answer 'Bass.' Similarly, several ROG_mouse benchmark questions mirror the attribute questions in the training data (brand, connection type, RGB lighting). If the benchmark questions are generated from the same KGs used for training, then the closed-ended tasks largely measure memorization of the training graph rather than the model's ability to reason over genuinely new personalized queries. The authors should clarify the exact construction process of the benchmark and, ideally, add held-out facts or paraphrased human-written questions that require composing known relations in new ways.
minor comments (5)
  1. [Section 4.1, Eq. (4)] Equation (4) writes \H = MLP(H_g) ∈ R^d, but X_emb in Eq. (5) is in R^{m×d}; the addition X_new = X_emb + \H requires either broadcasting or a sequence-alignment explanation. Please clarify the intended shapes.
  2. [Figures 2 and 3] Several placeholder glyphs appear in the figure text (e.g., '????????0', 'dram', and '????????1'), which makes the graph-prompting mechanism hard to read. The figures should be regenerated with actual token labels.
  3. [Section 5 (Training)] The training description reports 10 training images and about 20 CoT QA pairs per set, but does not state how the per-set models are trained and evaluated (e.g., one model per set, batch size, number of steps, or how the 10 epochs are scheduled across sets). This information is needed for reproducibility.
  4. [Section C, Table 7] The QA-pair quality evaluation reports that all LLM evaluators except GPT-4o answer 'Yes' for nearly all criteria; this near-perfect agreement may indicate evaluator leniency or prompt-induced bias rather than genuine quality. Reporting per-criterion disagreement statistics and including more negative control items would strengthen the evaluation.
  5. [Section I] The robustness analysis checks semantic consistency of generated descriptions, but not the stability of the numerical metrics in Tables 2-5. The claim 'confirming the robustness of our training pipeline' is therefore narrower than the robustness evidence needed for the main results.

Circularity Check

2 steps flagged · score 6.0 of 10

GPT-4o-generated KGs and benchmark references form a closed loop, making the relational-reasoning claim partly self-referential.

  1. other [Section 3 'CoT QA pairs Generation' (Eq. 3); Section 5 'Dataset' and 'Benchmark'.]
    "After constructing G, we proceed to generate P(QA,R) from the routes on G ... These routes are subsequently used as contextual prompts to guide GPT-4o in generating questions Q, where each route R serves as the 'thinking process' for answering the question. Finally, Q and R are jointly provided to GPT-4o to generate a CoT reasoning answer ACoT, formulated as: ACoT = GPT(Q,R). ... Benchmark. For each set of personalized knowledge, we construct a diverse set of closed-ended questions. ... Each set contains an entire graph, over 20 CoT QA pairs and the corresponding subgraph."

    The training labels (CoT QA pairs) are GPT-4o verbalizations of routes through each set's knowledge graph, and the benchmark is then constructed per set from the same personalized-knowledge sets, which are defined as images plus KGs plus CoT QA pairs. A model that has memorized the GPT-4o-generated graph can therefore answer the relational-reasoning questions without performing externally grounded reasoning. The benchmark's ground truth is not independent of the training target: both derive from the same GPT-4o-generated KG by construction, so the reported closed-ended gains partly measure fidelity to the generator's graph rather than verified relational reasoning.

  2. self definitional [Section 6.2 'Open-ended QA'; Table 18 and Table 19 prompts.]
    "Then, we employ GPT-4o with personalized knowledge and images to generate reference answers and subsequently perform as an evaluator to score model outputs based on the reference ... the GPT-Score serve as a reference, which reflects the alignment with GPT's styles and preferences rather than an absolute measure of answer quality."

    The same model family (GPT-4o) that produced the CoT training answers in Eq. (3) also generates the open-ended reference answers and then judges the model's outputs against those references. Thus the GPT-Score is a self-consistency score with the teacher that wrote the training targets, not an independent measure of relational reasoning. The paper explicitly concedes that GPT-Score reflects alignment with GPT's styles, making any performance claim built on that metric definitionally tied to the training-data generator rather than to independently verified knowledge.

full rationale

The central relational-reasoning claim is evaluated on the ReGraP benchmark, whose underlying knowledge source is the same GPT-4o-generated KGs from which the CoT training targets are derived, and whose open-ended references and scoring are also GPT-4o-based. This creates a partially closed loop: the model is trained on GPT-4o's graph-derived reasoning, then judged against GPT-4o's reference answers and preferences. Section J admits the system 'may occasionally generate fabricated information,' and no systematic verification of the generated KG facts is reported, so the benchmark's ground truth itself is ungrounded. The paper is not fully circular: the graph-prompting ablation (ReGraP-LLaVA vs. LLaVA(CoT)) is a controlled comparison on identical data, the Point metric is more defensible, and the external Yo'LLaVA/MyVLM recognition and captioning evaluation is independent. Self-citations are background references only and are not load-bearing. Overall this is partial circularity centered on the evaluation/data-generation loop rather than on a formal derivation, so the score is 6.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central claim rests on assumptions about the fidelity of GPT-4o-generated knowledge graphs, the correctness of benchmark ground truth, and the suitability of a frozen LLaVA backbone. The main free parameters are dataset and benchmark design choices (pair counts, question counts, image counts) rather than fitted constants. One new model artifact, reasoning tokens, is introduced without independent external evidence.

free parameters (3)
  • Number of CoT QA pairs per set = about 20
    Hand-chosen data budget. The paper does not ablate the total number of QA pairs, only the answer length (Section A).
  • Benchmark question counts per set = 40 MC, 15 TF, 10 fill, 3 closed descriptive, 5 open descriptive
    Arbitrary distribution chosen for the benchmark. No justification or sensitivity analysis is given for these counts.
  • Training images per set = 10
    Matches prior personalization datasets (Table 1). No sensitivity study on image count is provided.
assumptions (4)
  • domain assumption GPT-4o with the provided prompts produces relation graphs that faithfully represent the true relations among the personalized concepts in the images.
    Invoked in Section 3, Eqs. (1)-(2) and Tables 13-14. If the graphs contain hallucinated or missing edges, both training and benchmark inherit the errors.
  • domain assumption The benchmark questions and reference answers, including GPT-4o-generated reference answers, agree with human judgments of correctness.
    Used in Sections 5 and 6.2 (Tables 17-19). Human evaluation samples only a subset, so most benchmark scoring relies on this assumption.
  • domain assumption The frozen LLaVA-v1.6-vicuna-7b backbone is a competent vision-language model whose pretrained knowledge is appropriate for personalization.
    Used throughout Sections 4 and 5. The paper does not test whether results transfer to other backbones.
  • domain assumption CoT QA pairs generated by GPT-4o are valid chain-of-thought reasoning steps, not merely answer restatements.
    Section 3, Eq. (3). Section C reports GPT-4o judged 12% of sampled answers as not step-by-step, so this axiom is partially violated.
invented entities (1)
  • Reasoning tokens (entity tokens and relation tokens)
    purpose: Represent personalized concepts and their relations in the hard-prompting scheme.
    These are per-set learnable embeddings for graph nodes and edges, a new model component rather than an externally verifiable entity. The ablation in Section 6.3 shows small differences between prompting modes, so the unique contribution of these tokens is not firmly isolated.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ReGraP-LLaVA: Reasoning enabled Graph-based Personalized Large Language and Vision Assistant." pith.science (2026). https://pith.science/paper/LVVZIA52

@misc{pith2026250503654,
  author       = {Pith},
  title        = {Pith review of: ReGraP-LLaVA: Reasoning enabled Graph-based Personalized Large Language and Vision Assistant},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LVVZIA52}},
  note         = {Machine review of arXiv:2505.03654}
}
read the original abstract

Recent advances in personalized MLLMs enable effective capture of user-specific concepts, supporting both recognition of personalized concepts and contextual captioning. However, humans typically explore and reason over relations among objects and individuals, transcending surface-level information to achieve more personalized and contextual understanding. To this end, existing methods may face three main limitations: Their training data lacks multi-object sets in which relations among objects are learnable. Building on the limited training data, their models overlook the relations between different personalized concepts and fail to reason over them. Their experiments mainly focus on a single personalized concept, where evaluations are limited to recognition and captioning tasks. To address the limitations, we present a new dataset named ReGraP, consisting of 120 sets of personalized knowledge. Each set includes images, KGs, and CoT QA pairs derived from the KGs, enabling more structured and sophisticated reasoning pathways. We propose ReGraP-LLaVA, an MLLM trained with the corresponding KGs and CoT QA pairs, where soft and hard graph prompting methods are designed to align KGs within the model's semantic space. We establish the ReGraP Benchmark, which contains diverse task types: multiple-choice, fill-in-the-blank, True/False, and descriptive questions in both open- and closed-ended settings. The proposed benchmark is designed to evaluate the relational reasoning and knowledge-connection capability of personalized MLLMs. We conduct experiments on the proposed ReGraP-LLaVA and other competitive MLLMs. Results show that the proposed model not only learns personalized knowledge but also performs relational reasoning in responses, achieving the SoTA performance compared with the competitive methods. All the codes and datasets are released at: https://github.com/xyfyyds/ReGraP.

Figures

Figures reproduced from arXiv: 2505.03654 by the authors.

Figure 1
Figure 1. The comparison between ReGraP-LLaVA and other personalized MLLMs. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. The data generation pipeline. We first construct knowledge graph that represents the [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. The framework of ReGraP-LLaVA. The left side shows the framework to soft prompt [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (3 more)
Figure 4
Figure 4. Figure 4: The ablation study of the number of personalized objects. [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: The ablation study of the length of answers in CoT QA pairs. [PITH_FULL_IMAGE:figures/full_fig_p013_5.png]
Figure 6
Figure 6. Figure 6: The human evaluation results. The numbers are the counts of each case. [PITH_FULL_IMAGE:figures/full_fig_p014_6.png]

Discussion (0). Continue with ORCID to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. ForgetMe: Evaluating Selective Forgetting in Generative Models

    cs.CV 2025-04 reject novelty 5.0 of 10

    The authors propose the ForgetMe dataset and the Entangled metric to evaluate selective unlearning in diffusion models, using SAM, CLIP, GPT-4o, and LaMa to build paired original/background images.

Reference graph

Works this paper leans on

42 extracted references · 15 canonical work pages · cited by 1 Pith paper

  1. [1]

    Qwen technical report

    Jinze Bai, Shuai Bai, Yunfei Chu, Zeyu Cui, Kai Dang, Xiaodong Deng, Yang Fan, Wenbin Ge, Yu Han, Fei Huang, et al. Qwen technical report. arXiv preprint arXiv:2309.16609, 2023

    arXiv 2023

  2. [2]

    Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning.Advances in neural information processing systems, 36:34892–34916, 2023

    2023

  3. [3]

    Gemini: a family of highly capable multimodal models

    Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805, 2023

    arXiv 2023

  4. [4]

    Minigpt-4: Enhanc- ing vision-language understanding with advanced large language models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mohamed Elhoseiny. Minigpt-4: Enhanc- ing vision-language understanding with advanced large language models. In ICLR, 2024

    2024

  5. [5]

    Yo'llava: Your personalized language and vision assistant

    Thao Nguyen, Haotian Liu, Yuheng Li, Mu Cai, Utkarsh Ojha, and Yong Jae Lee. Yo'llava: Your personalized language and vision assistant. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Processing Systems, volume 37, pages 40913–40951. Curran Associates, Inc., 2024

    work page 2024

  6. [6]

    Personalized multimodal large language models: A survey

    Junda Wu, Hanjia Lyu, Yu Xia, Zhehao Zhang, Joe Barrow, Ishita Kumar, Mehrnoosh Mirtaheri, Hongjie Chen, Ryan A Rossi, Franck Dernoncourt, et al. Personalized multimodal large language models: A survey. arXiv preprint arXiv:2412.02142, 2024

    arXiv 2024

  7. [7]

    Personalized large vision-language models

    Chau Pham, Hoang Phan, David Doermann, and Yunjie Tian. Personalized large vision-language models. arXiv preprint arXiv:2412.17610, 2024

    arXiv 2024

  8. [8]

    Remember, retrieve and generate: Understanding infinite visual concepts as your personalized assistant

    Haoran Hao, Jiaming Han, Changsheng Li, Yu-Feng Li, and Xiangyu Yue. Remember, retrieve and generate: Understanding infinite visual concepts as your personalized assistant. arXiv preprint arXiv:2410.13360, 2024

    arXiv 2024

Show all 42 references
  1. [9]

    Myvlm: Personalizing vlms for user-specific queries

    Yuval Alaluf, Elad Richardson, Sergey Tulyakov, Kfir Aberman, and Daniel Cohen-Or. Myvlm: Personalizing vlms for user-specific queries. In European Conference on Computer Vision, pages 73–91. Springer, 2024

    2024

  2. [10]

    Structure-mapping: A theoretical framework for analogy

    Dedre Gentner. Structure-mapping: A theoretical framework for analogy. Cognitive Science, 7(2):155–170, 1983

    1983

  3. [11]

    Aminoff, Shira Baror, Eric W

    Elissa M. Aminoff, Shira Baror, Eric W. Roginek, and Daniel D. Leeds. Contextual associations represented both in neural networks and human behavior. Scientific Reports, 12(1):5570, 2022

    2022

  4. [12]

    Llava-sg: Leveraging scene graphs as visual semantic expression in vision-language models

    Jingyi Wang, Jianzhong Ju, Jian Luan, and Zhidong Deng. Llava-sg: Leveraging scene graphs as visual semantic expression in vision-language models. InICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2025

    2025

  5. [13]

    Llava needs more knowledge: Retrieval augmented natural language generation with knowledge graph for explaining thoracic pathologies

    Ameer Hamza, Abdullah , Yong Hyun Ahn, Sungyoung Lee, and Seong Tae Kim. Llava needs more knowledge: Retrieval augmented natural language generation with knowledge graph for explaining thoracic pathologies. Proceedings of the AAAI Conference on Artificial Intelligence, 39(3):3...

    2025

  6. [14]

    Chain-of-thought prompting elicits reasoning in large language models

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in neural information processing systems, 35:24824–24837, 2022. 10

    2022

  7. [15]

    Llava-o1: Let vision language models reason step-by-step

    Guowei Xu, Peng Jin, Li Hao, Yibing Song, Lichao Sun, and Li Yuan. Llava-o1: Let vision language models reason step-by-step. arXiv preprint arXiv:2411.10440, 2024

    2024 arXiv

  8. [16]

    Demystifying long chain-of-thought reasoning in LLMs

    Edward Yeo, Yuxuan Tong, Xinyao Niu, Graham Neubig, and Xiang Yue. Demystifying long chain-of-thought reasoning in LLMs. In ICLR 2025 Workshop on Navigating and Addressing Data Problems for Foundation Models, 2025

    2025

  9. [17]

    Gpt-4 technical report

    Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023

    2023 arXiv

  10. [18]

    Towards visual-prompt temporal answer ground- ing in instructional video

    Shutao Li, Bin Li, Bin Sun, and Yixuan Weng. Towards visual-prompt temporal answer ground- ing in instructional video. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(12):8836–8853, 2024

    2024

  11. [19]

    Large language models with holistically thought could be better doctors

    Yixuan Weng, Bin Li, Fei Xia, Minjun Zhu, Bin Sun, Shizhu He, Shengping Liu, Kang Liu, Shutao Li, and Jun Zhao. Large language models with holistically thought could be better doctors. In CCF International Conference on Natural Language Processing and Chinese Computing, pages ...

    2024

  12. [20]

    Deepseek-v3 technical report

    Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024

    2024 arXiv

  13. [21]

    Gpt-4 technical report, 2024

    OpenAI. Gpt-4 technical report, 2024

    2024

  14. [22]

    Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966, 2023

    2023 arXiv

  15. [23]

    Hello again! llm-powered personalized agent for long-term dialogue

    Hao Li, Chenghao Yang, An Zhang, Yang Deng, Xiang Wang, and Tat-Seng Chua. Hello again! llm-powered personalized agent for long-term dialogue. NAACL, 2025

    2025

  16. [24]

    Training- free personalization via retrieval and reasoning on fingerprints.arXiv preprint arXiv:2503.18623, 2025

    Deepayan Das, Davide Talon, Yiming Wang, Massimiliano Mancini, and Elisa Ricci. Training- free personalization via retrieval and reasoning on fingerprints.arXiv preprint arXiv:2503.18623, 2025

    2025

  17. [25]

    Personaliza- tion toolkit: Training free personalization of large vision language models

    Soroush Seifi, Vaggelis Dorovatas, Daniel Olmeda Reino, and Rahaf Aljundi. Personaliza- tion toolkit: Training free personalization of large vision language models. arXiv preprint arXiv:2502.02452, 2025

    2025 arXiv

  18. [26]

    Concept-as-tree: Synthetic data is all you need for vlm personalization

    Ruichuan An, Kai Zeng, Ming Lu, Sihan Yang, Renrui Zhang, Huitong Ji, Qizhe Zhang, Yulin Luo, Hao Liang, and Wentao Zhang. Concept-as-tree: Synthetic data is all you need for vlm personalization. arXiv preprint arXiv:2503.12999, 2025

    2025 arXiv

  19. [27]

    Phi-3 technical report: A highly capable language model locally on your phone

    Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, et al. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219, 2024

    2024 arXiv

  20. [28]

    You only look once: Unified, real-time object detection

    Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 779–788, 2016

    2016

  21. [29]

    Gpt-4o system card

    Aaron Hurst, Adam Lerer, Adam P Goucher, Adam Perelman, Aditya Ramesh, Aidan Clark, AJ Ostrow, Akila Welihinda, Alan Hayes, Alec Radford, et al. Gpt-4o system card. arXiv preprint arXiv:2410.21276, 2024

    2024 arXiv

  22. [30]

    Lora: Low-rank adaptation of large language models

    Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, Weizhu Chen, et al. Lora: Low-rank adaptation of large language models. ICLR, 1(2):3, 2022

    2022

  23. [31]

    Adam: A method for stochastic optimization

    Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. 11

    2014 arXiv

  24. [32]

    Llava-v1.6-vicuna-7b

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Llava-v1.6-vicuna-7b. In https: // huggingface. co/ liuhaotian/ llava-v1. 6-vicuna-7b . Huggingface, 2023

    2023

  25. [33]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024

    2024 arXiv

  26. [34]

    Qwen2.5-vl, January 2025

    Qwen Team. Qwen2.5-vl, January 2025

    2025

  27. [35]

    Improved baselines with visual instruction tuning

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 26296–26306, 2024

    2024

  28. [36]

    Qwen2.5-vl-72b

    Qwen team. Qwen2.5-vl-72b. In https: // huggingface. co/ Qwen/ Qwen2. 5-VL-72B-Instruct . Huggingface, 2025

    2025

  29. [37]

    Long is more for alignment: A simple but tough-to-beat baseline for instruction fine-tuning

    Hao Zhao, Maksym Andriushchenko, Francesco Croce, and Nicolas Flammarion. Long is more for alignment: A simple but tough-to-beat baseline for instruction fine-tuning. In Ruslan Salakhutdinov, Zico Kolter, Katherine Heller, Adrian Weller, Nuria Oliver, Jonathan Scarlett, and Fe...

    2024

  30. [38]

    Multimodal chain-of-thought reasoning in language models

    Zhuosheng Zhang, Aston Zhang, Mu Li, hai zhao, George Karypis, and Alex Smola. Multimodal chain-of-thought reasoning in language models. Transactions on Machine Learning Research, 2024

    2024

  31. [39]

    Compositional chain-of- thought prompting for large multimodal models

    Chancharik Mitra, Brandon Huang, Trevor Darrell, and Roei Herzig. Compositional chain-of- thought prompting for large multimodal models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 14420–14431, 2024

    2024

  32. [40]

    Visual cot: Advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning

    Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, and Hongsheng Li. Visual cot: Advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning. In A. Globerson, L. Mackey, D. Bel- grave, A. Fan, ...

    2024

  33. [41]

    Kam-cot: Knowledge augmented multimodal chain-of-thoughts reasoning

    Debjyoti Mondal, Suraj Modi, Subhadarshi Panda, Rituraj Singh, and Godawari Sudhakar Rao. Kam-cot: Knowledge augmented multimodal chain-of-thoughts reasoning. In Proceedings of the AAAI conference on artificial intelligence, volume 38, pages 18798–18806, 2024

    2024

  34. [42]

    response 1 is better

    Yihe Deng, Chenchen Ye, Zijie Huang, Mingyu Derek Ma, Yiwen Kou, and Wei Wang. Graphvis: Boosting llms with visual knowledge graph integration. In A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang, editors, Advances in Neural Information Process...

    2024

Pith tools

Reviewed August 15, 2026 · model on record in the stance chip above.