Pith. sign in

REVIEW 5 major objections 6 minor 52 references

ComprehendEdit: A Comprehensive Dataset and Evaluation Framework for Multimodal Knowledge Editing

T0 review · 5 major / 6 minor · reviewed 2026-08-11 · deepseek-v4-flash

Pith's one-line read The paper argues that current multimodal knowledge editing evaluations are incomplete and biased, and that new in-domain metrics reveal hidden damage to nearby knowledge that older metrics overlook.

desk verdict Useful benchmark and metrics for in-domain multimodal editing evaluation, but the reported numbers contain an internal range inconsistency that makes the central HICE claim unverifiable as printed. read the letter →

arxiv 2412.12821 v1 pith:N6563AT6 submitted 2024-12-17 cs.CV

classification cs.CV
keywords multimodalknowledgeeditingbenchmarkin-domaingeneralizationpreservationin-contextlearningvisualquestionansweringevaluationmetrics
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Knowledge editing for multimodal models is usually judged by whether the edited answer changes, whether rephrased questions still get the new answer, and whether unrelated questions are unchanged. The paper argues this misses what matters most: what happens to nearby, same-domain knowledge. To close that gap it builds ComprehendEdit, an eight-task benchmark of visual question answering, and adds two metrics computed on real in-domain samples—Knowledge Generalization Index (does the edit help other samples the model previously got wrong?) and Knowledge Preservation Index (does the edit preserve samples the model previously got right?). On this benchmark, methods that looked strong in earlier evaluations (fine-tuning, IKE, SERAC, MEND) score poorly on the new metrics, while the paper's baseline, Hierarchical In-Context Editing (HICE), matches prior methods on older metrics and balances the new ones. If these results hold, future editing methods need to be judged on in-domain side effects, not only on editing accuracy and out-of-domain locality.

What carries the argument

ComprehendEdit supplies the testbed: 17,932 samples across eight tasks built from five source datasets, with a 3:1 train/test split and rephrased questions plus text and multimodal locality samples attached to each editing sample. The two proposed metrics partition the in-domain samples of each edited sample by the original model's behavior: $D_{KGI}(s)$ holds samples the original model answered incorrectly and $D_{KPI}(s)$ holds samples it answered correctly, with KGI and KPI averaging edited-model accuracy over those sets. To keep this affordable, the paper samples the $k=4$ nearest and farthest neighbors of each edited sample by CLIP feature distance, yielding image-based and text-based variants I-KGI/I-KPI and T-KGI/T-KPI. HICE is the method: a pretrained text encoder $h$, a random projection $W_r$ to dimension $M=10{,}000$, a ridge classifier $W^*=(F_p^\top F_p+\lambda I)^{-1}F_p^\top Y$, a text memory $M_1$ of clustered demonstrations, and a memory $M_2$ of hard external samples, so that test inputs are either answered with $k_0=16$ retrieved demonstrations appended to the prompt or passed to the original model unchanged.

What would settle it

Run KGI and KPI exhaustively over all in-domain samples rather than only the $k=4$ nearest and farthest neighbors for several editing methods. If the relative rankings change, or if a method the paper reports as weak matches HICE under full evaluation, the central claim fails.

Watch

Extended reading notes

Core claim

The paper's central claim is that the standard evaluation of multimodal knowledge editing is incomplete and that the omission changes the verdict on real methods. Editing a model to answer 'parrot' instead of 'eagle' for one bird image is only part of the task; the edit should also help neighboring bird images that were misclassified and must not flip neighboring bird images that were correct. The paper reports that current methods fail on these two behaviors: fine-tuning overfits to the edited sample, IKE's demonstrations do not transfer to neighbors, SERAC underperforms on multimodal locality, and MEND's distilled loss, which uses only out-of-domain data, leaves in-domain generalization weak. HICE is reported to restore the balance by classifying each input as in-scope or out-of-scope and applying retrieved in-context demonstrations only to in-scope inputs, leaving the model parameters unchanged.

Load-bearing premise

The load-bearing premise is that four nearest and four farthest in-domain neighbors of each edited sample, chosen by a pretrained similarity measure, represent how editing affects the whole in-domain set; if that small sample is unrepresentative, the reported findings about which methods fail may not generalize.

Editorial extensions

If this is right

  • Editing methods that score well on existing metrics can still silently corrupt in-domain knowledge, so benchmarks should measure KGI and KPI alongside reliability, generality, and locality.
  • HICE demonstrates a parameter-free editing route: a scope classifier plus in-context demonstrations can preserve out-of-domain behavior and improve in-domain measures at the same time.
  • The reported KGI results imply that a correct edit does not automatically transfer to similar misclassified samples, so future methods need mechanisms for propagating edits to neighbors.
  • The reported KPI results imply that edits can flip previously correct answers on nearby samples, so preservation of correct in-domain behavior should be a first-class objective.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If KGI and KPI were computed exhaustively rather than on the four nearest and farthest neighbors, the numerical rankings could shift, and HICE's reported lead might shrink or grow; the paper does not provide a convergence check.
  • The KGI/KPI split depends on which base model defines 'correct' and 'incorrect', so the metrics are not model-independent; a different base model would assign samples to the two subsets differently.
  • The same in-domain evaluation idea could be applied to language-only knowledge editing, where the distinction between generalization and preservation at the distribution level is currently less emphasized.
  • The observed trade-off between parametric editing methods' strong reliability and their weak in-domain preservation suggests a possible structural limit: methods that change weights to fit one fact tend to disturb neighboring facts, whereas memory or in-context methods avoid this at the cost of prompt sensitivity.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper introduces ComprehendEdit, a multimodal knowledge editing benchmark covering eight tasks derived from five datasets (GQA, TallyQA, VSR, TextVQA, MathVista). It proposes two new metrics, Knowledge Generalization Index (KGI) and Knowledge Preservation Index (KPI), to measure editing effects on in-domain samples. It also presents Hierarchical In-Context Editing (HICE), a two-stage method that first classifies inputs as in-domain or out-of-domain and then applies in-context editing only to in-domain inputs. The authors compare FT-V, FT-L, IKE, SERAC, and MEND across conventional metrics and the new KGI/KPI metrics, reporting that existing methods struggle on the new metrics while HICE achieves more balanced performance.

Significance. If the results are reliable, this work fills a genuine gap: current multimodal knowledge editing evaluation (e.g., MMEdit) relies heavily on synthetic content and out-of-domain locality, ignoring in-domain side effects. The KGI/KPI metrics are a reasonable and well-motivated attempt to quantify these effects, and the proposed benchmark broadens task coverage beyond the narrow object-recognition focus of prior evaluations. The HICE baseline is clearly specified and the dataset construction process is documented in detail. However, the quantitative evidence for HICE's superiority is currently undermined by an internal inconsistency in the reported metric ranges and by the unvalidated sampling approximation used to compute KGI/KPI. The significance is therefore conditional on the authors resolving these evaluation issues.

major comments (5)
  1. [Results, Figure 3 caption; Table 2] Figure 3's caption states that I-KGI, T-KGI, I-KPI, and T-KPI lie in [0,16] for BLIP-2 OPT and [0,25] for MiniGPT-4, and Figure 4's caption states [0,40] for ComprehendEdit. However, Table 2 (E-VQA, MiniGPT-4) reports T-KPI = 46.20 for several rows and T-KPI = 46.34 for HICE, and Tables 9-12 report T-KPI values between 45.87 and 46.80 on the same backbone and benchmark. Since MKPI in Eq. (7) is an expectation of an indicator function, its valid range on a percentage scale is [0,100], and a value of 46.20 cannot lie in [0,25]. Either the figure captions misstate the metric maxima, or the table values use a different normalization (e.g., raw counts) that does not match Eq. (7). As written, the quantitative support for the claim that HICE achieves superior and more balanced KPI is not verifiable. Please correct the ranges, state the exact normalization used, and report the numerical values for Figures 3 and 4.
  2. [Proposed Evaluation Metrics, 'Similarity-based Sampling'] Equations (6) and (7) define KGI and KPI as expectations over all in-domain samples in DKGI(s) and DKPI(s). The subsequent paragraph, 'Similarity-based Sampling', replaces these sets with only the k=4 nearest and k=4 farthest neighbors selected by CLIP similarity. The paper neither defines this sampled metric formally nor provides a convergence check, confidence interval, or comparison with full-set evaluation. This is load-bearing because the central finding that existing methods underperform in-domain is based on this 8-neighbor subsample. Please either define the sampled metric explicitly (e.g., KGI_k, KPI_k) and report its uncertainty, or validate the approximation by computing both full-set and sampled estimates on a subset of editing samples.
  3. [Implementation Details; Ablation Study] In 'Implementation Details', HICE's hyperparameters (λ, M, T, k0, memory ratio) are selected using a validation split of the training set. In contrast, the baseline methods (FT-V, FT-L, IKE, SERAC, MEND) are run with fixed settings taken from prior work, as listed in Appendix Tables 5-8. This asymmetry gives HICE a potential advantage, especially on the newly introduced KGI/KPI metrics for which the baselines were not tuned. To ensure a fair comparison, please tune each baseline on the same validation split of ComprehendEdit, or report both fixed and tuned baseline results so the reader can gauge the effect.
  4. [Task Formulation; Similarity-based Sampling] The definition of D(s) as 'samples from the same source dataset as s' is ambiguous regarding whether it includes training samples. If D(s) includes the training split, then KGI/KPI evaluation for HICE can be inflated because HICE's memory M1 and classifier are constructed from that same training split. The paper must specify whether D(s) is test-only or train+test, and must confirm that no in-domain evaluation sample overlaps with the samples used to build HICE's memory or classifier. If overlap exists, the reported KGI/KPI improvements for HICE may partly reflect memorization rather than editing generalization.
  5. [Experimental Results (Figures 3 and 4)] The main results in Figures 3 and 4 are presented as bar charts with single numerical labels and no error bars, standard deviations, or significance tests. Because each test sample is edited independently, the variability across editing samples is expected to be substantial. Please report means with standard deviations (e.g., across editing samples or multiple runs), or at least provide a table of the underlying numbers so that the differences between methods can be assessed quantitatively.
minor comments (6)
  1. [Appendix, Figures 5-7 captions] The captions of Figures 5 and 6 refer to 'Eq. (11)' and 'Eq. (2)', but the main text contains equations numbered up to (9); please renumber or remove these cross-references.
  2. [Table 2 caption] The text mentions a '†' symbol to indicate reproduced results, but the symbol does not appear in Table 2 or its caption; please add the appropriate footnote or remove the reference.
  3. [Results section] There is a typo in 'performannce' (should be 'performance'); also, in the Ablation Study section, 'rephrase questions' should be 'rephrased questions'.
  4. [Ablation Study] The ablation study uses '1 nearest and 1 farthest neighbor' for KGI/KPI, whereas the main experiments use k=4; please clarify whether the conclusions are sensitive to this choice and how the two settings relate.
  5. [Proposed Evaluation Metrics] The phrase 'the original model answered incorrectly (exclude s)' is ambiguous: it is unclear whether s is excluded only from DKGI(s) or from both DKGI(s) and DKPI(s); please state the exclusion rule explicitly.
  6. [Appendix, Table 3] Table 3 is not referenced in the main text; please add a cross-reference so readers know where to find the comparison with E-VQA and KEBench.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: KGI/KPI are defined from ground-truth and original-model predictions, and HICE is trained and evaluated without fitting the target metrics.

full rationale

KGI (Eq. 6) and KPI (Eq. 7) are expectations of indicator functions over D_KGI(s)/D_KPI(s), which are defined by ground-truth labels and the pre-edit model's correctness, not by HICE or by any fitted parameter. HICE's classifier and memories (Eqs. 8-9; M1/M2 construction) are trained on the benchmark's training split, and the reported Rel/T-G/T-L/M-L/KGI/KPI numbers are measurements on the test split and neighbor samples, so no reported 'prediction' is the value of a fitted parameter renamed as a result. The paper cites prior work (Cheng et al., Huang et al., IKE, SERAC, MEND) for baselines and motivations, but no load-bearing premise is justified by a self-citation; the authors' own earlier work is not invoked. The proposal of both metrics and a method aimed at them is a normal benchmark-plus-baseline pattern and does not make the comparison circular. The apparent mismatch between T-KPI values (e.g., 46.20 in Table 2) and the [0,25] range stated in the Figure 3 caption is an internal consistency/verifiability issue, not a circularity. Hence score 0.

Assumptions & free parameters 6 free parameters · 4 assumptions · 0 invented entities

The central claim rests on the KGI/KPI sampling procedure (k nearest/farthest neighbors) and on the CLIP similarity assumption for defining in-domain relatedness. These are model-dependent and subsampling-based, which introduces uncertainty. HICE's hyperparameters are tuned on the validation split, which is standard but should be reported alongside baseline tuning for fair comparison.

free parameters (6)
  • lambda (ridge penalty) = 10^-4 to 10^4 grid, best on validation
    Used in Eq. (9) to solve for classifier W*; selected on validation set.
  • M (projected feature dimension) = 10000
    Random projection dimension for classifier; ablation shows 5000-15000; 10000 chosen.
  • T (similarity threshold) = 0.85 (default from ablation)
    Threshold for classifying a sample as in-domain vs out-of-domain; ablation varies 0.75-0.90.
  • k0 (number of demonstrations) = 16
    Number of similar demonstrations retrieved from memory M1 for in-context editing.
  • memory ratio for M1 = 5%
    Percentage of training samples stored in M1 after k-means clustering; ablation varies 1%, 5%, 20%.
  • k (neighbors for KGI/KPI) = 4
    Number of nearest and farthest neighbors used to estimate KGI and KPI; no sensitivity analysis.
assumptions (4)
  • standard math Least-squares solution with ridge penalty provides a valid classifier for in-domain/out-of-domain discrimination
    Used in Eq. (8)-(9) to compute W*.
  • domain assumption CLIP feature similarity is a reliable measure of semantic relatedness between images and between questions
    Used both for selecting KGI/KPI neighbors and for constructing memory M1 via k-means on CLIP features.
  • domain assumption The split of in-domain samples into DKGI (previously wrong) and DKPI (previously correct) is stable enough to define meaningful metrics
    Depends on the original model's predictions; if the model's errors change, the splits would change.
  • ad hoc to paper Sampling the k nearest and k farthest neighbors estimates the full in-domain editing effect
    Adopted for computational efficiency with no theoretical or empirical justification of representativeness.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ComprehendEdit: A Comprehensive Dataset and Evaluation Framework for Multimodal Knowledge Editing." pith.science (2026). https://pith.science/paper/N6563AT6

@misc{pith2026241212821,
  author       = {Pith},
  title        = {Pith review of: ComprehendEdit: A Comprehensive Dataset and Evaluation Framework for Multimodal Knowledge Editing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/N6563AT6}},
  note         = {Machine review of arXiv:2412.12821}
}
read the original abstract

Large multimodal language models (MLLMs) have revolutionized natural language processing and visual understanding, but often contain outdated or inaccurate information. Current multimodal knowledge editing evaluations are limited in scope and potentially biased, focusing on narrow tasks and failing to assess the impact on in-domain samples. To address these issues, we introduce ComprehendEdit, a comprehensive benchmark comprising eight diverse tasks from multiple datasets. We propose two novel metrics: Knowledge Generalization Index (KGI) and Knowledge Preservation Index (KPI), which evaluate editing effects on in-domain samples without relying on AI-synthetic samples. Based on insights from our framework, we establish Hierarchical In-Context Editing (HICE), a baseline method employing a two-stage approach that balances performance across all metrics. This study provides a more comprehensive evaluation framework for multimodal knowledge editing, reveals unique challenges in this field, and offers a baseline method demonstrating improved performance. Our work opens new perspectives for future research and provides a foundation for developing more robust and effective editing techniques for MLLMs. The ComprehendEdit benchmark and implementation code are available at https://github.com/yaohui120/ComprehendEdit.

Figures

Figures reproduced from arXiv: 2412.12821 by the authors.

Figure 1
Figure 1. Concept of Multimodal Knowledge Editing. The [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Knowledge Distortion in Multimodal Knowledge [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. Performance comparison of knowledge editing methods on E-VQA benchmark. The range of values for Rel, T-G, [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (6 more)
Figure 4
Figure 4. Figure 4: Performance comparison of knowledge editing methods on ComprehendEdit benchmark. The range of values for Rel, [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Illustration of constructing classifier W∗ . We first exact features of questions by pre-trained model h, and then project these features to obtain Fp. Fp are used to calculate W∗ by Eq. (11). OneHot Project to higher dimension by 𝑊𝑟 4𝑁 Classifier Eq. (2) Questions Lab…
Figure 6
Figure 6. Figure 6: Illustration of constructing memories M1 and M2. We use CLIP-extracted features for k-means clustering. Then we randomly select a sample from each class, construct and store the demonstration in M1. We use pre-trained model h and the classifier W∗ to make predictions f…
Figure 7
Figure 7. Figure 7: Illustration of constructing new question [PITH_FULL_IMAGE:figures/full_fig_p011_7.png]
Figure 8
Figure 8. Figure 8: Some examples of ComprehendEdit. Q, G, P, S, C mean Question, Ground-truth, Prediction, Source, task Category independently. culate similarity between their relationships. The relation￾ships in these samples were replaced with their most similar counterparts, and the a…
Figure 9
Figure 9. Figure 9: Results of the edited model on DKGI and DKP I using SERAC. Q, G, P, S mean Question, Ground-truth, Prediction, Source independently. The first row is the performance of edited model on I-KPI and T-KPI, while the second row is the performance of edited model on I-KGI an…

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

52 extracted references · 10 canonical work pages

  1. [1]

    Acharya, M.; Kafle, K.; and Kanan, C. 2019. TallyQA: Answering complex counting questions. In Proceedings of the AAAI conference on artificial intelligence, 01, 8076--8084

  2. [2]

    Bai, J.; Bai, S.; Yang, S.; Wang, S.; Tan, S.; Wang, P.; Lin, J.; Zhou, C.; and Zhou, J. 2023. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond. arXiv preprint arXiv:2308.12966

  3. [3]

    Brown, T. B. 2020. Language models are few-shot learners. arXiv preprint ArXiv:2005.14165

  4. [4]

    Cao, B.; Lin, H.; Han, X.; Sun, L.; Yan, L.; Liao, M.; Xue, T.; and Xu, J. 2021. Knowledgeable or educated guess? revisiting language models as knowledge bases. arXiv preprint arXiv:2106.09231

  5. [5]

    Chen, Q.; Zhang, T.; Li, D.; Huang, L.; Xue, H.; Wang, C.; and He, X. 2024. Lifelong Knowledge Editing for LLMs with Retrieval-Augmented Continuous Prompt Learning. arXiv preprint arXiv:2405.03279

  6. [6]

    Chen, X.; Fang, H.; Lin, T.-Y.; Vedantam, R.; Gupta, S.; Doll \'a r, P.; and Zitnick, C. L. 2015. Microsoft coco captions: Data collection and evaluation server. arXiv preprint arXiv:1504.00325

  7. [7]

    Cheng, S.; Tian, B.; Liu, Q.; Chen, X.; Wang, Y.; Chen, H.; and Zhang, N. 2023. Can We Edit Multimodal Large Language Models? arXiv preprint arXiv:2310.08475

  8. [8]

    Dai, D.; Dong, L.; Hao, Y.; Sui, Z.; Chang, B.; and Wei, F. 2021. Knowledge neurons in pretrained transformers. arXiv preprint arXiv:2104.08696

Show all 52 references
  1. [9]

    De Cao, N.; Aziz, W.; and Titov, I. 2021. Editing factual knowledge in language models. arXiv preprint arXiv:2104.08164

  2. [10]

    Deng, J.; Wei, Z.; Pang, L.; Ding, H.; Shen, H.; and Cheng, X. 2024. UnKE: Unstructured Knowledge Editing in Large Language Models. arXiv preprint arXiv:2405.15349

  3. [11]

    R.; Eisenschlos, J

    Dhingra, B.; Cole, J. R.; Eisenschlos, J. M.; Gillick, D.; Eisenstein, J.; and Cohen, W. W. 2022. Time-aware language models as temporal knowledge bases. Transactions of the Association for Computational Linguistics, 10: 257--273

  4. [12]

    Du, Z.; Qian, Y.; Liu, X.; Ding, M.; Qiu, J.; Yang, Z.; and Tang, J. 2021. Glm: General language model pretraining with autoregressive blank infilling. arXiv preprint arXiv:2103.10360

  5. [13]

    Elazar, Y.; Kassner, N.; Ravfogel, S.; Ravichander, A.; Hovy, E.; Sch \"u tze, H.; and Goldberg, Y. 2021. Measuring and improving consistency in pretrained language models. Transactions of the Association for Computational Linguistics, 9: 1012--1031

  6. [14]

    Fu, C.; Chen, P.; Shen, Y.; Qin, Y.; Zhang, M.; Lin, X.; Yang, J.; Zheng, X.; Li, K.; Sun, X.; et al. 2023. MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models. arXiv preprint arXiv:2306.13394

  7. [15]

    Goyal, Y.; Khot, T.; Summers-Stay, D.; Batra, D.; and Parikh, D. 2017. Making the v in vqa matter: Elevating the role of image understanding in visual question answering. In Proceedings of the IEEE conference on computer vision and pattern recognition, 6904--6913

  8. [16]

    J.; Guo, A.; Lin, C.; Grauman, K.; Luo, J.; and Bigham, J

    Gurari, D.; Li, Q.; Stangl, A. J.; Guo, A.; Lin, C.; Grauman, K.; Luo, J.; and Bigham, J. P. 2018. Vizwiz grand challenge: Answering visual questions from blind people. In Proceedings of the IEEE conference on computer vision and pattern recognition, 3608--3617

  9. [17]

    Han, X.; Li, R.; Li, X.; and Pan, J. Z. 2023. A divide and conquer framework for Knowledge Editing. Knowledge-Based Systems, 279: 110826

  10. [18]

    Hartvigsen, T.; Sankaranarayanan, S.; Palangi, H.; Kim, Y.; and Ghassemi, M. 2024. Aging with grace: Lifelong model editing with discrete key-value adaptors. Advances in Neural Information Processing Systems, 36

  11. [19]

    Hase, P.; Diab, M.; Celikyilmaz, A.; Li, X.; Kozareva, Z.; Stoyanov, V.; Bansal, M.; and Iyer, S. 2023. Methods for measuring, updating, and visualizing factual beliefs in language models. In Proceedings of the 17th Conference of the European Chapter of the Association for Com...

  12. [20]

    Hu, C.; Cao, P.; Chen, Y.; Liu, K.; and Zhao, J. 2024. WilKE: Wise-Layer Knowledge Editor for Lifelong Knowledge Editing. arXiv preprint arXiv:2402.10987

  13. [21]

    Huang, H.; Zhong, H.; Liu, Q.; Wu, S.; Wang, L.; and Tan, T. 2024. KEBench: A Benchmark on Knowledge Editing for Large Vision-Language Models. arXiv preprint arXiv:2403.07350

  14. [22]

    Huang, Z.; Shen, Y.; Zhang, X.; Zhou, J.; Rong, W.; and Xiong, Z. 2023. Transformer-patcher: One mistake worth one neuron. arXiv preprint arXiv:2301.09785

  15. [23]

    A.; and Manning, C

    Hudson, D. A.; and Manning, C. D. 2019. Gqa: A new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 6700--6709

  16. [24]

    Jiang, D.; Liu, Y.; Liu, S.; Zhang, X.; Li, J.; Xiong, H.; and Tian, Q. 2023. From clip to dino: Visual encoders shout in multi-modal large language models. arXiv preprint arXiv:2310.08825

  17. [25]

    Kwiatkowski, T.; Palomaki, J.; Redfield, O.; Collins, M.; Parikh, A.; Alberti, C.; Epstein, D.; Polosukhin, I.; Devlin, J.; Lee, K.; et al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguistics, 7: 453--466

  18. [26]

    X.; and Wen, J.-R

    Li, Y.; Du, Y.; Zhou, K.; Wang, J.; Zhao, W. X.; and Wen, J.-R. 2023. Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10355

  19. [27]

    Liu, F.; Emerson, G.; and Collier, N. 2023. Visual spatial reasoning. Transactions of the Association for Computational Linguistics, 11: 635--651

  20. [28]

    Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2024 a . Visual instruction tuning. Advances in neural information processing systems, 36

  21. [29]

    Liu, H.; Xue, W.; Chen, Y.; Chen, D.; Zhao, X.; Wang, K.; Hou, L.; Li, R.; and Peng, W. 2024 b . A survey on hallucination in large vision-language models. arXiv preprint arXiv:2402.00253

  22. [30]

    Liu, Y.; Duan, H.; Zhang, Y.; Li, B.; Zhang, S.; Zhao, W.; Yuan, Y.; Wang, J.; He, C.; Liu, Z.; et al. 2023. Mmbench: Is your multi-modal model an all-around player? arXiv preprint arXiv:2307.06281

  23. [31]

    Lu, P.; Bansal, H.; Xia, T.; Liu, J.; Li, C.; Hajishirzi, H.; Cheng, H.; Chang, K.-W.; Galley, M.; and Gao, J. 2023. Mathvista: Evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255

  24. [32]

    Marino, K.; Rastegari, M.; Farhadi, A.; and Mottaghi, R. 2019. Ok-vqa: A visual question answering benchmark requiring external knowledge. In Proceedings of the IEEE/cvf conference on computer vision and pattern recognition, 3195--3204

  25. [33]

    D.; Gong, D.; Parvaneh, A.; Abbasnejad, E.; and van den Hengel, A

    McDonnell, M. D.; Gong, D.; Parvaneh, A.; Abbasnejad, E.; and van den Hengel, A. 2024. Ranpac: Random projections and pre-trained models for continual learning. Advances in Neural Information Processing Systems, 36

  26. [34]

    Meng, K.; Bau, D.; Andonian, A.; and Belinkov, Y. 2022. Locating and editing factual associations in GPT. Advances in Neural Information Processing Systems, 35: 17359--17372

  27. [35]

    Mitchell, E.; Lin, C.; Bosselut, A.; Finn, C.; and Manning, C. D. 2021. Fast model editing at scale. arXiv preprint arXiv:2110.11309

  28. [36]

    D.; and Finn, C

    Mitchell, E.; Lin, C.; Bosselut, A.; Manning, C. D.; and Finn, C. 2022. Memory-based model editing at scale. In International Conference on Machine Learning, 15817--15831. PMLR

  29. [37]

    O.; Aljundi, R.; and Turner, R

    Panos, A.; Kobe, Y.; Reino, D. O.; Aljundi, R.; and Turner, R. E. 2023. First session adaptation: A strong replay-free baseline for class-incremental learning. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 18820--18830

  30. [38]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748--8763. PMLR

  31. [39]

    Rawte, V.; Rani, A.; Sharma, H.; Anand, N.; Rajbangshi, K.; Sheth, A.; and Das, A. 2024. Visual hallucination: Definition, quantification, and prescriptive remediations. arXiv preprint arXiv:2403.17306

  32. [40]

    Reimers, N.; and Gurevych, I. 2019. Sentence-bert: Sentence embeddings using siamese bert-networks. arXiv preprint arXiv:1908.10084

  33. [41]

    Rombach, R.; Blattmann, A.; Lorenz, D.; Esser, P.; and Ommer, B. 2022. High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 10684--10695

  34. [42]

    Singh, A.; Natarajan, V.; Shah, M.; Jiang, Y.; Chen, X.; Batra, D.; Parikh, D.; and Rohrbach, M. 2019. Towards vqa models that can read. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 8317--8326

  35. [43]

    Sun, Z.; Liu, Y.; Wang, J.; Meng, F.; Xu, J.; Chen, Y.; and Zhou, J. 2024. Outdated Issue Aware Decoding for Factual Knowledge Editing. arXiv preprint arXiv:2406.02882

  36. [44]

    Tong, S.; Liu, Z.; Zhai, Y.; Ma, Y.; LeCun, Y.; and Xie, S. 2024. Eyes wide shut? exploring the visual shortcomings of multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 9568--9578

  37. [45]

    Touvron, H.; Martin, L.; Stone, K.; Albert, P.; Almahairi, A.; Babaei, Y.; Bashlykov, N.; Batra, S.; Bhargava, P.; Bhosale, S.; et al. 2023. Llama 2: Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

  38. [46]

    Yu, L.; Chen, Q.; Zhou, J.; and He, L. 2024. Melo: Enhancing model editing with neuron-indexed dynamic lora. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, 19449--19457

  39. [47]

    Yu, W.; Yang, Z.; Li, L.; Wang, J.; Lin, K.; Liu, Z.; Wang, X.; and Wang, L. 2023. Mm-vet: Evaluating large multimodal models for integrated capabilities. arXiv preprint arXiv:2308.02490

  40. [48]

    X.; Zhou, K.; Li, J.; Tang, T.; Wang, X.; Hou, Y.; Min, Y.; Zhang, B.; Zhang, J.; Dong, Z.; et al

    Zhao, W. X.; Zhou, K.; Li, J.; Tang, T.; Wang, X.; Hou, Y.; Min, Y.; Zhang, B.; Zhang, J.; Dong, Z.; et al. 2023. A survey of large language models. arXiv preprint arXiv:2303.18223

  41. [49]

    Zheng, C.; Li, L.; Dong, Q.; Fan, Y.; Wu, Z.; Xu, J.; and Chang, B. 2023. Can We Edit Factual Knowledge by In-Context Learning? arXiv preprint arXiv:2305.12740

  42. [50]

    Zhu, D.; Chen, J.; Shen, X.; Li, X.; and Elhoseiny, M. 2023. Minigpt-4: Enhancing vision-language understanding with advanced large language models. arXiv preprint arXiv:2304.10592

  43. [51]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all...

  44. [52]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 gl...

Pith tools

Reviewed August 11, 2026 · model on record in the stance chip above.