Pith. sign in

REVIEW 5 major objections 6 minor 26 references

This paper introduces ViLD, an enterprise-grounded benchmark for generative vision-language models, and reports that a LoRA-finetuned 7B model matches or exceeds a 32B model on logo, OCR, and media description tasks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

VLM-in-the-Wild provides an enterprise-focused benchmark and the BlockWeaver OCR matching algorithm, reporting that a small fine-tuned model can rival a 32B model on some tasks.

T0 review reviewed 2026-08-05 challenge →

load-bearing objection Useful benchmark scaffolding and a clever OCR matcher, but the headline numbers rest on a possible train/test leak and an unvalidated Gemini-built ground truth. the 5 major comments →

arxiv 2509.06994 v1 pith:DSP3HJQP submitted 2025-09-03 cs.CV cs.CL

VLMs-in-the-Wild: Bridging the Gap Between Academic Benchmarks and Enterprise Reality

Srihari Bandraupalli , Anupam Purwar This is my paper
classification cs.CV cs.CL
keywords vision-language modelsenterprise benchmarkingOCR evaluationLLM-as-judgeLoRA fine-tuningsocial media analysismultimodal evaluationgenerative VLM
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Current VLM benchmarks, built on multiple-choice questions and synthetic data, do not tell an enterprise whether a model will handle real social-media content. This paper proposes ViLD, an evaluation framework built around ten business tasks—logo detection, OCR, object detection, human and demographic analysis, activity and appearance, scene detection, camera and quality assessment, dominant colors, comprehensive description, and NSFW detection—judged against a 7,898-sample dataset stratified from one million real-world posts. To grade free-form generative outputs, ViLD replaces bounding boxes with a spatio-temporal grid, adds the BlockWeaver algorithm for matching unordered OCR blocks without embeddings or LLM calls, and uses a Key Information Units (KIU) framework to measure completeness and faithfulness of descriptions. The benchmark results show that a 7B model fine-tuned with LoRA on just 2,000 samples can match or beat a 32B model on logo, OCR, and media description tasks, suggesting small domain-adapted models may be enough for many enterprise workloads.

Core claim

ViLD is presented as the first systematic, enterprise-grounded methodology for evaluating generative VLMs. Rather than bounding boxes, it uses a 3×3 spatial grid for images and interval-based temporal tracking for videos, matching how generative models describe locations. The BlockWeaver algorithm solves the many-to-many OCR block matching problem through a coverage-score function, greedy assignment, and super-block formation, enabling standard CER/WER metrics at O(n×m) cost. The KIU algorithm decomposes descriptions into atomic verifiable units to compute separate completeness and faithfulness scores. On the constructed dataset, the paper reports that MIMO-SFT-7B is the strongest 7B model o

What carries the argument

The two load-bearing mechanisms are BlockWeaver and the KIU framework. BlockWeaver converts unordered, variably-grouped OCR outputs into matched super-block pairs using only a substring-coverage score (no embeddings, no LLM calls), with greedy assignment followed by ordered concatenation of prediction fragments and then of ground-truth fragments. KIU splits predicted and reference descriptions into atomic factual units and semantically matches them to separately report completeness (recall of ground-truth units) and faithfulness (precision of predicted units). Around these sits the spatio-temporal grid: a 3×3 position grid for images and start/mid/end/interval tags for videos, replacing boun

Load-bearing premise

The correctness labels were produced by Gemini 2.5 Flash, an unverified proprietary model; every precision, recall, completeness, and faithfulness score measures agreement with that one annotator, not with independent visual facts.

What would settle it

Take a random sample of a few hundred of the 7,898 benchmark items, have trained human annotators apply the same JSON schema, and recompute model rankings against the human labels; if the gap between the LoRA-tuned 7B model and the 32B model shrinks or reverses, the central result is an artifact of Gemini's annotation style.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

If this is right

  • LoRA fine-tuning on 2,000 curated samples lets a 7B model match or exceed a 32B model on logo, OCR, and media description tasks, so domain adaptation can substitute for raw scale in some enterprise workloads.
  • BlockWeaver makes OCR evaluation of generative VLMs cheap enough to run at scale, since it avoids embedding similarity and LLM-as-judge calls.
  • Fine-tuning sharpens task accuracy but lowers reliability—most acutely on video—so deployment decisions must weigh the reliability-accuracy trade-off.
  • The persistent video gap across all models indicates that video understanding remains an open problem for open-source VLMs, regardless of model size or adaptation.
  • The 3×3 grid and temporal intervals provide a localization standard that does not penalize generative outputs for lacking pixel coordinates.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Because all ground truth is Gemini-generated, the reported scores likely reward models whose output style aligns with Gemini's; a human-verified subset could reveal annotation-driven biases in the rankings.
  • The BlockWeaver coverage-matching idea is general: any evaluation that must align two ordered-but-differently-segmented sequences (e.g., ASR transcripts, structured extraction) could reuse it.
  • If the 2,000-sample LoRA result generalizes, it suggests enterprises can build competitive task-specific VLMs without large proprietary backbones, and that benchmark-driven data curation is the lever.
  • The ViLD task suite is built from social-media streams; adapting the taxonomy (logo to storefront, NSFW to compliance, activity to customer behavior) could transfer to retail, healthcare, or industrial inspection, though the dataset itself would need rebuilding.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

5 major / 6 minor

Summary. The paper introduces ViLD, a framework and benchmark for evaluating generative vision-language models (VLMs) on ten enterprise-oriented tasks, using a corpus of ~7,500 real-world images/videos plus 500 NSFW samples. A spatio-temporal grid is proposed for localization without bounding boxes, two matching algorithms are presented (BlockWeaver for OCR block alignment and KIU for evaluating descriptive completeness/faithfulness), and three open-source 7B VLMs, a 32B VLM, and a 7B LoRA-fine-tuned variant are benchmarked. The central claim is that LoRA fine-tuning on only 2,000 samples lets Qwen2.5-VL-7B match or exceed Qwen2.5-VL-32B on logo, OCR, and media description tasks (Table II), alongside a reliability–accuracy trade-off. The manuscript includes machine-readable evaluation methodology and a deterministic OCR matcher, but the empirical grounding has several unresolved load-bearing gaps.

Significance. If validated, ViLD would be a useful step toward enterprise-oriented VLM evaluation: the task taxonomy is practical, the spatio-temporal grid is a reasonable adaptation to generative outputs, and BlockWeaver provides a deterministic, O(n×m) OCR-alignment method that avoids expensive LLM/embedding calls. The KIU decomposition for completeness and faithfulness is also a sensible direction. However, the current manuscript does not yet provide sufficient evidence for its headline conclusions. All ground-truth annotations come from a single proprietary VLM (Gemini 2.5 Flash) with no human verification, the 'reliability' column in Table II is never defined, NSFW results are absent despite being a listed task, the LoRA training set may overlap the evaluation set, and no uncertainty/error bars accompany any comparison. These issues directly affect the validity of the benchmark as an independent measure and the LoRA-vs-32B claim. The contributions are promising but the empirical evidence needs substantial strengthening before the paper can be accepted.

major comments (5)
  1. [§II-D, §III-B, Table II] Ground truth annotations are generated exclusively by Gemini 2.5 Flash with no human validation, and model outputs are then matched using LLM-based semantic matching. Consequently, all reported precision/recall/F1/completeness/faithfulness scores measure agreement with Gemini's annotations, not with an independent visual ground truth. This is load-bearing for the benchmark's validity. The authors acknowledge 'LLM-as-judge' limitations but do not quantify them. Please add a human-validated subset (e.g., a few hundred samples per task) with inter-annotator agreement, and report the agreement of Gemini annotations with human labels. Without this, the framework cannot support claims such as 'reliable method to assess VLMs.'
  2. [§IV-A, Appendix A] The paper never states that the 2,000 samples used for LoRA fine-tuning are disjoint from the 7,500-sample evaluation set. Section IV says the model was fine-tuned on '2,000 curated enterprise samples,' while Appendix A says 'our curated dataset of 2,000 video samples.' Both sets are drawn from the same one-million-sample corpus, and the evaluation set includes 1,889 videos. If the fine-tuning set overlaps or shares near-duplicates with the evaluation set, the Table II result that Qwen2.5-VL-7B-LoRA matches/exceeds Qwen2.5-VL-32B is inflated by train/test leakage. This is a necessary condition for the central empirical claim. Please explicitly report the disjointness (e.g., sample IDs or hash-based exclusion) and, if overlap exists, rerun the evaluation on a truly held-out subset.
  3. [Table II] The 'Reliability' column is a headline metric in Table II and drives the claimed 'reliability–accuracy trade-off' and the finding that fine-tuning 'sharply lowers overall reliability.' However, the paper nowhere defines what 'Reliability' measures, how it is computed, or what scale it uses. This is not a presentation nit: the main findings in §IV-A rely on this column. Please define the metric precisely, including the formula and the data sources (e.g., JSON parse success, schema violations, or output consistency), and report it for all models in Appendix B.
  4. [§II-D, Table II, Appendix B] NSFW detection is listed as one of the ten business-critical tasks and 500 NSFW samples are included in the benchmark, yet no NSFW results appear in Table II or anywhere in the appendices. The paper therefore does not actually benchmark the full ten-task suite it advertises. Please either report the NSFW evaluation results with the same metrics as other tasks, or explicitly state why the task is excluded from the reported experiments. As it stands, the claim of a 'comprehensive' ten-task evaluation is unsupported.
  5. [Table II, Appendix B] All reported metrics are point estimates with no confidence intervals, standard deviations, or inter-run variability. Generative VLM outputs are stochastic, and LLM-as-a-judge matching is itself stochastic, so small differences in Table II (e.g., Qwen2.5-VL-7B-LoRA image OCR F1 0.87 vs. MIMO-SFT 0.83) may be within noise. The absence of error bars or significance testing makes the cross-model ordering and the headline LoRA-vs-32B comparison unverifiable. Please provide uncertainty estimates, e.g., by repeating evaluation runs or bootstrapping over samples, and state the number of runs.
minor comments (6)
  1. [Title/Abstract] The title uses 'VLMs-in-the-Wild' while the abstract and body use 'VLM-in-the-Wild'; please make the naming consistent.
  2. [Abstract] The phrase 'To this framework, we bring an innovative BlockWeaver Algorithm' is grammatically awkward; consider revising for clarity.
  3. [§IV vs Appendix A] Section IV says LoRA was fine-tuned on '2,000 curated enterprise samples,' but Appendix A says 'our curated dataset of 2,000 video samples.' Clarify whether the fine-tuning set is images, videos, or a mix; this matters for interpreting the video results.
  4. [Table II] The column 'Media F1' in Table II appears to correspond to 'Description (F1)' in Table VIII; please use consistent terminology and indicate whether Media F1 is the same as the description F1 from the KIU evaluation.
  5. [Eq. (1), §III-F] Coverage-Score(A,B) as defined is asymmetric with respect to A and B. The paper should state explicitly whether the score is computed with the shorter text as the numerator and how asymmetry is handled in Steps 1–3, since the threshold is then applied to different quantities.
  6. [References] Reference [22] is a ResearchGate preprint with a DOI-like handle; consider citing the published venue or a more stable version. Also, the footnote for the corresponding author omits an email domain consistently (e.g., 'anupam.aiml@gmail.com' is fine, but check formatting).

Circularity Check

0 steps flagged

No significant circularity: the benchmark results are empirical comparisons, not derivations from the ground-truth generator; minor self-citations are not load-bearing.

full rationale

The paper's central claims are empirical measurements: Table II reports F1-style scores obtained by running open-source VLMs and a LoRA-adapted model against a fixed annotation schema. These scores are not algebraically forced by the ground-truth generation process; they depend on the models' actual outputs. The closest issue is that the ground truth is generated by Gemini 2.5 Flash (Section II-D: 'Ground truth annotations conforming to our detailed JSON schema were generated using a powerful proprietary VLM (Gemini 2.5 Flash)') and the evaluation uses LLM-based semantic matching. Consequently, metrics like completeness and faithfulness (Eqs. 4-5) measure agreement with Gemini's annotations rather than with independent human labels. This is a real external-validity limitation and is acknowledged in the paper ('we acknowledge the limitations of an LLM-as-judge methodology'), but it is not a circular reduction in the sense used here: the models under evaluation are not Gemini, and the metric equations do not substitute a model's own outputs as ground truth. The two self-citations ([25], [26]) support generic methodological claims about enterprise evaluation and LLM-as-judge practice; they are not the load-bearing justification for the benchmark results, which stand on the reported experiments. A separate data-integrity concern is that the 2,000-sample LoRA fine-tuning set is never stated to be disjoint from the ~7,500-sample evaluation set; this could inflate the headline comparison, but it is a missing-evidence/leakage risk, not a circularity by construction. Overall, no formal circularity is exhibited, so the score is low.

Axiom & Free-Parameter Ledger

1 free parameters · 5 axioms · 0 invented entities

The central evaluation rests on model-generated annotations and model-based judgment, with no human-verified ground truth. The benchmark's validity depends on these assumptions.

free parameters (1)
  • BlockWeaver threshold tau = 0.30
    Threshold for coverage-score-based block matching in Eq. (1)-(3); chosen by hand, no sensitivity analysis.
axioms (5)
  • domain assumption Gemini 2.5 Flash annotations are accurate ground truth
    Section II-D uses Gemini 2.5 Flash to generate all ground-truth JSON; if these annotations contain errors or biases, every metric in Tables II and V-IX inherits them.
  • domain assumption LLM-as-judge matching correctly determines semantic correspondence
    Section III-B relies on LLM prompts for entity matching; validation is only relative to embedding-based matching, not to human agreement.
  • domain assumption The 3x3 spatial grid and start/mid/end temporal intervals capture localization sufficiently
    Section II-B adopts this grid as ground truth localization; it is coarse and may miss precise positions, but is treated as sufficient.
  • domain assumption The 7,898-sample benchmark represents the one-million-sample enterprise corpus
    Section II-D describes stratified sampling but provides no formal representativeness analysis.
  • domain assumption The 2,000-sample fine-tuning set is disjoint from the evaluation set
    Section IV fine-tunes on curated enterprise samples from the same corpus without stating disjointness; if overlapping, the LoRA results are inflated.

reviewed 2026-08-05 · how reviews work

0 comments
Cite this review

Pith. "Pith review of VLMs-in-the-Wild: Bridging the Gap Between Academic Benchmarks and Enterprise Reality." pith.science (2026). https://pith.science/paper/DSP3HJQP

@misc{pith2026250906994,
  author       = {Pith},
  title        = {Pith review of: VLMs-in-the-Wild: Bridging the Gap Between Academic Benchmarks and Enterprise Reality},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DSP3HJQP}},
  note         = {Machine review of arXiv:2509.06994}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Open-source Vision-Language Models show immense promise for enterprise applications, yet a critical disconnect exists between academic evaluation and enterprise deployment requirements. Current benchmarks rely heavily on multiple-choice questions and synthetic data, failing to capture the complexity of real-world business applications like social media content analysis. This paper introduces VLM-in-the-Wild (ViLD), a comprehensive framework to bridge this gap by evaluating VLMs on operational enterprise requirements. We define ten business-critical tasks: logo detection, OCR, object detection, human presence and demographic analysis, human activity and appearance analysis, scene detection, camera perspective and media quality assessment, dominant colors, comprehensive description, and NSFW detection. To this framework, we bring an innovative BlockWeaver Algorithm that solves the challenging problem of comparing unordered, variably-grouped OCR outputs from VLMs without relying on embeddings or LLMs, achieving remarkable speed and reliability. To demonstrate efficacy of ViLD, we constructed a new benchmark dataset of 7,500 diverse samples, carefully stratified from a corpus of one million real-world images and videos. ViLD provides actionable insights by combining semantic matching (both embedding-based and LLM-as-a-judge approaches), traditional metrics, and novel methods to measure the completeness and faithfulness of descriptive outputs. By benchmarking leading open-source VLMs (Qwen, MIMO, and InternVL) against a powerful proprietary baseline as per ViLD framework, we provide one of the first industry-grounded, task-driven assessment of VLMs capabilities, offering actionable insights for their deployment in enterprise environments.

Figures

Figures reproduced from arXiv: 2509.06994 by Anupam Purwar, Srihari Bandraupalli.

Figure 1
Figure 1. Figure 1: Overview of our multi-task annotation and localization framework. [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Distribution of posts across different languages in the dataset. [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Overview of the VLM-in-the-Wild evaluation framework used to benchmark vision-language models on real-world media data. [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: The algorithm begins with these raw block sets and [PITH_FULL_IMAGE:figures/full_fig_p006_4.png] view at source ↗
Figure 4
Figure 4. Figure 4: BlockWeaver Algorithm Stepwise Illustration: (a) Example image and associated texts. (b) Sets of ground truth and prediction blocks, showing complex [PITH_FULL_IMAGE:figures/full_fig_p007_4.png] view at source ↗
Figure 5
Figure 5. Figure 5: Training loss progression during LoRA fine-tuning of Qwen2.5-VL [PITH_FULL_IMAGE:figures/full_fig_p011_5.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

26 extracted references · 2 canonical work pages

  1. [1]

    Radford, J

    A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, et al., ”Learning Transferable Visual Models From Natural Language Supervision,” arXiv preprint arXiv:2103.00020, 2021

    Pith/arXiv arXiv 2021

  2. [2]

    C. Jia, Y . Yang, Y . Xia, Y .-T. Chen, Z. Parekh, H. Pham, et al., ”Scaling Up Visual and Vision-Language Representation Learning With Noisy Text Supervision,” arXiv preprint arXiv:2102.05918, 2021

    Pith/arXiv arXiv 2021

  3. [3]

    J. Li, D. Li, S. Savarese, and S. Hoi, ”BLIP-2: Bootstrapping Language- Image Pre-training with Frozen Image Encoders and Large Language Models,” arXiv preprint arXiv:2301.12597, 2023

    Pith/arXiv arXiv 2023

  4. [4]

    W. Dai, J. Li, D. Li, A. M. H. Tiong, J. Zhao, W. Wang, et al., ”InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning,” arXiv preprint arXiv:2305.06500, 2023

    Pith/arXiv arXiv 2023

  5. [5]

    H. Liu, C. Li, Q. Wu, and Y . J. Lee, ”Visual Instruction Tuning,” arXiv preprint arXiv:2304.08485, 2023

    Pith/arXiv arXiv 2023

  6. [6]

    D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny, ”MiniGPT- 4: Enhancing Vision-Language Understanding with Advanced Large Language Models,” arXiv preprint arXiv:2304.10592, 2023

    Pith/arXiv arXiv 2023

  7. [7]

    Z. Chen, J. Wu, W. Wang, W. Su, G. Chen, S. Xing, et al., ”InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual- Linguistic Tasks,” arXiv preprint arXiv:2312.14238, 2024

    Pith/arXiv arXiv 2024

  8. [8]

    X. Yue, Y . Ni, K. Zhang, T. Zheng, R. Liu, G. Zhang, et al., ”MMMU: A Massive Multi-discipline Multimodal Understanding and Reasoning Benchmark for Expert AGI,” arXiv preprint arXiv:2311.16502, 2024

    Pith/arXiv arXiv 2024

  9. [9]

    P. Lu, S. Mishra, T. Xia, L. Qiu, K.-W. Chang, S.-C. Zhu, et al., ”Learn to Explain: Multimodal Reasoning via Thought Chains for Science Question Answering,” in Proc. 36th Conf. Neural Inf. Process. Syst. (NeurIPS), 2022

    work page 2022

  10. [10]

    P. Lu, H. Bansal, T. Xia, J. Liu, C. Li, H. Hajishirzi, et al., ”MathVista: Evaluating Mathematical Reasoning of Foundation Models in Visual Contexts,” arXiv preprint arXiv:2310.02255, 2024

    Pith/arXiv arXiv 2024

  11. [11]

    C. Fu, P. Chen, Y . Shen, Y . Qin, M. Zhang, X. Lin, et al., ”MME: A Comprehensive Evaluation Benchmark for Multimodal Large Language Models,” arXiv preprint arXiv:2306.13394, 2024

    Pith/arXiv arXiv 2024

  12. [12]

    Y . Liu, H. Duan, Y . Zhang, B. Li, S. Zhang, W. Zhao, et al., ”MMBench: Is Your Multi-modal Model an All-around Player?,” arXiv preprint arXiv:2307.06281, 2024

    Pith/arXiv arXiv 2024

  13. [13]

    Y . Li, Y . Du, K. Zhou, J. Wang, W. X. Zhao, and J.-R. Wen, ”Evaluating Object Hallucination in Large Vision-Language Models,” arXiv preprint arXiv:2305.10355, 2023

    Pith/arXiv arXiv 2023

  14. [14]

    Singh, V

    A. Singh, V . Natarajan, M. Shah, Y . Jiang, X. Chen, D. Batra, et al., ”To- wards VQA Models That Can Read,” arXiv preprint arXiv:1904.08920, 2019

    Pith/arXiv arXiv 1904

  15. [15]

    Mathew, D

    M. Mathew, D. Karatzas, and C. V . Jawahar, ”DocVQA: A Dataset for VQA on Document Images,” arXiv preprint arXiv:2007.00398, 2021

    Pith/arXiv arXiv 2007

  16. [16]

    Masry, D

    A. Masry, D. X. Long, J. Q. Tan, S. Joty, and E. Hoque, ”ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning,” arXiv preprint arXiv:2203.10244, 2022

    Pith/arXiv arXiv 2022

  17. [17]

    Kembhavi, M

    A. Kembhavi, M. Salvato, E. Kolve, M. Seo, H. Hajishirzi, and A. Farhadi, ”A Diagram Is Worth A Dozen Images,” arXiv preprint arXiv:1603.07396, 2016

    Pith/arXiv arXiv 2016

  18. [18]

    C. Fu, Y . Dai, Y . Luo, L. Li, S. Ren, R. Zhang, et al., ”Video-MME: The First-Ever Comprehensive Evaluation Benchmark of Multi-modal LLMs in Video Analysis,” arXiv preprint arXiv:2405.21075, 2025

    Pith/arXiv arXiv 2025

  19. [19]

    Redmon, S

    J. Redmon, S. Divvala, R. Girshick and A. Farhadi, ”You Only Look Once: Unified, Real-Time Object Detection,” 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV , USA, 2016, pp. 779-788, doi: 10.1109/CVPR.2016.91. keywords: Object detection;Real-time systems;Convolutional neural networks;Computer vision;YOLO;Unified arch...

    doi:10.1109/cvpr.2016.91 2016

  20. [20]

    S. Wang, D. Kim, A. Taalimi, C. Sun and W. Kuo, ”Learning Visual Grounding from Generative Vision and Language Model,” 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (W ACV), Tucson, AZ, USA, 2025, pp. 8057-8067, doi: 10.1109/W ACV61041.2025.00782. keywords: Training;Visualization;Image segmentation;Computer vision;Grounding;Crops;Obje...

    arXiv 2025

  21. [21]

    Z. Tan, D. Li, S. Wang, A. Beigi, B. Jiang, A. Bhattacharjee, M. Karami, J. Li, L. Cheng and H. Liu, ”Large Language Models for Data Annotation and Synthesis: A Survey,” Proceedings of the 2024 Confer- ence on Empirical Methods in Natural Language Processing (EMNLP), Miami, Florida, USA, 2024, pp. 930–957, doi: 10.18653/v1/2024.emnlp- main.54. keywords: L...

    doi:10.18653/v1/2024.emnlp- 2024

  22. [22]

    Li, Zongxia & Wu, Xiyang & Du, Hongyang & Liu, Fuxiao & Nghiem, Huy & Shi, Guangyao. (2025). A Survey of State of the Art Large Vision Language Models: Alignment, Benchmark, Evaluations and Challenges. 10.13140/RG.2.2.14334.91209

    work page arXiv 2025

  23. [23]

    S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, H. Zhong, Y . Zhu, M. Yang, Z. Li, J. Wan, P. Wang, W. Ding, Z. Fu, Y . Xu, J. Ye, X. Zhang, T. Xie, Z. Cheng, H. Zhang, Z. Yang, H. Xu, and J. Lin, ”Qwen2.5-VL Technical Report,” arXiv preprint arXiv:2502.13923, 2025

    Pith/arXiv arXiv 2025

  24. [24]

    Core Team, Z. Yue, Z. Lin, Y . Song, W. Wang, S. Ren, S. Gu, S. Li, P. Li, L. Zhao, L. Li, K. Bao, H. Tian, H. Zhang, G. Wang, D. Zhu, Cici, C. He, B. Ye, B. Shen, Z. Zhang, Z. Jiang, Z. Zheng, Z. Song, Z. Luo, Y . Yu, Y . Wang, Y . Tian, Y . Tu, Y . Yan, Y . Huang, X. Wang, X. Xu, X. Song, X. Zhang, X. Yong, X. Zhang, X. Deng, W. Yang, W. Ma, W. Lv, W. Z...

    Pith/arXiv arXiv 2025

  25. [25]

    KnowsLM: A framework for evaluation of small language models for knowledge augmentation and humanised conversations,

    Chitranshu Harbola and Anupam Purwar, “KnowsLM: A framework for evaluation of small language models for knowledge augmentation and humanised conversations,” arXiv preprint arXiv:2504.04569, Apr. 2025

    Pith/arXiv arXiv 2025

  26. [26]

    Gautam and A

    B. Gautam and A. Purwar, Evaluating the Efficacy of Open-Source LLMs in Enterprise-Specific RAG Systems: A Comparative Study of Performance and Scalability, arXiv preprint arXiv:2406.11424, 2024. APPENDIX A. Fine-tuning Details We fine-tuned the Qwen2.5-VL-7B-Instruct model using Low-Rank Adaptation (LoRA) on our curated dataset of 2,000 video samples. Th...

    Pith/arXiv arXiv 2024

This paper was first reviewed by deepseek-v4-flash on August 5, 2026.