Pith

open record

sign in

arxiv: 2607.05859 · v1 · pith:VMPMWSOB · submitted 2026-07-07 · cs.CV

AVA-VLM: Adaptive Visual Attention-Vision Language Model for In-the-Wild Construction Site Monitoring

Younggun Kim , Taeheon Kim , Youngseo Kim , Seunghee Park This is my paper

Reviewed by Pith T0 review T1 audit T2 compute T3 formal T4 kernel 2026-07-08 22:09 UTCglm-5.2pith:VMPMWSOBrecord.jsonopen to challenge →

classification cs.CV
keywords vision-language modelconstruction safetyadaptive visual attentioncoarse-to-fine reasoningvisual token efficiencyPPE violation detectionchain-of-thought
0
0 comments X

The pith

VLM learns to zoom in, cuts visual tokens by 70%

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

A vision-language model for construction-site monitoring that first looks at a downsampled global image and selectively crops high-resolution regions only when needed, improving PPE-violation detection while using less than a third of the visual tokens of standard approaches.

Core claim

The paper introduces a coarse-to-fine reasoning strategy where a VLM trained on a region-aware chain-of-thought dataset learns to decide when local inspection is needed, where to crop, and how to incorporate cropped evidence, achieving 75.1 F1 on PPE-violation identification versus 62.0 for direct-QA baselines while using only 30.6% of the visual-token budget.

What carries the argument

Region-aware CoT dataset, adaptive crop tool, decoupled training

Load-bearing premise

During training, the model is given ground-truth crop regions rather than having to predict them itself, so the framework's success at inference depends entirely on the model's ability to identify the right regions from a low-resolution global image—a capability never directly measured.

What would settle it

If crop-region prediction quality (IoU between predicted and ground-truth crop boxes) is poor at inference, the model either wastes tokens on irrelevant regions or misses critical evidence, collapsing the performance gains.

Figures

Figures reproduced from arXiv: 2607.05859 by Seunghee Park, Taeheon Kim, Younggun Kim, Youngseo Kim.

Figure 1
Figure 1. Figure 1: Overview of the limitations of existing construction-tailored VLMs and the proposed AVA-VLM. [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of the region-aware CoT dataset curation pipeline. Starting from the source construction-site [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 3
Figure 3. Figure 3: Examples of the region-aware CoT dataset. [PITH_FULL_IMAGE:figures/full_fig_p011_3.png] view at source ↗
Figure 4
Figure 4. Figure 4: Framework of AVA-VLM. Top: Training pipeline, where AVA-VLM is trained with ground-truth response traces for both direct-answer and crop-required samples. For crop-required samples, the model first learns to generate a local-inspection request and then learns to produce the final answer using the ground-truth cropped image, which decouples crop-region learning from crop-conditioned answer generation. Botto… view at source ↗
Figure 5
Figure 5. Figure 5: Tool-call usage ratio of AVA-VLM on the VI task, showing selective local inspection across overall samples, [PITH_FULL_IMAGE:figures/full_fig_p019_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Qualitative results of PPE-violation identification, showing how AVA-VLM selectively uses local inspection [PITH_FULL_IMAGE:figures/full_fig_p021_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: Qualitative results of PPE-violation identification under global image downsampling, showing AVA-VLM’s [PITH_FULL_IMAGE:figures/full_fig_p025_7.png] view at source ↗
Figure 8
Figure 8. Figure 8: Tool-call usage ratio of AVA-VLM on the OD task, showing selective local inspection across overall samples, [PITH_FULL_IMAGE:figures/full_fig_p026_8.png] view at source ↗
read the original abstract

Vision-Language Models (VLMs) are promising for construction-site monitoring, and recent construction-tailored VLMs have primarily adapted pretrained VLMs through direct QA-style fine-tuning from a single global image. We argue that this direct paradigm remains limited for in-the-wild deployment in terms of operational range, reliability under reduced-resolution inputs, and inference efficiency. To address these challenges, we propose AVA-VLM, an Adaptive Visual Attention-Vision Language Model that follows a human-inspired coarse-to-fine reasoning strategy. AVA-VLM first reasons over a low-resolution global image and selectively requests a high-resolution local crop only when detailed inspection is needed, similar to how a human inspector zooms in on hard-to-see yet important areas. We further introduce a region-aware Chain-of-Thought dataset that teaches the model when to inspect, where to crop, and how to use local evidence. Experiments show that AVA-VLM improves reliability under long-distance and reduced-resolution conditions while substantially reducing visual-token usage.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

3 major / 7 minor

Summary. The paper proposes AVA-VLM, a construction-site VLM that combines low-resolution global reasoning with selective high-resolution local cropping via a tool-call mechanism. The model is trained on a region-aware CoT dataset derived from ConstructionSite10K, where crop decisions and regions are generated from ground-truth annotations and a YOLO detector. At inference, the model predicts whether to crop and where, using a downsampled global image and mapping predicted crop coordinates back to full resolution. Experiments compare AVA-VLM against direct-QA and example-image CoT baselines on violation identification (VI), object detection (OD), and image captioning (IC), evaluating across camera distances and reduced resolutions. The central claim is that AVA-VLM achieves better F1 on PPE-violation identification (75.1 vs. 62.0 baseline) while using only 30.6% of the visual-token budget, and maintains advantages under reduced resolution and long-distance scenarios.

Significance. The paper addresses a practically important problem: making VLMs more efficient and robust for construction-site monitoring under realistic deployment constraints. The experimental design is commendable—same backbone (Qwen2.5-VL 7B), same dataset, same hyperparameters across all methods, with multiple evaluation axes (camera distance, resolution, per-class metrics). The region-aware CoT dataset construction is a useful contribution, and the selective cropping mechanism is a reasonable approach to the efficiency-accuracy trade-off. The paper is transparent about performance degradation under aggressive downsampling and about the OD trade-off. However, the central claim rests on the model's ability to predict crop regions from downsampled inputs at inference, and this capability is never directly evaluated, which weakens the evidentiary basis for the headline results.

major comments (3)
  1. The paper never directly evaluates crop-region prediction quality (e.g., IoU between predicted crop boxes and ground-truth crop boxes from Algorithms 1–2). The entire framework depends on the model's ability to accurately predict WHERE to crop from a downsampled global image, yet this is only indirectly assessed through downstream task metrics. The IoU metrics in Table 7 measure violation-localization quality, which conflates crop quality with answer-generation quality and only covers VI positive samples where the model predicts a violation. For no-violation samples that trigger cropping and for all OD samples, crop quality is entirely unassessed. Adding a direct crop-box IoU evaluation would substantially strengthen the paper's central claim. (§4.2, Eqs. 14–15; Algorithms 1–2; Table 7)
  2. Tables 4, 7, and 10 have inconsistent or unclear descriptions of the AVA-VLM configuration. Table 4's caption states 'AVA-VLM uses 1/4 downsampled global images,' but Table 6 shows that 1/4 downsampling yields F1=64.5%, while 1/2 downsampling yields F1=75.1%. The text in §5.2.1 says the 1/2 setting is used as the 'default AVA-VLM configuration in the overall comparison reported in Table 4.' This means Table 4's caption is incorrect—it should say 1/2, not 1/4. This inconsistency affects interpretation of the efficiency claims. (Table 4 caption; Table 6; §5.2.1 final paragraph)
  3. The train/inference resolution gap is a load-bearing concern. During training (§4.1, Algorithm 3), the model always receives full-size global images and learns to predict crop boxes from them. At inference (§4.2, Eqs. 14–15), the default configuration uses 1/2-downsampled global images, and the predicted crop box in downsampled coordinates is linearly scaled back to full resolution. The model has never been trained on downsampled inputs, so its crop-prediction behavior in this regime is uncharacterized. Table 6 shows F1 degrades from 82.7% (full-size) to 75.1% (1/2) to 64.5% (1/4), and Figure 5(d) shows tool-call ratio drops from 35.8% to 20.0% as resolution decreases, suggesting the model increasingly fails to trigger crops when it should. The paper should discuss this train/test distribution shift explicitly and consider whether training with augmented downsampled inputs could mitigate
minor comments (7)
  1. Section 3.2: The free parameters (tau_obj_VI, tau_crop_VI, tau_obj_OD, tau_crop_OD, alpha) are used in Algorithms 1–2 but their values are not reported. Please include them in the implementation details or supplementary material.
  2. Table 2: The 'Test images' column (1,360 / 1,310 / 334) does not match the test image count in Table 1 (3,004 total). Please clarify whether Table 2 reports a subset or a different grouping.
  3. Section 5.1: The downsampling factor s is described in §4.2 but its specific values for each Table 6 setting should be stated explicitly in the implementation details for reproducibility.
  4. Figure 5(d) is referenced in the text as showing tool-call ratio vs. resolution, but the figure label says 'reduced global image resolutions' without specifying the exact scales (1/2, 1/4). Consider adding scale labels directly on the figure.
  5. Table 10: AVA-VLM's OD performance (60.3% Avg. IoU) is below the baseline (68.1%). The text acknowledges this trade-off, but the abstract's claim of 'substantially reducing visual-token usage' could be read as implying no accuracy cost. Consider qualifying the abstract to note the OD trade-off.
  6. Reference [12] (Chen and Zou, 2026) and several other references have 2026 dates. Please verify these are correct publication dates and not placeholder dates.
  7. Section 4.1: The notation m^(·)_i uses a dot placeholder for the task type but this is not formally defined. Consider explicitly stating that the dot denotes the task type (VI or OD).

Simulated Author's Rebuttal

3 responses · 0 unresolved

We thank the referee for a careful and constructive review. The referee correctly identifies that the central mechanism of AVA-VLM—predicting crop regions from downsampled global images—deserves direct evaluation, and also identifies a caption error and an important train/test distribution shift. We address each point below.

read point-by-point responses

  1. Referee: The paper never directly evaluates crop-region prediction quality (e.g., IoU between predicted crop boxes and ground-truth crop boxes from Algorithms 1–2). The entire framework depends on the model's ability to accurately predict WHERE to crop from a downsampled global image, yet this is only indirectly assessed through downstream task metrics. The IoU metrics in Table 7 measure violation-localization quality, which conflates crop quality with answer-generation quality and only covers VI positive samples where the model predicts a violation. For no-violation samples that trigger cropping and for all OD samples, crop quality is entirely unassessed. Adding a direct crop-box IoU evaluation would substantially strengthen the paper's central claim.

    Authors: The referee is correct that we do not directly evaluate crop-region prediction quality, and we agree this is a gap. We will add a direct crop-box IoU evaluation in the revision. Specifically, for all samples where the model triggers a crop at inference (covering VI positive, VI negative, and OD samples), we will compute IoU between the predicted crop box and the ground-truth crop box derived from Algorithms 1–2. This will be reported separately from the downstream task metrics to decouple crop quality from answer-generation quality. We note that the current Table 7 IoU metrics do conflate crop quality with answer quality as the referee states, so the direct crop-box IoU evaluation will provide a cleaner assessment of the mechanism the referee rightly identifies as load-bearing. revision: yes

  2. Referee: Tables 4, 7, and 10 have inconsistent or unclear descriptions of the AVA-VLM configuration. Table 4's caption states 'AVA-VLM uses 1/4 downsampled global images,' but Table 6 shows that 1/4 downsampling yields F1=64.5%, while 1/2 downsampling yields F1=75.1%. The text in §5.2.1 says the 1/2 setting is used as the 'default AVA-VLM configuration in the overall comparison reported in Table 4.' This means Table 4's caption is incorrect—it should say 1/2, not 1/4. This inconsistency affects interpretation of the efficiency claims.

    Authors: The referee is correct. Table 4's caption states '1/4 downsampled global images,' but the text in §5.2.1 explicitly states that the 1/2-width-and-height setting is used as the default configuration for the overall comparison in Table 4. The caption is erroneous. We will correct the captions of Tables 4, 7, and 10 to state '1/2 downsampled global images' to match the actual configuration used. We will also carefully audit all table captions for consistency. We note that the efficiency claims themselves are unaffected: the 30.6% visual-token figure in Table 4 corresponds to the 1/2 setting (25% from the downsampled global image plus 5.6% from cropped images), as confirmed by Table 6's 1/2 row. revision: yes

  3. Referee: The train/inference resolution gap is a load-bearing concern. During training (§4.1, Algorithm 3), the model always receives full-size global images and learns to predict crop boxes from them. At inference (§4.2, Eqs. 14–15), the default configuration uses 1/2-downsampled global images, and the predicted crop box in downsampled coordinates is linearly scaled back to full resolution. The model has never been trained on downsampled inputs, so its crop-prediction behavior in this regime is uncharacterized. Table 6 shows F1 degrades from 82.7% (full-size) to 75.1% (1/2) to 64.5% (1/4), and Figure 5(d) shows tool-call ratio drops from 35.8% to 20.0% as resolution decreases, suggesting the model increasingly fails to trigger crops when it should. The paper should discuss this train/test distribution shift explicitly and consider whether training with augmented downsampled inputs could mitigate

    Authors: The referee raises a valid concern. There is indeed a train/test resolution gap: the model is trained on full-size global images but evaluated on downsampled inputs at inference. We agree this distribution shift is not currently discussed in the paper, and we will add an explicit discussion in the revision. The degradation pattern the referee identifies—F1 dropping from 82.7% to 75.1% to 64.5% and tool-call ratio dropping from 35.8% to 20.0%—is consistent with the hypothesis that downsampled inputs make it harder for the model to recognize when cropping is needed, leading to missed crops and increased false negatives. We will acknowledge this limitation transparently and discuss training with resolution augmentation (i.e., randomly downsampling global images during training) as a promising mitigation strategy. We are not able to run new training experiments with resolution augmentation before the revision deadline, so we will frame this as a concrete future direction rather than reporting results. We believe this is an honest accounting of what we can and cannot address at this stage. revision: partial

Circularity Check

0 steps flagged

No circularity: supervised training with ground-truth crop annotations, evaluated on held-out test set against external baselines

full rationale

The paper's central claim is that AVA-VLM achieves better PPE-violation F1 (75.1 vs. 62.0 baseline) while using only 30.6% of visual tokens. The derivation chain is standard supervised learning: (1) crop regions are derived from dataset bounding-box annotations and a YOLO detector (Algorithms 1-2), (2) the model is trained with ground-truth crop regions and CoT annotations (Algorithm 3, Eq. 12-13), and (3) evaluation is on a held-out test set (Table 4) against externally-defined baselines (Qwen2.5, LLaVA, GPT). The training pipeline explicitly decouples crop-region learning from answer generation by using ground-truth crops during training (Section 4.1: 'the crop tool is not executed based on the model's predicted crop region. Instead, each training sample already specifies whether local inspection is required and, if so, which ground-truth cropped image should be used'). At inference (Section 4.2, Eq. 14-15), the model predicts crop regions from a downsampled global image and the predicted box is linearly scaled back to full resolution. The evaluation metrics (F1, IoU, SPICE, METEOR, BERTScore) are computed against ground-truth annotations on the test split, which is kept unchanged. No prediction is equivalent to its training input by construction. The YOLO detector is used only for generating training annotations for no-violation samples, not for evaluation. There are no self-citation chains where the central premise depends on unverified prior work by the same authors. The ConstructionSite10K benchmark [12] is an external dataset. The paper does have a train/inference resolution gap (training on full-size images, testing on downsampled images) and does not directly evaluate crop-box prediction quality (IoU between predicted and ground-truth crop boxes), but these are correctness/generalization risks, not circularity. The F1 improvement is not forced by construction: the model could have performed worse on the test set, and the ablation in Table 6 showing degradation under aggressive downsampling confirms the results are not trivially guaranteed.

Axiom & Free-Parameter Ledger

7 free parameters · 4 axioms · 2 invented entities

The axiom ledger captures the key assumptions and parameters that the paper's claims depend on. The free parameters (thresholds, expansion ratio, downsampling factor) are not explicitly stated in the paper, which is a reproducibility gap. The domain assumptions about annotation quality and YOLO reliability are standard for applied ML but should be validated. The resolution-gap assumption (train on full-res, infer on low-res) is the most paper-specific and least validated.

free parameters (7)
  • tau_obj_VI (object-level area threshold for VI) = not explicitly stated
    Threshold determining whether a bounding box is 'small enough' to require cropping in the VI task. Controls the crop/no-crop decision in Algorithm 1.
  • tau_crop_VI (crop-region area threshold for VI) = not explicitly stated
    Threshold determining whether the union of small boxes is too large to crop. Controls crop rejection in Algorithm 1, line 21.
  • tau_obj_OD (object-level area threshold for OD) = not explicitly stated
    Same role as tau_obj_VI but for object detection task (Algorithm 2).
  • tau_crop_OD (crop-region area threshold for OD) = not explicitly stated
    Same role as tau_crop_VI but for OD task.
  • alpha (expansion ratio) = not explicitly stated
    Controls how much the crop box is expanded beyond the union of small boxes before clipping to image bounds.
  • s (downsampling factor for inference) = 4 (1/4 width and height, i.e., 1/2 each dimension)
    Controls the global image resolution at inference. The paper uses 1/2-width-and-height as the default (Table 4 uses 1/4, but text says 1/2 is default based on Table 6 analysis).
  • LoRA rank = 128
    Standard PEFT hyperparameter, not paper-specific.
axioms (4)
  • domain assumption Ground-truth bounding boxes from ConstructionSite10K accurately represent the spatial extent of query-relevant evidence.
    The entire crop annotation pipeline (Algorithms 1-2) depends on bounding-box annotations being reliable proxies for where the model should look. If boxes are loose or tight, the crop supervision is noisy.
  • domain assumption A YOLO26-large detector finetuned on the Construction-Hazard-Detection dataset provides reliable worker and PPE detections for generating no-violation crop annotations.
    Section 3.2 uses D_YOLO to identify candidate crop regions for no-violation samples. If YOLO misses workers or produces false positives, the training data for 'when to crop' on negative samples is corrupted.
  • ad hoc to paper The crop decision can be learned from a low-resolution global image at inference time, even though training uses full-resolution global images.
    Training uses full-size global images (Section 4.1: 'we use the full-size global image to fully expose the model'), but inference uses downsampled global images (Section 4.2). The paper does not validate that the crop-decision model transfers across this resolution gap, though Table 6 shows degradation under aggressive downsampling.
  • standard math Qwen2.5-VL 7B is a representative backbone for evaluating construction-site VLM adaptation strategies.
    All comparisons use the same backbone, which is appropriate for isolating adaptation strategy effects, but results may not transfer to other VLM architectures.
invented entities (2)
  • Region-aware CoT dataset (extended from ConstructionSite10K) no independent evidence
    purpose: Training data that teaches the model when to inspect, where to crop, and how to use local evidence
    The dataset is constructed by the authors using rule-based annotation generators and a YOLO detector. It is not independently validated against human annotations of crop decisions. The crop decisions are algorithmically determined, not human-verified.
  • Crop tool (tool-call mechanism) independent evidence
    purpose: Allows the VLM to request a high-resolution local crop during inference by outputting bounding-box coordinates
    The tool-call mechanism is evaluated on a held-out test set, and the downstream task performance (Tables 4-13) provides evidence that the mechanism works. The tool-call usage ratios (Figures 5, 8) show the model uses the tool selectively.

pith-pipeline@v1.1.0-glm · 42767 in / 4686 out tokens · 403384 ms · 2026-07-08T22:09:28.197885+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

52 extracted references · 52 canonical work pages · 15 internal anchors

  1. [1]

    AlShami, A.K., Rabinowitz, R., Shoman, M., Fang, J., Picek, L., Lo, S.Y., Cruz, S., Lam, K.N., Kamod, N., Li, L.L., Kalita, J., Boult, T.E.,

    work page

  2. [2]

    2coool: 2nd workshop on the challenge of out-of-label hazards in autonomous driving, in: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, pp. 764–771

    work page

  3. [3]

    Spice: Semantic propositional image caption evaluation, in: European conference on computer vision, Springer

    Anderson, P., Fernando, B., Johnson, M., Gould, S., 2016. Spice: Semantic propositional image caption evaluation, in: European conference on computer vision, Springer. pp. 382–398

    work page 2016

  4. [4]

    Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond

    Bai, J., Bai, S., Yang, S., Wang, S., Tan, S., Wang, P., Lin, J., Zhou, C., Zhou, J., 2023. Qwen-vl: A versatile vision-language model for understanding, localization, text reading, and beyond

    work page 2023

  5. [5]

    Banerjee,S.,Lavie,A.,2005.Meteor:Anautomaticmetricformtevaluationwithimprovedcorrelationwithhumanjudgments,in:Proceedings of the acl workshop on intrinsic and extrinsic evaluation measures for machine translation and/or summarization, pp. 65–72

    work page 2005

  6. [6]

    Context-awarevision-languagemodelagentenriched with domain-specific ontology for construction site safety monitoring

    Chan,C.F.,Wong,P.K.Y.,Guo,X.,Cheng,J.C.,Chan,J.P.C.,Leung,P.H.,Tao,X.,2025. Context-awarevision-languagemodelagentenriched with domain-specific ontology for construction site safety monitoring. Automation in Construction 177, 106305

    work page 2025

  7. [7]

    Enhancing vision-language model for construction safety inspection via visually grounded reasoning and reinforcement learning

    Chan, J.C.F., WONG, P.K.Y., Guo, X., Liang, Z., Wu, Y., Cheng, J.C., . Enhancing vision-language model for construction safety inspection via visually grounded reasoning and reinforcement learning. Available at SSRN 6294267

    work page

  8. [8]

    Augmented reality, deep learning and vision-language query system for construction worker safety

    Chen, H., Hou, L., Wu, S., Zhang, G., Zou, Y., Moon, S., Bhuiyan, M., 2024a. Augmented reality, deep learning and vision-language query system for construction worker safety. Automation in Construction 157, 105158. URL:https://www.sciencedirect.com/science/ article/pii/S0926580523004181, doi:https://doi.org/10.1016/j.autcon.2023.105158

    work page doi:10.1016/j.autcon.2023.105158 2023

  9. [9]

    MiniGPT-v2: large language model as a unified interface for vision-language multi-task learning

    Chen, J., Zhu, D., Shen, X., Li, X., Liu, Z., Zhang, P., Krishnamoorthi, R., Chandra, V., Xiong, Y., Elhoseiny, M., 2023. Minigpt- v2: large language model as a unified interface for vision-language multi-task learning. URL:https://arxiv.org/abs/2310.09478, arXiv:2310.09478

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  10. [10]

    An Image is Worth 1/2 Tokens After Layer 2: Plug-and-Play Inference Acceleration for Large Vision-Language Models

    Chen, L., Zhao, H., Liu, T., Bai, S., Lin, J., Zhou, C., Chang, B., 2024b. An image is worth 1/2 tokens after layer 2: Plug-and-play inference acceleration for large vision-language models. URL:https://arxiv.org/abs/2403.06764,arXiv:2403.06764

    work page internal anchor Pith review Pith/arXiv arXiv

  11. [11]

    Tailored vision-language framework for automated hazard identification and report generation in construc- tion sites

    Chen, Q., Yin, X., 2025. Tailored vision-language framework for automated hazard identification and report generation in construc- tion sites. Advanced Engineering Informatics 66, 103478. URL:https://www.sciencedirect.com/science/article/pii/ S1474034625003714, doi:https://doi.org/10.1016/j.aei.2025.103478

    work page doi:10.1016/j.aei.2025.103478 2025

  12. [12]

    Canmultimodallargelanguagemodelstrulyperform multimodalin-contextlearning?,in:2025IEEE/CVFWinterConferenceonApplicationsofComputerVision(WACV),IEEE.pp.6000–6010

    Chen,S.,Han,Z.,He,B.,Liu,J.,Buckley,M.,Qin,Y.,Torr,P.,Tresp,V.,Gu,J.,2025a. Canmultimodallargelanguagemodelstrulyperform multimodalin-contextlearning?,in:2025IEEE/CVFWinterConferenceonApplicationsofComputerVision(WACV),IEEE.pp.6000–6010

    work page

  13. [13]

    Are large pre-trained vision language models effective construction safety inspectors

    Chen, X., Zou, Z., 2026. Are large pre-trained vision language models effective construction safety inspectors. Data-Centric Engineering 7. URL:http://dx.doi.org/10.1017/dce.2026.10044, doi:10.1017/dce.2026.10044

    work page doi:10.1017/dce.2026.10044 2026

  14. [14]

    Expandingperformanceboundariesof open-source multimodal models with model, data, and test-time scaling

    Chen, Z., Wang, W., Cao, Y., Liu, Y., Gao, Z., Cui, E., Zhu, J., Ye, S., Tian, H., Liu, Z., Gu, L., Wang, X., Li, Q., Ren, Y., Chen, Z., Luo, J., Wang, J., Jiang, T., Wang, B., He, C., Shi, B., Zhang, X., Lv, H., Wang, Y., Shao, W., Chu, P., Tu, Z., He, T., Wu, Z., Deng, H., Ge, J., Chen, K.,Zhang,K.,Wang,L.,Dou,M.,Lu,L.,Zhu,X.,Lu,T.,Lin,D.,Qiao,Y.,Dai,J....

    work page

  15. [15]

    Chain of Thought Prompt Tuning in Vision Language Models

    Ge, J., Luo, H., Qian, S., Gan, Y., Fu, J., Zhang, S., 2023. Chain of thought prompt tuning in vision language models. arXiv preprint arXiv:2304.07919

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  16. [16]

    Intelligent virtual assistants with llm-based process automation

    Guan, Y., Wang, D., Chu, Z., Wang, S., Ni, F., Song, R., Li, L., Gu, J., Zhuang, C., 2023. Intelligent virtual assistants with llm-based process automation

    work page 2023

  17. [17]

    LoRA: Low-Rank Adaptation of Large Language Models

    Hu,E.J.,Shen,Y.,Wallis,P.,Allen-Zhu,Z.,Li,Y.,Wang,S.,Wang,L.,Chen,W.,2021. Lora:Low-rankadaptationoflargelanguagemodels. URL:https://arxiv.org/abs/2106.09685,arXiv:2106.09685

    work page internal anchor Pith review Pith/arXiv arXiv 2021

  18. [18]

    Jian,Y.,Liu,T.,Tao,Y.,Zhang,C.,Vosoughi,S.,Yang,H.,2024. Expeditedtrainingofvisualconditionedlanguagegenerationviaredundancy reduction, in: Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 300– 314

    work page 2024

  19. [19]

    Vru-accident: A vision-language benchmark for video question answering and dense captioning for accident scene understanding

    Kim, Y., Abdelrahman, A.S., Abdel-Aty, M., 2025a. Vru-accident: A vision-language benchmark for video question answering and dense captioning for accident scene understanding

    work page

  20. [20]

    Safe-llava:Aprivacy-preservingvision-languagedatasetandbenchmarkforbiometricsafety

    Kim,Y.,Swetha,S.,Kagdi,F.,Shah,M.,2025b. Safe-llava:Aprivacy-preservingvision-languagedatasetandbenchmarkforbiometricsafety. URL:https://arxiv.org/abs/2509.00192,arXiv:2509.00192. Y. Kim et al.:Preprint submitted to ElsevierPage 32 of 34 AVA-VLM: Adaptive Visual Attention-Vision Language Model for In-the-Wild Construction Site Monitoring

    work page arXiv

  21. [21]

    Res-bench: Benchmarking the robustness of multimodal large language models to dynamic resolution input, in: Proceedings of the AAAI Conference on Artificial Intelligence, pp

    Li, C., Wang, Z., Sheng, Y., Zhu, X., Hao, Y., Wang, X., 2026a. Res-bench: Benchmarking the robustness of multimodal large language models to dynamic resolution input, in: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 31545–31553

    work page

  22. [22]

    Construction site fall hazard identification and automated captioning using adapted vision-language models

    Li, Y., Xu, F., Zhang, Z., Mei, X., Huang, H., 2026b. Construction site fall hazard identification and automated captioning using adapted vision-language models. Automation in Construction 183, 106790. URL:https://www.sciencedirect.com/science/article/pii/ S0926580526000312, doi:https://doi.org/10.1016/j.autcon.2026.106790

    work page doi:10.1016/j.autcon.2026.106790 2026

  23. [23]

    Adaptvision: Efficient vision-language models via adaptive visual acquisition

    Lin, Z., Liu, Y., Yang, Y., Tao, L., Ye, D., 2026. Adaptvision: Efficient vision-language models via adaptive visual acquisition. URL: https://arxiv.org/abs/2512.03794,arXiv:2512.03794

    work page arXiv 2026

  24. [24]

    Llava-next: Improved reasoning, ocr, and world knowledge

    Liu, H., Li, C., Li, Y., Li, B., Zhang, Y., Shen, S., Lee, Y.J., 2024. Llava-next: Improved reasoning, ocr, and world knowledge. URL: https://llava-vl.github.io/blog/2024-01-30-llava-next/

    work page 2024

  25. [25]

    Visual instruction tuning, in: Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., Levine, S

    Liu, H., Li, C., Wu, Q., Lee, Y.J., 2023a. Visual instruction tuning, in: Oh, A., Naumann, T., Globerson, A., Saenko, K., Hardt, M., Levine, S. (Eds.), Advances in Neural Information Processing Systems, Curran Associates, Inc.. pp. 34892–34916. URL:https://proceedings. neurips.cc/paper_files/paper/2023/file/6dcf277ea32ce3288914faf369fe6de0-Paper-Conference.pdf

    work page 2023

  26. [26]

    Grounding DINO: Marrying DINO with Grounded Pre-Training for Open-Set Object Detection

    Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., Li, C., Yang, J., Su, H., Zhu, J., et al., 2023b. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499

    work page internal anchor Pith review Pith/arXiv arXiv

  27. [27]

    The Llama 3 Herd of Models

    Meta, 2024. The llama 3 herd of models. URL:https://arxiv.org/abs/2407.21783,arXiv:2407.21783

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  28. [28]

    GPT-4 Technical Report

    OpenAI, 2024. Gpt-4 technical report. URL:https://arxiv.org/abs/2303.08774,arXiv:2303.08774

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  29. [29]

    Privacy-Aware Visual Language Models

    Samson, L., Barazani, N., Ghebreab, S., Asano, Y.M., 2024. Privacy-aware visual language models. arXiv preprint arXiv:2405.17423

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  30. [30]

    Vlm-robustbench: A comprehensive benchmark for robustness of vision-language models

    Saxena, R., Suglia, A., Minervini, P., 2026. Vlm-robustbench: A comprehensive benchmark for robustness of vision-language models. URL: https://arxiv.org/abs/2603.06148,arXiv:2603.06148

    work page arXiv 2026

  31. [31]

    Vision-language models for edge networks: A comprehensive survey

    Sharshar, A., Khan, L.U., Ullah, W., Guizani, M., 2025. Vision-language models for edge networks: A comprehensive survey. IEEE Internet of Things Journal

    work page 2025

  32. [32]

    Upop: Unified and progressive pruning for compressing vision-language transformers, in: International Conference on Machine Learning, PMLR

    Shi, D., Tao, C., Jin, Y., Yang, Z., Yuan, C., Wang, J., 2023. Upop: Unified and progressive pruning for compressing vision-language transformers, in: International Conference on Machine Learning, PMLR. pp. 31292–31311

    work page 2023

  33. [33]

    Real-time safety detection on construction sites using a vision-language and nlp- based model

    Sivanraj, S., Uduwage, D., Tripathi, M., 2026. Real-time safety detection on construction sites using a vision-language and nlp- based model. Advanced Engineering Informatics 69, 103889. URL:https://www.sciencedirect.com/science/article/pii/ S1474034625007827, doi:https://doi.org/10.1016/j.aei.2025.103889

    work page doi:10.1016/j.aei.2025.103889 2026

  34. [34]

    A double thinking enabled visual language model for open-set construction site safety inspections

    Tang, Y., Yan, H., Gao, Z., Zhang, Z., Luo, X., . A double thinking enabled visual language model for open-set construction site safety inspections. Available at SSRN 5179874

    work page

  35. [35]

    Region-level vision-language model for detecting distraction behavior and mobility attributes of vulnerable road users

    Tran, D.Q., Abdel-Aty, M., Kim, Y., Abdelrahman, A.S., Islam, Z., 2026. Region-level vision-language model for detecting distraction behavior and mobility attributes of vulnerable road users. IEEE Transactions on Intelligent Transportation Systems 27, 5784–5803. doi:10.1109/TITS.2026.3657271

    work page doi:10.1109/tits.2026.3657271 2026

  36. [36]

    Visual question answering-based referring expression segmentation for construction safety analysis

    Tran, D.Q., Aboah, A., Jeon, Y., Do, M.T., Abdel-Aty, M., Park, M., Park, S., 2025. Visual question answering-based referring expression segmentation for construction safety analysis. Automation in Construction 174, 106127. URL:https://www.sciencedirect.com/ science/article/pii/S0926580525001670, doi:https://doi.org/10.1016/j.autcon.2025.106127

    work page doi:10.1016/j.autcon.2025.106127 2025

  37. [37]

    Selma: A speech-enabled language model for virtual assistant interactions

    Wagner, D., Churchill, A., Sigtia, S., Marchi, E., 2025. Selma: A speech-enabled language model for virtual assistant interactions

    work page 2025

  38. [38]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Wang, P., Bai, S., Tan, S., Wang, S., Fan, Z., Bai, J., Chen, K., Liu, X., Wang, J., Ge, W., Fan, Y., Dang, K., Du, M., Ren, X., Men, R., Liu, D., Zhou, C., Zhou, J., Lin, J., 2024. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    work page 2024

  39. [39]

    InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency

    Wang,W.,Gao,Z.,Gu,L.,Pu,H.,Cui,L.,Wei,X.,Liu,Z.,Jing,L.,Ye,S.,Shao,J.,Wang,Z.,Chen,Z.,Zhang,H.,Yang,G.,Wang,H.,Wei, Q.,Yin,J.,Li,W.,Cui,E.,Chen,G.,Ding,Z.,Tian,C.,Wu,Z.,Xie,J.,Li,Z.,Yang,B.,Duan,Y.,Wang,X.,Hou,Z.,Hao,H.,Zhang,T.,Li,S., Zhao,X.,Duan,H.,Deng,N.,Fu,B.,He,Y.,Wang,Y.,He,C.,Shi,B.,He,J.,Xiong,Y.,Lv,H.,Wu,L.,Shao,W.,Zhang,K.,Deng,H.,Qi,B., Ge,J....

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  40. [40]

    Efficient Vision-Language Models by Summarizing Visual Tokens into Compact Registers

    Wen,Y.,Cao,Q.,Fu,Q.,Mehta,S.,Najibi,M.,2024. Efficientvision-languagemodelsbysummarizingvisualtokensintocompactregisters. arXiv preprint arXiv:2410.14072

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  41. [41]

    PyramidDrop: Accelerating Your Large Vision-Language Models via Pyramid Visual Redundancy Reduction

    Xing,L.,Huang,Q.,Dong,X.,Lu,J.,Zhang,P.,Zang,Y.,Cao,Y.,He,C.,Wang,J.,Wu,F.,Lin,D.,2025.Pyramiddrop:Acceleratingyourlarge vision-language models via pyramid visual redundancy reduction. URL:https://arxiv.org/abs/2410.17247,arXiv:2410.17247

    work page internal anchor Pith review Pith/arXiv arXiv 2025

  42. [42]

    Llava-cot: Let vision language models reason step-by-step, in: Proceedings of the IEEE/CVF International Conference on Computer Vision

    Xu, G., Jin, P., Wu, Z., Li, H., Song, Y., Sun, L., Yuan, L., 2025. Llava-cot: Let vision language models reason step-by-step, in: Proceedings of the IEEE/CVF International Conference on Computer Vision

    work page 2025

  43. [43]

    Edgevideoanalytics:Asurveyonapplications,systemsandenablingtechniques

    Xu,R.,Razavi,S.,Zheng,R.,2023. Edgevideoanalytics:Asurveyonapplications,systemsandenablingtechniques. IEEECommunications Surveys & Tutorials 25, 2951–2982

    work page 2023

  44. [44]

    Qwen2.5 Technical Report

    Yang, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Li, C., Liu, D., Huang, F., Wei, H., Lin, H., Yang, J., Tu, J., Zhang, J., Yang, J., Yang, J., Zhou, J., Lin, J., Dang, K., Lu, K., Bao, K., Yang, K., Yu, L., Li, M., Xue, M., Zhang, P., Zhu, Q., Men, R., Lin, R., Li, T., Xia, T., Ren, X., Ren, X., Fan, Y., Su, Y., Zhang, Y., Wan, Y., Liu, Y., Cui...

    work page internal anchor Pith review Pith/arXiv arXiv

  45. [45]

    Vision transformer-based visual language understanding of the con- struction process

    Yang, B., Zhang, B., Han, Y., Liu, B., Hu, J., Jin, Y., 2024b. Vision transformer-based visual language understanding of the con- struction process. Alexandria Engineering Journal 99, 242–256. URL:https://www.sciencedirect.com/science/article/pii/ S1110016824004873, doi:https://doi.org/10.1016/j.aej.2024.05.015

    work page doi:10.1016/j.aej.2024.05.015 2024

  46. [46]

    Visionzip: Longer is better but not necessary in vision language models, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp

    Yang, S., Chen, Y., Tian, Z., Wang, C., Li, J., Yu, B., Jia, J., 2025. Visionzip: Longer is better but not necessary in vision language models, in: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 19792–19802

    work page 2025

  47. [47]

    Visionthink: Smart and efficient vision language model via reinforcement learning

    Yang, S., Li, J., Lai, X., Wu, J., Li, W., MA, Z., Yu, B., Zhao, H., Jia, J., 2026. Visionthink: Smart and efficient vision language model via reinforcement learning. Advances in Neural Information Processing Systems 38, 95187–95227. Y. Kim et al.:Preprint submitted to ElsevierPage 33 of 34 AVA-VLM: Adaptive Visual Attention-Vision Language Model for In-t...

    work page 2026

  48. [48]

    BERTScore: Evaluating Text Generation with BERT

    Zhang, T., Kishore, V., Wu, F., Weinberger, K.Q., Artzi, Y., 2019. Bertscore: Evaluating text generation with bert. arXiv preprint arXiv:1904.09675

    work page internal anchor Pith review Pith/arXiv arXiv 2019

  49. [49]

    SparseVLM: Visual Token Sparsification for Efficient Vision-Language Model Inference

    Zhang, Y., Fan, C.K., Ma, J., Zheng, W., Huang, T., Cheng, K., Gudovskiy, D., Okuno, T., Nakata, Y., Keutzer, K., et al., 2024. Sparsevlm: Visual token sparsification for efficient vision-language model inference. arXiv preprint arXiv:2410.04417

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  50. [50]

    Multimodal Chain-of-Thought Reasoning in Language Models

    Zhang, Z., Zhang, A., Li, M., Zhao, H., Karypis, G., Smola, A., 2023. Multimodal chain-of-thought reasoning in language models. arXiv preprint arXiv:2302.00923

    work page internal anchor Pith review Pith/arXiv arXiv 2023

  51. [51]

    Mmicl: Empowering vision-language model with multi-modal in-context learning, in: International Conference on Learning Representations, pp

    Zhao, H., Cai, Z., Si, S., Ma, X., An, K., Chen, L., Liu, Z., Wang, S., Han, W., Chang, B., 2024. Mmicl: Empowering vision-language model with multi-modal in-context learning, in: International Conference on Learning Representations, pp. 14942–14980

    work page 2024

  52. [52]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models

    Zhu,J.,Wang,W.,Chen,Z.,Liu,Z.,Ye,S.,Gu,L.,Tian,H.,Duan,Y.,Su,W.,Shao,J.,Gao,Z.,Cui,E.,Wang,X.,Cao,Y.,Liu,Y.,Wei,X., Zhang, H., Wang, H., Xu, W., Li, H., Wang, J., Deng, N., Li, S., He, Y., Jiang, T., Luo, J., Wang, Y., He, C., Shi, B., Zhang, X., Shao, W., He, J., Xiong, Y., Qu, W., Sun, P., Jiao, P., Lv, H., Wu, L., Zhang, K., Deng, H., Ge, J., Chen, K.,...

    work page 2025