Advancing the Understanding and Evaluation of AR-Generated Scenes: When Vision-Language Models Shine and Stumble

Lin Duan; Maria Gorlatova; Yanming Xiu

REVIEW 4 major objections 6 minor 25 references

Advancing the Understanding and Evaluation of AR-Generated Scenes: When Vision-Language Models Shine and Stumble

T0 review · 4 major / 6 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Vision-language models can see obvious augmented-reality objects but routinely miss seamlessly integrated virtual content, a new evaluation shows.

desk verdict A useful first-step AR benchmark with a real circularity flaw in the complexity labels and a numeric mismatch; worth reviewing with revisions. read the letter →

arxiv 2501.13964 v3 pith:ZDS5LQID submitted 2025-01-21 cs.CV cs.AIcs.HC

Lin Duan , Yanming Xiu , Maria Gorlatova This is my paper · ORCID

classification cs.CVcs.AIcs.HC

keywords AugmentedrealityVision-languagemodelsSceneunderstandingContentevaluationARdatasetVirtualperceptioncomplexityMultimodal

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether commercial vision-language models (VLMs) can automatically evaluate augmented-reality (AR) scenes — that is, whether they can see virtual content placed in real photos and say what it is. To answer it, the authors built DiverseAR, a dataset of 318 AR and non-AR images spanning three levels of scene complexity, and tested GPT, Gemini, and Claude with two kinds of prompts. They report that the models identify obvious virtual objects well, reaching a perception true-positive rate as high as 93% and a description true-positive rate up to 71%, but that performance collapses as virtual content becomes seamlessly integrated with realistic shadows and lighting, with reported ranges falling to 22.8% for perception and 34.2% for description on the hardest scenes. The paper concludes that today's VLMs could screen for glaring AR artifacts but are not yet dependable automated judges of polished AR experiences.

What carries the argument

The argument is carried by two newly designed instruments. The first is DiverseAR, a dataset of 318 images collected from commercial AR apps, research prototypes, and public sources, with each AR image labeled as easy, medium, or hard according to the obviousness of its virtual content. The second is a five-category response taxonomy—accurate AR recognition, partial recognition, missed recognition, false detection, and accurate non-AR recognition—that converts each VLM answer into a countable class. Two prompts are used, one general ('what is happening?') and one task-aware (asking whether virtual content is superimposed), and performance is scored with true-positive rates for perception (is it AR?) and for description (is the virtual content correctly identified and named?). This apparatus produces the paper's main empirical curve: both metrics decline monotonically as complexity rises.

What would settle it

Have multiple independent annotators re-label the 298 AR images without seeing the VLM results; if the easy–medium–hard ordering does not reproduce, or if a larger sample of hard images yields perception and description true-positive rates comparable to easy images, the central complexity-decline claim would collapse. Alternatively, demonstrating that a VLM fine-tuned on AR images maintains description true-positive rate above 90% on seamlessly integrated content would directly contradict the claim that current VLMs cannot judge polished AR.

Watch

Extended reading notes

Core claim

The central claim, stated on the paper's own terms, is that state-of-the-art commercial VLMs are generally capable of perceiving and describing AR scenes, yet their competence is sharply conditional on how well the virtual content is integrated. On DiverseAR, the perception true-positive rate reaches 93% and the description true-positive rate reaches 71% when the best-performing model and a task-aware prompt are used, but both metrics fall steeply as scenes move from 'easy' (transparent or low-fidelity overlays) to 'hard' (high-quality virtual objects with proper shadows, plausible size, and physical consistency). The paper characterizes the failure modes: with seamlessly blended content, models miss the AR elements or misclassify real objects as virtual when the scene already contains other AR objects; with obvious content, they succeed readily. A user study shows humans also decline with complexity but outperform the best VLM on the hardest scenes, especially in describing what is virtual. The paper's takeaway is that VLMs have potential as AR quality evaluators but currently fall short of what reliable automated high-quality evaluation would require.

Load-bearing premise

The complexity labels (easy, medium, hard) were assigned by a single annotator and are treated as ground truth, so the finding that VLM performance declines with complexity depends entirely on those labels being reproducible.

Editorial extensions

If this is right

A task-aware prompt that explicitly asks about virtual content raises perception true-positive rate by 30–47 percentage points across models, so prompt design is a major lever in any VLM-based AR screening tool.
Because all three models maintain a 100% true-negative rate, non-AR images are never flagged as AR; the practical bottleneck is missed detection, not false alarms.
Human and VLM performance decline in the same direction with complexity, and the images that trip up people largely overlap with those that trip up VLMs, suggesting shared cues drive both successes and failures.
For hard, seamlessly integrated AR scenes, human viewers beat the best VLM by about 8 percentage points in perception and 27 in description, so an AR quality-checking pipeline that relies on VLMs alone would currently underperform a human reviewer on precisely the content that needs the most checking.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

Because 'hard' is defined as seamless integration, part of the observed decline is built into the label; a sharper test would ask whether VLMs can identify the specific rendering cue (shadow direction, lighting mismatch, depth inconsistency) that reveals a virtual object, separating perceptual ability from label design.
The finding that hands or QR codes in the scene improve recognition suggests these models rely on contextual priors more than pixel-level rendering analysis; this predicts that VLM-based AR detectors could be improved substantially by giving the model interactive cues or a two-stage 'what looks off?' reasoning prompt, rather than only by adding more training images.
The dataset's current size (318 images, 79 of them 'hard') makes the hard-scene numbers sensitive to a few dozen examples; a scaled-up independently labeled collection would likely shift the precise percentages even if the qualitative decline persists.

Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Two things to know: (1) This is a genuinely useful first-step benchmark — a new 318-image AR dataset with released code, and a clean complexity-stratified evaluation of three commercial VLMs. (2) The central 'decline with complexity' result is partly baked into the labeling rubric, and the paper's own intro numbers don't match the results section. Still worth engaging, with revisions.

What's new: DiverseAR, first dataset specifically for VLM evaluation of AR images, with images from multiple sources including their own AR apps. They test GPT, Gemini, Claude with general vs task-aware prompts, report perception/description TPR, and run a small human comparison. The observation that task-aware prompts help a lot (e.g., Gemini perception from 61.4% to 93.3%) is useful and practical. The dataset is released on GitHub, so others can build on it.

Where it's soft: The complexity levels (easy/medium/hard) are defined by how easily virtual content can be distinguished from real — 'Hard: seamlessly integrated... making them more challenging to distinguish.' That means the finding that VLMs do worse on hard images partly restates the definition of hard. Any detector, human or model, would be expected to drop on images defined by their indistinguishability. The labels come from a single annotator with no inter-rater check, so we don't know how robust the levels are. A more defensible approach would be to label hard by objective properties (correct shadows, physical plausibility) independent of human detectability. Also, the introduction cites TPR drops 'from 75.8% to 22.8% and from 97.8% to 34.2%,' but the results section gives different numbers (e.g., GPT under general prompt: 71.4% to 11.4%). That inconsistency needs fixing. The user study has only 5 participants; it's a pilot, not a strong comparison. And the 'first to apply VLMs to AR evaluation' claim is undercut by their own ViDDAR reference [23], which is exactly a VLM-based AR content detector. That should be reworded.

Who it's for: Researchers working on automated AR/XR quality assessment or VLM evaluation. The benchmark is a reasonable starting point, and the released data makes it citable. The paper deserves a serious referee, but it needs revision before acceptance — chiefly, clarifying the label circularity, fixing the numeric inconsistency, and softening the novelty claim.

Recommendation: Send to peer review, not desk reject. The dataset and prompt effects are real contributions; just demand the authors address the circularity and the intro/results mismatch.

Referee Report

4 major / 6 minor

Summary. The paper introduces DiverseAR, a 318-image dataset of AR and non-AR scenes collected from commercial apps, research prototypes, and custom-built applications. It evaluates three commercial VLMs (GPT-4o, Claude-3.5-Sonnet, Gemini-1.5-Pro) on two tasks—perceiving whether an image contains AR content and describing the virtual elements—under a general captioning prompt and a task-aware prompt. The authors define three AR scene complexity levels, categorize model responses into five qualitative groups, and report perception/description true positive rates across complexity levels, plus a five-participant user study. The central claim is that VLMs are quite capable on obvious AR content (perception TPR up to 93%, description TPR around 71–74%) but their performance degrades sharply on seamlessly integrated virtual content, implying that current VLMs can screen for obvious AR failures but cannot be trusted as automated evaluators of high-quality AR scenes.

Significance. If the headline findings hold, this paper provides a useful first benchmark for VLM-based AR content evaluation. The dataset is publicly released, spans diverse devices and rendering conditions, and the five-category error taxonomy gives actionable insight into where VLMs confuse real and virtual objects. The paper also makes a concrete applied recommendation—VLMs are suitable for catching obvious virtual-object failures but not well-integrated content—that could inform AR quality-assurance practice. However, the main decline-with-complexity result rests on a single annotator's detectability-based complexity labels, and the reported numerical claims lack uncertainty quantification. These issues are load-bearing for the paper's central message, so the benchmark is not yet fully reliable as a reference result.

major comments (4)

[§4.2, Figs. 3–4] The Easy/Medium/Hard complexity definitions in Section 4.2 are stated directly in terms of how distinguishable the virtual content is: Easy is 'obvious virtual content ... easily distinguishable from the real world,' and Hard is 'high-quality virtual content seamlessly integrated ... making them more challenging to distinguish as virtual.' The central result that VLM perception/description TPR declines across these levels therefore partly restates the labeling criterion rather than independently measuring VLM performance against an objective notion of AR scene complexity. To support the applied conclusion that VLMs 'cannot be trusted to catch well-integrated content,' the Hard subset needs to be characterized by objective rendering properties (e.g., shadow direction, lighting consistency, physical plausibility) collected from multiple annotators with reported inter-rater reliability, and the analysis should be re-run with such labels or with human detectability explicitly controlled.
[§5.1, §5.3] The paper reports model rankings and per-complexity trends without confidence intervals or significance tests. For example, Section 5.1 states that Gemini achieves 93.3% TPR_P under prompt T and that GPT's TPR_D is 73.8% versus Gemini's 71.8%, and Figure 4 shows GPT 'outperforming' the other models on medium and hard scenes. With only 79 hard-level AR images, a 2% TPR_D difference is well within binomial sampling error. The same issue affects the user-study comparison, where five participants are used to claim large human advantages on hard scenes. Provide bootstrap confidence intervals, McNemar tests for paired model comparisons, or per-item error tables before asserting that one model outperforms another or that performance significantly declines with complexity.
[Introduction (contributions bullet 3) vs §5.1 and §4.4] The reported magnitudes are internally inconsistent. The Introduction states that as complexity increases, perception TPR drops from 75.8% to 22.8% and description TPR from 97.8% to 34.2%, while Section 5.1 reports GPT's prompt-G perception decline as 71.4% to 11.4% and gives GPT's overall prompt-G description TPR as 38.9%. The abstract says description TPR 'up to 71%,' but Section 5.1 reports 73.8% for GPT under prompt T. More importantly, under the Section 4.4 definitions TPR_D = N1/(N1+N2+N3) is always less than or equal to TPR_P = (N1+N2)/(N1+N2+N3), so a description easy-level value of 97.8% alongside a perception easy-level value of 75.8% is arithmetically impossible on the same subset. Clarify whether the Introduction reports a different prompt/model aggregation, and align all reported numbers.
[§5.3] The human comparison is not a clean head-to-head evaluation. Participants were explicitly informed that they would see 'a mix of AR and non-AR images,' which gives them knowledge of the base rate and primes them to search for AR content; the paper even attributes human false positives to this priming. The VLMs, by contrast, are not given this meta-information. This design inflates human TPRs and makes the reported human advantages (e.g., 27.1% higher TPR_D on hard-level images) difficult to interpret. Report the exact participant instructions, or match the information conditions, or present the user study as a primed-detection condition rather than a general human baseline.

minor comments (6)

[Abstract] The sentence 'Could Vision-Language Models (VLMs) offer a solution for the automated evaluation of AR-generated scenes?' appears twice in the abstract; remove the duplicate.
[§4.4, §5.1] Section 5.1 refers to a Description True Negative Rate (TNR_D) and reports it as 100%, but Section 4.4 only defines TNR_P. Add the corresponding definition or state explicitly that TNR_D is the non-AR analogue of TNR_P.
[§4.2] The phrase 'unrealistic attributes like informal size or placement' appears to contain a typo; 'informal size' should likely be 'unusual size' or 'abnormal size.'
[Figs. 3–4] Figures 3 and 4 do not show numerical values or error bars, so the per-complexity TPRs cannot be checked from the figures alone. Add a table with exact per-level values and, ideally, confidence intervals.
[§4.1] The paper says the task-aware prompt without explicitly referencing AR was chosen 'for consistency and simplicity,' but it does not state whether any reported result uses the explicit-AR prompt (prompt T is described as 'task-aware' in Figures 3–4). Clarify which prompt variant generated the numbers in Section 5.
[§5.2] The attribution of specific VLM behaviors to the transformer self-attention mechanism (citing a general vision-transformer survey) is speculative and unsupported by the experiments; it would be safer to describe these as observed failure patterns rather than architectural explanations.

Circularity Check

1 steps flagged · score 3.0 of 10

Complexity rubric in §4.2 defines Hard as 'more challenging to distinguish as virtual,' so the headline decline result is partly a restatement of the label; the underlying VLM measurements remain independent.

self definitional [Section 4.2, 'Classification of AR Scene Complexity Levels' and Section 5.1, 'Perception Performance']
"Easy: Images with obvious virtual content, such as transparent or glowing overlays, or virtual objects with low rendering quality that are easily distinguishable from the real world. ... Hard: Images with high-quality virtual content seamlessly integrated into the real-world environment, including proper shadows, realistic size and shape, and adherence to physical laws, making them more challenging to distinguish as virtual. ... Further analysis of T PRP across different AR scene complexity levels reveals a consistent decline in performance as complexity increases."

The independent variable (complexity level) is defined partly by the very difficulty the dependent variable measures: Easy means 'easily distinguishable from the real world' and Hard means 'more challenging to distinguish as virtual.' The central result that VLM perception and description TPR decline with complexity is therefore, to some degree, a restatement of the labeling rubric rather than an independent discovery. The paper does not provide an objective, detectability-independent definition of complexity (e.g., measured shadow accuracy or physical plausibility), and the single annotator labels are not validated for inter-rater reliability.

full rationale

This is an empirical benchmark paper rather than a derivation, so the circularity burden is low. The only load-bearing step that approaches circularity is the complexity taxonomy in Section 4.2: the Easy/Medium/Hard labels encode how distinguishable the virtual content is, and the paper's headline 'decline with complexity' and 'struggle with seamlessly integrated content' conclusions echo that encoding. This makes the direction of the main trend unsurprising by construction. Against that, the paper's quantitative results (TPR up to 93%/71%, specific drops such as GPT 71.4% to 11.4% under prompt G, the 100% TNR, the prompt-T gains of 30.5-46.7%, and the human comparison in Section 5.3) are measured behaviors of external commercial VLMs; none are fitted parameters or outputs of the authors' own models, and no uniqueness theorem or prior work is invoked to force the conclusions. The dataset does include images from the authors' prior AR applications [6,17,23] and from ViDDAR [23], but those are data sources, not load-bearing evidence. There is also an internal inconsistency between the Introduction's aggregate TPR drops (75.8% to 22.8%, 97.8% to 34.2%) and the Section 5.1 per-model numbers, but inconsistency is a correctness/consistency issue, not circularity. Overall, one definitional circularity affects the interpretation of the complexity trend, while the core benchmark measurements stand independently, so a score of 3 is appropriate.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

The paper's central results rest on three human-made taxonomies: the complexity levels (Section 4.2), the response categories (Section 4.3), and the dataset's representativeness (Section 3). These are domain assumptions, not derived results, and they are not validated with inter-rater reliability or external benchmarks.

assumptions (4)

domain assumption The easy/medium/hard complexity labels assigned by the single annotator are a valid ordinal scale for AR scene difficulty.
Section 4.2; the entire complexity-stratified analysis depends on this subjective labeling, with no inter-rater reliability reported.
domain assumption The 318 images in DiverseAR are representative of the broad space of AR scenes.
Section 3; images come from a few sources, many from the authors' own apps and lab, so representativeness is assumed rather than demonstrated.
domain assumption The five response categories and the manual assignment of each VLM response to a category are consistent and unbiased.
Section 4.3; the authors classify the model outputs themselves, with no inter-rater reliability check.
domain assumption VLM API responses are treated as deterministic outputs of the specified model versions.
Section 4; temperature and other sampling parameters are not reported, yet all numbers are presented as fixed values.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Advancing the Understanding and Evaluation of AR-Generated Scenes: When Vision-Language Models Shine and Stumble." pith.science (2026). https://pith.science/paper/ZDS5LQID

@misc{pith2026250113964,
  author       = {Pith},
  title        = {Pith review of: Advancing the Understanding and Evaluation of AR-Generated Scenes: When Vision-Language Models Shine and Stumble},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZDS5LQID}},
  note         = {Machine review of arXiv:2501.13964}
}

read the original abstract

Augmented Reality (AR) enhances the real world by integrating virtual content, yet ensuring the quality, usability, and safety of AR experiences presents significant challenges. Could Vision-Language Models (VLMs) offer a solution for the automated evaluation of AR-generated scenes? Could Vision-Language Models (VLMs) offer a solution for the automated evaluation of AR-generated scenes? In this study, we evaluate the capabilities of three state-of-the-art commercial VLMs -- GPT, Gemini, and Claude -- in identifying and describing AR scenes. For this purpose, we use DiverseAR, the first AR dataset specifically designed to assess VLMs' ability to analyze virtual content across a wide range of AR scene complexities. Our findings demonstrate that VLMs are generally capable of perceiving and describing AR scenes, achieving a True Positive Rate (TPR) of up to 93% for perception and 71% for description. While they excel at identifying obvious virtual objects, such as a glowing apple, they struggle when faced with seamlessly integrated content, such as a virtual pot with realistic shadows. Our results highlight both the strengths and the limitations of VLMs in understanding AR scenarios. We identify key factors affecting VLM performance, including virtual content placement, rendering quality, and physical plausibility. This study underscores the potential of VLMs as tools for evaluating the quality of AR experiences.

Figures

Figures reproduced from arXiv: 2501.13964 by the authors.

**Figure 2.** Examples of key characteristics of the virtual and real [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗

**Figure 3.** The True Positive Rate for Perception (T PRP) of the three VLMs across varying complexity levels using G (general) and T (task-aware) image captioning prompts. VLMs show a consistent decline in performance as AR scene complexity increases. a) GPT b) Claude c) Gemini [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (4 more)

**Figure 4.** Figure 4: The True Positive Rate for Description ( [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

**Figure 5.** Figure 5: These images feature an AR-generated “Exit” sign inte [PITH_FULL_IMAGE:figures/full_fig_p004_5.png]

**Figure 8.** Figure 8: This image features an AR-generated pot, heater, apple, [PITH_FULL_IMAGE:figures/full_fig_p005_8.png]

**Figure 7.** Figure 7: These images showcase an AR-generated image with a [PITH_FULL_IMAGE:figures/full_fig_p005_7.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

25 extracted references · 22 canonical work pages

[23]

Y . Xiu, T. Scargill, and M. Gorlatova. ViDDAR: Vision language model-based task-detrimental content detection for augmented reality. IEEE Transactions on Visualization and Computer Graphics (TVCG), 2025

work page 2025
[1]

Introducing the next generation of Claude

Anthropic. Introducing the next generation of Claude. https://www. anthropic.com/news/claude-3-family, 2024

work page 2024
[2]

Bitton-Guetta, Y

N. Bitton-Guetta, Y . Bitton, J. Hessel, L. Schmidt, Y . Elovici, G. Stanovsky, and R. Schwartz. Breaking common sense: Whoops! a vision-and-language benchmark of synthetic and compositional images. In Proceedings of IEEE/CVF ICCV, 2023

work page 2023
[3]

W. Cai, I. Ponomarenko, J. Yuan, X. Li, W. Yang, H. Dong, and B. Zhao. SpatialBot: Precise spatial understanding with vision language models. arXiv preprint arXiv:2406.13642, 2024

arXiv 2024
[4]

Z. Chen, B. Hu, C. Niu, T. Chen, Y . Li, H. Shan, and G. Wang. IQAGPT: Image quality assessment with vision-language and ChatGPT models. arXiv preprint arXiv:2312.15663, 2023

arXiv 2023
[5]

AR shopping for glasses, 2024

DeepAR. AR shopping for glasses, 2024. https://www.deepar.ai

work page 2024
[6]

S. Eom, D. Sykes, S. Rahimpour, and M. Gorlatova. NeuroLens: Augmented reality-based contextual guidance through surgical tool tracking in neurosurgery. In Proceedings of IEEE ISMAR, 2022

work page 2022
[7]

Gemini: A family of highly capable multimodal models

Google. Gemini: A family of highly capable multimodal models. https://gemini.google.com/app, 2024

work page 2024

Show all 25 references

[8]

K. Han, Y . Wang, H. Chen, X. Chen, J. Guo, Z. Liu, Y . Tang, A. Xiao, C. Xu, Y . Xu, et al. A survey on vision transformer.IEEE Transactions on Pattern Analysis and Machine Intelligence, 45(1):87–110, 2022

2022
[9]

Hore and D

A. Hore and D. Ziou. Image quality metrics: PSNR vs. SSIM. In Proceedings of IEEE ICPR, 2010

2010
[10]

T. Hu, T. Scargill, F. Yang, Y . Chen, G. Lan, and M. Gorlatova. SEESys: Online pose error estimation system for visual SLAM. In Proceedings of ACM SenSys, 2024

2024
[11]

Langlotz, T

T. Langlotz, T. Nguyen, D. Schmalstieg, and R. Grasset. Next- generation augmented reality browsers: Rich, seamless, and adaptive. In Proceedings of the IEEE, 2014

2014
[12]

Lin and G

K. Lin and G. Wang. Hallucinated-IQA: No-reference image quality assessment via adversarial learning. In Proceedings of IEEE CVPR, 2018

2018
[13]

Malakouti, A

S. Malakouti, A. Aghazadeh, A. Khandelwal, and A. Kovashka. Bench- marking VLMs’ reasoning about persuasive atypical images. arXiv preprint arXiv:2409.10719, 2024

2024 arXiv
[14]

Merino, M

L. Merino, M. Schwarzl, M. Kraus, M. Sedlmair, D. Schmalstieg, and D. Weiskopf. Evaluating mixed and augmented reality: A systematic literature review (2009-2019). In Proceedings of IEEE ISMAR, 2020

2009
[15]

Scaniverse - 3D scanner

Niantic, Inc. Scaniverse - 3D scanner. Mobile application software,
[16]

GPT-4 technical report

OpenAI. GPT-4 technical report. arXiv:2303.08774, 2023

2023 arXiv
[17]

Z. Qu, R. Byrne, and M. Gorlatova. “Looking” into attention patterns in extended reality: An eye tracking-based study. In Proceedings of IEEE ISMAR, 2024

2024
[18]

E. E. Sabelman and R. Lam. The real-life dangers of augmented reality. In Proceedings of IEEE Spectrum, 2015

2015
[19]

Scargill, S

T. Scargill, S. Hurli, J. Chen, and M. Gorlatova. Will it move? In- door scene characterization for hologram stability in mobile AR. In Proceedings of MobiCom Workshop, 2021

2021
[20]

S. Tong, Z. Liu, Y . Zhai, Y . Ma, Y . LeCun, and S. Xie. Eyes wide shut? Exploring the visual shortcomings of multimodal LLMs. In Proceedings of IEEE/CVF CVPR, 2024

2024
[21]

J. Wang, K. C. Chan, and C. C. Loy. Exploring CLIP for assessing the look and feel of images. In Proceedings of AAAI, 2023

2023
[22]

Wasenm¨uller, M

O. Wasenm¨uller, M. Meyer, and D. Stricker. Augmented reality 3D discrepancy check in industrial applications. In Proceedings of IEEE ISMAR, 2016

2016
[24]

Zhang, J

J. Zhang, J. Huang, S. Jin, and S. Lu. Vision-language models for vision tasks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(8):5625–5644, 2024

2024
[25]

K. Zhou, E. Lai, W. B. A. Yeong, K. Mouratidis, and J. Jiang. ROME: Evaluating pre-trained vision-language models on reasoning beyond visual common sense. arXiv preprint arXiv:2310.19301, 2023

2023 arXiv

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.

[1] [23]

Y . Xiu, T. Scargill, and M. Gorlatova. ViDDAR: Vision language model-based task-detrimental content detection for augmented reality. IEEE Transactions on Visualization and Computer Graphics (TVCG), 2025

work page 2025

[2] [1]

Introducing the next generation of Claude

Anthropic. Introducing the next generation of Claude. https://www. anthropic.com/news/claude-3-family, 2024

work page 2024

[3] [2]

Bitton-Guetta, Y

N. Bitton-Guetta, Y . Bitton, J. Hessel, L. Schmidt, Y . Elovici, G. Stanovsky, and R. Schwartz. Breaking common sense: Whoops! a vision-and-language benchmark of synthetic and compositional images. In Proceedings of IEEE/CVF ICCV, 2023

work page 2023

[4] [3]

W. Cai, I. Ponomarenko, J. Yuan, X. Li, W. Yang, H. Dong, and B. Zhao. SpatialBot: Precise spatial understanding with vision language models. arXiv preprint arXiv:2406.13642, 2024

arXiv 2024

[5] [4]

Z. Chen, B. Hu, C. Niu, T. Chen, Y . Li, H. Shan, and G. Wang. IQAGPT: Image quality assessment with vision-language and ChatGPT models. arXiv preprint arXiv:2312.15663, 2023

arXiv 2023

[6] [5]

AR shopping for glasses, 2024

DeepAR. AR shopping for glasses, 2024. https://www.deepar.ai

work page 2024

[7] [6]

S. Eom, D. Sykes, S. Rahimpour, and M. Gorlatova. NeuroLens: Augmented reality-based contextual guidance through surgical tool tracking in neurosurgery. In Proceedings of IEEE ISMAR, 2022

work page 2022

[8] [7]

Gemini: A family of highly capable multimodal models

Google. Gemini: A family of highly capable multimodal models. https://gemini.google.com/app, 2024

work page 2024