Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

Real-world clock photos plus hand-swap preference training raise vision-language models from near-chance to 46% full-time accuracy on analog clocks.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · grok-4.5

2026-07-15 12:52 UTC pith:VOIVCA4O

load-bearing objection Solid empirical paper: real-world TickTockVQA data drives most of the gain over synthetic clocks, and a simple geometric Swap-DPO construction cleanly reduces residual hand-role errors. the 3 major comments →

arxiv 2603.08011 v2 pith:VOIVCA4O submitted 2026-03-09 cs.CV

It's Time to Get It Right: Improving Analog Clock Reading and Clock-Hand Spatial Reasoning in Vision-Language Models

Jaeha Choi , Jin Won Lee , Siwoo You , Jangho Lee This is my paper
classification cs.CV
keywords analog clock readingvision-language modelsspatiotemporal reasoningdirect preference optimizationTickTockVQAhand-swap errorsreal-world VQA
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Vision-language models that handle complex multimodal tasks still fail at reading ordinary analog clocks in photographs, often scoring near random chance and routinely swapping the hour and minute hands. The paper argues this gap comes from training data that is mostly synthetic or clean and planar, so models never learn the lighting, occlusion, clutter, and design variety of real scenes. The authors release TickTockVQA, a roughly 12,000-image human-annotated collection of clocks found in the wild, labeled with hour, minute, and AM/PM when the scene makes it clear. They then fine-tune models first with ordinary supervised learning on this data and second with Swap-DPO, a preference method that forces the model to prefer the true time over a geometrically swapped hour-minute pair. On their test set the full pipeline lifts Llama-3.2-11B full-time accuracy from 1.41% zero-shot to 46.22%, beats much larger synthetic sets, and narrows the hand-swap error gap, establishing clock reading as a concrete probe of fine-grained spatiotemporal reasoning.

Core claim

Combining the real-world TickTockVQA dataset with Swap-DPO preference alignment that explicitly penalizes hour-minute hand role reversal substantially improves analog clock reading in vision-language models. Full-time accuracy on the TickTockVQA test set rises from roughly 1-6% zero-shot to 46.22% for Llama-3.2-11B, while the same real images outperform equal-scale or larger synthetic clocks (SynClock, CtrlClock) under identical fine-tuning.

What carries the argument

Swap-DPO: after supervised fine-tuning, preference pairs set the preferred answer to the ground-truth time and the rejected answer to either the model's own error or a closed-form geometric swap of the hands (θ_h = 30h + m/2, θ_m = 6m mapped to new hour and minute). Direct preference optimization then teaches the model the correct semantic roles of the two hands.

Load-bearing premise

After ordinary supervised fine-tuning, the dominant remaining error is that the model simply reverses the hour and minute hands, so constructing rejected answers by that geometric swap is the highest-leverage preference signal.

What would settle it

Train the same base model on TickTockVQA with supervised fine-tuning plus DPO that uses only random incorrect times (no hand-swap construction) and check whether the hand-swap accuracy gap and full-time accuracy still improve by the same margins reported for Swap-DPO; if random DPO matches Swap-DPO, the geometric construction is not essential.

Watch this falsifier — get emailed when new claim-graph text bears on it.

If this is right

  • Real-scene diversity matters more than photorealistic synthetic scale for fine-grained spatial time reading.
  • Preference optimization can be specialized to a single geometric error mode rather than only random negatives.
  • Analog clock reading becomes a reusable diagnostic for spatiotemporal reasoning in vision-language models.
  • Once hand roles are correctly learned, models gain robustness to the lighting, occlusion, and numeral-style variation present in everyday photos.
  • Multimodal systems that need visual time perception (assistants, robots, document understanding) gain a clearer training recipe.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • The same closed-form hand-swap preference construction could transfer to other continuous-to-discrete spatial mappings such as gauge needles or compass dials.
  • Persistent low accuracy on wristwatches and flipped clocks points to multi-scale and rotation-equivariant vision features as the next bottleneck after hand-role learning.
  • If occlusion, numeral style, and small faces are also major residual errors, hybrid preference sets that mix geometric swaps with those error types could close more of the remaining gap to human performance.
  • The dataset's context-dependent AM/PM tags open a direct test of whether models can ground clock time in scene lighting and activity cues.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper argues that state-of-the-art VLMs remain weak at reading analog clocks in real scenes, largely because existing resources are synthetic/planar and because models systematically confuse hour and minute hands. It introduces TickTockVQA (~12.5k human-annotated real-world clocks with hour/minute and optional AM/PM labels) and a two-stage fine-tuning recipe: LoRA SFT on TickTockVQA followed by Swap-DPO, which builds preference pairs by preferring the ground-truth time over a geometrically swapped hour–minute hard negative (or the SFT model’s own error). Across Qwen2.5-VL-7B, Llama-3.2-11B, and Gemma3-12B, real-data SFT yields large gains over zero-shot and over equal- or larger-scale synthetic sets (SynClock, CtrlClock); Swap-DPO further reduces the Baseline vs. Swap-equivalence gap and modestly improves full-time accuracy (e.g., Llama 1.41% → 45.78% SFT → 46.22% Swap-DPO). The authors position analog clock reading as a compact testbed for fine-grained spatiotemporal reasoning.

Significance. If the reported numbers hold, the work supplies a useful real-world benchmark and a simple, falsifiable preference construction for a concrete spatial-role failure mode. Strengths that should be credited: multi-backbone evaluation; controlled real-vs-synthetic and synthetic-scale ablations (up to 1M); explicit Baseline vs. Swap-equivalence metrics that isolate hand-role confusion; a fixed geometric SwapHands map rather than a metric-fitted negative; and an openly described collection/annotation protocol with source-separated train/test splits. Absolute accuracy remains far below human level, so the primary impact is as a diagnostic benchmark and domain-adaptation study rather than a solved capability, but that limitation is acknowledged and does not erase the empirical contribution.

major comments (3)
  1. Table 3: On Llama-3.2-11B the Swap-DPO lift over TickTockVQA SFT is only 45.78% → 46.22% full-time accuracy (and the B–S gap barely moves). On Gemma the lift is similarly small. The abstract and §5.3 frame Swap-DPO as substantially correcting hand confusion and enhancing accuracy; that framing is better supported for Qwen (20.34% → 23.06%) than for the strongest backbone. Please restate the contribution of Swap-DPO proportionally to the measured deltas, and clarify whether the method is primarily a residual hand-role regularizer rather than a large end-to-end accuracy driver.
  2. §4.2 / Algorithm 1 and §5.3: The design of Swap-DPO assumes that geometrically consistent hour–minute role reversal is a sufficiently dominant residual error after SFT to justify the closed-form angular swap as the preferred hard negative. The manuscript itself reports only a ~2–3 pp B–S gap after SFT. A short residual-error taxonomy on the SFT model (hand-swap vs. occlusion, small faces/wristwatches, missing numerals, flipped orientation—cf. Supp. B) would make the targeting claim load-bearing rather than assumed. Without that, the method remains valid but its claimed specificity is under-supported.
  3. §3.2 and abstract: AM/PM tags are presented as part of TickTockVQA’s annotation design, yet main tables report only hour/minute/full-time accuracy and MAE. Either evaluate AM/PM when labeled (or state explicitly that AM/PM is unused in training/eval) so the claimed annotation richness matches the experimental protocol.
minor comments (6)
  1. Figure 1 caption and axis: MAE is reported in minutes; make the unit explicit in the y-axis label as well as the caption for standalone readability.
  2. Table 1 vs. Table 3: Zero-shot Llama full-time accuracy is 1.43% in Table 1 and 1.41% in Table 3; reconcile the rounding or seed difference.
  3. §4.2 equations for θ_h, θ_m and the swap map: state the assumed 12-hour face and that second hands are ignored, matching the prompt protocol in the supplement.
  4. Figure 2: Several proprietary model names/versions appear; ensure the exact API/model identifiers and date of evaluation are listed so the qualitative comparison is reproducible.
  5. Supp. C references Table S2 for Random-DPO vs. Swap-DPO; ensure that table is complete and that the hybrid negative strategy (SFT error vs. pure swap) is summarized in the main text in one sentence.
  6. Minor prose: “It’s Time to Get It Right” / ITGR naming is fine, but keep one consistent model name in tables and figures to avoid reader confusion with the paper title.

Circularity Check

0 steps flagged

No significant circularity: empirical ML results measured on held-out real-world test data with fixed geometric preference construction.

full rationale

This is a standard empirical vision-language paper. TickTockVQA annotations and the train/test split (7 236 / 5 247) are human-provided and never used to define the evaluation metrics. Swap-DPO constructs rejected responses either from the SFT model’s own errors on the training set or via the closed-form angular map (θ_h = 30h + m/2, θ_m = 6m → h_new = ⌊θ_m/30⌋, m_new = (θ_h/6) mod 60); that map is a fixed geometric identity, not a parameter fitted to any test quantity. Full-time accuracy, hour/minute accuracy and MAE are ordinary held-out measurements. No uniqueness theorem, self-referential definition, or post-hoc re-labeling of a fitted quantity as a “prediction” appears. The modest residual B–S gap after SFT is reported transparently and does not force the headline numbers. Consequently the derivation chain is self-contained against external benchmarks and exhibits zero circularity of the kinds enumerated.

Axiom & Free-Parameter Ledger

4 free parameters · 3 axioms · 2 invented entities

Standard empirical VLM fine-tuning paper. The load-bearing elements are the new dataset construction choices, the geometric definition of the hard negative, and a handful of training hyper-parameters; no new physical entities or unproved mathematical lemmas are introduced.

free parameters (4)
  • DPO β = 0.3
    Set to 0.3 (stronger than the common default 0.1) to emphasize preference signal; chosen by the authors rather than derived.
  • LoRA rank / alpha = 64 / 64
    Rank 64, alpha 64 applied to vision and language linear layers; standard but free choice that affects capacity.
  • projector learning-rate multiplier = 5×
    Projector receives 5× higher LR (1e-5 vs 2e-6) during DPO; ad-hoc differential schedule.
  • minute tolerance for accuracy = ±2 min
    ±2 minutes used for minute and full-time accuracy; conventional but arbitrary threshold that affects reported numbers.
axioms (3)
  • domain assumption Direct Preference Optimization with a frozen reference model improves alignment on the constructed preference pairs without requiring an explicit reward model.
    Invoked throughout Sec. 4.2 and Eq. (1); taken from Rafailov et al. 2023.
  • standard math Hour and minute hands can be uniquely recovered from their angular positions via the standard clock equations θ_h = 30h + m/2, θ_m = 6m.
    Used to define the SwapHands map that generates hard negatives.
  • domain assumption Human double annotation with consensus yields sufficiently accurate ground-truth times for supervised and preference training.
    Stated in Sec. 3.2; no inter-annotator agreement statistics are reported.
invented entities (2)
  • TickTockVQA no independent evidence
    purpose: Provide the first large-scale, human-annotated, in-the-wild analog-clock VQA benchmark with hour/minute/AM-PM labels.
    New dataset constructed by the authors; independent evidence will exist once the images and labels are publicly released and re-used by others.
  • Swap-DPO no independent evidence
    purpose: Generate geometrically consistent hard negatives that specifically penalize hour–minute hand role reversal.
    Novel preference-pair construction; the method itself is the contribution and has no external falsifiable prediction beyond the reported accuracy gains.

pith-pipeline@v1.1.0-grok45 · 23163 in / 2960 out tokens · 28908 ms · 2026-07-15T12:52:26.347025+00:00 · methodology

0 comments
read the original abstract

Advances in vision-language models (VLMs) have achieved remarkable success on complex multimodal reasoning tasks, leading to the assumption that they should also excel at reading analog clocks. However, contrary to this expectation, our study reveals that reading analog clocks in real-world environments remains a significant challenge for state-of-the-art VLMs. Existing analog clock datasets are largely synthetic or planar with limited stylistic diversity and minimal background context, failing to capture the visual variability of real-world scenes. As a result, VLMs trained on such data exhibit weak spatiotemporal reasoning, frequently confusing the hour and minute hands and struggling under common visual conditions such as occlusion, lighting variation, and cluttered backgrounds. To address this issue, we introduce TickTockVQA, a human-annotated dataset containing analog clocks in diverse real-world scenarios. TickTockVQA provides explicit hour and minute annotations, and includes an AM/PM tag when it is inferable from the visual context. Furthermore, we propose Swap-DPO, a direct preference optimization-based fine-tuning framework to align model reasoning toward accurate time interpretation. Experimental results demonstrate that our approach substantially enhances clock reading accuracy and robustness under real-world conditions, establishing a foundation for future research on spatiotemporal reasoning and visual understanding in VLMs.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. State Beyond Appearance: Diagnosing and Improving State Consistency in Dial-Based Measurement Reading

    cs.CV 2026-04 unverdicted novelty 6.0

    MLLMs ignore dial state geometry and cluster by appearance, causing inconsistency under variations; TriSCA's state-distance alignment, metadata supervision, and objective alignment improve robustness on clock and gaug...

Reference graph

Works this paper leans on

44 extracted references · 10 linked inside Pith · cited by 1 Pith paper

  1. [1]

    As VLMs [2, 6, 11, 37, *Equal contribution

    Introduction Reading time from an analog clock is an everyday skill for humans, yet it remains a challenging problem for modern vision-language models (VLMs). As VLMs [2, 6, 11, 37, *Equal contribution. 0 500 1000 1500 2000 2500 3000 3500 4000 Training Steps 90 100 110 120 130 140 150Mean Absolute Error (minutes) Best: 99.9 min Zero-shot 148.6 min Zero-sh...

    Pith/arXiv arXiv 2000

  2. [2]

    Visual Question Answering Visual question answering (VQA) has become a central component in both evaluating and improving the capabili- ties of modern vision-language models

    Related Work 2.1. Visual Question Answering Visual question answering (VQA) has become a central component in both evaluating and improving the capabili- ties of modern vision-language models. While benchmarks such as TextVQA [35], ChartQA [25], DocVQA [26], OK-VQA [24] and VQAv2 [10] are widely used for performance measurement, numerous studies have show...

  3. [3]

    TickTockVQA: A Real-World Benchmark 3.1. Collection Pipeline We curated approximately 12k real-world clock images from diverse sources, including COCO [22], SBU Cap- tions [27], Visual Genome (VG) [16], ImageNet [7], Open Images (OID) [17], Conceptual Captions 12M (CC12M) [3], and movie frames (e.g., theClock Movie) [39]. For COCO, VG, ImageNet [7], and O...

    1953

  4. [4]

    Base Models Our method is applied to multiple state-of-the-art open- source VLMs to demonstrate its generality and scalabil- ity

    Method 4.1. Base Models Our method is applied to multiple state-of-the-art open- source VLMs to demonstrate its generality and scalabil- ity. Specifically, we adopt Qwen2.5-VL-7B [2], Llama-3.2- 11B [11], and Gemma3-12B [37] as the base architectures. 4.2. Fine-tuning Strategy To enhance the clock-reading capabilities of the base VLM, we propose a two-sta...

  5. [5]

    If the SFT model produces a clearly incorrect prediction, we directly use it asy l

  6. [6]

    This strategy yields approximately 7k preference pairs, enabling the model to focus on clock-specific ambigui- ties—particularly hour–minute hand confusion

    If the prediction is near-correct, we generate a hard neg- ative by applying the Swap-DPO transformation to the ground-truth time. This strategy yields approximately 7k preference pairs, enabling the model to focus on clock-specific ambigui- ties—particularly hour–minute hand confusion. This entire process is summarized in Algorithm 1

  7. [7]

    Experiments Unless otherwise stated, all reported results and figures use Llama-3.2-11B [11] as the backbone. 5.1. Implementation Details In the SFT stage, we adapt the model to the clock domain using LoRA [13]. It is applied to all linear layers in both the vision tower and the language model, excluding the lan- guage model head and embedding layers. Dur...

  8. [8]

    Conclusion In this work, we examine the long-standing challenge of analog clock reading for vision-language models—a task that requires fine-grained spatiotemporal reasoning. We in- troduceTickTockVQA, a 12k real-world benchmark with diverse visual–temporal annotations, and proposeSwap- DPO, a targeted preference-alignment method designed to correct hour–...

  9. [9]

    SFT establishes strong baseline but exhibits hand con- fusion:Supervised fine-tuning substantially improves per- formance across all models (e.g., Llama: 1.41% zero-shot �45.8% SFT). However, a persistent 2.32–2.90pp gap be- tween B and S metrics indicates that 5–7% of errors are (1) Qwen2.5-VL-7B:We apply full fine-tuning (no LoRA) due to its relatively ...

  10. [10]

    The larger per-device batch size (8 vs

    on all linear layers including the vision tower to reduce memory footprint. The larger per-device batch size (8 vs. 4 for Qwen/Gemma) is possible due to LoRA’s parameter ef- ficiency—only∼2% of parameters are trainable. Lazy pre- processing (on-the-fly image loading) accelerates training. Flash Attention 2 is supported and enabled. (3) Gemma3-12B:Similar ...

  11. [11]

    D.2.3) when swap-based negatives are used

    Times near 12:00: • Input: 12:00→Output: 12:00 (degenerate case, both hands point up) •Handling:These cases yield degenerate swaps (i.e., SWAPHANDS(y gt) =y gt) and are filtered by our valida- tion checks (Sec. D.2.3) when swap-based negatives are used

  12. [12]

    Near-overlapping hands: • Example:1:05(θ h = 32.5◦,θ m = 30◦) • Swapped:1:05(nearly identical) • Handling: Such cases are filtered out by our distinctness / temporal-distance checks Section. D.2.3

  13. [13]

    Half-hour positions: • Example: 3:30→Swapped: 6:18 • Hour hand at 105° (halfway between 3 and 4) • Minute hand at 180° (pointing at 6) •Handling:Works as intended

  14. [14]

    Rounding behavior: • We use floor division for hours:h new =⌊θ m/30⌋ • We use modulo for minutes:m new = (θh/6) mod 60 • This ensures outputs remain in valid ranges:h∈[0,11], m∈[0,59] D.2.3. Quality Control and Validation We perform the following validation checks on generated preference pairs: 1.Format validation:Bothy w andy l must be valid HH:MM string...

  15. [15]

    Select the most prominent analog clock (ignore digital displays)

  16. [16]

    Hour hand = short/thick; Minute hand = long/thin (ignore seconds)

  17. [17]

    Ambiguity resolution: if between marks, use lower hour and nearest minute

  18. [18]

    Output format: HH:MM (12-hour, leading zero, no AM/PM)

  19. [19]

    It omits training-specific instructions (multi-clock selection, NO CLOCK fallback) while preserving core task re- quirements

    Fallback: output ”NO CLOCK” if no analog clock visible Table S6.Inference prompt for evaluation.This prompt is used consistently across all test evaluations, ablation studies, and model comparisons. It omits training-specific instructions (multi-clock selection, NO CLOCK fallback) while preserving core task re- quirements. Inference Prompt Find the most p...

  20. [20]

    A survey of state of the art large vision language models: Alignment, benchmark, evaluations and challenges.arXiv preprint arXiv:2501.02189, 2025

    Zongxia Li, Xiyang Wu, Hongyang Du, Fuxiao Liu, Huy Nghiem, and Guangyao Shi. A survey of state of the art large vision language models: Alignment, benchmark, evaluations and challenges.arXiv preprint arXiv:2501.02189, 2025. 3

    Pith/arXiv arXiv 2025

  21. [21]

    Reasoning paths with reference objects elicit quan- titative spatial reasoning in large vision-language models

    Yuan-Hong Liao, Rafid Mahmood, Sanja Fidler, and David Acuna. Reasoning paths with reference objects elicit quan- titative spatial reasoning in large vision-language models. arXiv preprint arXiv:2409.09788, 2024. 2

    Pith/arXiv arXiv 2024

  22. [22]

    Ctrl- adapter: An efficient and versatile framework for adapt- ing diverse controls to any diffusion model.arXiv preprint arXiv:2404.09967, 2024

    Han Lin, Jaemin Cho, Abhay Zala, and Mohit Bansal. Ctrl- adapter: An efficient and versatile framework for adapt- ing diverse controls to any diffusion model.arXiv preprint arXiv:2404.09967, 2024. 7

    Pith/arXiv arXiv 2024

  23. [23]

    Microsoft coco: Common objects in context

    Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ar, and C Lawrence Zitnick. Microsoft coco: Common objects in context. In European conference on computer vision, pages 740–755. Springer, 2014. 3

    2014

  24. [24]

    Mirage: A multi- modal benchmark for spatial perception, reasoning, and in- telligence.arXiv preprint arXiv:2505.10604, 2025

    Chonghan Liu, Haoran Wang, Felix Henry, Pu Miao, Ya- jie Zhang, Yu Zhao, and Peiran Wu. Mirage: A multi- modal benchmark for spatial perception, reasoning, and in- telligence.arXiv preprint arXiv:2505.10604, 2025. 3

    Pith/arXiv arXiv 2025

  25. [25]

    Ok-vqa: A visual question answering benchmark requiring external knowledge

    Kenneth Marino, Mohammad Rastegari, Ali Farhadi, and Roozbeh Mottaghi. Ok-vqa: A visual question answering benchmark requiring external knowledge. InProceedings of the IEEE/cvf conference on computer vision and pattern recognition, pages 3195–3204, 2019. 3

    2019

  26. [26]

    Chartqa: A benchmark for question answer- ing about charts with visual and logical reasoning

    Ahmed Masry, Xuan Long Do, Jia Qing Tan, Shafiq Joty, and Enamul Hoque. Chartqa: A benchmark for question answer- ing about charts with visual and logical reasoning. InFind- ings of the association for computational linguistics: ACL 2022, pages 2263–2279, 2022. 3

    2022

  27. [27]

    Docvqa: A dataset for vqa on document images

    Minesh Mathew, Dimosthenis Karatzas, and CV Jawahar. Docvqa: A dataset for vqa on document images. InProceed- ings of the IEEE/CVF winter conference on applications of computer vision, pages 2200–2209, 2021. 3

    2021

  28. [28]

    Im2text: Describing images using 1 million captioned pho- tographs.Advances in neural information processing sys- tems, 24, 2011

    Vicente Ordonez, Girish Kulkarni, and Tamara Berg. Im2text: Describing images using 1 million captioned pho- tographs.Advances in neural information processing sys- tems, 24, 2011. 3

    2011

  29. [29]

    Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023

    Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas M ¨uller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion mod- els for high-resolution image synthesis.arXiv preprint arXiv:2307.01952, 2023. 7

    Pith/arXiv arXiv 2023

  30. [30]

    Spatial preference rewarding for mllms spa- tial understanding

    Han Qiu, Peng Gao, Lewei Lu, Xiaoqin Zhang, Ling Shao, and Shijian Lu. Spatial preference rewarding for mllms spa- tial understanding. InProceedings of the IEEE/CVF Inter- national Conference on Computer Vision, pages 720–730,

  31. [31]

    Direct preference optimization: Your language model is secretly a reward model.Advances in neural information processing systems, 36:53728–53741, 2023

    Rafael Rafailov, Archit Sharma, Eric Mitchell, Christo- pher D Manning, Stefano Ermon, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model.Advances in neural information processing systems, 36:53728–53741, 2023. 2, 4

    2023

  32. [32]

    Under pressure: learning-based analog gauge reading in the wild

    Maurits Reitsma, Julian Keller, Kenneth Blomqvist, and Roland Siegwart. Under pressure: learning-based analog gauge reading in the wild. In2024 IEEE International Con- ference on Robotics and Automation (ICRA), pages 14–20. IEEE, 2024. 1

    2024

  33. [33]

    High-resolution image synthesis with latent diffusion models

    Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Bj ¨orn Ommer. High-resolution image synthesis with latent diffusion models. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 10684–10695, 2022. 7

    2022

  34. [34]

    Clockbench: Visual time benchmark where hu- mans beat the clock, llms don’t, 2025

    Alek Safar. Clockbench: Visual time benchmark where hu- mans beat the clock, llms don’t, 2025. 2, 8

    2025

  35. [35]

    Lost in time: Clock and calendar understanding challenges in multimodal llms.arXiv preprint arXiv:2502.05092, 2025

    Rohit Saxena, Aryo Pradipta Gema, and Pasquale Minervini. Lost in time: Clock and calendar understanding challenges in multimodal llms.arXiv preprint arXiv:2502.05092, 2025. 1, 2, 3

    Pith/arXiv arXiv 2025

  36. [36]

    Towards vqa models that can read

    Amanpreet Singh, Vivek Natarajan, Meet Shah, Yu Jiang, Xinlei Chen, Dhruv Batra, Devi Parikh, and Marcus Rohrbach. Towards vqa models that can read. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8317–8326, 2019. 3

    2019

  37. [37]

    Mind the gap: Benchmarking spatial reasoning in vision-language models.arXiv preprint arXiv:2503.19707,

    Ilias Stogiannidis, Steven McDonagh, and Sotirios A Tsaf- taris. Mind the gap: Benchmarking spatial reasoning in vision-language models.arXiv preprint arXiv:2503.19707,

    Pith/arXiv arXiv

  38. [38]

    Gemma 3 technical report.arXiv preprint arXiv:2503.19786, 2025

    Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Ta- tiana Matejovicova, Alexandre Ram ´e, Morgane Rivi `ere, et al. Gemma 3 technical report.arXiv preprint arXiv:2503.19786, 2025. 1, 3, 4, 5, 6

    Pith/arXiv arXiv 2025

  39. [39]

    Vapr–vision-language preference alignment for reasoning

    Rohan Wadhawan, Fabrice Y Harel-Canada, Zi-Yi Dou, Suhaila Shakiah, Robinson Piramuthu, and Nanyun Peng. Vapr–vision-language preference alignment for reasoning. arXiv preprint arXiv:2510.01700, 2025. 3

    arXiv 2025

  40. [40]

    It’s about time: Analog clock reading in the wild

    Charig Yang, Weidi Xie, and Andrew Zisserman. It’s about time: Analog clock reading in the wild. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 2508–2517, 2022. 2, 3, 4, 6, 7

    2022

  41. [41]

    Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for ex- pert agi

    Xiang Yue, Yuansheng Ni, Kai Zhang, Tianyu Zheng, Ruoqi Liu, Ge Zhang, Samuel Stevens, Dongfu Jiang, Weiming Ren, Yuxuan Sun, et al. Mmmu: A massive multi-discipline multimodal understanding and reasoning benchmark for ex- pert agi. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 9556– 9567, 2024. 3

    2024

  42. [42]

    Adding conditional control to text-to-image diffusion models

    Lvmin Zhang, Anyi Rao, and Maneesh Agrawala. Adding conditional control to text-to-image diffusion models. In Proceedings of the IEEE/CVF international conference on computer vision, pages 3836–3847, 2023. 7

    2023

  43. [43]

    Roborefer: Towards spatial referring with reasoning in vision-language models for robotics.arXiv preprint arXiv:2506.04308, 2025

    Enshen Zhou, Jingkun An, Cheng Chi, Yi Han, Shanyu Rong, Chi Zhang, Pengwei Wang, Zhongyuan Wang, Tiejun Huang, Lu Sheng, et al. Roborefer: Towards spatial referring with reasoning in vision-language models for robotics.arXiv preprint arXiv:2506.04308, 2025. 1

    arXiv 2025

  44. [44]

    Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025

    Jinguo Zhu, Weiyun Wang, Zhe Chen, Zhaoyang Liu, Shen- glong Ye, Lixin Gu, Hao Tian, Yuchen Duan, Weijie Su, Jie Shao, et al. Internvl3: Exploring advanced training and test-time recipes for open-source multimodal models.arXiv preprint arXiv:2504.10479, 2025. 1, 3, 6

    Pith/arXiv arXiv 2025