Pith. sign in

REVIEW 4 major objections 4 minor 33 references

VectraYX-Vision-1B: A Sub-2B Spanish/LATAM Cybersecurity Vision-Language Model with Structured Visual Reasoning and Native Tool Use

T0 review · 4 major / 4 minor · reviewed 2026-08-14 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read A sub-2B Spanish-language cybersecurity model with a complete vision pipeline is released, and its central result is negative: after about 16 million training tokens it writes fluent Spanish but still scores near zero at recognizing what…

desk verdict Honest negative result with a genuinely new NoPE-visual question, but the headline 'ignores image content' claim is undercut by a mismatched text-only control and missing error bars. read the letter →

arxiv 2608.08477 v1 pith:UT4WVILY submitted 2026-08-09 cs.CL

Juan S. Santillana This is my paper · ORCID
classification cs.CL
keywords vision-languagemodelcybersecuritySpanishNLPvisualgroundingnegativeresultpositionalencodingtooluse
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper builds and releases VectraYX-Vision-1B, a sub-2B Spanish/LATAM cybersecurity vision-language model, but the result it reports is a negative one: after roughly 16 million tokens of vision fine-tuning, the model writes fluent Spanish yet ignores the image, scoring only 0.08 on tool identification in a 50-item held-out benchmark. The design is complete and released—a frozen vision encoder, a projector, a 1.04B Spanish security decoder, native reasoning and tool-call tokens, and an offline export path—so the bottleneck identified is training volume, not plumbing. The paper's central claim is that visual grounding at this scale requires substantially more vision SFT, and it releases a falsifiable three-variant ablation to test whether periodic no-positional-encoding layers help or hurt attention over the injected 729-token visual block. If that claim is right, the practical payoff is an unoccupied niche: a sub-4GB, air-gapped, Spanish-native security assistant that can actually read a disassembly pane or a packet capture.

What carries the argument

The load-bearing mechanism is the injection of a 729-token visual block into a decoder whose layers alternate rotary position embeddings (RoPE, on three of every four layers) with layers that carry no positional encoding (NoPE, on every fourth layer). RoPE imposes an arbitrary left-to-right order on a 2D patch grid, while NoPE layers treat the visual block as permutation-equivariant; whether this asymmetry helps or hurts is the paper's central open question. The discriminating experiment is a three-variant ablation: V0 keeps the released NoPE-every-4 schedule, V1 makes every layer RoPE, and V2 adds a learned 2D positional embedding to the visual tokens, all trained through the same alignment, instruct, and think-plus-tools pipeline. The sign of the difference in visual-grounding scores between V0 and V1 is what separates the hypothesis that NoPE helps from the hypothesis that an explicit 2D signal is needed.

What would settle it

Run the released pipeline with the prescribed remediation (longer SFT, at least 60 percent replay, lower LLM learning rate) and score the same 50 B6 items: if tool-identification stays near 0.08 while the text-only control stays near 0.02, the negative result is a training-volume finding; if it climbs well above the control, the deficit was volume all along. Separately, train the V1 all-RoPE variant on identical data: if its visual-grounding score is not clearly different from V0's, the periodic-NoPE question is answered as having no effect rather than as helpful or harmful.

Watch

Extended reading notes

Core claim

On the paper's own terms, the central finding is that VectraYX-Vision-1B does not visually ground at the current training volume. The best valid checkpoint reaches B6 tool-identification 0.08 and answer-correctness 0.06 on 50 held-out synthetic screenshots, while a text-only control scores 0.02, showing that the benchmark cannot be answered from text alone and that the image signal, though real, is barely above chance. The paper reports this as a negative result rather than a placeholder: it prescribes a longer SFT run with at least 60 percent replay and a lower learning rate, and it documents a checkpoint-loader bug that had briefly masqueraded as training collapse. The architectural question the system is built to answer is whether the backbone's periodic layers without positional encoding help or hurt attention over the injected 729-token visual block; that question remains open and is released as a falsifiable three-variant ablation.

Load-bearing premise

The load-bearing premise is that the 50-item benchmark honestly measures visual grounding: the synthetic screenshots must stand in for real tool UIs, the generated answers must be factually correct, and the text-only control must stay near zero—conditions the paper itself says are not yet audited.

Editorial extensions

If this is right

  • A remediated run with longer vision SFT and at least 60 percent text replay is the paper's prescribed next step; if it lifts B6 substantially, the negative result becomes a scaling finding rather than a design failure.
  • Because the text-only control sits at 0.02 tool-identification, any future B6 score can be attributed to vision only if it clearly clears that floor, which makes the released benchmark reusable as a grounding probe.
  • The V0-versus-V1 ablation outcome will bear on every future vision-language model built on a NoPE-family text backbone, since the same visual-block injection pattern will recur.
  • The loader-bug diagnostic—checking that the embedding norm returns to its trained value after export—is a concrete safeguard for any projector-style model export pipeline.
  • Retrained to competence, the released stack would imply a sub-4GB offline Spanish-language security tool that identifies tools and reads on-screen technical facts without cloud connectivity.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: the near-zero answer-correctness scores, including 0.000 for a strong cloud VLM, suggest the exact-fact metric is extremely unforgiving, and a token-overlap or partial-credit variant would separate 'sees the image' from 'quotes the exact string' more cleanly.
  • Editorial inference: the unquantified gap between synthetic renders and real tool screenshots may be as large a bottleneck as training volume; a cheap test would be to fine-tune on a few hundred real screenshots and compare B6 on real versus rendered images.
  • Editorial inference: attention maps on NoPE versus RoPE layers over a synthetic 2D grid task could give a preliminary read on the central hypothesis without another full fine-tune.
  • Editorial inference: if the V0/V1 comparison favors NoPE, future VLM designers may deliberately interleave position-free layers to preserve permutation-equivariance over visual tokens; if it favors RoPE, all-RoPE becomes the safer default for the injection recipe.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper presents VectraYX-Vision-1B, a LLaVA-style sub-2B Spanish/LATAM cybersecurity vision-language model coupling a frozen SigLIP encoder to a 1.04B Spanish security decoder. It describes the architecture, a four-phase curriculum, a 14,596-pair multimodal corpus, two new benchmarks (B6_vision, B7_think), and reports a negative preliminary result: after roughly 16M vision-SFT tokens the model shows near-zero visual grounding, with best B6 tool-identification 0.08 on 50 held-out synthetic screenshots versus 0.02 for a text-only control. The paper also documents a checkpoint-loader bug that initially masqueraded as training collapse, releases code, configs, weights, checkpoints, and the benchmark harness, and proposes a three-variant NoPE×vision ablation as a falsifiable architectural question whose results are pending a remediated run.

Significance. If the negative result is properly supported, it is a useful and honestly reported finding: it suggests that the standard LLaVA-style recipe at roughly 16M vision-SFT tokens is insufficient for text-dense security-tool imagery, and it gives a concrete remediation (longer SFT, ≥60% replay, lower LR). The released artifacts—weights, per-step checkpoints, benchmark harness, synthetic renderer, and ablation designs—are a genuine strength and make the work auditable. The NoPE×vision question is interesting and clearly scoped. However, the headline empirical claim currently rests on a confounded comparison and on single-seed 50-item numbers without the bootstrap intervals promised in Section 6.5, so the significance is contingent on fixing those points.

major comments (4)
  1. [Section 7.3 / Table 5] The text-only control row is the v3A-p2 backbone, whereas the with-image row it is compared with is v3B. Table 4 shows these checkpoints differ in step count, learning rate, and tok_emb freezing status, so the 0.08-vs-0.02 difference is confounded by checkpoint identity rather than isolating image presence. The sentence in Section 7.3 claiming a "small but positive vision signal ... once training volume is held constant" is not supported by the table. Please evaluate v3B with the image removed (and ideally v3A-p2 with the image present) before claiming the model "ignores image content"; otherwise the correct conclusion is only that the visual pathway has not been shown to help on this benchmark.
  2. [Section 6.5 vs. Section 7.3 / Table 5] The protocol in Section 6.5 explicitly commits to paired bootstrap confidence intervals for every pairwise comparison, and the abstract says the paper compensates with per-item bootstrap intervals, but Table 5 contains no intervals and I could not find item-level scores in the text. With n=50 and a single seed, the observed 0.08-vs-0.02 difference has a simple binomial 95% interval that includes zero, so the claimed positive vision signal is not statistically supported. Please report the intervals or explicitly withdraw the comparison as a strength claim.
  3. [Sections 5.3, 6.6, and 9] The B6 benchmark and the training corpus are produced by the same synthetic renderer and LLM-in-the-loop pipeline, and Section 9 admits that no human audit has been performed and that the real-screenshot domain gap is unquantified. The text-only control rules out leakage through the question text, but it does not rule out the possibility that the model has overfit the synthetic image distribution or that the benchmark is insensitive to real visual grounding. For a negative result this is less damaging than for a positive one, but the paper should explicitly scope the claim to the synthetic benchmark and, if feasible, add a small real-screenshot subset with human-audited labels.
  4. [Section 7.3 / Table 5] GPT-4o achieves B6 tool-id 0.940 but B6_correct 0.000. The explanation that the correct-fact metric is harder does not account for a frontier model scoring zero on all 50 items; this suggests either the metric is mis-calibrated or the ground-truth annotations are unreliable, both of which affect the benchmark's sensitivity. Please show a few scored examples, including GPT-4o failures, and clarify the rubric for answer_correctness.
minor comments (4)
  1. [Section 6.2 / Table 5] The terms "answer_correctness" and "B6_correct" are used interchangeably; please choose one and use it consistently.
  2. [Section 7.2] The statement that freezing the backbone "preserves language competence exactly" is too strong for single-seed evaluation; "no regression was observed" is the supportable claim.
  3. [Section 3.4] The paper says RoPE tables are extended to cover 729+max_seq_len positions; please state the numeric max_seq_len used in training so the positional budget is reproducible.
  4. [Section 5.4] The reported generation cost of $3.74 for 6,077 pairs would be more useful if the exact LLM and version used for QA generation were stated.

Circularity Check

0 steps flagged · score 1.0 of 10

No circular derivation: the negative visual-grounding result is an empirical measurement, not a self-fulfilling prediction; the main validity concerns are a mismatched text-only control and shared synthetic benchmark/training pipeline, which are methodological limitations rather than circularity.

full rationale

The paper makes no claim that reduces by construction to its inputs. The central empirical result—B6 tool-identification 0.08 for v3B versus 0.02 for the text-only control (Table 5, Section 7.3)—is a measured outcome, not a derived prediction, and its near-zero value works against, not for, any same-distribution inflation from the shared synthetic renderer. The benchmark and training corpus do come from the same pipeline (Sections 5.3 and 6.2), and Section 9 admits "The benchmark and the model share design DNA, risking benchmark-overfitting"; that is a real evaluation-validity limitation but not circular equivalence, because B6 is not derived from the training loss or fitted parameters. The text-only control is run on a different checkpoint (v3A-p2) than the with-image row (v3B), so the 0.08-vs-0.02 contrast is confounded by checkpoint identity; this weakens the 'ignores image content' inference but is a methodological flaw, not a self-referential reduction. The only self-citations (VectraYX-Nano [24] for the inherited curriculum and B1–B5) are infrastructure and are not load-bearing for the visual-grounding claim. The NoPE×vision ablation is explicitly open ("the outcome we genuinely cannot predict," Section 10), so no prediction is being smuggled in. Overall: no significant circularity; score 1 reflects the minor, non-load-bearing self-citation and the admitted shared-benchmark risk, not a circular derivation.

Assumptions & free parameters 0 free parameters · 4 assumptions · 0 invented entities

No numerical parameters are fitted to produce the central claim. Training hyperparameters (LR 1e-3, 2e-5, 1e-5; replay ratios 60:40, 50:50; step counts 228/819/546) are disclosed engineering choices, not fitted constants. Special tokens (<|think|>, <|tool_call|>) are vocabulary additions, not postulated scientific entities, so no invented entities are recorded.

assumptions (4)
  • domain assumption SigLIP-so400m features transfer to synthetic and real security-tool screenshots.
    Assumed when freezing the encoder; the paper flags a frozen encoder ceiling in Section 9 and has not run the frozen-vs-unfrozen ablation.
  • domain assumption Synthetic renders are representative of real tool UIs.
    B6 relies on synthetic images when the private set is absent; Section 9 admits the domain gap is unquantified.
  • domain assumption LLM-generated QA pairs are factually correct and do not leak answers into questions.
    Section 9 states no human audit has been performed; the paper itself lists answer leakage and factual errors as open risks.
  • domain assumption The 50-item B6 suite can distinguish visual grounding from noise.
    At n=50, a score of 0.08 corresponds to 4 items; the paper provides no bootstrap intervals in Table 5 despite the protocol in Section 6.5.

how reviews work

0 comments
Cite this review

Pith. "Pith review of VectraYX-Vision-1B: A Sub-2B Spanish/LATAM Cybersecurity Vision-Language Model with Structured Visual Reasoning and Native Tool Use." pith.science (2026). https://pith.science/paper/UT4WVILY

@misc{pith2026260808477,
  author       = {Pith},
  title        = {Pith review of: VectraYX-Vision-1B: A Sub-2B Spanish/LATAM Cybersecurity Vision-Language Model with Structured Visual Reasoning and Native Tool Use},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/UT4WVILY}},
  note         = {Machine review of arXiv:2608.08477}
}
read the original abstract

We present VectraYX-Vision-1B, a sub-2B vision-language model (VLM) for Spanish/LATAM cybersecurity imagery, coupling a frozen SigLIP-so400m encoder to a 1.04B Spanish/LATAM security decoder via an MLP. To our knowledge, it is the first sub-2B VLM specialized for cyber UI (IDA, Ghidra, Wireshark, Nmap, Metasploit, Volatility) that answers in Spanish, emits structured reasoning via native <|think|> tokens, invokes tools via Model Context Protocol (<|tool_call|>), and exports to llama.cpp's LLaVA mmproj format for air-gapped deployment. We report a negative preliminary visual-grounding result: despite fully functional pipelines, the current vision SFT (400-1900 steps, ~16M tokens) yields near-zero B6 scores (0.08 tool-identification), ignoring image content. We specify remediation (longer SFT, >=60% replay, lower LR) and expose a checkpoint-loader bug (unstripped llm. prefix) masquerading as training collapse. Crucially, we introduce a 3-variant ablation matrix (V0: NoPE-every-4, V1: all-RoPE, V2: NoPE+learned 2D) to study if periodic no-positional-encoding (NoPE) layers help or hurt attention over the 729-token visual block. Code, configs, and weights are released to establish priority on this architectural question. We provide B1-B5 for the text backbone, text controls, preliminary B6/B7 scores, wall times, GGUF efficiency on CPU, and a corpus of 14,596 QA pairs across 10 domains. We open-source all models and trajectories: jsantillana/vectrayx-1b, jsantillana/vectrayx-vision-1b, and jsantillana/vectrayx-vision-1b-checks.

Figures

Figures reproduced from arXiv: 2608.08477 by the authors.

Figure 1
Figure 1. VectraYX-Vision-1B data flow. The projector maps 729 SigLIP patch tokens to the 2048-dim decoder space; the projected tokens replace the <image> placeholder slot in the token sequence and are consumed by the decoder exactly like text tokens. frozen in phase 4a and optionally unfrozen thereafter (Section 4); we default to keeping it frozen throughout because the projector plus a trainable backbone already provide suf… view at source ↗

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

33 extracted references · 22 canonical work pages

  1. [1]

    Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, et al. 2024. Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone.arXiv preprint arXiv:2404.14219(2024)

    arXiv 2024

  2. [2]

    Ehsan Aghaei, Xi Niu, Waseem Shadid, and Ehab Al-Shaer. 2022. SecureBERT: A Domain-Specific Language Model for Cybersecurity. InSecurity and Privacy in Communication Networks (SecureComm)

    work page 2022

  3. [3]

    Anthropic. 2024. Model Context Protocol Specification. https:// modelcontextprotocol.io

    work page 2024

  4. [4]

    Gilles Baechler, Srinivas Sunkara, Maria Wang, Fedir Zubach, Hassan Mansoor, Vincent Etter, Victor Cărbune, Jason Lin, Jindong Chen, and Abhanshu Sharma

  5. [5]

    Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. 2023. Qwen-VL: A Versatile Vision-Language Model for Understanding, Localization, Text Reading, and Beyond.arXiv preprint arXiv:2308.12966(2023)

    arXiv 2023

  6. [6]

    Manish Bhatt, Sahana Chennabasappa, Cyrus Nikolaidis, Shengye Wan, Ivan Evtimov, Dominik Gabi, Daniel Song, Faizan Ahmad, Cornelius Aschermann, Lorenzo Fontana, et al . 2023. Purple Llama CyberSecEval: A Secure Coding Benchmark for Language Models.arXiv preprint arXiv:2312.04724(2023)

    arXiv 2023

  7. [7]

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. 2024. InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks. IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)(2024)

    work page 2024

  8. [8]

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xi- aohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al . 2021. An Image is Worth 16x16 Words: Trans- formers for Image Recognition at Scale.International Conference on Learning Representations (ICLR)(2021)

    work page 2021

Show all 33 references
  1. [9]

    Robert M French. 1999. Catastrophic Forgetting in Connectionist Networks. Trends in Cognitive Sciences3, 4 (1999), 128–135

    1999

  2. [10]

    Georgi Gerganov and llama.cpp contributors. 2023. llama.cpp: LLM inference in C/C++. https://github.com/ggerganov/llama.cpp

    2023

  3. [11]

    Yichen Gong, Delong Ran, Jinyuan Liu, Conglei Wang, Tianshuo Cong, Anyu Wang, Sisi Duan, and Xiaoyun Wang. 2023. Eyes Shut, Safety Off: Evaluating the Robustness of Multimodal LLMs against Visual Jailbreaks.arXiv preprint arXiv:2311.05608(2023)

    2023 arXiv

  4. [12]

    Hazim Hanif and Sergio Maffeis. 2022. VulBERTa: Simplified Source Code Pre- Training for Vulnerability Detection. InInternational Joint Conference on Neural Networks (IJCNN)

    2022

  5. [13]

    Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al . 2022. Training Compute-Optimal Large Language Models.Advances in Neural Information Processing Sys...

    2022

  6. [14]

    Wenyi Hong, Weihan Wang, Qingsong Lv, Jiazheng Xu, Wenmeng Yu, Junhui Ji, Yan Wang, Zihan Wang, Yuxiao Dong, Ming Ding, and Jie Tang. 2024. CogAgent: A Visual Language Model for GUI Agents. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2024

  7. [15]

    Adam Ibrahim, Benjamin Thérien, Kshitij Gupta, Mats L Richter, Quentin An- thony, Timothée Lesort, Eugene Belilovsky, and Irina Rish. 2024. Simple and Scalable Strategies to Continually Pre-train Large Language Models.Transactions on Machine Learning Research (TMLR)(2024)

    2024

  8. [16]

    Amirhossein Kazemnejad, Inkit Padhi, Karthikeyan Natesan Ramamurthy, Payel Das, and Siva Reddy. 2023. The Impact of Positional Encoding on Length Gener- alization in Transformers.Advances in Neural Information Processing Systems (NeurIPS)(2023)

    2023

  9. [17]

    Geewook Kim, Teakgyu Hong, Moonbin Yim, JeongYeon Nam, Jinyoung Park, Jinyeong Yim, Wonseok Hwang, Sangdoo Yun, Dongyoon Han, and Seunghyun Park. 2022. OCR-free Document Understanding Transformer. InEuropean Con- ference on Computer Vision (ECCV)

    2022

  10. [18]

    Hugo Laurençon, Léo Tronchon, Matthieu Cord, and Victor Sanh. 2024. What Matters When Building Vision-Language Models?Advances in Neural Informa- tion Processing Systems (NeurIPS)(2024)

    2024

  11. [19]

    Kenton Lee, Mandar Joshi, Iulia Turc, Hexiang Hu, Fangyu Liu, Julian Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, and Kristina Toutanova. 2023. Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding. InInternational Conference on Machin...

    2023

  12. [20]

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. 2023. BLIP-2: Boot- strapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models. InInternational Conference on Machine Learning (ICML)

    2023

  13. [21]

    Haotian Liu, Chunyuan Li, Yuheng Li, and Yong Jae Lee. 2024. Improved Baselines with Visual Instruction Tuning. InIEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2024

  14. [22]

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. 2023. Visual In- struction Tuning.Advances in Neural Information Processing Systems (NeurIPS) (2023)

    2023

  15. [23]

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sand- hini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al

  16. [24]

    Santillana

    Juan S. Santillana. 2026. VectraYX-Nano: A 42M-Parameter Spanish Cyberse- curity Language Model with Curriculum Learning and Native Tool Use.arXiv preprint arXiv:2605.13989(2026)

    2026 arXiv

  17. [25]

    Hao Shao, Shengju Qian, Han Xiao, Guanglu Song, Zhuofan Zong, Letian Wang, Yu Liu, and Hongsheng Li. 2024. Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning. InAdvances in Neural Information Processing S...

    2024

  18. [26]

    Jianlin Su, Murtadha Ahmed, Yu Lu, Shengfeng Pan, Wen Bo, and Yunfeng Liu. 2024. RoFormer: Enhanced Transformer with Rotary Position Embedding. Neurocomputing568 (2024)

    2024

  19. [27]

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. 2024. Qwen2-VL: Enhancing Vision-Language Model’s Perception of the World at Any Resolution.arXiv preprint arXiv:2409.12191(2024)

    2024 arXiv

  20. [28]

    Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain-of-Thought Prompting Elicits Reasoning in Large Language Models. InAdvances in Neural Information Processing Systems (NeurIPS)

    2022

  21. [29]

    Keen You, Haotian Zhang, Eldon Schoop, Floris Weers, Amanda Swearngin, Jeffrey Nichols, Yinfei Yang, and Zhe Gan. 2024. Ferret-UI: Grounded Mobile UI Understanding with Multimodal LLMs. InEuropean Conference on Computer Vision (ECCV)

    2024

  22. [30]

    Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. Sig- moid Loss for Language Image Pre-Training.IEEE/CVF International Conference on Computer Vision (ICCV)(2023)

    2023

  23. [31]

    Hong-Yu Zhou, Subathra Adithan, Julián N Acosta, Eric J Topol, and Pranav Rajpurkar. 2024. A Generalist Learner for Multifaceted Medical Image Interpre- tation.arXiv preprint arXiv:2405.07988(2024)

    2024 arXiv

  24. [2021]

    InInternational Conference on Machine Learning (ICML)

    Learning Transferable Visual Models from Natural Language Supervision. InInternational Conference on Machine Learning (ICML)

  25. [2024]

    arXiv preprint arXiv:2402.04615(2024)

    ScreenAI: A Vision-Language Model for UI and Infographics Understanding. arXiv preprint arXiv:2402.04615(2024)

    2024 arXiv

Pith tools

Reviewed August 14, 2026 · model on record in the stance chip above.