Pith. sign in

REVIEW 3 major objections 3 minor 70 references

MagicVL-2B: Empowering Vision-Language Models on Mobile Devices with Lightweight Visual Encoders via Curriculum Learning

T0 review · 3 major / 3 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read MagicVL-2B matches state-of-the-art vision-language accuracy while cutting on-device power use by 41.1%.

desk verdict The abstract's two headline numbers are unverifiable from this copy, and the body is corrupted, but the recipe is plausible and the paper deserves peer review if the full text matches the abstract. read the letter →

arxiv 2508.01540 v1 pith:WP7EHRVX submitted 2025-08-03 cs.CV cs.AI

Yi Liu , Xiao Xu , Zeyu Xu , Meng Zhang , Yibo Li , Haoyu Chen , Junkang Zhang , Qiang Wang
show 5 more authors
Jifa Sun Siling Lin Shengxun Cheng Lingshu Zhang Kang Wang
This is my paper
classification cs.CVcs.AI
keywords vision-languagemodelsmobileAIlightweightvisualencoderdynamicresolutioncurriculumlearningon-devicepowerconsumptionefficientmultimodalinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces MagicVL-2B, a vision-language model built for flagship smartphones. It claims that a visual encoder with fewer than 100 million parameters, combined with a redesigned dynamic resolution scheme and a multimodal curriculum learning strategy, reaches benchmark accuracy on par with current state-of-the-art models while reducing on-device power consumption by 41.1%. The point is to show that advanced multimodal intelligence can run directly on phones rather than in the cloud. The reader should care because accuracy and efficiency usually trade off, and the paper argues that the trade-off can be broken by training a small encoder more carefully instead of making it larger.

What carries the argument

The load-bearing mechanism is the pairing of a lightweight visual encoder with a training curriculum. The encoder, under 100M parameters, keeps vision-side compute low. The redesigned dynamic resolution scheme lets the model allocate image tokens adaptively to the input's natural aspect ratio rather than resizing it into a fixed grid, avoiding wasteful tokens. The multimodal curriculum learning strategy incrementally raises task difficulty and data information density, which the paper argues lets the small encoder learn stronger visual representations than the same architecture trained on homogeneous data. Together these mechanisms lower both the number of image tokens and the effective compute needed per image, which is what produces the reported power saving.

What would settle it

Place MagicVL-2B and one quoted state-of-the-art model on the same flagship phone, using the same inference engine, quantization, and image resolutions, and measure wall-clock power draw and benchmark accuracy; if the power gap falls well below 41.1% or accuracy drops significantly, the central claim fails.

Watch

Extended reading notes

Core claim

On its own terms, the paper's central discovery is that the performance ceiling of a compact vision-language model is set less by encoder size than by how the encoder's limited capacity is trained and how its image tokens are produced. MagicVL-2B uses a sub-100M-parameter visual encoder; a dynamic resolution scheme that adaptively chooses image tokenization without distorting image dimensions; and a curriculum schedule that ramps up task difficulty and data information density during training. With these three components, the paper reports accuracy matching current state-of-the-art models across standard VLM benchmarks while measuring 41.1% lower on-device power consumption.

Load-bearing premise

The entire result rests on the power measurement and benchmark comparisons being run under the same conditions for MagicVL-2B and the state-of-the-art baselines; if the baselines ran with different resolution, token counts, or quantization, the 41.1% saving and accuracy parity would not be a fair comparison.

Editorial extensions

If this is right

  • If the claim holds, flagship phones can run vision-language assistants locally, removing cloud latency and privacy concerns.
  • The sub-100M encoder makes the vision backbone small enough to fit the on-device memory budget of a phone while leaving room for the language model.
  • A 41.1% power saving at equal accuracy makes energy a key axis of comparison for mobile AI models, not just benchmark scores.
  • The curriculum strategy provides a recipe for training other compact multimodal models without scaling up parameters.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If the power saving comes mostly from fewer image tokens, then the design could transfer to any vision-language model whose inference cost is token-bound, not just the 2B model.
  • The curriculum schedule may be the most portable insight: it suggests a general recipe for coaxing stronger features out of small encoders in other multimodal settings, such as audio or video, where data density can be staged.
  • A direct testable extension is to measure how the 41.1% power figure changes under different quantization levels or image resolutions; if the saving is robust, the design could generalize beyond flagship phones to mid-range devices.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 3 minor

Summary. The manuscript introduces MagicVL-2B, a 2-billion-parameter vision-language model designed for flagship smartphones. The proposed contributions are a lightweight visual encoder with fewer than 100M parameters, a redesigned dynamic resolution scheme that controls image token generation, and a multimodal curriculum learning strategy that progressively increases task difficulty and information density. The abstract claims that MagicVL-2B matches the accuracy of current state-of-the-art models while reducing on-device power consumption by 41.1%. The body text, however, is rendered as unreadable mojibake in the provided version, and the only legible page header refers to a different arXiv record (2508.01547, cs.HC) rather than the paper under review (2508.01540, cs.CV).

Significance. If the claimed results are correct, MagicVL-2B would represent a practically important step toward deploying vision-language models on mobile devices: a compact model with a sub-100M-parameter visual encoder that matches the accuracy of much larger state-of-the-art systems while consuming substantially less power would be of clear value to the mobile AI community. The proposed curriculum learning strategy and dynamic resolution scheme are also potentially interesting as general techniques for training compact VLMs. However, the significance cannot be assessed from the submitted manuscript because the central empirical claims are stated in the abstract without any of the protocol details needed for verification, and the body text is unreadable. The paper therefore provides no verifiable evidence for its headline results.

major comments (3)
  1. [Abstract (final two sentences)] The central claim that MagicVL-2B "matches the accuracy of current state-of-the-art models while reducing on-device power consumption by 41.1%" is not accompanied by any specification of the comparison protocol. The abstract does not name the state-of-the-art baseline models, the mobile device hardware, the inference engine, the quantization settings, the input resolutions, the number of generated tokens used in power measurement, or the error bars on the accuracy and power numbers. Without these details, the 41.1% figure and the accuracy-parity claim cannot be independently checked; unequal test conditions would make both claims uninformative.
  2. [Full text (all sections)] The provided body text is almost entirely mojibake or corrupted characters, making it impossible to read the methodology, the curriculum learning schedule, the dynamic resolution implementation, the benchmark setup, or the evaluation tables. The only legible fragment is a page header reading "arXiv:2508.01547v1 [cs.HC] 3 Aug 2025", which does not match the record under review (arXiv:2508.01540, cs.CV). As a result, no section of the paper can be checked for internal consistency, correctness of the training procedure, or validity of the reported measurements. A manuscript in this state is not reviewable.
  3. [Abstract (method description)] The abstract introduces a multimodal curriculum learning strategy and a dynamic resolution scheme with token-budget settings, but it does not specify how these are determined or whether the reported results are sensitive to them. In particular, the curriculum stage schedule and the token-budget hyperparameters are free parameters; since the body text is unreadable, there is no evidence that these choices are principled or that the 41.1% power reduction is not simply an artifact of a smaller token budget for MagicVL-2B than for the baselines. The paper needs at least an ablation or a sensitivity analysis for these parameters to support the claimed advantage.
minor comments (3)
  1. [Abstract (terminology)] The phrase "current state-of-the-art models" is too vague for a quantitative claim; the paper should name the specific baselines and report the benchmark scores in a table rather than only in prose.
  2. [Abstract (power claim)] The 41.1% power reduction is presented without any statistical or measurement context; the manuscript should state the number of repeated measurements, the measurement device, and the variance or confidence interval.
  3. [Full text (metadata)] The page header displaying a different arXiv ID appears to be a metadata error; this should be corrected to avoid misattribution, although the more serious problem is the unreadable body text.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity found; the available abstract reports external benchmark measurements and no derivation reduces to its own inputs.

full rationale

The only legible portion of the manuscript is the abstract, which claims that MagicVL-2B matches state-of-the-art accuracy on standard VLM benchmarks while reducing on-device power consumption by 41.1%. This is an empirical claim against external benchmarks and measurements, not a mathematical derivation that could reduce to its own inputs. The supplied body text is corrupted mojibake, and the visible page header carries a different arXiv identifier, but text corruption and unverifiability are not circularity. No equation, fitted parameter renamed as a prediction, self-citation load-bearing step, or ansatz smuggled in via citation can be identified from the available evidence. Under the hard rule that circularity may only be claimed when a specific reduction is quoted and exhibited, the honest finding is no significant circularity, with a score of 0.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

An empirical engineering paper: there are no invented entities and no derivation-level fitted constants. The ledger flags two things: the curriculum schedule and resolution token budgets, hand-designed training choices whose tuning is not disclosed, and the unstated measurement assumptions behind the headline power figure. The compact-encoder premise is a domain assumption that the abstract asserts but does not justify.

free parameters (2)
  • Curriculum stage schedule (task difficulty and information-density progression)
    The abstract credits the multimodal curriculum strategy for substantial improvement but does not specify how stages were ordered or how difficulty was increased; if the schedule was tuned against the evaluation benchmarks, the reported gains are partly fitted rather than predicted.
  • Dynamic resolution token-budget settings
    The redesigned resolution scheme adaptively decides image token counts; the specific budgets, aspect-ratio rules, or thresholds are not reported in the abstract and would be hand-chosen design parameters if tuned on the evaluation set.
assumptions (3)
  • domain assumption Benchmark accuracy on standard VLM benchmarks is a valid proxy for practical model capability
    The abstract's parity claim is stated entirely in benchmark terms, so the validity of these benchmarks as proxies for real-world capability is assumed without discussion.
  • domain assumption On-device power measurements are representative of real deployment conditions
    The 41.1% power claim requires a measurement setup (hardware, inference engine, quantization, token counts) that the abstract does not disclose, so the representativeness of the measurement is assumed.
  • domain assumption A sub-100M-parameter visual encoder retains enough visual information for VLM tasks
    The design premise is that a compact encoder plus dynamic resolution does not cost accuracy; the abstract asserts the outcome but does not justify the premise.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MagicVL-2B: Empowering Vision-Language Models on Mobile Devices with Lightweight Visual Encoders via Curriculum Learning." pith.science (2026). https://pith.science/paper/WP7EHRVX

@misc{pith2026250801540,
  author       = {Pith},
  title        = {Pith review of: MagicVL-2B: Empowering Vision-Language Models on Mobile Devices with Lightweight Visual Encoders via Curriculum Learning},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/WP7EHRVX}},
  note         = {Machine review of arXiv:2508.01540}
}
read the original abstract

Vision-Language Models (VLMs) have achieved remarkable breakthroughs in recent years, enabling a diverse array of applications in everyday life. However, the substantial computational and storage demands of VLMs pose significant challenges for their efficient deployment on mobile devices, which represent the most ubiquitous and accessible computing platforms today. In this work, we introduce MagicVL-2B, a novel VLM meticulously optimized for flagship smartphones. MagicVL-2B leverages a lightweight visual encoder with fewer than 100M parameters and features a redesigned dynamic resolution scheme that adaptively generates image tokens without excessive modification of image dimensions. To further enhance the performance of this compact encoder within VLMs, we propose a multimodal curriculum learning strategy that incrementally increases task difficulty and data information density throughout training. This approach substantially improves the model's performance across a variety of sub-tasks. Extensive evaluations on standard VLM benchmarks demonstrate that MagicVL-2B matches the accuracy of current state-of-the-art models while reducing on-device power consumption by 41.1%. These results establish MagicVL-2B as a practical and robust solution for real-world mobile vision-language applications, enabling advanced multimodal intelligence to run directly on smartphones.

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

70 extracted references · 15 canonical work pages

  1. [1]

    , " * write output.state after.block = add.period write newline

    ENTRY address archivePrefix author booktitle chapter edition editor eid eprint howpublished institution isbn journal key month note number organization pages publisher school series title type volume year label extra.label sort.label short.list INTEGERS output.state before.all mid.sentence after.sentence after.block FUNCTION init.state.consts #0 'before.a...

  2. [2]

    write newline

    " write newline "" before.all 'output.state := FUNCTION n.dashify 't := "" t empty not t #1 #1 substring "-" = t #1 #2 substring "--" = not "--" * t #2 global.max substring 't := t #1 #1 substring "-" = "-" * t #2 global.max substring 't := while if t #1 #1 substring * t #2 global.max substring 't := if while FUNCTION word.in bbl.in capitalize " " * FUNCT...

  3. [3]

    A.; Awan, A

    Abdin, M.; Jacobs, S. A.; Awan, A. A.; Aneja, J.; Awadallah, A.; Awadalla, H.; Bach, N.; Bahree, A.; Bakhtiari, A.; Behl, H.; et al. 2024. Phi-3 technical report: A highly capable language model locally on your phone. arXiv preprint arXiv:2404.14219

    arXiv 2024

  4. [5]

    Alayrac, J.-B.; Donahue, J.; Luc, P.; Miech, A.; Barr, I.; Hasson, Y.; Lenc, K.; Mensch, A.; Millican, K.; Reynolds, M.; et al. 2022. Flamingo : A visual language model for few-shot learning. NeurIPS, 35: 23716--23736

    work page 2022

  5. [6]

    M.; Firat, O.; Johnson, M.; Lepikhin, D.; Passos, A.; Shakeri, S.; Taropa, E.; Bailey, P.; Chen, Z.; et al

    Anil, R.; Dai, A. M.; Firat, O.; Johnson, M.; Lepikhin, D.; Passos, A.; Shakeri, S.; Taropa, E.; Bailey, P.; Chen, Z.; et al. 2023. Palm 2 technical report. arXiv preprint arXiv:2305.10403

    arXiv 2023

  6. [7]

    L.; and Paul, M

    Ankner, Z.; Blakeney, C.; Sreenivasan, K.; Marion, M.; Leavitt, M. L.; and Paul, M. 2024. Perplexed by perplexity: Perplexity-based data pruning with small reference models. arXiv preprint arXiv:2405.20541

    arXiv 2024

  7. [8]

    Computational Bottlenecks of Training Small-scale Large Language Models

    Ashkboos, S.; Mirzadeh, I.; Alizadeh, K.; Sekhavat, M. H.; Nabi, M.; Farajtabar, M.; and Faghri, F. 2024. Computational Bottlenecks of Training Small-scale Large Language Models. arXiv preprint arXiv:2410.19456

    work page Pith review arXiv 2024

  8. [9]

    Bai, J.; Bai, S.; Yang, S.; Wang, S.; Tan, S.; Wang, P.; Lin, J.; Zhou, C.; and Zhou, J. 2023. Qwen-vl: A frontier large vision-language model with versatile abilities. arXiv preprint arXiv:2308.12966

    arXiv 2023

Show all 70 references
  1. [10]

    Bai, S.; Chen, K.; Liu, X.; Wang, J.; Ge, W.; Song, S.; Dang, K.; Wang, P.; Wang, S.; Tang, J.; et al. 2025. Qwen2. 5-vl technical report. arXiv preprint arXiv:2502.13923

    2025 arXiv

  2. [11]

    Brown, T. B. 2020. Language models are few-shot learners. arXiv preprint arXiv:2005.14165

    2020 arXiv

  3. [12]

    Cai, M.; Yang, J.; Gao, J.; and Lee, Y. J. 2024. Matryoshka Multimodal Models. arXiv preprint arXiv:2405.17430

    2024 arXiv

  4. [13]

    Cha, J.; Kang, W.; Mun, J.; and Roh, B. 2024. Honeybee: Locality-enhanced Projector for Multimodal LLM. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2024

  5. [14]

    Chen, Z.; Wang, W.; Cao, Y.; Liu, Y.; Gao, Z.; Cui, E.; Zhu, J.; Ye, S.; Tian, H.; Liu, Z.; et al. 2024 a . Expanding performance boundaries of open-source multimodal models with model, data, and test-time scaling. arXiv preprint arXiv:2412.05271

    2024 arXiv

  6. [15]

    Chen, Z.; Wang, W.; Tian, H.; Ye, S.; Gao, Z.; Cui, E.; Tong, W.; Hu, K.; Luo, J.; Ma, Z.; et al. 2024 b . How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. arXiv preprint arXiv:2404.16821

    2024 arXiv

  7. [16]

    Chen, Z.; Wu, J.; Wang, W.; Su, W.; Chen, G.; Xing, S.; Zhong, M.; Zhang, Q.; Zhu, X.; Lu, L.; Li, B.; Luo, P.; Lu, T.; Qiao, Y.; and Dai, J. 2023. InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks. arXiv preprint arXiv:2312.14238

    2023 arXiv

  8. [17]

    Chen, Z.; Wu, J.; Wang, W.; Su, W.; Chen, G.; Xing, S.; Zhong, M.; Zhang, Q.; Zhu, X.; Lu, L.; et al. 2024 c . Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patte...

    2024

  9. [18]

    Chu, X.; Qiao, L.; Lin, X.; Xu, S.; Yang, Y.; Hu, Y.; Wei, F.; Zhang, X.; Zhang, B.; Wei, X.; et al. 2023. Mobilevlm: A fast, reproducible and strong vision language assistant for mobile devices. arXiv preprint arXiv:2312.16886

    2023 arXiv

  10. [19]

    Chu, X.; Qiao, L.; Zhang, X.; Xu, S.; Wei, F.; Yang, Y.; Sun, X.; Hu, Y.; Lin, X.; Zhang, B.; et al. 2024. MobileVLM V2: Faster and Stronger Baseline for Vision Language Model. arXiv preprint arXiv:2402.03766

    2024 arXiv

  11. [20]

    Cui, C.; Sun, T.; Lin, M.; Gao, T.; Zhang, Y.; Liu, J.; Wang, X.; Zhang, Z.; Zhou, C.; Liu, H.; Zhang, Y.; Lv, W.; Huang, K.; Zhang, Y.; Zhang, J.; Zhang, J.; Liu, Y.; Yu, D.; and Ma, Y. 2025. PaddleOCR 3.0 Technical Report. arXiv:2507.05595

    2025 arXiv

  12. [21]

    Dai, W.; Li, J.; Li, D.; Tiong, A. M. H.; Zhao, J.; Wang, W.; Li, B.; Fung, P.; and Hoi, S. 2023. InstructBLIP: Towards General-purpose Vision-Language Models with Instruction Tuning

    2023

  13. [22]

    Ding, Y.; Niu, C.; Wu, F.; Tang, S.; Lyu, C.; and Chen, G. 2024. Enhancing On-Device LLM Inference with Historical Cloud-Based LLM Interactions. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, 597--608

    2024

  14. [23]

    Dosovitskiy, A.; Beyer, L.; Kolesnikov, A.; Weissenborn, D.; Zhai, X.; Unterthiner, T.; Dehghani, M.; Minderer, M.; Heigold, G.; Gelly, S.; et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929

    2020 arXiv

  15. [24]

    M.; Jain, A.; Schmidt, L.; Toshev, A.; and Shankar, V

    Fang, A.; Jose, A. M.; Jain, A.; Schmidt, L.; Toshev, A.; and Shankar, V. 2023. Data Filtering Networks. arXiv preprint arXiv:2309.17425

    2023 arXiv

  16. [25]

    Gu, S.; Zhang, J.; Zhou, S.; Yu, K.; Xing, Z.; Wang, L.; Cao, Z.; Jia, J.; Zhang, Z.; Wang, Y.; et al. 2024. Infinity-mm: Scaling multimodal performance with large-scale and high-quality instruction data. arXiv preprint arXiv:2410.18558

    2024 arXiv

  17. [26]

    Hu, S.; Tu, Y.; Han, X.; He, C.; Cui, G.; Long, X.; Zheng, Z.; Fang, Y.; Huang, Y.; Zhao, W.; et al. 2024 a . MiniCPM : Unveiling the Potential of Small Language Models with Scalable Training Strategies. arXiv preprint arXiv:2404.06395

    2024 arXiv

  18. [27]

    H.; Kamath, A.; Peng, N.; and Chang, K.-W

    Hu, W.; Dou, Z.-Y.; Li, L. H.; Kamath, A.; Peng, N.; and Chang, K.-W. 2024 b . Matryoshka Query Transformer for Large Vision-Language Models

    2024

  19. [28]

    Hua, W.; Wan, M.; Vadrevu, S.; Nadel, R.; Zhang, Y.; and Wang, C. 2024. Interactive Speculative Planning: Enhance Agent Efficiency through Co-design of System and User Interface. arXiv:2410.00079

    2024 arXiv

  20. [29]

    Huang, M.; Liu, Y.; Liang, D.; Jin, L.; and Bai, X. 2024. Mini-monkey: Multi-scale adaptive cropping for multimodal large language models. arXiv preprint arXiv:2408.02034

    2024 arXiv

  21. [30]

    Karamcheti, S.; Nair, S.; Balakrishna, A.; Liang, P.; Kollar, T.; and Sadigh, D. 2024. Prismatic VLMs: Investigating the Design Space of Visually-Conditioned Language Models. In International Conference on Machine Learning (ICML)

    2024

  22. [31]

    Kettunen, K. 2014. Can type-token ratio be used to show morphological complexity of languages? Journal of Quantitative Linguistics, 21(3): 223--245

    2014

  23. [32]

    Li, B.; Zhang, Y.; Guo, D.; Zhang, R.; Li, F.; Zhang, H.; Zhang, K.; Li, Y.; Liu, Z.; and Li, C. 2024 a . Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326

    2024 arXiv

  24. [33]

    Li, J.; Li, D.; Savarese, S.; and Hoi, S. 2023. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In International conference on machine learning, 19730--19742. PMLR

    2023

  25. [34]

    Li, L.; Qian, S.; Lu, J.; Yuan, L.; Wang, R.; and Xie, Q. 2024 b . Transformer-lite: High-efficiency deployment of large language models on mobile phone gpus. arXiv preprint arXiv:2403.20041

    2024 arXiv

  26. [35]

    Lin, J.; Yin, H.; Ping, W.; Lu, Y.; Molchanov, P.; Tao, A.; Mao, H.; Kautz, J.; Shoeybi, M.; and Han, S. 2023. VILA: On Pre-training for Visual Language Models. arXiv:2312.07533

    2023 arXiv

  27. [36]

    Liu, H.; Li, C.; Li, Y.; and Lee, Y. J. 2023 a . Improved Baselines with Visual Instruction Tuning

    2023

  28. [37]

    Liu, H.; Li, C.; Li, Y.; Li, B.; Zhang, Y.; Shen, S.; and Lee, Y. J. 2024 a . LLaVA-NeXT : Improved reasoning, OCR , and world knowledge

    2024

  29. [38]

    Liu, H.; Li, C.; Wu, Q.; and Lee, Y. J. 2024 b . Visual Instruction Tuning. NeurIPS, 36

    2024

  30. [39]

    Liu, S.; Zeng, Z.; Ren, T.; Li, F.; Zhang, H.; Yang, J.; Li, C.; Yang, J.; Su, H.; Zhu, J.; et al. 2023 b . Grounding dino: Marrying dino with grounded pre-training for open-set object detection. arXiv preprint arXiv:2303.05499

    2023 arXiv

  31. [40]

    Liu, Z.; Mao, H.; Wu, C.-Y.; Feichtenhofer, C.; Darrell, T.; and Xie, S. 2022. A ConvNet for the 2020s. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)

    2022

  32. [41]

    Liu, Z.; Zhu, L.; Shi, B.; Zhang, Z.; Lou, Y.; Yang, S.; Xi, H.; Cao, S.; Gu, Y.; Li, D.; et al. 2024 c . NVILA: Efficient frontier visual language models. arXiv preprint arXiv:2412.04468

    2024 arXiv

  33. [42]

    Loshchilov, I.; and Hutter, F. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101

    2017 arXiv

  34. [43]

    Lu, X.; Chen, Y.; Chen, C.; Tan, H.; Chen, B.; Xie, Y.; Hu, R.; Tan, G.; Wu, R.; Hu, Y.; et al. 2024. Bluelm-v-3b: Algorithm and system co-design for multimodal large language models on mobile devices. arXiv preprint arXiv:2411.10640

    2024 arXiv

  35. [44]

    Luo, G.; Yang, X.; Dou, W.; Wang, Z.; Dai, J.; Qiao, Y.; and Zhu, X. 2024. Mono-InternVL: Pushing the Boundaries of Monolithic Multimodal Large Language Models with Endogenous Visual Pre-training. arXiv preprint arXiv:2410.08202

    2024 arXiv

  36. [45]

    B.; Lozhkov, A.; Tazi, N.; et al

    Marafioti, A.; Zohar, O.; Farr \'e , M.; Noyan, M.; Bakouch, E.; Cuenca, P.; Zakka, C.; Allal, L. B.; Lozhkov, A.; Tazi, N.; et al. 2025. SmolVLM: Redefining small and efficient multimodal models. arXiv preprint arXiv:2504.05299

    2025 arXiv

  37. [46]

    McKinzie, B.; Gan, Z.; Fauconnier, J.-P.; Dodge, S.; Zhang, B.; Dufter, P.; Shah, D.; Du, X.; Peng, F.; Weers, F.; et al. 2024. MM1 : Methods, analysis & insights from multimodal LLM pre-training. arXiv preprint arXiv:2403.09611

    2024 arXiv

  38. [47]

    H.; Cao, Q.; Horton, M.; Jin, Y.; Sun, C.; Mirzadeh, S

    Mehta, S.; Sekhavat, M. H.; Cao, Q.; Horton, M.; Jin, Y.; Sun, C.; Mirzadeh, S. I.; Najibi, M.; Belenko, D.; Zatloukal, P.; et al. 2024. Openelm: An efficient language model family with open training and inference framework. In Workshop on Efficient Systems for Foundation Mode...

    2024

  39. [48]

    OpenAI. 2023. GPT-4 Technical Report. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, volume abs/2303.08774

    2023 arXiv

  40. [49]

    Qu, G.; Chen, Q.; Wei, W.; Lin, Z.; Chen, X.; and Huang, K. 2024. Mobile edge intelligence for large language models: A contemporary survey. arXiv preprint arXiv:2407.18921

    2024 arXiv

  41. [50]

    W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al

    Radford, A.; Kim, J. W.; Hallacy, C.; Ramesh, A.; Goh, G.; Agarwal, S.; Sastry, G.; Askell, A.; Mishkin, P.; Clark, J.; et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning, 8748--8763. PMLR

    2021

  42. [51]

    J.; and Yan, Y

    Shang, Y.; Cai, M.; Xu, B.; Lee, Y. J.; and Yan, Y. 2024. LLaVA-PruMerge: Adaptive Token Reduction for Efficient Large Multimodal Models. arXiv preprint arXiv:2403.15388

    2024

  43. [52]

    Shi, M.; Liu, F.; Wang, S.; Liao, S.; Radhakrishnan, S.; Huang, D.-A.; Yin, H.; Sapra, K.; Yacoob, Y.; Shi, H.; Catanzaro, B.; Tao, A.; Kautz, J.; Yu, Z.; and Liu, G. 2024. Eagle: Exploring The Design Space for Multimodal LLMs with Mixture of Encoders. arXiv:2408.15998

    2024 arXiv

  44. [53]

    Sun, Q.; Fang, Y.; Wu, L.; Wang, X.; and Cao, Y. 2023. EVA-CLIP: Improved Training Techniques for CLIP at Scale. arXiv preprint arXiv:2303.15389

    2023 arXiv

  45. [54]

    C.; Yang, J.; Yang, S.; Iyer, A.; Pan, X.; Wang, A.; Fergus, R.; LeCun, Y.; and Xie, S

    Tong, S.; Brown, E.; Wu, P.; Woo, S.; Middepogu, M.; Akula, S. C.; Yang, J.; Yang, S.; Iyer, A.; Pan, X.; Wang, A.; Fergus, R.; LeCun, Y.; and Xie, S. 2024 a . Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs

    2024

  46. [55]

    C.; Yang, J.; Yang, S.; Iyer, A.; Pan, X.; et al

    Tong, S.; Brown, E.; Wu, P.; Woo, S.; Middepogu, M.; Akula, S. C.; Yang, J.; Yang, S.; Iyer, A.; Pan, X.; et al. 2024 b . Cambrian-1: A fully open, vision-centric exploration of multimodal llms. arXiv preprint arXiv:2406.16860

    2024 arXiv

  47. [56]

    Touvron, H.; Lavril, T.; Izacard, G.; Martinet, X.; Lachaux, M.-A.; Lacroix, T.; Rozi \`e re, B.; Goyal, N.; Hambro, E.; Azhar, F.; et al. 2023 a . Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971

    2023 arXiv

  48. [57]

    Touvron, H.; Martin, L.; Stone, K.; Albert, P.; Almahairi, A.; Babaei, Y.; Bashlykov, N.; Batra, S.; Bhargava, P.; Bhosale, S.; et al. 2023 b . Llama 2 : Open foundation and fine-tuned chat models. arXiv preprint arXiv:2307.09288

    2023 arXiv

  49. [58]

    H.; Wu, Y.; Le, Q

    Trinh, T. H.; Wu, Y.; Le, Q. V.; He, H.; and Luong, T. 2024. Solving olympiad geometry without human demonstrations. Nature, 625(7995): 476--482

    2024

  50. [59]

    F.; Alabdulmohsin, I.; Parthasarathy, N.; Evans, T.; Beyer, L.; Xia, Y.; Mustafa, B.; et al

    Tschannen, M.; Gritsenko, A.; Wang, X.; Naeem, M. F.; Alabdulmohsin, I.; Parthasarathy, N.; Evans, T.; Beyer, L.; Xia, Y.; Mustafa, B.; et al. 2025. Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. arXiv pr...

    2025 arXiv

  51. [60]

    Vasu, P. K. A.; Gabriel, J.; Zhu, J.; Tuzel, O.; and Ranjan, A. 2023. FastViT: A Fast Hybrid Vision Transformer using Structural Reparameterization. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV)

    2023

  52. [61]

    Wang, P.; Bai, S.; Tan, S.; Wang, S.; Fan, Z.; Bai, J.; Chen, K.; Liu, X.; Wang, J.; Ge, W.; et al. 2024. Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution. arXiv preprint arXiv:2409.12191

    2024 arXiv

  53. [62]

    Wei, J.; Cao, S.; Cao, T.; Ma, L.; Wang, L.; Zhang, Y.; and Yang, M. 2024. T-mac: Cpu renaissance via table lookup for low-bit llm deployment on edge. arXiv preprint arXiv:2407.00088

    2024 arXiv

  54. [63]

    Wei, J.; Tay, Y.; Bommasani, R.; Raffel, C.; Zoph, B.; Borgeaud, S.; Yogatama, D.; Bosma, M.; Zhou, D.; Metzler, D.; et al. 2022. Emergent abilities of large language models. arXiv preprint arXiv:2206.07682

    2022 arXiv

  55. [64]

    Xue, Z.; Song, Y.; Mi, Z.; Chen, L.; Xia, Y.; and Chen, H. 2024. PowerInfer-2: Fast Large Language Model Inference on a Smartphone. arXiv preprint arXiv:2406.06282

    2024 arXiv

  56. [65]

    Yang, A.; Li, A.; Yang, B.; Zhang, B.; Hui, B.; Zheng, B.; Yu, B.; Gao, C.; Huang, C.; Lv, C.; Zheng, C.; Liu, D.; Zhou, F.; Huang, F.; Hu, F.; Ge, H.; Wei, H.; Lin, H.; Tang, J.; Yang, J.; Tu, J.; Zhang, J.; Yang, J.; Yang, J.; Zhou, J.; Zhou, J.; Lin, J.; Dang, K.; Bao, K.; ...

    2025 arXiv

  57. [66]

    Yang, A.; Yang, B.; Zhang, B.; Hui, B.; Zheng, B.; Yu, B.; Li, C.; Liu, D.; Huang, F.; Wei, H.; Lin, H.; Yang, J.; Tu, J.; Zhang, J.; Yang, J.; Yang, J.; Zhou, J.; Lin, J.; Dang, K.; Lu, K.; Bao, K.; Yang, K.; Yu, L.; Li, M.; Xue, M.; Zhang, P.; Zhu, Q.; Men, R.; Lin, R.; Li, ...

    2024 arXiv

  58. [67]

    Yao, Y.; Yu, T.; Zhang, A.; Wang, C.; Cui, J.; Zhu, H.; Cai, T.; Li, H.; Zhao, W.; He, Z.; et al. 2024. Minicpm-v: A gpt-4v level mllm on your phone. arXiv preprint arXiv:2408.01800

    2024 arXiv

  59. [68]

    Zhai, X.; Mustafa, B.; Kolesnikov, A.; and Beyer, L. 2023. Sigmoid loss for language image pre-training. International Conference on Computer Vision ( ICCV )

    2023

  60. [69]

    Zhang, H.; Gao, M.; Gan, Z.; Dufter, P.; Wenzel, N.; Huang, F.; Shah, D.; Du, X.; Zhang, B.; Li, Y.; Dodge, S.; You, K.; Yang, Z.; Timofeev, A.; Xu, M.; Chen, H.-Y.; Fauconnier, J.-P.; Lai, Z.; You, H.; Wang, Z.; Dehghan, A.; Grasch, P.; and Yang, Y. 2024. MM1.5: Methods, Anal...

    2024 arXiv

  61. [70]

    Zhang, R.; Han, J.; Zhou, A.; Hu, X.; Yan, S.; Lu, P.; Li, H.; Gao, P.; and Qiao, Y. 2023. LLaMA-Adapter: Efficient Fine-tuning of Language Models with Zero-init Attention. arXiv preprint arXiv:2303.16199

    2023 arXiv

  62. [71]

    Zhu, J.; Wang, W.; Chen, Z.; Liu, Z.; Ye, S.; Gu, L.; Duan, Y.; Tian, H.; Su, W.; Shao, J.; et al. 2025. InternVL3: Exploring Advanced Training and Test-Time Recipes for Open-Source Multimodal Models. arXiv preprint arXiv:2504.10479

    2025 arXiv

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.