Pith. sign in

REVIEW 3 major objections 6 minor 1 cited by

IRGPT: Understanding Real-world Infrared Image with Bi-cross-modal Curriculum on Large-scale Benchmark

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read IRGPT builds a vision-language model on real infrared images and claims state-of-the-art scores on nine infrared tasks, outperforming larger models.

desk verdict A genuinely new real-infrared-text dataset and a sensible curriculum idea, but the headline SOTA claim is only demonstrated in-distribution and needs external validation. read the letter →

arxiv 2507.14449 v1 pith:CLJOZZNH submitted 2025-07-19 cs.CV

Zhe Cao , Jin Zhang , Ruiheng Zhang This is my paper
classification cs.CV
keywords infraredvision-languagemodelreal-worldimagesbi-cross-modalcurriculuminfrared-textdatasettransferlearningmultimodallargelanguageimage-textalignmentbenchmark
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper proposes IRGPT, a vision-language model built specifically for real-world infrared images, and claims it is the first such model trained on authentic infrared imagery rather than synthetic images converted from visible ones. To support this, the authors assemble IR-TD, a dataset of more than 260,000 real infrared images paired with text, where captions and Q&A pairs come from two pipelines: LLM-generated descriptions of aligned visible images and rule-based construction from existing annotations. They also introduce a bi-cross-modal curriculum that orders training samples by difficulty along two axes: how far an infrared image sits from the visible-image domain, and how well the infrared image aligns with its text. On a nine-task benchmark covering recognition, grounding, counting, and related skills, IRGPT reports higher scores than several larger models, including InternVL2-26B, in both zero-shot and fine-tuned evaluation. The authors' central argument is that data authenticity plus difficulty-aware transfer, not model size, is what closes the gap between visible and infrared understanding.

What carries the argument

The central mechanism is the bi-cross-modal curriculum transfer learning strategy. It generates two complementary difficulty scores for every infrared-text pair: one based on infrared-to-visible domain distance (computed via intra-domain projection onto the inter-domain direction plus an MMD term), and one based on infrared-to-text semantic alignment (computed with a pre-warmed CLIP model and a dynamic loss-variation rate). The two scores are integrated into a difficulty ranking, the training data is split into ascending-difficulty tiers, and pre-training draws samples randomly within each tier; the loss-variation rate also acts as an adaptive per-sample weight. The curriculum is applied during incremental pre-training of the vision encoder and projector, before a LoRA-based instruction fine-tuning stage.

What would settle it

Have human experts independently annotate a held-out set of real infrared images, with captions, recognition, grounding, and counting labels, using no LLM-generated descriptions or rule-based templates drawn from IR-TD, then compare IRGPT and InternVL2-26B on that set; the central claim holds only if IRGPT's positive-sum advantage persists on such out-of-pipeline data.

Watch

Extended reading notes

Core claim

IRGPT's central claim is that real-world infrared understanding can be transferred from a pre-trained visible-domain vision-language model by sequencing training samples according to a two-component difficulty score. Lesson 1 assigns each infrared image a geometric domain-projection distance: the sample's vector is projected onto the inter-domain direction between infrared and visible centers in a Reproducing Kernel Hilbert Space, with an MMD term added, so samples closer to the visible domain are treated as easier. Lesson 2 measures infrared-text alignment with a pre-warmed CLIP model and augments the static loss with a dynamic loss-variation rate that down-weights misaligned pairs and up-weights hard ones. The model, initialized from InternVL2-8B, is pre-trained on difficulty tiers in ascending order with stratified random sampling and adaptive sample weights, then instruction-tuned. The paper reports state-of-the-art results on all nine benchmark tasks, with a fine-tuned positive-score total of 485.79 versus 473.78 for InternVL2-26B and a negative-score total of 4.39 versus 19.57, and shows that reversing the curriculum order sharply degrades performance.

Load-bearing premise

The benchmark that establishes the state-of-the-art claim is built from the same IR-TD image pool and the same LLM-generated and rule-based annotation pipeline used for training, so the test samples are in-distribution; the margins over models that have never seen infrared images may reflect adaptation to this pipeline rather than general infrared understanding.

Editorial extensions

If this is right

  • Real infrared-text pairs, rather than synthetic style-transferred pairs, can serve as a sufficient training signal for visible-to-infrared transfer in vision-language models.
  • The two-score curriculum, domain distance plus alignment difficulty, provides a sample ordering that outperforms random and anti-curriculum schedules on infrared tasks.
  • The model's largest gains appear in identity and reasoning-heavy tasks such as ReID, where raw perception alone is insufficient.
  • The released IR-TD dataset and nine-task benchmark give the community a common reference for comparing infrared vision-language models on authentic data.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Because the benchmark is carved from the same IR-TD pool and the same annotation pipeline that trained IRGPT, the reported 12-point positive-sum advantage over InternVL2-26B partly reflects in-distribution adaptation; a fair test would use independently annotated infrared data to see how much generalizes.
  • The same bi-cross-modal curriculum idea could be applied to other under-resourced sensing modalities, such as night-vision, radar, or multispectral imaging, where paired visible imagery and sparse text exist but large aligned corpora do not.
  • A testable extension is to use the loss-variation-rate weighting alone, without the tiered ordering, to isolate whether the gains come from curriculum sequencing or from sample reweighting.
  • The paper's approximation of cross-modal distance via intra-domain projection is explicitly imprecise; a sensitivity analysis on the distance metric would clarify how robust the difficulty ranking is.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces IRGPT, a multimodal large language model for real-world infrared images, together with a new large-scale infrared-text dataset (IR-TD) containing over 260K authentic image-text pairs. The dataset is constructed through LLM-based descriptions of paired visible images and rule-based annotations. The authors also propose a bi-cross-modal curriculum transfer learning strategy that orders pre-training samples by infrared-visible domain distance and infrared-text alignment difficulty. On a self-built 9-task benchmark, IRGPT reports state-of-the-art results in both zero-shot and fine-tuned settings, outperforming larger baselines such as InternVL2-26B.

Significance. If the claims are substantiated, the paper would provide a valuable open-source dataset and a practical training recipe for infrared vision-language understanding, a genuinely underexplored area. The bi-cross-modal curriculum idea is interesting, and the paper includes thorough ablations on curriculum components and sampling schedules. However, the evaluation is currently undermined by two load-bearing issues: the benchmark is generated from the same dataset and annotation pipeline as the training data, and the most relevant prior infrared-specific model, Infrared-LLaVA, is not compared. The reported 'state-of-the-art' performance therefore remains an in-distribution claim until those issues are addressed.

major comments (3)
  1. [Section 3.4 / Section 5.1, Table 2] The benchmark is built from the same IR-TD dataset and uses the same LLM-based and rule-based generation processes as the training data. Because the test questions and answers therefore share templates, vocabulary, and annotation statistics with the training set, the large gains in Table 2 (e.g., psum 485.79 vs 473.78 for InternVL2-26B) may be driven by adaptation to the generation pipeline rather than by generalizable infrared understanding. The paper should either add an external held-out benchmark (e.g., reusing existing infrared datasets with manually curated questions) or provide quantitative evidence that performance holds when the test distribution is varied (e.g., different question formats, unseen datasets, or human-written annotations).
  2. [Section 5.1, Table 2] The most relevant prior work, Infrared-LLaVA [14], is discussed in Section 2 but never experimentally compared. Since Infrared-LLaVA is also an infrared-specific vision-language model (though trained on synthetic infrared data), its omission makes it impossible to judge whether IRGPT actually surpasses the prior state of the art in this niche. The authors should include Infrared-LLaVA (and, if feasible, other infrared-specific MLLMs) in Table 2 under both zero-shot and fine-tuned settings.
  3. [Section 5.1, Table 2 footnote] The 'zero-shot' comparison is asymmetric: IRGPT is evaluated after incremental pre-training on 190k IR-TD pairs, while the baselines (InternVL2-8B, Qwen2-VL, etc.) have received no infrared training. Thus the reported zero-shot improvements (e.g., +76.35 psum over InternVL2-8B) conflate domain adaptation with zero-shot generalization. The authors should either report true zero-shot results (no IR pre-training) or pre-train the baselines on the same IR-TD pre-training subset with identical steps and then compare, so that the effect of the curriculum can be isolated.
minor comments (6)
  1. [Section 3.4 (title)] The subsection title 'Benchmark Defination' contains a typo; it should read 'Benchmark Definition'.
  2. [Table 2] The compared method listed as 'Pandagpt-7B' should be capitalized consistently as 'PandaGPT-7B'.
  3. [Section 5.2] The reported curriculum improvement of 14.47 in psum and the 'marginal decrease of merely 2.12' in nsum do not match the numbers in Table 2 (zero-shot CL vs Random: psum +21.47, nsum -3.62). Please reconcile the text with the table.
  4. [Section 5.1 / Section 4.2.3] The number of curriculum tiers M is not reported anywhere. Please provide the value used in the experiments and, ideally, a sensitivity analysis over M.
  5. [Section 4.2.1, Eq. (3)] The sign convention for the projection term in Eq. (3) is unclear; if d_i is intended as a distance it should be non-negative, but as written the projected term can be negative. Please clarify or use an absolute value.
  6. [Section 5.4] The claim that CLIP losses are 'normally distributed ... from the Central Limit Theorem' is not a valid justification, since the theorem applies to sums of independent random variables, not to model losses on a fixed dataset. Please rephrase this explanation.

Circularity Check

0 steps flagged · score 0.0 of 10

No circular derivation: IRGPT's claims rest on empirical evaluation and independently computed curriculum signals; the in-IR-TD benchmark raises generalization validity, not circularity.

full rationale

The paper's central contributions—a large infrared-text dataset, a bi-cross-modal curriculum, and the IRGPT model—are not derived from their evaluation targets by construction. The two difficulty signals used for curriculum ordering are computed externally: Lesson 1 uses MMD-based geometric projection on features from an existing retrained encoder, and Lesson 2 uses a pre-warmed CLIP's loss-variation rate. Neither signal is the model's own training loss, and the reported psum/nsum scores are measured on a held-out split rather than solved for. The only self-citation with overlapping authorship, reference [47], is used as general support for the benefit of stochasticity in curriculum sampling and is accompanied by external support [50]; the paper also directly ablatates the curriculum schedule in Tables 3 and 4, so the self-citation is not load-bearing. A genuine caveat is that the benchmark is constructed from the same IR-TD pipeline used to build the training data, making the evaluation in-distribution and limiting claims about general real-world infrared understanding. That is an external-validity concern, however, not a circularity: no benchmark number reduces by construction to a fitted parameter or to a training objective, and the train/test splits are disjoint. The paper's derivation chain is therefore not circular.

Assumptions & free parameters 1 free parameters · 4 assumptions · 0 invented entities

The central claims rest on assumptions about the validity of LLM-derived text as infrared annotations, the usefulness of heuristic difficulty scores, and the representativeness of the self-constructed benchmark. No new physical entities are introduced, and the only explicit free parameter is the number of curriculum tiers, which is not reported.

free parameters (1)
  • number of curriculum tiers M = not reported
    Sec. 4.2.3 partitions training data into M tiers; M is a hand-chosen hyperparameter that affects the curriculum schedule but is not specified.
assumptions (4)
  • domain assumption LLM-generated descriptions of visible images are valid semantic annotations for corresponding infrared images.
    Section 3.2: visible images processed by LLMs to generate descriptive texts used as annotations for infrared images; this assumes semantic equivalence across modalities.
  • domain assumption The sample-level distance di computed via intra-domain projection and MMD provides a meaningful difficulty ranking.
    Section 4.2.1, Eq. (3); authors state the distances are not precise but acceptable for ranking.
  • domain assumption The CLIP loss variation rate alpha reflects sample alignment difficulty.
    Section 4.2.2, Eq. (4); assumes that a larger positive variation rate indicates a hard but learnable sample.
  • domain assumption The nine benchmark tasks measure infrared image understanding.
    Section 3.4; tasks are defined by the authors with no external validation of their difficulty or representativeness.

how reviews work

0 comments
Cite this review

Pith. "Pith review of IRGPT: Understanding Real-world Infrared Image with Bi-cross-modal Curriculum on Large-scale Benchmark." pith.science (2026). https://pith.science/paper/CLJOZZNH

@misc{pith2026250714449,
  author       = {Pith},
  title        = {Pith review of: IRGPT: Understanding Real-world Infrared Image with Bi-cross-modal Curriculum on Large-scale Benchmark},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CLJOZZNH}},
  note         = {Machine review of arXiv:2507.14449}
}
read the original abstract

Real-world infrared imagery presents unique challenges for vision-language models due to the scarcity of aligned text data and domain-specific characteristics. Although existing methods have advanced the field, their reliance on synthetic infrared images generated through style transfer from visible images, which limits their ability to capture the unique characteristics of the infrared modality. To address this, we propose IRGPT, the first multi-modal large language model for real-world infrared images, built upon a large-scale InfraRed-Text Dataset (IR-TD) comprising over 260K authentic image-text pairs. The proposed IR-TD dataset contains real infrared images paired with meticulously handcrafted texts, where the initial drafts originated from two complementary processes: (1) LLM-generated descriptions of visible images, and (2) rule-based descriptions of annotations. Furthermore, we introduce a bi-cross-modal curriculum transfer learning strategy that systematically transfers knowledge from visible to infrared domains by considering the difficulty scores of both infrared-visible and infrared-text. Evaluated on a benchmark of 9 tasks (e.g., recognition, grounding), IRGPT achieves state-of-the-art performance even compared with larger-scale models.

Figures

Figures reproduced from arXiv: 2507.14449 by the authors.

Figure 1
Figure 1. Exist MLLMs exhibit hallucinations when interpreting [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. IRGPT effectively handles complex infrared image-text tasks (e.g., grounding, recognition), demonstrating successful knowledge [PITH_FULL_IMAGE:figures/full_fig_p002_2.png] view at source ↗
Figure 3
Figure 3. The translation of tri-modal data. (a) Tri-modal data [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figures from the paper (4 more)
Figure 4
Figure 4. Figure 4: Training strategy of IRGPT. (a) Incremental pre-training [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]
Figure 5
Figure 5. Figure 5: Bi-cross-modal curriculum transfer strategy. [PITH_FULL_IMAGE:figures/full_fig_p005_5.png]
Figure 6
Figure 6. Figure 6: The distribution of difficulty score. The CLIP loss ex [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Sample pairs with ranking results. Demonstrating varia [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score. Full citation record

  1. Revealing Physical-World Semantic Vulnerabilities: Universal Adversarial Patch for Infrared Vision-Language Models

    cs.CV 2026-04 unverdicted novelty 7.0 of 10

    UCGP is a universal physical adversarial patch that compromises cross-modal semantic alignment in IR-VLMs through curved-grid parameterization and representation-space disruption.

Reference graph

Works this paper leans on

55 extracted references · 43 canonical work pages · cited by 1 Pith paper

  1. [14]

    Infrared-llava: Enhancing understanding of infrared images in multi-modal large language models

    Shixin Jiang, Zerui Chen, Jiafeng Liang, Yanyan Zhao, Ming Liu, and Bing Qin. Infrared-llava: Enhancing understanding of infrared images in multi-modal large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 8573–8591, 2024. 1, 3, 4

    work page 2024

  2. [1]

    Flamingo: a visual language model for few-shot learning

    Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Men- sch, Katherine Millican, Malcolm Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in neural information processing systems, 35:23716–23736,

  3. [2]

    Curriculum learning

    Yoshua Bengio, J ´erˆome Louradour, Ronan Collobert, and Ja- son Weston. Curriculum learning. In Proceedings of the 26th Annual International Conference on Machine Learn- ing, page 41–48, New York, NY , USA, 2009. Association for Computing Machinery. 3

    work page 2009

  4. [3]

    In- formation maximizing curriculum: A curriculum-based ap- proach for learning versatile skills

    Denis Blessing, Onur Celik, Xiaogang Jia, Moritz Reuss, Maximilian Li, Rudolf Lioutikov, and Gerhard Neumann. In- formation maximizing curriculum: A curriculum-based ap- proach for learning versatile skills. In Advances in Neural Information Processing Systems, pages 51536–51561. Cur- ran Associates, Inc., 2023. 3

    work page 2023

  5. [4]

    The (r) evolution of multi- modal large language models: A survey

    Davide Caffagni, Federico Cocchi, Luca Barsellotti, Nicholas Moratelli, Sara Sarto, Lorenzo Baraldi, Marcella Cornia, and Rita Cucchiara. The (r) evolution of multi- modal large language models: A survey. arXiv preprint arXiv:2402.12451, 2024. 3

    arXiv 2024

  6. [5]

    Retinexformer: One-stage retinex- based transformer for low-light image enhancement

    Yuanhao Cai, Hao Bian, Jing Lin, Haoqian Wang, Radu Tim- ofte, and Yulun Zhang. Retinexformer: One-stage retinex- based transformer for low-light image enhancement. In Pro- ceedings of the IEEE/CVF international conference on com- puter vision, pages 12504–12513, 2023. 3

    work page 2023

  7. [6]

    Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks

    Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation mod- els and aligning for generic visual-linguistic tasks. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 24185–24198, 2024. 3, 7

    work page 2024

  8. [7]

    Cross-lingual lan- guage model pretraining

    Alexis Conneau and Guillaume Lample. Cross-lingual lan- guage model pretraining. Advances in neural information processing systems, 32, 2019. 6

    work page 2019

Show all 55 references
  1. [8]

    An image is worth 16x16 words: Trans- formers for image recognition at scale

    Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Syl- vain Gelly, et al. An image is worth 16x16 words: Trans- formers for image recognition at scale. arXiv preprint a...

    2010 arXiv

  2. [9]

    Cross-domain gradient discrepancy minimization for unsupervised domain adaptation

    Zhekai Du, Jingjing Li, Hongzu Su, Lei Zhu, and Ke Lu. Cross-domain gradient discrepancy minimization for unsupervised domain adaptation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3937–3946, 2021. 5

    2021

  3. [10]

    Unsupervised domain adaptation via deep conditional adap- tation network

    Pengfei Ge, Chuan-Xian Ren, Xiao-Lin Xu, and Hong Yan. Unsupervised domain adaptation via deep conditional adap- tation network. Pattern Recognition, 134:109088, 2023. 5

    2023

  4. [11]

    Imagebind: One embedding space to bind them all

    Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind: One embedding space to bind them all. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15180–15190, 2023. 1, 3

    2023

  5. [12]

    Learning with noisy correspondence for cross-modal matching.Advances in Neu- ral Information Processing Systems, 34:29406–29419, 2021

    Zhenyu Huang, Guocheng Niu, Xiao Liu, Wenbiao Ding, Xinyan Xiao, Hua Wu, and Xi Peng. Learning with noisy correspondence for cross-modal matching.Advances in Neu- ral Information Processing Systems, 34:29406–29419, 2021. 6

    2021

  6. [13]

    Quilt-1m: One million image-text pairs for histopathology

    Wisdom Ikezogwo, Saygin Seyfioglu, Fatemeh Ghezloo, Dylan Geva, Fatwir Sheikh Mohammed, Pavan Kumar Anand, Ranjay Krishna, and Linda Shapiro. Quilt-1m: One million image-text pairs for histopathology. Advances in neural information processing systems, 36:37995–38017,

  7. [15]

    C-sfda: A curriculum learning aided self- training framework for efficient source free domain adapta- tion

    Nazmul Karim, Niluthpol Chowdhury Mithun, Abhinav Ra- jvanshi, Han-pang Chiu, Supun Samarasekera, and Nazanin Rahnavard. C-sfda: A curriculum learning aided self- training framework for efficient source free domain adapta- tion. In Proceedings of the IEEE/CVF conference on com...

  8. [16]

    Segment any- thing

    Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer White- head, Alexander C Berg, Wan-Yen Lo, et al. Segment any- thing. In Proceedings of the IEEE/CVF international confer- ence on computer vision, pages 4015–4026, 2023. 1, 3

    2023

  9. [17]

    What matters when building vision-language mod- els? Advances in Neural Information Processing Systems , 37:87874–87907, 2025

    Hugo Laurenc ¸on, L´eo Tronchon, Matthieu Cord, and Victor Sanh. What matters when building vision-language mod- els? Advances in Neural Information Processing Systems , 37:87874–87907, 2025. 3

    2025

  10. [18]

    Edge-guided multi-domain rgb-to-tir image translation for training vision tasks with challenging labels

    Dong–Guw Lee, Myung–Hwan Jeon, Younggun Cho, and Ayoung Kim. Edge-guided multi-domain rgb-to-tir image translation for training vision tasks with challenging labels. In 2023 IEEE International Conference on Robotics and Au- tomation (ICRA), pages 8291–8298, 2023. 3

    2023

  11. [19]

    Seed-bench: Bench- marking multimodal large language models

    Bohao Li, Yuying Ge, Yixiao Ge, Guangzhi Wang, Rui Wang, Ruimao Zhang, and Ying Shan. Seed-bench: Bench- marking multimodal large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, pages 13299–13308, 2024. 1

    2024

  12. [20]

    The stability- efficiency dilemma: Investigating sequence length warmup for training gpt models

    Conglong Li, Minjia Zhang, and Yuxiong He. The stability- efficiency dilemma: Investigating sequence length warmup for training gpt models. Advances in Neural Information Processing Systems, 35:26736–26750, 2022. 3

    2022

  13. [21]

    Domain generalization with adversarial feature learning

    Haoliang Li, Sinno Jialin Pan, Shiqi Wang, and Alex C Kot. Domain generalization with adversarial feature learning. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 5400–5409, 2018. 5

    2018

  14. [22]

    Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models

    Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models. In In- ternational conference on machine learning , pages 19730– 19742. PMLR, 2023. 3

    2023

  15. [23]

    Infmae: A foundation model in the infrared modality

    Fangcen Liu, Chenqiang Gao, Yaming Zhang, Junjie Guo, Jinghao Wang, and Deyu Meng. Infmae: A foundation model in the infrared modality. In European Conference on Computer Vision, pages 420–437. Springer, 2025. 3, 5

    2025

  16. [24]

    Visual instruction tuning

    Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. InAdvances in Neural Information Processing Systems, pages 34892–34916. Curran Associates, Inc., 2023. 1, 3, 4, 7

    2023

  17. [25]

    A survey on hallucination in large vision-language models

    Hanchao Liu, Wenyuan Xue, Yifei Chen, Dapeng Chen, Xiu- tian Zhao, Ke Wang, Liping Hou, Rongjun Li, and Wei Peng. A survey on hallucination in large vision-language models. arXiv preprint arXiv:2402.00253, 2024. 3

    2024 arXiv

  18. [26]

    Lsotb-tir: A large-scale high-diversity thermal infrared object tracking benchmark

    Qiao Liu, Xin Li, Zhenyu He, Chenglong Li, Jun Li, Zikun Zhou, Di Yuan, Jing Li, Kai Yang, Nana Fan, et al. Lsotb-tir: A large-scale high-diversity thermal infrared object tracking benchmark. In Proceedings of the 28th ACM international conference on multimedia, pages 3847–385...

    2020

  19. [27]

    Let’s learn step by step: Enhancing in-context learning ability with curriculum learning

    Yinpeng Liu, Jiawei Liu, Xiang Shi, Qikai Cheng, Yong Huang, and Wei Lu. Let’s learn step by step: Enhancing in-context learning ability with curriculum learning. arXiv preprint arXiv:2402.10738, 2024. 3

    2024 arXiv

  20. [28]

    Decoupled weight de- cay regularization

    Ilya Loshchilov and Frank Hutter. Decoupled weight de- cay regularization. In International Conference on Learning Representations, 2019. 7

    2019

  21. [29]

    What is being transferred in transfer learning? In Advances in Neural Information Processing Systems , pages 512–523

    Behnam Neyshabur, Hanie Sedghi, and Chiyuan Zhang. What is being transferred in transfer learning? In Advances in Neural Information Processing Systems , pages 512–523. Curran Associates, Inc., 2020. 5

    2020

  22. [30]

    Learning transferable visual models from natural language supervi- sion

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervi- sion. In International conference on machine learning, ...

    2021

  23. [31]

    Laion-5b: An open large-scale dataset for training next generation image-text models

    Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Worts- man, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. Advances in neural in- f...

    2022

  24. [32]

    Curriculum learning: A survey

    Petru Soviany, Radu Tudor Ionescu, Paolo Rota, and Nicu Sebe. Curriculum learning: A survey. International Journal of Computer Vision, 130(6):1526–1565, 2022. 3

    2022

  25. [33]

    less is more

    Valentin I Spitkovsky, Hiyan Alshawi, and Dan Jurafsky. From baby steps to leapfrog: How “less is more” in unsuper- vised dependency parsing. In Human Language Technolo- gies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguis...

    2010

  26. [34]

    Pandagpt: One model to instruction-follow them all

    Yixuan Su, Tian Lan, Huayang Li, Jialu Xu, Yan Wang, and Deng Cai. Pandagpt: One model to instruction-follow them all. arXiv preprint arXiv:2305.16355, 2023. 3, 7

    2023 arXiv

  27. [35]

    Alignment-free rgb-t salient object detection: A large-scale dataset and progressive correlation network

    Kunpeng Wang, Keke Chen, Chenglong Li, Zhengzheng Tu, and Bin Luo. Alignment-free rgb-t salient object detection: A large-scale dataset and progressive correlation network. arXiv preprint arXiv:2412.14576, 2024. 4

    2024 arXiv

  28. [36]

    Alignment-free rgbt salient object de- tection: Semantics-guided asymmetric correlation network and a unified benchmark

    Kunpeng Wang, Danying Lin, Chenglong Li, Zhengzheng Tu, and Bin Luo. Alignment-free rgbt salient object de- tection: Semantics-guided asymmetric correlation network and a unified benchmark. IEEE Transactions on Multime- dia, 2024. 4

    2024

  29. [37]

    Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution

    Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-vl: Enhancing vision-language model’s perception of the world at any resolution. arXiv preprint arXiv:2409.12191, 2024. 3, 7

    2024 arXiv

  30. [38]

    Learning a multi-domain cur- riculum for neural machine translation

    Wei Wang, Ye Tian, Jiquan Ngiam, Yinfei Yang, Isaac Caswell, and Zarana Parekh. Learning a multi-domain cur- riculum for neural machine translation. InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7711–7723, 2020. 3

    2020

  31. [39]

    Rethinking maxi- mum mean discrepancy for visual domain adaptation

    Wei Wang, Haojie Li, Zhengming Ding, Feiping Nie, Jun- yang Chen, Xiao Dong, and Zhihui Wang. Rethinking maxi- mum mean discrepancy for visual domain adaptation. IEEE Transactions on Neural Networks and Learning Systems, 34 (1):264–277, 2023. 5

    2023

  32. [40]

    A survey on curriculum learning

    Xin Wang, Yudong Chen, and Wenwu Zhu. A survey on curriculum learning. IEEE transactions on pattern analysis and machine intelligence, 44(9):4555–4576, 2021. 3

    2021

  33. [41]

    Efficienttrain: Exploring gener- alized curriculum learning for training visual backbones

    Yulin Wang, Yang Yue, Rui Lu, Tianjiao Liu, Zhao Zhong, Shiji Song, and Gao Huang. Efficienttrain: Exploring gener- alized curriculum learning for training visual backbones. In Proceedings of the IEEE/CVF International Conference on Computer Vision, pages 5852–5864, 2023. 3

    2023

  34. [42]

    Deepseek-vl2: Mixture-of- experts vision-language models for advanced multimodal understanding

    Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, et al. Deepseek-vl2: Mixture-of- experts vision-language models for advanced multimodal understanding. arXiv preprint arXiv:2412.10302, 2024. 3

    2024 arXiv

  35. [43]

    Training large language models for reasoning through reverse curriculum reinforcement learning

    Zhiheng Xi, Wenxiang Chen, Boyang Hong, Senjie Jin, Rui Zheng, Wei He, Yiwen Ding, Shichun Liu, Xin Guo, Junzhe Wang, et al. Training large language models for reasoning through reverse curriculum reinforcement learning. In In- ternational Conference on Machine Learning, pages...

    2024

  36. [44]

    Curriculum learning for natural language understanding

    Benfeng Xu, Licheng Zhang, Zhendong Mao, Quan Wang, Hongtao Xie, and Yongdong Zhang. Curriculum learning for natural language understanding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 6095–6104, Online, 2020. Association f...

    2020

  37. [45]

    An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, et al. Qwen2. 5 technical report. arXiv preprint arXiv:2412.15115, 2024. 1

    2024 arXiv

  38. [46]

    Bicro: Noisy correspon- dence rectification for multi-modality data via bi-directional cross-modal similarity consistency

    Shuo Yang, Zhaopan Xu, Kai Wang, Yang You, Hongxun Yao, Tongliang Liu, and Min Xu. Bicro: Noisy correspon- dence rectification for multi-modality data via bi-directional cross-modal similarity consistency. In Proceedings of the IEEE/CVF Conference on Computer Vision and Patter...

    2023

  39. [47]

    Mind the boundary: Coreset selection via reconstructing the decision boundary

    Shuo Yang, Zhe Cao, Sheng Guo, Ruiheng Zhang, Ping Luo, Shengping Zhang, and Liqiang Nie. Mind the boundary: Coreset selection via reconstructing the decision boundary. In Forty-first International Conference on Machine Learn- ing, 2024. 6

    2024

  40. [48]

    Vision-language models for vision tasks: A survey

    Jingyi Zhang, Jiaxing Huang, Sheng Jin, and Shijian Lu. Vision-language models for vision tasks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence,

  41. [49]

    Irsam: Advancing segment anything model for infrared small target detection

    Mingjin Zhang, Yuchun Wang, Jie Guo, Yunsong Li, Xinbo Gao, and Jing Zhang. Irsam: Advancing segment anything model for infrared small target detection. In European Con- ference on Computer Vision, pages 233–249. Springer, 2025. 3

    2025

  42. [50]

    Coverage-centric coreset selection for high pruning rates

    Haizhong Zheng, Rui Liu, Fan Lai, and Atul Prakash. Coverage-centric coreset selection for high pruning rates. In 11th International Conference on Learning Representations, ICLR 2023, 2023. 6

    2023

  43. [51]

    Intra-and inter-modal curriculum for multi- modal learning

    Yuwei Zhou, Xin Wang, Hong Chen, Xuguang Duan, and Wenwu Zhu. Intra-and inter-modal curriculum for multi- modal learning. In Proceedings of the 31st ACM Interna- tional Conference on Multimedia , pages 3724–3735, 2023. 3

    2023

  44. [52]

    Languagebind: Extending video-language pretraining to n-modality by language-based semantic align- ment

    Bin Zhu, Bin Lin, Munan Ning, Yang Yan, Jiaxi Cui, W ANG HongFa, Yatian Pang, Wenhao Jiang, Junwu Zhang, Zong- wei Li, et al. Languagebind: Extending video-language pretraining to n-modality by language-based semantic align- ment. In The Twelfth International Conference on Lea...

    2024

  45. [53]

    Minigpt-4: Enhancing vision-language understanding with advanced large language models

    Deyao Zhu, Jun Chen, Xiaoqian Shen, Xiang Li, and Mo- hamed Elhoseiny. Minigpt-4: Enhancing vision-language understanding with advanced large language models. In 12th International Conference on Learning Representations, ICLR 2024, 2024. 3

    2024

  46. [54]

    Task- customized mixture of adapters for general image fusion

    Pengfei Zhu, Yang Sun, Bing Cao, and Qinghua Hu. Task- customized mixture of adapters for general image fusion. In Proceedings of the IEEE/CVF Conference on Computer Vi- sion and Pattern Recognition, pages 7099–7108, 2024. 3

    2024

  47. [55]

    A comprehensive survey on transfer learning

    Fuzhen Zhuang, Zhiyuan Qi, Keyu Duan, Dongbo Xi, Yongchun Zhu, Hengshu Zhu, Hui Xiong, and Qing He. A comprehensive survey on transfer learning. Proceedings of the IEEE, 109(1):43–76, 2021. 5

    2021

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.