Eddy-VL 1.9B: Structural Pruning and Layered Distillation for Edge-Deployable Multimodal Embedding

Changwoo Kim; HanYeong Cho; Jimin Park; Taeuk Chu

REVIEW 2 major objections 4 minor 23 references

Eddy-VL 1.9B: Structural Pruning and Layered Distillation for Edge-Deployable Multimodal Embedding

T0 review · 2 major / 4 minor · reviewed 2026-08-02 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Pruning four decoder layers and layering CKA distillation produces a 1.93B embedder at 91.7% of teacher score and 10% lower latency.

desk verdict Useful compression recipe and a released 1.9B embedder, but the headline 91.7% retention number is not trustworthy as measured because the distillation data overlap MMEB-V2 images and the pruning layer set was chosen on a subset of the same benchmark. read the letter →

arxiv 2607.16316 v1 pith:CASFFQQQ submitted 2026-07-15 cs.CV

HanYeong Cho , Changwoo Kim , Taeuk Chu , Jimin Park This is my paper · ORCID

classification cs.CV

keywords multimodalembeddingstructuralpruningknowledgedistillationCKAvision-languageretrievaledgedeploymentMatryoshkarepresentationlearningMMEB-V2

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

Eddy-VL 1.9B is a compressed multimodal embedding model for offline, edge retrieval. The authors show that depth pruning alone drops the MMEB-V2 score from 68.9 to 56.8, and that a two-part recipe—probe-driven selection of four layers to remove, then layered knowledge distillation from the unchanged teacher—recovers the score to 63.2, or 91.7% of teacher performance. The compression also cuts forward latency by about 10% at fixed batch size. The claim matters because it offers a route to run capable vision-language retrieval on air-gapped forensic hardware without a cloud API.

What carries the argument

Probe-driven structural pruning scores each text-decoder layer by the sum of linear CKA between its hidden states and those of its neighbors, so layers whose representations are most similar to adjacent ones are candidates for removal. A greedy top-4 choice fails on classification-heavy tasks, so the paper adds a balance score and chooses D={3,22,24,25}. Then 'hole-covering' mapping M={(3,4),(21,23),(22,26),(23,27),(24,28)} brackets each removed block, and layered distillation matches teacher and student attention maps with 1−CKA in the middle and MRL MSE+cosine at the final layer.

What would settle it

Retrain the same pruned student with a corpus that provably excludes all MMEB-V2 images (e.g., an independent image-caption set) and re-run the 78-task benchmark; if the overall score drops materially below 63.2 toward the 56.8 prune-only baseline, the reported recovery is partly memorization of evaluation images rather than genuine distillation.

Watch

Extended reading notes

Core claim

Using adjacent-layer linear CKA on 1,000 calibration pairs, the paper ranks text-decoder blocks by redundancy and removes blocks 3, 22, 24 and 25, shrinking the decoder from 28 to 24 layers. A 'hole-covering' teacher–student layer map aligns each removed block's neighbors, and distillation with mid-layer attention-map 1−CKA plus final-layer MSE/cosine and Matryoshka dimensions {128,...,2048} recovers 6.4 of the 12.1 points lost by pruning alone. Under the authors' fixed evaluation protocol the release model scores 63.2 overall versus 68.9 for the teacher, with near-parity on SugarCrepe, MR2-Bench and ARO, and 6.8 versus 8.5 on Winoground as the main weakness.

Load-bearing premise

The load-bearing premise is that the distillation data—Korean captions on MS COCO images plus an image-only set built partly from COCO and SUN—are disjoint from MMEB-V2 evaluation tasks; the paper provides no exclusion or overlap analysis, so if those datasets leak into evaluation, the 63.2 score is inflated.

Editorial extensions

If this is right

Depth pruning is a structurally cheap way to compress a VLM embedder: removing 4 of 28 decoder layers costs 12.1 points, and most of that is recoverable.
Layered attention-map CKA beats final-hidden-only distillation (63.2 vs 60.8) and beats RCKA, STRUCTURE, and mutual-kNN alternatives under the same recipe.
Compositional benchmarks (SugarCrepe, MR2-Bench, ARO) recover to near teacher level, indicating the removed layers are not the main carriers of compositional reasoning; Winoground remains the residual gap.
A ~10% latency reduction at a fixed batch size on a single edge GPU makes the model a practical candidate for real-time candidate surfacing in air-gapped settings.
Matryoshka dimensions are preserved, so one encoder can serve embeddings of dimension 128–2048 without retraining.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The reported 63.2 may overstate the true recovery because the distillation corpora include MS COCO and SUN images, and MMEB-V2 contains retrieval and classification tasks over those same datasets; the paper gives no exclusion or overlap analysis. This is my inference, not a claim of the paper.
The layer-choice story suggests a general recipe: probe-sum ranking alone is insufficient; a symmetry or balance measure over neighboring CKA values is needed to avoid removing layers before a sharp representation ramp.
A natural extension would test whether the same prune-and-distill recipe transfers to other 28-layer decoders or to video and document branches, and whether the 6.4-point recovery scales with distillation corpus size.
The mid-layer mapping brackets only four holes; an open question is whether more aggressive pruning (e.g., 6–8 layers) with the same layered CKA objective retains the 91.7% ratio or breaks down.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

You should know two things. First, this is a genuinely useful engineering paper: it shows how to take Qwen3-VL-Embedding-2B down to 24 decoder layers with a layered CKA distillation that recovers most of the pruned gap, and it ships weights. Second, the central measured claim—63.2 vs 68.9, 91.7% retention—should not be taken at face value. The stress-test note has it right: Korean COCO is literally MS COCO images, OurDataset mixes in COCO and SUN, and those datasets appear in MMEB-V2 tasks. With a frozen vision encoder, the text decoder can learn to map those exact images' pooled representations to the teacher's outputs, inflating the student relative to a teacher that never saw them. The paper never offers an overlap analysis or exclusion list; the limitations section doesn't mention it.

What is genuinely new: they pick four layers by adjacent CKA, find the greedy choice collapses on classification tasks, and swap in layer 3 for early-depth balance. The ablation in Table 4 is a useful caution about probe-sum pruning. The distillation mapping 'hole-covering' brackets each removed block, and the comparison of hidden vs layered vs RCKA/STRUCTURE/mutual-kNN is clean. The protocol is transparent: fixed prompt, 8 frames, teacher re-eval 68.9 with their own number, not the leaderboard 73.2.

Soft spots in proportion: the overlap issue is the big one, and it directly threatens the headline 'recovering 6.4 of 12.1 points.' Also, the pruning ablation selects the layer set using 8 MMEB-Image tasks that are part of the 78-task MMEB-V2. That is model selection on the test benchmark, which biases the 63.2 upward regardless of data overlap. The fix is straightforward: re-evaluate on a clean held-out set, or at least show which MMEB-V2 tasks share images with their training corpora. The latency reduction is modest but real. Winoground being weak is a known limitation, not a red flag.

Who this is for: people doing edge deployment of multimodal embedders and model compression practitioners. They get a useful recipe and a clear warning about probe-sum pruning. As a referee, I would ask for a held-out re-measurement before trusting the numbers. The paper deserves a serious referee, not a desk reject, because the engineering is transparent and the release is useful. I would accept peer review but with heavy revision demands. I would not cite the 91.7% number; I might cite the recipe.

Referee Report

2 major / 4 minor

Summary. Eddy-VL 1.9B is a compressed multimodal embedding model derived from Qwen3-VL-Embedding-2B by removing four text-decoder layers (28→24) and fine-tuning the remaining decoder with layered knowledge distillation on two proprietary corpora (§3): Korean COCO (image–caption) and OurDataset (image-only). The paper reports MMEB-V2 63.2 vs. 68.9 for the teacher, i.e., 91.7% retention and recovery of 6.4 of the 12.1 points lost by pruning alone; near-teacher scores on SugarCrepe, MR2-Bench, and ARO; and a ~10% latency reduction (§6–§7). Model weights and inference code are released. The central claim is that probe-driven depth pruning plus layered CKA distillation preserves most of a strong multimodal embedder's retrieval quality for edge deployment.

Significance. If the headline MMEB-V2 result were cleanly measured, this would be a practically useful contribution: it demonstrates a concrete compression path for a leading multimodal embedder, with a fairly complete comparison of distillation objectives and an honest discussion of the difference between the authors' fixed protocol and the Qwen leaderboard score. The release of weights, the transparent protocol caveats, and the inclusion of a pruned-baseline ablation are strengths. However, the two protocol issues below directly affect the validity of the central 63.2 / 91.7% claim, so the paper's main empirical conclusion is not currently established.

major comments (2)

[§3.1–§3.2 vs. §6.1, Table 4] The headline result is not a clean held-out measurement. §3.1 says Korean COCO consists of MS COCO images; §3.2 says OurDataset includes MS COCO and SUN images in addition to other public benchmarks. MMEB-V2 is described as a 78-task benchmark whose image subset includes tasks such as SUN397 (visible in the paper's own Table 4). The paper provides no exclusion analysis or overlap check between the distillation corpora and the MMEB-V2 task images, and §6.1's statement that fine-grained probes do not overlap MMEB-V2 is irrelevant to this issue. Because the vision encoder is frozen, the trained text decoder can become tuned to embeddings of the specific COCO/SUN images seen in distillation, which can inflate zero-shot retrieval/classification on those images. Thus the 63.2 and 91.7% retention figures are not supported as stated.
[§4.1, Table 4] The pruning layer set D={3,22,24,25} is not chosen without using the test benchmark. To choose between this set and {2,21,22,24}, the authors evaluate both on 'MMEB-Image (8 tasks)' and use those scores to select the release set. Since MMEB-V2 overall is the unweighted mean over 78 tasks that includes these same image tasks, the final reported MMEB-V2 score is partly a consequence of model selection on the test metric. This upward bias is independent of the training-data overlap issue. The paper should use a validation split disjoint from the final benchmark for all pruning and checkpoint decisions, or report results for all candidate layer sets with a properly held-out estimate.

minor comments (4)

[§5 / Table 5] The text says dynamic loss scaling targets a 40/60 mid:final ratio, while Table 5 reports mid/final loss weights 0.3/0.7. Please reconcile these two numbers.
[§4.1] The ranking description is internally inconsistent: it first states greedy top-4 CKA would remove {21,22,23,24}, then says ranking only by s(ℓ) suggests {2,21,22,24} even though block 2 is ranked 7th by the numbers in Table 3. Clarify the ranking rule, any adjacency/early-depth constraints, and the role of Table 3 in choosing D.
[§6.1 / Table 6] The public leaderboard row reports only Overall=73.2. Since the paper stresses that the authors' protocol differs from the leaderboard, it would be informative to include the public leaderboard's image/video/VisDoc breakdown if it is available.
[§3.2] Minor typo: 'R VL-CDIP' should read 'RVL-CDIP'.

Circularity Check

2 steps flagged · score 6.0 of 10

Headline retention claim is partly tuned to the evaluation: the pruning layer set is selected on MMEB-Image tasks that are included in MMEB-V2, and the distillation corpora contain COCO/SUN images used in MMEB-V2 without any exclusion analysis.

fitted input called prediction [§4.1 (Table 4) and §6.1]
"We validate the layer choice with a structural-pruning ablation (weight remapping only, no distillation): export a 24-layer student with {2,21,22,24} removed and score it on MMEB-Image (batch size 128; Table 4). ... supporting D={3,22,24,25} for the release pruned student. ... MMEB-V2 overall is the unweighted mean of 78 per-task metrics (36 image + 18 video Hit@1; 24 VisDoc nDCG@5)."

The removal set D—the central architectural choice of the paper—is selected by comparing scores on eight MMEB-Image tasks (HatefulMemes, ImageNet-1K, SUN397, VOC2007, etc.). Those same image tasks are a subset of the 36 image tasks that constitute the MMEB-V2 overall score later reported as the headline 63.2. Thus the layer set is tuned on a subset of the benchmark used to claim 91.7% teacher retention, so the reported number is not an independent held-out measurement of the pruning decision.
other [§3.1, §3.2, §6.1]
"Korean COCO denotes the Korean Image Captioning Dataset ... MS COCO images paired with Korean text captions ... OurDataset ... built by merging curated subsets from multiple sources: ... public benchmarks (MS COCO [13], Korean COCO [9], SUN [20], R VL-CDIP [5], CORD v2 [16]) ... each sample is a single image with the default retrieval instruction only (image-only distillation). ... We evaluate on the official MMEB-V2 benchmark [14] with the public VLM2Vec harness [8] (78 tasks)."

The distillation corpora contain MS COCO and SUN images, and MMEB-V2 includes retrieval/classification tasks on the same datasets. The paper provides no exclusion procedure or overlap analysis; the only non-overlap assertion in §6.1 concerns the separate fine-grained probes, not MMEB-V2. With the vision encoder frozen, training the text decoder on these images can directly improve the pooled representations that are scored, so the claimed 63.2 vs. 68.9 retention is not an uncontaminated out-of-sample result.

full rationale

The compression pipeline itself—adjacent-layer CKA ranking, hole-covering layer mapping, attention-map 1−CKA mid-losses, MRL MSE+cosine final loss, and the teacher re-evaluation protocol—is internally coherent and not circular on its own. The circularity comes from two benchmark-coupling decisions that make the headline 91.7% retention partly a consequence of the evaluation rather than an independent test. First, the pruning set D is chosen by scoring candidate layer sets on MMEB-Image tasks, which are part of the MMEB-V2 overall metric; this is model selection on a subset of the reported test benchmark. Second, the distillation corpora include COCO and SUN images, and MMEB-V2 is evaluated on those same datasets, with no exclusion analysis; the paper's limitations section does not address this threat. These two issues bias the same headline score upward. The fine-grained compositional numbers (SugarCrepe, ARO, Winoground, MR2) are separate but could be affected by COCO image overlap as well. Apart from these coupling problems, the derivation does not reduce to a self-citation or an ansatz-imported uniqueness claim.

Assumptions & free parameters 4 free parameters · 4 assumptions · 0 invented entities

The central claims rest on the choice of pruned layers, distillation hyperparameters, and the implicit assumption that the proprietary training data is both effective and disjoint from the benchmark. The most consequential assumption—data disjointness—is likely violated.

free parameters (4)

Pruned layer set D={3,22,24,25} = blocks 3, 22, 24, 25
Chosen by CKA ranking plus an ablation on 8 MMEB-Image tasks; the alternative set {2,21,22,24} was rejected because it scored far worse. This is a discrete hyperparameter tuned on the evaluation benchmark.
Distillation loss weights = mid/final = 0.3/0.7; dynamic scaling target 40/60
Reported as default hyperparameters; no sensitivity analysis is provided.
CKA calibration size = N=1,000
Number of image-caption pairs used to rank layers; chosen without justification.
Training recipe = batch=32, lr=3.63e-5, warmup=0.05, steps=15068, best checkpoint=12366
Standard hyperparameters; best checkpoint selected by training loss, not independently validated.

assumptions (4)

domain assumption Linear CKA on adjacent-layer activations is a reliable proxy for layer redundancy.
Central to the pruning procedure in §4.1; if CKA does not capture what is actually redundant for retrieval, the pruned set is unjustified.
domain assumption Knowledge distillation on Korean COCO + OurDataset transfers teacher performance to the pruned student.
Assumes the proprietary training mixture is representative of the MMEB-V2 distribution; the paper does not test transfer to a held-out clean distribution.
domain assumption The MMEB-V2 benchmark under a fixed generic instruction measures general retrieval capability.
The authors note their protocol differs from the public leaderboard (generic instruction, 8 frames); this weaker protocol is used for all conclusions.
domain assumption Training and evaluation image sets are disjoint.
Implied by the evaluation design but contradicted by the dataset descriptions: Korean COCO uses MS COCO images and OurDataset includes COCO/SUN, which appear in MMEB-V2 tasks. No exclusion mechanism is described.

how reviews work

0 comments

Cite this review

Pith. "Pith review of Eddy-VL 1.9B: Structural Pruning and Layered Distillation for Edge-Deployable Multimodal Embedding." pith.science (2026). https://pith.science/paper/CASFFQQQ

@misc{pith2026260716316,
  author       = {Pith},
  title        = {Pith review of: Eddy-VL 1.9B: Structural Pruning and Layered Distillation for Edge-Deployable Multimodal Embedding},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/CASFFQQQ}},
  note         = {Machine review of arXiv:2607.16316}
}

read the original abstract

In this report, we introduce Eddy-VL 1.9B, a compressed multimodal embedding model built on Qwen3-VL-Embedding-2B for offline, edge-deployable vision-language retrieval. Eddy-VL targets air-gapped forensic and investigative settings where cloud APIs are unavailable and low latency is essential. Compression combines (i) probe-driven structural pruning that removes four redundant text-decoder layers (28 to 24) ranked by adjacent-layer linear CKA, and (ii) layered knowledge distillation with hole-covering teacher-student mappings, mid-layer attention-map 1-CKA, and final-layer MSE and cosine losses with Matryoshka dimensions {128, 256, 512, 1024, 2048}. The released model contains 1,926,188,032 parameters (3.85 GB bf16), representing approximately 9.5% fewer parameters than the 2.13B teacher model. Empirical evaluations on MMEB-V2 (78 tasks, VLM2Vec protocol) show that Eddy-VL achieves an overall score of 63.2 compared with 68.9 for the teacher, retaining 91.7% of the teacher's performance while recovering 6.4 of the 12.1 points lost through pruning alone (56.8). Compositional reasoning performance remains close to the teacher on SugarCrepe (86.1 vs. 86.4), MR2-Bench (24.5 vs. 24.7), and ARO (59.5 vs. 60.4), while Winoground group performance (6.8 vs. 8.5) remains the primary limitation. Depth pruning also reduces forward latency by approximately 10% (150.0 to 136.4 ms per image on NVIDIA DGX Spark using FlashAttention-2). We present the architecture, compression methodology, training procedures, and evaluation results, demonstrating the effectiveness of Eddy-VL for multimodal retrieval under constrained edge deployment. Model weights and inference code are publicly available on Hugging Face.

Figures

Figures reproduced from arXiv: 2607.16316 by the authors.

**Figure 2.** Teacher layer probe: s(ℓ) vs. 0-based decoder block index ℓ ∈ {0, . . . , 27}. Red bars: removed blocks D = {3, 22, 24, 25}. Beyond the s(ℓ) sum. Ranking only by s(ℓ) also suggests an alternative four-block set {2, 21, 22, 24} (blocks 2, 21, 22, and 24 rank 7th, 3rd, 1st, and 4th among middle blocks). We therefore inspect the decomposition s(ℓ) = CKA(Xℓ−1, Xℓ) + CKA(Xℓ , Xℓ+1) rather than the sum alone ( [PITH_FULL… view at source ↗

**Figure 3.** Release distillation loss curve (best checkpoint highlighted). [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗

Figures from the paper (2 more)

**Figure 4.** Figure 4: MMEB-V2 overall for the main compression path: teacher [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

**Figure 5.** Figure 5: Capability retention vs. teacher on compositional benchmarks. [PITH_FULL_IMAGE:figures/full_fig_p009_5.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

23 extracted references · 6 linked inside Pith

[1]

Qwen3-VL technical report.arXiv preprint arXiv:2511.21631, 2025

Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Liang- hao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, 9 Dayiheng Liu, Shix...

arXiv 2025
[2]

Eddy-VL Embedding 1.9B

Hanyeong Cho, Changwoo Kim, Taeuk Chu, and Jimin Park. Eddy-VL Embedding 1.9B. https://huggingface.co/Urock-AI/Eddy-vl_embedding_1.9B_v1, 2026

2026
[3]

FlashAttention: Fast and memory-efficient exact attention with IO-awareness

Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher R´ e. FlashAttention: Fast and memory-efficient exact attention with IO-awareness. InNeurIPS, 2022

2022
[4]

With limited data for multimodal alignment, let the STRUCTURE guide you

Fabian Gr¨ oger, Shuo Wen, Huyen Le, and Maria Brbi´ c. With limited data for multimodal alignment, let the STRUCTURE guide you. InNeurIPS, 2025. arXiv:2506.16895

arXiv 2025
[5]

Evaluation of deep convolu- tional nets for document image classification and retrieval

Adam W Harley, Alex Ufkes, and Konstantinos G Derpanis. Evaluation of deep convolu- tional nets for document image classification and retrieval. InICDAR, 2015

2015
[6]

Distilling the knowledge in a neural network

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015

arXiv 2015
[7]

SugarCrepe: Fixing hackable benchmarks for vision-language compositionality

Cheng-Yu Hsieh, Zixian Zhang, Zixiang Ma, Aniruddha Kembhavi, and Ranjay Krishna. SugarCrepe: Fixing hackable benchmarks for vision-language compositionality. InNeurIPS, 2023

2023
[8]

VLM2Vec: Training vision-language models for massive multimodal embedding tasks

Ziyan Jiang, Rui Meng, Xinyi Yang, Semih Yavuz, Yingbo Zhou, and Wenhu Chen. VLM2Vec: Training vision-language models for massive multimodal embedding tasks. In ICLR, 2025. arXiv:2410.05160

arXiv 2025

Show all 23 references

[9]

Korean Image Captioning Dataset (ai hub dataset no

Korea Electronics Technology Institute. Korean Image Captioning Dataset (ai hub dataset no. 261).https://aihub.or.kr/aihubdata/data/view.do?dataSetSn=261, 2022. Ko- rean captions for MS COCO images; machine translation from English with error correc- tion

2022
[10]

Similarity of neural network representations revisited

Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. Similarity of neural network representations revisited. InICML, 2019

2019
[11]

Matryoshka representation learning

Aditya Kusupati, Gantavya Bhatt, Aniruddha Rege, Karthik Narasimhan, Kaushik Jataval- labhula, Manish Khaitan, Bommasani Pramanick, Aniruddha Kembhavi, and Ali Farhadi. Matryoshka representation learning. InNeurIPS, 2022

2022
[12]

Qwen3-VL- Embedding and Qwen3-VL-Reranker: A unified framework for state-of-the-art multimodal retrieval and ranking.arXiv preprint arXiv:2601.04720, 2026

Mingxin Li, Yanzhao Zhang, Dingkun Long, Keqin Chen, Sibo Song, Shuai Bai, Zhibo Yang, Pengjun Xie, An Yang, Dayiheng Liu, Jingren Zhou, and Junyang Lin. Qwen3-VL- Embedding and Qwen3-VL-Reranker: A unified framework for state-of-the-art multimodal retrieval and ranking.arXiv ...

2026 arXiv
[13]

Lawrence Zitnick

Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Doll´ ar, and C. Lawrence Zitnick. Microsoft COCO: Common objects in context. In ECCV, pages 740–755, 2014

2014
[14]

VLM2Vec- V2: Advancing multimodal embedding for videos, images, and visual documents.Transac- tions on Machine Learning Research, 2026

Rui Meng, Ziyan Jiang, Ye Liu, Mingyi Su, Xinyi Yang, Yuepeng Fu, Can Qin, Zeyuan Chen, Ran Xu, Caiming Xiong, Yingbo Zhou, Wenhu Chen, and Semih Yavuz. VLM2Vec- V2: Advancing multimodal embedding for videos, images, and visual documents.Transac- tions on Machine Learning Rese...

2026 arXiv
[15]

AI Hub (korean public ai training data portal)

National Information Society Agency. AI Hub (korean public ai training data portal). https://www.aihub.or.kr, 2024. Licensed training datasets; redistributable subsets not included in our release

2024
[16]

CORD: A consolidated receipt dataset for post-OCR parsing

Seunghyun Park, Seung Shin, Bado Lee, Junyeop Lee, Jaeheung Surh, Minjoon Seo, and Hwalsuk Lee. CORD: A consolidated receipt dataset for post-OCR parsing. InNeurIPS Document Intelligence Workshop, 2019

2019
[17]

Relational knowledge distillation

Wonpyo Park, Dongju Kim, Yan Lu, and Minsu Cho. Relational knowledge distillation. In CVPR, 2019

2019
[18]

Qwen3-VL-Embedding-2B.https://huggingface.co/Qwen/ Qwen3-VL-Embedding-2B, 2025

Qwen Team. Qwen3-VL-Embedding-2B.https://huggingface.co/Qwen/ Qwen3-VL-Embedding-2B, 2025

2025
[19]

Winoground: Probing vision and language models for visio-linguistic compositionality

Tristan Thrush, Ryan Jiang, Max Bartolo, Amanpreet Singh, Adina Williams, Douwe Kiela, and Candace Ross. Winoground: Probing vision and language models for visio-linguistic compositionality. InCVPR, pages 5238–5248, 2022

2022
[20]

SUN database: Large-scale scene recognition from abbey to zoo

Jianxiong Xiao, James Hays, Krista A Ehinger, Antonio Oliva, and Antonio Torralba. SUN database: Large-scale scene recognition from abbey to zoo. InCVPR, pages 3485–3492, 2010

2010
[21]

When and why vision-language models behave like bags-of-words, and what to do about it? InICLR, 2023

Mert Yuksekgonul, Federico Bianchi, Pratyusha Kalluri, Dan Jurafsky, and James Zou. When and why vision-language models behave like bags-of-words, and what to do about it? InICLR, 2023

2023
[22]

MR2-bench: Going beyond matching to reasoning in multimodal retrieval.arXiv preprint arXiv:2509.26378, 2025

Junjie Zhou, Ze Liu, Lei Xiong, Jin-Ge Yao, Yueze Wang, Shitao Xiao, Fenfen Lin, Miguel Hu Chen, Zhicheng Dou, Siqi Bao, Defu Lian, Yongping Xiong, and Zheng Liu. MR2-bench: Going beyond matching to reasoning in multimodal retrieval.arXiv preprint arXiv:2509.26378, 2025

2025
[23]

Rethinking centered kernel alignment in knowledge distillation

Zikai Zhou, Yunhang Shen, Shitong Shao, Linrui Gong, and Shaohui Lin. Rethinking centered kernel alignment in knowledge distillation. InIJCAI, 2024. arXiv:2401.11824. 11

2024 arXiv

Pith tools

Reviewed August 2, 2026 · model on record in the stance chip above.

[1] [1]

Qwen3-VL technical report.arXiv preprint arXiv:2511.21631, 2025

Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Liang- hao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, 9 Dayiheng Liu, Shix...

arXiv 2025

[2] [2]

Eddy-VL Embedding 1.9B

Hanyeong Cho, Changwoo Kim, Taeuk Chu, and Jimin Park. Eddy-VL Embedding 1.9B. https://huggingface.co/Urock-AI/Eddy-vl_embedding_1.9B_v1, 2026

2026

[3] [3]

FlashAttention: Fast and memory-efficient exact attention with IO-awareness

Tri Dao, Dan Fu, Stefano Ermon, Atri Rudra, and Christopher R´ e. FlashAttention: Fast and memory-efficient exact attention with IO-awareness. InNeurIPS, 2022

2022

[4] [4]

With limited data for multimodal alignment, let the STRUCTURE guide you

Fabian Gr¨ oger, Shuo Wen, Huyen Le, and Maria Brbi´ c. With limited data for multimodal alignment, let the STRUCTURE guide you. InNeurIPS, 2025. arXiv:2506.16895

arXiv 2025

[5] [5]

Evaluation of deep convolu- tional nets for document image classification and retrieval

Adam W Harley, Alex Ufkes, and Konstantinos G Derpanis. Evaluation of deep convolu- tional nets for document image classification and retrieval. InICDAR, 2015

2015

[6] [6]

Distilling the knowledge in a neural network

Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531, 2015

arXiv 2015

[7] [7]

SugarCrepe: Fixing hackable benchmarks for vision-language compositionality

Cheng-Yu Hsieh, Zixian Zhang, Zixiang Ma, Aniruddha Kembhavi, and Ranjay Krishna. SugarCrepe: Fixing hackable benchmarks for vision-language compositionality. InNeurIPS, 2023

2023

[8] [8]

VLM2Vec: Training vision-language models for massive multimodal embedding tasks

Ziyan Jiang, Rui Meng, Xinyi Yang, Semih Yavuz, Yingbo Zhou, and Wenhu Chen. VLM2Vec: Training vision-language models for massive multimodal embedding tasks. In ICLR, 2025. arXiv:2410.05160

arXiv 2025