REVIEW 3 major objections 5 minor 126 references
The paper argues that human visual similarity is not one thing: whether two images look alike depends on which aspect — color, shape, lighting, background — is being compared, and that a perceptual metric should let a user name that aspect
Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →
T0 review · deepseek-v4-flash
2026-08-01 15:35 UTC pith:KO42CD76
load-bearing objection A careful, well-executed benchmark-and-metric paper; the core claims hold, but the 'many senses' claim is bounded by VLM-proposed aspects, which the authors themselves concede. the 3 major comments →
The Many Senses of Visual Similarity: A Text-Prompted Image Perceptual Metric
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
Core claim
The central claim is that visual similarity is not a single scalar but a family of distances, one per aspect (color, shape, lighting, background, pose, etc.), and that a metric can be made to pick the right family member from a free-form text prompt. To support this, the paper builds a dataset of 24,342 synthetic image triplets with 257,391 triplet-aspect conditions and 1,044,495 human votes, each vote identifying the odd-one-out under a named aspect. Fine-tuning a vision-language embedding model on this data with a softmax choice model yields TPIPS, which on the in-distribution odd-one-out test reaches 64.1–64.7% rater agreement against a 67.5% human consensus — narrowing the gap from 9.1 t
What carries the argument
The load-bearing object is the text-conditioned pairwise similarity function f(x1, x2 | c), trained end-to-end from odd-one-out judgments: within a triplet, the excluded pair must score highest. The training signal is a softmax over the three pairwise scores matched to human vote distributions with cross-entropy. The paper compares three fusion architectures — late (independent embeddings compared by cosine), mid (per-layer activation distances with text-conditioned channel weights), and early (both images fed into one vision-language model with two tied register tokens and a symmetric attention mask, which guarantees symmetry f(x1,x2|c)=f(x2,x1|c) and identity f(x,x|c)=1 by construction). T
Load-bearing premise
The load-bearing premise is that every similarity sense that matters to humans can be named as a short English noun phrase by the proposing VLM; the paper concedes (Section 3.1, Section 6) that aspects a VLM cannot verbalize are systematically missing from the data, and TPIPS inherits that blind spot.
What would settle it
Build a fresh odd-one-out set in which the aspects are written by human annotators (no VLM in the loop), including attributes that are hard to verbalize or culturally specific, render triplets that vary only along those aspects, and measure whether TPIPS recovers human choices above chance. If human consensus is high but TPIPS's agreement is near chance — or if TPIPS ranks pairs nearly identically under two aspects that humans treat as opposite (e.g., 'shape' vs. 'silhouette') — the claim that the text prompt genuinely steers the comparison fails.
If this is right
- A single metric can serve multiple senses of similarity: the same query image returns different nearest neighbors when prompted with 'object color' versus 'background' versus 'camera distance'.
- Compositional search works by adding aspect-conditioned scores: combining a subject-matter query with a brushwork query retrieves paintings that satisfy both, and swapping one component swaps the result.
- Generative models can be audited along chosen visual axes rather than by one overall number, so a model may be flagged better in lighting but worse in texture.
- Training on synthetic triplets transfers to outputs of real algorithms — image editing, compositing, novel-view synthesis, and single-image 3D — so the metric is usable beyond its training distribution.
- The 'overall' condition also improves generic perceptual benchmarks (BAPPS, NIGHTS), indicating the collected data captures shared perceptual structure, not just aspect-specific quirks.
Where Pith is reading between the lines
- The paper's data-scaling curve (performance plateaus at 60% of training data) suggests the bottleneck is aspect diversity, not triplet count; a testable extension is to invest in harder or more diverse aspect sets rather than more repetitions.
- Because aspect proposals come from the VLM, the dataset cannot include similarities that lack a verbal label; a human-written aspect corpus would directly test how much of human similarity space TPIPS actually covers.
- Compositional retrieval by simple score addition is a linear approximation of logical conjunction; negative constraints or weighted aspect clauses are a natural, unexplored extension.
- The identity property f(x,x|c)=1 for all c means the model cannot express that an image is more self-similar under some aspects than others; applications measuring aspect salience would need a different normalization.
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper introduces TPIPS, a text-prompted image perceptual similarity metric, together with a large-scale human-annotation dataset. The dataset consists of about 25K image triplets generated by text-to-image models, with over 1M human odd-one-out judgments collected across 257K triplet-aspect conditions. Aspect conditions are proposed by LLM/VLM pipelines (Qwen3 for prompt variation, GPT-5.2 for refinement) and then pruned by a human 'can't tell' filter. The authors benchmark a broad set of existing metrics, embedding models, and VLMs, showing a gap to human consensus. They then fine-tune Qwen3-VL-Embedding-8B with late-, mid-, and early-fusion architectures, achieving the best model-human agreement on both the in-distribution odd-one-out test and a separately collected out-of-distribution 2AFC set built from image editing, compositing, novel view synthesis, and single-image 3D outputs. Finally, they demonstrate aspect-conditioned retrieval and compositional retrieval as applications.
Significance. The dataset and benchmark are a substantial contribution: the odd-one-out annotation protocol with multiple free-form aspects per triplet goes beyond existing scalar perceptual similarity datasets, and the paper's evaluation is careful in several respects — separate test set, standard errors, human-consensus ceiling, multiple baselines, ablations over fusion architectures and data scale, and an out-of-distribution 2AFC set spanning real vision algorithms. The authors also commit to releasing code, data, and trained models, which materially strengthens the work. If the main claims hold, TPIPS would be a useful tool for aspect-conditioned image retrieval and for fine-grained evaluation of generative models. The architecture analysis, especially the construction of symmetry and identity properties in the early-fusion model, is thoughtful and clearly described. However, as discussed below, the breadth of the 'many senses' and 'reliable generalization' claims is currently limited by the way aspect conditions are generated and evaluated.
major comments (3)
- [Section 3.1, Table 2, Section 6] The claim that TPIPS captures 'many senses' of similarity and 'generalizes reliably beyond the training distribution' is not fully supported for the aspect space. In the odd-one-out dataset, aspect proposals come from LLM/VLM pipelines (Qwen3-4B for prompt variation, GPT-5.2 for refinement) and are only pruned by a human 'can't tell' filter; this filter can delete proposed aspects that are not visibly varying, but it cannot add aspects that the proposing VLM failed to verbalize. In the OOD 2AFC set, the editing subset uses GPT-5.2-proposed aspects and the other subsets use fixed researcher-selected lists; none of the aspects are elicited from naive human annotators in an open-ended manner. Thus the OOD evaluation shifts the image distribution but not the aspect-generation mechanism. The paper's own limitation statement in Section 6 admits this: 'our aspect proposals are VLM-generated, an
- [Section 3.2 / Appendix A.2] The 2AFC evaluation removes aspects whose post-filter human votes are exactly tied, described as 'not useful for evaluation.' This selection removes ambiguous comparisons from the test set, which raises both the human-consensus ceiling and model agreement. The reported 10.0 percentage-point gap to human consensus on the OOD set is therefore computed on a subset of aspects that are resolvable by majority vote. The practice is defensible, but because it affects the headline generalization number, the paper should either report the tied-aspect rate and its effect on agreement, or evaluate on the full aspect set with an appropriate chance-level handling of ties. Without this, the reader cannot tell how much of the OOD performance depends on excluding the most difficult cases.
- [Section 5, Figure 5 / Table 3] The in-distribution odd-one-out test shares the synthetic text-to-image generation pipeline with the training set. This is acknowledged, and the OOD 2AFC set is a reasonable mitigation. However, the two evaluation sets differ in protocol (odd-one-out vs. 2AFC), image source, and aspect source simultaneously, so it is difficult to attribute the OOD improvement to a single factor. I do not see this as a fatal flaw, but the Discussion should explicitly state that generalization is demonstrated on OOD image distributions plus a mix of VLM-proposed and researcher-fixed aspects, not on OOD aspect distributions. A cleaner decomposition — e.g., a 2AFC set on synthetic images with the same protocol, or an odd-one-out set on algorithm outputs — would strengthen the causal claim that the model learned a transferable conditional similarity function.
minor comments (5)
- [Abstract / Section 1] Typos: 'concensus' in the abstract and 'syntheic' in Section 1 should be corrected.
- [Section 4.2 / Figure 3] The notation for the mid-fusion similarity in Eq. (3) uses w_l(c) as text-conditioned channel weights, but the text says 'averaged across image tokens, and summed over layers.' The equation's placement of the 1/T average inside the sum is clear, but a parenthetical defining T and d in the main text would help readers who skip Appendix C.
- [Table 3] The table is dense and the 'Ours' row appears under multiple fusion families with different numbers. It would be helpful to bold the single best TPIPS variant per column and to clarify in the caption which 'Ours' rows are late, early, and mid fusion, since the main text says the late-fusion model is the default but Figure 5 highlights the early-fusion result.
- [Appendix C.2] The identity property f_theta(x,x|c)=1 is argued by symmetry of the attention mask and tied registers. The reasoning is sound, but it relies on the two image segments being token-for-token identical after preprocessing; this should be stated explicitly in the identity paragraph, since resolution/padding differences between the two copies would break the argument.
- [Section 3.2 / Table 2] For the image-editing subset, the aspect list is 'predicted by GPT-5.2 per sample and human-pruned.' It would be useful to report how many aspects survive the human-pruning step and how often the pruning changes the majority vote, to quantify the human contribution to the final aspect labels.
Circularity Check
No significant circularity: TPIPS is trained on held-out human odd-one-out votes and evaluated on disjoint held-out test splits; the VLM-generated aspect space is an acknowledged coverage limitation, not a circular reduction.
full rationale
The derivation chain is a standard supervised pipeline. Human odd-one-out votes define the target y (Section 3.1), Equation (2) trains f_theta by cross-entropy against those votes, and the headline gaps (9.1% to 2.8% odd-one-out; 15.8% to 10.0% 2AFC) are measured on the disjoint held-out test splits of Table 1 (2,000 OOO triplets; 1,127 2AFC queries), never on training votes. Hyperparameters (tau, LoRA rank) were selected on the 185-triplet validation set (Appendix C.1), so no fitted constant is reported as a prediction. The OOD 2AFC set shifts image content (editing, compositing, NVS, image-to-3D outputs) and uses independently collected human votes, so the transfer claim is empirically grounded. The one load-bearing caveat is the aspect space: aspects are LLM/VLM-proposed (Qwen3-4B, GPT-5.2) and human-pruned in both the training set (Section 3.1) and the editing portion of the 2AFC set (Table 2: '6-10 aspects predicted by GPT-5.2 per sample and human-pruned'). Section 6 concedes: 'Our aspect proposals are VLM-generated, and we can systematically miss aspects that VLMs cannot capture.' This bounds the 'many senses'/'reliable generalization' claims to VLM-verbalizable, human-detectable aspects and would require a human-authored-aspect evaluation set to fully stress-test; but it is a scope limitation, not a circular step, because the human vote labels are collected independently of the model and the model's outputs are compared against those votes rather than derived from the aspect proposals. Self-citations (LPIPS [2], DreamSim [3], TF-QARE [48]) are used as benchmarks, baselines, or protocol precedents; none is load-bearing for the central fine-tuning result, and architecture choices (early vs. mid vs. late fusion) are validated by ablations (Table 6) rather than by citation.
Axiom & Free-Parameter Ledger
free parameters (4)
- softmax temperature τ =
0.05 (late/early fusion), 0.0025 (mid-fusion)
- LoRA rank r =
16
- mid-fusion probe layers =
20
- retrieval diversity weight =
0.3
axioms (6)
- domain assumption Human similarity judgments are context-dependent and can be elicited reliably via odd-one-out triplet comparisons.
- domain assumption Synthetic text-to-image variations from FLUX prompts produce triplets with multiple independent visual aspects in the intended difficulty regime.
- domain assumption VLM/LLM-proposed aspect lists, after human 'can't tell' pruning, cover the visually relevant aspects of each triplet.
- domain assumption Annotator consensus from 5 workers provides a valid ground-truth distribution.
- standard math The softmax choice model with cross-entropy correctly ties pairwise similarities to odd-one-out probabilities.
- domain assumption The Qwen3-VL-Embedding backbone provides a suitable base representation for fine-tuning.
invented entities (1)
-
Register tokens r1, r2 in early-fusion architecture
no independent evidence
read the original abstract
Human visual similarity judgments are context-dependent. For example, two images may be similar in shape but distinct in color. Existing perceptual similarity metrics, however, collapse these nuances into a single scalar value, offering no mechanism to condition on specific aspects. To bridge this gap, we introduce a large-scale dataset of human similarity judgments over image triplets, where each triplet is annotated across multiple, free-form semantic aspects of similarity. Benchmarking a broad range of frontier vision-language models (VLMs) reveals a considerable performance gap compared to human annotators' consensus. Leveraging our data, we fine-tune a VLM to produce our Text-Prompted Image Perceptual Similarity (TPIPS) metric, capturing multiple senses of visual similarity depending on the specified text prompt. We demonstrate that TPIPS aligns more closely with human perception and generalizes reliably beyond the training distribution. Finally, we show that TPIPS unlocks new capabilities in text-guided retrieval, compositional search, and the fine-grained evaluation of generative models. Our code, data, and trained models are at https://peterwang512.github.io/TPIPS
Figures
Reference graph
Works this paper leans on
-
[1]
Image quality assessment: from error visibility to structural similarity.IEEE Transactions on Image Processing, 13(4):600–612, 2004
Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli. Image quality assessment: from error visibility to structural similarity.IEEE Transactions on Image Processing, 13(4):600–612, 2004. 10
2004
-
[2]
The unreasonable effectiveness of deep features as a perceptual metric
Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang. The unreasonable effectiveness of deep features as a perceptual metric. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 586–595, 2018
2018
-
[3]
DreamSim: Learning new dimensions of human visual similarity using synthetic data
Stephanie Fu, Netanel Y Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. DreamSim: Learning new dimensions of human visual similarity using synthetic data. InAdvances in Neural Information Processing Systems (NeurIPS), 2023
2023
-
[4]
Features of similarity.Psychological Review, 84(4):327–352, 1977
Amos Tversky. Features of similarity.Psychological Review, 84(4):327–352, 1977
1977
-
[5]
Respects for similarity.Psychological Review, 100(2):254–278, 1993
Douglas L Medin, Robert L Goldstone, and Dedre Gentner. Respects for similarity.Psychological Review, 100(2):254–278, 1993
1993
-
[6]
FSIM: A feature similarity index for image quality assessment.IEEE Transactions on Image Processing, 20(8):2378–2386, 2011
Lin Zhang, Lei Zhang, Xuanqin Mou, and David Zhang. FSIM: A feature similarity index for image quality assessment.IEEE Transactions on Image Processing, 20(8):2378–2386, 2011
2011
-
[7]
HDR-VDP-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions.ACM Transactions on Graphics (Proc
Rafat Mantiuk, Kil Joong Kim, Allan G Rempel, and Wolfgang Heidrich. HDR-VDP-2: A calibrated visual metric for visibility and quality predictions in all luminance conditions.ACM Transactions on Graphics (Proc. SIGGRAPH), 30(4):40:1–40:14, 2011
2011
-
[8]
GeneCIS: A benchmark for general conditional image similarity
Sagar Vaze, Nicolas Carion, and Ishan Misra. GeneCIS: A benchmark for general conditional image similarity. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6862–6872, 2023
2023
-
[9]
Cheng-Yu Hsieh, Pavan Kumar Anasosalu Vasu, Fartash Faghri, Raviteja Vemulapalli, Chun-Liang Li, Ranjay Krishna, Oncel Tuzel, and Hadi Pouransari. Focallens: Instruction tuning enables zero-shot conditional image representations.arXiv preprint arXiv:2504.08368, 2025
Pith/arXiv arXiv 2025
-
[10]
Tsai-Shien Chen, Aliaksandr Siarohin, Guocheng Gordon Qian, Kuan-Chieh Jackson Wang, Egor Nemchinov, Moayed Haji-Ali, Riza Alp Guler, Willi Menapace, Ivan Skorokhodov, Anil Kag, et al. Omni-attribute: Open-vocabulary attribute encoder for visual concept personalization.arXiv preprint arXiv:2512.10955, 2025
Pith/arXiv arXiv 2025
-
[11]
THINGS: A database of 1,854 object concepts and more than 26,000 naturalistic object images.PLOS ONE, 14(10):e0223792, 2019
Martin N Hebart, Adam H Dickter, Alexis Kidder, Wan Y Kwok, Anna Corriveau, Caitlin Van Wicklin, and Chris I Baker. THINGS: A database of 1,854 object concepts and more than 26,000 naturalistic object images.PLOS ONE, 14(10):e0223792, 2019
2019
-
[12]
GPT-4V(ision) system card.OpenAI Technical Report, 2023
OpenAI. GPT-4V(ision) system card.OpenAI Technical Report, 2023
2023
-
[13]
Gemini: A family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023
Gemini Team, Google. Gemini: A family of highly capable multimodal models.arXiv preprint arXiv:2312.11805, 2023
Pith/arXiv arXiv 2023
-
[14]
Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025
Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, et al. Qwen3-vl technical report.arXiv preprint arXiv:2511.21631, 2025
Pith/arXiv arXiv 2025
-
[15]
Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks
Zhe Chen, Jiannan Wu, Wenhai Wang, Weijie Su, Guo Chen, Sen Xing, Muyan Zhong, Qinglong Zhang, Xizhou Zhu, Lewei Lu, et al. Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks. InProceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 24185–24198, 2024
2024
-
[17]
Qwen3-VL-Embedding and Qwen3-VL- Reranker: A unified framework for state-of-the-art multimodal retrieval and ranking.arXiv preprint, 2026
Mingxin Li, Yanzhao Zhang, Dingkun Long, Keqin Chen, Sibo Song, Shuai Bai, Zhibo Yang, Pengjun Xie, An Yang, Dayiheng Liu, Jingren Zhou, and Junyang Lin. Qwen3-VL-Embedding and Qwen3-VL- Reranker: A unified framework for state-of-the-art multimodal retrieval and ranking.arXiv preprint, 2026
2026
-
[18]
ImageNet classification with deep convolutional neural networks
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. ImageNet classification with deep convolutional neural networks. InAdvances in Neural Information Processing Systems (NeurIPS), 2012
2012
-
[19]
Very deep convolutional networks for large-scale image recognition
Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for large-scale image recognition. InInternational Conference on Learning Representations (ICLR), 2015
2015
-
[20]
Deep residual learning for image recognition
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016
2016
-
[21]
An image is worth 16x16 words: Transformers for image recognition at scale
Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations (ICLR), 2021. 11
2021
-
[22]
Learning transferable visual models from natural language supervision
Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InInternational conference on machine learning, pages 8748–8763. PmLR, 2021
2021
-
[23]
DINOv2: Learning robust visual features without supervision.Transactions on Machine Learning Research (TMLR), 2024
Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V o, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jégou, Julien Mairal, Patrick La...
2024
-
[24]
Perceptual losses for real-time style transfer and super-resolution
Justin Johnson, Alexandre Alahi, and Li Fei-Fei. Perceptual losses for real-time style transfer and super-resolution. InEuropean Conference on Computer Vision (ECCV), pages 694–711. Springer, 2016
2016
-
[25]
Image style transfer using convolutional neural networks
Leon A Gatys, Alexander S Ecker, and Matthias Bethge. Image style transfer using convolutional neural networks. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 2414–2423, 2016
2016
-
[26]
Understanding and simplifying perceptual distances
Dan Amir and Yair Weiss. Understanding and simplifying perceptual distances. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 12226–12235, 2021
2021
-
[27]
Vlic: Vision-language models as perceptual judges for human-aligned image compression
Kyle Sargent, Ruiqi Gao, Philipp Henzler, Charles Herrmann, Aleksander Holynski, Li Fei-Fei, Jiajun Wu, and Jason Y Zhang. Vlic: Vision-language models as perceptual judges for human-aligned image compression. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026
2026
-
[28]
PieAPP: Perceptual image-error assess- ment through pairwise preference
Ekta Prashnani, Hong Cai, Yasamin Mostofi, and Pradeep Sen. PieAPP: Perceptual image-error assess- ment through pairwise preference. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1808–1817, 2018
2018
-
[29]
Image quality assessment: Unifying structure and texture similarity.IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 44(5):2567–2581, 2020
Keyan Ding, Kede Ma, Shiqi Wang, and Eero P Simoncelli. Image quality assessment: Unifying structure and texture similarity.IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI), 44(5):2567–2581, 2020
2020
-
[30]
Human alignment of neural network representations
Lukas Muttenthaler, Jonas Dippel, Lorenz Linhardt, Robert A Vandermeulen, and Simon Kornblith. Human alignment of neural network representations. InInternational Conference on Learning Represen- tations (ICLR), 2023
2023
-
[31]
Shift-tolerant perceptual similarity metric
Abhijay Ghildyal and Feng Liu. Shift-tolerant perceptual similarity metric. InEuropean Conference on Computer Vision (ECCV), pages 91–107. Springer, 2022
2022
-
[32]
LipSim: A provably robust perceptual similarity metric
Sara Ghazanfari, Alexandre Araujo, Prashanth Krishnamurthy, Farshad Khorrami, and Siddharth Garg. LipSim: A provably robust perceptual similarity metric. InInternational Conference on Learning Representations (ICLR), 2024
2024
-
[33]
Markus Kettunen, Erik Härkönen, and Jaakko Lehtinen. E-LPIPS: Robust perceptual image similarity via random transformation ensembles.arXiv preprint arXiv:1906.03973, 2019
Pith/arXiv arXiv 1906
-
[34]
A similarity measure for illustration style
Elena Garces, Aseem Agarwala, Diego Gutierrez, and Aaron Hertzmann. A similarity measure for illustration style. 2014
2014
-
[35]
Measuring style similarity in diffusion models
Gowthami Somepalli, Anubhav Gupta, Kamal Gupta, Shramay Palta, Micah Goldblum, Jonas Geiping, Abhinav Shrivastava, and Tom Goldstein. Measuring style similarity in diffusion models. InEuropean Conference on Computer Vision (ECCV), 2024
2024
-
[36]
Relational visual similarity
Thao Nguyen, Sicheng Mo, Krishna Kumar Singh, Yilin Wang, Jing Shi, Nicholas Kolkin, Eli Shechtman, Yong Jae Lee, and Yuheng Li. Relational visual similarity. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026
2026
-
[37]
Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. ImageReward: Learning and evaluating human preferences for text-to-image generation.arXiv preprint arXiv:2304.05977, 2023
Pith/arXiv arXiv 2023
-
[38]
Pick-a-pic: An open dataset of user preferences for text-to-image generation
Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation. InNeural Information Processing Systems (NeurIPS), 2023
2023
-
[39]
Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text-to-image synthesis. arXiv preprint arXiv:2306.09341, 2023
Pith/arXiv arXiv 2023
-
[40]
Editreward: A human- aligned reward model for instruction-guided image editing
Keming Wu, Sicong Jiang, Max Ku, Ping Nie, Minghao Liu, and Wenhu Chen. Editreward: A human- aligned reward model for instruction-guided image editing. InInternational Conference on Learning Representations (ICLR), 2026. 12
2026
-
[41]
Multiview triplet embedding: Learning attributes in multiple maps
Ehsan Amid and Antti Ukkonen. Multiview triplet embedding: Learning attributes in multiple maps. In International Conference on Machine Learning (ICML), pages 1472–1480. PMLR, 2015
2015
-
[42]
Conditional similarity networks
Andreas Veit, Serge Belongie, and Theofanis Karaletsos. Conditional similarity networks. InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 830–838, 2017
2017
-
[43]
Learning type-aware embeddings for fashion compatibility
Mariya I Vasileva, Bryan A Plummer, Krishna Dusad, Shreya Rajpal, Ranjitha Kumar, and David Forsyth. Learning type-aware embeddings for fashion compatibility. InEuropean Conference on Computer Vision (ECCV), pages 390–405, 2018
2018
-
[44]
William Thong, Cees GM Snoek, and Arnold WM Smeulders. Cooperative embeddings for instance, attribute and category retrieval.arXiv preprint arXiv:1904.01421, 2019
Pith/arXiv arXiv 1904
-
[45]
Learning similarity conditions without explicit supervision
Reuben Tan, Mariya I Vasileva, Kate Saenko, and Bryan A Plummer. Learning similarity conditions without explicit supervision. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 10373–10382, 2019
2019
-
[46]
Training-free conditional image embedding framework leveraging large vision language models
Masayuki Kawarada, Kosuke Yamada, Antonio Tejero-de Pablos, and Naoto Inoue. Training-free conditional image embedding framework leveraging large vision language models. InProceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, pages 7636–7646, 2026
2026
-
[47]
Highlighting what matters: Promptable embeddings for attribute-focused image retrieval
Siting Li, Xiang Gao, and Simon Shaolei Du. Highlighting what matters: Promptable embeddings for attribute-focused image retrieval. InAdvances in Neural Information Processing Systems (NeurIPS), 2025
2025
-
[48]
Towards text-guided attribute-disentangled multimodal representation learning
Yibing Wei, Sudeep Katakol, Manuel Brack, Jinhong Lin, Haoyue Bai, Yu-Teng Li, Richard Zhang, Eli Shechtman, Hareesh Ravi, and Ajinkya Kale. Towards text-guided attribute-disentangled multimodal representation learning. InCVPR, 2026
2026
-
[49]
Open ad-hoc categorization with contextualized feature learning
Zilin Wang, Sangwoo Mo, Stella X Yu, Sima Behpour, and Liu Ren. Open ad-hoc categorization with contextualized feature learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2025
2025
-
[50]
Image clustering conditioned on text criteria
Sehyun Kwon, Jaeseung Park, Minkyu Kim, Jaewoong Cho, Ernest K Ryu, and Kangwook Lee. Image clustering conditioned on text criteria. InInternational Conference on Learning Representations (ICLR), 2024
2024
-
[51]
Composing text and image for image retrieval-an empirical odyssey
Nam V o, Lu Jiang, Chen Sun, Kevin Murphy, Li-Jia Li, Li Fei-Fei, and James Hays. Composing text and image for image retrieval-an empirical odyssey. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 6439–6448, 2019
2019
-
[52]
Image retrieval on real-life images with pre-trained vision-and-language models
Zheyuan Liu, Cristian Rodriguez-Opazo, Damien Teney, and Stephen Gould. Image retrieval on real-life images with pre-trained vision-and-language models. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 2125–2134, 2021
2021
-
[53]
Pic2Word: Mapping pictures to words for zero-shot composed image retrieval
Kuniaki Saito, Kihyuk Sohn, Xiang Zhang, Chun-Liang Li, Chen-Yu Lee, Kate Saenko, and Tomas Pfister. Pic2Word: Mapping pictures to words for zero-shot composed image retrieval. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19305–19314, 2023
2023
-
[54]
Zero-shot composed image retrieval with textual inversion
Alberto Baldrati, Lorenzo Agnolucci, Marco Bertini, and Alberto Del Bimbo. Zero-shot composed image retrieval with textual inversion. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 15338–15347, 2023
2023
-
[55]
Sigmoid loss for language image pre-training
Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. Sigmoid loss for language image pre-training. InProceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 11975–11986, 2023
2023
-
[56]
Reproducible scaling laws for contrastive language-image learning
Mehdi Cherti, Romain Beaumont, Ross Wightman, Mitchell Wortsman, Gabriel Ilharco, Cade Gordon, Christoph Schuhmann, Ludwig Schmidt, and Jenia Jitsev. Reproducible scaling laws for contrastive language-image learning. InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2818–2829, 2023
2023
-
[57]
BLIP: Bootstrapping language-image pre- training for unified vision-language understanding and generation
Junnan Li, Dongxu Li, Caiming Xiong, and Steven Hoi. BLIP: Bootstrapping language-image pre- training for unified vision-language understanding and generation. InInternational Conference on Machine Learning (ICML), pages 12888–12900. PMLR, 2022
2022
-
[58]
BLIP-2: Bootstrapping language-image pre- training with frozen image encoders and large language models
Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi. BLIP-2: Bootstrapping language-image pre- training with frozen image encoders and large language models. InInternational Conference on Machine Learning (ICML), 2023
2023
-
[59]
Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-VL: A versatile vision-language model for understanding, localization, text reading, and beyond.arXiv preprint arXiv:2308.12966, 2023
Pith/arXiv arXiv 2023
-
[60]
Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al. Qwen2-VL: Enhancing vision-language model’s perception of the world at any resolution.arXiv preprint arXiv:2409.12191, 2024. 13
Pith/arXiv arXiv 2024
-
[61]
Qwen3.5: Unified vision-language foundation with early-fusion multimodal training
Qwen Team. Qwen3.5: Unified vision-language foundation with early-fusion multimodal training. https://github.com/QwenLM/Qwen3.5, 2026. Technical blog post
2026
-
[62]
Flamingo: a visual language model for few-shot learning
Jean-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, Roman Ring, Eliza Rutherford, Serkan Cabi, Tengda Han, Zhitao Gong, Sina Samangooei, Marianne Monteiro, Jacob Menick, Sebastian Borgeaud, Andrew Brock, Aida Nematzadeh, Sahand Sharifzadeh, Mikolaj Binkow...
2022
-
[63]
LLaV A-OneVision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024
Bo Li, Yuanhan Zhang, Dong Guo, Renrui Zhang, Feng Li, Hao Zhang, Kaichen Zhang, Yanwei Li, Ziwei Liu, and Chunyuan Li. LLaV A-OneVision: Easy visual task transfer.arXiv preprint arXiv:2408.03326, 2024
Pith/arXiv arXiv 2024
-
[64]
Deepseek-vl2: Mixture-of-experts vision-language models for advanced multimodal understanding, 2024
Zhiyu Wu, Xiaokang Chen, Zizheng Pan, Xingchao Liu, Wen Liu, Damai Dai, Huazuo Gao, Yiyang Ma, Chengyue Wu, Bingxuan Wang, Zhenda Xie, Yu Wu, Kai Hu, Jiawei Wang, Yaofeng Sun, Yukun Li, Yishi Piao, Kang Guan, Aixin Liu, Xin Xie, Yuxiang You, Kai Dong, Xingkai Yu, Haowei Zhang, Liang Zhao, Yisong Wang, and Chong Ruan. Deepseek-vl2: Mixture-of-experts visio...
2024
-
[65]
Rui Meng, Ziyan Jiang, Ye Liu, Mingyi Su, Xinyi Yang, Yuepeng Fu, Can Qin, Zeyuan Chen, Ran Xu, Caiming Xiong, et al. VLM2Vec-V2: Advancing multimodal embedding for videos, images, and visual documents.arXiv preprint arXiv:2507.04590, 2025
Pith/arXiv arXiv 2025
-
[66]
Rongyao Fang, Aldrich Yu, Chengqi Duan, Linjiang Huang, Shuai Bai, Yuxuan Cai, Kun Wang, Si Liu, Xihui Liu, and Hongsheng Li. Flux-reason-6m & prism-bench: A million-scale text-to-image reasoning dataset and comprehensive benchmark.arXiv preprint arXiv:2509.09680, 2025
Pith/arXiv arXiv 2025
-
[67]
VLM2Vec: Training vision-language models for massive multimodal embedding tasks
Ziyan Jiang, Rui Meng, Xinyi Yang, Semih Yavuz, Yingbo Zhou, and Wenhu Chen. VLM2Vec: Training vision-language models for massive multimodal embedding tasks. InInternational Conference on Learning Representations (ICLR), 2025
2025
-
[68]
NV-Embed: Improved techniques for training LLMs as generalist embedding models
Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. NV-Embed: Improved techniques for training LLMs as generalist embedding models. In International Conference on Learning Representations (ICLR), 2025
2025
-
[69]
Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen
Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. InInternational Conference on Learning Representations (ICLR), 2022
2022
-
[70]
Steerable visual representations.arXiv preprint arXiv:2604.02327, 2026
Jona Ruthardt, Manu Gaur, Deva Ramanan, Makarand Tapaswi, and Yuki M Asano. Steerable visual representations.arXiv preprint arXiv:2604.02327, 2026
Pith/arXiv arXiv 2026
-
[71]
Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, et al. Internvl3. 5: Advancing open-source multimodal models in versatility, reasoning, and efficiency.arXiv preprint arXiv:2508.18265, 2025
Pith/arXiv arXiv 2025
-
[72]
The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale.International Journal of Computer Vision (IJCV), 2020
Alina Kuznetsova, Hassan Rom, Neil Alldrin, Jasper Uijlings, Ivan Krasin, Jordi Pont-Tuset, Shahab Kamali, Stefan Popov, Matteo Malloci, Alexander Kolesnikov, Tom Duerig, and Vittorio Ferrari. The Open Images Dataset V4: Unified image classification, object detection, and visual relationship detection at scale.International Journal of Computer Vision (IJCV), 2020
2020
-
[73]
Improved artgan for conditional synthesis of natural image and artwork.IEEE Transactions on Image Processing, 28(1):394–409, 2019
Wei Ren Tan, Chee Seng Chan, Hernan Aguirre, and Kiyoshi Tanaka. Improved artgan for conditional synthesis of natural image and artwork.IEEE Transactions on Image Processing, 28(1):394–409, 2019
2019
-
[74]
Learning an image editing model without image editing pairs
Nupur Kumari, Sheng-Yu Wang, Nanxuan Zhao, Yotam Nitzan, Yuheng Li, Krishna Kumar Singh, Richard Zhang, Eli Shechtman, Jun-Yan Zhu, and Xun Huang. Learning an image editing model without image editing pairs. InICLR, 2026
2026
-
[75]
Grace Luo, Jonathan Granskog, Aleksander Holynski, and Trevor Darrell. Dual-process image generation. arXiv preprint arXiv:2506.01955, 2025
Pith/arXiv arXiv 2025
-
[76]
DanceGRPO: Unleashing GRPO on visual generation.arXiv preprint arXiv:2505.07818, 2025
Zeyue Xue, Jie Wu, Yu Gao, Fangyuan Kong, Lingting Zhu, Mengzhao Chen, Zhiheng Liu, Wei Liu, Qiushan Guo, Weilin Huang, and Ping Luo. DanceGRPO: Unleashing GRPO on visual generation.arXiv preprint arXiv:2505.07818, 2025
Pith/arXiv arXiv 2025
-
[77]
A multimodal automated interpretability agent
Tamar Rott Shaham, Sarah Schwettmann, Franklin Wang, Achyuta Rajaram, Evan Hernandez, Jacob Andreas, and Antonio Torralba. A multimodal automated interpretability agent. InICML, 2024
2024
-
[78]
Viacheslav Surkov, Chris Wendler, Antonio Mari, Mikhail Terekhov, Justin Deschenaux, Robert West, Caglar Gulcehre, and David Bau. One-step is enough: Sparse autoencoders for text-to-image diffusion models.arXiv preprint arXiv:2410.22366, 2024
arXiv 2024
-
[79]
Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025
An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report.arXiv preprint arXiv:2505.09388, 2025. 14
Pith/arXiv arXiv 2025
-
[80]
FLUX.1: Open-weight rectified flow transformers for text-to-image generation
Black Forest Labs. FLUX.1: Open-weight rectified flow transformers for text-to-image generation. https://blackforestlabs.ai/announcing-black-forest-labs/, 2024
2024
-
[81]
Zhangqi Jiang, Zheng Sun, Xianfang Zeng, Yufeng Yang, Xuanyang Zhang, Yongliang Wu, Wei Cheng, Gang Yu, Xu Yang, and Bihan Wen. Geditbench v2: A human-aligned benchmark for general image editing.arXiv preprint arXiv:2603.28547, 2026
arXiv 2026
discussion (0)
Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.