Pith. sign in

REVIEW 3 major objections 5 minor 23 references

Do Vision Encoders Exhibit Human-like Color Thresholds?

T0 review · 3 major / 5 minor · reviewed 2026-08-01 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read Best vision encoders score under 0.25 on human color-threshold test

desk verdict A broad, clearly written benchmark of local color-discrimination geometry in 50+ encoders, but the headline mIoU scores sit below the chance level (1/3) and the paper never reports a null baseline—so the central negative claim is not yet quantitatively supported. read the letter →

arxiv 2607.16540 v1 pith:H64DQ6YV submitted 2026-07-17 cs.CV

Engy Ehab , Pablo Hernández-Cámara , Nahla Belal , Jesús Malo , Javier Vazquez-Corral , Alexandra Gomez-Villa This is my paper
classification cs.CV
keywords colorperceptionchromaticdiscriminationthresholdsvisionencodersrepresentationgeometryCIEDE2000MacAdamellipsesself-supervisedlearningfoundationmodels
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper asks whether modern vision encoders—convolutional networks, transformers, and multimodal foundation models—organize colors in latent space the way human observers do. To find out, the authors probe more than 50 pretrained encoders with uniform color patches at four saturation levels, deriving a 'chromatic similarity field' from cosine distances in each model's feature space. They then measure the overlap (mean intersection-over-union) between each model's 250 nearest-neighbor colors and the human discrimination ellipses defined by the CIEDE2000 color-difference standard. The best model reaches only about 0.22, and no encoder comes close to human-like alignment; self-supervised models lead, language-supervised models are most polarized, and a VAE-based model lags. The conclusion is that human-like chromatic sensitivity does not emerge naturally from current large-scale visual training objectives.

What carries the argument

The central object is the chromatic similarity field: the local ordering of colors in an encoder's feature space, computed as cosine distance between the reference color's embedding and embeddings of sampled neighboring colors. This field is compared against human discrimination ellipses derived from the CIEDE2000 color-difference formula, which serve as the ground-truth region G. The predicted region P is the set of 250 sampled points with lowest cosine distance to the reference, and the mean intersection-over-union (mIoU) of G and P across 18 reference hues and four chroma levels quantifies alignment. The mIoU is the device that converts the abstract question of 'does the model see color l

What would settle it

Run the identical protocol on a null control model whose similarity field is defined directly by CIEDE2000 or CIELAB distance; if that human-aligned baseline also scores below 0.25 mIoU, the metric cannot detect human-like geometry and the paper's negative conclusion is a measurement artifact. Also, state the exact CIEDE2000 threshold used to define the ground-truth ellipses and verify that the resulting regions reproduce published MacAdam/BFD discrimination ellipses; without that, the ground truth itself is not fixed.

Watch

Extended reading notes

Core claim

The central discovery is that the local geometry of chromatic similarity in pretrained vision encoders is structured and anisotropic, yet systematically misaligned with human discrimination ellipses. Across all 50 evaluated models, the mean intersection-over-union between the set of the 250 most chromatically similar latent neighbors and the CIEDE2000-defined human discrimination region never exceeds 0.22. Alignment varies with hue and chroma—blues are consistently worst, and high-chroma conditions differ from low-chroma ones—and with training paradigm, with self-supervised encoders (e.g., DINOv2, Perception Encoder) achieving the highest scores and language-supervised models spanning both t

Load-bearing premise

The evaluation assumes that the mIoU between the model's 250 nearest cosine-neighbors and a CIEDE2000-defined ellipse is a faithful measure of perceptual alignment, which depends on an unspecified CIEDE2000 threshold and on the cosine-neighbor set being a valid proxy for the model's similarity field.

Editorial extensions

If this is right

  • If the negative result holds, downstream applications that require human-like color fidelity—digital art preservation, product inspection, color-critical design—cannot trust frozen foundation-model embeddings without a perceptual correction step.
  • The consistent advantage of self-supervised over supervised objectives suggests that training signals based on local image structure preserve low-level chromatic relationships better than category-level supervision.
  • The finding that scale and parameter count do not drive alignment implies that simply training larger models will not solve the problem; new objectives or explicit perceptual constraints are needed.
  • The hue- and chroma-dependent variation, particularly the poor performance on blues, indicates that any remedy must be targeted at specific regions of color space rather than a global adjustment.
  • The stable ranking across evaluation conditions suggests the misalignment reflects an intrinsic property of the learned latent geometry, not an artifact of a particular test set.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The reported mIoU values (best ~0.22) fall below the 1/3 expected chance overlap for two random sets of equal size; this suggests the metric may be measuring systematic anti-alignment or a sampling artifact, and a random baseline should be reported before interpreting the absolute scores.
  • The authors never specify the exact CIEDE2000 threshold that defines the 'inside' of the human ellipse; different thresholds would change G and therefore the IoU, so the ground truth is underdetermined until that value is stated and validated against the original MacAdam/BFD data.
  • A sharper test would extract the orientation and eccentricity of the model's similarity ellipses and compare them directly to the psychophysical ellipses, decoupling shape from spatial extent; the paper lists this as future work but does not quantify it.
  • The polarized performance of language-supervised models suggests that within a single training paradigm, the choice of data and objective details can swing alignment from top to bottom; probing what drives that swing may reveal how perceptual geometry is (or is not) encoded.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 5 minor

Summary. The paper presents a large-scale benchmark of 50+ pretrained vision encoders (CNNs, ViTs, self-supervised, language-supervised, and VAE-based) probed for local chromatic discrimination behavior. Around 18 reference CIELAB colors and at four chroma levels (C=5,20,60,80), the authors define human discrimination regions from CIEDE2000 ellipses, sample 500 points per region (250 inside, 250 outside), embed each color patch with a frozen encoder, and take the 250 points with lowest cosine distance to the reference embedding as the model's predicted similar set. Alignment is quantified by mean intersection-over-union (mIoU, Eq. 1) between this predicted set and the human-defined set. The central empirical claim is that all models achieve weak alignment, with the best mIoU below 0.25, and therefore human-like chromatic thresholds do not emerge from current large-scale visual training objectives. Secondary claims concern training-paradigm trends: self-supervised models rank highest on average, language-supervised models are the most polarized, and scale does not help.

Significance. If the measurement were properly calibrated, this would be a valuable large-scale negative result for the vision-and-perception community, with practical implications for color-sensitive downstream tasks and for representation-learning objectives. The strength of the paper lies in its breadth: 50 models spanning many architectures and training paradigms, public pretrained weights, and a controlled stimulus protocol that connects classical color psychophysics to modern encoders. The authors also provide clear visualizations of local similarity fields. However, the central metric is not calibrated against a chance baseline, and the reported values fall below the level expected from random selection, which makes the main quantitative conclusion uninterpretable as stated. The secondary ranking claims also lack statistical support. The manuscript is therefore a useful exploratory study in need of substantial analytic revision.

major comments (3)
  1. [Evaluation Metrics, Eq. (1)] The mIoU metric is not chance-normalized. With |G|=|P|=250 over a fixed 500-point sample, a uniformly random P has expected intersection 125 and expected IoU = 125/(500−125) = 1/3. Every reported value, including the best mIoU≈0.22, is below this chance level. The paper interprets these numbers as 'weak alignment', but a score below chance indicates systematic anti-overlap between the model's 250 nearest cosine-distance points and the human CIEDE2000 ellipse. The authors must report the chance baseline, provide a chance-normalized metric (e.g., IoU minus expected IoU, or an adjusted Rand index), and explain why the nearest-neighbor sets avoid the human-defined regions. Without this, the central claim that human-like chromatic sensitivity 'does not emerge' is not quantitatively supported.
  2. [Stimuli: Reference Colors and Discrimination Regions] The paper states that the human discrimination region is 'derived from standard CIE discrimination data, with boundaries determined by the CIEDE2000 metric (ΔE00)', but it never specifies the CIEDE2000 threshold value (e.g., ΔE00 = 1.0, 2.3, or a JND-based value). The size and shape of the ground-truth ellipses depend directly on this threshold, and the mIoU values are highly sensitive to the resulting |G|. Please report the exact threshold and the source of the ellipse parameters, and state whether the threshold varies across reference colors or chroma levels. This is essential for reproducibility and for interpreting the absolute mIoU numbers.
  3. [Results and Figures 2–3] No variance or significance testing is reported. Each model receives a single mIoU value (averaged over 18 ellipses × 4 chroma levels), and the ranking of training paradigms (e.g., 'self-supervised encoders consistently outperform supervised ones') is asserted from point estimates. Given the compressed range of observed scores (roughly 0.05–0.22), these differences may be within noise. Provide confidence intervals (e.g., bootstrap over ellipses/chroma levels) and, where family-level claims are made, appropriate statistical tests. This is particularly important because the mIoU scale is not calibrated (see above).
minor comments (5)
  1. [Evaluation Metrics, Eq. (1)] Eq. (1) defines IoU for a single ellipse, while the paper reports mIoU as the mean. Clarify the averaging procedure (over ellipses, chroma levels, or both) in the main text rather than only in the caption of Figure 2.
  2. [Figure 1 caption] The caption says 'chromaticity 60' while the text uses 'chroma level C=60'. Use consistent terminology and notation for chroma (C*_ab) throughout.
  3. [Stimuli] The paper states 'Luminance is held constant at L=50 cd/m²', but the stimuli are described in CIELAB with L* values elsewhere. L* is a perceptual lightness coordinate, not luminance in cd/m². Please correct the units or clarify which quantity is fixed.
  4. [Discussion/Limitations] The limitations paragraph acknowledges that cosine distance in frozen feature spaces 'may not fully capture nonlinear decoding processes', but it does not mention the chance-level issue of Eq. (1). Add a specific discussion of the chance baseline and the anti-alignment result.
  5. [Supplementary Table 1] Model names in the appendix table contain missing punctuation and spaces (e.g., 'senet154.gluonin1k', 'haloregnetzb.ra3in1k'). Format the table for readability and consistency with the model zoo names used in Figure 2.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the encoder benchmark is externally grounded in CIEDE2000/MacAdam discrimination data, and no prediction is fitted to or derived from the target.

full rationale

The paper's central claim is an empirical negative result: across more than 50 pretrained vision encoders, model-derived chromatic similarity regions overlap human CIEDE2000-defined discrimination ellipses only weakly (best mIoU ≈ 0.22). The human ground truth is an external psychophysical standard (MacAdam, BFD, RIT-DuPont, CIEDE2000), and the model predictions are simply 250 nearest neighbors in frozen feature space, selected without any fitting to the human regions. No parameter is estimated from the human labels, no equation reduces the prediction to the ground-truth construction, and no uniqueness theorem or ansatz from the authors' prior work is invoked to force the conclusion. The self-citations [9]–[11] appear only as motivation that foundation models can exhibit some human-like visual behavior; they do not supply the measured mIoU values or the benchmark design. The limitations section flags restricted stimuli and cosine-distance evaluation, but it does not reveal a circular dependency. The skeptical concern about mIoU being uncalibrated relative to a chance baseline (expected IoU ≈ 1/3 for random 250-point sets) is a validity/interpretation issue, not circularity, because the model-selected set P is not constructed from the human region G. Thus the derivation chain is self-contained: external benchmark in, measured alignment out, with no load-bearing self-reference.

Assumptions & free parameters 3 free parameters · 4 assumptions · 0 invented entities

The paper introduces no new entities or fitted physical constants. Its evidential burden lies entirely in the choice of ground-truth definition and in the calibration of the region-overlap metric.

free parameters (3)
  • CIEDE2000 discrimination threshold (ΔE00) = not stated (an implied constant, e.g., 1.0)
    Defines the boundary of the human ground-truth set G and therefore every IoU value; never specified in the Methodology.
  • Sampling-region radius factor = 2.0 × ellipse semi-major axis
    Hand-chosen; determines the pool of outside points and thus the achievable IoU range.
  • Balanced sample sizes (250 inside / 250 outside) = 250/250 out of 500
    Sets the random-chance IoU to 1/3, which is central to calibrating the metric; the paper does not report this baseline.
assumptions (4)
  • domain assumption CIEDE2000 ΔE00 regions are a valid proxy for human chromatic discrimination thresholds
    Ground-truth ellipses in Figure 1 are asserted to be derived from CIEDE2000; no psychophysical threshold measurement is used directly (Methodology: Reference Colors and Discrimination Regions).
  • domain assumption Cosine distance in the frozen latent space reflects the model's chromatic similarity geometry
    The Evaluation Protocol uses cosine distance between embeddings; the authors acknowledge in Limitations that downstream nonlinear decoding may differ.
  • domain assumption Uniform flat patches are adequate stimuli for probing chromatic sensitivity
    All stimuli are uniform patches at fixed luminance; encoders trained on natural images may be off-distribution, as acknowledged in Limitations.
  • domain assumption The 18 reference hues and four chroma levels are representative of color space
    IoU varies strongly with hue and chroma (Figure 3), so the choice of reference colors affects the global ranking.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Do Vision Encoders Exhibit Human-like Color Thresholds?." pith.science (2026). https://pith.science/paper/H64DQ6YV

@misc{pith2026260716540,
  author       = {Pith},
  title        = {Pith review of: Do Vision Encoders Exhibit Human-like Color Thresholds?},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H64DQ6YV}},
  note         = {Machine review of arXiv:2607.16540}
}
read the original abstract

Understanding and characterizing human color perception is a longstanding research goal. One of the most traditional approaches is looking for the human color discrimination thresholds, the minimum chromatic differences perceptible to human observers. In recent years, deep neural networks have become the standard networks for computer vision tasks. In particular, deep vision encoders, foundation models trained on large-scale visual data, map images into latent feature representations. Despite the widespread use of deep vision encoders, few studies have investigated whether their internal representations exhibit human-like discrimination thresholds. In this work, we present a large-scale exploratory study probing the chromatic sensitivity of more than 50 pretrained vision encoders, including convolutional networks and vision transformers, against human discrimination thresholds. Using controlled chromatic stimuli at multiple chroma levels, we compare model-derived chromatic discrimination thresholds with human discrimination ellipses through a region-overlap metric (mIoU). Our analysis reveals generally weak alignment between model representations and human perceptual thresholds across all model families, with the best mIoU < 0.25. Moreover, we find that self-supervised encoders consistently outperform supervised ones, while language-supervised models show the most polarized behavior, occupying both the top and bottom of the ranking. These findings suggest that human-like chromatic sensitivity does not emerge naturally from current large-scale visual training objectives for any of the analyzed architectures.

Figures

Figures reproduced from arXiv: 2607.16540 by the authors.

Figure 1
Figure 1. Chromatic similarity fields for representative vision encoders across multiple reference colors at chromaticity 60. The values shown above each reference color chip correspond to its CIELAB (a*, b*). Each column corresponds to a different reference hue, while rows correspond to different model families. Heatmaps visualize cosine distance in the local latent neighborhood around each reference color, with ellipses ove… view at source ↗
Figure 2
Figure 2. Mean intersection over union (mIoU) between model and human chromatic discrimination thresholds for all evaluated vision encoders. Models are colored according to training paradigm: supervised, self-supervised, language-supervised, and unsupervised (VAE-based) goals. Higher mIoU indicates stronger alignment between the model and human discrimination thresholds. Across the entire benchmark, alignment with human per￾c… view at source ↗
Figure 3
Figure 3. Intersection over union (IoU) scores across hue and chroma levels for representative vision encoders. Each subplot corresponds to a different chroma level, while curves show IoU variation across hue ellipses. The results reveal strong dependence on both chromatic location and saturation regime. crimination thresholds is not governed by architecture type or parameter count. However, the superiority of self-supervised… view at source ↗

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

23 extracted references · 4 linked inside Pith

  1. [1]

    Visual sensitivities to color differences in day- light.Journal of the Optical Society of America, 32(5):247–274, 1942

    David L MacAdam. Visual sensitivities to color differences in day- light.Journal of the Optical Society of America, 32(5):247–274, 1942

    1942

  2. [2]

    Chromaticity-discrimination ellipses for surface colours.Color Research & Application, 11(1):25–42, 1986

    Ming Ronnier Luo and Bob Rigg. Chromaticity-discrimination ellipses for surface colours.Color Research & Application, 11(1):25–42, 1986

    1986

  3. [3]

    Berns, David H

    Roy S. Berns, David H. Alman, Lisa Reniff, G. D. Snyder, and M. R. Balonon-Rosen. Visual determination of suprathreshold color-difference tolerances using probit analysis.Color Research & Application, 16(5):297–316, 1991

    1991

  4. [4]

    Colorimetry

    Commission Internationale de l’ ´Eclairage. Colorimetry. Techni- cal Report Publication No. 15, Central Bureau of the CIE, Vienna, Austria, 1976

    1976

  5. [5]

    Berns.Billmeyer and Saltzman’s Principles of Color Tech- nology

    Roy S. Berns.Billmeyer and Saltzman’s Principles of Color Tech- nology. John Wiley & Sons, Hoboken, NJ, 4th edition, 2019

    2019

  6. [6]

    F. J. J. Clarke, R. McDonald, and B. Rigg. Modification to the JPC79 colour-difference formula.Journal of the Society of Dyers and Colourists, 100:128–132, 1984

    1984

  7. [7]

    Industrial colour- difference evaluation

    Commission Internationale de l’ ´Eclairage. Industrial colour- difference evaluation. Technical Report Publication No. 116, Cen- tral Bureau of the CIE, Vienna, Austria, 1995

    1995

  8. [8]

    M Ronnier Luo, Guihua Cui, and Bryan Rigg. The development of the cie 2000 colour-difference formula: Ciede2000.Color Re- search & Application: Endorsed by Inter-Society Color Council, The Colour Group (Great Britain), Canadian Society for Color, Color Science Association of Japan, Dutch Society for the Study of Color, The Swedish Colour Centre Foundation...

    2000

Show all 23 references
  1. [9]

    Alejandro Parraga, Bart- lomiej Twardowski, Jesus Malo, Javier Vazquez-Corral, and Joost van den Weijer

    Alexandra Gomez-Villa, Kai Wang, C. Alejandro Parraga, Bart- lomiej Twardowski, Jesus Malo, Javier Vazquez-Corral, and Joost van den Weijer. The art of deception: Color visual illusions and diffusion models. InProceedings of the IEEE/CVF Conference on Computer Vision and Patte...

    2025

  2. [10]

    Color illusions also deceive cnns for low-level vision tasks: Analysis and implications.Vision Research, 176:156–174, 2020

    Alex Gomez-Villa, Adrian Mart ´ın, Javier Vazquez-Corral, Marcelo Bertalm´ıo, and Jes ´us Malo. Color illusions also deceive cnns for low-level vision tasks: Analysis and implications.Vision Research, 176:156–174, 2020

    2020

  3. [11]

    Brainard, Thomas S

    Jaykishan Patel, Alban Flachot, Javier Vazquez-Corral, David H. Brainard, Thomas S. A. Wallis, Marcus A. Brubaker, and Richard F. Murray. A deep convolutional neural network trained for lightness constancy is susceptible to lightness illusions.Journal of Vision, 26(3):14–14, 03 2026

    2026

  4. [12]

    Color encoding in latent space of stable diffusion models.arXiv preprint arXiv:2512.09477, 2025

    Guillem Arias, Ariadna Sol `a, Mart´ı Armengod, and Maria Vanrell. Color encoding in latent space of stable diffusion models.arXiv preprint arXiv:2512.09477, 2025

    2025

  5. [13]

    Yancheng Cai, Fei Yin, Dounia Hammou, and Rafal Mantiuk. Do computer vision foundation models learn the low-level characteris- tics of the human visual system? InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 20039–20048, June 2025

    2025

  6. [14]

    Nipun Ravindu Wickramanayaka and Masafumi Oizumi. System- atic comparison of color representations between humans and deep neural networks: towards predicting human color perception in a vast color space.bioRxiv, pages 2025–12, 2025

    2025

  7. [15]

    Learning transferable visual models from natural language supervision.arXiv preprint arXiv:2103.00020, 2021

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, et al. Learning transferable visual models from natural language supervision.arXiv preprint arXiv:2103.00020, 2021

    2021 arXiv

  8. [16]

    Openclip, 2021

    Gabriel Ilharco et al. Openclip, 2021. GitHub repository

    2021

  9. [17]

    Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    Maxime Oquab, Timoth ´ee Darcet, Theo Moutakanni, et al. Dinov2: Learning robust visual features without supervision.arXiv preprint arXiv:2304.07193, 2023

    2023 arXiv

  10. [18]

    Stable diffusion 3.https://stability.ai, 2024

    Stability AI. Stable diffusion 3.https://stability.ai, 2024

    2024

  11. [19]

    Zhang et al

    X. Zhang et al. Pe-clip: Perception-enhanced clip for image-text representation learning.arXiv preprint, 2023

    2023

  12. [20]

    Very deep convolu- tional networks for large-scale image recognition.arXiv preprint arXiv:1409.1556, 2014

    Karen Simonyan and Andrew Zisserman. Very deep convolu- tional networks for large-scale image recognition.arXiv preprint arXiv:1409.1556, 2014

    2014 arXiv

  13. [21]

    Gaurav Sharma, Wencheng Wu, and Edul N. Dalal. The ciede2000 color-difference formula: Implementation notes, supplementary test data, and mathematical observations.Color Research & Ap- plication, 30(1):21–30, 2005

    2005

  14. [22]

    A standard default color space for the internet - srgb, 1996

    IEC. A standard default color space for the internet - srgb, 1996. IEC 61966-2-1

    1996

  15. [23]

    An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020

    Alexey Dosovitskiy et al. An image is worth 16x16 words: Transformers for image recognition at scale.arXiv preprint arXiv:2010.11929, 2020. Technical appendices and supplementary material Evaluated models and their training paradigms. Model Training Paradigm senet154.gluonin1k...

    2010 arXiv

Pith tools

Reviewed August 1, 2026 · model on record in the stance chip above.