{"id":"1e53da2c-71b4-4710-ab51-11a9fcfd631e","arxiv_id":"2505.06934","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":5.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"Whitening CLIP embeddings makes their squared norm a likelihood-like score that ranks images and captions by how typical they are under a Gaussian approximation.","lead":"The authors show that after a simple linear rescaling, CLIP's image and text embeddings behave like a standard bell-shaped distribution, so the distance from the center scores how typical an image or caption is. This gives a fast, training-free way to detect artifacts, domain shifts, and unusual text without training new models.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Joint normality of whitened CLIP embeddings is inferred from marginal tests and in-sample covariance only; Eq. 6 therefore lacks a validated multivariate basis.","rationale":"Reader's weakest assumption is exactly the joint-normality gap; I agree. The paper deserves credit for a training-free method, code release, cross-dataset Tab. 3, and explicit acknowledgment that the model is an approximation. The applications (artifact detection, domain shift, SLERP) may work as ranking heuristics regardless of exact calibration. But those demonstrations do not require Eq. 6 to be a true density, whereas the central claim does. The missing Jacobian term is a smaller, separate issue: within a fixed whitening matrix it cancels, but the paper states a likelihood for images/captions, not just for y. The proposed test is decisive: if held-out multivariate normality and chi-square calibration pass, the conditional concern is resolved; if not, the paper should be revised to describe W-CLIP as a heuristic likelihood surrogate, i.e., a Mahalanobis score, and the conditional verdict should be revisited. Because the concern is unresolved and exactly matches the reader's condition, the reader's CONDITIONAL verdict should stand.","tokens_in":17435,"tokens_out":5146,"duration_ms":53018,"concrete_test":"Split MS-COCO validation into a whitening fit split (e.g., 4,000) and a held-out split (1,000), and also hold out Flickr8k. Fit W on the fit split; on held-out whitened embeddings run Mardia's multivariate skewness/kurtosis (or Henze-Zirkler) and compare the empirical distribution of ||y||^2 to chi-square_768 via quantile-quantile and Kolmogorov-Smirnov. If the multivariate test rejects or the Q-Q tails deviate on held-out data, Eq. 6 is not a calibrated log-likelihood and should be presented only as a heuristic anomaly score.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim (Eq. 6) treats the whitened embedding y = W(x - mu) as if its density were standard normal, so the negative log-likelihood is d/2 log(2pi) + 1/2 ||y||^2. The evidence in Sec. 3.3 does not establish this. Anderson-Darling and D'Agostino-Pearson are applied per feature, and covariance diagonality is checked on the same 5,000 MS-COCO validation vectors used to fit W. Marginal Gaussianity plus zero correlation is not sufficient for joint Gaussianity; the text in Sec. 3.3 ('in normal distribution, non-correlation is a sufficient condition for independence') assumes the joint normality it is trying to justify. Consequently the norm in Eq. (6) need not follow the assumed chi distribution. Tab. 2 already hints at this for text: the empirical norm std dev differs from the theoretical chi_d value by 13.24%, and no goodness-of-fit for the full vector norm is reported. This is load-bearing because the method's probabilistic interpretation, and any use as a likelihood rather than a Mahalanobis rank, depends on y being jointly N(0,I). Secondary: for raw image/caption density, the invertible change of variables introduces a log|det W| term omitted from Eq. 6; constant within a fixed W, it matters for cross-matrix or cross-modality comparisons.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Whitened CLIP (W-CLIP), an invertible linear transformation of CLIP image and text embeddings via PCA whitening. Under the assumption that the whitened embeddings are jointly standard normal, the log-likelihood of an embedding is approximated as -1/2(d log(2π) + ||y||^2) (Eq. 6). The authors validate per-feature normality with Anderson-Darling and D'Agostino-Pearson tests, argue that whitening yields i.i.d. features, and demonstrate applications including artifact detection, domain-shift ranking, generation-bias measurement, and full-circle SLERP interpolation. They also compare their text likelihood scores with language-model NLLs and perform cross-dataset and cross-CLIP-model ablations.","tokens_in":17728,"tokens_out":6460,"duration_ms":62786,"significance":"If the central normality assumption is justified, W-CLIP offers a fast, training-free, and invertible likelihood surrogate for CLIP latents, with a clean derivation and a wide range of downstream uses. The paper's strengths include a reproducible code release, a principled derivation from the Gaussian assumption to Eq. (6), cross-dataset (Flickr8k) and cross-model (ViT-B/32) ablations, and a clear set of falsifiable experimental probes (artifact detection, ImageNet-C/R shifts, generation bias). The probabilistic claim is, however, the load-bearing component: Eq. (6) is only a likelihood if the whitened embeddings are jointly standard normal. The current validation establishes, at most, approximate marginal Gaussianity and enforced decorrelation in-sample; it does not establish joint normality, and an omitted Jacobian term further muddies the interpretation as a likelihood of the original embeddings. These gaps are fixable with additional multivariate tests and a clarifying statement, so the paper merits a major revision rather than rejection.","major_comments":[{"comment":"The validation of the normal-distribution assumption is performed per feature on the same 5,000 MS-COCO validation embeddings used to compute the whitening matrix W. Since W is constructed to make the sample mean zero, the sample variance unity, and the sample covariance the identity on this same set, the covariance diagonality reported in Fig. 4(b) and the zero-mean/unit-variance claims in Fig. 4(d) are enforced by construction and do not constitute evidence about the population distribution. The Anderson-Darling and D'Agostino-Pearson tests are applied to individual coordinates and cannot detect non-Gaussian dependence among coordinates; the statement in Sec. 3.3 that 'in normal distribution, non-correlation is a sufficient condition for independence' assumes joint normality, which is precisely the property needing validation. As a result, the distribution of ||y||^2 need not be chi-squared, so Eq. (6) is not established as a log-likelihood. This is load-bearing because the entire probabilistic interpretation, the chi-distribution comparison in Tab. 2, and the use of the norm as a probability score depend on y being jointly N(0,I). The 13.24% relative deviation of the text norm standard deviation in Tab. 2 already signals a departure. Please add a direct test of joint normality or of the norm/chi-squared fit, and perform it on a held-out set with W computed on a separate training set (e.g., a 2k/3k split).","section":"Sec. 3.3, Eq. (6)"},{"comment":"Equation (6) is the log-density of the whitened vector y under a standard normal model, as Eq. (5) makes clear. If the paper claims to provide the likelihood of a raw CLIP embedding or of an image/caption, the invertible change of variables y = W(x - μ) introduces an additive log|det W| term that is missing. For a fixed W this constant cancels in relative comparisons, but it differs between the image and text whitening matrices, so the paper's image and text 'log-likelihood' scores are not directly comparable as likelihood values, and cross-modality comparisons (or any use in a setting with multiple W matrices) require the constant. Please state explicitly what random variable Eq. (6) describes and either include the Jacobian term or restrict the claims to within-space relative scores.","section":"Sec. 3.4, Eq. (6)"}],"minor_comments":[{"comment":"The sentence 'in normal distribution, non-correlation is a sufficient condition for independence' is only valid under joint normality; please rephrase to avoid appearing to assume the very property being tested.","section":"Sec. 3.3"},{"comment":"The whitening procedure replaces highly correlated features with random noise r ~ N(0,0.1), and the correlation threshold τ is a free parameter. Please report the chosen τ and provide a sensitivity analysis; the current statement that the impact is minimal is not backed by quantitative evidence.","section":"App. D.1, Alg. 1"},{"comment":"Table 3 is difficult to parse because the column values for the two modalities are interleaved without clear separation. Please reformat so each row has explicit Data T, Data W, Avg. AD, Avg. DP, and Correlation columns, one row per modality.","section":"Tab. 3"},{"comment":"The text contains a typo: 'a dog with a bottle of bear' should read 'a dog with a bottle of beer'.","section":"Sec. 4.3"},{"comment":"The Anderson-Darling formula is displayed with an awkward line break; please ensure the prefactor is typeset clearly as -n - (1/n) Σ ... so that the statistic is unambiguous.","section":"App. D.3, Eq. (12)"}],"recommendation":"major_revision","confidential_remarks":"The core issue is the gap between marginal normality tests and the joint-normality assumption needed for Eq. (6). I believe this is fixable with a norm-based or multivariate normality check on properly split data, and the paper otherwise contains interesting and potentially useful applications. I would also gently encourage the authors to be more cautious about the 'first direct likelihood computation' claim in the related-work section, since the novelty framing depends on a precise definition of 'likelihood' that the manuscript currently leaves ambiguous."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Punchline: this is a neat, honest little paper. The core observation — that after PCA whitening, CLIP embeddings look marginally normal, so squared norm becomes a cheap likelihood-ish score — is genuinely new in the CLIP literature, even though whitening and Mahalanobis distances are old tools. The paper is well written, releases code, and includes sensible cross-dataset and cross-architecture checks (Flickr8k, ViT-B/32) that go beyond the usual single-benchmark demo.\n\nThe soft spot is exactly where the reader put it: the step from marginal normality to joint normality is not made. They test per-feature Anderson-Darling and D'Agostino-Pearson on the same 5,000 COCO vectors used to fit the whitening matrix, so the zero mean, unit variance, and decorrelation are enforced by construction. The claim that 'in normal distribution, non-correlation is a sufficient condition for independence' assumes the joint normality it is supposed to justify. The paper's own Table 2 quietly shows the problem: text norm std dev is off by 13.24% from the chi_d prediction, and no full-vector norm goodness-of-fit is reported. So the probabilistic interpretation of Eq. (6) as an actual log-likelihood is not established.\n\nOn the other hand, the score itself is still useful. The ImageNet-C/A/R separations, the artifact detection, and the full-circle SLERP examples are all consistent with norm being a sensible outlier and semantic-drift measure, even if it is not a calibrated density. I also think the omitted log|det W| term is a minor issue for the applications shown, since W is fixed for a given whitening dataset; it only matters for absolute cross-modality or cross-dataset likelihood comparisons, which the paper doesn't really make.\n\nMy overall read: this deserves peer review and, after a revision that either adds a proper multivariate normality test (e.g., Mardia or Henze-Zirkler on a held-out set) or explicitly reframes the method as a 'score' rather than a 'likelihood', it should be published. It is the kind of simple, efficient tool that practitioners will pick up. The paper is not overclaiming in a malicious way, but the central claim needs either more evidence or a more modest title.","headline":"A useful and clearly-written heuristic for scoring CLIP embeddings, but the likelihood framing leans on an unvalidated joint-normality assumption that the paper's own norm statistics partially contradict.","tokens_in":18236,"tokens_out":3067,"would_cite":true,"duration_ms":29764,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Whitening CLIP embeddings turns their squared norm into a direct log-likelihood estimate for images and captions.","keywords":["Whitened CLIP","CLIP embeddings","likelihood estimation","whitening transform","multimodal representation","out-of-distribution detection","image artifacts","spherical interpolation"],"falsifier":"Take a held-out set of images never used to fit the whitening matrix, whiten their CLIP embeddings, and compare the empirical distribution of $\\|y\\|^2$ with the $\\chi^2_d$ distribution predicted by Eq. (6). A large tail excess or a mean norm far from $\\sqrt{d-\\tfrac{1}{2}}$ would show the likelihood surrogate does not hold outside the fitting set.","tokens_in":17248,"feed_emoji":"🖼️","tokens_out":11540,"duration_ms":98701,"temperature":0.7,"pith_summary":"The paper proposes Whitened CLIP (W-CLIP), a training-free linear whitening of the CLIP latent space, and argues that after this transform, image and caption embeddings behave approximately like standard normal vectors. If that approximation holds, the log-likelihood of an image or caption under CLIP's learned distribution is just $-\\frac{1}{2}(d\\log(2\\pi)+\\|y\\|^2)$, where $y$ is the whitened embedding, so a meaningful probability score costs only one matrix multiplication. The authors support the normality assumption with Anderson-Darling and D'Agostino-Pearson tests and show the resulting scores separate real images from generated images with artifacts, detect domain shifts such as ImageNet-C and ImageNet-R, reveal generation bias, and make full-circle interpolation between images possible in the whitened space. A reader should care because explicit likelihoods for images have been hard to obtain, and this offers a fast, semantic alternative that plugs into existing CLIP-based pipelines.","feed_headline":"Whiten CLIP and the embedding norm becomes a likelihood score","feed_subtitle":"A training-free linear transform turns CLIP's latent space into a probability model for images and captions.","key_machinery":"The carrying mechanism is PCA whitening: from a representative set of CLIP embeddings the paper builds the empirical covariance $\\Sigma$, diagonalizes it as $\\Sigma = V\\Lambda V^\\top$, and applies the invertible map $W = \\Lambda^{-1/2}V^\\top$ to centered embeddings, so $y = W(x-\\mu)$ has zero mean and identity covariance. Under the paper's normality claim, this reduces the probability model to the standard normal density, making log-likelihood a function of the squared Euclidean norm. The companion object is the chi distribution of the norm, whose mean $\\sqrt{d-\\tfrac{1}{2}}$ drives the thin-shell intuition used to stabilize iterative generation and to explain why nearly all mass sits near a sphere of radius $\\sqrt{d}$.","core_discovery":"The paper's central claim is that the CLIP embedding distribution, treated separately for images and for text, is approximately elliptical and becomes approximately isotropic standard normal after the invertible whitening map $y = \\Lambda^{-1/2}V^\\top(x-\\mu)$. Under that model the paper defines the log-likelihood surrogate $\\ell(x) = -\\frac{1}{2}(d\\log(2\\pi)+\\|y\\|^2)$ and identifies the norm $\\|y\\|$ with a chi distribution whose typical radius is $\\sqrt{d-\\tfrac{1}{2}}$. The authors state that to their knowledge this is the first direct computation of likelihood functions for images and text prompts under the CLIP-learned distribution; the experiments are presented as preliminary demonstrations of the properties and applications of these scores.","pith_inferences":["Editorial: the same whitening-plus-norm recipe should transfer to other contrastive vision-language embeddings; the paper tests two CLIP variants, so a held-out norm-distribution check on another model would test the generality.","Editorial: the thin-shell result implies a calibrated 'typical set' around radius $\\sqrt{d}$; a threshold on normalized norm deviation might be more stable across datasets than raw likelihood.","Editorial: because image and text whitening matrices are fit separately, W-CLIP likelihoods are not directly comparable across modalities; a cross-modal calibration step would be needed before using the score to compare an image with its caption.","Editorial: the moderate correlation with language-model likelihoods (0.33-0.48) suggests W-CLIP carries complementary information, so a combined score could be more robust to text type and length than either measure alone."],"forward_implications":["W-CLIP can rank real images above generated images with artifacts, giving a fast generated-image detector.","Corrupted ImageNet-C images and stylized ImageNet-R renditions receive lower likelihoods than clean ImageNet, with severity moving the norm histogram further away from the in-distribution set.","Iterative UnCLIP re-encoding drifts into noise, while normalizing each whitened embedding to radius $\\sqrt{d}$ keeps outputs within the natural image space, exposing a likelihood bias of the generator.","Full-circle SLERP in W-CLIP keeps interpolated and extrapolated images natural and semantically meaningful, whereas raw-CLIP full-circle SLERP collapses into structured noise for off-segment angles.","Caption likelihoods decrease when captions become more specific or contain grammatical errors, and W-CLIP separates captions from general text more sharply than language-model likelihoods."],"supporting_citations":[{"why":"Supplies the CLIP model whose image and text latent spaces are whitened and scored.","marker":"Radford et al., 2021"},{"why":"Provides the MS-COCO validation set used to fit the whitening matrix and to run normality tests.","marker":"Lin et al., 2014"},{"why":"Supplies the Anderson-Darling test used to validate that whitened features approximate a normal distribution.","marker":"Anderson & Darling, 1954"},{"why":"Supplies the D'Agostino-Pearson test used as the second normality check.","marker":"D'Agostino & Pearson, 1973"},{"why":"Establishes the modality gap that justifies whitening and scoring images and captions separately.","marker":"Liang et al., 2022"},{"why":"Documents the double-ellipsoid geometry of CLIP embeddings, the structure the whitening transform removes.","marker":"Levi & Gilboa, 2025"},{"why":"Supplies the SynArtifact dataset of generated images with artifacts used in the likelihood comparison.","marker":"Cao et al., 2024"},{"why":"Provides ImageNet-C corruptions used to show likelihood drops under domain shift.","marker":"Hendrycks & Dietterich, 2019"},{"why":"Provides ImageNet-R renditions used to show style-based likelihood shifts.","marker":"Hendrycks et al., 2021a"},{"why":"Supplies the UnCLIP generator and the original SLERP procedure that full-circle SLERP extends.","marker":"Ramesh et al., 2022"}],"fun_headline_variants":["Whitened CLIP turns embedding norm into a likelihood score","CLIP embeddings whitened to give image and caption likelihoods","Training-free whitening makes CLIP a likelihood model","CLIP latent space whitened to estimate likelihoods","Norm in whitened CLIP space approximates log-likelihood"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the whitened embeddings are jointly standard normal, not merely coordinate-by-coordinate normal and uncorrelated; the paper's checks are per-feature and are run on the same validation set used to fit the whitening transform.","fun_headline_variants_meta":{"raw":{"variants":["Whitened CLIP turns embedding norm into a likelihood score","CLIP embeddings whitened to give image and caption likelihoods","Training-free whitening makes CLIP a likelihood model","CLIP latent space whitened to estimate likelihoods","Norm in whitened CLIP space approximates log-likelihood"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000496,"raw_usage":{"total_tokens":2392,"prompt_tokens":863,"completion_tokens":1529,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":479,"completion_tokens_details":{"reasoning_tokens":1457}},"tokens_in":479,"tokens_out":1529,"duration_ms":9066,"temperature":1.0,"reasoning_tokens":1457,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:29:07.595774+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a held-out set of images never used to fit the whitening matrix, whiten their CLIP embeddings, and compare the empirical distribution of $\\|y\\|^2$ with the $\\chi^2_d$ distribution predicted by Eq. (6). A large tail excess or a mean norm far from $\\sqrt{d-\\tfrac{1}{2}}$ would show the likelihood surrogate does not hold outside the fitting set.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the Anderson-Darling test used to validate that whitened features approximate a normal distribution."},{"cited_title":"and Pearson, E","cited_arxiv_id":null,"evidence_quote":"Supplies the D'Agostino-Pearson test used as the second normality check."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Documents the double-ellipsoid geometry of CLIP embeddings, the structure the whitening transform removes."}],"review_version":1}