{"id":"80af3e1b-de36-4591-9709-571f1341c187","arxiv_id":"2502.02048","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":3.0,"correctness_risk":"high","formal_verification":"none","parameter_count":6,"one_line_summary":"Training a small supervised contrastive projection on frozen embeddings gives modest F1 gains for some downstream classifiers, but the paper's stated loss function is internally inconsistent.","lead":"The paper proposes a cheap way to adapt frozen AI embeddings to a new task by training a small contrastive projection on top of them, instead of fine-tuning the large model. Tests on clinical notes and movie data show some F1 gains, mainly for simple classifiers, but the paper contains an error in its core loss equation.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Equation (2) uses the same sigmoid term for positive and negative pairs, so the stated loss cannot separate classes; as written, the central method cannot produce the claimed improvements.","rationale":"The reader's verdict is REJECT, and my analysis supports that rejection, so no change to the verdict is needed. The reader's weakest_assumption focuses on possible cross-validation leakage, which is a real and substantial concern: Section 3.2 describes training the projection on the full dataset, while Section 4 claims out-of-sample F1 from 5-fold CV, and the paper never explicitly states that the contrastive projection is retrained inside each fold. That ambiguity could invalidate all reported improvements. However, I see an even more fundamental problem in the paper's own derivation: Eq. (2), as typeset, applies the same sigmoid term to positive and negative pairs, so the stated objective cannot encourage separation. This is the most load-bearing issue because it breaks the mechanism claimed for the method, not just the evaluation. The timing comparison in Section 4.3 is useful and the computational-resource claim is plausible, and the method could in principle work if Eq. (2) is a typographical error for a standard binary cross-entropy loss. But as written, the central claim that contrastive training produces task-specific embeddings with better class separation is unsupported. I therefore agree with the REJECT verdict, while noting that a corrected loss and a clarified fold protocol could make a revised version worth reconsidering.","tokens_in":21126,"tokens_out":2995,"duration_ms":33807,"concrete_test":"Re-implement the projection with the loss exactly as written in Eq. (2), using the hyperparameters in Table 1, on the diabetes dataset; then compute the average cosine similarity between projected embeddings of same-label versus different-label pairs on held-out data, and evaluate downstream F1. If Eq. (2) is the true training objective, different-label pairs should become more similar, not less, and the reported per-modality F1 of 76.3 should not reproduce. If the authors' implementation instead minimizes ℓ·log σ(sim) + (1−ℓ)·log(1−σ(sim)), then the paper must be corrected to that equation and the empirical results re-verified under that corrected objective.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing defect is in the loss definition, Eq. (2). For a pair with label ℓ=0, the loss term is log σ(g_j(u)^T g_j(v)/τ), not log(1 − σ(...)). Maximizing this term drives projected embeddings of different classes toward each other, which is exactly opposite to the stated goal of separating labels. The loss therefore cannot train a projection with the class-clustering behavior claimed in Section 3.1 and visualized in Figure 4, so the central claim that contrastive learning improves downstream F1 is not supported by the paper's own equations. This is not a minor sign convention: binary cross-entropy requires the negative-pair term to use log(1 − σ(...)), otherwise the objective has no discriminative signal and the reported per-modality gains from 58.3 to 76.3 (diabetes) and 56.2 to 76.4 (movies) have no mechanism in the described method. A second concern is the experimental protocol: Section 4 reports average out-of-sample F1 from 5-fold cross-validation, but Section 3.2 describes training the projection on the original dataset without specifying fold separation; if the projection was fit before splitting, test-fold labels would be visible and the gains would be leakage artifacts. The equation error is primary because it makes the method as described nonfunctional, regardless of the evaluation protocol.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a low-resource method for adapting frozen multimodal embeddings to a binary classification task: extract task-agnostic embeddings from pretrained models (e.g., ClinicalBERT, Qwen2+GTE, BERT, ViT), train a small nonlinear projection with a supervised contrastive objective, and then train a downstream classifier on the projected embeddings. The method is evaluated on a proprietary healthcare dataset (diabetes, hypertension) and the public mmimdb movie dataset, comparing unprojected embeddings, PCA projections, and contrastive projections with multiple downstream classifiers. The central claim is that this efficient adaptation improves F1 scores substantially (e.g., average F1 from 58.3 to 76.3 for diabetes and 56.2 to 76.4 for movies) with minimal computational overhead, as summarized in Table 2 and the timing benchmark in Table 6.","tokens_in":21391,"tokens_out":3978,"duration_ms":44897,"significance":"If the method worked as described, it would be a practically useful contribution: it is modality-agnostic, operates on frozen embeddings, and the timing comparison is informative (34 seconds on GPU versus 39,562 seconds for full fine-tuning, with F1 0.845 versus 0.909). The paper also has the merit of reporting results across several downstream classifiers and two datasets. However, the published objective in Eq. (2) is label-independent as written, and the experimental protocol does not specify whether the projection is trained inside each cross-validation fold. These two issues are load-bearing: the first removes the mechanism for the claimed class separation, and the second makes the reported out-of-sample gains uninterpretable. The paper ships no code and relies on a proprietary dataset, so these points cannot be checked externally. The contribution is therefore not supported by the manuscript in its current form.","major_comments":[{"comment":"In Eq. (2), both the positive and negative pair terms contain the same expression log sigma(g_j(u)^T g_j(v)/tau), multiplied by ell and (1-ell) respectively. These multipliers cancel, so the loss reduces to L_b^{(j)} = -sum log sigma(...), which is independent of the labels. Maximizing this objective drives all projected vectors to coincide; it has no mechanism to bring same-label pairs together and push different-label pairs apart. Consequently, the class-clustering behavior claimed in Section 3.1 and visualized in Figure 4 cannot be produced by the method as written, and the reported gains in Table 2 (e.g., 58.3 to 76.3 for diabetes, 56.2 to 76.4 for movies) have no basis in the stated objective. If this is a typographical error, the correct negative-pair term log(1 - sigma(...)) must be stated explicitly and the experiments rerun with that loss; a code release would resolve the ambiguity.","section":"Section 3.2, Eq. (2)"},{"comment":"The evaluation reports 'average out-of-sample F1 score using 5-fold cross validation' but does not state whether the contrastive projection is trained inside each training fold. Section 3.2 describes training the projection on 'the original dataset' without mentioning fold boundaries, and the figures and tables do not clarify the fit/split order. If the projection is fit on the full dataset before splitting, test-fold labels are visible during projection training, and the reported F1 improvements would be leakage artifacts rather than genuine out-of-sample gains. The authors must specify the exact protocol, ideally with the projection retrained per fold, before any of the reported numbers can be interpreted.","section":"Section 4 vs Section 3.2"},{"comment":"The headline claim of 'significant performance improvements across various downstream tasks' is based on averages over classifiers and feature extractors in Table 2, but the detailed tables show that the gains are concentrated in particular model/extractor combinations (chiefly CART, SVC, RF, and XGB), while LR and MLP frequently lose F1 relative to the unprojected baseline (e.g., Table 3: CBERT LR -2.7, GQW MLP -3.5; Table 4: CBERT MLP -4.1). The paper should report paired comparisons across folds and per-model statistical tests, and the abstract and conclusion should be moderated to the settings where the improvement is consistent. This issue is secondary to Eq. (2) but is relevant to the strength of the central claim.","section":"Tables 3-5 and Table 12"}],"minor_comments":[{"comment":"There are numerous typographical errors, including 'Constrastive' in the title, 'drwabacks' in Section 3.1, 'projectiontion' in Section 4, 'feairness' and 'showin' in Section 4.1, and 'constraints' in the abstract; these should be corrected.","section":"Throughout"},{"comment":"Several references are missing years or venues, such as 'Chen et al.', 'Radford et al.', 'He et al.', 'Caron et al.', and 'Zhang et al.'; the bibliography should be completed to journal style.","section":"References"},{"comment":"The hyperparameters in Table 1 (learning rate, epochs, projection size, etc.) are presented as fixed choices, but no sensitivity analysis or selection procedure is described; the downstream classifiers are said to use 'default parameters' without listing software versions or default specifications.","section":"Section 4, Table 1"},{"comment":"The t-SNE visualization is not reproducible as reported because no perplexity, learning rate, or random seed is given; more importantly, the claimed separation cannot be true under Eq. (2) as written.","section":"Section 4.1, Figure 4"}],"recommendation":"reject","confidential_remarks":"The Eq. (2) error is disqualifying as written, and the missing fit/split specification prevents the experimental numbers from being interpreted as out-of-sample. The proprietary HHC dataset and the absence of code make external verification impossible; if a resubmission is considered, the editor should require code, a corrected loss, and a per-fold projection protocol."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Read through it. The short version: as written, the central loss in Eq. (2) is a no-op—the same sigmoid term is used for positive and negative pairs, so the objective is just log σ(sim) for every pair and the labels never enter. The authors say it's binary cross-entropy, but it isn't. That means the claimed class separation in Figure 4 and the F1 gains in Tables 2–5 have no mechanism in the described method. This is a load-bearing flaw, not a sign typo you can wave off.\n\nThe second issue is the CV protocol. Section 3.2 says the projection is trained on the original dataset; Section 4 reports 'average out-of-sample F1 using 5-fold cross validation' but never states whether the projection is refit inside each fold. If it's fit before splitting, test-fold labels are visible during projection training and the gains are leakage. The paper needs to state this explicitly.\n\nWhat's genuinely useful here: the framing—train a small task-specific nonlinear projection on top of frozen embeddings—is a sensible low-resource recipe, and the timing comparison (34 seconds vs ~11 hours for full fine-tuning, F1 0.845 vs 0.909) is a genuinely informative datapoint for resource-constrained settings. The per-modality versus single-projection comparison is also worth having. The prose is clear.\n\nBut even if Eq. (2) is fixed to the standard BCE, the contribution is thin. This is supervised contrastive learning (SupCon and related work) applied to a projection head on frozen embeddings, and the paper doesn't cite SupCon. The experimental gains are concentrated in weak baselines—SVC with F1 near zero—and the 'best F1 across models' table shows the contrastive projection does not beat the unprojected baseline for the best model in hypertension and movies (35.5 vs 32.5; 77.9 vs 77.4). So the abstract's 'significant performance improvements across various downstream tasks' is oversold.\n\nWho's this for? Practitioners in low-resource settings who want a cheap adaptation trick would be the audience, assuming the method is actually the one described. A serious referee should engage: the idea is worth testing and the computational claims are relevant, but the current version does not support its own conclusions. I'd send it to review with a firm request to fix Eq. (2), clarify the CV protocol, and temper the claims. Not a desk reject—the kernel is fine—but as-is it's not close to acceptable.","headline":"The method as written can't work—Eq. (2) discards the labels—and the CV protocol is under-specified; the kernel is worth a reviewer's time but the current claims don't follow.","tokens_in":21921,"tokens_out":3133,"would_cite":false,"duration_ms":30066,"reading_group":"maybe","serious_thinker":"no","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A small contrastive projection on frozen multimodal embeddings can recover most of fine-tuning's F1 gain at a tiny fraction of the compute, e.g. raising average clinical F1 from 58.3 to 76.3 in the reported experiments.","keywords":["contrastive learning","multimodal embeddings","frozen foundation models","domain adaptation","clinical natural language processing","embedding projection","efficient fine-tuning","binary classification"],"falsifier":"Re-run the medical experiments with the contrastive projection fit separately on each of the five training folds and evaluated on the held-out fold, and compare the average out-of-sample F1 to Table 2's 76.3 for diabetes; if it drops back toward the 58.3 unprojected baseline, the reported gains are leakage rather than adaptation.","tokens_in":20892,"feed_emoji":"⚡","tokens_out":11925,"duration_ms":107312,"temperature":0.7,"pith_summary":"The paper sets out to make foundation-model embeddings usable where full fine-tuning is too expensive, such as hospital settings. It freezes large language and vision models, extracts embeddings once, and trains a small nonlinear projection with a supervised contrastive objective that pulls same-label embeddings together and pushes different-label ones apart; the projected low-dimensional vectors go into an ordinary classifier. In the reported experiments on clinical notes (diabetes, hypertension) and a movie genre dataset, this per-modality projection raises average out-of-sample F1 from 58.3 to 76.3 (diabetes), 19.8 to 27.6 (hypertension), and 56.2 to 76.4 (movies). The timing benchmark puts the contrastive route at 34 seconds on a GPU versus 39,562 seconds for full fine-tuning, with F1 0.845 versus 0.909. If these numbers hold, the method offers a practical middle path between generic embeddings and full fine-tuning.","feed_headline":"A 34-second projection captures most of fine-tuning's F1 gains","feed_subtitle":"Per-modality contrastive projection raises average clinical F1 from 58.3 to 76.3 using 3 MB, not 33 GB, of GPU memory.","key_machinery":"The load-bearing object is a small feed-forward ReLU network with one hidden layer and output size 128, applied per modality (or once to the concatenated vector), trained with the contrastive loss in Eq. (2): a binary cross-entropy with sigmoid applied to the temperature-scaled similarity $g_j(u)^\\top g_j(v)/\\tau$, with $\\tau = 0.1$, where the pair label is 1 if the two inputs share the downstream binary label and 0 otherwise. This reshapes the frozen embedding space so that simple decision-boundary models find clean separation, which the paper argues is why CART and SVC show the largest gains, and it doubles as label-aware dimensionality reduction, which is why it outperforms PCA at the same projected size.","core_discovery":"On the paper's own terms, the central discovery is that a task-specific embedding subspace, learned by a small ReLU projection on frozen embeddings using only labels, carries most of the value that full backpropagation fine-tuning would extract. The projection is trained per modality (or once on the concatenated embeddings) with a SimCLR-style contrastive loss, so same-class examples cluster and different-class examples separate, and the paper reports average out-of-sample F1 gains of 18.0 points for diabetes, 7.8 points for hypertension, and 20.2 points for movies over unprojected embeddings, with larger per-model gains in some cells. The cost comparison on the diabetes task is the sharpest statement: contrastive projection reaches F1 0.845 in 34 seconds with 3 MB VRAM, while full ClinicalBERT fine-tuning reaches 0.909 in 39,562 seconds (about 11 hours) with more than 32 GB VRAM.","pith_inferences":["The paper does not test it, but the small projection could be retrained incrementally as new labeled cases arrive in deployment, enabling continual adaptation without ever fine-tuning the foundation model.","The largest gains concentrate in classifiers with simple decision boundaries, which suggests the projection mainly linearizes the label geometry; a natural follow-up is to test whether the learned projection transfers to other classifiers, tasks, or patient cohorts.","Although only binary tasks are shown, the same-label/different-label pair objective extends directly to multi-class and multi-label settings, so the machinery should apply to richer clinical classification problems.","The hypertension results are the least dramatic (F1 19.8 to 27.6 with a low baseline), so the method's value on highly imbalanced conditions is the least established; class-balanced contrastive pairs are an obvious untested variant."],"forward_implications":["If the results hold, a hospital or other compute-limited site can adapt frozen clinical embeddings to a new binary condition in about a minute on CPU, with no GPU required.","Because the foundation models run once and their embeddings are stored, adapting to a new task or adding a new modality requires retraining only the small projections, not rerunning the large models.","Per-modality projection dominates single projection and PCA in the averaged results, so the paper's design choice of reshaping each modality independently is itself a claimed contribution.","Full fine-tuning still wins on F1 (0.909 vs 0.845), so the claim is not replacement of fine-tuning; it is a resource-constrained alternative that trades about 6 F1 points for three orders of magnitude less compute and memory."],"supporting_citations":[{"why":"Supplies the SimCLR contrastive loss that the paper adapts to supervised same-label/different-label pairs.","marker":"Chen et al."},{"why":"Provides ClinicalBERT, the clinical text feature extractor used for the EHR embeddings in the medical experiments.","marker":"Huang et al., 2019"},{"why":"Provides Qwen2, the 7B LLM used as the second text feature extractor for clinical notes.","marker":"Yang et al., 2024"},{"why":"GTE pretraining that adapts Qwen2 into the embedding model used to generate text embeddings.","marker":"Li et al., 2023"},{"why":"Provides BERT, the text encoder for the movie plot and tabular modality in the mmimdb experiment.","marker":"Devlin et al., 2019"},{"why":"Provides ViT-16, the pretrained image encoder for movie posters.","marker":"Dosovitskiy et al., 2021"},{"why":"Introduces mmimdb, the multimodal movie dataset on which the comedy/drama classification is evaluated.","marker":"Arevalo et al., 2017"},{"why":"Establishes the frozen-embedding multimodal prediction pipeline (concatenate and classify) that serves as the unprojected baseline.","marker":"Soenksen et al., 2022"},{"why":"Supports converting structured or tabular data (ICD codes, movie metadata) into text before embedding, a preprocessing step both datasets rely on.","marker":"Carballo et al., 2023"}],"fun_headline_variants":["34-second contrastive projection recovers ~93% of fine-tuning F1","Small projection, big gains: contrastive learning adapts embeddings in 3MB","From 11 hours to 34 seconds: efficient multimodal embedding adaptation","Contrastive projection: fine-tuning-level F1 with minimal compute"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The reported out-of-sample numbers assume the contrastive projection is trained inside each cross-validation fold; if the projection is fit on the entire dataset before the 5-fold split, test labels would be visible during projection training and the F1 gains would not measure true generalization.","fun_headline_variants_meta":{"raw":{"variants":["34-second contrastive projection recovers ~93% of fine-tuning F1","Small projection, big gains: contrastive learning adapts embeddings in 3MB","From 11 hours to 34 seconds: efficient multimodal embedding adaptation","Contrastive projection: fine-tuning-level F1 with minimal compute"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00042,"raw_usage":{"total_tokens":2186,"prompt_tokens":998,"completion_tokens":1188,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":614,"completion_tokens_details":{"reasoning_tokens":1108}},"tokens_in":614,"tokens_out":1188,"duration_ms":11766,"temperature":1.0,"reasoning_tokens":1108,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-09T13:32:05.983006+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Re-run the medical experiments with the contrastive projection fit separately on each of the five training folds and evaluated on the held-out fold, and compare the average out-of-sample F1 to Table 2's 76.3 for diabetes; if it drops back toward the 58.3 unprojected baseline, the reported gains are leakage rather than adaptation.","supporting_citations":[],"review_version":1}