{"id":"6cef259a-b9ed-44e0-b165-f25d969c465d","arxiv_id":"2507.14828","paper_version":1,"verdict":"REJECT","confidence":"MODERATE","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"An adaptive margin added to InfoNCE improves time series clustering metrics but hurts linear-probe classification, exposing a disconnect between clustering scores and downstream utility.","lead":"Researchers added an adaptive margin to a contrastive learning loss for time series and found it creates tight, well-separated clusters that actually classify worse. The result is a warning that common clustering scores can overstate how useful learned representations really are.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Margin=5 against cosine similarity in [-1,1] makes the squared margin penalty always active, so the proposed adaptive eMargin mechanism never operates as described and the reported cluster geometry may be an artifact.","rationale":"The reader identified the margin/similarity-scale incompatibility as the weakest assumption, and I agree. Eq. 2 plus margin=5 means the hinge term is strictly positive for every Y=1 pair because cosine similarity is bounded above by 1. This makes the 'adaptive' part of the loss non-functional: the margin never saturates, and all dissimilar pairs receive a large, monotonically increasing penalty based on a target of 5 that is unreachable. The paper's own Section 3.3 justification ('If M is already larger than the margin, no penalty is applied') is factually false under the chosen hyperparameters. This is an internal inconsistency, not merely a disagreement with community practice. It undermines the attribution of the clustering results to adaptive margin-based separation, and therefore the central negative claim—that adaptive margin inflates cluster metrics while degrading downstream utility—is not established by the reported experiments. The missing InfoNCE-only baseline compounds the issue by leaving open the possibility that the observed DBI/Silhouette improvements come from InfoNCE itself or from the constant repulsion, not from adaptivity. I would still give credit for the honest reporting of poor downstream performance, the public code, and the t-SNE observations, but these do not rescue the central claim as stated. The path to a conditional accept would be a corrected margin value within scope, a proper InfoNCE baseline, and a quantitative clustering-evaluation protocol that does not rely solely on metrics that can be inflated by degenerate geometry.","tokens_in":9646,"tokens_out":2438,"duration_ms":30689,"concrete_test":"Re-run the HARTH and ECG experiments (same architecture, threshold 0.4, three seeds) with margin values within the cosine range, e.g., margin ∈ {0.2, 0.5, 0.8}, plus a plain InfoNCE baseline with no margin. Report DBI, Silhouette, and linear-probe accuracy/F1. If the high clustering scores and low downstream scores disappear or change substantially, the reported effects are artifacts of the always-on margin=5 penalty; if they persist, the negative result may be robust, though eMargin must then be redefined as a static margin, not an adaptive one.","verdict_should_be":"REJECT","load_bearing_attack":"The load-bearing flaw is the margin value relative to the similarity range. In Section 3.2, Eq. 2 defines M_margin = 0.5(1-Y)M^2 + 0.5 Y max(0, margin-M)^2, where M = sim(z_t, z_{t+1}) is cosine similarity in [-1,1]. Training details state margin=5. Therefore, for every pair labeled Y=1, margin-M >= 4, so max(0, margin-M) is always positive and equals margin-M. The penalty is applied uniformly to all dissimilar adjacent steps; no pair can ever be 'already larger than the margin', directly contradicting Section 3.3's claim that no penalty is applied when M is large. The threshold-based adaptivity therefore reduces to a static, always-on squared penalty, not an adaptive margin. This is load-bearing because the paper's central story—that eMargin adaptively separates dissimilar pairs while preserving similar ones, producing tight clusters that inflate DBI/Silhouette yet fail downstream—depends on the margin being within the similarity scale. With margin=5, every positive pair is being pushed apart with force proportional to (5-M)^2, which likely dominates the loss and can create artificial geometry. Consequently, the experiments do not evaluate the proposed method as described. The missing plain InfoNCE baseline further prevents attribution of the clustering gains to the margin, but the margin-scale issue is the primary correctness risk.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes eMargin, a modification of InfoNCE that uses a pseudo-label derived from raw-data cosine similarity to apply a margin penalty to adjacent time steps deemed dissimilar. The authors evaluate eMargin on three time series datasets (HARTH, SLEEPEEG, ECG) using the Davies-Bouldin Index and Silhouette score as clustering metrics and linear probing as a downstream classification evaluation. Their reported results show that eMargin improves clustering metrics but does not improve downstream classification, which they interpret as evidence that unsupervised clustering metrics can be misleading surrogates for representation quality. The source code is publicly available.","tokens_in":9932,"tokens_out":6610,"duration_ms":76106,"significance":"If the mechanism worked as described, the paper would provide a useful cautionary empirical result about the reliability of clustering metrics in time series self-supervised learning. The paper has strengths: public code, a consistent encoder architecture across all baselines, multiple real-world datasets, an honest negative result, and t-SNE evidence of the geometric artifacts induced by the loss. As written, however, the central claim is undermined by a scale mismatch between the margin parameter and the cosine similarity range, and by the absence of a plain InfoNCE control that would isolate the effect of the margin term.","major_comments":[{"comment":"The margin parameter is set to 5 while M is cosine similarity in [-1,1]. Therefore, for every Y=1 pair, margin - M >= 4, so max(0, margin-M)^2 is always positive. This contradicts Section 3.3's statement that \"If M is already larger than the margin, no penalty is applied,\" since that regime is unreachable. The threshold-based adaptivity described in Section 3.2 is thus not realized in the reported experiments: the implemented loss is an always-active squared penalty on all pseudo-dissimilar positive pairs, and the clustering results are attributable to that penalty rather than to an adaptive margin. The authors should either re-run with a margin within the similarity range (e.g., 0.5) or reframe the method and claims to match the actual loss.","section":"Section 3.2, Eq. (2); Section 3.3; Section 4 (Training Details)"},{"comment":"The central comparison lacks a plain InfoNCE baseline. The tables list TNC, InfoTS, CoST, SimMTM, TimeDRL, TS2Vec, SoftCLT, and MF-CLR, but there is no row for InfoNCE without eMargin. Since eMargin is defined as InfoNCE plus the margin term, the margin's effect cannot be isolated without this control. This omission is especially problematic because the margin term changes the scale of the positive-pair similarity in the loss and, with margin=5, can dominate the objective; the reported clustering gains and downstream losses could therefore be caused by this scale shift rather than by margin-based separation.","section":"Tables 1 and 2; Sections 4.1 and 4.2"},{"comment":"The clustering evaluation is incompletely specified. It is not stated how clusters are formed before computing DBI and Silhouette (e.g., k-means with known k, or assignments based on the pseudo-labels Y). This matters because Eq. (2) explicitly optimizes compactness and separation, so DBI/Silhouette gains under eMargin are partly circular: the loss directly shapes the geometry that these metrics measure. The linear-probe results are the only non-circular evidence, so the conclusion that clustering metrics are unreliable should be supported by a more explicit analysis, such as a rank correlation between clustering metrics and downstream accuracy across all methods, and by stating the cluster-assignment procedure.","section":"Section 4.1; Section 5"},{"comment":"The denominator in Algorithm 1 uses M_margin,t,k for k not equal to t,t+1, but M_margin is defined only for consecutive pairs. Please clarify whether negative pairs use the original similarity sim(z_t,z_k) or a modified margin value, since this changes the effective loss and could affect all reported results.","section":"Algorithm 1, line 8"}],"minor_comments":[{"comment":"The baseline called \"Soft\" in Section 4.1 appears as \"SoftCLT\" and \"TS2Vec+SoftCLT\" in tables; please unify the notation and cite reference [16] precisely.","section":"Section 4.1; Table 2"},{"comment":"The Davies-Bouldin Index is cited but not defined; please include the formula or a precise description so the reader knows exactly how it is computed in this paper.","section":"Section 4.1"},{"comment":"The term \"adaptive\" is used to describe threshold-dependent behavior, but the threshold only creates the binary pseudo-label Y; the margin itself is a fixed scalar. Please clarify the distinction between the pseudo-label threshold and the margin magnitude.","section":"Section 3.1 and 3.2"},{"comment":"There are inconsistent dataset name spellings, including \"Harth\" versus \"HARTH\" and \"SleepEeg\" versus \"SLEEPEEG\"; please standardize these names.","section":"Throughout"},{"comment":"Some rows report accuracy values higher than both precision and recall (e.g., HARTH Random Init.); please state whether precision and recall are macro-averaged and provide class-balance information to aid interpretation.","section":"Table 2"}],"recommendation":"major_revision","confidential_remarks":"I have chosen major_revision rather than reject because the primary flaw is correctable: re-running with a feasible margin within [-1,1] and adding a plain InfoNCE baseline would allow the authors to test the proposed adaptive mechanism as described. However, if the authors do not re-run the experiments and instead only revise the text, I would not recommend acceptance, since the current numbers do not evaluate the claimed method."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Here's my take on 2507.14828. The headline finding—that a contrastive loss which explicitly tightens clusters can inflate Silhouette and DBI while making linear-probe accuracy worse—is real, and worth saying once with clean evidence. The paper does that with three datasets, several strong baselines, and public code. The t-SNE spirals are a nice visual: they make the point that geometric compactness can be semantically meaningless. So the paper has a useful core.\n\nBut the loss as written cannot do what it claims. In Eq. (2), M is a cosine similarity in [-1,1], and the training details set margin=5. Then for every Y=1 pair, margin-M >= 4, so max(0, margin-M)^2 is always positive. The 'if M is already larger than the margin, no penalty is applied' regime from Section 3.3 is unreachable. The adaptive cutoff never operates; instead you have an always-on squared penalty that scales with dissimilarity. That changes the method being evaluated. The clustering gains may come from this constant repulsion, not from the threshold-based adaptivity that the paper sells. This is the load-bearing flaw, and I don't see a way around it in the text.\n\nSecond, Tables 1 and 2 never include plain InfoNCE with the same CNN backbone. Without that, you can't attribute the clustering improvements to the margin at all; it could be the backbone or the pseudo-label Y. This is a standard control that should have been there.\n\nMinor: DBI computation is not described beyond citing [23]; with three seeds and no fixed clustering protocol, Table 1 numbers are provisional. The circularity point—the loss optimizes cluster compactness and then achieves good cluster metrics—is real but not disqualifying; the paper's claim is about transfer failure, which is external.\n\nOverall: the caution that unsupervised cluster metrics are unreliable proxies for downstream utility is plausible and already part of the literature, but the paper as submitted doesn't establish it for eMargin because the proposed mechanism is not what was run. The fix is straightforward: set margin within the similarity range, add an InfoNCE-only baseline, and report the clustering protocol. Then the negative result would be worth a careful look.\n\nThis deserves a serious referee—the question is meaningful, the empirical design is mostly sound, and the flaw is fixable. I would not cite it in its current form, but I'd read a revised version.","headline":"A useful caution about cluster metrics, but the central mechanism is disabled by a margin=5 vs cosine-in-[-1,1] mismatch; fixable, but as-is not a valid test.","tokens_in":10458,"tokens_out":3370,"would_cite":false,"duration_ms":31456,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Adding an adaptive margin to InfoNCE improves clustering metrics but hurts downstream classification.","keywords":["contrastive learning","time series representation learning","adaptive margin","InfoNCE","clustering metrics","linear probing","unsupervised evaluation","representation collapse"],"falsifier":"Retrain eMargin on the same three datasets with margin values of 0.5, 1.0, and 2.0; if the clustering improvements persist at margins at or below 1, the always-on-penalty explanation is wrong, while if they vanish or weaken substantially, the reported cluster geometry is an artifact of the overscaled margin.","tokens_in":9415,"feed_emoji":"📉","tokens_out":6834,"duration_ms":66967,"temperature":0.7,"pith_summary":"This paper attempts to establish that an adaptive margin added to the InfoNCE contrastive loss, which the authors call eMargin, creates time series embeddings that look better under unsupervised clustering metrics yet transfer worse to downstream classification. Across three real-world datasets (HARTH, SLEEPEEG, and ECG), eMargin consistently achieves the best Davies-Bouldin Index and Silhouette scores among nine methods, but underperforms several baselines when a linear classifier is trained on the frozen embeddings. The point the authors want a fair reader to take away is that cluster compactness and separation in embedding space are not reliable proxies for task-relevant utility, and that an evaluation protocol built only on clustering scores can rank a model highly for the wrong reasons.","feed_headline":"A contrastive margin lifts clustering scores but fails linear probing","feed_subtitle":"On three time-series datasets, eMargin beats baselines on Silhouette and DBI, then loses with a linear probe.","key_machinery":"The load-bearing object is the eMargin-modified similarity matrix $M_{\\mathrm{margin}} = \\frac{1}{2}(1-Y)M^2 + \\frac{1}{2}Y[\\max(0,\\mathrm{margin}-M)]^2$, where $Y$ is a pseudo-label computed from the cosine similarity of consecutive raw time steps against a threshold, and $M$ is the cosine similarity of the learned embeddings of those same steps. The $Y=0$ branch squares the similarity to reinforce local continuity, while the $Y=1$ branch applies a squared hinge that pushes dissimilar adjacent steps apart in embedding space. This reshaped similarity is fed into the softmax of InfoNCE, and the paper's argument is that this dynamic reshaping should prevent trivial solutions that overfit local smoothness; the experiments then show that the reshaping instead yields tight but semantically misaligned clusters.","core_discovery":"The central discovery is a disconnect between unsupervised geometry and supervised utility. eMargin replaces entries of the cosine-similarity matrix for adjacent time steps with a squared-hinge penalty when a pseudo-label built from raw-data similarity marks the pair as dissimilar, producing embeddings that score best on DBI and Silhouette across HARTH, SLEEPEEG, and ECG. The same embeddings, however, fall behind state-of-the-art baselines in linear-probe accuracy, F1, precision, and recall, with particularly low precision and recall on SLEEPEEG. The paper interprets this as evidence that the eMargin objective promotes geometric compactness without aligning clusters to class-relevant decision boundaries.","pith_inferences":["A cleaner test of the adaptive-separation idea would calibrate the margin to the actual range of the similarity function, such as $\\mathrm{margin}\\le 1$ or a data-dependent quantile of $M$; the paper's margin of 5 may prevent adaptivity from ever engaging.","The spiral-like geometry suggests that indices sensitive only to pairwise distances can be gamed by continuous, locally tight manifolds; connectivity-based or shape-aware cluster validity indices might not be fooled the same way.","If the margin is the culprit, then a margin that activates only at true transition points, where adjacent steps genuinely differ in data space, could plausibly improve downstream tasks; that hypothesis is testable by threshold sweeps on the raw-data similarity."],"forward_implications":["When the central claim holds, clustering metrics such as DBI and Silhouette are unsafe for model selection in time series representation learning, because the best-scoring method can be one of the worst at downstream classification.","Adding a margin to InfoNCE does not by itself improve transfer; it can trade class-relevant linear separability for geometric compactness.","Tight, elongated (spiral-like) manifolds can inflate compactness-and-separation indices even when they entangle multiple ground-truth classes.","An evaluation protocol for learned time series representations should include a frozen-backbone linear probe alongside unsupervised clustering scores."],"supporting_citations":[{"why":"Supplies the InfoNCE objective that eMargin modifies.","marker":"[18]"},{"why":"Provides the extended InfoNCE formulation used in the paper.","marker":"[5]"},{"why":"Defines temporal neighborhood coding and the evaluation protocol for clustering metrics.","marker":"[23]"},{"why":"TS2Vec is the main baseline and the source of the representation-dimension setting.","marker":"[28]"},{"why":"Soft contrastive learning motivates the dynamic adjustment of similarity penalties.","marker":"[16]"},{"why":"Posits that natural clustering is a property of good representations, the assumption tested here.","marker":"[2]"},{"why":"Defines the Davies-Bouldin Index used as one of the two clustering metrics.","marker":"[6]"},{"why":"Defines the Silhouette score used as the other clustering metric.","marker":"[20]"}],"fun_headline_variants":["eMargin boosts clustering but fails linear probing","Contrastive margin: clusters shine, classifiers stumble","eMargin: good geometry, poor linear probe results","Margin-based separation aids clusters, not classification","eMargin wins on Silhouette, loses on accuracy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The margin is fixed at 5 while cosine similarity can only reach 1, so the penalty $\\max(0,5-M)^2$ is strictly positive for every pair labeled dissimilar, which means the threshold-based adaptivity never actually shuts off and the method may be an always-on push rather than an adaptive one.","fun_headline_variants_meta":{"raw":{"variants":["eMargin boosts clustering but fails linear probing","Contrastive margin: clusters shine, classifiers stumble","eMargin: good geometry, poor linear probe results","Margin-based separation aids clusters, not classification","eMargin wins on Silhouette, loses on accuracy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000138,"raw_usage":{"total_tokens":1094,"prompt_tokens":826,"completion_tokens":268,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":442,"completion_tokens_details":{"reasoning_tokens":195}},"tokens_in":442,"tokens_out":268,"duration_ms":4096,"temperature":1.0,"reasoning_tokens":195,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T15:46:11.525918+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain eMargin on the same three datasets with margin values of 0.5, 1.0, and 2.0; if the clustering improvements persist at margins at or below 1, the always-on-penalty explanation is wrong, while if they vanish or weaken substantially, the reported cluster geometry is an artifact of the overscaled margin.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the extended InfoNCE formulation used in the paper."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"TS2Vec is the main baseline and the source of the representation-dimension setting."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Soft contrastive learning motivates the dynamic adjustment of similarity penalties."},{"cited_title":"Bengio, A","cited_arxiv_id":null,"evidence_quote":"Posits that natural clustering is a property of good representations, the assumption tested here."}],"review_version":1}