{"id":"06f0fa29-5b19-40b5-b9e8-2a0c2de8c274","arxiv_id":"2607.10548","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"CAOT augments optimal transport with an attention-based semantic-consistency term so that similar short texts receive consistent pseudo-labels, improving clustering accuracy over prior OT baselines.","lead":"The paper introduces CAOT, an optimal-transport pseudo-labeler that adds sample-to-sample semantic consistency (via instance attention) to the usual sample-to-cluster cost. This yields more reliable pseudo-labels and stronger short-text clustering than prior OT methods, with some transfer to long text and images.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"Closed-loop S_att training on CAOT pseudo-labels risks reinforcing early errors rather than correcting them; the paper never measures whether S_att remains faithful to true neighborhoods after warm-up.","rationale":"The reader correctly isolates the closed-loop construction of S_att as the weakest link in the argument that CAOT truly harmonizes global structure with local semantic consistency. All other pieces (OT solver, contrastive pre-training, soft marginals) are either standard or ablated; the novel claim lives or dies on whether the attention term supplies independent neighborhood information rather than merely echoing the current pseudo-labels. The paper never measures this independence (no held-out neighborhood purity, no human similarity correlation, no frozen-attention control). The concrete freeze-or-oracle test above would settle the issue with a single controlled experiment already inside the authors’ codebase. Because the main-table gains are real but not uniform, and because the methodological risk is precisely the one the reader flagged, the CONDITIONAL verdict remains appropriate; no stronger rejection is warranted without the diagnostic.","tokens_in":23570,"tokens_out":651,"duration_ms":7464,"concrete_test":"On StackOverflow and SearchSnippets, freeze G_h after the warm-up epoch and continue training with a static S_att (or replace S_att by a fixed ground-truth same-class adjacency matrix). Recompute Table 2 ACC/NMI and the coupling-matrix visualizations of Fig. 7. If performance collapses or the soft assignments become less confident, the closed-loop training of S_att is load-bearing and the faithfulness assumption is unsupported; if numbers stay within 1–2 points, the concern is largely mitigated.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central claim rests on the assertion that the term -ε3⟨S,QQᵀ⟩ in Eq. (3) injects genuine sample-to-sample semantic consistency into the transport plan. S = S_cos + S_att, and S_att is produced by the instance-level attention network G_h that is itself supervised by the identical CAOT-derived pseudo-labels via L_A (Eq. 10) and the same-cluster sets R_i (Eq. 9). After the K-means warm-up (Algorithm 1), the loop is fully closed: noisy early ŷ shape S_att, which then shapes the next Q, which shapes the next ŷ. The paper supplies no independent diagnostic that S_att continues to track true semantic neighborhoods (or even ground-truth same-class pairs) once EM begins; the only evidence is the final ACC/NMI numbers. If the attention matrix merely amplifies the warm-up partition, the claimed “local-consistency correction” is illusory and the gains over RSTC/SCPCL could be explained by other components (contrastive fine-tuning, soft uniformity via ε₂, etc.).","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes CAOT-STC, an EM-style short-text clustering framework whose E-step solves a consistency-aware adaptive optimal transport problem (Eq. 3). The objective augments the usual sample-to-cluster cost with an entropy term, a soft uniformity penalty Ψ(b) that accommodates class imbalance, and a quadratic semantic-consistency term −ε3⟨S,QQᵀ⟩ where S = S_cos + S_att. S_att is produced by an instance-level attention network G_h that is itself trained with the same CAOT pseudo-labels (L_A, Eq. 10). After a K-means warm-up the three modules (PGM, SSCM, SGM) are jointly optimized. Experiments on eight short-text benchmarks, plus long-text and image transfers, report gains over RSTC, SCPCL and FNSCC; ablations, representation-quality plots, coupling-matrix visualizations and a CV-based rule for ε2 are supplied. Code is released.","tokens_in":23963,"tokens_out":1268,"duration_ms":12583,"significance":"If the claimed local-consistency correction is genuine, the work supplies a practical and scalable improvement to OT-based pseudo-labeling that is especially useful for short, sparse text and for imbalanced label distributions. Strengths that raise the paper above a pure empirical tweak include: (i) an explicit, solvable OT formulation that jointly encodes global structure and neighborhood consistency; (ii) a mini-batch solver whose runtime scales better than full-dataset OT (Tables 9–10); (iii) a simple CV heuristic that removes manual tuning of the imbalance weight ε2; (iv) public code and cross-domain checks on long text and images. These elements make the contribution reusable beyond the eight short-text tables.","major_comments":[{"comment":"The central claim that −ε3⟨S,QQᵀ⟩ injects genuine sample-to-sample semantic consistency rests on S_att remaining a faithful neighborhood proxy after the K-means warm-up. After Algorithm 1 leaves the warm-up stage, S_att is trained exclusively with CAOT-derived same-cluster sets R_i (Eq. 9) via L_A (Eq. 10). The manuscript never reports an independent diagnostic—e.g., precision/recall of S_att against ground-truth same-class pairs, or correlation of S_att with frozen SBERT cosine neighborhoods—once EM begins. Without such a check it is possible that S_att merely amplifies the warm-up partition, rendering the “local-consistency correction” illusory and leaving open the possibility that the observed gains over RSTC/SCPCL arise from contrastive fine-tuning or the soft uniformity term alone. A short diagnostic table or plot would make the load-bearing assumption falsifiable.","section":null},{"comment":"Table 2 shows that OURS is not uniformly superior: on SearchSnippets ACC drops 5.04 points relative to FNSCC, and on GoogleNews-TS ACC drops 3.44 points relative to SCPCL. The abstract and §4.2 claim that the method “outperforms state-of-the-art methods” and ranks first on eleven of sixteen indicators; the mixed results should be acknowledged more carefully and the conditions under which CAOT helps versus hurts should be discussed, otherwise the central empirical claim is overstated.","section":null}],"minor_comments":[{"comment":"Eq. (3) writes a = 1/n 1_n×1 while Appendix A.1 writes a = 1/N 1_n×1; the symbol N is never defined. Unify notation.","section":null},{"comment":"Figure 1 caption and the surrounding text refer to “hexagons” and “blue triangles”; the figure itself is schematic and hard to parse without color. A clearer legend or grayscale-safe markers would help.","section":null},{"comment":"In Algorithm 1 the interactive-stage loss is written as Eq. (12) but the text defines it as Eq. (13); fix the cross-reference.","section":null},{"comment":"Table 3 “w/o All” still contains S_cos in some rows according to the prose; clarify whether the ablation removes both similarities or only the attention term.","section":null},{"comment":"The CV-based mapping for ε2 (Table 4 and §4.5) is useful but the thresholds [0,0.2), [0.2,0.4), … appear chosen post-hoc; a short sensitivity check or leave-one-dataset-out validation would strengthen the claim of automatic configuration.","section":null},{"comment":"Typos: “short in modeling” (abstract), “Pseduo-labeling” (Appendix B.1 heading), and inconsistent capitalization of dataset names.","section":null}],"recommendation":"major_revision","confidential_remarks":"The closed-loop concern is real but fixable with a modest diagnostic; I would not reject on that ground alone. The mixed Table-2 results are more serious for the abstract’s wording than for the technical contribution. Scope is appropriate for a methods-oriented ML venue; the cross-domain tables are a plus. Code release is already present, which lowers the bar for reproducibility."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"The one thing worth knowing is that they add a clean bilinear term −ε3⟨S,QQᵀ⟩ to the OT objective so that transport vectors of similar samples are pulled together, and they back it with a jointly trained instance-level attention matrix. That is the actual novelty relative to SELA, RSTC, PPOT and SCPCL; everything else is standard EM + contrastive fine-tuning of SBERT.\n\nWhat they do well: the failure mode in Figure 1 is real and common near decision boundaries of pretrained embeddings. The derivation in Appendix A is ordinary Lagrange + first-order Taylor linearization of the quadratic term; it converges in practice (Figure 6). Ablations in Table 3 isolate S_att, S_cos, contrastive learning and the LP loss. Main results (Table 2) are competitive to best on most of the eight short-text sets, with a clear jump on StackOverflow; they also show modest transfer to long text and images and release code. The CV-based schedule for ε2 is a practical touch that removes one manual knob.\n\nThe soft spot is exactly the closed loop the stress-test flags. After the K-means warm-up, S_att is trained with the same CAOT pseudo-labels that it then feeds back into the next OT solve (Eqs. 9–10 and Algorithm 1). There is no independent check that S_att continues to track true neighborhoods once EM starts—only final ACC/NMI. That does not kill the paper; ordinary self-training has the same risk and the warm-up plus dual similarities (cos + att) give some buffer. But it means the “local-consistency correction” claim is only indirectly supported. Gains are also not uniform (they lose to FNSCC on SearchSnippets ACC), and free parameters remain (ε1–3, λ, temperatures).\n\nMath and citation pattern look solid; no load-bearing circularity beyond the usual EM loop. This is for people already working on OT pseudo-labeling or short-text clustering who want a reproducible incremental win. I would send it to peer review; a referee can demand the missing neighborhood-fidelity diagnostic and tighter statistical testing. Worth engaging if you care about the subfield.","headline":"Solid engineering fix for OT pseudo-labeling: the bilinear consistency term is new and the gains are real, but the closed-loop attention training is the soft spot the paper never fully diagnoses.","tokens_in":24532,"tokens_out":564,"would_cite":true,"duration_ms":6241,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"Optimal transport that also looks at neighboring samples yields more reliable pseudo-labels for short-text clustering.","keywords":["short text clustering","optimal transport","pseudo-labeling","instance-level attention","semantic consistency","EM clustering","imbalanced clustering"],"falsifier":"Replace the learned attention matrix S_att with a pure cosine matrix (or with random similarities) while keeping every other component fixed; if clustering accuracy on the eight benchmarks no longer improves over plain OT, the claimed benefit of the consistency term disappears.","tokens_in":24484,"feed_emoji":"🔗","tokens_out":570,"duration_ms":8477,"temperature":0.7,"pith_summary":"Short-text clustering often relies on optimal transport to assign pseudo-labels that respect the global layout of samples versus clusters. Prior transport methods ignore whether nearby, semantically similar texts receive the same label, so ambiguous samples near decision boundaries can be mislabeled and then poison later training. This paper introduces consistency-aware adaptive optimal transport (CAOT): an instance-level attention network first estimates pairwise semantic similarity, and that similarity matrix is inserted as an extra term into the transport objective. Solving the resulting problem produces soft assignments that simultaneously minimize sample-to-cluster cost and encourage similar samples to share labels. Those labels then supervise both the clustering head and the attention network itself inside an EM loop. On eight standard short-text benchmarks the resulting clusters beat recent OT baselines on most accuracy and NMI scores, and the same pipeline also works for long documents and images.","feed_headline":"Transport that looks at neighbors fixes short-text clusters","feed_subtitle":"Adding a local-similarity term to optimal transport yields more reliable pseudo-labels and higher accuracy","key_machinery":"Consistency-aware adaptive optimal transport (CAOT): the minimization of sample-to-cluster cost plus entropy and imbalance penalties minus an inner-product term that aligns the transport matrix Q with a learned similarity matrix S (Eq. 3).","core_discovery":"When the classic OT objective is augmented with a neighborhood-consistency term that rewards transport vectors of attention-similar samples to be alike, the recovered pseudo-labels become more accurate and confident than pure sample-to-cluster transport, and the labels can be used to train a joint clustering model that outperforms prior short-text methods.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Neighbor-aware OT yields more reliable short-text pseudo-labels","Local consistency term makes OT fix short-text clusters","Attention-guided transport improves short text clustering accuracy","OT that aligns similar samples produces better short-text labels","Harmonizing global OT structure with local neighborhood consistency"],"cache_read_input_tokens":16512,"weakest_assumption_plain":"That the attention similarity matrix, which is itself trained with the same pseudo-labels it helps produce, is a faithful enough proxy for true semantic neighborhoods that the extra consistency term corrects early mistakes rather than locking them in.","fun_headline_variants_meta":{"raw":{"variants":["Neighbor-aware OT yields more reliable short-text pseudo-labels","Local consistency term makes OT fix short-text clusters","Attention-guided transport improves short text clustering accuracy","OT that aligns similar samples produces better short-text labels","Harmonizing global OT structure with local neighborhood consistency"]},"model":"grok-4.5","effort":"low","cost_usd":0.004782,"raw_usage":{"total_tokens":1367,"prompt_tokens":759,"num_sources_used":0,"completion_tokens":81,"cost_in_usd_ticks":47820000,"prompt_tokens_details":{"text_tokens":759,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":527,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":759,"tokens_out":81,"duration_ms":5707,"temperature":1.0,"reasoning_tokens":527,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-14T10:53:49.906618+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"Replace the learned attention matrix S_att with a pure cosine matrix (or with random similarities) while keeping every other component fixed; if clustering accuracy on the eight benchmarks no longer improves over plain OT, the claimed benefit of the consistency term disappears.","supporting_citations":[],"review_version":1}