{"id":"4ac20e26-e2c9-4853-a98a-ce3f7ad67e89","arxiv_id":"2607.27357","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":8,"one_line_summary":"SSCD lifts unpaired cross-modal medical classification by aligning global and class-conditional code distributions in a shared codebook, improving student macro-F1 by 5.7 and 2.5 points on two settings.","lead":"A new training method transfers diagnostic knowledge from a costly medical imaging modality, such as OCT or CT, to a cheaper one like fundus photos or chest X-rays, without needing the same patients in both image types. The trick is to convert both image types into a shared codebook of visual concepts, then align how each disease uses the codes.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Missing control: a teacher with no class knowledge under the same SSCD pipeline; without it, the +5.7/+2.5 F1 gains may be codebook regularization rather than cross-modal semantic transfer.","rationale":"The reader's weakest assumption is the frozen random teacher projection g_T (Sec. 3.3), with the suggestion to ablate a learned g_T. I agree that the teacher-side signal must be meaningful, but that specific ablation would not settle the more fundamental question of whether the teacher's disease knowledge causes the improvement. A learned g_T could change the numbers while the baseline gains over student-only might still come from the codebook losses acting as a regularizer. The decisive control is to remove the teacher's class knowledge entirely: if a teacher with no valid class structure produces the same improvements, then the central claim of disease-level knowledge transfer is false; if it fails, the frozen random projection is at least adequate for the transfer claim. This is why the concern is more load-bearing than the reader's formulation: it targets the semantic-transfer interpretation directly. The paper's ablations and Table 4 are consistent with either hypothesis, so the missing control is the gap between what is shown (a method that improves F1) and what is claimed (a mechanism that transfers disease knowledge). Since the reader already returned a conditional verdict and this concern adds a concrete verification step rather than overturning the empirical results, the verdict remains unchanged.","tokens_in":11102,"tokens_out":11379,"duration_ms":102277,"concrete_test":"Run the identical SSCD pipeline on both settings (OCT→fundus and CT→CXR) with the teacher's disease knowledge removed, keeping all hyperparameters, the frozen random g_T, and the codebook losses. The cleanest version is to use a randomly initialized ResNet-50 teacher with no training (or a teacher trained on shuffled labels), then freeze it and run SSCD as usual. Compare student macro-F1 against the trained-teacher SSCD result. If the no-knowledge teacher matches or exceeds the trained-teacher gains over student-only, the improvement is not evidence of cross-modal semantic transfer; if the gains shrink substantially, teacher knowledge is causally responsible.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that SSCD transfers disease-level knowledge is not yet fully supported because the experiments never separate the teacher's class information from the regularizing effect of the auxiliary codebook losses. The Table 3 ablations remove Ldist and Lcls-cb and show these terms contribute, but they do not show that a teacher with actual disease knowledge is necessary. Equations (7) and (9) are aggregate distribution-matching losses with the teacher distribution detached; such losses can improve a student's marginal or class-conditional behavior even if the teacher codes carry no class structure. The only direct evidence for semantic content in the teacher side is Table 4, which reports cosine similarity between teacher and student class-mean code distributions (0.60–0.80 on OCT→fundus, 0.49–0.55 on CT→CXR). These values are not compared against a null model (e.g., random teacher or shuffled labels), and high cosine similarity can arise simply because both distributions are close to the shared marginal that Ldist already enforces. Thus the headline improvement could be a codebook-induced regularizer rather than evidence of knowledge transfer. The frozen random teacher projection (Sec. 3.3) is a secondary aspect of this same gap: even if g_T preserved class structure, the causal role of teacher knowledge would remain untested. A no-knowledge teacher control is the decisive experiment.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes Shared Semantic Codebook Distillation (SSCD) for unpaired cross-modal medical image classification. Teacher and student encoders project their features into a shared low-dimensional space, softly assign them to a common discrete codebook, and are trained to align batch-marginal and class-conditional code distributions, with the codebook updated by EMA and kept diverse via entropy regularization and dead-code restart. The teacher and codebook are discarded at inference. Experiments on OCT→fundus (6-class MultiEYE) and CT→CXR (binary COVIDx/NIH) report macro-F1 gains of +5.7 and +2.5 over the student-only baseline, outperforming the evaluated baselines (DistillGlobMean, DistillClsMean, FDDM). The method is well specified, includes ablations and three-seed error bars, and releases code.","tokens_in":11453,"tokens_out":3729,"duration_ms":33345,"significance":"If the central claim holds, SSCD provides a practical, generic mechanism for transferring disease-level knowledge between unpaired, heterogeneous medical modalities without paired data or direct feature matching, at no inference cost. The paper's strengths include a clear method description, two heterogeneous benchmark settings, ablations of each loss term, and public code. However, the key interpretive claim—that the gains arise from the teacher's disease knowledge rather than from the codebook losses acting as a regularizer—is not currently supported by the experiments as designed. The missing no-knowledge teacher control is therefore load-bearing for the paper's central contribution.","major_comments":[{"comment":"The central claim that SSCD transfers disease-level knowledge is not yet supported because no control separates the teacher's class information from the regularizing effect of the auxiliary codebook losses. The losses in Eq. (7) and Eq. (9) operate on aggregate, detached teacher distributions, so they can improve a student's marginal and class-conditional behavior even if the teacher codes carry no class structure. Table 4 reports cosine similarities between teacher and student class-mean code distributions (0.60–0.80 and 0.49–0.55), but these values are not compared against a null model such as a randomly initialized teacher, a label-shuffled teacher, or a teacher whose codes are random; high similarity could arise simply because both distributions are close to the shared marginal enforced by Ldist. The authors should run a no-knowledge teacher control (e.g., an untrained teacher encoder, or a teacher trained on shuffled labels) through the identical SSCD pipeline and report whether the +5.7/+2.5 F1 gains persist. If they do, the results would indicate a codebook-induced regularizer rather than cross-modal semantic transfer, and the paper's framing would need to change accordingly.","section":"Sec. 3.5, Eq. (7)–(9); Sec. 4.4, Table 4"},{"comment":"The justification for freezing the teacher projector g_T at random initialization relies on an analogy to the Johnson–Lindenstrauss phenomenon, but JL guarantees apply to linear maps, while g_T is a two-layer nonlinear ReLU MLP. The paper's cited empirical evidence is the 'teacher class separability in Fig. 2', yet Fig. 2 shows UMAP of backbone features h_T, not of the projected embeddings z_T used for codebook assignment; it therefore does not directly validate the crucial assumption that class structure survives the random nonlinear projection. Because all teacher-side signal flows through g_T, a degradation of class structure there would make the distillation target noisy. The authors should either (a) measure class separability or k-NN accuracy on z_T itself, or (b) ablate against a learned or fine-tuned g_T, to demonstrate that the frozen random projection is not a major source of information loss.","section":"Sec. 3.3, 'Frozen teacher projection'"},{"comment":"OCT-CoDA, the strongest prior unpaired cross-modal method on OCT→fundus, is excluded from the main comparison because it requires LLM-generated disease concepts and a pretrained vision-language backbone. The stated reasons are practical, but the paper's contribution claims that SSCD 'outperforms all evaluated distillation baselines' and that what 'remains open' is a generic framework without external modality-specific priors. As written, the comparison does not establish that SSCD is competitive with the previous state of the art on the MultiEYE benchmark; it only establishes competitiveness with the reimplemented FDDM and simple mean-matching baselines. The authors should either include an OCT-CoDA comparison under the original protocol, or clearly state that the claim is limited to methods that do not use external language/vision-language resources.","section":"Sec. 4.1, 'Baselines'"}],"minor_comments":[{"comment":"There are several LaTeX artifacts in the captions, e.g., 'T able 1' and 'T able 2', which should be fixed.","section":"Throughout"},{"comment":"The per-class cosine similarities are reported without confidence intervals or a statistical test against a null distribution; since the mean values are moderate (0.49–0.55 on CT→CXR), a permutation or bootstrap test, or a comparison with a random-projection baseline, would make the alignment claim more convincing.","section":"Sec. 4.4, Table 4"},{"comment":"The initialization of the codebook C is not specified; please state how the K code vectors are initialized before the EMA updates begin.","section":"Sec. 3.4"},{"comment":"The hyperparameter selection procedure for the baselines sweeps λ over a grid, but the text does not state whether SSCD's own hyperparameters (τ, λdist, λcls-cb, λent, K, d, γ) were selected on validation; reporting this would clarify the comparison's fairness.","section":"Sec. 4.1"},{"comment":"In Eq. (8), the notation N_c^T and N_c^S is used inconsistently with the earlier N_T and N_S; consider unifying the notation for count variables.","section":"Sec. 3.5"}],"recommendation":"major_revision","confidential_remarks":"The paper is within scope and the method is promising, but the missing no-knowledge teacher control is a substantive gap that affects the interpretation of the headline results. The OCT-CoDA exclusion is also worth addressing. I do not see evidence of circularity; the concern is about the strength of the causal claim, not about the mechanics of the method."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nWhat you should know: SSCD is a clean, well-tested distillation recipe for unpaired cross-modal medical classification, but its central claim—that the gains come from transferring teacher disease knowledge—is not yet backed by the right control. The method uses a shared discrete codebook through which teacher and student features are routed; each image becomes a soft distribution over codes, and the student is trained to match the teacher's global and class-conditional code distributions. At test time only the student remains. That's a sound idea, clearly written, with code, ablations, and three-seed error bars.\n\nThe genuinely new part is combining a shared codebook with global and class-conditional distribution matching, without paired samples. The improvements are consistent: +5.7 macro-F1 on OCT→fundus and +2.5 on CT→CXR over the student alone, beating FDDM and two mean-matching baselines. Ablations show all three loss terms contribute, and the authors are honest about rare-class noise and the fact that they recover only part of the teacher-student gap.\n\nThe soft spots are real. Most importantly, there is no experiment that separates teacher knowledge from codebook regularization. The teacher projector is frozen at random initialization, and the Johnson-Lindenstrauss justification is weak for a two-layer ReLU MLP. No learned-projector ablation, and no control with a teacher that lacks class knowledge. Without those, the +5.7/+2.5 could be a regularizer rather than semantic transfer. Table 4's codebook cosine similarities are not compared to a null model, so they don't rescue the interpretation. Excluding OCT-CoDA, the strongest prior on the OCT→fundus setting, is understandable given its LLM requirement, but it weakens the superiority claim.\n\nWho this is for: people working on unpaired cross-modal distillation or multi-modal medical imaging. It deserves a serious referee; the method is useful and the experiments are transparent, but the transfer mechanism needs the missing control before the conclusion is solid.\n\nRecommendation: send to peer review with a request for a no-knowledge-teacher control and a learned-projector ablation.","headline":"Useful new distillation recipe with real gains, but the paper doesn't yet show the gains come from teacher knowledge rather than codebook regularization; needs a no-knowledge teacher control.","tokens_in":11918,"tokens_out":3367,"would_cite":true,"duration_ms":30735,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A shared discrete codebook lets an unpaired student model learn disease-level knowledge from a stronger medical modality, improving OCT-to-fundus from 64.5 to 70.2 macro-F1 and CT-to-chest-X-ray from 73.8 to 76.3 with no added inference…","keywords":["unpaired cross-modal distillation","shared semantic codebook","medical image classification","knowledge distillation","distribution alignment","class-conditional alignment","OCT-to-fundus","CT-to-chest-X-ray"],"falsifier":"Train the student with the same objective but replace the frozen random teacher projection with a learned projection fit to preserve teacher class separability, or with no projection at all, and compare on the same splits; if the gains disappear or reverse, the frozen projection is not preserving the class structure the method relies on. Alternatively, measure teacher class separability in the projected codebook space before any student training: if classes are already indistinguishable there, the alignment signal is not semantic.","tokens_in":10946,"feed_emoji":"🩻","tokens_out":7245,"duration_ms":57397,"temperature":0.7,"pith_summary":"The paper proposes a way to transfer diagnostic knowledge between two medical imaging modalities when no patient-level pairing exists, such as teaching a fundus-photo model from OCT scans or a chest-X-ray model from CT scans. The method represents each image as a distribution over a common set of learned code vectors, then aligns teacher and student code distributions both globally and per disease class, sidestepping the need for paired samples or directly comparable feature spaces. On two unpaired settings, the student improves from 64.5 to 70.2 macro-F1 and from 73.8 to 76.3 macro-F1, beating all evaluated distillation baselines; at inference only the student encoder and classifier remain. A sympathetic reader would care because it suggests that disease-level structure, rather than pixel or feature correspondence, is a sufficient bridge for cross-modal medical transfer.","feed_headline":"Shared codebook lifts unpaired medical model by 5.7 macro-F1","feed_subtitle":"Improves OCT-to-fundus and CT-to-chest-X-ray classification with no added inference cost.","key_machinery":"The central object is the shared semantic codebook: a set of $K$ unit-norm vectors in a common projection space, maintained as a buffer updated by exponential moving average rather than gradient descent. Both teacher and student projected embeddings are softly assigned to these codes by temperature-scaled cosine similarity, turning each image into a distribution over the same vocabulary. The load-bearing mechanism is the pair of distribution-alignment losses: a global term that matches teacher and student batch-marginal code usage under forward KL divergence, and a class-conditional term that matches per-class mean code distributions with squared error. A frozen, randomly initialized teacher projection head keeps the teacher-side distribution stationary while the student projector is trained; the paper invokes the random-projection geometry phenomenon to justify that class structure survives this untrained projection. Diversity guards, including entropy regularization on student code usage and restart of dead codes, prevent collapse onto a few dominant codes.","core_discovery":"The central claim is that unpaired cross-modal knowledge distillation can be achieved by routing both teacher and student through a shared discrete codebook and matching their aggregate code-usage statistics, instead of matching instances or continuous features. The teacher's frozen feature extractor and the student's trainable one are each projected into a shared low-dimensional space, softly assigned to a set of $K$ learnable code vectors, and compared through two losses: a global KL divergence between batch-marginal code distributions and a per-class squared-error term between class-mean code distributions. The codebook is evolved by exponential moving average, with entropy regularization and dead-code restart to keep the vocabulary diverse. On OCT-to-fundus retinal classification the student model rises from 64.5 to 70.2 macro-F1, and on CT-to-chest-X-ray pneumonia classification from 73.8 to 76.3, outperforming all evaluated distillation baselines. Because all teacher-side and codebook modules are discarded at inference, the method adds no deployment cost.","pith_inferences":["Because the alignment is distributional rather than instance-level, the method should generalize to settings where teacher and student labels are only partially overlapping or where the student set is unlabeled, as long as class-conditional teacher statistics are available.","The frozen random projection assumption could be tested directly by ablating a learned teacher projection; if performance holds, the shared codebook is robust to that choice, and if not, part of the measured gain is a projection artifact.","The per-class alignment table suggests that co-occurrence-aware or class-balanced sampling should stabilize rare-class transfer, which the paper itself names as future work.","If the codebook codes turn out to be semantically interpretable, the method could double as an explanation tool for what the student model actually learned from the teacher modality."],"forward_implications":["If the paper is right, disease-level semantic transfer works across unpaired, heterogeneous modalities without paired samples or direct feature matching.","The same codebook mechanism transfers unchanged between very different modality pairs, namely OCT to fundus and CT to chest X-ray.","At inference the student model is identical to a plain student encoder plus classifier, so the transfer comes at zero deployment cost.","Rare classes receive weaker class-conditional alignment because they co-occur less often in both mini-batches, making their transferred code signatures less stable across training runs.","SSCD recovers only part of the gap to the teacher upper bound, indicating that some teacher information is modality-specific and cannot be expressed from the student image alone."],"supporting_citations":[{"why":"Justifies the frozen random teacher projection through the random-projection geometry phenomenon.","marker":"[12]"},{"why":"Supplies the EMA codebook update and dead-code restart mechanism used to evolve the shared vocabulary.","marker":"[17]"},{"why":"The closest unpaired distillation baseline; the method must outperform it under the same backbone and protocol.","marker":"[18]"},{"why":"Supplies the OCT and fundus retinal dataset with a shared six-class label space used for the first setting.","marker":"[19]"},{"why":"Supplies the CT teacher dataset for the second setting.","marker":"[7]"},{"why":"Supplies the chest-X-ray student dataset for the second setting.","marker":"[20]"}],"fun_headline_variants":["Unpaired distillation via shared codebook lifts medical F1","Codebook alignment improves unpaired medical classification","No pairs needed: shared codebook distills medical models","Shared semantic codebook for unpaired medical distillation","Codebook-based distillation boosts student with zero pairs"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that a frozen, randomly initialized projection of the teacher's features preserves enough disease-class structure for the shared codebook to carry genuine semantic signal; if that projection scrambles class structure, the teacher-side alignment target becomes noise and the measured gains would not reflect real knowledge transfer.","fun_headline_variants_meta":{"raw":{"variants":["Unpaired distillation via shared codebook lifts medical F1","Codebook alignment improves unpaired medical classification","No pairs needed: shared codebook distills medical models","Shared semantic codebook for unpaired medical distillation","Codebook-based distillation boosts student with zero pairs"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001008,"raw_usage":{"total_tokens":4288,"prompt_tokens":1003,"completion_tokens":3285,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":619,"completion_tokens_details":{"reasoning_tokens":3211}},"tokens_in":619,"tokens_out":3285,"duration_ms":21036,"temperature":1.0,"reasoning_tokens":3211,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T15:23:18.888195+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train the student with the same objective but replace the frozen random teacher projection with a learned projection fit to preserve teacher class separability, or with no projection at all, and compare on the same splits; if the gains disappear or reverse, the frozen projection is not preserving the class structure the method relies on. Alternatively, measure teacher class separability in the projected codebook space before any student training: if classes are already indistinguishable there, the alignment signal is not semantic.","supporting_citations":[{"cited_title":"Contemporary mathematics26(189-206), 1 (1984)","cited_arxiv_id":null,"evidence_quote":"Justifies the frozen random teacher projection through the random-projection geometry phenomenon."},{"cited_title":"In: International Conference on Medical Image Computing and Computer-Assisted Intervention","cited_arxiv_id":null,"evidence_quote":"The closest unpaired distillation baseline; the method must outperform it under the same backbone and protocol."},{"cited_title":"IEEE Transactions on Medical Imaging44(4), 1711–1722 (2024)","cited_arxiv_id":null,"evidence_quote":"Supplies the OCT and fundus retinal dataset with a shared six-class label space used for the first setting."},{"cited_title":"COVIDx CT-3: A Large-scale, Multinational, Open-Source Benchmark Dataset for Computer-aided COVID-19 Screening from Chest CT Images","cited_arxiv_id":"2206.03043","evidence_quote":"Supplies the CT teacher dataset for the second setting."}],"review_version":2}