{"id":"5bc6f15a-742e-4092-ae32-c110d9c06458","arxiv_id":"2501.00162","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A Wasserstein-distance linear program selects and reweights source classes to reduce transfer distance, improving few-shot target accuracy under extreme label shift including disjoint label sets.","lead":"This paper proposes WaSS, a method that reweights or selects source-domain classes by minimizing the Wasserstein distance between source and target feature distributions, then fine-tunes a classifier on a few labeled target examples. The method is tested under label shift including fully disjoint label sets, with modest accuracy gains over baselines on several image benchmarks.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Section 3.3's claim that DS(Y|Z) is invariant under class reweighting is false for overlapping features; the bound-minimization proof fails, leaving Eq. (3) without the stated theoretical support.","rationale":"The reader's weakest assumption concerns the fixed ResNet feature space; that is a valid external caveat, acknowledged in Appendix B. The more load-bearing issue is internal: the proof of the bound-minimization claim relies on a false invariance. This is not a matter of consensus but of mathematical correctness. If the invariance fails, the central theoretical sentence in Section 3.3 is wrong. The empirical wins in Table 1 and Figure 1 are the main remaining support, so I keep the CONDITIONAL verdict rather than rejecting: the method may work, but the stated theory does not. The paper's own Appendix B admits the embedding limitation; that limitation is secondary because the presented experiments can still provide evidence. No code or error bars are provided, but that is a reproducibility concern, not a mathematical refutation. Hence verdict unchanged.","tokens_in":19597,"tokens_out":7225,"duration_ms":73564,"concrete_test":"Estimate the term E_{DS(w)(Z)}[W1(DS(Y|Z), DT(Y|Z))] on a real split, e.g., CIFAR-10 with ResNet-50 features, for two weight vectors: uniform and the WaSS optimum. If the term differs substantially (e.g., >5% relative), the invariance assertion fails and Eq. (4) is not minimizing the displayed bound. An analytic two-Gaussian counterexample also settles the mathematical claim: take class conditionals N(-1,1) and N(1,1) and verify p_w(Y|Z) changes with w. Either check directly tests the proof step in Section 3.3.","verdict_should_be":"UNCHANGED","load_bearing_attack":"Section 3.3 states, 'the conditional distribution DS(Y | Z) is invariant under the reweighting.' This step connects the LP in Eq. (4) to the generalization bound. For the reweighted source DS(w)(Z) = Σ_i w_i DS_i(Z), the posterior is DS(w)(Y=y|Z=z) = w_y p_y(z) / Σ_i w_i p_i(z), where p_i are the class-conditional densities in feature space. This depends on w unless the supports of the p_i are disjoint. ResNet-50 features of natural images are not class-disjoint, so the conditional is not invariant. Additionally, even if the conditional were fixed, the expectation E_{DS(w)(Z)}[W1(DS(Y|Z), DT(Y|Z))] in the bound varies with w through the outer marginal DS(w)(Z). Thus the upper bound contains w-dependent terms that Eq. (4) does not minimize. The paper's claim that WaSS 'corresponds to a bound minimization algorithm' is therefore not established by the presented analysis. The empirical comparison may still favor WaSS, but the theoretical justification in Section 3.3 has a concrete gap.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes WaSS, a two-stage transfer learning method for settings where the source and target label sets need not overlap. In the first stage, a linear program (Eq. 4) reweights source classes so that the reweighted mixture of source class-conditional feature distributions minimizes the empirical 1-Wasserstein distance to the target feature distribution in a fixed pretrained ResNet-50 embedding. In the second stage, a classifier is trained on the reweighted source data and its final layer is fine-tuned on limited target labels. The authors provide a generalization analysis intended to show that this procedure minimizes an upper bound on target error, and they report experiments in disjoint and open-set label shift settings on several datasets.","tokens_in":19865,"tokens_out":19235,"duration_ms":188426,"significance":"If the empirical results are reliable, WaSS is a simple and inexpensive recipe for source class selection under extreme label shift, and the DDA setting (no shared labels) is well motivated. The class-level reweighting by optimal transport in a fixed embedding is a natural idea that could be practically useful. However, the paper's theoretical justification is not currently valid, and the empirical presentation contains inconsistencies that need to be corrected before the claims can be evaluated. The paper does not release code, and the proofs are not machine-checked; the main strengths are the breadth of datasets, the comparison with several baselines, and the clear statement of the intended bound-minimization rationale.","major_comments":[{"comment":"Lemma 3.1 as stated is false. Take Z={0,1}, Y={0,1}, d_Z=d_Y=1, P_Z=δ_0, Q_Z=δ_1, P(Y|Z=0)=δ_0 and Q(Y|Z=1)=δ_1. With arbitrary conditional values at null points set to make the conditional expectations vanish, the right-hand side is W1(P_Z,Q_Z)=1, while W1(P,Q)=2 because the single joint atoms are (0,0) and (1,1). The proof fails at Eqs. (18)-(20), where L(y_m,y_t) is set to zero because the two conditionals are 'the same'; the label cost does not vanish when z_m≠z_t. Since this lemma is the main tool connecting feature-space W1 to target error, the proposed upper bound and the 'bound minimization' claim are not established.","section":"§3.3, Lemma 3.1"},{"comment":"The statement that 'the conditional distribution DS(Y | Z) is invariant under the reweighting' is only true if the class-conditional feature supports are disjoint. In general, for the reweighted source DS(w)(Z)=Σ_i w_i DS_i(Z), the posterior is DS(w)(Y=y|Z=z)=w_y p_y(z)/(Σ_i w_i p_i(z)), which depends on w whenever the class-conditional densities p_i overlap; ResNet-50 features of natural images are not class-disjoint. Moreover, even if the posterior were fixed, the expectation E_DS(w)(Z)[W1(DS(Y|Z), DT(Y|Z))] depends on w through the outer marginal. Thus Eq. (4) minimizes only the second additive term in the bound, and the claim that it suffices to minimize the first two terms does not follow.","section":"§3.3, after the upper bound"},{"comment":"Theorem 3.1 is quoted with undefined notation (Ω, X) and an equality that is not stated as a distance or a bound, making the cited result impossible to verify. In the proof of Lemma 3.2, the 1/2 factor from Proposition 3.1 is dropped, the Lipschitz step around Eq. (35) omits the expectation on the right-hand side, and Proposition F.2 is misapplied to a matrix difference although it is stated for vectors. The fine-tuning bound αβσmax(VS−VT) is therefore not rigorously established, which matters because the final stage of WaSS is precisely a fine-tuning step.","section":"§3.3, Theorem 3.1 and Lemma 3.2"},{"comment":"The text says WaSS is outperformed in two experiments, but the table shows three losses: PACS [0,1,2] versus OSS, PACS [3,4,5] versus ALL, and C-100 [16,61,9] versus ALL. No standard deviations, confidence intervals, or per-run results are reported for any of the ten iterations, and the caption's paired t-test statement is not substantiated by the visible data; the p-value is only reported for Figure 1, not for Table 1. The 'average 1–2 absolute percentage point increase' claim should also be reconciled with the actual per-split differences.","section":"§4.1, Table 1"},{"comment":"The test-class indices are outside the valid label ranges of the datasets. Office-31 has classes 0-30, but the table reports [31,32,33] and [62,63,64]; Office-Home has classes 0-64, but the table reports [65,66,67] and [130,131,132]; VisDA has classes 0-11, but the table reports [12,13,14] and [24,25,26]. If the experimental setup concatenates domains into a larger label set, the paper must state this explicitly; as written, these rows cannot correspond to any class in the datasets, and the results are not reproducible or interpretable.","section":"Appendix C, Table 2"}],"minor_comments":[{"comment":"The cross-entropy formula is missing a minus sign; as written, ℓCE(g(f(x)), y) is a positive log-likelihood rather than the usual negative log-likelihood.","section":"§2"},{"comment":"Equation (4) is stated as an exact linear program, but the experiments use Sinkhorn regularization; the entropy regularization parameter is a free parameter, and its effect on the recovered weights is neither discussed nor ablated.","section":"§3.1 and Appendix A"},{"comment":"The claim that an optimal subset is obtained presumes that w* is sparse; the simplex constraint alone does not enforce sparsity, and the paper reports no statistics on the number of nonzero weights produced by the LP.","section":"§3.1"},{"comment":"The target class set for Fashion-MNIST is given as {T-shirt/top, Trouser, Pullover} in the text but as {Ankle boot, T-shirt/top, Trousers} in the caption of Figure 4; these should be reconciled.","section":"§4.2 and Figure 4"},{"comment":"The dataset name is spelled 'VizDa' in the table but 'VisDA' in the text; also, the sentence 'three of these datasets are used to emulate distribution shifts for the same set of classes' is unclear.","section":"Appendix C, Table 2"},{"comment":"There are several typos: 'This allos us' should be 'This allows us', 'Lipchitz' should be 'Lipschitz', and the learning rate '1e4' should probably be '1e-4'.","section":"Appendix A and F"},{"comment":"The paper states that code will be released upon publication; for a method whose value hinges on solving a numerical LP, a pseudocode description or code release would substantially aid reproducibility.","section":"General"}],"recommendation":"major_revision","confidential_remarks":"The theoretical section appears to have been assembled without full verification: Lemma 3.1 is false, the conditional-invariance claim is incorrect, and the quotation of Theorem 3.1 is garbled. The Appendix C class indices suggest that the experiments may have been run with a different label convention than the one described, and the text overstates the number of wins in Table 1. I would ask the authors to rewrite or remove the theoretical claim, correct the empirical reporting, and verify the extra-dataset experiments before sending the paper back for review."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"First thing to know: the core idea is genuinely new and simple — solve a class-level Wasserstein LP on pretrained features to reweight source classes for transfer, even when label spaces are disjoint. Table 1 and the larger-scale Figure 1 mostly back it up, with typical 1-2 point gains over PADA, OSS, and other baselines. That is a useful empirical result in transfer learning, though incremental.\n\nWhat the paper does well: the LP formulation is clean, the disjoint-label DDA benchmark is a real addition, and the authors are candid about finite-sample effects and the fragility of using a fixed embedding. The UMAP and pie charts are a nice touch. The citation pattern is fine — the OT-DA and partial-DA related work is the right set.\n\nThe soft spots are real. Section 3.3's theory does not hold together. The paper states that DS(Y|Z) is invariant under class reweighting; that is only true when class-conditional supports are disjoint, which is false for real features. Even if the conditional were fixed, the expectation over DS(w)(Z) inside the bound changes with w, so the upper bound is not minimized by Eq. (4). The claim that WaSS is a bound-minimization algorithm is not established. Lemma 3.2 is also shaky: the quoted theorem from Mousavi Kalan et al. is garbled, the proof relies on unproved propositions, and the term rho*W1 is underspecified. This is fixable — the LP can be presented as a heuristic motivated by the bound — but as written it overclaims.\n\nThere is a reproducibility red flag: Appendix C reports test class indices that cannot exist (Office-31 has 31 classes, yet lists [31,32,33] and [62,63,64]; Office-Home has 65 classes, yet lists [130,131,132]; VisDA has 12 classes, yet lists [24,25,26]). That looks like a copy-paste error, and it undercuts confidence in the appendix numbers. Table 1 also gives no error bars; the paired t-test is mentioned but the p-values for those specific results are not shown.\n\nOverall, the empirical core is credible enough to matter. The paper deserves a serious referee, but it needs major revision: fix the theory or reframe it, correct the appendix, and release code. I would not cite it in its current form.","headline":"Simple, plausible class-reweighting method with mostly convincing empirical wins, but the theory section is wrong on a central claim and the appendix has corrupted indices.","tokens_in":20354,"tokens_out":5166,"would_cite":false,"duration_ms":44257,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A linear program reweighting source classes by Wasserstein distance in a pretrained feature space identifies the classes that best transfer to a target domain, even under disjoint label sets.","keywords":["transfer learning","extreme label shift","Wasserstein distance","class subset selection","domain adaptation","few-shot learning","optimal transport","negative transfer"],"falsifier":"Run WaSS on a target task where ImageNet features are known to conflate semantically different classes, and compare against the ALL baseline; if the Wasserstein-selected subset does not improve target accuracy over uniform source weighting across several splits, the feature-space similarity premise fails.","tokens_in":19396,"feed_emoji":"🎯","tokens_out":5969,"duration_ms":52631,"temperature":0.7,"pith_summary":"The paper tries to establish that a sparse reweighting of source classes, computed by minimizing the Wasserstein distance between the reweighted source feature mixture and the target feature distribution, is the right way to choose what to transfer. It argues that this choice matters when the source and target label sets are largely or completely disjoint, where using all source classes causes negative transfer. If correct, transfer learning under extreme label shift can be reduced to a class-selection step followed by standard fine-tuning, with measurable accuracy gains over methods that restrict to shared classes. The paper also provides a generalization analysis that frames the Wasserstein minimization as bound minimization rather than a heuristic.","feed_headline":"Wasserstein reweighting picks transfer-friendly source classes","feed_subtitle":"Even with no shared labels, the class-selection step improves target accuracy over the compared baselines.","key_machinery":"The central object is the linear program in Eq. (4): a transport-matrix formulation of the Wasserstein distance in which each source class receives a weight $w_i$, the target marginal is uniform, and the source marginal is a class-reweighted mixture. Solving it over the simplex produces the sparse weight vector $w^*$ that selects which source classes to keep and how much to weight each. This class-level reweighting, rather than sample-level transport, is what makes the method computationally practical, and the Sinkhorn algorithm lets it scale to large datasets. Around this optimization the paper builds a theory of induced classifiers and a lifted output space to prove that minimizing this distance is a bound-minimization strategy for target error.","core_discovery":"The central claim is that solving the linear program that minimizes $W_1(\\sum_i w_i D^S_i(Z), D^T(Z))$ in a pretrained feature space yields the source class mixture that best facilitates transfer, and that a classifier trained on that mixture and fine-tuned on limited target labels reaches the highest target accuracy among the compared methods. The paper frames this as bound minimization: a generalization analysis shows the target error of an induced classifier is bounded by the source error plus the Wasserstein distance between reweighted source and target plus a term measuring the change of the final layer after fine-tuning. The method (WaSS) handles the disjoint-label setting by lifting the output space to the union of source and target labels, so the bound does not collapse even when no classes are shared.","pith_inferences":["Our inference: a direct test of the method's core premise would be to compute the class weights with a randomly initialized encoder; if the selected subset still improves target accuracy, the Wasserstein geometry is not doing the claimed semantic work, and if it fails, the dependence on pretrained features is confirmed.","Our inference: the paper fixes the pretrained encoder and never re-solves the linear program after fine-tuning, so a natural extension is to iterate selection and fine-tuning; substantial weight shifts across iterations would indicate that the fixed-feature assumption is material.","Our inference: the same class-reweighting logic should carry to other modalities and to very large, uncurated source pools, where dropping far-away classes could reduce both compute and negative transfer simultaneously."],"forward_implications":["Transfer can work even when the source and target label sets are completely disjoint, because selection is driven by feature-space similarity rather than shared labels.","The optimal source mixture can be found without re-training the encoder, reducing transfer learning to a linear program plus fine-tuning.","Class-level reweighting with Sinkhorn scaling keeps the selection step feasible for large source datasets.","The generalization bound implies that minimizing the Wasserstein term is a principled proxy for target accuracy, not just a heuristic."],"supporting_citations":[{"why":"Supplies the base Wasserstein generalization bound ($\\epsilon_T(h) \\le \\epsilon_S(h)+2\\rho W_1+\\lambda^*$) that WaSS extends to the fine-tuned, disjoint-label setting.","marker":"[Shen et al., 2018]"},{"why":"Introduces the joint-distribution Wasserstein formulation and the conditional-decomposition argument WaSS adapts for class reweighting.","marker":"[Courty et al., 2017]"},{"why":"Provides the Sinkhorn algorithm WaSS uses to solve the linear program at scale.","marker":"[Cuturi, 2013]"},{"why":"Supplies the ImageNet-pretrained ResNet-50 encoder whose feature space defines the similarity metric for class selection.","marker":"[Deng et al., 2009a]"},{"why":"Supplies the transfer-distance theorem for models that differ only in the final layer, used to bound the fine-tuning gap.","marker":"[Mousavi Kalan et al., 2020]"},{"why":"Establishes negative transfer under label-space shift, motivating why reweighting away from divergent source classes is necessary.","marker":"[Zhao et al., 2019]"}],"fun_headline_variants":["WaSS: Wasserstein class selection beats extreme label shift","Even with disjoint labels, Wasserstein picks transferable classes","Reweight source classes with Wasserstein to combat label shift","Bound-minimizing source class mixture lifts transfer under shift","Wasserstein-based class selection improves few-shot transfer"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that distances between ImageNet-pretrained ResNet-50 feature vectors are a trustworthy measure of how useful a source class is to a target task; the selection is computed once in that fixed space and never revised after the downstream encoder is trained, and the paper itself notes that a pretrained network may not always supply a meaningful embedding.","fun_headline_variants_meta":{"raw":{"variants":["WaSS: Wasserstein class selection beats extreme label shift","Even with disjoint labels, Wasserstein picks transferable classes","Reweight source classes with Wasserstein to combat label shift","Bound-minimizing source class mixture lifts transfer under shift","Wasserstein-based class selection improves few-shot transfer"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001166,"raw_usage":{"total_tokens":4808,"prompt_tokens":912,"completion_tokens":3896,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":528,"completion_tokens_details":{"reasoning_tokens":3816}},"tokens_in":528,"tokens_out":3896,"duration_ms":25951,"temperature":1.0,"reasoning_tokens":3816,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T22:58:34.152653+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run WaSS on a target task where ImageNet features are known to conflate semantically different classes, and compare against the ALL baseline; if the Wasserstein-selected subset does not improve target accuracy over uniform source weighting across several splits, the feature-space similarity premise fails.","supporting_citations":[],"review_version":1}