{"id":"a2a0b8e7-747e-48f1-b843-97c7b27ce0ba","arxiv_id":"2412.13036","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":2,"one_line_summary":"RL-OSHeDA, a two-stage representation learning method with pseudo-labeling, outperforms existing domain adaptation baselines on 56 open-set heterogeneous domain adaptation tasks.","lead":"This paper introduces open-set heterogeneous domain adaptation (OSHeDA), where source and target data differ in both feature spaces and label sets, and proposes a representation learning method with theoretical guarantees. The method aligns known-class representations across domains while separating unknown-class samples, and it reports consistent gains over baselines across text, image, and clinical datasets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Theorem 2's proof invokes a triangle inequality for Jensen–Shannon divergence that is false; the reader's alternative concern about unlabeled unknown-class risk misreads P^u_t and does not land.","rationale":"The reader's weakest assumption is not load-bearing because it conflates P^u_t with Pt,u. The paper defines P^u_t via f^u_t(X_t,Y)=(X_t,unk), so all target draws receive the surrogate label 'unk'; no unknown-class labels are needed to compute bE(P^u_t). Eq. (5) is precisely an empirical estimate of the theorem's open-set difference. After checking that, the remaining genuine defect is in the proof of Theorem 2: a triangle inequality for JS divergence that is false. The sqrt-JS metric gives a closely related but different inequality, and its use changes the constant in the noise term. This matters because Theorem 2 is the stated motivation for pseudo-labeling and the non-negative risk estimator. The flaw is local and likely repairable; the empirical evaluation (56 tasks, 10 seeds, Nemenyi test) is independent evidence that RL-OSHeDA works. I therefore keep the reader's CONDITIONAL verdict but replace the supporting reason: the condition should be 'fix the proof of Theorem 2 or state the corrected bound', not 'justify computability of unknown-class empirical errors'.","tokens_in":46523,"tokens_out":18852,"duration_ms":163824,"concrete_test":"Re-derive Eq. (21) of Appendix A.2 using the valid inequality sqrt(DJS(A||B)) ≤ sqrt(DJS(A||C)) + sqrt(DJS(C||B)) in place of the false DJS triangle inequality, and re-prove Theorem 2 from that step. If the resulting finite-sample bound contains sqrt(2N(Pt,k,g)) or an extra factor rather than sqrt(N(Pt,k,g)), Theorem 2's stated inequality is not established as written. As a quick sanity check, evaluate the discrete counterexample A=(1,0), B=(0,1), C=(0.8,0.2); if the printed DJS triangle inequality appears to hold, the computation is in error.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The reader's stated concern is that Theorem 2 requires empirical errors bE(P^u_t, h∘ft) on the unknown-class conditional distribution. This misreads the notation: P^u_t is the distribution induced from Pt by f^u_t(X_t,Y)=(X_t,unk), so every target sample is assigned the surrogate label 'unk'. It is not Pt,u = Pt(·|Y∉Ys). Therefore bE(P^u_t) is computable from all unlabeled target data, and Eq. (5) is exactly an empirical estimate of E(P^u_t) - λE(P^u_s). The load-bearing gap is elsewhere. In Appendix A.2, the proof of Theorem 2, Eq. (21), uses DJS(A||B) ≤ DJS(A||C) + DJS(C||B). Jensen–Shannon divergence does not satisfy triangle inequality. Counterexample: A=(1,0), B=(0,1), C=(0.8,0.2) gives DJS(A||B)=0.693, DJS(A||C)=0.075, DJS(C||B)=0.423, so the sum is 0.498 < 0.693. The correct tool is the metric property of sqrt(DJS): sqrt(DJS(A||B)) ≤ sqrt(DJS(A||C)) + sqrt(DJS(C||B)). Repairing this changes the bound: applying Lemma 2 to DJS(Pt,k(Z,g(Z))||Pt,k(Z,Y)) yields an extra factor of 2, so the stated sqrt(N(Pt,k,g)) term likely becomes sqrt(2N(Pt,k,g)) or requires an additional constant. Thus Theorem 2's finite-sample bound, which is central to the pseudo-labeling and non-negative risk estimator motivation, is not proven as written. The empirical claims remain strong; the theoretical narrative is weakened.","agreement_with_reader":"disagree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper formalizes open-set heterogeneous domain adaptation (OSHeDA), in which source and target feature spaces differ and the target label space strictly contains the source label space. It develops infinite-sample upper and lower bounds for the target error (Theorem 1 and Proposition 1), a finite-sample bound with pseudo-label noise (Theorem 2), and a homogeneous-DA comparison (Proposition 3). Guided by these bounds, it proposes RL-OSHeDA, a representation-learning method with two feature mappings, MMD-based alignment and segregation terms, a non-negative open-set risk estimator, and a two-stage pseudo-labeling scheme. The method is evaluated on 56 tasks across seven datasets, where it reports consistent improvements over baselines and releases code.","tokens_in":46875,"tokens_out":8054,"duration_ms":69811,"significance":"The OSHeDA scenario is timely, and the empirical evaluation is broad, spanning clinical, vision, and text data. The theoretical decomposition into source error, open-set difference, and known-class domain distance is a useful organizing principle, and the lower-bound argument for segregating unknown classes is intuitive. The paper also makes a concrete algorithmic contribution and publicly releases code, which strengthens reproducibility. However, the theoretical narrative currently contains a false inequality in the proof of the main finite-sample theorem and a statement-level typo in the proof of Proposition 1; these need repair before the bounds can be cited as proven.","major_comments":[{"comment":"Equation (21) in the proof of Theorem 2 invokes a triangle inequality for Jensen–Shannon divergence, DJS(A||B) ≤ DJS(A||C)+DJS(C||B), which is false. For example, with point masses A=(1,0), B=(0,1), C=(0.8,0.2), DJS(A||B)=0.693 while DJS(A||C)+DJS(C||B)=0.498. The inequality holds for the square root of JS divergence: sqrt(DJS(A||B)) ≤ sqrt(DJS(A||C))+sqrt(DJS(C||B)). Since the final bound in Theorem 2 is exactly of the square-root form, the theorem can be repaired by applying the metric property of sqrt(DJS) in place of the current Eq. (21), but the proof as written is not valid.","section":"Appendix A.2, Eq. (21)"},{"comment":"The final displayed inequality in the proof of Proposition 1 reads λ E(Pt,u,h∘ft), whereas the theorem statement and the preceding Eqs. (19)–(20) imply λ E(Pt,k,h∘ft). As written, the proof does not match the proposition; this should be corrected.","section":"Appendix A.2, Proposition 1 proof"},{"comment":"Theorem 2 treats the pseudo-label model g as a fixed object and states a bound for all h with N(Pt,k,g) appearing in the bound. In the algorithm of Section 5.2, g is re-derived from the current h and the unlabeled target data at every stage-2 update. The theorem therefore does not, as stated, cover the adaptively coupled (g,h) used by RL-OSHeDA. The authors should either restrict the theoretical claim to a fixed g and state that the algorithm is an approximation, or provide a fixed-point argument.","section":"Section 4.2 / Section 5.2"}],"minor_comments":[{"comment":"In the proof of Lemma 5, the text says 'because of Assumption 2' when bounding the squared losses by C^2; this should cite Assumption 1 instead.","section":"Appendix A.1, Lemma 5 proof"},{"comment":"The KPG row reports N/A for PTB-XL but KPG is included in the average row; the authors should state explicitly that the average excludes the N/A entry.","section":"Table 1"},{"comment":"The dataset names in these tables use the typo 'Reset-50' for ResNet-50; this should be corrected throughout.","section":"Tables 12 and 19"},{"comment":"The notation P^u_t is easy to confuse with Pt,u; the authors should add a one-sentence reminder in Section 4.2 when empirical errors are introduced, since P^u_t is the surrogate-unk distribution, not the unknown-class conditional.","section":"Section 4.1"}],"recommendation":"major_revision","confidential_remarks":"The stress-test concern about Theorem 2 requiring labels for unknown-class conditionals does not survive close reading: P^u_t is the distribution induced by assigning the surrogate label 'unk' to every target sample, so Eq. (5) is indeed computable from unlabeled target data. The genuine issue is the false JS triangle inequality in Eq. (21), which is repairable using the metric property of sqrt(DJS). The Proposition 1 proof typo is also easy to fix. I recommend asking for a revision that corrects these two proof points and clarifies the status of the theorem with respect to the adaptive pseudo-label model; I do not see grounds for rejection."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Take a look at this one. It defines OSHeDA — open-set heterogeneous domain adaptation — and backs it with a broad empirical study: 56 tasks across text, image, and ECG, ten seeds, significance tests, code released. The gains over the closest baselines are consistent, and on PTB-XL the unknown-class detection is much stronger than anything else. That part is solid and worth having in the literature.\n\nThe new setting is a natural join of HeDA and OSDA, and the algorithm is a reasonable composition: MMD alignment, non-negative risk for the unknown class, pseudo-labeling with a two-stage schedule. Nothing exotic, but the benchmark is useful.\n\nThe reader's concern about Theorem 2 misses. P^u_t is the distribution induced by f^u_t(X,Y)=(X,unk), so it is observable from all target data; Eq. (5) really is an empirical estimate of the open-set difference. The actual soft spot is in the proof of Theorem 2, Eq. (21): the paper invokes a triangle inequality for Jensen–Shannon divergence, D_JS(A||B) ≤ D_JS(A||C) + D_JS(C||B). That inequality is false — JS divergence is not a metric, and a concrete counterexample gives 0.693 > 0.075 + 0.423. The function that is a metric is sqrt(D_JS), and swapping in that metric property yields the same final shape of bound only if the intermediate step uses square roots throughout. The paper's written derivation is therefore not correct, and since Theorem 2 is the stated motivation for the pseudo-labeling and non-negative risk terms, the theoretical guarantee is not established as stated. (The same step also seems to drop a label-marginal term when applying Lemma 2, which would need to be handled.) The theorem may be repairable, but it is not a minor typo — it is a load-bearing step in the theory.\n\nBottom line: the empirical contribution and the OSHeDA benchmark are worth serious attention; the theory needs another pass. I would send it to peer review with the requirement that the proof of Theorem 2 be rewritten, and I would not cite the bound as proven until then. The paper is a good candidate for the reading group a few months from now, when the proof is cleaned up.","headline":"Fresh DA setting with a very solid benchmark, but the central finite-sample bound has a proof error that should not be ignored.","tokens_in":47442,"tokens_out":7720,"would_cite":true,"duration_ms":66451,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"A new three-term error bound for open-set heterogeneous domain adaptation shows exactly what a model must optimize, and a method built on that bound outperforms all baselines on 56 tasks.","keywords":["open-set heterogeneous domain adaptation","heterogeneous domain adaptation","open-set domain adaptation","learning bounds","representation learning","pseudo-labeling","non-negative risk estimator","novel class detection"],"falsifier":"Run a synthetic OSHeDA task with oracle labels for unknown-class target samples, train a variant of RL-OSHeDA that uses those oracle labels to compute the true open-set difference, and compare its target error against the version using the non-negative risk estimator in Eq. (5); if the two diverge when the unknown-class loss on known samples is nonzero, the finite-sample guarantee's reliance on unavailable empirical errors is exposed.","tokens_in":46295,"feed_emoji":"🎯","tokens_out":9755,"duration_ms":81127,"temperature":0.7,"pith_summary":"This paper defines and analyzes open-set heterogeneous domain adaptation (OSHeDA), the realistic setting where source and target data live in different feature spaces and the target also contains classes never seen in the source, as when a model trained on digital ECG traces must read scanned paper ECGs and catch rare abnormalities. The authors prove an upper bound on target prediction error that splits into three pieces: source error, an open-set difference between how the two domains treat the unknown class, and the distance between source and target representations of known classes. A companion lower bound shows that pushing unknown-class representations away from known ones is necessary to keep target error from growing. Guided by these bounds, they build RL-OSHeDA, which aligns known-class representations with an MMD-based term, segregates unknowns, and uses a non-negative risk estimator with two-stage pseudo-labeling. Across 56 tasks on text, image, and clinical ECG data, the method reports higher harmonic-mean accuracy than every compared baseline.","feed_headline":"Model adapts across feature spaces and spots unseen classes","feed_subtitle":"Theory splits target error into three controllable terms; the new method outperforms all baselines on 56 tasks.","key_machinery":"The load-bearing object is the error decomposition in Theorem 1, which separates target error into source error, open-set difference, and known-class domain distance. The open-set difference $E(P^u_t,h\\circ f_t)-\\lambda E(P^u_s,h\\circ f_s)$ is the term unique to open-set settings; Proposition 1's lower bound motivates the segregation term by showing that separating the unknown-class target distribution from the source distribution lowers the floor on target error. The finite-sample Theorem 2 adds two pieces of machinery: Natarajan-dimension complexity terms from Lemma 5, and a pseudo-label noise $N(P_{t,k},g)$ that enters the bound through the triangle inequality for JS divergence. On the algorithm side, these terms are implemented with centroid-based MMD distances for $L_{inv}$ and $L_{seg}$, and with the non-negative risk estimator of Eq. (5) for $L_{osd}$. Together the bound and its finite-sample refinement tell the method what to optimize and why.","core_discovery":"On its own terms, the paper claims that target-domain error in OSHeDA obeys a three-term decomposition. With $\\lambda=P_t(Y\\in\\mathcal{Y}_s)$ the share of target samples from known classes and $C$ the loss bound, Theorem 1 gives $E(P_t,h\\circ f_t)\\le \\lambda E(P_s,h\\circ f_s)+\\bigl(E(P^u_t,h\\circ f_t)-\\lambda E(P^u_s,h\\circ f_s)\\bigr)+\\sqrt{2\\lambda C}\\bigl(D_{JS}(P_s(Z)\\|P_{t,k}(Z))^{1/2}+D_{JS}(P_s(Z,Y)\\|P_{t,k}(Z,Y))^{1/2}\\bigr)$. The three terms are the source error, the open-set difference, and the known-class domain distance in representation space. The proposed method, Representation Learning for OSHeDA (RL-OSHeDA), is constructed to minimize exactly these quantities: classification loss on labeled data, MMD alignment of marginal and label-conditioned known-class representations, a segregation term for the unknown class, and a non-negative risk estimator whose optimum is the open-set difference. The finite-sample version of the bound adds a pseudo-label noise term, which the two-stage pseudo-labeling procedure is designed to reduce. The empirical claim is that this combination outperforms prior HeDA, OSDA, and OS-SSL methods on all 56 evaluated tasks.","pith_inferences":["A testable extension the paper leaves implicit: the three-term decomposition predicts that RL-OSHeDA's margin over baselines should grow as the unknown-class fraction $1-\\lambda$ increases and as feature-space heterogeneity grows, because both enlarge the open-set difference and the JS-distance terms; sweeping $\\lambda$ on a fixed benchmark would check this.","The theory collapses all unknown classes into one 'unk' label; extending the bound to multiple unknown classes with their own conditional distributions would change the open-set difference term and likely require per-class segregation terms.","Equation (5) is the only component that estimates the open-set difference without unknown-class labels; on a synthetic task with oracle unknown labels, comparing its value with the true open-set difference would show how tightly the finite-sample motivation tracks the implemented loss.","Because open-set domain adaptation and positive-unlabeled learning are special cases of the same decomposition, the analysis could serve as a template for unifying those neighboring problems."],"forward_implications":["If Theorem 1 is right, a successful OSHeDA method must simultaneously reduce source error, control the open-set difference, and align known-class marginal and joint distributions; dropping any one of these terms should raise target error.","Because the joint-distribution term includes the target label distribution, labeled target data or high-quality pseudo-labels are necessary in OSHeDA; unlabeled target data alone cannot minimize the bound.","The lower bound implies that separating unknown-class representations from known-class ones is a requirement, not a heuristic, because it directly prevents the lower bound on target error from growing.","The finite-sample bound shows pseudo-label noise enters additively, so the two-stage pseudo-labeling scheme is not an engineering convenience but a way to make the bound tighter.","The reported consistent ranking advantage over eight baselines, with pairwise significance p-values below 0.05 on 56 tasks, is the empirical consequence predicted by the theory."],"supporting_citations":[{"why":"Supplies the domain-adaptation learning-bound framework and the bounded-loss assumption that Theorem 1 extends to open-set heterogeneous settings.","marker":"Ben-David et al. 2010"},{"why":"Source of the bounded-loss and discrepancy style of DA guarantee that the OSHeDA upper bound adapts.","marker":"Mansour, Mohri, and Rostamizadeh 2009"},{"why":"Prior open-set domain adaptation bound whose 'open-set difference' term the paper shows is a special case of Theorem 1's second term.","marker":"Fang et al. 2020"},{"why":"Non-negative risk estimator for positive-unlabeled learning that Eq. (5) adapts to keep the open-set difference from going excessively negative.","marker":"Kiryo et al. 2017"},{"why":"GAN theory from which Proposition 2 derives the finite-data rate for minimizing JS divergence.","marker":"Biau et al. 2020"},{"why":"Defines the Natarajan dimension used in Theorem 2's finite-sample complexity terms.","marker":"Natarajan 1989"},{"why":"Grounds the choice to align label-conditioned distributions rather than joint distributions in the invariant-representation loss.","marker":"Pham, Zhang, and Zhang 2023"}],"fun_headline_variants":["New theory and algorithm for open-set heterogeneous domain adaptation","Handling feature shift and unknown classes in domain adaptation","Model exceeds baselines on 56 open-set heterogeneous tasks","New error bound for open-set heterogeneous domain adaptation","Splitting target error into three terms for open-set DA"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The finite-sample guarantee depends on being able to measure the classifier's error on the very unknown-class target samples that come unlabeled, and the algorithm's substitute estimator only matches that measure under a condition the setup does not guarantee.","fun_headline_variants_meta":{"raw":{"variants":["New theory and algorithm for open-set heterogeneous domain adaptation","Handling feature shift and unknown classes in domain adaptation","Model exceeds baselines on 56 open-set heterogeneous tasks","New error bound for open-set heterogeneous domain adaptation","Splitting target error into three terms for open-set DA"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000888,"raw_usage":{"total_tokens":3890,"prompt_tokens":1064,"completion_tokens":2826,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":680,"completion_tokens_details":{"reasoning_tokens":2748}},"tokens_in":680,"tokens_out":2826,"duration_ms":20514,"temperature":1.0,"reasoning_tokens":2748,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-11T13:29:42.028358+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run a synthetic OSHeDA task with oracle labels for unknown-class target samples, train a variant of RL-OSHeDA that uses those oracle labels to compute the true open-set difference, and compare its target error against the version using the non-negative risk estimator in Eq. (5); if the two diverge when the unknown-class loss on known samples is nonzero, the finite-sample guarantee's reliance on unavailable empirical errors is exposed.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Prior open-set domain adaptation bound whose 'open-set difference' term the paper shows is a special case of Theorem 1's second term."},{"cited_title":"C.; and Sugiyama, M","cited_arxiv_id":null,"evidence_quote":"Non-negative risk estimator for positive-unlabeled learning that Eq. (5) adapts to keep the open-set difference from going excessively negative."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"GAN theory from which Proposition 2 derives the finite-data rate for minimizing JS divergence."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Defines the Natarajan dimension used in Theorem 2's finite-sample complexity terms."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Grounds the choice to align label-conditioned distributions rather than joint distributions in the invariant-representation loss."}],"review_version":1}