Pith. sign in

REVIEW 4 major objections 4 minor 25 references

Weighting each one-vs-all classifier's open-set entropy by the closed-set classifier's class probability per target sample focuses universal domain adaptation on the most relevant class boundaries, improving consistency.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-04 19:49 UTC pith:XROLIXRN

load-bearing objection A modest, honestly-described tweak to MLNet's OEM loss; the mechanism is plausible but the empirical win claim is overstated and unquantified. the 4 major comments →

arxiv 2509.09006 v1 pith:XROLIXRN submitted 2025-09-10 cs.CV

E-MLNet: Enhanced Mutual Learning for Universal Domain Adaptation with Sample-Specific Weighting

Samuel Felipe dos Santos , Tiago Agostinho de Almeida , Jurandy Almeida This is my paper
classification cs.CV
keywords universal domain adaptationopen-set entropy minimizationone-vs-all classifierssample-specific weightingclosed-set classifiermutual learningH-scoreopen-partial domain adaptation
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

In universal domain adaptation, a model must classify target samples that share classes with the labeled source while also flagging target samples from novel classes as unknown. Existing mutual-learning methods, such as MLNet, adapt a bank of one-vs-all 'open-set' classifiers by minimizing the average binary entropy over all classes, effectively treating every classifier as equally relevant to every target sample. E-MLNet claims this uniform treatment dilutes the learning signal, and proposes to weight each classifier's entropy term by the closed-set classifier's class probability for that sample. The authors report that this single change makes the model more consistent, beating MLNet in 22 of 31 open-partial and 19 of 31 open-set tasks, and reaching the best average H-scores on VisDA-2017 and ImageCLEF. The broader interest is that a cheap, parameter-free weighting of existing losses can sharpen open-set decisions without introducing new training objectives.

Core claim

On the paper's own terms, the central discovery is that the Open-set Entropy Minimization (OEM) loss used to adapt one-vs-all classifiers benefits from a dynamic, sample-specific weighting. Instead of averaging the binary entropy of all K open-set classifiers, E-MLNet multiplies each classifier's entropy term by the closed-set classifier's prediction probability pc(k|x) for that class. This concentrates the gradient on the class boundaries that are most relevant to the current target sample, sharpening the distinction between known classes and unknown 'outlier' classes. The paper's empirical claim is that this modification alone, without changing any other component of MLNet, yields higher o

What carries the argument

The key mechanism is the weighted open-set entropy loss of Eq. (6): for a target sample x, the binary entropy of the k-th one-vs-all classifier O_k is multiplied by the closed-set classifier's softmax probability pc(k|x), then averaged over classes. This turns the uniform OEM loss into a per-sample attention over decision boundaries: when C is confident the sample belongs to class k, the loss is dominated by the inlier/outlier entropy of O_k, while irrelevant classifiers are down-weighted. This weighting is what carries the argument, since it is the only change relative to MLNet.

Load-bearing premise

The load-bearing premise is that the closed-set classifier's softmax predictions reliably indicate which one-vs-all classifiers are relevant for a given target sample, so that when the closed-set classifier is wrong under severe domain shift or early in training, the weighting can steer adaptation toward incorrect boundaries and propagate errors.

What would settle it

A reader could test the claim by constructing a target domain with an extreme shift such that the closed-set classifier's initial predictions are no better than chance; if E-MLNet's H-score then falls below MLNet's, the weighting's reliance on closed-set confidence is the reason. Alternatively, replacing the predicted weights with ground-truth class memberships should produce a large H-score gap if the weighting is the source of the improvements.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • The weighted OEM loss can be swapped into any MLNet-style framework as a drop-in replacement, requiring no architectural changes or new hyperparameters beyond those already used.
  • Because the weighting is per sample, it naturally emphasizes different class boundaries for different target images, which should matter more as the number of source classes grows and uniform averaging becomes noisier.
  • The larger win-rate in OPDA (22/31) than ODA (19/31) suggests the weighting is especially valuable when both source-private and target-private classes are present, a setting where picking the right boundaries is hardest.
  • On VisDA-2017 and ImageCLEF, the weighting lifts average H-scores to the highest among the compared methods, indicating a benefit on large-scale and small-scale benchmarks alike.
  • If the weighting is robust, uniform OEM can be seen as a special case where all closed-set probabilities are equal, making E-MLNet a generalisation of the standard loss.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • We infer that the closed-set softmax weights act as an implicit attention mechanism over the classifier bank, and that making the weights a learned function of the feature representation could relax the dependence on closed-set accuracy.
  • We infer that the same sample-specific weighting could be applied to other per-class losses in the MLNet objective, such as the consistency constraint, possibly compounding the benefit even if the paper does not test this.
  • We infer that the early-training instability the authors flag could be mitigated with a confidence threshold or a curriculum that starts with uniform weights and anneals to weighted ones, since the paper's own analysis suggests unreliable initial predictions are the main failure mode.
  • We infer that a practical diagnostic would be to measure the closed-set classifier's target-domain accuracy during training; the weighting should help most when that accuracy is already above chance, and may need a safeguard when it is not.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

4 major / 4 minor

Summary. The paper proposes E-MLNet, a modification of the MLNet universal domain adaptation method. Instead of averaging the open-set entropy minimization (OEM) loss uniformly over all one-vs-all classifiers, E-MLNet weights each classifier's entropy term by the closed-set classifier's predicted probability for the corresponding class (Eq. 6). The authors argue that this focuses adaptation on class boundaries that are relevant to each target sample, improving robustness and consistency. They evaluate on Office-31, Office-Home, VisDA-2017, and ImageCLEF under OPDA and ODA settings, reporting H-scores and claiming that E-MLNet outperforms MLNet in a majority of individual tasks (22/31 OPDA, 19/31 ODA) and achieves the highest average H-scores on VisDA and ImageCLEF.

Significance. If the empirical claim is established, E-MLNet is a simple and potentially useful improvement over MLNet: it requires no architectural change, only a sample-specific reweighting of an existing term, and the authors release their code. The controlled comparison with MLNet on four standard benchmarks is a strength, as is the use of the standard H-score metric. However, the central claim is empirical and the current evidence is not sufficient. The reported win counts are inaccurate, the average-score statement is contradicted by one of the tables, and all results come from single runs without variance or significance testing. The contribution is incremental but could be acceptable if these issues are fixed; as it stands, the robustness/majority-win claim is not established.

major comments (4)
  1. [Abstract and §IV-D (Tables I–II)] The abstract and conclusion state that E-MLNet outperforms MLNet in 22 of 31 OPDA tasks, but a direct count from Tables I and II gives 20 wins, 9 losses, and 2 ties (Office-31: 2 wins/4 losses; Office-Home: 5 wins/5 losses/2 ties; VisDA: 1 win; ImageCLEF: 12 wins). The 22/31 number is not supported by the tables and must be corrected. Additionally, the unqualified claim that E-MLNet achieves the highest average H-scores on VisDA and ImageCLEF is false in the ODA setting: in Table III, VisDA shows TNT at 71.6, GATE at 70.8, and NCAL at 69.1, while E-MLNet is 66.6. The claim should be restricted to the settings that actually support it.
  2. [§IV-C and Tables I–IV] All reported numbers are single runs, with no error bars, no multiple seeds, and no paired significance tests. Many per-task differences are below 0.5 H-score, e.g., Table I D2A (89.7 vs. 89.4), C2A (73.6 vs. 73.8), and Table III C2A (63.1 vs. 63.0). Such differences are within the run-to-run variability expected for ResNet-50 on these benchmarks. Since the central claim is that E-MLNet is more robust and wins in the majority of tasks, the empirical foundation is not established. The authors should provide at least three seeds with means and standard deviations, and preferably a paired significance test, or explicitly justify why a single run is sufficient.
  3. [§IV-A] The paper omits CDA and PDA results with the justification that 'our proposed modification to the OEM loss does not affect that specific scenario.' This is not self-evident. Equation (6) changes the OEM loss for every target sample, and in PDA the closed-set weighting could alter how source-private one-vs-all classifiers are treated even when there are no target-private classes. Either the CDA/PDA results should be reported, or the paper should provide an argument or ablation showing that the weighting is indeed inert in those settings. Otherwise the 'universal' claim is not fully supported.
  4. [§III-C, Eq. (6), and Conclusion] The weighting mechanism depends critically on the closed-set classifier's softmax predictions pc(k|x) being reliable for target samples. The paper acknowledges in the Conclusion that severe domain shift could cause error propagation, but it does not test this. The hypothesis that MLNet wins in a few tasks because of 'severe domain shifts' is asserted without any per-task shift measure or early-training diagnostic. Given that the paper claims superior robustness, this risk should be evaluated directly—e.g., by ablating the weighting under increasing shift or by monitoring the quality of pc during training.
minor comments (4)
  1. [Tables I and III] The method name is typeset inconsistently: 'MLNET' appears in the Table I and III headers while 'MLNet' is used elsewhere. Please standardize.
  2. [§III-C, Eq. (6)] The weighting uses raw softmax probabilities from the closed-set classifier without temperature scaling or calibration. The paper cites [8] as inspiration; it would be helpful to state explicitly which element of self-calibrated tuning is adopted and whether calibration was considered.
  3. [§IV-D] The sentence 'both MLNet and E-MLNet significantly outperform OVANet' uses 'significantly' in a colloquial sense, since no significance tests are reported. Please rephrase to avoid implying statistical significance.
  4. [§III-C] Equation (6) is written with a minus sign outside the sum; since the binary entropy terms are non-positive (the terms inside the brackets are negative), the overall loss remains non-negative. Clarifying the sign convention would help readability.

Circularity Check

0 steps flagged

No significant circularity: the weighting is a design heuristic evaluated against an external baseline, and no load-bearing step reduces to its inputs.

full rationale

The paper's derivation chain is straightforward: it takes MLNet's Open-set Entropy Minimization (OEM) loss and replaces the uniform average over one-vs-all classifiers with a weighted average using closed-set classifier predictions (Eq. 6). The statement that this weighting 'focuses adaptation on the most relevant class boundaries' is a description of the weighting mechanism itself, not a conclusion derived from the empirical target result. The empirical claim—that E-MLNet outperforms MLNet in most tasks—is not forced by construction: the weighting does not mathematically guarantee a higher H-score, and the experiments compare against an external baseline on standard benchmarks. Hyperparameters are taken from prior work (MLNet and OVANet) rather than fitted to the evaluation metric. The only external citation used as inspiration ([8]) is not self-citational and is not load-bearing. The authors' self-citations ([1], [2], [3]) appear only in related work and are not used to justify the method's validity. The acknowledged limitation—that the weighting depends on closed-set classifier accuracy—is a potential failure mode, not circular reasoning. Any discrepancy in the reported win counts (e.g., 22 vs. 20) or the absence of variance estimates is a correctness/statistical concern, not a circularity concern. Therefore, no circular step is present.

Axiom & Free-Parameter Ledger

4 free parameters · 4 axioms · 0 invented entities

No invented entities. The method's key assumption is that closed-set predictions are a valid weighting signal. Hyperparameters are carried over from MLNet and OVANet. The paper provides no theoretical justification beyond a plausibility argument.

free parameters (4)
  • gamma (OEM loss weight) = 0.1
    Adopted from OVANet [6]; balances the weighted OEM loss in Eq. 6. Not tuned in this paper.
  • beta1 (NIL loss weight) = 0.5
    Adopted from MLNet [7]; weight for neighborhood invariance learning.
  • beta2 (CMM loss weight) = 0.1
    Adopted from MLNet [7]; weight for cross-domain manifold mixup.
  • eta (CC loss weight) = 0.16 (0.32 for VisDA)
    Adopted from MLNet [7]; weight for consistency constraint.
axioms (4)
  • domain assumption A ResNet-50 pretrained on ImageNet provides transferable features for domain adaptation.
    Used as the backbone for all experiments; standard in the field but an empirical assumption.
  • domain assumption The closed-set classifier's softmax predictions pc(k|x) are a meaningful relevance measure for weighting open-set classifiers.
    This is the core hypothesis of E-MLNet; Section III-C states that a confident prediction implies the corresponding classifier is relevant. The paper itself acknowledges this can fail under severe domain shift.
  • domain assumption MLNet's components (NIL, CMM, CC) are improvements over OVANet and remain beneficial when OEM is weighted.
    E-MLNet inherits all MLNet losses; the paper provides no new analysis of these components.
  • domain assumption Standard UniDA protocols and class splits from prior work are appropriate.
    Section IV-A references prior protocols; the evaluation depends on these splits.

pith-pipeline@v1.3.0-alltime-deepseek · 11311 in / 13738 out tokens · 110877 ms · 2026-08-04T19:49:03.502388+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of E-MLNet: Enhanced Mutual Learning for Universal Domain Adaptation with Sample-Specific Weighting." pith.science (2026). https://pith.science/paper/XROLIXRN

@misc{pith2026250909006,
  author       = {Pith},
  title        = {Pith review of: E-MLNet: Enhanced Mutual Learning for Universal Domain Adaptation with Sample-Specific Weighting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/XROLIXRN}},
  note         = {Machine review of arXiv:2509.09006}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

Universal Domain Adaptation (UniDA) seeks to transfer knowledge from a labeled source to an unlabeled target domain without assuming any relationship between their label sets, requiring models to classify known samples while rejecting unknown ones. Advanced methods like Mutual Learning Network (MLNet) use a bank of one-vs-all classifiers adapted via Open-set Entropy Minimization (OEM). However, this strategy treats all classifiers equally, diluting the learning signal. We propose the Enhanced Mutual Learning Network (E-MLNet), which integrates a dynamic weighting strategy to OEM. By leveraging the closed-set classifier's predictions, E-MLNet focuses adaptation on the most relevant class boundaries for each target sample, sharpening the distinction between known and unknown classes. We conduct extensive experiments on four challenging benchmarks: Office-31, Office-Home, VisDA-2017, and ImageCLEF. The results demonstrate that E-MLNet achieves the highest average H-scores on VisDA and ImageCLEF and exhibits superior robustness over its predecessor. E-MLNet outperforms the strong MLNet baseline in the majority of individual adaptation tasks -- 22 out of 31 in the challenging Open-Partial DA setting and 19 out of 31 in the Open-Set DA setting -- confirming the benefits of our focused adaptation strategy.

Figures

Figures reproduced from arXiv: 2509.09006 by Jurandy Almeida, Samuel Felipe dos Santos, Tiago Agostinho de Almeida.

Figure 1
Figure 1. Figure 1: Different settings for Unsupervised Domain Adaptation (UDA) based [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: Overview of E-MLNet. (a) Schematic of our proposed E-MLNet, highlighting the data flow for the source and target domains and the application of [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

25 extracted references · 1 linked inside Pith

  1. [1]

    Tightening classification bound- aries in open set domain adaptation through unknown exploitation,

    L. F. A. Silva, N. Sebe, and J. Almeida, “Tightening classification bound- aries in open set domain adaptation through unknown exploitation,” in Conf. Graphics, Patterns and Images – SIBGRAPI, 2023, pp. 157–162

    2023

  2. [2]

    Beyond the known: Enhancing open set domain adaptation with unknown exploration,

    L. F. A. Silva, S. F. dos Santos, N. Sebe, and J. Almeida, “Beyond the known: Enhancing open set domain adaptation with unknown exploration,”Pattern Recognit. Lett., vol. 189, pp. 265–272, 2025

    2025

  3. [3]

    Improving transferability of domain adaptation networks through domain alignment layers,

    L. F. A. Silva, D. C. G. Pedronette, F. A. Faria, J. P. Papa, and J. Almeida, “Improving transferability of domain adaptation networks through domain alignment layers,” inConf. Graphics, Patterns and Images – SIBGRAPI, 2021, pp. 168–175

    2021

  4. [4]

    Universal domain adaptation through self supervision,

    K. Saito, D. Kim, S. Sclaroff, and K. Saenko, “Universal domain adaptation through self supervision,”NeurIPS, vol. 33, pp. 16 282– 16 292, 2020

    2020

  5. [5]

    Universal domain adaptation,

    K. You, M. Long, Z. Cao, J. Wang, and M. I. Jordan, “Universal domain adaptation,” inCVPR, 2019, pp. 2720–2729

    2019

  6. [6]

    Ovanet: One-vs-all network for universal domain adaptation,

    K. Saito and K. Saenko, “Ovanet: One-vs-all network for universal domain adaptation,” inICCV, 2021, pp. 9000–9009

    2021

  7. [7]

    Mlnet: Mutual learning network with neighborhood invariance for universal domain adaptation,

    Y . Lu, M. Shen, A. J. Ma, X. Xie, and J.-H. Lai, “Mlnet: Mutual learning network with neighborhood invariance for universal domain adaptation,” inAAAI, vol. 38, no. 4, 2024, pp. 3900–3908

    2024

  8. [8]

    Self-calibrated tuning of vision- language models for out-of-distribution detection,

    G. Yu, J. Zhu, J. Yao, and B. Han, “Self-calibrated tuning of vision- language models for out-of-distribution detection,”NeurIPS, vol. 37, pp. 56 322–56 348, 2024

    2024

  9. [9]

    Learning to detect open classes for universal domain adaptation,

    B. Fu, Z. Cao, M. Long, and J. Wang, “Learning to detect open classes for universal domain adaptation,” inECCV, 2020, pp. 567–583

    2020

  10. [10]

    Domain consensus clustering for universal domain adaptation,

    G. Li, G. Kang, Y . Zhu, Y . Wei, and Y . Yang, “Domain consensus clustering for universal domain adaptation,” inCVPR, 2021, pp. 9757– 9766

    2021

  11. [11]

    Learning classifiers of prototypes and reciprocal points for universal domain adaptation,

    S. Hur, I. Shin, K. Park, S. Woo, and I. S. Kweon, “Learning classifiers of prototypes and reciprocal points for universal domain adaptation,” in WACV, 2023, pp. 531–540

    2023

  12. [12]

    On the effectiveness of image rotation for open set domain adaptation,

    S. Bucci, M. R. Loghmani, and T. Tommasi, “On the effectiveness of image rotation for open set domain adaptation,” inECCV, 2020, pp. 422–438

    2020

  13. [13]

    Unsupervised learning of visual features by contrasting cluster assign- ments,

    M. Caron, I. Misra, J. Mairal, P. Goyal, P. Bojanowski, and A. Joulin, “Unsupervised learning of visual features by contrasting cluster assign- ments,”NeurIPS, vol. 33, pp. 9912–9924, 2020

    2020

  14. [14]

    Unified optimal transport framework for universal domain adaptation,

    W. Chang, Y . Shi, H. Tuan, and J. Wang, “Unified optimal transport framework for universal domain adaptation,”NeurIPS, vol. 35, pp. 29 512–29 524, 2022

    2022

  15. [15]

    Evidential neighborhood contrastive learning for universal domain adaptation,

    L. Chen, Y . Lou, J. He, T. Bai, and M. Deng, “Evidential neighborhood contrastive learning for universal domain adaptation,” inAAAI, vol. 36, no. 6, 2022, pp. 6258–6267

    2022

  16. [16]

    Geometric anchor correspondence mining with uncertainty mod- eling for universal domain adaptation,

    ——, “Geometric anchor correspondence mining with uncertainty mod- eling for universal domain adaptation,” inCVPR, 2022, pp. 16 134– 16 143

    2022

  17. [17]

    Neighborhood-based credibility anchor learning for universal domain adaptation,

    W. Su, Z. Han, R. He, B. Wei, X. He, and Y . Yin, “Neighborhood-based credibility anchor learning for universal domain adaptation,”Pattern Recognition, vol. 142, p. 109686, 2023

    2023

  18. [18]

    Adapting visual category models to new domains,

    K. Saenko, B. Kulis, M. Fritz, and T. Darrell, “Adapting visual category models to new domains,” inECCV, 2010, pp. 213–226

    2010

  19. [19]

    Deep hashing network for unsupervised domain adaptation,

    H. Venkateswara, J. Eusebio, S. Chakraborty, and S. Panchanathan, “Deep hashing network for unsupervised domain adaptation,” inCVPR, 2017, pp. 5018–5027

    2017

  20. [20]

    Visda: The visual domain adaptation challenge,

    X. Peng, B. Usman, N. Kaushik, J. Hoffman, D. Wang, and K. Saenko, “Visda: The visual domain adaptation challenge,”CoRR, abs/1710.06924, 2017

    Pith/arXiv arXiv 2017

  21. [21]

    Nearest neighbors distance ratio open-set classifier,

    P. R. Mendes J ´unior, R. M. De Souza, R. d. O. Werneck, B. V . Stein, D. V . Pazinato, W. R. De Almeida, O. A. Penatti, R. d. S. Torres, and A. Rocha, “Nearest neighbors distance ratio open-set classifier,”Machine Learning, vol. 106, no. 3, pp. 359–386, 2017

    2017

  22. [22]

    Separate to adapt: Open set domain adaptation via progressive separation,

    H. Liu, Z. Cao, M. Long, J. Wang, and Q. Yang, “Separate to adapt: Open set domain adaptation via progressive separation,” inCVPR, 2019, pp. 2927–2936

    2019

  23. [23]

    Open set domain adaptation by backpropagation,

    K. Saito, S. Yamamoto, Y . Ushiku, and T. Harada, “Open set domain adaptation by backpropagation,” inECCV, 2018, pp. 153–168

    2018

  24. [24]

    Open set domain adaptation: Theoretical bound and algorithm,

    Z. Fang, J. Lu, F. Liu, J. Xuan, and G. Zhang, “Open set domain adaptation: Theoretical bound and algorithm,”IEEE Trans. Neural Networks and Learning Systems, vol. 32, no. 10, pp. 4309–4322, 2020

    2020

  25. [25]

    Deep anomaly detection with outlier exposure,

    D. Hendrycks, M. Mazeika, and T. Dietterich, “Deep anomaly detection with outlier exposure,” inICLR, 2019

    2019