Pith. sign in

REVIEW 4 major objections 5 minor 34 references

SCoDA: Self-supervised Continual Domain Adaptation

T0 review · 4 major / 5 minor · reviewed 2026-08-04 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read SCoDA claims that replacing supervised source pretraining with a self-supervised BYOL teacher and adding a Space Similarity Loss that preserves latent geometry yields strong source-free domain adaptation, with 86.59% average accuracy on thr

desk verdict The paper's headline claim of state-of-the-art performance is contradicted by its own results tables; the SSL-initialization idea is worth exploring, but this manuscript is not ready for peer review. read the letter →

arxiv 2509.09935 v1 pith:VVVOQGKR submitted 2025-09-12 cs.CV

Chirayu Agrawal , Snehasis Mukherjee This is my paper · ORCID
classification cs.CV
keywords Source-freedomainadaptationSelf-supervisedlearningKnowledgedistillationGeometricmanifoldalignmentTeacher-studentframeworkBYOLSpacesimilaritylossCatastrophicforgetting
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

SCoDA is a source-free domain adaptation (SFDA) framework that discards the standard supervised-pretrained source model and instead starts from a teacher pretrained entirely with self-supervised learning (BYOL). The student adapts to unlabeled target data by minimizing a composite loss that combines instance-level cosine matching with a Space Similarity Loss, which aligns the transposed feature matrices to preserve the teacher's latent manifold geometry. The teacher is updated as a slow EMA of the student, preventing catastrophic forgetting. The paper reports 86.59% average accuracy on three Office-Home adaptation tasks (Ar→Cl, Ar→Pr, Ar→Rw) and 37.68% on a DomainNet I→C task, and argues these results support the two departures: label-agnostic initialization and geometric manifold alignment.

What carries the argument

The composite loss L_total = L_cos + λ L_space: L_cos is the standard L2-normalized per-sample cosine similarity between teacher and student features, and L_space is the Space Similarity Loss, which applies cosine similarity to the transposed feature matrices (d×B), comparing each feature dimension's activation pattern over the batch. The teacher is a slowly evolving exponential moving average (EMA) of the student, providing a stable target that mitigates catastrophic forgetting.

What would settle it

A decisive test: evaluate SCoDA on all 12 Office-Home domain pairs (the paper only reports Ar→Cl, Ar→Pr, Ar→Rw) and compare the average to the three-task subset. A second test: initialize the teacher with a supervised ImageNet-pretrained ResNet-50 (with its classifier) and run the same adaptation; if the supervised start performs equally well, the paper's claim that self-supervised initialization is key would be undercut.

Watch

Extended reading notes

Core claim

The central claim is that two modifications to the SFDA recipe—starting with a self-supervised rather than supervised teacher, and preserving the teacher's feature-space geometry rather than only matching per-sample L2-normalized vectors—lead to better adaptation and lower catastrophic forgetting. The Space Similarity Loss (adapted from Singh et al., 2022) is the pivotal mechanism: instead of comparing individual feature vectors, it compares the rows of the transposed feature matrices, i.e., the activation of each feature dimension across the mini-batch. This captures the 'internal grammar' of the latent space. In an EMA teacher-student loop, the student learns to match both instance identit

Load-bearing premise

The load-bearing premise is that an ImageNet-pretrained BYOL model, without any source-domain training, can serve as the teacher for Office-Home and DomainNet, and that the reported accuracies are comparable to methods that train on labeled source data—a premise the paper does not fully substantiate because it never specifies how a classifier for 65 or 345 classes is attached.

Editorial extensions

If this is right

  • SFDA can be performed without any labeled source data, removing the need for source supervision.
  • Preserving latent geometry via L_space avoids the information loss caused by L2-normalized instance matching.
  • The EMA teacher-student design yields near-zero source forgetting (only 0.11% drop on DomainNet I→I), balancing plasticity and stability.
  • The method transfers across CNN-based SSL initializations (BYOL, SimCLR) but fails with transformer-based DINO, defining a compatibility frontier.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • If an ImageNet-pretrained SSL encoder suffices as the 'source,' SFDA becomes closer to test-time adaptation, and the term 'source-free' could be reinterpreted as 'label-free' rather than 'access-free.'
  • The reported 86.59% average is computed on only three of the twelve Office-Home direction pairs; a full 12-task evaluation might change the ranking and should be the next check.
  • The DINO failure suggests the space-similarity loss may be incompatible with representations built on prototypes or centering/sharpening; adding such stabilization could extend the method to transformers.
  • Comparing accuracies against supervised-source baselines (e.g., SHOT, AaD with Co-learn) is complicated by the different classifier protocols; a controlled comparison using the same classifier head would clarify the source of the gains.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes SCoDA, a source-free domain adaptation method that replaces supervised source pretraining with a self-supervised (BYOL) initialization and adapts a teacher-student framework with an EMA update rule. The student is trained on unlabeled target data with a composite loss combining cosine feature similarity (Eq. 1) and a Space Similarity Loss (Eq. 2) imported from unsupervised knowledge distillation. The authors claim, in the abstract and introduction, that SCoDA 'significantly outperforms state-of-the-art SFDA methods.' Experiments are reported on two tasks from Office-Home and one from DomainNet, along with ablations over SSL initializations and loss components. The paper's own results, however, show SCoDA below several published baselines on the same benchmarks.

Significance. The idea of replacing supervised source training with a self-supervised initialization and preserving geometric manifold structure during source-free adaptation is interesting and could be a useful direction for the SFDA community. The paper also has some strengths: it explicitly reports the pre-adaptation versus post-adaptation performance, it includes an ablation of the two loss terms, and it documents a catastrophic forgetting failure mode with DINO, which is informative. However, the central empirical claim is contradicted by the paper's own tables. The evaluation protocol is also under-specified, and the benchmark coverage is too limited to support the stated conclusions. If the method were properly evaluated and the claims scaled back, the core idea might merit further study, but as presented the evidence does not support the headline result.

major comments (4)
  1. [Abstract and Section 5.3.1] The abstract and introduction claim that SCoDA 'significantly outperforms state-of-the-art SFDA methods.' This is directly contradicted by the paper's own results. Section 5.3.1 states that SCoDA 'falls short of the state-of-the-art ViLAaD++ method, which achieves 90.23% average accuracy,' while Table 1 reports SCoDA at 86.59% on the same three Office-Home tasks. The claim is therefore not supported by the evidence presented, and this is a load-bearing issue for the paper's central contribution.
  2. [Section 5.3.2 and Table 2] On DomainNet I→C, Table 2 shows SCoDA achieves 37.68%, while the AaD baseline achieves 47.12%. The text acknowledges a gap of approximately 9.44 percentage points. This further contradicts the abstract's claim of state-of-the-art performance. Since the paper's own numbers place SCoDA behind multiple existing methods, the empirical case for the method is not made.
  3. [Sections 3.1.1 and 4.2] There is a critical inconsistency in the initialization protocol. Section 3.1.1 says the model is 'pre-trained on the source domain using a self-supervised learning objective,' but Section 4.2 says the backbone is 'pretrained on ImageNet.' These are different data distributions, and the paper never states how a classifier for 65 or 345 classes is attached to the SSL feature extractor. Without a clear description of the classifier head and how it is initialized or trained, the reported accuracies are difficult to compare with SFDA methods that use a supervised source classifier. This undermines the claim that the method performs SFDA in the standard sense.
  4. [Table 1 and Section 1] The first claimed contribution—that replacing supervised source pretraining with SSL is beneficial—is not tested. The paper compares SCoDA (with BYOL initialization) against published SFDA methods that use supervised source models, but it does not include an ablation where SCoDA is initialized with a supervised source model and run with the same adaptation procedure. Such a controlled comparison is necessary to support the claim that the SSL initialization, rather than the adaptation loss or EMA framework, is responsible for the observed results. Without this baseline, the contribution is not validated.
minor comments (5)
  1. [General] The paper has many typos and formatting issues, including inconsistent capitalization of 'SCoDA'/'SCODA', incomplete ACM CCS placeholder text ('Do Not Use This Code'), and inconsistent reference formatting (some entries use full author names, others initials). These should be cleaned up.
  2. [Figure 4] The confusion matrices are not legible in the provided figure and are not described with class labels. This makes the qualitative analysis very difficult to follow.
  3. [Equation (2)] The Space Similarity Loss is said to be 'adapted from [16]', but the adaptation to the SFDA setting is not discussed in detail. In particular, the role of the teacher's feature space versus the student's feature space, and whether the loss is computed on all dimensions or a subset, would benefit from clarification.
  4. [Algorithm 1] The teacher's BatchNorm statistics are updated via EMA, but the exact procedure and the momentum used are not specified. The main EMA momentum m is given in Section 4.2, but the BN-specific update is not described.
  5. [Table 2] The domain pair I→C is not expanded in the text; Section 5.3.2 refers to 'Infograph→Clipart' but the dataset description only defines DomainNet broadly. A clear statement of the domain names would improve readability.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity found: the method combines external, published losses and an EMA teacher–student update without fitting any parameter to the target result or importing the central claim from self-citations.

full rationale

SCoDA's derivation chain is not circular. The proposed loss (Eq. 1 and Eq. 2) is a direct adoption of the cosine feature similarity and the Space Similarity loss from Singh et al. [16], an external source not authored by the present authors. The EMA teacher–student update (Eq. 5) is the standard Mean Teacher rule, attributed to Tarvainen et al. [23] and CoSDA [4], also external. The hyperparameters λ and m are set by hand (Section 4.2: λ = 1.0, m = 0.999) and are balancing coefficients, not fitted to the reported accuracies. The 'pre-adaptation' baseline is computed from the same SSL-initialized model before adaptation, and 'post-adaptation' is after the defined optimization loop; the improvement is a before/after measurement, not a prediction constructed from the same data. There are no self-citations by Agrawal/Mukherjee in the reference list, so no self-citation load-bearing step exists. The abstract's claim of 'significantly outperforms state-of-the-art SFDA methods' is contradicted by the paper's own tables (e.g., Section 5.3.1 admits SCoDA falls short of ViLAaD++ on Office-Home, and Section 5.3.2 admits it falls short of AaD on DomainNet I→C), but that is an internal empirical inconsistency, not circular reasoning. Similarly, the discrepancy between 'pre-trained on the source domain using SSL' (Section 3.1.1) and 'backbone pretrained on ImageNet' (Section 4.2) is a protocol/correctness concern, not a circularity. Accordingly, the circularity score is 0.

Assumptions & free parameters 2 free parameters · 3 assumptions · 0 invented entities

The paper introduces no new entities. Its load-bearing assumptions are all domain assumptions about the evaluation protocol: that ImageNet SSL pretraining substitutes for a source-domain model, that a classifier is available without explanation, and that a loss designed for unsupervised knowledge distillation works unchanged for source-free domain adaptation. The only free hyperparameters are lambda and m, both set by hand.

free parameters (2)
  • lambda (space loss weight) = 1.0
    Set by hand in Section 4.2 with no sensitivity analysis shown.
  • EMA momentum m = 0.999
    Set by hand in Section 4.2 with no sensitivity analysis shown.
assumptions (3)
  • domain assumption An ImageNet-pretrained SSL model is a valid source model for Office-Home and DomainNet SFDA even though the benchmark source domain (e.g., Art) is not used.
    Section 3.1.1 says the model is pre-trained on the source domain using SSL, but Section 4.2 uses a ResNet-50 pretrained on ImageNet. No argument shows these coincide.
  • domain assumption A classifier for the 65 or 345 target classes can be attached to the SSL feature extractor without breaking the source-free protocol.
    The paper never describes how predictions are produced from a BYOL feature extractor or where the classifier head comes from.
  • domain assumption The Space Similarity loss from [16] transfers unchanged to the SFDA setting.
    Equation (2) is imported verbatim from [16] with no derivation or analysis of its behavior under large domain shift in Section 3.2.2.

how reviews work

0 comments
Cite this review

Pith. "Pith review of SCoDA: Self-supervised Continual Domain Adaptation." pith.science (2026). https://pith.science/paper/VVVOQGKR

@misc{pith2026250909935,
  author       = {Pith},
  title        = {Pith review of: SCoDA: Self-supervised Continual Domain Adaptation},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/VVVOQGKR}},
  note         = {Machine review of arXiv:2509.09935}
}
read the original abstract

Source-Free Domain Adaptation (SFDA) addresses the challenge of adapting a model to a target domain without access to the data of the source domain. Prevailing methods typically start with a source model pre-trained with full supervision and distill the knowledge by aligning instance-level features. However, these approaches, relying on cosine similarity over L2-normalized feature vectors, inadvertently discard crucial geometric information about the latent manifold of the source model. We introduce Self-supervised Continual Domain Adaptation (SCoDA) to address these limitations. We make two key departures from standard practice: first, we avoid the reliance on supervised pre-training by initializing the proposed framework with a teacher model pre-trained entirely via self-supervision (SSL). Second, we adapt the principle of geometric manifold alignment to the SFDA setting. The student is trained with a composite objective combining instance-level feature matching with a Space Similarity Loss. To combat catastrophic forgetting, the teacher's parameters are updated via an Exponential Moving Average (EMA) of the student's parameters. Extensive experiments on benchmark datasets demonstrate that SCoDA significantly outperforms state-of-the-art SFDA methods.

Figures

Figures reproduced from arXiv: 2509.09935 by the authors.

Figure 1
Figure 1. The core contribution of SCODA. We replace biased, [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. A detailed schematic of the proposed SCoDA framework. Stage 1 (Initialization): The process begins by leveraging a [PITH_FULL_IMAGE:figures/full_fig_p004_2.png] view at source ↗
Figure 3
Figure 3. A detailed view of the Teacher-Student adaptation dynamics in SCODA.An input mini-batch (X) is processed by both [PITH_FULL_IMAGE:figures/full_fig_p005_3.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Confusion matrices of the SCoDA-adapted BYOL model evaluated on both source and target domains, demonstrating [PITH_FULL_IMAGE:figures/full_fig_p008_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

34 extracted references · 2 linked inside Pith

  1. [1]

    D. Chen, D. Wang, T. Darrell, and S. Ebrahimi. 2022. Contrastive Test-Time Adaptation. InCVPR. IEEE, 295–305

    2022

  2. [2]

    Diamant, A

    I. Diamant, A. Rosenfeld, I. Achituve, J. Goldberger, and A. Netzer. 2024. De- confusing Pseudo-Labels in Source-Free Domain Adaptation. InECCV. Springer, 108–125

    2024

  3. [3]

    N. Ding, Y. Xu, Y. Tang, S. Xu, Y. Wang, and D. Tao. 2022. Source-Free Domain Adaptation via Distribution Estimation. InCVPR. IEEE, 7212–7222

    2022

  4. [4]

    Haozhe Feng, Zhaorui Yang, Hesun Chen, Tianyu Pang, Chao Du, Minfeng Zhu, Wei Chen, and Shuicheng Yan. 2023. CoSDA: Continual Source-Free Domain Adaptation. https://arxiv.org/abs/2304.06627

    arXiv 2023

  5. [5]

    Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhao- han Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, and Michal Valko

    Jean-Bastien Grill, Florian Strub, Florent Altché, Corentin Tallec, Pierre H. Richemond, Elena Buchatskaya, Carl Doersch, Bernardo Avila Pires, Zhao- han Daniel Guo, Mohammad Gheshlaghi Azar, Bilal Piot, Koray Kavukcuoglu, Rémi Munos, and Michal Valko. 2020. Bootstrap your own latent: A new ap- proach to self-supervised learning. InNeurIPS. Advances in Ne...

    2020

  6. [6]

    Le, Zhicheng Ding, Qing Tian, and Truong-Son Hy

    Luu Tung Hai, Thinh D. Le, Zhicheng Ding, Qing Tian, and Truong-Son Hy. 2025. Topology-Guided Knowledge Distillation for Efficient Point Cloud Processing. https://arxiv.org/pdf/2505.08101

    arXiv 2025

  7. [7]

    Y. Kim, D. Cho, K. Han, P. Panda, and S. Hong. 2021. Domain Adaptation Without Source Data.IEEE Transactions on Artificial Intelligence2, 6 (2021), 508–518

    2021

  8. [8]

    S. Lee, D. Jung, J. Yim, and S. Yoon. 2022. Confidence Score for Source-Free Unsupervised Domain Adaptation. InICML. PMLR

    2022

Show all 34 references
  1. [9]

    L. Li, Y. Gu, X. Pu, J. Li, R. Pu, C. Ling, A. J. McLeod, and B. Wang. 2023. When Source-free Domain Adaptation Meets Learning with Noisy Labels. InICLR

    2023

  2. [10]

    Liang, D

    J. Liang, D. Hu, and J. Feng. 2020. Do We Really Need to Access the Source Data? Source Hypothesis Transfer for Unsupervised Domain Adaptation. In ICML. PMLR

    2020

  3. [11]

    Liang, D

    J. Liang, D. Hu, Y. Wang, R. He, and J. Feng. 2022. Source-Data Absent Unsuper- vised Domain Adaptation Through Hypothesis Transfer and Labeling Transfer. IEEE Transactions on Pattern Analysis and Machine Intelligence44, 11 (2022), 8602–8617

    2022

  4. [12]

    M. Lyu, T. Hao, X. Hu, H. Chen, Z. Lin, Z. Han, and G. Ding. 2024. Learn from the Learnt: Source-Free Active Domain Adaptation via Contrastive Sampling and Visual Persistence. InECCV. Springer, 228–246

    2024

  5. [13]

    Roy Miles, Ismail Elezi, and Jiankang Deng. 2024. VkD: Improving Knowledge Distillation using Orthogonal Projections. InCVPR. IEEE, 15720–15730

    2024

  6. [14]

    Mitsuzumi, A

    Y. Mitsuzumi, A. Kimura, and H. Kashima. 2024. Understanding and Improving Source-Free Domain Adaptation from a Theoretical Perspective. InCVPR. IEEE, 28515–28524

    2024

  7. [15]

    X. Peng, K. Bai, X. Li, Z. Dou, C. Li, and J. Feng. 2019. Moment matching for multi-source domain adaptation. InICCV. IEEE, 1406–1415

    2019

  8. [16]

    Singh and H

    A. Singh and H. Wang. 2022. Simple unsupervised knowledge distillation with space similarity. InECCV Workshops. Springer, 147–164

    2022

  9. [17]

    Y. Song, T. S. Kim, L. Nam, T. Kooi, and C. Yoo. 2024. Is User Feedback Always Informative? Retrieval Latent Defending for Semisupervised Domain Adaptation without Source Data. InECCV. Springer, 72–92

    2024

  10. [18]

    Szatkowski, M

    F. Szatkowski, M. Pyla, M. Przewięźlikowski, S. Cygert, B. Twardowski, and T. Trzciński. 2024. Adapt Your Teacher: Improving Knowledge Distillation for Exemplar-free Continual Learning. InW ACV. IEEE, 1977–1987

    2024

  11. [19]

    S. Tang, A. Chang, K. Zhang, Z. Zhu, M. Ye, and C. Zhang. 2024. Source-Free Domain Adaptation via Target Prediction Distribution Searching. InICCV. IEEE

    2024

  12. [20]

    S. Tang, Y. Shi, Z. Ma, J. Li, J. Lyu, Q. U, and J. Zhang. 2022. Model Adaptation through Hypothesis Transfer with Gradual Knowledge Distillation. InIROS. IEEE

    2022

  13. [21]

    S. Tang, W. Su, M. Ye, and X. Zhu. 2024. Source-Free Domain Adaptation with Frozen Multimodal Foundation Model. InCVPR. IEEE, 23711–23720

    2024

  14. [22]

    S. Tang, Y. Zou, Z. Song, J. Lyu, C. Chen, M. Ye, S. Zhong, and J. Zhang. 2022. Semantic Consistency Learning on Manifold for Source Datafree Unsupervised Domain Adaptation.Neural Networks152 (2022), 467–478

    2022

  15. [23]

    Tarvainen and H

    A. Tarvainen and H. Valpola. 2017. Mean teachers are better role models: Weight- averaged consistency targets for semi-supervised learning. InNeurIPS. Advances in Neural Information Processing Systems, 1195–1204

    2017

  16. [24]

    Venkateswara, J

    H. Venkateswara, J. Eusebio, S. Chakraborty, and S. Panchanathan. 2017. Deep hashing network for unsupervised domain adaptation. InCVPR. IEEE, 5018–5027

    2017

  17. [25]

    H. Xia, S. Xia, and Z. Ding. 2024. Discriminative Pattern Calibration Mechanism for Source-Free Domain Adaptation. InCVPR. IEEE

    2024

  18. [26]

    H. Xia, H. Zhao, and X. Ding. 2021. Adaptive Adversarial Network for Source-Free Domain Adaptation. InICCV. IEEE

    2021

  19. [27]

    B. Xing, R. Yang, R. Guo, J. Shi, and W. Yue. 2024. Hierarchical Unsupervised Relation Distillation for Source Free Domain Adaptation. InECCV. Springer, 393–409

    2024

  20. [28]

    S. Yang, Y. Wang, J. van de Weijer, L. Herranz, and S. Jui. 2021. Generalized Source-Free Domain Adaptation. InICCV. IEEE

    2021

  21. [29]

    S. Yang, Y. Wang, K. Wang, S. Iui, and J. van de Weijer. 2021. Exploiting the Intrinsic Neighborhood Structure for Source-free Domain Adaptation. InNeurIPS. Advances in Neural Information Processing Systems, 29393–29405

    2021

  22. [30]

    S. Yang, Y. Wang, K. Wang, S. Iui, and J. van de Weijer. 2022. Attracting and Dispersing: A Simple Approach for Source-free Domain Adaptation. InNeurIPS. Advances in Neural Information Processing Systems, 5802–5815

    2022

  23. [31]

    Zhang, L

    L. Zhang, L. Shen, and C-S. Foo. 2025. Source-Free Domain Adaptation Guided by Vision and Vision-Language Pre-Training.International Journal of Computer Vision133 (2025), 844–866

    2025

  24. [32]

    Zhang, W

    Z. Zhang, W. Chen, H. Cheng, Z. Li, S. Li, L. Lin, and G. Li. 2022. Divide and Contrast: Source-free Domain Adaptation via Adaptive Contrastive Learning. In NeurIPS. Advances in Neural Information Processing Systems, 5137–5149

    2022

  25. [33]

    Z. Zhou, Y. Shi, Z. Ma, J. Li, J. Lyu, Q. U, and J. Zhang. 2021. Source-free Domain Adaptation via Avatar Prototype Generation and Adaptation. InIJCAI. IJCAI, 2921–2927

    2021

  26. [34]

    Ronghang Zhu, Mengxuan Hu, Weiming Zhuang, Lingjuan Lyu, Xiang Yu, and Sheng Li. 2025. Revisiting Source-Free Domain Adaptation: Insights into Repre- sentativeness, Generalization, and Variety. InCVPR. IEEE, 25688–25697. Received 20 February 2007; revised 12 March 2009; accept...

    2025

Pith tools

Reviewed August 4, 2026 · model on record in the stance chip above.