Pith. sign in

REVIEW 3 major objections 6 minor 2 cited by

Bridging Domain Generalization to Multimodal Domain Generalization via Unified Representations

T0 review · 3 major / 6 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read One shared space lets domain-generalization tricks cross modalities.

desk verdict A genuinely new way to reuse single-modal DG tools in multimodal settings, but the mechanism is confounded and the SOTA gains are often within noise. read the letter →

arxiv 2507.03304 v1 pith:LHEQK5XC submitted 2025-07-04 cs.CV

Hai Huang , Yan Xia , Sashuai Zhou , Hanting Wang , Shulei Wang , Zhou Zhao This is my paper
classification cs.CV
keywords domaingeneralizationmultimodalunifiedrepresentationsupervisedcontrastivelearninginformationdisentanglementMixupJigsawself-supervisionIBN-Net
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper attacks the problem that single-modal domain generalization (DG) methods lose most of their benefit when applied to multimodal data: Mixup, JiGen, and IBN-Net each improve single-modality models clearly, but their gains shrink under joint multimodal training. The authors argue the cause is modality asynchrony—different modalities drift in divergent generalization directions—and propose mapping all modalities into a unified representation space before applying the DG method. The space is built by supervised contrastive learning that pulls same-category samples together across modalities and by a disentanglement loss that separates modal-general from modal-specific information. Inside this space, Mixup, Jigsaw self-supervision, and IBN normalization act on shared category semantics, producing synchronized multimodal improvements. The empirical claim is that this bridge turns multimodal DG into ordinary DG and outperforms the previous multimodal-specific state of the art on EPIC-Kitchens and Human-Animal-Cartoon.

What carries the argument

The load-bearing object is the supervised contrastive decoupling module. For each modality m, a category encoder Φ maps an input to general information z, and a category-agnostic encoder Ψ maps it to specific information z̄; a supervised contrastive loss L_scl aligns z across modalities for same-category samples, a CLUB mutual-information bound L_club pushes z and z̄ apart, and a decoder with reconstruction loss keeps the pair informationally complete. Once this space is learned, each DG method is redefined to act on z alone: UR-Mixup interpolates z and feeds the mixed semantics through the decoder; UR-JiGen builds jigsaw permutations from segments of z; UR-IBN applies instance and batch normalization to z. The unified space is what lets the three methods improve all modalities at once instead of pulling them in conflicting directions.

What would settle it

Compute the mutual information between the general and specific representations on an unseen target domain, or measure same-category alignment of the general representation across modalities on that domain. If the disentanglement collapses there—mutual information rises or category clusters no longer coincide across modalities—the unified space does not transfer, and the method should degrade to or below direct DG transfer.

Watch

Extended reading notes

Core claim

The central claim is that a unified representation—category-aligned across modalities and stripped of domain- and modality-specific detail—is sufficient to make single-modal DG methods effective in the multimodal setting. Concretely, the paper shows that applying Mixup, JiGen, or IBN-Net to the general information alone, rather than to each modality's raw features, yields larger gains than direct transfer and matches or beats models built specifically for multimodal DG. On EPIC-Kitchens, UR-Mixup raises the multi-modality average from 63.42 to 67.79 in the multi-source setting; on HAC, UR-Mixup reaches 73.40 against 72.44 for the prior best. The same recipe also improves single-source multimodal DG and recovers uni-modal performance that joint training usually suppresses.

Load-bearing premise

The whole bridge rests on the assumption that aligning category semantics on the source domains produces a general-information space that stays category-aligned and domain-invariant on unseen target domains; the paper offers no proof of transfer, only benchmark accuracy.

Editorial extensions

If this is right

  • Any single-modal DG method that operates on representations can in principle be ported to multimodal problems by running it on the unified general space, not per modality.
  • Multimodal joint training need not sacrifice uni-modal performance: UR-Mixup's video-only accuracy under joint training surpasses single-modality Mixup training on HAC.
  • The method needs no domain labels, so it applies to single-source multimodal DG as well as multi-source DG.
  • The unified-representation wrapper can be attached to other DG losses; the paper reports consistent gains when adding it to mDSDI and RDM.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A natural test the authors do not run is whether the bridge survives when one modality is much weaker or noisier than the others, since the contrastive alignment would then pull semantics toward the dominant modality.
  • The disentanglement idea may extend to open-set multimodal DG, where unseen categories appear at test time; the general space would need to cluster by semantic similarity rather than by fixed labels.
  • If the unified space genuinely strips domain information, it should also improve cross-modal retrieval or zero-shot classification on unseen domains, consequences not explored in the paper.
  • A theoretical account of why category-aligned contrastive alignment prevents modality competition is missing; without one, the recipe's applicability to other modality triples is an extrapolation.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper proposes URMMDG, a framework that maps paired multimodal inputs into a unified representation by disentangling modal-general and modal-specific information using supervised contrastive learning, CLUB mutual-information minimization, and reconstruction. Single-modal DG methods (Mixup, JiGen, IBN-Net) are then applied to the general information, with the goal of synchronizing multimodal improvement and reformulating multimodal domain generalization as a domain generalization problem. Experiments on EPIC-Kitchens and HAC under multi-source and single-source settings, across bimodal and trimodal combinations, report consistent gains over direct DG baselines and results comparable to or better than SimMMDG and CMRF.

Significance. If the central claim holds, the paper offers a practical bridge between two otherwise separate literatures: existing single-modal DG algorithms could be reused in the multimodal setting by operating on an aligned semantic space, rather than being redesigned per modality. The empirical study is broad in scope—three modalities, two datasets, multiple modality subsets, both multi-source and single-source settings, plus extensions to mDSDI and RDM in the supplementary material. The method is not circular: the losses are standard, the hyperparameters are selected on validation data, and the reported accuracy is a measured outcome. However, the key mechanism—that the unified representation itself, rather than the added auxiliary losses, drives the gains—is not isolated by the current experiments, and the SOTA claims rest on differences that are often near one accuracy point without error bars.

major comments (3)
  1. [§3.2, Eq. (12), Tables 5/10] The load-bearing claim that applying DG methods in the unified representation space enables synchronized MMDG improvement is not isolated from the auxiliary losses. UR-Mixup, UR-JiGen, and UR-IBN each add L_scl, L_club, and L_rec (Eqs. 5, 7, 8) to the final loss in Eq. (12). The ablations in Tables 5 and 10 only vary components inside the UR-Mixup pipeline; they do not include a control that adds these auxiliary losses while applying Mixup, JiGen, or IBN to the original per-modality representations, or to the concatenation of general and specific features. Without this control, the reported gains over the direct DG baselines in Tables 2 and 3 could be caused by the extra losses improving the shared feature extractor rather than by the unified-representation bridge. Please add this control.
  2. [§4.2, Tables 2/3] The empirical SOTA claim lacks error bars and statistical significance tests. Several improvements over CMRF are within roughly one accuracy point (e.g., EPIC V-A-F multi-source: 67.79 vs 67.36 in Table 2; EPIC single-source: 60.84 vs 60.12 and HAC single-source: 64.91 vs 64.09 in Table 3). Without multiple seeds and significance testing, the reader cannot determine whether these differences are stable. Please report mean plus/minus standard deviation over at least three runs and, where appropriate, paired significance tests.
  3. [§4.3, Table 5] The comparison between supervised and unsupervised contrastive learning, and between CLUB and MSE-based decoupling, is currently based on a single target split (D2, D3 -> D1 in EPIC-Kitchens) with no variance information. The text concludes that supervised contrastive learning is better and that CLUB is more effective than MSE, but the evidence is thin and the row labels in Table 5 (UCL, SCL, MID, CID) are not fully defined. Please clarify the table and provide results across all domain splits or report averages with errors.
minor comments (6)
  1. [Eq. (6)] The first expectation in the CLUB definition is missing a closing bracket: Ep(x,y[log qθ(y|x)] should be Ep(x,y)[log qθ(y|x)]. Please fix the notation.
  2. [Eq. (10)] Equation (10) has an extra closing parenthesis: D(˜zm_i, ¯zm_i)) should be D(˜zm_i, ¯zm_i). Please correct it.
  3. [Figure 2 caption] The caption contains typos: 'Supvised Contrastive Learning' should be 'Supervised Contrastive Learning' and 'Minimazation' should be 'Minimization'. Please fix them.
  4. [Table 5] The header abbreviation 'CID' is used for CLUB information decoupling, but Section 3.2 and the text consistently refer to CLUB. Please unify the terminology.
  5. [Supplementary, §7] The sentence 'As shown in Table 8, the results are consistent with those in Table 8' is self-referential and should reference the corresponding main-text table (Table 4) or be rephrased.
  6. [Introduction, first paragraph] The phrase 'an different third category' is a grammatical error; it should be 'a different third category'. Please proofread the text.

Circularity Check

0 steps flagged · score 2.0 of 10

No circular reduction; minor same-author citations are contextual and not load-bearing, so the paper is essentially self-contained.

full rationale

The paper's derivation chain is empirical rather than definitional. The claimed bridge (applying Mixup, JiGen, and IBN-Net to a supervised-contrastive, MI-minimized general representation) is implemented by standard external losses (SupCon, CLUB, MSE reconstruction) and evaluated on held-out target domains of EPIC-Kitchens and HAC; no equation defines the reported accuracy into the loss terms, and no fitted parameter is later renamed as a prediction. Hyperparameters (alpha_i, tau, P, Beta) are fixed or selected on validation, which is model selection, not circularity. The same-author references [15,23,24,25,46,50] appear in related-work and motivation for unified representations, but they are not invoked as a uniqueness theorem or as the proof that the method must work; the core mechanism is tested against external MMDG baselines (SimMMDG, CMRF, RNA-Net). The main weakness is a confound in the ablations: UR-Mixup/UR-JiGen/UR-IBN add L_scl, L_club, and L_rec relative to the direct DG baselines, so the marginal gain is not uniquely attributable to the unified-space bridge. This is an experimental identification issue, not a circular construction, because the UR outputs are not algebraically forced to equal the loss inputs. Since the central claim has independent empirical content and self-citations are not load-bearing, the circularity score is low.

Assumptions & free parameters 7 free parameters · 4 assumptions · 0 invented entities

The method introduces no new physical entities. It relies on standard learning-theoretic assumptions (CLUB bound), a domain assumption about label consistency across modalities, and two ad hoc assumptions about the disentangled general representation being sufficient and Mixup being valid in the unified space.

free parameters (7)
  • alpha_1 = 1.0
    Weight for classification loss in Eq. (12). Tuned on validation.
  • alpha_2 = 2.0
    Weight for supervised contrastive loss. Tuned on validation.
  • alpha_3 = 2.0
    Weight for CLUB mutual information loss. Tuned on validation.
  • alpha_4 = 1.0
    Weight for reconstruction loss. Tuned on validation.
  • temperature tau = 0.1
    Temperature in supervised contrastive loss, set as in prior contrastive works.
  • Mixup Beta parameter = 0.2
    Shape parameter for Beta distribution in UR-Mixup.
  • Jigsaw number P = 256
    Number of permutations in UR-JiGen, selected based on validation performance (Table 6).
assumptions (4)
  • standard math CLUB provides an upper bound on mutual information
    Used in Eq. (6-7) to encourage disentanglement of general and specific information.
  • domain assumption Paired modalities share the same category label set
    Required for supervised contrastive learning across modalities; stated in Section 1.
  • ad hoc to paper General information is sufficient for classification and independent of domain and modality
    The disentanglement assumption that underlies applying DG methods only to general information; not proven, only empirically validated.
  • ad hoc to paper Mixup in the unified representation space yields semantically valid samples
    The paper assumes that linear interpolation of category-aligned general representations produces meaningful augmented samples; motivated by Figure 1 but not formally justified.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Bridging Domain Generalization to Multimodal Domain Generalization via Unified Representations." pith.science (2026). https://pith.science/paper/LHEQK5XC

@misc{pith2026250703304,
  author       = {Pith},
  title        = {Pith review of: Bridging Domain Generalization to Multimodal Domain Generalization via Unified Representations},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/LHEQK5XC}},
  note         = {Machine review of arXiv:2507.03304}
}
read the original abstract

Domain Generalization (DG) aims to enhance model robustness in unseen or distributionally shifted target domains through training exclusively on source domains. Although existing DG techniques, such as data manipulation, learning strategies, and representation learning, have shown significant progress, they predominantly address single-modal data. With the emergence of numerous multi-modal datasets and increasing demand for multi-modal tasks, a key challenge in Multi-modal Domain Generalization (MMDG) has emerged: enabling models trained on multi-modal sources to generalize to unseen target distributions within the same modality set. Due to the inherent differences between modalities, directly transferring methods from single-modal DG to MMDG typically yields sub-optimal results. These methods often exhibit randomness during generalization due to the invisibility of target domains and fail to consider inter-modal consistency. Applying these methods independently to each modality in the MMDG setting before combining them can lead to divergent generalization directions across different modalities, resulting in degraded generalization capabilities. To address these challenges, we propose a novel approach that leverages Unified Representations to map different paired modalities together, effectively adapting DG methods to MMDG by enabling synchronized multi-modal improvements within the unified space. Additionally, we introduce a supervised disentanglement framework that separates modal-general and modal-specific information, further enhancing the alignment of unified representations. Extensive experiments on benchmark datasets, including EPIC-Kitchens and Human-Animal-Cartoon, demonstrate the effectiveness and superiority of our method in enhancing multi-modal domain generalization.

Figures

Figures reproduced from arXiv: 2507.03304 by the authors.

Figure 1
Figure 1. Illustration of conflicts when applying Mixup across [PITH_FULL_IMAGE:figures/full_fig_p001_1.png] view at source ↗
Figure 2
Figure 2. Illustration of the URMMDG framework. The framework first constructs a unified representation through supervised con￾trastive learning and disentanglement with multimodal data, and then applies single-modal DG methods on the unified representation. In subfigure (a), C and D stand for Class and Domain, respectively, and the figure exemplifies our framework with two domains and n classes. (b) Data Manipulation, (c) Le… view at source ↗
Figure 3
Figure 3. Visualization of the learned embeddings using t-SNE [PITH_FULL_IMAGE:figures/full_fig_p007_3.png] view at source ↗
Figures from the paper (2 more)
Figure 4
Figure 4. Figure 4: Parameter sensitivity analysis on HAC with video and audio data [PITH_FULL_IMAGE:figures/full_fig_p013_4.png]
Figure 5
Figure 5. Figure 5: Visualization of the learned embeddings using t-SNE (D2, D3 [PITH_FULL_IMAGE:figures/full_fig_p014_5.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 2 Pith papers

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. TAP: Parameter-efficient Task-Aware Prompting for Adverse Weather Removal

    cs.CV 2025-08 conditional novelty 6.0 of 10

    A two-stage prompt-tuning method with low-rank and contrastive prompt enhancement claims all-in-one adverse weather removal at 2.75M parameters.

  2. Open-set Cross Modal Generalization via Multimodal Unified Representation

    cs.CV 2025-07 conditional novelty 5.0 of 10

    The authors propose OSCMG, an open-set version of Cross Modal Generalization, and show their MICU method with masked contrastive learning and unified jigsaw puzzles outperforms prior methods.

Reference graph

Works this paper leans on

58 extracted references · 36 canonical work pages · cited by 2 Pith papers

  1. [1]

    Robust cross-modal representation learning with progressive self- distillation

    Alex Andonian, Shixing Chen, and Raffay Hamid. Robust cross-modal representation learning with progressive self- distillation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 16430– 16441, 2022. 3

    2022

  2. [2]

    Person30k: A dual-meta general- ization network for person re-identification

    Yan Bai, Jile Jiao, Wang Ce, Jun Liu, Yihang Lou, Xuetao Feng, and Ling-Yu Duan. Person30k: A dual-meta general- ization network for person re-identification. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 2123–2132, 2021. 1

    work page 2021

  3. [3]

    Ex- ploiting domain-specific features to enhance domain gener- alization

    Manh-Ha Bui, Toan Tran, Anh Tran, and Dinh Phung. Ex- ploiting domain-specific features to enhance domain gener- alization. Advances in Neural Information Processing Sys- tems, 34:21189–21201, 2021. 1

    work page 2021

  4. [4]

    Domain generalization by solving jigsaw puzzles

    Fabio M Carlucci, Antonio D’Innocente, Silvia Bucci, Bar- bara Caputo, and Tatiana Tommasi. Domain generalization by solving jigsaw puzzles. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 2229–2238, 2019. 1, 2, 3, 5, 6

    work page 2019

  5. [5]

    Vggsound: A large-scale audio-visual dataset

    Honglie Chen, Weidi Xie, Andrea Vedaldi, and Andrew Zisserman. Vggsound: A large-scale audio-visual dataset. In ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) , pages 721–725. IEEE, 2020. 1

    work page 2020

  6. [6]

    Uniter: Universal image-text representation learning

    Yen-Chun Chen, Linjie Li, Licheng Yu, Ahmed El Kholy, Faisal Ahmed, Zhe Gan, Yu Cheng, and Jingjing Liu. Uniter: Universal image-text representation learning. In European conference on computer vision , pages 104–120. Springer,

  7. [7]

    Club: A contrastive log-ratio up- per bound of mutual information

    Pengyu Cheng, Weituo Hao, Shuyang Dai, Jiachang Liu, Zhe Gan, and Lawrence Carin. Club: A contrastive log-ratio up- per bound of mutual information. In International confer- ence on machine learning, pages 1779–1788. PMLR, 2020. 4, 8

    work page 2020

  8. [8]

    Robustnet: Improving domain generalization in urban-scene segmentation via in- stance selective whitening

    Sungha Choi, Sanghun Jung, Huiwon Yun, Joanne T Kim, Seungryong Kim, and Jaegul Choo. Robustnet: Improving domain generalization in urban-scene segmentation via in- stance selective whitening. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , pages 11580–11590, 2021. 1

    work page 2021

Show all 58 references
  1. [9]

    Openmmlab’s next generation video understanding toolbox and benchmark

    MMAction Contributors. Openmmlab’s next generation video understanding toolbox and benchmark. 2020. 1

    2020

  2. [10]

    Scaling egocentric vision: The epic-kitchens dataset

    Dima Damen, Hazel Doughty, Giovanni Maria Farinella, Sanja Fidler, Antonino Furnari, Evangelos Kazakos, Davide Moltisanti, Jonathan Munro, Toby Perrett, Will Price, et al. Scaling egocentric vision: The epic-kitchens dataset. In Proceedings of the European conference on comput...

    2018

  3. [11]

    Simmmdg: A simple and effective framework for multi-modal domain generalization

    Hao Dong, Ismail Nejjar, Han Sun, Eleni Chatzi, and Olga Fink. Simmmdg: A simple and effective framework for multi-modal domain generalization. Advances in Neural In- formation Processing Systems, 36:78674–78695, 2023. 1, 3, 4, 5, 6, 7

    2023

  4. [12]

    Towards mul- timodal open-set domain generalization and adaptation through self-supervision

    Hao Dong, Eleni Chatzi, and Olga Fink. Towards mul- timodal open-set domain generalization and adaptation through self-supervision. In European Conference on Com- puter Vision, pages 270–287. Springer, 2024. 3

    2024

  5. [13]

    Multi-modal align- ment using representation codebook

    Jiali Duan, Liqun Chen, Son Tran, Jinyu Yang, Yi Xu, Belinda Zeng, and Trishul Chilimbi. Multi-modal align- ment using representation codebook. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15651–15660, 2022. 3

    2022

  6. [14]

    Cross-modal representation flattening for multi-modal do- main generalization

    Yunfeng Fan, Wenchao Xu, Haozhao Wang, and Song Guo. Cross-modal representation flattening for multi-modal do- main generalization. Advances in Neural Information Pro- cessing Systems, 37:66773–66795, 2025. 1, 3, 5, 6, 7

    2025

  7. [15]

    Ace: A generative cross-modal retrieval framework with coarse-to-fine semantic modeling

    Minghui Fang, Shengpeng Ji, Jialong Zuo, Hai Huang, Yan Xia, Jieming Zhu, Xize Cheng, Xiaoda Yang, Wenrui Liu, Gang Wang, et al. Ace: A generative cross-modal retrieval framework with coarse-to-fine semantic modeling. arXiv preprint arXiv:2406.17507, 2024. 3

    2024

  8. [16]

    Slowfast networks for video recognition

    Christoph Feichtenhofer, Haoqi Fan, Jitendra Malik, and Kaiming He. Slowfast networks for video recognition. In Proceedings of the IEEE/CVF international conference on computer vision, pages 6202–6211, 2019. 1

    2019

  9. [17]

    Sharpness-aware minimization for efficiently improving generalization

    Pierre Foret, Ariel Kleiner, Hossein Mobahi, and Behnam Neyshabur. Sharpness-aware minimization for efficiently improving generalization. arXiv preprint arXiv:2010.01412,

    2010 arXiv

  10. [18]

    Domain-adversarial training of neural networks

    Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pas- cal Germain, Hugo Larochelle, Franc ¸ois Laviolette, Mario March, and Victor Lempitsky. Domain-adversarial training of neural networks. Journal of machine learning research , 17(59):1–35, 2016. 1, 3

    2016

  11. [19]

    Imagebind: One embedding space to bind them all

    Rohit Girdhar, Alaaeldin El-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra. Imagebind: One embedding space to bind them all. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 15180–15190, 2023. 2

    2023

  12. [20]

    Learning shared semantic space for speech-to-text translation

    Chi Han, Mingxuan Wang, Heng Ji, and Lei Li. Learning shared semantic space for speech-to-text translation. arXiv preprint arXiv:2105.03095, 2021. 3

    2021 arXiv

  13. [21]

    Mixgen: A new multi- modal data augmentation

    Xiaoshuai Hao, Yi Zhu, Srikar Appalaraju, Aston Zhang, Wanqian Zhang, Bo Li, and Mu Li. Mixgen: A new multi- modal data augmentation. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 379–389, 2023. 3

    2023

  14. [22]

    Deep residual learning for image recognition

    Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceed- ings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 1

    2016

  15. [23]

    Un- locking the potential of multimodal unified discrete represen- tation through training-free codebook optimization and hier- archical alignment

    Hai Huang, Yan Xia, Shengpeng Ji, Shulei Wang, Hanting Wang, Jieming Zhu, Zhenhua Dong, and Zhou Zhao. Un- locking the potential of multimodal unified discrete represen- tation through training-free codebook optimization and hier- archical alignment. arXiv preprint arXiv:2403....

    2024 arXiv

  16. [24]

    Semantic residual for multimodal unified discrete representation

    Hai Huang, Shulei Wang, and Yan Xia. Semantic residual for multimodal unified discrete representation. InICASSP 2025- 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE, 2025. 3

    2025

  17. [25]

    Overcoming both domain shift and label shift for referring video segmentation

    Hai Huang, Sashuai Zhou, and Yan Xia. Overcoming both domain shift and label shift for referring video segmentation. In Findings of the Association for Computational Linguis- tics: NAACL 2025, pages 3058–3069, 2025. 3

    2025

  18. [26]

    Modality competition: What makes joint training of multi-modal network fail in deep learn- ing?(provably)

    Yu Huang, Junyang Lin, Chang Zhou, Hongxia Yang, and Longbo Huang. Modality competition: What makes joint training of multi-modal network fail in deep learn- ing?(provably). In International conference on machine learning, pages 9226–9259. PMLR, 2022. 2, 7

    2022

  19. [27]

    Self-challenging improves cross-domain generalization

    Zeyi Huang, Haohan Wang, Eric P Xing, and Dong Huang. Self-challenging improves cross-domain generalization. In Computer vision–ECCV 2020: 16th European conference, Glasgow, UK, August 23–28, 2020, proceedings, part II 16 , pages 124–140. Springer, 2020. 1, 3

    2020

  20. [28]

    The kinetics hu- man action video dataset

    Will Kay, Joao Carreira, Karen Simonyan, Brian Zhang, Chloe Hillier, Sudheendra Vijayanarasimhan, Fabio Viola, Tim Green, Trevor Back, Paul Natsev, et al. The kinetics hu- man action video dataset. arXiv preprint arXiv:1705.06950,

    arXiv

  21. [29]

    Supervised contrastive learning

    Prannay Khosla, Piotr Teterwak, Chen Wang, Aaron Sarna, Yonglong Tian, Phillip Isola, Aaron Maschinot, Ce Liu, and Dilip Krishnan. Supervised contrastive learning. Advances in neural information processing systems, 33:18661–18673,

  22. [30]

    Learning to generalize: Meta-learning for do- main generalization

    Da Li, Yongxin Yang, Yi-Zhe Song, and Timothy Hospedales. Learning to generalize: Meta-learning for do- main generalization. In Proceedings of the AAAI conference on artificial intelligence, 2018. 1, 3

    2018

  23. [31]

    Domain generalization for med- ical imaging classification with linear-dependency regular- ization

    Haoliang Li, YuFei Wang, Renjie Wan, Shiqi Wang, Tie- Qiang Li, and Alex Kot. Domain generalization for med- ical imaging classification with linear-dependency regular- ization. Advances in neural information processing systems, 33:3118–3129, 2020. 1

    2020

  24. [32]

    Cross- modal discrete representation learning

    Alexander H Liu, SouYoung Jin, Cheng-I Jeff Lai, An- drew Rouditchenko, Aude Oliva, and James Glass. Cross- modal discrete representation learning. arXiv preprint arXiv:2106.05438, 2021. 3

    2021 arXiv

  25. [33]

    Feddg: Federated domain generalization on medical image segmentation via episodic learning in continuous fre- quency space

    Quande Liu, Cheng Chen, Jing Qin, Qi Dou, and Pheng-Ann Heng. Feddg: Federated domain generalization on medical image segmentation via episodic learning in continuous fre- quency space. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition , page...

    2021

  26. [34]

    Unified-io: A unified model for vision, language, and multi-modal tasks

    Jiasen Lu, Christopher Clark, Rowan Zellers, Roozbeh Mot- taghi, and Aniruddha Kembhavi. Unified-io: A unified model for vision, language, and multi-modal tasks. In The Eleventh International Conference on Learning Representa- tions, 2022. 3

    2022

  27. [35]

    Do- main generalisation via risk distribution matching

    Toan Nguyen, Kien Do, Bao Duong, and Thin Nguyen. Do- main generalisation via risk distribution matching. In Pro- ceedings of the IEEE/CVF Winter Conference on Applica- tions of Computer Vision, pages 2790–2799, 2024. 1

    2024

  28. [36]

    Unsupervised learning of visual representations by solving jigsaw puzzles

    Mehdi Noroozi and Paolo Favaro. Unsupervised learning of visual representations by solving jigsaw puzzles. In Euro- pean conference on computer vision, pages 69–84. Springer,

  29. [37]

    Causality-inspired single- source domain generalization for medical image segmenta- tion

    Cheng Ouyang, Chen Chen, Surui Li, Zeju Li, Chen Qin, Wenjia Bai, and Daniel Rueckert. Causality-inspired single- source domain generalization for medical image segmenta- tion. IEEE Transactions on Medical Imaging , 42(4):1095– 1106, 2022. 1

    2022

  30. [38]

    Two at once: Enhancing learning and generalization capacities via ibn-net

    Xingang Pan, Ping Luo, Jianping Shi, and Xiaoou Tang. Two at once: Enhancing learning and generalization capacities via ibn-net. In Proceedings of the european conference on computer vision (ECCV), pages 464–479, 2018. 1, 2, 3, 4, 5, 6

    2018

  31. [39]

    Audio-visual speech recognition with a hybrid ctc/attention architecture

    Stavros Petridis, Themos Stafylakis, Pingchuan Ma, Geor- gios Tzimiropoulos, and Maja Pantic. Audio-visual speech recognition with a hybrid ctc/attention architecture. In 2018 IEEE Spoken Language Technology Workshop (SLT), pages 513–520. IEEE, 2018. 3

    2018

  32. [40]

    Domain generalization through audio- visual relative norm alignment in first person action recog- nition

    Mirco Planamente, Chiara Plizzari, Emanuele Alberti, and Barbara Caputo. Domain generalization through audio- visual relative norm alignment in first person action recog- nition. In Proceedings of the IEEE/CVF winter conference on applications of computer vision, pages 1807–18...

    2022

  33. [41]

    Learn- ing transferable visual models from natural language super- vision

    Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learn- ing transferable visual models from natural language super- vision. In International Conference on Machine Learning...

    2021

  34. [42]

    Domain generalization of 3d semantic segmenta- tion in autonomous driving

    Jules Sanchez, Jean-Emmanuel Deschaud, and Franc ¸ois Goulette. Domain generalization of 3d semantic segmenta- tion in autonomous driving. InProceedings of the IEEE/CVF International Conference on Computer Vision, pages 18077– 18087, 2023. 1

    2023

  35. [43]

    Xkd: Cross-modal knowl- edge distillation with domain alignment for video represen- tation learning

    Pritam Sarkar and Ali Etemad. Xkd: Cross-modal knowl- edge distillation with domain alignment for video represen- tation learning. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 14875–14885, 2024. 3

    2024

  36. [44]

    Domain randomization for transferring deep neural networks from simulation to the real world

    Josh Tobin, Rachel Fong, Alex Ray, Jonas Schneider, Woj- ciech Zaremba, and Pieter Abbeel. Domain randomization for transferring deep neural networks from simulation to the real world. In 2017 IEEE/RSJ international conference on intelligent robots and systems (IROS) , pages 2...

    2017

  37. [45]

    Deep domain confusion: Maximizing for domain invariance

    Eric Tzeng, Judy Hoffman, Ning Zhang, Kate Saenko, and Trevor Darrell. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv:1412.3474 , 2014. 1, 3

    2014 arXiv

  38. [46]

    Irbridge: Solving image restoration bridge with pre-trained generative diffusion mod- els

    Hanting Wang, Tao Jin, Wang Lin, Shulei Wang, Hai Huang, Shengpeng Ji, and Zhou Zhao. Irbridge: Solving image restoration bridge with pre-trained generative diffusion mod- els. arXiv preprint arXiv:2505.24406, 2025. 3

    2025 arXiv

  39. [47]

    Generalizing to unseen domains: A survey on do- main generalization

    Jindong Wang, Cuiling Lan, Chang Liu, Yidong Ouyang, Tao Qin, Wang Lu, Yiqiang Chen, Wenjun Zeng, and S Yu Philip. Generalizing to unseen domains: A survey on do- main generalization. IEEE transactions on knowledge and data engineering, 35(8):8052–8072, 2022. 3, 4

    2022

  40. [48]

    Towards transformer-based aligned generation with self-coherence guidance

    Shulei Wang, Wang Lin, Hai Huang, Hanting Wang, Si- hang Cai, WenKang Han, Tao Jin, Jingyuan Chen, Jiacheng Sun, Jieming Zhu, et al. Towards transformer-based aligned generation with self-coherence guidance. arXiv preprint arXiv:2503.17675, 2025. 3

    2025 arXiv

  41. [49]

    Vlmixer: Unpaired vision-language pre-training via cross-modal cutmix

    Teng Wang, Wenhao Jiang, Zhichao Lu, Feng Zheng, Ran Cheng, Chengguo Yin, and Ping Luo. Vlmixer: Unpaired vision-language pre-training via cross-modal cutmix. In In- ternational Conference on Machine Learning, pages 22680– 22690. PMLR, 2022. 3

    2022

  42. [50]

    Achiev- ing cross modal generalization with multimodal unified rep- resentation

    Yan Xia, Hai Huang, Jieming Zhu, and Zhou Zhao. Achiev- ing cross modal generalization with multimodal unified rep- resentation. Advances in Neural Information Processing Sys- tems, 36, 2024. 2, 3

    2024

  43. [51]

    mixup: Beyond empirical risk minimiza- tion

    Hongyi Zhang, Moustapha Cisse, Yann N Dauphin, and David Lopez-Paz. mixup: Beyond empirical risk minimiza- tion. arXiv preprint arXiv:1710.09412, 2017. 1, 2, 3, 5, 6

    2017 arXiv

  44. [52]

    Towards effective multi-modal interchanges in zero-resource sounding object localization

    Yang Zhao, Chen Zhang, Haifeng Huang, Haoyuan Li, and Zhou Zhao. Towards effective multi-modal interchanges in zero-resource sounding object localization. Advances in Neural Information Processing Systems , 35:38089–38102,

  45. [53]

    Deep domain-adversarial image generation for do- main generalisation

    Kaiyang Zhou, Yongxin Yang, Timothy Hospedales, and Tao Xiang. Deep domain-adversarial image generation for do- main generalisation. In Proceedings of the AAAI conference on artificial intelligence, pages 13025–13032, 2020. 1, 3 Bridging Domain Generalization to Multimodal Dom...

    2020

  46. [54]

    The feature dimensions for video, audio, and optical flow are 2304, 512, and 2048, respec- tively

    Inplementation Details To ensure fair experimental comparisons, we adopt the same modality backbones as SimMMDG [11] and CMRF [14], with our experimental setup based on the MMAction2 toolkit [9]. The feature dimensions for video, audio, and optical flow are 2304, 512, and 2048...

  47. [55]

    In contrast, our proposed approach sub- stantially improves their performance in the MMDG set- ting

    More Experiments More Experiments about Multi-modal single-source DG: As shown in Table 7, directly transferring DG meth- ods [4, 38, 51] to MMDG results in significantly inferior performance compared to models specifically designed for MMDG [11, 14]. In contrast, our proposed...

  48. [56]

    Notably, our method exhibits minimal fluctuations across all parame- ter settings, indicating a lower sensitivity to hyperparameter selection

    Parameter Sensitivity Analysis As shown in Figure 4, we conduct a comprehensive analysis of four loss hyperparameters in UR-Mixup by varying one parameter at a time while keeping the others fixed. Notably, our method exhibits minimal fluctuations across all parame- ter setting...

  49. [57]

    We do not ab- late Lcls since it is essential for classification

    Loss Function Ablation Study As shown in Table 5 of the main paper, we ablate Lscl and Lclub, where rows 1, 3, and 7 correspond to using Lcls only, Lcls+Lscl, and the full objective, respectively. We do not ab- late Lcls since it is essential for classification. Lrec is mean- ...

  50. [58]

    It can be observed that the gen- eral and specific information of each modality are well- separated and consistently aligned across domains

    More Visualization As shown in Figure 5, we provide additional visualizations of the learned embeddings. It can be observed that the gen- eral and specific information of each modality are well- separated and consistently aligned across domains. Fur- thermore, the embeddings o...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.