pith. sign in

arxiv: 2510.14244 · v2 · pith:QGYOHDLQnew · submitted 2025-10-16 · 📡 eess.IV · cs.AI· cs.CV

Reinforcement Learning for Unsupervised Domain Adaptation in Spatio-Temporal Echocardiography Segmentation

Arnaud Judge , Nicolas Duchateau , Thierry Judge , Roman A. Sandler , Joseph Z. Sokol , Christian Desrosiers , Olivier Bernard , Pierre-Marc Jodoin This is my paper

Pith reviewed 2026-05-18 06:50 UTC · model grok-4.3

classification 📡 eess.IV cs.AIcs.CV
keywords reinforcement learningunsupervised domain adaptationechocardiography segmentationspatio-temporal segmentationmedical image analysistemporal consistencyanatomical validityuncertainty estimation
0
0 comments X p. Extension
pith:QGYOHDLQ Add to your LaTeX paper What is a Pith Number? 
\usepackage{pith}
\pithnumber{QGYOHDLQ}

Prints a linked pith:QGYOHDLQ badge after your title and writes the identifier into PDF metadata. Compiles on arXiv with no extra files. Learn more

The pith

Reinforcement learning enables accurate echocardiography segmentation across different datasets without target labels.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The paper introduces RL4Seg3D, a reinforcement learning framework for unsupervised domain adaptation in segmenting heart structures from 2D plus time echocardiography videos. It develops custom reward functions that promote precise identification of key landmarks and a fusion scheme that maintains consistency across video frames, all without requiring any labeled examples from the target imaging domain. This setup is tested on a collection of more than 30,000 echocardiographic videos where it surpasses conventional domain adaptation approaches. A sympathetic reader would care because reliable automated segmentation of cardiac ultrasound videos can support clinical diagnosis while reducing the costly need for expert annotations on every new dataset or scanner.

Core claim

RL4Seg3D integrates novel reward functions and a fusion scheme to enhance key landmark precision in its segmentations while processing full-sized input videos. By leveraging reinforcement learning for image segmentation, our approach improves accuracy, anatomical validity, and temporal consistency while also providing a robust uncertainty estimator which can be used at test time to further enhance segmentation performance. We demonstrate the effectiveness of our framework on over 30,000 echocardiographic videos, showing that it outperforms standard domain adaptation techniques without the need for any labels on the target domain.

What carries the argument

RL4Seg3D framework using reinforcement learning with novel reward functions to promote landmark precision and a fusion scheme to enforce anatomical validity and temporal consistency in full-sized spatio-temporal inputs.

Load-bearing premise

The novel reward functions and fusion scheme are assumed to reliably enforce anatomical validity and temporal consistency in the absence of target-domain labels.

What would settle it

A comparison on a target domain echocardiography dataset with available ground truth labels where the method's segmentation accuracy or landmark errors do not exceed those from a standard unsupervised domain adaptation baseline.

read the original abstract

Domain adaptation methods aim to bridge the gap between datasets by enabling knowledge transfer across domains, reducing the need for additional expert annotations. However, many approaches struggle with reliability in the target domain, an issue particularly critical in medical image segmentation, where accuracy and anatomical validity are essential. This challenge is further exacerbated in spatio-temporal data, where the lack of temporal consistency can significantly degrade segmentation quality, and particularly in echocardiography, where the presence of artifacts and noise can further hinder segmentation performance. To address these issues, we present RL4Seg3D, an unsupervised domain adaptation framework for 2D + time echocardiography segmentation. RL4Seg3D integrates novel reward functions and a fusion scheme to enhance key landmark precision in its segmentations while processing full-sized input videos. By leveraging reinforcement learning for image segmentation, our approach improves accuracy, anatomical validity, and temporal consistency while also providing, as a beneficial side effect, a robust uncertainty estimator, which can be used at test time to further enhance segmentation performance. We demonstrate the effectiveness of our framework on over 30,000 echocardiographic videos, showing that it outperforms standard domain adaptation techniques without the need for any labels on the target domain. Code is available at https://github.com/arnaudjudge/RL4Seg3D.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

2 major / 2 minor

Summary. The paper introduces RL4Seg3D, a reinforcement learning framework for unsupervised domain adaptation in 2D+t echocardiography segmentation. It proposes novel reward functions targeting landmark precision, anatomical validity, and temporal consistency, combined with a fusion scheme, to produce segmentations on unlabeled target-domain videos. The approach also yields an uncertainty estimator usable at test time. Effectiveness is demonstrated on more than 30,000 echocardiographic videos, with claims of outperforming standard domain adaptation techniques without any target labels.

Significance. If the central results hold, the work would be significant for label-efficient adaptation in cardiac ultrasound, where expert annotations are costly and temporal consistency is clinically important. The large-scale evaluation (>30k videos) and public code release are clear strengths that support reproducibility and generalizability claims. The side-effect uncertainty estimator is a useful practical contribution. However, the reliance on hand-crafted rewards means the significance is conditional on those rewards successfully substituting for missing target labels.

major comments (2)
  1. [Methods (reward design and fusion scheme)] The central claim that the framework outperforms standard UDA techniques without target labels rests on the novel reward functions (landmark precision, anatomical validity, temporal consistency) and fusion scheme reliably enforcing clinical constraints. The manuscript provides limited direct evidence that these hand-crafted priors capture domain-specific artifacts such as valve motion or noise patterns in 2D+t echo sequences; if they do not, the reported gains would not generalize beyond the tested data.
  2. [Experiments and Results] The large-scale demonstration on >30,000 videos is presented as evidence of effectiveness, but the evaluation lacks sufficient detail on how the rewards were validated against clinical ground truth or failure modes (e.g., specific artifact types). This is load-bearing because the outperformance claim depends on the rewards substituting for absent target labels.
minor comments (2)
  1. [Abstract] The abstract states 'over 30,000 echocardiographic videos' but does not specify the exact count, source/target split, or acquisition parameters; adding these details would strengthen the reproducibility of the large-scale claim.
  2. [Methods] Notation for the fusion scheme and uncertainty estimator should be defined more explicitly with equations to allow readers to follow how they integrate with the RL policy.

Simulated Author's Rebuttal

2 responses · 0 unresolved

We thank the referee for their constructive and insightful review. The comments highlight important aspects of reward design and validation that we will address to strengthen the manuscript. We provide point-by-point responses below.

read point-by-point responses

  1. Referee: [Methods (reward design and fusion scheme)] The central claim that the framework outperforms standard UDA techniques without target labels rests on the novel reward functions (landmark precision, anatomical validity, temporal consistency) and fusion scheme reliably enforcing clinical constraints. The manuscript provides limited direct evidence that these hand-crafted priors capture domain-specific artifacts such as valve motion or noise patterns in 2D+t echo sequences; if they do not, the reported gains would not generalize beyond the tested data.

    Authors: We appreciate this observation. The reward functions were derived from established echocardiographic principles to enforce clinically relevant constraints on structure, motion, and consistency, which are known to be affected by common artifacts. Ablation studies in the manuscript already quantify the contribution of each reward term to overall performance. To provide more direct evidence, we will add qualitative examples and quantitative analysis of artifact mitigation (e.g., valve motion and shadowing) in a revised experimental section or appendix, along with failure-case discussion. revision: yes

  2. Referee: [Experiments and Results] The large-scale demonstration on >30,000 videos is presented as evidence of effectiveness, but the evaluation lacks sufficient detail on how the rewards were validated against clinical ground truth or failure modes (e.g., specific artifact types). This is load-bearing because the outperformance claim depends on the rewards substituting for absent target labels.

    Authors: We agree that additional detail on reward validation and failure modes would improve clarity. In the revision we will expand the results section with a dedicated failure-mode analysis focused on artifact-heavy sequences and will report any available correlations between reward signals and segmentation quality on labeled subsets. Because the target domain is completely unlabeled, direct clinical ground-truth validation for every case is not feasible; however, the reported gains on multiple metrics and the public code release allow independent verification of the approach. revision: partial

Circularity Check

0 steps flagged

No circularity: empirical validation independent of method definition

full rationale

The paper introduces RL4Seg3D as an unsupervised domain adaptation method using novel reward functions for landmark precision, anatomical validity, and temporal consistency, then reports empirical outperformance on over 30,000 unlabeled target-domain echocardiographic videos against standard UDA baselines. No derivation step reduces a claimed prediction or result to its own inputs by construction, no fitted parameter is relabeled as a prediction, and no load-bearing premise rests solely on self-citation chains. The central claims rest on external experimental comparison rather than tautological re-expression of the reward definitions or fusion scheme.

Axiom & Free-Parameter Ledger

0 free parameters · 0 axioms · 0 invented entities

Abstract provides no explicit free parameters, axioms, or invented entities; all technical assumptions about reward design and fusion are implicit and unstated.

pith-pipeline@v0.9.0 · 5786 in / 1041 out tokens · 24112 ms · 2026-05-18T06:50:40.531759+00:00 · methodology

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Lean theorems connected to this paper

Citations machine-checked in the Pith Canon. Every link opens the source theorem in the public Lean library.

What do these tags mean?
matches
The paper's claim is directly supported by a theorem in the formal canon.
supports
The theorem supports part of the paper's argument, but the paper may add assumptions or extra steps.
extends
The paper goes beyond the formal theorem; the theorem is a base layer rather than the whole result.
uses
The paper appears to rely on the theorem as machinery.
contradicts
The paper's claim conflicts with a theorem or certificate in the canon.
unclear
Pith found a possible connection, but the passage is too broad, indirect, or ambiguous to say the theorem truly supports the claim.

Reference graph

Works this paper leans on

48 extracted references · 48 canonical work pages · 2 internal anchors

  1. [1]

    Domain adaptation for medical image analysis: a survey,

    H. Guan and M. Liu, “Domain adaptation for medical image analysis: a survey,”IEEE Trans Biomed Eng, vol. 69, pp. 1173–85, 2022

    work page 2022

  2. [2]

    Domain adaptation of echocardiography segmentation via reinforcement learning,

    A. Judge, T. Judge, N. Duchateauet al., “Domain adaptation of echocardiography segmentation via reinforcement learning,” inProc. MICCAI, LNCS, vol. 15009, 2024, pp. 235–44

    work page 2024

  3. [3]

    Dynamic-guided spatiotemporal attention for echocardiography video segmentation,

    J. Lin, W. Xie, L. Kanget al., “Dynamic-guided spatiotemporal attention for echocardiography video segmentation,”IEEE Trans Med Imaging, vol. 43, pp. 3843–55, 2024

    work page 2024

  4. [4]

    Echocardiography segmentation with enforced temporal consistency,

    N. Painchaud, N. Duchateau, O. Bernardet al., “Echocardiography segmentation with enforced temporal consistency,”IEEE Trans Med Imaging, vol. 41, p. 2867–78, 2022

    work page 2022

  5. [5]

    Cardiac segmentation with strong anatomical guarantees,

    N. Painchaud, Y . Skandarani, T. Judgeet al., “Cardiac segmentation with strong anatomical guarantees,”IEEE Trans Med Imaging, vol. 39, pp. 3703–13, 2020

    work page 2020

  6. [6]

    SimLVSeg: Simplifying left ventricular segmentation in 2-D+time echocardiograms with self-and weakly supervised learning,

    F. Maani, A. Ukaye, N. Saadiet al., “SimLVSeg: Simplifying left ventricular segmentation in 2-D+time echocardiograms with self-and weakly supervised learning,”Ultrasound Med Biol, vol. 50, pp. 1945– 54, 2024

    work page 1945

  7. [7]

    Segmenting cardiac ultrasound videos using self-supervised learning,

    E. Lamoureux, S. Ayromlou, S. N. Ahmadi Amiriet al., “Segmenting cardiac ultrasound videos using self-supervised learning,” inProc. EMBC, 2023, pp. 1–7

    work page 2023

  8. [8]

    Self-supervised learning for label-free segmentation in cardiac ultrasound,

    D. L. Ferreira, C. Lau, Z. Salaymanget al., “Self-supervised learning for label-free segmentation in cardiac ultrasound,”Nat Commun, vol. 16, p. 4070, 2025

    work page 2025

  9. [9]

    Unsupervised domain adaptation for medical image segmentation via self-training of early features,

    R. Sheikh and T. Schultz, “Unsupervised domain adaptation for medical image segmentation via self-training of early features,” inProc. MIDL, vol. 172, 2022, pp. 1096–107

    work page 2022

  10. [10]

    Pseudo labels for unsupervised domain adaptation: A review,

    Y . Li, L. Guo, and Y . Ge, “Pseudo labels for unsupervised domain adaptation: A review,”Electronics, vol. 12, 2023

    work page 2023

  11. [11]

    Self-labeled techniques for semi- supervised learning: taxonomy, software and empirical study,

    I. Triguero, S. Garc ´ıa, and F. Herrera, “Self-labeled techniques for semi- supervised learning: taxonomy, software and empirical study,”Knowl Inf Syst, vol. 42, pp. 245–84, 2015

    work page 2015

  12. [12]

    Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation,

    L. Yu, S. Wang, X. Liet al., “Uncertainty-aware self-ensembling model for semi-supervised 3D left atrium segmentation,” inProc. MICCAI, LNCS, vol. 11765, 2019, pp. 605–13

    work page 2019

  13. [13]

    Co-training with high-confidence pseudo labels for semi-supervised medical image segmentation,

    Z. Shen, P. Cao, H. Yanget al., “Co-training with high-confidence pseudo labels for semi-supervised medical image segmentation,”arXiv, 2023

    work page 2023

  14. [14]

    Do- main adaptation for medical image segmentation using transformation- invariant self-training,

    N. Ghamsarian, J. Gamazo Tejero, P. M ´arquez-Neilaet al., “Do- main adaptation for medical image segmentation using transformation- invariant self-training,” inProc. MICCAI, LNCS, vol. 14220, 2023, pp. 331–41

    work page 2023

  15. [15]

    Segment anything,

    A. Kirillov, E. Mintun, N. Raviet al., “Segment anything,” inProc. ICCV, 2023, pp. 4015–26

    work page 2023

  16. [16]

    Segment anything in medical images,

    J. Ma, Y . He, F. Liet al., “Segment anything in medical images,”Nat Commun, vol. 15, p. 654, 2024

    work page 2024

  17. [17]

    Customized segment anything model for medical image segmentation,

    K. Zhang and D. Liu, “Customized segment anything model for medical image segmentation,”arXiv:2304.13785, 2023

    work page arXiv 2023

  18. [18]

    Medical SAM adapter: Adapting segment anything model for medical image segmentation,

    J. Wu, Z. Wang, M. Honget al., “Medical SAM adapter: Adapting segment anything model for medical image segmentation,”Med Image Anal, vol. 102, p. 103547, 2025

    work page 2025

  19. [19]

    Segment and track anything,

    Y . Cheng, L. Li, Y . Xuet al., “Segment and track anything,” arXiv:2305.06558, 2023

    work page arXiv 2023

  20. [20]

    Track anything: Segment anything meets videos

    J. Yang, M. Gao, Z. Liet al., “Track anything: Segment anything meets videos,”arXiv:2304.11968, 2023

    work page arXiv 2023

  21. [21]

    Beyond adapting SAM: Towards end- to-end ultrasound image segmentation via auto prompting,

    X. Lin, Y . Xiang, L. Yuet al., “Beyond adapting SAM: Towards end- to-end ultrasound image segmentation via auto prompting,” inProc. MICCAI, LNCS, vol. 15008, 2024, pp. 24–34

    work page 2024

  22. [22]

    MemSAM: taming segment anything model for echocardiography video segmentation,

    X. Deng, H. Wu, R. Zenget al., “MemSAM: taming segment anything model for echocardiography video segmentation,” inProc, CVPR, 2024, pp. 9622–31

    work page 2024

  23. [23]

    ZoDi: Zero-shot domain adaptation with diffusion-based image transfer,

    H. Azuma, Y . Matsui, and A. Maki, “ZoDi: Zero-shot domain adaptation with diffusion-based image transfer,” inProc. ECCV, 2025, pp. 151–67

    work page 2025

  24. [24]

    GANs for medical image synthesis: An empirical study,

    Y . Skandarani, P.-M. Jodoin, and A. Lalande, “GANs for medical image synthesis: An empirical study,”J Imaging, vol. 9, p. 69, 2023

    work page 2023

  25. [25]

    Unpaired image-to-image translation using cycle-consistent adversarial networks,

    J.-Y . Zhu, T. Park, P. Isolaet al., “Unpaired image-to-image translation using cycle-consistent adversarial networks,” inProc. ICCV, 2017, pp. 2223–32

    work page 2017

  26. [26]

    Generative Adversarial Networks in Medical Image augmentation: A review,

    Y . Chen, X.-H. Yang, Z. Weiet al., “Generative Adversarial Networks in Medical Image augmentation: A review,”Comput Biol Med, vol. 144, p. 105382, 2022

    work page 2022

  27. [27]

    Structure preserving cycle-GAN for unsuper- vised medical image domain adaptation,

    P. Iacono and N. Khan, “Structure preserving cycle-GAN for unsuper- vised medical image domain adaptation,” inProc. EMBC, 2024, pp. 1–5

    work page 2024

  28. [28]

    A stability-enhanced CycleGAN for effective domain transformation of unpaired ultrasound images,

    L. Huang, Z. Zhou, Y . Guoet al., “A stability-enhanced CycleGAN for effective domain transformation of unpaired ultrasound images,”Biomed Signal Process Control, vol. 77, p. 103831, 2022

    work page 2022

  29. [29]

    An invitation to deep reinforcement learning,

    B. Jaeger and A. Geiger, “An invitation to deep reinforcement learning,” F ound Trends Optim, vol. 7, pp. 1–80, 2024

    work page 2024

  30. [30]

    A survey of reinforcement learning from human feedback

    T. Kaufmann, P. Weng, V . Bengset al., “A survey of reinforcement learning from human feedback,”arXiv:2312.14925, vol. 10, 2023

    work page arXiv 2023

  31. [31]

    Deep Reinforcement Learning from Human Preferences,

    P. F. Christiano, J. Leike, T. Brownet al., “Deep Reinforcement Learning from Human Preferences,” inProc. NeurIPS, vol. 30, 2017

    work page 2017

  32. [32]

    Fine-Tuning Language Models from Human Preferences

    D. M. Ziegler, N. Stiennon, J. Wuet al., “Fine-tuning language models from human preferences,”arXiv:1909.08593, 2019

    work page internal anchor Pith review Pith/arXiv arXiv 1909

  33. [33]

    Training language models to follow instructions with human feedback,

    L. Ouyang, J. Wu, X. Jianget al., “Training language models to follow instructions with human feedback,”Proc. NeurIPS, vol. 35, pp. 27 730– 44, 2022

    work page 2022

  34. [34]

    Learning to summarize with human feedback,

    N. Stiennon, L. Ouyang, J. Wuet al., “Learning to summarize with human feedback,”Proc. NeurIPS, vol. 33, pp. 3008–21, 2020

    work page 2020

  35. [35]

    Evaluating reinforcement learning agents for anatomical landmark detection,

    A. Alansary, O. Oktay, Y . Liet al., “Evaluating reinforcement learning agents for anatomical landmark detection,”Med Image Anal, vol. 53, pp. 156–64, 2019

    work page 2019

  36. [36]

    Multi-scale deep reinforce- ment learning for real-time 3D-landmark detection in CT scans,

    F.-C. Ghesu, B. Georgescu, Y . Zhenget al., “Multi-scale deep reinforce- ment learning for real-time 3D-landmark detection in CT scans,”IEEE Trans Pattern Anal Mach Intell, vol. 41, pp. 176–89, 2019

    work page 2019

  37. [37]

    Reinforcement Learning in Med- ical Image Analysis: Concepts, Applications, Challenges, and Future Directions,

    M. Hu, J. Zhang, L. Matkovicet al., “Reinforcement Learning in Med- ical Image Analysis: Concepts, Applications, Challenges, and Future Directions,”J Appl Clin Med Phys, vol. 24, p. e13898, 2023

    work page 2023

  38. [38]

    Rl-coseg: A reinforcement learning-based collaborative localization and segmentation framework for medical image,

    F. Xu, F. Yang, X. Zhang, and Z. Liu, “Rl-coseg: A reinforcement learning-based collaborative localization and segmentation framework for medical image,”Expert Systems with Applications, vol. 298, p. 129661, 2026

    work page 2026

  39. [39]

    Proximal Policy Optimization Algorithms

    J. Schulman, F. Wolski, P. Dhariwalet al., “Proximal policy optimization algorithms,”arXiv:1707.06347, 2017

    work page internal anchor Pith review Pith/arXiv arXiv 2017

  40. [40]

    nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation,

    F. Isensee, P. F. Jaeger, S. A. A. Kohlet al., “nnU-Net: a self-configuring method for deep learning-based biomedical image segmentation,”Nat Methods, vol. 18, pp. 203–11, 2021

    work page 2021

  41. [41]

    On calibration of modern neural networks,

    C. Guo, G. Pleiss, Y . Sunet al., “On calibration of modern neural networks,” inProc. ICML, 2017, pp. 1321–30

    work page 2017

  42. [42]

    Deep learning for segmentation using an open large-scale dataset in 2D echocardiography,

    S. Leclerc, E. Smistad, J. Pedrosaet al., “Deep learning for segmentation using an open large-scale dataset in 2D echocardiography,”IEEE Trans Med Imaging, vol. 38, pp. 2198–210, 2019

    work page 2019

  43. [43]

    U-net: Convolutional networks for biomedical image segmentation,

    O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” inProc. MICCAI, LNCS, vol. 9351, 2015, pp. 234–41

    work page 2015

  44. [44]

    Metrics reloaded: recom- mendations for image analysis validation,

    L. Maier-Hein, A. Reinke, P. Godauet al., “Metrics reloaded: recom- mendations for image analysis validation,”Nat Methods, vol. 21, pp. 195–212, 2024

    work page 2024

  45. [45]

    Anatomy of the mitral valve apparatus: role of 2D and 3D echocardiography,

    J. P. Dal-Bianco and R. A. Levine, “Anatomy of the mitral valve apparatus: role of 2D and 3D echocardiography,”Cardiol Clin, vol. 31, pp. 151–64, 2013

    work page 2013

  46. [46]

    Dropout as a Bayesian approximation: Representing model uncertainty in deep learning,

    Y . Gal and Z. Ghahramani, “Dropout as a Bayesian approximation: Representing model uncertainty in deep learning,” inProc. ICML, 2016, pp. 1050–9

    work page 2016

  47. [47]

    Simple and scal- able predictive uncertainty estimation using deep ensembles,

    B. Lakshminarayanan, A. Pritzel, and C. Blundell, “Simple and scal- able predictive uncertainty estimation using deep ensembles,” inProc. NeurIPS, vol. 30, 2017

    work page 2017

  48. [48]

    Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with con- volutional neural networks,

    G. Wang, W. Li, M. Aertsenet al., “Aleatoric uncertainty estimation with test-time augmentation for medical image segmentation with con- volutional neural networks,”Neurocomputing, vol. 338, pp. 34–45, 2019

    work page 2019