REVIEW 3 major objections 5 minor 38 references
MimickNet, Matching Clinical Post-Processing Under Realistic Black-Box Constraints
T0 review · 3 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read MimickNet shows that proprietary ultrasound post-processing can be approximated from unpaired images, reaching an SSIM of 0.930 on a clinical test set.
desk verdict Solid, honest empirical contribution showing CycleGAN can mimic a proprietary ultrasound post-processing pipeline from unpaired data, but the headline SSIM overstates fidelity on high-dynamic-range targets and the evaluation has some fixable methodological gaps. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The load-bearing object is a CycleGAN, a pair of generators and discriminators trained with cycle consistency to translate between two image domains without paired examples, here implemented with U-Net encoder-decoder generators with skip connections and PatchGAN/LSGAN discriminators. The selected MimickNet generator has 117,697 parameters and is trained from scratch with a cycle-consistency loss (Eq. 3) and an adversarial loss. The cycle-consistency loss is what permits unpaired training, and the adversarial loss is what pushes outputs toward the clinical look. SSIM, broken into luminance and contrast-structure components, is the evaluation metric that localizes the remaining gap to luminance outliers.
What would settle it
Feed MimickNet raw frames whose clinical counterparts were produced under a deliberately different dynamic-range window, for example clipping at $-80$ dB instead of the scanner's default, and measure the luminance component of SSIM; if the model cannot track the new clipping and the luminance component drops well below the reported mean of $0.967 \pm 0.073$, the cycle-consistency objective has not learned the true post-processing function.
Extended reading notes
Core claim
MimickNet treats clinical post-processing as an unpaired style-transfer problem and shows that a CycleGAN with U-Net generators can reproduce a manufacturer's post-processed output from raw delay-and-summed beams. In the black-box setting, with no access to the pre-processed partner images, the trained generator reaches a mean structural similarity of $0.930 \pm 0.089$ and, on cardiac cineloops outside the fetal, liver, and phantom training distribution, $0.967 \pm 0.002$. A gray-box model trained on paired images reaches $0.979 \pm 0.013$, which the paper treats as the theoretical ceiling for this architecture. The main remaining failure mode is luminance: contrast and structure are captured well, but the cycle-consistency objective discourages the network from learning irreversible dynamic-range clipping, so bright reflectors in outlier frames remain too bright.
Load-bearing premise
The argument stands on the assumption that the scanner's post-processing is a deterministic, stationary function of the raw delay-and-summed image, so that cycle-consistency plus adversarial training can recover it from unpaired samples; if the mapping is not identifiable or depends on hidden scanner state, the reported SSIM could reflect distribution matching rather than faithful reproduction.
Editorial extensions
If this is right
- Novel beamforming results can be shown next to a clinical-grade post-processing baseline without tapping proprietary scanner code or paired data.
- MimickNet runs in real time at 92 FPS on an NVIDIA P100 and uses about 2000 times fewer FLOPS than MobileNetV2, so the mimic could run on clinical or mobile ultrasound systems.
- It generalizes beyond its training targets: cardiac cineloops yield SSIM 0.967, and resolution and contrast improvements from the REFocUS beamforming method are preserved after mimic post-processing.
- Because a model trained only on Verasonics data performs comparably to the full-data model, the framework may need relatively little data to mimic a given scanner.
Reading between the lines
- A natural testable extension is to train MimickNet on one manufacturer's outputs and evaluate on another manufacturer's clinical images; success there would show the network learns a generalizable post-processing style rather than dataset-specific statistics.
- If the cycle-consistency loss is the reason dynamic-range clipping is not learned, replacing or relaxing it with a one-way adversarial loss plus a histogram-matching or luminance-calibration term should reduce the luminance outlier gap.
- The gray-box ceiling of SSIM 0.979 sets a target: any future architecture that closes the gap to that ceiling under black-box constraints would demonstrate that irreversible post-processing operations can be learned without paired data.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes MimickNet, a CycleGAN-based framework that maps raw delay-and-sum (DAS) ultrasound beamformed images to images resembling clinical-grade scanner post-processing, using only unpaired raw and post-processed cineloops. The authors first report a gray-box upper bound with paired data (best SSIM 0.979 ± 0.013), then train an unpaired CycleGAN and report a test-set SSIM of 0.930 ± 0.089, with out-of-distribution cardiac data achieving SSIM 0.967 ± 0.002. They analyze worst-case SSIM components, show that luminance errors dominate failures, and release code and models. The stated contribution is to provide a practical clinical post-processing baseline for comparing novel beamforming methods.
Significance. If the central claim is supported, the framework is practically useful: it offers researchers a way to generate clinical-grade baselines without proprietary scanner access. The manuscript has several strengths: evaluation on held-out cineloops, an out-of-distribution cardiac test, a gray-box upper-bound comparison, runtime measurements, and open-source code. The empirical finding that a small generator fails to converge while a 117,697-parameter generator succeeds, and the relative consistency across objective functions in Table III, are useful contributions. However, the headline SSIM is selected from test-set performance without an independent validation split, and the paper's own worst-case analysis shows a known failure mode (dynamic-range clipping) that is central to the claimed approximation. These issues need to be addressed before the central claim is fully established.
major comments (3)
- [Section III-B, Table III] The final MimickNet configuration is selected as the 117,697-parameter MSE model "since it achieves the highest SSIM" on the same test set used for the reported headline number. Without an independent validation split, the reported performance is an optimistic selection artifact. The same issue affects the gray-box model selection in Table II. I request a validation split for any hyperparameter or checkpoint selection, or a corrected estimate such as nested cross-validation, and a statement of how many configurations were tried.
- [Section IV] The Discussion states that clipping raw DAS data from -120 dB to -80 dB makes bright scatterers "practically match clinical-grade post-processing without any additional changes." This is a simple baseline that should be quantified on the full test set, along with other non-learned baselines such as histogram matching or global gain adjustment. Without such comparisons, the claim that MimickNet "closely approximates" the clinical operator is not demonstrated; the learned network must be shown to beat these trivial baselines on the same metrics.
- [Section II-B, Eq. (3), Figure 5, Section IV] The worst-case examples in Figure 5 have SSIM as low as 0.414, with a luminance component of 0.419, and the Discussion attributes this to the cycle-consistency loss making dynamic-range clipping "challenging to learn." Because the stated purpose is to provide a fair clinical baseline for novel beamforming research, the paper should quantify how often such failures occur (e.g., the fraction of test frames below a threshold such as SSIM 0.8) and evaluate whether the failure affects comparisons for high-dynamic-range targets. Alternatively, the authors should temper the "closely approximate" claim to reflect that the mapping is distribution-matched rather than operator-identical.
minor comments (5)
- [Abstract] The abstract reports performance on a "300 cineloop test set," but Table I lists 8,509 test frames from what appears to be many more cineloops; please clarify what the number 300 refers to.
- [Figure 4] The bottom panel's x-axis is labeled "similarity ratio"; specify that this is the SSIM luminance component, and add consistent axis labels for both panels.
- [Tables II and III] The phrase "not significantly different" is used without any significance test or confidence interval; please add statistical comparisons or remove the claim.
- [References and text] The beamforming method is written "REFoCUS" in reference [3] but "REFocUS" in the main text; please make the spelling consistent.
- [Section III-A] The sentence "We are primarily interested in the best SSIM metric since it was originally formulated to model the human visual system" is presented without justification; SSIM is not universally accepted as the best proxy for clinical image quality, so a brief rationale or reference would help.
Circularity Check
No significant circularity: empirical benchmark with held-out and out-of-distribution evaluation.
full rationale
This paper does not contain a derivation chain; its central claim is an empirical measurement of SSIM between MimickNet outputs and clinical-grade post-processing ground truth. The raw DAS images and post-processed images are independent measured inputs, and the reported test-set and cardiac evaluations use data not used to train the generators. CycleGAN is a standard external method (ref [30]); no uniqueness theorem or load-bearing result is imported from the authors' prior work. The admitted clipping limitation (Sec. IV) is an identifiability failure, not a circular reduction: the learned mapping is still compared to ground truth. The mildest concern is that the final configuration was selected by test-set SSIM ('We select the 117697 parameter network optimizing MSE for subsequent analysis since it achieves the highest SSIM with fewest parameters'), so the headline 0.930 is partly a selection statistic; however, this is a methodological optimism loop rather than an equivalence-by-construction, and the independent cardiac generalization (0.967±0.002) plus the consistency across Table III architectures provide external grounding. No step reduces its output to its input by definition.
Assumptions & free parameters
free parameters (2)
- Generator capacity and architecture =
117,697 parameters for final model
- Training objective for cycle-consistency =
MSE for final MimickNet
assumptions (3)
- domain assumption Cycle-consistency loss is sufficient to identify the true post-processing mapping from unpaired images.
- domain assumption SSIM is an adequate measure of clinical-grade image similarity for this task.
- domain assumption The proprietary post-processing is a deterministic, stationary function of the raw DAS image within the dataset.
Cite this review
Pith. "Pith review of MimickNet, Matching Clinical Post-Processing Under Realistic Black-Box Constraints." pith.science (2026). https://pith.science/paper/YWOPJVM6
@misc{pith2026190805782,
author = {Pith},
title = {Pith review of: MimickNet, Matching Clinical Post-Processing Under Realistic Black-Box Constraints},
year = {2026},
howpublished = {\url{https://pith.science/paper/YWOPJVM6}},
note = {Machine review of arXiv:1908.05782}
}
abstract
Image post-processing is used in clinical-grade ultrasound scanners to improve image quality (e.g., reduce speckle noise and enhance contrast). These post-processing techniques vary across manufacturers and are generally kept proprietary, which presents a challenge for researchers looking to match current clinical-grade workflows. We introduce a deep learning framework, MimickNet, that transforms raw conventional delay-and-summed (DAS) beams into the approximate post-processed images found on clinical-grade scanners. Training MimickNet only requires post-processed image samples from a scanner of interest without the need for explicit pairing to raw DAS data. This flexibility allows it to hypothetically approximate any manufacturer's post-processing without access to the pre-processed data. MimickNet generates images with an average similarity index measurement (SSIM) of 0.930$\pm$0.0892 on a 300 cineloop test set, and it generalizes to cardiac cineloops outside of our train-test distribution achieving an SSIM of 0.967$\pm$0.002. We also explore the theoretical SSIM achievable by evaluating MimickNet performance when trained under gray-box constraints (i.e., when both pre-processed and post-processed images are available). To our knowledge, this is the first work to establish deep learning models that closely approximate current clinical-grade ultrasound post-processing under realistic black-box constraints where before and after post-processing data is unavailable. MimickNet serves as a clinical post-processing baseline for future works in ultrasound image formation to compare against. To this end, we have made the MimickNet software open source.
Figures
Figures from the paper (2 more)
Reference graph
Works this paper leans on
-
[1]
G. F. Pinton, G. E. Trahey, and J. J. Dahl, “Sources of image degradation in fundamental and harmonic ultrasound imaging using nonlinear, full- wave simulations,” IEEE Trans. Ultrason. Ferroelectr . Freq. Control , vol. 58, no. 4, pp. 754–765, Apr. 2011
work page 2011
-
[2]
Exploring nsight imaging, a totally new architecture for premium ultrasound,
K. Thiele, J. Jago, R. Entrekin, and R. Peterson, “Exploring nsight imaging, a totally new architecture for premium ultrasound,” Philips, Tech. Rep. 4522 962 95791, June 2013. [Online]. Available: https://www.usa.philips.com/healthcare/resources/feature-detail/nsight
work page 2013
-
[3]
REFoCUS: Ultrasound focusing for the software beam- forming age,
N. Bottenus, “REFoCUS: Ultrasound focusing for the software beam- forming age,” in 2018 IEEE International Ultrasonics Symposium (IUS) , Oct. 2018, pp. 1–4
work page 2018
-
[4]
G. E. Trahey, J. W. Allison, S. W. Smith, and O. T. von Ramm, “Speckle reduction achievable by spatial compounding and frequency compound- ing: Experimental results and implications for target detectability,” in Pattern Recognition and Acoustical Imaging , vol. 0768. International Society for Optics and Photonics, Sep. 1987, pp. 185–192
work page 1987
-
[5]
A primer on the physical principles of tissue harmonic imaging,
A. Anvari, F. Forsberg, and A. E. Samir, “A primer on the physical principles of tissue harmonic imaging,” Radiographics, vol. 35, no. 7, pp. 1955–1964, Nov. 2015
work page 1955
-
[6]
Clinical utility of fetal Short-Lag spatial coherence imaging,
W. Long, D. Hyun, K. R. Choudhury, D. Bradway, P. McNally, B. Boyd, S. Ellestad, and G. E. Trahey, “Clinical utility of fetal Short-Lag spatial coherence imaging,” Ultrasound Med. Biol. , vol. 44, no. 4, pp. 794–806, Apr. 2018
work page 2018
-
[7]
M. R. Morgan, D. Hyun, and G. E. Trahey, “Short-lag spatial coherence imaging in 1.5-d and 1.75-d arrays: Elevation performance and array design considerations,” IEEE Trans. Ultrason. Ferroelectr . Freq. Control, Mar. 2019
work page 2019
-
[8]
Understand- ing the advanced signal processing technique of Real-Time adaptive filters,
H. Ahman, L. Thompson, A. Swarbrick, and J. Woodward, “Understand- ing the advanced signal processing technique of Real-Time adaptive filters,” J. Diagn. Med. Sonogr ., vol. 25, no. 3, pp. 145–160, May 2009
work page 2009
Show all 38 references
-
[9]
Deep convolutional neural network for image deconvolution,
L. Xu, J. S. J. Ren, C. Liu, and J. Jia, “Deep convolutional neural network for image deconvolution,” in Advances in Neural Information Processing Systems 27 , Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, Eds. Curran Associates, Inc., 2014, pp. 1790–1798
2014
-
[10]
Least squares generative adversarial networks,
X. Mao, Q. Li, H. Xie, R. Y . K. Lau, Z. Wang, and S. P. Smolley, “Least squares generative adversarial networks,” in 2017 IEEE International Conference on Computer Vision (ICCV) , Oct 2017, pp. 2813–2821
2017
-
[11]
Robust kernel regression for restoration and reconstruction of images from sparse noisy data,
H. Takeda, S. Farsiu, and P. Milanfar, “Robust kernel regression for restoration and reconstruction of images from sparse noisy data,” in2006 International Conference on Image Processing , Oct. 2006, pp. 1257– 1260
2006
-
[12]
Bilateral filtering for gray and color images,
C. Tomasi and R. Manduchi, “Bilateral filtering for gray and color images,” in Sixth International Conference on Computer Vision (IEEE Cat. No.98CH36271) , Jan. 1998, pp. 839–846
1998
-
[13]
U-net: Convolutional networks for biomedical image segmentation,
O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” Med. Image Comput. Comput. Assist. Interv., 2015
2015
-
[14]
Fast and accurate image super resolution by deep CNN with skip connection and network in network,
J. Yamanaka, S. Kuwashima, and T. Kurita, “Fast and accurate image super resolution by deep CNN with skip connection and network in network,” in Neural Information Processing . Springer International Publishing, 2017, pp. 217–225
2017
-
[15]
Deep residual learning for image recognition,
K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2016, pp. 770–778
2016
-
[16]
Road extraction by deep residual U- Net,
Z. Zhang, Q. Liu, and Y . Wang, “Road extraction by deep residual U- Net,” IEEE Geoscience and Remote Sensing Letters , vol. 15, no. 5, pp. 749–753, May 2018
2018
-
[17]
Multi-level Wavelet- CNN for image restoration,
P. Liu, H. Zhang, K. Zhang, L. Lin, and W. Zuo, “Multi-level Wavelet- CNN for image restoration,” 2018
2018
-
[18]
Densely connected convolutional networks,
G. Huang, Z. Liu, L. Van Der Maaten, and K. Q. Weinberger, “Densely connected convolutional networks,” in Proceedings of the IEEE confer- ence on computer vision and pattern recognition , 2017, pp. 4700–4708
2017
-
[19]
The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation,
S. J ´egou, M. Drozdzal, D. Vazquez, A. Romero, and Y . Bengio, “The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops , 2017, pp. 11–19
2017
-
[20]
Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion,
P. Vincent, H. Larochelle, I. Lajoie, Y . Bengio, and P.-A. Manzagol, “Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion,” J. Mach. Learn. Res. , vol. 11, no. Dec, pp. 3371–3408, 2010
2010
-
[21]
Loss functions for image restoration with neural networks,
H. Zhao, O. Gallo, I. Frosio, and J. Kautz, “Loss functions for image restoration with neural networks,” IEEE Transactions on Computational Imaging, vol. 3, no. 1, pp. 47–57, March 2017
2017
-
[22]
Learning to generate images with perceptual similarity metrics,
J. Snell, K. Ridgeway, R. Liao, B. D. Roads, M. C. Mozer, and R. S. Zemel, “Learning to generate images with perceptual similarity metrics,” 2017
2017
-
[23]
Large scale GAN training for high fidelity natural image synthesis,
A. Brock, J. Donahue, and K. Simonyan, “Large scale GAN training for high fidelity natural image synthesis,” in International Conference on Learning Representations , 2019. [Online]. Available: https://openreview.net/forum?id=B1xsqj09Fm
2019
-
[24]
Generative adversarial nets,
I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y . Bengio, “Generative adversarial nets,” in Advances in Neural Information Processing Systems 27 , Z. Ghahramani, M. Welling, C. Cortes, N. D. Lawrence, and K. Q. Weinberger, Eds. ...
2014
-
[25]
Unsupervised representation learning with deep convolutional generative adversarial networks,
A. Radford, L. Metz, and S. Chintala, “Unsupervised representation learning with deep convolutional generative adversarial networks,” CoRR, vol. abs/1511.06434, 2016
2016 arXiv
-
[26]
Image-to-image translation with conditional adversarial networks,
P. Isola, J.-Y . Zhu, T. Zhou, and A. A. Efros, “Image-to-image translation with conditional adversarial networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 1125– 1134
2017
-
[27]
Ultrasound image enhancement using a deep learning architecture,
M. Abdel-Nasser and O. A. Omer, “Ultrasound image enhancement using a deep learning architecture,” in Proceedings of the International Conference on Advanced Intelligent Systems and Informatics 2016 . Springer International Publishing, 2017, pp. 639–649
2016
-
[28]
Ultrasound speckle reduction using generative adversial networks,
F. Dietrichson, E. Smistad, A. Ostvik, and L. Lovstakken, “Ultrasound speckle reduction using generative adversial networks,” in 2018 IEEE International Ultrasonics Symposium (IUS) , Oct. 2018, pp. 1–4
2018
-
[29]
Deep convolutional neural network for ultrasound image enhancement,
D. Perdios, M. V onlanthen, A. Besson, F. Martinez, and J.-P. Thiran, “Deep convolutional neural network for ultrasound image enhancement,” in 2018 IEEE International Ultrasonics Symposium (IUS) , Oct. 2018, pp. 1–4
2018
-
[30]
Unpaired image-to-image translation using cycle-consistent adversarial networks,
J.-Y . Zhu, T. Park, P. Isola, and A. A. Efros, “Unpaired image-to-image translation using cycle-consistent adversarial networks,” in Proceedings of the IEEE international conference on computer vision , 2017, pp. 2223–2232
2017
-
[31]
In vivo application of short-lag spatial coherence and harmonic spatial coherence imaging in fetal ultrasound,
V . Kakkad, J. Dahl, S. Ellestad, and G. Trahey, “In vivo application of short-lag spatial coherence and harmonic spatial coherence imaging in fetal ultrasound,” Ultrason. Imaging , vol. 37, no. 2, pp. 101–116, Apr. 2015
2015
-
[32]
Quantifying image quality improvement using elevated acoustic output in B-Mode harmonic imaging,
Y . Deng, M. L. Palmeri, N. C. Rouze, G. E. Trahey, C. M. Haystead, and K. R. Nightingale, “Quantifying image quality improvement using elevated acoustic output in B-Mode harmonic imaging,” Ultrasound Med. Biol. , vol. 43, no. 10, pp. 2416–2425, Oct. 2017
2017
-
[33]
Implications of lag-one coherence on real-time adaptive frequency selection,
J. Long, W. Long, N. Bottenus, G. F. Pintonl, and G. E. Trahey, “Implications of lag-one coherence on real-time adaptive frequency selection,” in 2018 IEEE International Ultrasonics Symposium (IUS) . IEEE, 2018, pp. 1–9
2018
-
[34]
Image quality assessment: from error visibility to structural similarity,
A. C. Bovik, H. R. Sheikh, and E. P. Simoncelli, “Image quality assessment: from error visibility to structural similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–612, Apr. 2004
2004
-
[35]
Mo- bilenetv2: Inverted residuals and linear bottlenecks,
M. Sandler, A. Howard, M. Zhu, A. Zhmoginov, and L. Chen, “Mo- bilenetv2: Inverted residuals and linear bottlenecks,” in 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition , June 2018, pp. 4510–4520
2018
-
[36]
Progressive growing of GANs for improved quality, stability, and variation,
T. Karras, T. Aila, S. Laine, and J. Lehtinen, “Progressive growing of GANs for improved quality, stability, and variation,” in International Conference on Learning Representations , 2018. [Online]. Available: https://openreview.net/forum?id=Hk99zCeAb
2018
-
[37]
Which training methods for GANs do actually converge?
L. Mescheder, A. Geiger, and S. Nowozin, “Which training methods for GANs do actually converge?” in Proceedings of the 35th International Conference on Machine Learning, ser. Proceedings of Machine Learning Research, J. Dy and A. Krause, Eds., vol. 80. Stockholmsmssan, Stockho...
2018
-
[38]
Mobile ultrafast ultrasound imaging system based on smartphone and tablet devices,
H. Hewener and S. Tretbar, “Mobile ultrafast ultrasound imaging system based on smartphone and tablet devices,” in 2015 IEEE International Ultrasonics Symposium (IUS) , Oct. 2015, pp. 1–4
2015
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.