REVIEW 4 major objections 5 minor 37 references
Restoration of Non-rigidly Distorted Underwater Images using a Combination of Compressive Sensing and Local Polynomial Image Representations
T0 review · 4 major / 5 minor · reviewed 2026-08-14 · deepseek-v4-flash
Pith's one-line read This paper claims that wavy-water video distortion can be undone by recovering the full motion field from a few tracked points using compressed sensing with a 3D Fourier sparsity basis.
desk verdict A genuinely new CS-based motion interpolation method with reproducible wins over baselines, but the center-of-trajectory assumption is a real, untested limitation. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
The central object is the complex-valued motion vector field $d = d_x + i d_y$, a 3D array over image space and time. Its key property is sparsity in the 3D DFT basis, justified by the spatial smoothness and temporal periodicity of water waves. The measurement model is $e = \Phi F \theta + \eta$, where $\Phi$ is a row-subsampled identity selecting tracked points and $F$ is the 3D DFT matrix; because $\Phi$ and $F$ are maximally incoherent, a LASSO solver can reconstruct the field from very few tracks. The second ingredient is PEOF, which fits local second-degree polynomials to image patches and solves for the displacement between each frame and the mean image, iterating over a pyramid for large motions.
What would settle it
Record a static target through wavy water with a known steady current or asymmetric wave train, track feature points, and compare each trajectory's mean against its known true position; if the mean offset exceeds a small threshold, the compressed-sensing reconstruction is fed biased measurements and the restored-image error should rise, whereas an algorithm that explicitly estimates drift would not degrade.
Extended reading notes
Core claim
The central claim is that the full 3D motion vector field generated by a wavy water surface is compressible in the 3D discrete Fourier basis, so it can be recovered from a small set of tracked point trajectories by solving an L1-regularized least-squares problem, the LASSO. The recovered field is then used to warp every video frame back toward a common undistorted coordinate system. The paper further claims that applying polynomial-expansion optical flow (PEOF) after the compressed-sensing step preserves straight lines, text, and grid structure better than the compressed-sensing step alone, and that both stages outperform the compared existing methods on synthetic and real underwater videos in terms of RMSE, SSIM, and normalized mutual information. The two-stage CS+PEOF pipeline is presented as the best overall restoration approach, at computational cost far below the spline-plus-RPCA baseline.
Load-bearing premise
The load-bearing premise is that the time-average of each tracked point's positions equals its true undistorted position, which requires the water surface motion at every point to be locally symmetric and zero-mean over the captured frames.
Editorial extensions
If this is right
- Dense non-rigid motion estimation for wavy water needs only a sparse set of tracked points: the paper reports reconstructions from fewer than 3% of pixel trajectories, with performance saturating as more tracks are added.
- The compressed-sensing stage alone removes more than 90% of the motion in most test videos, indicating that the 3D-DFT sparsity model captures the physical distortion well.
- A simple local-polynomial optical-flow stage, used alone, already beats the compared baselines on most numerical metrics, so elaborate models are not required for this task.
- The two-stage CS+PEOF combination gives the best structural preservation of lines, text, and grids, as well as the best numerical quality in most comparisons.
- The method needs no training data, no distortion-free template, and no special illumination or multiple viewpoints; only a short video on the order of 100 frames is sufficient.
Reading between the lines
- If the zero-mean water-motion assumption fails because of a steady current or wave drift, the center-of-trajectory estimate becomes biased; a natural extension would be to estimate and subtract a global or slowly varying drift term before compressed-sensing reconstruction.
- The same 3D-DFT sparsity idea could be tested on other temporally periodic non-rigid distortions, such as short-exposure atmospheric turbulence, where similar band-limited motion fields arise.
- The success of this parameter-free pipeline suggests that deep single-frame restoration methods lose information by ignoring temporal redundancy; training them on multiple frames, or combining them with a compressed-sensing motion-reduction front end, could close the performance gap.
- Because the row-subsampled identity and the DFT are maximally incoherent, the reconstruction guarantees are near-optimal; replacing the DFT with another basis would likely require more tracked points for the same accuracy.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper addresses restoration of a static scene imaged through a wavy water surface, where each video frame is a non-rigidly distorted view of the same underlying image. The authors propose a compressed sensing (CS) pipeline: salient points are tracked across frames, their displacement trajectories are used as sparse space-time measurements of the dense motion vector field (MVF), and the MVF is recovered by solving a LASSO problem under the assumption that the MVF is sparse in the 3D DFT basis. The restored image is then obtained by warping each frame by the estimated MVF and taking the mean. The paper also evaluates a simpler optical flow method based on local polynomial expansion (PEOF), and a two-stage CS+PEOF variant. Experiments on synthetic videos, a mechanically-paddled real tank dataset, and the Real2 dataset of Tian and Narasimhan report numerical improvements over SBR-RPCA, LWB, and a deep-learning baseline in terms of RMSE, SSIM, and NMI, with substantially lower computation time than SBR-RPCA and LWB.
Significance. If the reported results hold, the CS-based MVF interpolation is a genuinely new use of compressed sensing for refractive distortion correction, and the finding that a simple PEOF optical flow method is competitive with or better than dedicated state-of-the-art restoration methods is practically valuable. The paper is refreshingly clear about its assumptions, provides synthetic and real data with ground truth, and makes code and datasets available, which strengthens reproducibility. The central claims, however, depend on two assumptions that are not independently verified: the zero-mean local symmetry of water motion that justifies the center-of-trajectory approximation, and the empirical sparsity of real MVFs in the DFT basis. The current evidence for the latter is circular, and the former is only tested in a wave-tank regime where waves are generated by mechanical paddles and are likely near-zero-mean. The CS-theoretic justification also contains a coherence error.
major comments (4)
- [Sec. 3.3, 'Verification of Fourier Sparsity' and Fig. 1] The empirical verification of Fourier sparsity is circular. The MVF whose DFT coefficients are plotted in Fig. 1 is the output of the CS/LASSO estimator in Eq. (2), which explicitly minimizes the ℓ1 norm of the DFT coefficients and therefore enforces sparsity in the DFT basis. Showing that this estimate has a concentrated spectrum is a tautology, not evidence that real MVFs are sparse in the DFT basis. The authors do note that 'the actual MVF can only be estimated,' but they then use the CS estimate itself to support the modeling assumption. Please verify sparsity using an independently estimated MVF (for example, from PEOF/optical flow, or from the known synthetic MVF in the Synthetic dataset), or explicitly reword the section to state that Fig. 1 illustrates the estimator's output rather than validating the premise on which the estimator is built.
- [Sec. 2.3.2, center-of-trajectory (COT) approximation, with Sec. 2.3.1 filter] The COT approximation x_i0 ≈ mean_t x_it is load-bearing for the entire CS stage, because every displacement in Eq. (1) is computed relative to the COT. The justification by 'local symmetry of water motion' fails when there is a net current, Stokes drift, or an asymmetric wave train: each trajectory then has a bias b_i = COT_i - p_i^0, and all measurements e in Eq. (1) are systematically shifted by b_i. The trajectory-filtering rule in Sec. 2.3.1 (COT over the first vs. last T/2 frames differing by more than 3 pixels) cannot detect this bias, because a constant offset adds equally to both halves. All real data come from a mechanically-paddled tank, where waves are plausibly near-zero-mean, so the failure mode is untested. Please add a synthetic experiment with a nonzero-mean velocity component (a constant drift or a biased wave) and report the restoration metrics and the resulting motion reduction; if the method degrades, state the operating domain clearly, and if it does not, explain why the bias is not harmful in CS reconstruction.
- [Sec. 2.3.3, coherence argument after Eq. (1)] The claim that the coherence between the row-subsampled identity sampling matrix Φ and the DFT basis F 'reaches its lower bound of 1' is numerically incorrect as stated. With the definition given in the paper, |Φ_i F_j| = 1/√(NxNyT) for every pair of rows/columns, so the defined coherence is approximately 1/√(NxNyT), not 1; under the standard sensing-matrix coherence used in CS theory, the value depends on the chosen row set and for a random partial DFT is typically O(√(log(NxNyT))), not exactly 1. The lower bound of coherence is also not 1 in this setting. Please correct the coherence calculation and the resulting sample-complexity sentence; otherwise the CS-theoretic justification for the number of tracked points is unsupported. This is a local error in the theory section, but it is part of the key novelty claim, so it should be fixed.
- [Table 1 and Sec. 3.3] The central empirical claim that CS, PEOF, and CS+PEOF outperform SBR-RPCA, LWB, and DL rests on single runs without error bars, repeated trials, or statistical tests. Several of the differences are small (e.g., Real1 Checker: CS+PEOF SSIM 0.892 vs. PEOF 0.890; Synthetic BrickWall: CS+PEOF RMSE 0.141 vs. PEOF 0.142), and all of the Synthetic videos are generated from 'randomly chosen parameters' with no indication of how many random realizations were used. Please report means and standard deviations over at least several random wave/tracking realizations for the Synthetic dataset, or otherwise characterize the run-to-run variability of the metrics. Without this, the superiority claims over the baselines are not yet supported at the level the paper asserts.
minor comments (5)
- [Sec. 2.3.3, Eq. (2)] The LASSO objective in Eq. (2) uses an unsquared data-fidelity term ||e−ΦFθ||_2, whereas the standard LASSO and the YALL1 solver typically use a squared ℓ2 term. Please clarify the exact optimization problem solved by YALL1 in this paper, and ensure the objective function matches the solver's formulation.
- [Sec. 3.1 and Sec. 3.2] The description of the Synthetic dataset says the water surface is generated using a superposition of 2–6 sinusoidal waves with 'randomly chosen parameters,' but no details are given on the parameter ranges or the number of random trials. Adding this information would help reproducibility and would support the request for error bars in the major comments.
- [Fig. 2] The caption of Fig. 2 lists the datasets used but does not identify which curve corresponds to which dataset or parameter; the graph itself is not visible in the text. Please add a legend or descriptive labels, and state the axes clearly.
- [Sec. 3.3, 'Verification of Fourier Sparsity'] The motion-reduction measure MR is defined as a median over tracked points, but the denominator ||d_i||_2 is the norm of the original displacement trajectory; for points with very small displacement this ratio can be unstable. Please state whether any thresholding or regularization was applied, and report the distribution of the per-point ratios rather than only the median.
- [Throughout] There are several small typographical and formatting issues, such as 'Y ALL1' with a space in Sec. 3.2, inconsistent time formatting in Table 1 (e.g., '29s' vs. '0m 23s'), and the phrase 'nonrigid transport object' in reference [19]. These do not affect the technical content but should be cleaned up.
Circularity Check
Fourier-sparsity 'verification' reuses the CS estimator's own output; central restoration claims are independently benchmarked.
-
self definitional
[Section 3.3, 'Verification of Fourier Sparsity', Fig. 1]
"Verification of Fourier Sparsity: Here, we demonstrate the sparsity of the MVFs from real underwater sequences. This is shown in Fig. 1 for the 'Elephant' sequence ... We note that the actual MVF can only be estimated. However, we contend that the MVF estimated by our CS method is a good approximation to the actual one. ... scatter plot of frequencies which account for 99% of the squared magnitude of the estimated MVF using CS (right)."
The MVF d is estimated by solving Eq. (2), which minimizes λ||θ||1 + ||e - ΦFθ||2 with F the 3D DFT basis. This estimator is explicitly biased toward solutions with sparse DFT coefficients. The 'verification' then computes the DFT of that same estimated d and observes that few coefficients account for 99% of its energy. That concentration is imposed by the objective, not measured from an independent ground-truth MVF. The text even concedes 'the actual MVF can only be estimated' before using the estimate to validate the sparsity premise, so the demonstration is self-referential rather than independent confirmation.
full rationale
The paper's central derivation is not circular. It treats tracked displacement trajectories as CS measurements, assumes DFT sparsity as a physically motivated prior from fluid mechanics, and solves a standard LASSO problem; no fitted parameter is renamed as a prediction, and the restoration results are evaluated against external ground-truth still-water images and compared with SBR-RPCA, LWB, and deep-learning baselines. The center-of-trajectory approximation and the zero-mean wave assumption are genuine correctness risks, but they are assumptions, not circular reductions. The only circular step is the 'Verification of Fourier Sparsity' in Sec. 3.3, where the MVF estimated under the l1-DFT-sparsity objective is used to demonstrate the sparsity of real MVFs; this is self-consistent but does not independently confirm the premise. Because this circularity affects a supporting validation argument rather than the main restoration pipeline or its external benchmarks, a moderate score of 3 is appropriate.
Assumptions & free parameters
free parameters (3)
- lambda (Eq. 2, LASSO regularization weight) =
chosen by cross-validation; optimal value reported stable across datasets
- DT downsampling factor =
8
- track filtering threshold =
3 pixels
assumptions (5)
- domain assumption The complete motion vector field d is sparse or compressible in the 3D DFT basis.
- domain assumption Static planar scene at depth h, orthographic camera looking straight down.
- domain assumption Center of trajectory approximates the true undistorted location (zero-mean symmetric water motion).
- standard math The DFT basis and the row-subsampled identity sampling matrix have coherence 1, supporting CS recovery guarantees.
- domain assumption Tracker output errors are bounded and enter Eq. 1 as small noise eta.
Cite this review
Pith. "Pith review of Restoration of Non-rigidly Distorted Underwater Images using a Combination of Compressive Sensing and Local Polynomial Image Representations." pith.science (2026). https://pith.science/paper/MKAGTBYM
@misc{pith2026190801940,
author = {Pith},
title = {Pith review of: Restoration of Non-rigidly Distorted Underwater Images using a Combination of Compressive Sensing and Local Polynomial Image Representations},
year = {2026},
howpublished = {\url{https://pith.science/paper/MKAGTBYM}},
note = {Machine review of arXiv:1908.01940}
}
read the original abstract
Images of static scenes submerged beneath a wavy water surface exhibit severe non-rigid distortions. The physics of water flow suggests that water surfaces possess spatio-temporal smoothness and temporal periodicity. Hence they possess a sparse representation in the 3D discrete Fourier (DFT) basis. Motivated by this, we pose the task of restoration of such video sequences as a compressed sensing (CS) problem. We begin by tracking a few salient feature points across the frames of a video sequence of the submerged scene. Using these point trajectories, we show that the motion fields at all other (non-tracked) points can be effectively estimated using a typical CS solver. This by itself is a novel contribution in the field of non-rigid motion estimation. We show that this method outperforms state of the art algorithms for underwater image restoration. We further consider a simple optical flow algorithm based on local polynomial expansion of the image frames (PEOF). Surprisingly, we demonstrate that PEOF is more efficient and often outperforms all the state of the art methods in terms of numerical measures. Finally, we demonstrate that a two-stage approach consisting of the CS step followed by PEOF much more accurately preserves the image structure and improves the (visual as well as numerical) video quality as compared to just the PEOF stage.
Figures
Reference graph
Works this paper leans on
- [15]
-
[1]
https://github.com/ jeringeo/CompressiveFlows
Github Repository. https://github.com/ jeringeo/CompressiveFlows. 8
-
[2]
https: //in.mathworks.com/help/vision/ref/ vision.pointtracker-system-object.html
MATLAB implementation for KLT tracker. https: //in.mathworks.com/help/vision/ref/ vision.pointtracker-system-object.html. 3
-
[3]
https://www.cse.iitb.ac.in/ ˜ajitvr/publications.html
Project Page. https://www.cse.iitb.ac.in/ ˜ajitvr/publications.html. 8
- [4]
-
[5]
M. Alterman, Y . Schechner, P. Perona, and J. Shamir. Detect- ing motion through dynamic refraction. IEEE Trans. Pattern Anal. Mach. Intell., 35(1):245–251, 2013. 1
work page 2013
-
[6]
H. Bay, A. Ess, T. Tuytelaars, and L. V . Gool. SURF: Speeded up robust features. Computer Vision and Image Un- derstanding, 110(3):346–359, 2008. 3
work page 2008
-
[7]
E. Cand `es, X. Li, Y . Ma, and J. Wright. Robust principal component analysis? J. ACM, 58(3):11:1–11:37, 2011. 1, 5
work page 2011
Show all 37 references
-
[8]
Candes, J
E. Candes, J. Romberg, and T. Tao. Robust uncertainty prin- ciples: Exact signal reconstruction from highly incomplete. IEEE Trans. on Information Theory, 2006. 4
2006
-
[9]
Donate and E
A. Donate and E. Ribeiro. Improved reconstruction of im- ages distorted by water waves. In Advances in Computer Graphics and Computer Vision, 2007. 1
2007
-
[10]
W. Dong, G. Shi, X. Hu, and Y . Ma. Nonlocal sparse and low-rank regularization for optical flow estimation. IEEE Transactions on Image Processing, 23(10), 2014. 4
2014
-
[11]
Dosovitskiy et al
A. Dosovitskiy et al. Flownet: Learning optical flow with convolutional networks. In ICCV, pages 2758–2766, 2015. 5
2015
-
[12]
Efros, V
A. Efros, V . Isler, J. Shi, and M. Visontai. Seeing through water. In NIPS, pages 393–400, 2004. 1
2004
-
[13]
Farneb ¨ack
G. Farneb ¨ack. Two-frame motion estimation based on poly- nomial expansion. In Image Analysis, pages 363–370, 2003. 2, 4
2003
-
[14]
Fenton and M
J. Fenton and M. Rienecker. A fourier method for solv- ing nonlinear water-wave problems: application to solitary- wave interactions. Journal of Fluid Mechanics , 118:411– 443, 1982. 3
1982
-
[16]
K. Jia, X. Wang, and X. Tang. Optical flow estimation using learned sparse model. In CVPR, 2011. 4
2011
-
[17]
Leutenegger, M
S. Leutenegger, M. Chli, and R. Siegwart. BRISK: Binary robust invariant scalable keypoints. In ICCV, 2011. 3
2011
-
[18]
Z. Li, Z. Murez, D. Kriegman, R. Ramamoorthi, and M. Chandraker. Learning to see through turbulent water. In WACV, pages 512–520, 2018. 1, 2, 5, 6, 7, 8
2018
-
[19]
H. Murase. Surface shape reconstruction of a nonrigid trans- port object using refraction and motion. IEEE Trans. Pattern Anal. Mach. Intell., 14(10):1045–1052, 1992. 1, 2, 3, 5
1992
-
[20]
Oreifej, G
O. Oreifej, G. Shu, T. Pace, and M. Shah. A two-stage re- construction approach for seeing through water. In CVPR, pages 1153–1160, 2011. 1, 2, 5, 7, 8
2011
-
[21]
Y . Qian, Y . Zheng, M. Gong, and Y .-H. Yang. Simultaneous 3d reconstruction for water surface and underwater scene. In ECCV, pages 776–792, 2018. 2
2018
-
[22]
Revaud, P
J. Revaud, P. Weinzaepfel, Z. Harchaoui, and C. Schmid. Epicflow: Edge-preserving interpolation of correspondences for optical flow. In CVPR, 2015. 4, 5
2015
-
[23]
Rosenfeld
M. Rosenfeld. Utilization of fourier decomposition for ana- lyzing time-periodic flows. Computers and Fluids, 24(4):349 – 368, 1995. 3
1995
-
[24]
Rosten, R
E. Rosten, R. Porter, and T. Drummond. Faster and bet- ter: A machine learning approach to corner detection. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(1):105–119, 2010. 3
2010
-
[25]
Seemakurthy and A
K. Seemakurthy and A. N. Rajagopalan. Deskewing of underwater images. IEEE Trans. Image Processing , 24(3):1046–1059, 2015. 1, 2, 5
2015
-
[26]
Shen and Y
X. Shen and Y . Wu. Sparsity model for robust optical flow estimation at motion discontinuities. In CVPR, 2010. 4
2010
-
[27]
Simo-Serra, E
E. Simo-Serra, E. Trulls, L. Ferraz, I. Kokkinos, P. Fua, and F. Moreno-Noguer. Discriminative learning of deep convo- lutional feature point descriptors. In ICCV, pages 118–126,
-
[28]
D. Sun, X. Yang, M.-Y . Liu, and J. Kautz. PWC-Net: CNNs for optical flow using pyramid, warping, and cost volume. In CVPR, 2018. 5
2018
-
[29]
Tian and S
Y . Tian and S. Narasimhan. Seeing through water: Im- age restoration using model-based tracking. In ICCV, pages 2303–2310, 2009. 1, 2, 5, 7, 8
2009
-
[30]
Tian and S
Y . Tian and S. Narasimhan. Globally optimal estimation of nonrigid image distortion. International Journal of Com- puter Vision, 98(3):279–302, 2012. 2
2012
-
[31]
Tian and S
Y . Tian and S. Narasimhan. Theory and practice of hierar- chical data-driven descent for optimal deformation estima- tion. International Journal of Computer Vision, 115(1):44– 67, 2015. 2
2015
-
[32]
D. G. Turlaev and L. S. Dolin. On observing underwater objects through a wavy water surface: A new algorithm for image correction and laboratory experiment. Izvestiya Atmo- sph. Ocean. Phys., 49(3):339345, 2013. 1, 2
2013
-
[33]
Wang and A
Z. Wang and A. C. Bovik. Mean squared error: Love it or leave it? A new look at signal fidelity measures.IEEE Signal Processing Magazine, 26(1):98–117, 2009. 5
2009
-
[34]
R. Ward. Compressed sensing with cross validation. IEEE Trans. Inf. Theor., 55(12):5773–5782, 2009. 4
2009
-
[35]
Z. Wen, D. Fraser, and A. Lambert. Bicoherence: a new lucky region technique in anisoplanatic image restoration. Appl. Opt., 48(32):6111–6119, 2009. 1
2009
-
[36]
Z. Wen, A. Lambert, D. Fraser, and H. Li. Bispectral analysis and recovery of images distorted by a moving water surface. Appl. Opt., 49(33):6376–6384, 2010. 1, 4
2010
-
[37]
Willcox and A
K. Willcox and A. Megretski. Real-Time PDE-Constrained Optimization, chapter Fourier Model Reduction for Large- Scale Applications in Computational Fluid Dynamics. SIAM, 2007. 3
2007
Reviewed August 14, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.