Pith. sign in

REVIEW 3 major objections 4 minor 23 references

Training a U-Net to inpaint PIV velocity fields with vector-based loss functions—cosine similarity and magnitude-index terms—yields significantly better multi-scale reconstructions than standard per-component mean-square error, while a hybr

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

T0 review · deepseek-v4-flash

2026-08-05 04:57 UTC pith:H7PAVYYB

load-bearing objection Parameter-free MI loss genuinely improves multi-scale PIV inpainting over MSE; stronger claims for vector/hybrid losses are undercut by test-set tuning and single runs. the 3 major comments →

arxiv 2509.05787 v1 pith:H7PAVYYB submitted 2025-09-06 physics.flu-dyn cs.LG

Vector-based loss functions for turbulent flow field inpainting

Samuel J. Baker , Shubham Goswami , Xiaohang Fang , Felix C. P. Leach This is my paper
classification physics.flu-dyn cs.LG PACS 47.27.-i
keywords turbulent flowneural networkvector fieldloss functionreconstructioninpaintingparticle image velocimetryU-Net
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

Standard mean-square error treats each velocity component as an independent number, ignoring that PIV data are vector fields whose direction and magnitude carry the physical structure of a turbulent flow. This paper proposes four vector-based loss functions—cosine similarity, magnitude index, a combined vector loss, and a hybrid vector-MSE loss—and trains a U-Net to fill a large central gap in PIV images from the TCC-III engine. The MI and vector losses produce the best multi-scale predictions: KL divergences of 0.014 and 0.013 versus 0.046 for MSE, meaning they reproduce the distribution of velocity magnitudes across scales much more faithfully. The hybrid loss reduces MSE's KL divergence by a factor of 2.7 while incurring only a 0.3% penalty in L2 error. The paper also shows that a pure cosine loss fails, because it ignores magnitudes, and that worst-case predictions remain poor, limiting practical use of the reconstructed fields.

Core claim

The central claim is that embedding the vector character of velocity data into the training objective changes what a neural network learns. Training the same U-Net with a magnitude-index or combined vector loss yields held-out test predictions whose KL divergence (0.013–0.014) is about 3.5 times lower than that of the MSE-trained network (0.046), while the hybrid loss gives a KL divergence of 0.017 with an L2 error only 0.3% above MSE. In visual terms, the vector-loss models capture high-speed diagonal bands and multi-scale magnitude distributions that the MSE model flattens. Conversely, a cosine-similarity loss alone makes the network collapse to small, near-uniform vectors, since it contai

What carries the argument

The load-bearing objects are four loss functions defined per pair of predicted and ground-truth velocity vectors a and b: the cosine similarity loss L_cos = ½(1 − a·b/(|a||b|)), the magnitude index L_MI = |a−b|/(|a|+|b|), the vector loss L_vec = α₁L_cos + (1 − α₁)L_MI, and the hybrid L_hyb = α₂L_vec + (1 − α₂)L_MSE, with α₁ = 0.3 and α₂ = 0.2 chosen by grid search. These losses replace or augment the component-wise MSE as the training objective for a U-Net that reconstructs a 13×13 central gap in 128×128 PIV images. Their role is to make the training signal depend on direction alignment (cosine) and relative magnitude difference (MI) rather than on absolute coordinate-wise error, which the a

Load-bearing premise

The paper trains exactly one model per loss function and reports no confidence intervals or significance tests, so the claim that vector losses 'significantly' improve multi-scale predictions rests on the assumption that a single training run is a reliable estimate of each loss's typical performance.

What would settle it

Train the same U-Net with, say, ten random seeds per loss function and compare the distributions of held-out KL divergences; if the MSE and vector-loss distributions overlap substantially (or the overlap exceeds the reported gap), the claimed improvement is not robust. A secondary check: measure the variance of predicted vector magnitudes for the cosine-loss model—if it is near zero across all snapshots, the collapse to non-physical uniform vectors is confirmed.

Watch this falsifier. Get emailed when new claim-graph text bears on it.

Share X Bluesky LinkedIn Reddit HN

If this is right

  • Models trained with the MI or vector losses reproduce the multi-scale distribution of velocity magnitudes significantly better than MSE, as measured by KL divergence on held-out crank-angle data.
  • The hybrid loss offers a practical compromise: near-MSE pixel accuracy with a 2.7x lower KL divergence, so users can choose the balance between point-wise and structural fidelity.
  • A pure cosine-similarity loss is insufficient by itself; without a magnitude term it drives the network to small, near-uniform, non-physical vectors.
  • The worst 5% of predictions still deviate strongly from ground truth for all losses, indicating that large-gap inpainting remains ill-posed and that further constraints (e.g., pressure or the third velocity component) are likely needed.
  • The choice of loss function is a physics-aware design lever: embedding the vector nature of the data into training improves the fidelity of scientific ML reconstructs without changing the model architecture or dataset.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Inference: the same vector-loss recipe could transfer to other vector-field learning tasks—e.g., super-resolution, temporal interpolation, or CFD surrogate modelling—where per-component MSE also ignores directional structure; the hybrid form suggests a general template of 'structural loss + pixel loss'.
  • Inference: the cosine-failure result implies a design rule for similarity-based losses in physical settings: any angular similarity term should be paired with a magnitude or scale term; this is testable by adding such a pair to other contrastive or embedding losses on vector data.
  • Inference: a multi-seed reproducibility study would settle whether the reported KL gaps exceed run-to-run variability, since the paper trains one model per loss; treating the single-run numbers as provisional is warranted.
  • Inference: the authors' proposed future work of correlating worst-case performance with a physical/textural quantity could be turned into an uncertainty estimator—flagging images whose predicted fields are likely unreliable before they are used downstream.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

3 major / 4 minor

Summary. The paper proposes four vector-based loss functions (cosine similarity, magnitude index, vector, and hybrid) for inpainting missing velocity vectors in turbulent PIV flow fields, and evaluates them against a standard MSE loss using a U-Net on the EngineBench TCC-III dataset. The central claim is that vector-based losses, particularly the magnitude index (MI) and vector losses, significantly improve multi-scale prediction quality as measured by KL divergence, while the hybrid loss provides a favorable trade-off with pixel-wise L2 error. The reported median KL divergences are 0.014 (MI), 0.013 (vector), and 0.017 (hybrid) versus 0.046 for MSE, with L2 errors within 0.3% of MSE for the hybrid. The authors conclude that vector-based losses preserve multi-scale flow patterns better than MSE, at some cost in pixel-wise accuracy.

Significance. If the central claim holds, the contribution is practical and of interest to the fluid-mechanics ML community: simple, physically-motivated loss functions that improve the reconstruction of multi-scale turbulent structures without architectural changes. The MI loss is parameter-free, which is a notable strength—it provides a direct comparison with MSE that is not confounded by hyperparameter tuning. The use of the public EngineBench benchmark and a well-defined inpainting task also supports reproducibility and comparability. However, the significance of the quantitative improvement is currently undermined by the lack of statistical replication and by test-set-based hyperparameter selection for the vector and hybrid losses. The paper's contribution is therefore promising but not yet rigorously established.

major comments (3)
  1. [Methods, Loss functions] The manuscript states that the user-defined parameters α1 and α2 were 'chosen using a grid search of values and chosen based on the best L2 performance on the test data.' This is test-set leakage: the reported L2 and KL for the vector and hybrid losses are not evaluated on a truly held-out set, because the hyperparameters were selected to optimize the same test metric. The improvement of the hybrid loss over MSE (0.3% L2 penalty, 2.7× lower KL) is therefore partly an artifact of tuning to the test set. Please re-select α1 and α2 on a validation split (or via cross-validation) and report results on a fully untouched test set, or explicitly justify why the tuning does not affect the comparison.
  2. [Results, Table 2] Each loss function is trained only once, with no multiple seeds, confidence intervals, or significance tests. The abstract and conclusions state that vector-based losses 'significantly improve' multi-scale predictions, but the 0.013–0.014 versus 0.046 KL gap could in principle be within run-to-run variability of a single U-Net training. The reported floors also overlap substantially (e.g., vector floor 0.249 vs. MSE floor 0.364). Please train multiple independent models per loss (at least 3–5 seeds), report mean and standard deviation (or box plots) for L2 and KL, and, if appropriate, apply a paired statistical test across test snapshots or seeds. The parameter-free MI loss is the cleanest comparison and may alone support the claim, but it still needs replication.
  3. [Conclusions] The claim that 'the MI and vector losses are able to significantly improve the prediction of multi-scale turbulent flow patterns compared to a standard MSE loss' is load-bearing and appears verbatim in the conclusions. Given the test-set tuning and single-run limitation, the word 'significantly' should be either supported by statistical evidence or replaced with a more neutral formulation such as 'were observed to improve.' The current evidence supports a promising trend, not a statistically established improvement.
minor comments (4)
  1. [General] Several figure references appear as 'Error! Reference source not found.' (e.g., in Methods and Results). Please fix the cross-references before submission.
  2. [Methods, Dataset] Typo: 'turbulent flow s' should be 'turbulent flows.' Also, the caption of Figure 2 contains 'for to provide the model inputs,' which is ungrammatical.
  3. [Results, Table 2] The table heading 'L2 KL' is ambiguous. It would be clearer to use separate columns (or a multi-level header) for 'Normalized L2' and 'KL divergence.' Also define 'floor' precisely: it is stated as 'bottom fifth percentile,' but the reader may wonder whether lower or higher values are worse; given the metrics, clarify that the floor is the 5th percentile of errors (i.e., worst-case 5%).
  4. [Methods, Loss functions] The MSE loss is not explicitly defined in the paper, though it is referenced. For completeness, please provide the standard definition L_MSE = ||a - b||^2 (or the normalized variant used).

Circularity Check

1 steps flagged

Test-set-tuned loss weights compromise the reported L2 comparison, but the central KL-divergence claim is not definitionally forced.

specific steps
  1. fitted input called prediction [Methods – Loss functions; Results – Table 2]
    "The user-defined parameters (α1 and α2) were chosen using a grid search of values and chosen based on the best L2 performance on the test data. Therefore, 𝛼1 = 0.3 and 𝛼2 = 0.2 were used for the entirety of this study."

    The vector and hybrid loss weights are selected by minimizing the test-set L2 error; the same test-set L2 values are then reported as the quantitative outcome for those losses (Table 2). Consequently, the statement that the hybrid loss has an L2 error only 0.3% above MSE is not an independent hold-out evaluation: the loss form was tuned on the exact metric and data split used to produce that number. The comparison is therefore partly an artifact of test-set selection rather than a prediction of held-out performance. However, the KL-divergence metric was not used in the grid search, and the parameter-free MI loss also shows a large KL improvement, so the central multi-scale claim retains independent empirical content.

full rationale

The paper makes no formal derivation claim; it is an empirical comparison of loss functions for flow inpainting. The only load-bearing reduction I can exhibit is the explicit test-set tuning of α1 and α2 on the L2 metric followed by the use of that same test set to report L2 outcomes. This makes the reported L2 figures for the vector and hybrid losses partly self-referential, and it should be treated as evaluation leakage rather than a true held-out result. It does not, however, force the main conclusion: the KL-divergence improvement is obtained with the parameter-free MI loss as well as the tuned vector loss, and the hyperparameters were not selected on KL. I found no self-citation chain, uniqueness import, ansatz-by-citation, or renaming of a known result. The absence of multiple seeds and confidence intervals is a statistical robustness concern, not circularity, and I have not counted it in the score. Overall the work is mostly self-contained; the score reflects the one genuine test-set-contamination step.

Axiom & Free-Parameter Ledger

2 free parameters · 3 axioms · 0 invented entities

The paper introduces two free hyperparameters (alpha1, alpha2) that are fit to the test data, which is a form of parameter fitting. It also relies on the unstated assumption that the experimental PIV data are accurate ground truth and that the KL metric captures multi-scale fidelity. No new physical entities are proposed.

free parameters (2)
  • alpha1 = 0.3
    Weight between cosine similarity and MI in the vector loss; chosen via grid search on the test data (Section II-C), affecting L2 and KL results for the vector loss.
  • alpha2 = 0.2
    Weight between vector loss and MSE in the hybrid loss; chosen via grid search on the test data (Section II-C), directly influencing the hybrid loss's reported L2, which is used to claim the compromise.
axioms (3)
  • domain assumption PIV measurements in EngineBench are treated as ground truth labels without accounting for PIV noise or measurement error.
    The training and test targets are the published PIV snapshots; any systematic error in the experimental data is inherited by the models and the reported metrics.
  • domain assumption KL divergence computed on the test snapshots is a valid and sufficient proxy for multi-scale flow pattern fidelity.
    The paper's main claim of improved multi-scale prediction rests on the KL metric (Section III), but the metric's binning, normalization, and sensitivity are not described or justified.
  • domain assumption The U-Net architecture and its MONAI default hyperparameters are suitable for the inpainting task.
    The model is fixed from prior work [10,21] with no ablation or architecture sensitivity analysis; the conclusions may depend on this specific model choice.

pith-pipeline@v1.3.0-alltime-deepseek · 6141 in / 9544 out tokens · 109776 ms · 2026-08-05T04:57:14.225457+00:00 · methodology

0 comments
Cite this review

Pith. "Pith review of Vector-based loss functions for turbulent flow field inpainting." pith.science (2026). https://pith.science/paper/H7PAVYYB

@misc{pith2026250905787,
  author       = {Pith},
  title        = {Pith review of: Vector-based loss functions for turbulent flow field inpainting},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/H7PAVYYB}},
  note         = {Machine review of arXiv:2509.05787}
}
Share X Bluesky LinkedIn Reddit HN
read the original abstract

When developing scientific machine learning (ML) approaches, it is often beneficial to embed knowledge of the physical system in question into the training process. One way to achieve this is by leveraging the specific characteristics of the data at hand. In the case of turbulent flows, fluid velocities can be measured and recorded as multi-component vectors at discrete points in space, using techniques such as particle image velocimetry (PIV) or computational fluid mechanics (CFD). However, the vectorised nature of the data is ignored by standard ML approaches, as widely-used loss functions such as the mean-square error treat each component of a velocity vector in isolation. Therefore, the aim of this work is to better preserve the physical characteristics of the data by introducing loss functions that utilise vector similarity metrics. To this end, vector-based loss functions are developed here and implemented alongside a U-Net model for a turbulent flow field inpainting problem, amounting to the prediction of velocity vectors inside large gaps in PIV images. The intention is for the inpainting task to pose a significant challenge for the ML models in order to shed light on their capabilities. The test case uses PIV data from the highly turbulent flow in the well-known Transparent Combustion Chamber III (TCC-III) engine. Loss functions based on the cosine similarity and vector magnitude differences are proposed; the results show that the vector-based loss functions lead to significantly improved predictions of multi-scale flow patterns, while a hybrid (vector and mean-square error) loss function enables a good compromise to be found between preserving multi-scale behaviour and pixel-wise accuracy.

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

23 extracted references · 22 canonical work pages · 1 internal anchor

  1. [1]

    Image inpainting: A review,

    O. Elharrouss, N. Almaadeed, S. Al-Maadeed, and Y. Akbari, “Image inpainting: A review,” Neural Process Lett, vol. 51, pp. 2007–2028, 2020

    work page 2007

  2. [2]

    A comprehensive review of past and present image inpainting methods,

    J. Jam, C. Kendrick, K. Walker, V. Drouard, J. G.-S. Hsu, and M. H. Yap, “A comprehensive review of past and present image inpainting methods,” Computer Vision and Image Understanding, vol. 203, p. 103147, 2021

    work page 2021

  3. [3]

    R. J. Adrian and J. Westerweel, Particle image velocimetry. Cambridge University Press, 2011

    work page 2011

  4. [4]

    Measurement of laminar, transitional and turbulent pipe flow using stereoscopic-PIV,

    C. W. H. Van Doorne and J. Westerweel, “Measurement of laminar, transitional and turbulent pipe flow using stereoscopic-PIV,” Exp Fluids, vol. 42, pp. 259–279, 2007

    work page 2007

  5. [5]

    Development and evaluation of gappy-POD as a data reconstruction technique for noisy PIV measurements in gas turbine combustors,

    P. Saini, C. M. Arndt, and A. M. Steinberg, “Development and evaluation of gappy-POD as a data reconstruction technique for noisy PIV measurements in gas turbine combustors,” Exp Fluids, vol. 57, pp. 1– 15, 2016

    work page 2016

  6. [6]

    Extracting vector magnitudes of dominant structures in a cyclic engine flow with dimensionality reduction,

    S. J. Baker et al., “Extracting vector magnitudes of dominant structures in a cyclic engine flow with dimensionality reduction,” Physics of Fluids, vol. 36, no. 2, 2024. Proceedings of the 1st international Symposium on AI and Fluid Mechanics Paper No [leave blank] *Corresponding Author: Samuel J. Baker, samuel.baker@eng.ox.ac.uk

    work page 2024

  7. [7]

    Dynamic Mode Decomposition for the Comparison of Engine In-Cylinder Flow Fields from Particle Image Velocimetry (PIV) and Reynolds-Averaged Navier–Stokes (RANS) Simulations,

    S. Baker et al., “Dynamic Mode Decomposition for the Comparison of Engine In-Cylinder Flow Fields from Particle Image Velocimetry (PIV) and Reynolds-Averaged Navier–Stokes (RANS) Simulations,” Flow Turbul Combust, no. 0123456789, 2023, doi: 10.1007/s10494-023-00424-3

    work page doi:10.1007/s10494-023-00424-3 2023

  8. [8]

    Gappy data: To Krig or not to Krig?,

    H. Gunes, S. Sirisup, and G. E. Karniadakis, “Gappy data: To Krig or not to Krig?,” J Comput Phys, vol. 212, no. 1, pp. 358–382, 2006

    work page 2006

  9. [9]

    Gappy spectral proper orthogonal decomposition,

    A. Nekkanti and O. T. Schmidt, “Gappy spectral proper orthogonal decomposition,” J Comput Phys, vol. 478, p. 111950, 2023

    work page 2023

  10. [10]

    EngineBench: Flow Reconstruction in the Transparent Combustion Chamber III Optical Engine

    S. J. Baker, M. A. Hobley, I. Scherl, X. Fang, F. C. Leach, and M. H. Davy, “EngineBench: Flow Reconstruction in the Transparent Combustion Chamber III Optical Engine,” arXiv preprint arXiv:2406.03325, 2024

    work page internal anchor Pith review Pith/arXiv arXiv 2024

  11. [11]

    Deep learning method for super-resolution reconstruction of the spatio-temporal flow field,

    K. Bao, X. Zhang, W. Peng, and W. Yao, “Deep learning method for super-resolution reconstruction of the spatio-temporal flow field,” Advances in Aerodynamics, vol. 5, no. 1, p. 19, 2023

    work page 2023

  12. [12]

    Improved deep learning method for accurate flow field reconstruction from sparse data,

    J. Zhang, J. Liu, and Z. Huang, “Improved deep learning method for accurate flow field reconstruction from sparse data,” Ocean Engineering, vol. 280, p. 114902, 2023

    work page 2023

  13. [13]

    Experimental velocity data estimation for imperfect particle images using machine learning,

    M. Morimoto, K. Fukami, and K. Fukagata, “Experimental velocity data estimation for imperfect particle images using machine learning,” Physics of Fluids, vol. 33, no. 8, 2021

    work page 2021

  14. [14]

    Multi-scale reconstruction of turbulent rotating flows with proper orthogonal decomposition and generative adversarial networks,

    T. Li, M. Buzzicotti, L. Biferale, F. Bonaccorso, S. Chen, and M. Wan, “Multi-scale reconstruction of turbulent rotating flows with proper orthogonal decomposition and generative adversarial networks,” J Fluid Mech, vol. 971, 2023

    work page 2023

  15. [15]

    Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,

    M. Raissi, P. Perdikaris, and G. E. Karniadakis, “Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations,” J Comput Phys, vol. 378, pp. 686–707, 2019

    work page 2019

  16. [16]

    One loss for all: Deep hashing with a single cosine similarity based learning objective,

    JT Hoe, KW Ng, T Zhang, CS Chan, YZ Song, and T Xiang, “One loss for all: Deep hashing with a single cosine similarity based learning objective,” Adv Neural Inf Process Syst, vol. 34, pp. 24286–24298, 2021

    work page 2021

  17. [17]

    Learning cross-modal embeddings for cooking recipes and food images,

    A. Salvador et al., “Learning cross-modal embeddings for cooking recipes and food images,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, pp. 3020–3028

    work page 2017

  18. [18]

    Deep learning on small datasets without pre-training using cosine loss,

    B. Barz and J. Denzler, “Deep learning on small datasets without pre-training using cosine loss,” in Proceedings of the IEEE/CVF winter conference on applications of computer vision, 2020, pp. 1371–1380

    work page 2020

  19. [19]

    TCC-III engine benchmark for large- eddy simulation of IC engine flows,

    P. Schiffmann, S. Gupta, D. Reuss, V. Sick, X. Yang, and T.-W. Kuo, “TCC-III engine benchmark for large- eddy simulation of IC engine flows,” Oil & Gas Science and Technology–Rev. IFP Energies nouvelle, vol. 71, no. 1, p. 3, 2016

    work page 2016

  20. [20]

    U-net: Convolutional networks for biomedical image segmentation,

    O. Ronneberger, P. Fischer, and T. Brox, “U-net: Convolutional networks for biomedical image segmentation,” in Medical image computing and computer-assisted intervention–MICCAI 2015, 2015, pp. 234–241

    work page 2015

  21. [21]

    Monai: An open-source framework for deep learning in healthcare,

    M. J. Cardoso et al., “Monai: An open-source framework for deep learning in healthcare,” arXiv preprint arXiv:2211.02701, 2022

    Pith/arXiv arXiv 2022

  22. [22]

    Context encoders: Feature learning by inpainting,

    D. Pathak, P. Krahenbuhl, J. Donahue, T. Darrell, and A. A. Efros, “Context encoders: Feature learning by inpainting,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2016, pp. 2536–2544

    work page 2016

  23. [23]

    Numeric Metrics for Capturing Variations in Flow Fields: An Improvement Towards a Robust Comparison of Vector Fields,

    M. Nowruzi, S. Baker, F. Leach, and X. Fang, “Numeric Metrics for Capturing Variations in Flow Fields: An Improvement Towards a Robust Comparison of Vector Fields,” Flow Turbul Combust, 2025, doi: 10.1007/s10494-025-00637-8

    work page doi:10.1007/s10494-025-00637-8 2025