REVIEW 3 major objections 4 minor 46 references
Seeing the Many: Exploring Parameter Distributions Conditioned on Features in Surrogates
T0 review · 3 major / 4 minor · reviewed 2026-08-15 · deepseek-v4-flash
Pith's one-line read The paper proposes sampling from a Bayesian posterior to recover the full distribution of parameters that produce a user-specified simulation feature, rather than searching for a few optimal matches.
desk verdict Useful and novel distributional inverse-design paper; the main reservation is that 'plausible' parameters are never validated against the true simulator. read the letter →
The pith
A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.
The reading
What carries the argument
At the center is a neural field surrogate $g(\mathbf{x}, \boldsymbol{\theta})$ that predicts field values at spatial locations $\mathbf{x}$ given simulation parameters $\boldsymbol{\theta}$, so the same network serves both forward evaluation and feature-based queries. The prior is a kernel density estimate whose bandwidths come from nearest-neighbor distances measured with a Fisher-information metric on the surrogate's output, which makes the density drop in regions where small parameter changes produce large field changes; this is the mechanism that encodes surrogate error into the prior. The feature likelihood compares the surrogate's predicted field over the user's selected region to the specified feature, and Hamiltonian Monte Carlo with leapfrog integration samples the resulting posterior. Pairwise binned heatmaps of the collected samples then approximate all two-dimensional marginals of the posterior for visualization.
What would settle it
Take a feature located in a region where the surrogate is known to be inaccurate, for instance between two training configurations where the density map in Appendix B shows a low-density streak; run the full posterior sampler, then evaluate the high-density samples on the ground-truth simulator. If those samples frequently fail to reproduce the specified feature, the density prior has not captured surrogate error. A cheaper check is to compare the HMC sample distribution against a dense-grid posterior for the one-dimensional viscosity example and verify that the sample density matches the true posterior where the prior is peaked.
Extended reading notes
Core claim
The central claim is that the inverse problem 'which input parameters generate this output feature?' should be solved as Bayesian posterior sampling rather than as optimization. The paper defines a prior over parameters via a density estimator that rewards configurations close to training points in both input and output space, and a likelihood that measures how well the surrogate's field over a user-selected neighborhood matches the target feature. The posterior is then proportional to the product of this likelihood and prior, and Hamiltonian Monte Carlo is used to draw samples whose density approximates the posterior. The paper argues this procedure is fast enough for progressive, interactive visualization and that the resulting sample distributions reveal the full set of plausible inverse solutions, including approximate matches and the correlations among parameters that produce them.
Load-bearing premise
The load-bearing premise is that closeness to training parameters, measured with a Fisher-information distance (a distance that reflects how quickly simulated outputs change as parameters move) and nearest-neighbor bandwidths, is a reliable proxy for where the surrogate's predictions can be trusted; if that correspondence fails, the posterior will confidently sample parameters whose surrogate outputs do not match the true simulation.
Editorial extensions
If this is right
- A user can specify a local feature, such as a vortex or rotation region, and immediately see every parameter configuration that approximately reproduces it, rather than a single optimized configuration.
- Because the prior density is high only near training data, the collected samples are restricted to regions where the surrogate is likely to be accurate, so the visualization does not present extrapolated matches as credible.
- Comparing the posterior for two different features highlights which parameter combinations distinguish one feature from the other, supporting feature-driven hypothesis generation.
- The density prior gives a surrogate-accuracy signal that is cheaper than standard deep-ensemble or dropout uncertainty estimates while remaining predictive of error, enabling lower-latency interaction.
- Because the sampler updates progressively after burn-in, the interface can show a coarse posterior quickly and refine it, making high-dimensional exploration practical in a visual loop.
Reading between the lines
- The paper does not pursue this, but the same density estimate could serve as an acquisition function: configurations with low prior density yet high feature likelihood mark regions where the surrogate most needs additional training simulations.
- A natural extension, which the paper leaves to future work, is to decouple the surrogate-accuracy prior from the feature likelihood so that model uncertainty can be inspected separately from feature-based querying.
- The framework should transfer to time-dependent or gradient-derived features whenever the feature can be written as a differentiable function of the neural field; the paper notes this but does not implement it.
- One implicit empirical claim worth testing is that Fisher-information distance orders parameter-space proximity better than Euclidean distance for surrogate error; a direct comparison of the two metrics on the same datasets would settle it.
Signed reviews
Editorial analysis
A structured set of objections, weighed in public.
Referee Report
Summary. The paper proposes a Bayesian framework for sampling distributions of simulation parameters conditioned on a user-specified output feature in neural-field surrogates. The prior is a density estimate over training parameters based on Fisher information and Euclidean distances, and the likelihood is derived from the surrogate's negative log-likelihood between the predicted feature patch and the target feature (Eqs. 9-10). Hamiltonian Monte Carlo draws samples from the resulting posterior, and a scatterplot-matrix interface displays the marginal distributions. The paper evaluates HMC convergence via R-hat and MMD statistics and assesses feature-match quality using the same NLL, then demonstrates the workflow on three simulation datasets.
Significance. If the posterior samples were validated against ground-truth simulations, the method would be a practical and useful contribution to surrogate-based inverse exploration in visualization and scientific analysis. The paper has clear strengths: it ships source code, provides explicit HMC diagnostics (R-hat and MMD convergence), and demonstrates a progressive sampling scheme that supports interactive visualization. However, the central claim that sampled parameters are 'plausible' and 'generate a target output feature' is not yet established, because the evaluation is self-consistent with the surrogate that defines the posterior. The density prior is an interesting idea, but its reliability as a proxy for surrogate approximation error is asserted rather than demonstrated.
major comments (3)
- [Section 5 and Appendix C, Eqs. (9)-(10)] The central claim that sampled parameters are plausible and generate a target output feature is validated only against the surrogate's negative log-likelihood that defines the posterior. Figures 14 and 15 evaluate feature-match quality using the same NLL (Eq. 9) that appears in the likelihood, making the evaluation circular. No experiment runs the HMC-sampled parameters through the actual simulator that generated the training data to verify that the requested feature is present. I recommend adding ground-truth validation: for held-out target features, sample from the posterior, run the simulator, and measure the feature-match error, comparing against uniform random sampling and a baseline optimizer.
- [Section 3.4 and Appendix B] The density prior is the only mechanism protecting against surrogate approximation error, yet its validity as a proxy is not quantitatively established. The paper asserts in Section 1 that the density estimate is 'comparable in being predictive of surrogate error' to standard uncertainty estimates, but no experiment directly measures the correlation between the density and actual surrogate error on held-out inputs. Appendix B provides only a qualitative illustration for the Circle simulation. I recommend a quantitative study: compute the density and the true surrogate error over a held-out set of parameter configurations, report their correlation, and test sensitivity to the bandwidths sigma_f and sigma_s.
- [Section 3.5 and Appendix C] The HMC diagnostics (R-hat, MMD) demonstrate convergence to the surrogate-defined posterior, but they do not validate the correctness of that posterior relative to the true simulation. The free parameters C (feature likelihood scale), sigma_f, sigma_s, step size, and leapfrog steps are set heuristically, and no sensitivity analysis is reported. Without such an analysis, it is unclear how robust the displayed distributions are to reasonable variations in these choices. Please add a sensitivity study over C, sigma_f, sigma_s, and HMC step size, and report how the posterior marginals change.
minor comments (4)
- [Throughout] In several places the text uses 'c.f.' where the standard abbreviation is 'cf.' (e.g., Section 3.5 and Section 4); this is a minor style issue.
- [Section 6] The paragraph beginning 'Last, throughout the paper...' should use 'Finally' and could be split into two paragraphs for readability.
- [Section 5.2] The case-study observations are presented as user-made findings, but no user study is reported; the text should clarify that these are illustrative scenario walkthroughs rather than empirical evaluations.
- [Section 5.1 and Figure 12] The R-hat histograms would be easier to interpret if the text stated the specific threshold used to judge mixing (e.g., R-hat below 1.1), rather than only saying 'quite close to 1.'
Circularity Check
Feature-match validation is self-referential: the NLL used as the quality metric is the same likelihood that defines the posterior, so the reported match quality is forced by construction.
-
self definitional
[Appendix C (Figs. 14-15); Sec. 3.5 (posterior sampling)]
"We next show the quality of the feature matches found from our HMC sampler. Specifically, in Fig. 14 we show the negative log likelihood (NLL), specifically Eq. 9, normalized via dividing by the average vector norm in the respective simulations. Consistent with the previous results, we find 8 - 16 leapfrog steps gives small NLL values; visual inspection confirms that such values lead to good approximate feature matches."
The posterior (c.f. Eq. 3) is defined as the feature likelihood times the prior, and Eq. 9 is the negative log of that feature likelihood computed from the surrogate. HMC is run precisely to draw samples from this posterior (Sec. 3.5), i.e., to minimize Eq. 9 subject to the prior. Therefore reporting that the collected samples have small NLL, and that they beat uniform random sampling, is a check that the sampler is minimizing its own target function, not independent evidence that the sampled parameters generate the target feature in the true simulation. The evaluation metric and the optimized objective are the same expression by construction.
full rationale
The core derivation—posterior = density prior × feature likelihood, followed by HMC sampling—is internally consistent and not circular: the prior is an explicit modeling assumption based on closeness to training parameters, and the HMC procedure is a standard inference algorithm applied to a well-defined target. No load-bearing self-citation, imported uniqueness theorem, or ansatz smuggled via citation was found; self-citations (e.g., [2], [24]) are contextual only. The identified circularity is confined to validation: feature-match quality is measured with the same NLL (Eq. 9) that defines the likelihood in the posterior, so low NLL values and the comparison against uniform sampling are forced by construction rather than confirming 'plausible' parameters with respect to the true simulation. The paper's Discussion partially acknowledges this by noting that surrogate reliability is a central concern and that the approach conflates feature satisfaction with model uncertainty ('our approach emphasized parameters that, both, satisfy a given feature, and are likely to give good approximations to simulations. It would be worth decoupling these quantities'). Because the central sampling method still has independent content and the circularity is in the evaluation rather than the derivation, a score of 6 (partial circularity) is appropriate rather than a higher score.
Assumptions & free parameters
free parameters (5)
- sigma_f (FIM bandwidth) =
average nearest-neighbor Fisher information distance over training parameters
- sigma_s (Euclidean bandwidth) =
6 times average nearest-neighbor Euclidean distance over training parameters
- C (feature likelihood scale) =
per simulation, chosen so Eq. 10 yields reasonable approximations (e.g., 0.05 at average vector distance 0.2)
- HMC step size =
tuned online for approximately 50% Metropolis-Hastings acceptance
- HMC chain hyperparameters =
10 leapfrog steps nominal, 8-16 evaluated; 100-200 steps per chain; 50 burn-in; 1,000 chains
assumptions (4)
- domain assumption The neural surrogate provides a sufficiently accurate approximation of the simulation over the parameter regions the user explores.
- ad hoc to paper Closeness to training data in input-output space, measured with a Fisher information metric, is a valid proxy for surrogate approximation error.
- domain assumption A Gaussian likelihood with a scalar scale C adequately models whether a parameter configuration matches a user-specified feature.
- domain assumption HMC with the chosen hyperparameters converges to the posterior quickly enough for interactive exploration.
Cite this review
Pith. "Pith review of Seeing the Many: Exploring Parameter Distributions Conditioned on Features in Surrogates." pith.science (2026). https://pith.science/paper/S362PSI7
@misc{pith2026250813088,
author = {Pith},
title = {Pith review of: Seeing the Many: Exploring Parameter Distributions Conditioned on Features in Surrogates},
year = {2026},
howpublished = {\url{https://pith.science/paper/S362PSI7}},
note = {Machine review of arXiv:2508.13088}
}
read the original abstract
Recently, neural surrogate models have emerged as a compelling alternative to traditional simulation workflows. This is accomplished by modeling the underlying function of scientific simulations, removing the need to run expensive simulations. Beyond just mapping from input parameter to output, surrogates have also been shown useful for inverse problems: output to input parameters. Inverse problems can be understood as search, where we aim to find parameters whose surrogate outputs contain a specified feature. Yet finding these parameters can be costly, especially for high-dimensional parameter spaces. Thus, existing surrogate-based solutions primarily focus on finding a small set of matching parameters, in the process overlooking the broader picture of plausible parameters. Our work aims to model and visualize the distribution of possible input parameters that produce a given output feature. To achieve this goal, we aim to address two challenges: (1) the approximation error inherent in the surrogate model and (2) forming the parameter distribution in an interactive manner. We model error via density estimation, reporting high density only if a given parameter configuration is close to training parameters, measured both over the input and output space. Our density estimate is used to form a prior belief on parameters, and when combined with a likelihood on features, gives us an efficient way to sample plausible parameter configurations that generate a target output feature. We demonstrate the usability of our solution through a visualization interface by performing feature-driven parameter analysis over the input parameter space of three simulation datasets. Source code is available at https://github.com/matthewberger/seeing-the-many
Reference graph
Works this paper leans on
- [1]
- [2]
- [3]
-
[4]
P. Blanchard, D. J. Higham, and N. J. Higham. Accurately computing the log-sum-exp and softmax functions. IMA Journal of Numerical Analysis, 41(4):2311–2330, 2021. 12
work page 2021
-
[5]
S. Bruckner and T. M ¨oller. Result-driven exploration of simulation parameter spaces for visual effects design. IEEE Transactions on Vi- sualization and Computer Graphics , 16(6):1468–1476, 2010. 2
work page 2010
-
[6]
A. Challapalli, D. Patel, and G. Li. Inverse machine learning frame- work for optimizing lightweight metamaterials. Materials & Design , 208:109937, 2021. 1, 2
work page 2021
-
[7]
Y .-T. Chen, N. Shi, X. Luo, W. Xu, and H.-W. Shen. Explorable inr: an implicit neural representation for ensemble simulation enabling ef- ficient spatial and parameter exploration. IEEE Transactions on Visu- alization and Computer Graphics , 2025. 1, 2, 3, 9
work page 2025
-
[8]
D. Coffey, C.-L. Lin, A. G. Erdman, and D. F. Keefe. Design by drag- ging: An interface for creative forward and inverse design with sim- ulation ensembles. IEEE transactions on visualization and computer graphics, 19(12):2783–2791, 2013. 2, 3
work page 2013
Show all 46 references
-
[9]
Dupont, H
E. Dupont, H. Kim, S. Eslami, D. Rezende, and D. Rosenbaum. From data to functa: Your data point is a function and you can treat it like one. arXiv preprint arXiv:2201.12204, 2022. 4
2022 arXiv
-
[10]
Farokhmanesh, K
F. Farokhmanesh, K. H ¨ohlein, C. Neuhauser, and R. Westermann. Neural fields for interactive visualization of statistical dependencies in 3d simulation ensembles. InInternational Symposium on Vision, Mod- eling, and Visualization, pp. 81–88. Eurographics Association, 2023. 2
2023
-
[11]
Fernandes, L
M. Fernandes, L. Walls, S. Munson, J. Hullman, and M. Kay. Un- certainty displays using quantile dotplots or cdfs improve transit decision-making. In Proceedings of the 2018 CHI conference on hu- man factors in computing systems , pp. 1–12, 2018. 5
2018
-
[12]
Gal and Z
Y . Gal and Z. Ghahramani. Dropout as a bayesian approximation: Representing model uncertainty in deep learning. In international conference on machine learning, pp. 1050–1059. PMLR, 2016. 2
2016
-
[13]
Gelman, J
A. Gelman, J. B. Carlin, H. S. Stern, and D. B. Rubin. Bayesian data analysis. Chapman and Hall/CRC, 1995. 5, 7, 8, 12
1995
-
[14]
L. Goli, C. Reading, S. Sell ´an, A. Jacobson, and A. Tagliasacchi. Bayes’ rays: Uncertainty quantification for neural radiance fields. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 20061–20070, 2024. 2
2024
-
[15]
Gosink, K
L. Gosink, K. Bensema, T. Pulsipher, H. Obermaier, M. Henry, H. Childs, and K. I. Joy. Characterizing and visualizing predictive uncertainty in numerical ensembles through bayesian model aver- aging. IEEE transactions on visualization and computer graphics , 19(12):2703–2712, 2013. 3
2013
-
[16]
Gretton, K
A. Gretton, K. M. Borgwardt, M. J. Rasch, B. Sch ¨olkopf, and A. Smola. A kernel two-sample test. The journal of machine learning research, 13(1):723–773, 2012. 14
2012
-
[17]
Han and C
J. Han and C. Wang. Coordnet: Data generation and visualization gen- eration for time-varying volumes via a coordinate-based neural net- work. IEEE Transactions on Visualization and Computer Graphics , 29(12):4951–4963, 2022. 2
2022
-
[18]
Hazarika, H
S. Hazarika, H. Li, K.-C. Wang, H.-W. Shen, and C.-S. Chou. Nnva: Neural network assisted visual analysis of yeast cell polarization sim- ulation. IEEE Transactions on Visualization and Computer Graphics, 26(1):34–44, 2019. 2
2019
-
[19]
W. He, J. Wang, H. Guo, K.-C. Wang, H.-W. Shen, M. Raj, Y . S. Nashed, and T. Peterka. Insitunet: Deep image synthesis for parame- ter space exploration of ensemble simulations. IEEE transactions on visualization and computer graphics , 26(1):23–33, 2019. 2
2019
-
[20]
E. Ilg, O. Cicek, S. Galesso, A. Klein, O. Makansi, F. Hutter, and T. Brox. Uncertainty estimates and multi-hypotheses networks for optical flow. InProceedings of the European Conference on Computer Vision (ECCV), pp. 652–667, 2018. 7
2018
-
[21]
Jiang, B
W. Jiang, B. Lei, and K. Daniilidis. Fisherrf: Active view selection and uncertainty quantification for radiance fields using fisher information. arXiv preprint arXiv:2311.17874, 2023. 2
2023 arXiv
-
[22]
D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. 12
2014 arXiv
-
[23]
Lakshminarayanan, A
B. Lakshminarayanan, A. Pritzel, and C. Blundell. Simple and scal- able predictive uncertainty estimation using deep ensembles. Ad- vances in neural information processing systems , 30, 2017. 1, 2, 7
2017
-
[24]
Y . Lu, K. Jiang, J. A. Levine, and M. Berger. Compressive neural rep- resentations of volumetric scalar fields. InComputer Graphics F orum, vol. 40, pp. 135–146. Wiley Online Library, 2021. 2
2021
-
[25]
Mildenhall, P
B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoor- thi, and R. Ng. Nerf: Representing scenes as neural radiance fields for view synthesis. Communications of the ACM, 65(1):99–106, 2021. 2
2021
-
[26]
B. Y . Mirghani, E. M. Zechman, R. S. Ranjithan, and G. Mahinthaku- mar. Enhanced simulation-optimization approach using surrogate modeling for solving inverse problems. Environmental F orensics, 13(4):348–363, 2012. 1, 2
2012
-
[27]
Molnar, J
S. Molnar, J. Laurence-Chasen, Y . Duan, J. Bessac, and K. Potter. Un- certainty visualization challenges in decision systems with ensemble data & surrogate models. In 2024 IEEE Workshop on Uncertainty Vi- sualization: Applications, Techniques, Software, and Decision Frame- wo...
2024
-
[28]
M ¨uller, A
T. M ¨uller, A. Evans, C. Schied, and A. Keller. Instant neural graphics primitives with a multiresolution hash encoding. ACM transactions on graphics (TOG), 41(4):1–15, 2022. 3, 9
2022
-
[29]
R. M. Neal et al. Mcmc using hamiltonian dynamics. Handbook of markov chain monte carlo, 2(11):2, 2011. 4, 5, 12
2011
-
[30]
Obermaier, K
H. Obermaier, K. Bensema, and K. I. Joy. Visual trends analysis in time-varying ensembles. IEEE transactions on visualization and com- puter graphics, 22(10):2331–2342, 2015. 2
2015
-
[31]
Orban, D
D. Orban, D. F. Keefe, A. Biswas, J. Ahrens, and D. Rogers. Drag and track: A direct manipulation interface for contextualizing data instances within a continuous parameter space. IEEE transactions on visualization and computer graphics , 25(1):256–266, 2018. 2
2018
-
[32]
S. Popinet. Free computational fluid dynamics. ClusterWorld, 2(6):7,
-
[33]
Potter, A
K. Potter, A. Wilson, P.-T. Bremer, D. Williams, C. Doutriaux, V . Pas- cucci, and C. R. Johnson. Ensemble-vis: A framework for the sta- tistical visualization of ensemble data. In 2009 IEEE international conference on data mining workshops , pp. 233–240. IEEE, 2009. 5
2009
-
[34]
Saklani, C
S. Saklani, C. Goel, S. Bansal, Z. Wang, S. Dutta, T. M. Athawale, D. Pugmire, and C. R. Johnson. Uncertainty-informed volume visualization using implicit neural representation. arXiv preprint arXiv:2408.06018, 2024. 2
2024 arXiv
-
[35]
Sedlmair, C
M. Sedlmair, C. Heinzl, S. Bruckner, H. Piringer, and T. M ¨oller. Vi- sual parameter space analysis: A conceptual framework. IEEE Trans- actions on Visualization and Computer Graphics , 20(12):2161–2170,
-
[36]
J. Shen, Y . Duan, and H.-W. Shen. Surroflow: A flow-based surrogate model for parameter space exploration and uncertainty quantification. IEEE Transactions on Visualization and Computer Graphics, 2024. 1, 2
2024
-
[37]
N. Shi, J. Xu, H. Li, H. Guo, J. Woodring, and H.-W. Shen. Vdl- surrogate: A view-dependent latent-based model for parameter space exploration of ensemble simulations. IEEE Transactions on Visual- ization and Computer Graphics , 29(1):820–830, 2022. 1, 2
2022
-
[38]
N. Shi, J. Xu, S. W. Wurster, H. Guo, J. Woodring, L. P. Van Roekel, and H.-W. Shen. Gnn-surrogate: A hierarchical and adaptive graph neural network for parameter space exploration of unstructured-mesh ocean simulations. IEEE Transactions on Visualization and Computer Graphics...
2022
-
[39]
Sitzmann, J
V . Sitzmann, J. Martel, A. Bergman, D. Lindell, and G. Wetzstein. Implicit neural representations with periodic activation functions. Advances in neural information processing systems , 33:7462–7473,
-
[40]
Tancik, P
M. Tancik, P. Srinivasan, B. Mildenhall, S. Fridovich-Keil, N. Ragha- van, U. Singhal, R. Ramamoorthi, J. Barron, and R. Ng. Fourier features let networks learn high frequency functions in low dimen- sional domains. Advances in neural information processing systems , 33:7537–7...
2020
-
[41]
J. Wang, S. Hazarika, C. Li, and H.-W. Shen. Visualization and visual analysis of ensemble data: A survey. IEEE transactions on visualiza- tion and computer graphics , 25(9):2853–2872, 2018. 1, 2
2018
-
[42]
J. Wang, X. Liu, H.-W. Shen, and G. Lin. Multi-resolution climate ensemble parameter analysis with nested parallel coordinates plots. IEEE transactions on visualization and computer graphics , 23(1):81– 90, 2016. 2
2016
-
[43]
Weymouth and B
G. Weymouth and B. Font. Waterlily: A fast differentiable cpu/gpu flow simulator in julia. Bulletin of the American Physical Society ,
-
[44]
Wilkinson, A
L. Wilkinson, A. Anand, and R. Grossman. Graph-theoretic scagnos- tics. In Information visualization, IEEE symposium on , pp. 21–21. IEEE Computer Society, 2005. 9
2005
-
[45]
T. Wu, T. Maruyama, L. Wei, T. Zhang, Y . Du, G. Iaccarino, and J. Leskovec. Compositional generative inverse design. arXiv preprint arXiv:2401.13171, 2024. 1, 2
2024 arXiv
-
[46]
Xiong, S
T. Xiong, S. W. Wurster, H. Guo, T. Peterka, and H.-W. Shen. Regular- ized multi-decoder ensemble for an error-aware scene representation network. IEEE Transactions on Visualization and Computer Graph- ics, 2024. 2 (a) Circle (b) NACA airfoil (c) Rayleigh-Taylor instability Fi...
2024
Reviewed August 15, 2026 · model on record in the stance chip above.
Discussion (0). Continue with ORCID to comment.