Pith. sign in

REVIEW 2 major objections 5 minor 74 references

Expert-elicitation method for non-parametric joint priors using normalizing flows

T0 review · 2 major / 5 minor · reviewed 2026-08-12 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read Normalizing flows turn expert answers into trainable joint priors

desk verdict A clearly described, reproducible extension of simulation-based prior elicitation to normalizing flows, but the joint-prior payoff currently rests on oracle correlation input that experts cannot provide. read the letter →

arxiv 2411.15826 v2 pith:A7UPSEY2 submitted 2024-11-24 stat.ME stat.ML

Florence Bockting , Stefan T. Radev , Paul-Christian Bürkner This is my paper · ORCID
classification stat.MEstat.ML MSC 62F15
keywords priorelicitationexpertknowledgejointdistributionnon-parametricpriorsnormalizingflowssimulation-basedinferencemaximummeandiscrepancyBayesianworkflow
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper adapts an existing simulation-based prior-elicitation workflow so that it can learn non-parametric joint priors over a Bayesian model's parameters, instead of only independent parametric priors. The prior is represented by a normalizing flow, a neural generative model that maps a simple base distribution to a flexible target density with exact density evaluation. The expert supplies interpretable predictive statistics—five quantiles of prior predictions, plus $R^2$—and the flow is trained so its prior predictive statistics match those answers; pairwise correlations among parameters are also supplied. Across four simulation studies, the paper reports recovery of target marginals, including skew, and capture of dependence when correlation information is provided. The wider claim is that one modular workflow can cover parametric and non-parametric, independent and joint elicitation.

What carries the argument

The load-bearing object is a normalizing flow $g_\lambda = g_{\lambda_H} \circ \cdots \circ g_{\lambda_1}$, a composition of invertible affine coupling layers whose density is $p_\lambda(\theta) = p(u = g_\lambda(\theta))\,|\det g_\lambda'(\theta)|$. Sampling in the generative direction $u \sim p(u)$, $\theta = g_\lambda^{-1}(u)$ turns the flow into a trainable joint prior. Training couples this flow to the elicitation pipeline: forward simulation from prior and likelihood produces model-implied target quantities (predictive quantiles, $R^2$, correlations), a weighted multi-objective loss (maximum mean discrepancy, with squared error for correlations) measures discrepancy from expert answers, and stochastic gradients update the flow weights. Supporting machinery includes a sensitivity analysis for testing whether each elicited statistic responds to prior changes, an oracle simulation for assessing identifiability before real experts are consulted, and loss-weighted model averaging for handling the non-uniqueness of learned priors.

What would settle it

Run the elicitation pipeline on a known correlated prior, but give the 'expert' only the five quantiles per group and $R^2$—no correlation values. If the ensemble of learned flow priors across 30 seeds shows pairwise correlation estimates whose spread is comparable to the prior range rather than collapsing around the true matrix, then the method is not actually learning dependence from interpretable expert quantities and the joint-prior claim is unsupported.

Watch

Extended reading notes

Core claim

The paper's central claim is that the same mini-batch stochastic-gradient-descent workflow used for parametric prior elicitation can, with only minor adjustments, learn a non-parametric joint prior $p_\lambda(\theta)$ by treating the prior as a normalizing flow with trainable network weights $\lambda$. A weighted sum of loss components, chiefly maximum mean discrepancy between model-implied and expert-elicited statistics, drives the training; the flow's change-of-variables formula makes density evaluation exact, and sampling proceeds in the generative direction $\theta = g_\lambda^{-1}(u)$. The simulations lead the authors to conclude that joint priors can be learned from primarily interpretable quantities, with the explicit caveat that the correlation information the method currently requires is not yet obtainable from real experts through a developed elicitation technique.

Load-bearing premise

The method's joint-prior learning depends on someone supplying the exact pairwise correlations between model parameters, and the paper states that current elicitation techniques cannot yet obtain this information from real experts.

Editorial extensions

If this is right

  • Users can specify priors without choosing parametric families: skew and dependence are represented by the flow whenever the elicited statistics carry that information.
  • The same modular workflow extends to independent, joint, parametric, and non-parametric priors, so evaluation diagnostics and model averaging transfer across settings.
  • The oracle-based sensitivity analysis lets researchers check before consulting an expert which aspects of the prior are identifiable from the planned questions.
  • Loss-weighted model averaging across random-seed replications converts the non-uniqueness of learned priors into a final prior plus a diagnostic for degenerate runs.
  • Because the framework only needs forward simulation and a differentiable loss, alternative generative models such as diffusion or flow-matching could replace normalizing flows without changing the elicitation protocol.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • A realistic elicitation protocol that replaces the exact correlation oracle—for example, pairwise comparison or ranking of scenarios—would be the decisive test of whether the joint-prior claim holds with real experts.
  • The same pipeline could be applied to hierarchical or latent-variable models by choosing elicited statistics at the group or latent level; the sensitivity analysis would reveal whether those statistics are informative enough.
  • Seed-to-seed variation in learned marginals could serve as a practical identifiability diagnostic: large variation after convergence signals that the elicitation protocol needs more quantiles, more target quantities, or regularization.
  • A learned flow prior from one model could seed elicitation for a related model, reducing the number of expert queries needed, since the flow provides a smooth, trainable initial density.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

2 major / 5 minor

Summary. The paper extends the simulation-based prior-elicitation framework of Bockting et al. (2024) to non-parametric joint priors by representing the prior with a normalizing flow (NF). The method learns the flow weights by minimizing a weighted sum of discrepancies between model-implied and expert-elicited statistics, which in the simulations are five quantiles of prior predictive distributions, R^2, and pairwise parameter correlations. Four oracle-based simulation studies are presented: a binomial regression with independent normal priors, and three normal-regression scenarios with independent, skewed-normal, and correlated multivariate-normal priors. The authors assess convergence through loss trajectories and slope diagnostics, examine learned priors and model-averaged results, and provide a sensitivity analysis for target-quantity informativeness. The central claim is that the same modular, simulation-based framework can learn flexible non-parametric joint priors using only minor workflow adjustments.

Significance. If the central claim holds, this would be a useful methodological advance: it extends simulation-based prior elicitation beyond parametric, independent priors to flexible joint distributions while retaining interpretable, observable target quantities. The paper contributes an open-source implementation (elicito), a reproducible workflow with code and results archived, and a thoughtful set of diagnostics (convergence slopes, sensitivity analysis, model averaging, oracle-based evaluation). The emphasis on non-identifiability and the honest discussion of when learned priors are not unique are valuable for practitioners. However, the experimental support for the joint-prior aspect is weaker than the abstract suggests: the only dependence scenario supplies the exact correlation matrix as an 'elicited statistic', and the evaluation of prior recovery is entirely visual. The framework itself is sound and the paper is clearly written, but the evidence for the advertised capability needs strengthening or the claims need qualification.

major comments (2)
  1. [Section 4.1 and Section 4.3.3] The only demonstration of learning a dependent joint prior (Simulation Study 2, Scenario 3) uses the exact correlation matrix of the model parameters as an 'elicited statistic'. In Section 4.1 you state: 'we use the exact correlation structure indicated by the ground truth' and 'We acknowledge that this information cannot be reasonably demanded from a domain expert in most cases.' This means the joint-prior result is an oracle exercise: the dependence structure is provided as input, not learned from interpretable quantities that a domain expert can meaningfully provide. The Discussion (Section 5) does concede this ('One exception is the use of correlation information...'), but the Abstract and the opening claim of Section 5 ('we demonstrated successful learning of joint priors based primarily on interpretable quantities that can be meaningfully provided by domain experts') overstate the empirical support. Please reframe the abstract and conclusions to state clearly that the current evidence for learning dependence is conditional on supplying correlation targets that are not currently elicitable from real experts.
  2. [Section 4.1, Section 4.3.1, and Figures 8-9] The evaluation of whether the method has learned the prior is based entirely on visual inspection of marginal and joint plots; no quantitative measure of prior recovery is reported. This matters because the training objective directly minimizes the discrepancy between model-implied and expert-elicited statistics, so a close match of those statistics (e.g., Figure 8) is the fitted outcome rather than an independent confirmation that the prior has been identified. The paper's 'successful learning' claim rests on visual judgments, which are especially concerning in Scenario 1: Figure 9 shows substantial between-seed variation for beta1 and beta2, and the sensitivity analysis in Appendix A.2 shows these parameters are informed by only one group-specific target quantity each, leaving them weakly identified. I recommend reporting quantitative metrics of prior recovery (e.g., MMD or energy distance between learned and true joint priors, coverage of true parameter values, or a scalar summary of cross-seed variability) and explicitly stating in the abstract and discussion which model parameters are identifiable under the chosen elicited statistics.
minor comments (5)
  1. [Section 3 (Implementation)] There is a typo: 'Stimulation Study 2' should read 'Simulation Study 2' in the sentence about the fixed learning rate for Scenario 3.
  2. [Figure 14 caption] The caption is missing a period after 'replications': it currently reads 'Learned joint prior for all 30 replicationsEach replication is shown in a different color.'
  3. [Section 4.3.2] The sentence 'we except to observe more variation in the learned prior distributions' should read 'we expect to observe'.
  4. [References] The reference 'lgorzata Roos M, Martins TG, Held L, et al' appears to have a typographical error in the author name; it should likely be 'Małgorzata Roos' or 'Roos M'.
  5. [Section 2.5 (Convergence Checks)] The statement 'With a proper discrepancy measure as loss function, it is guaranteed that the total loss approaches zero as learning progresses' is not generally true for stochastic gradient descent, since minibatch noise and finite training time prevent exact convergence to zero; please rephrase to avoid the implication of a formal guarantee.

Circularity Check

0 steps flagged · score 0.0 of 10

No circularity: the paper's fitted statistics are explicitly training targets and the sole oracle-dependent joint-prior demonstration is disclosed as such; the remaining limitations affect external validity, not the derivation chain.

full rationale

The paper does not rename a fitted quantity as a prediction. Its total loss (Eq. 1) is defined as the weighted mismatch between model-implied and expert-elicited statistics, and the reported matches in Figures 6, 8, 10, and 12 are convergence checks of that same objective ('supporting the assumption of successful convergence'), not out-of-sample predictions. The correlated-prior scenario is transparent: Section 4.1 states that 'we use the exact correlation structure indicated by the ground truth' for the correlation target quantity, and Section 4.3.3 concedes that the resulting correlation recovery is 'not particularly surprising, as we provided the method with the exact correlation information.' Thus the joint-prior dependence demonstration is an oracle/input test, not a claim derived from an independent source; this is a validity limitation (further stressed by the authors' admission that 'we currently lack suitable elicitation methods' for correlation information), but not a circular step. The acknowledged underidentification of β1 and β2 in Scenario 1 ('insufficient to uniquely identify the priors for β1 and β2') likewise undermines the strength of the empirical claims without equating output to input. Self-citations to Bockting et al. (2024) provide the underlying simulation-based workflow, but the NF extension, diagnostics, and simulations are new content; the universality of affine coupling flows is cited to independent work (Draxler et al. 2024). No uniqueness theorem or ansatz is imported from the authors' prior work to force the construction. The main risks are lack of real-expert validation, oracle-only evaluation, and visually assessed prior recovery, all of which are correctness concerns rather than circularity.

Assumptions & free parameters 7 free parameters · 7 assumptions · 0 invented entities

The method's central claim rests less on mathematical derivation than on a set of modeling and elicitation choices. The most consequential are the hand-picked loss weights and flow architecture, the oracle-based evaluation protocol, and the assumption that correlation information is available as an expert input. The axioms listed above are the premises that would have to hold for the four simulation studies to generalize to real prior elicitation.

free parameters (7)
  • Loss component weights alpha_m = alpha_corr=0.1, alpha_R2=10.0, others=1.0
    Section 4.1: hand-chosen to balance scale differences between MMD and L2 losses; they directly shape the objective being optimized.
  • Learning rate = 0.00025 (SS2 scenarios 1 and 2), 0.0001 (SS1 and SS2 scenario 3)
    Section 3 states these were obtained by manual tuning until good performance across case studies was achieved.
  • NF architecture capacity = 3 affine coupling blocks, 2 dense layers with 128 units, ReLU; |lambda| in {202,776, 205,872}
    Section 3: chosen architecture determines expressiveness of the learned prior; no ablations are reported.
  • Number of training epochs = 500 (SS1), 800 (SS2)
    Section 3: fixed stopping point; adequacy is assessed via convergence slopes and visual checks.
  • Batch size and sample sizes = batch size 128, S=200 for training, S=10,000 for oracle statistics
    Section 3: fixed inputs to the SGD and MMD estimation; set without reported sensitivity analysis.
  • Elicited quantile set = 5%, 25%, 50%, 75%, 95%
    Section 4.1: chosen following quantile-based elicitation recommendations; only this set is evaluated.
  • Model averaging scaling factor gamma = 1
    Table 1: gamma=1 in all simulation studies; controls how strongly loss differences translate into prior weights.
assumptions (7)
  • standard math Change-of-variables formula for normalizing flows yields the prior density p_lambda(theta).
    Section 2.1 uses this to evaluate densities and to sample from the joint prior by inversion.
  • standard math Affine coupling flows can represent any target distribution (universal approximation).
    Section 2.1 cites Draxler et al (2024) to justify restricting to affine coupling flows.
  • domain assumption Oracle-simulated expert statistics faithfully represent real expert elicitation.
    All four studies replace real experts with statistics derived from a known ground-truth prior; no expert noise, bias, or task misunderstanding is modeled.
  • ad hoc to paper Experts can provide point estimates of correlations between model parameters.
    Section 4.1 and Discussion state that correlation information cannot currently be reasonably elicited from domain experts, yet Scenario 3 relies on it as an input.
  • domain assumption The selected target quantities and five quantiles are sufficiently informative for the prior aspects of interest.
    Sensitivity analyses (Figures 2, A1, A6, A11) support informativeness only under the ground-truth prior; for beta1 and beta2 the paper shows the statistics are insufficient to identify the prior (Section 4.3.1).
  • domain assumption MMD with energy kernel plus L2 loss for correlations is an appropriate discrepancy measure.
    Section 3 adopts MMD from Bockting et al (2024) and an L2 loss for correlations; no comparison with alternative losses is reported.
  • standard math The generative model and likelihood are correctly specified in the simulation studies.
    The method's consistency checks assume the model used to simulate expert stats equals the model used to train the prior.

how reviews work

0 comments
Cite this review

Pith. "Pith review of Expert-elicitation method for non-parametric joint priors using normalizing flows." pith.science (2026). https://pith.science/paper/A7UPSEY2

@misc{pith2026241115826,
  author       = {Pith},
  title        = {Pith review of: Expert-elicitation method for non-parametric joint priors using normalizing flows},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/A7UPSEY2}},
  note         = {Machine review of arXiv:2411.15826}
}
read the original abstract

We propose an expert-elicitation method for learning non-parametric joint prior distributions using normalizing flows. Normalizing flows are a class of generative models that enable exact, single-step density evaluation and can capture complex density functions through specialized deep neural networks. Building on our previously introduced simulation-based framework, we adapt and extend the methodology to accommodate non-parametric joint priors. Our framework thus supports the development of elicitation methods for learning both parametric and non-parametric priors, as well as independent or joint priors for model parameters. To evaluate the performance of the proposed method, we perform four simulation studies and present an evaluation pipeline that incorporates diagnostics and additional evaluation tools to support decision-making at each stage of the elicitation process.

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

74 extracted references · 38 canonical work pages

  1. [1]

    In: 12th USENIX symposium on operating systems design and implementation (OSDI 16), pp 265--283, doi:10.5555/3026877.3026899

    Abadi M, Barham P, Chen J, et al (2016) TensorFlow : A system for large-scale machine learning. In: 12th USENIX symposium on operating systems design and implementation (OSDI 16), pp 265--283, doi:10.5555/3026877.3026899

    arXiv 2016

  2. [2]

    Authority EFS (2014) Guidance on expert knowledge elicitation in food and feed safety risk assessment. EFSA J. 12(6):3734. doi:10.2903/j.efsa.2014.3734

    arXiv 2014

  3. [3]

    Blog Post, ://betanalpha.github.io/assets/case_studies/principled_bayesian_workflow.html

    Betancourt M (2020) Towards a principled bayesian workflow. Blog Post, ://betanalpha.github.io/assets/case_studies/principled_bayesian_workflow.html

    work page 2020

  4. [4]

    arXiv preprint doi:10.48550/arXiv.2110.09813

    Bischof R, Kraus M (2021) Multi-objective loss balancing for physics-informed deep learning. arXiv preprint doi:10.48550/arXiv.2110.09813

    work page doi:10.48550/arxiv.2110.09813 2021

  5. [5]

    Bissiri PG, Holmes CC, Walker SG (2016) A general framework for updating belief distributions. J. R. Stat. Soc. Series. B Stat. Methodol. 78(5):1103--1130. doi:10.1111/rssb.12158

    doi:10.1111/rssb.12158 2016

  6. [6]

    Bockting F, Radev ST, B \"u rkner PC (2024) Simulation-based prior knowledge elicitation for parametric bayesian models. Sci. Rep. 14(1):17330. doi:10.1038/s41598-024-68090-7

    work page doi:10.1038/s41598-024-68090-7 2024

  7. [7]

    IEEE Trans

    Bond-Taylor S, Leach A, Long Y, et al (2021) Deep generative modelling: A comparative review of vaes, gans, normalizing flows, energy-based and autoregressive models. IEEE Trans. Pattern Anal. Mach. Intell. 44(11):7327--7347. doi:10.1109/TPAMI.2021.3116668

    arXiv 2021

  8. [8]

    Biometrics pp 603--618

    Buckland ST, Burnham KP, Augustin NH (1997) Model selection: an integral part of inference. Biometrics pp 603--618. doi:10.2307/2533961

    work page doi:10.2307/2533961 1997

Show all 74 references
  1. [9]

    IEEE Trans

    Cao H, Tan C, Gao Z, et al (2024) A survey on generative diffusion models. IEEE Trans. Knowl. Data Eng. 36(7):2814--2830. doi:10.1109/TKDE.2024.3361474

    2024

  2. [10]

    Cambridge books

    Claeskens G, Hjort NL (2008) Model selection and model averaging. Cambridge books

    2008

  3. [11]

    Clemen RT, Fischer GW, Winkler RL (2000) Assessing dependence: Some experimental results. Manage. Sci. 46(8):1100--1115. doi:10.1287/mnsc.46.8.1100.12023

    2000

  4. [12]

    Oxford University Press, USA

    Cooke RM (1991) Experts in uncertainty: opinion and subjective probability in science. Oxford University Press, USA

    1991

  5. [13]

    arXiv preprint doi:10.48550/arXiv.2009.09796

    Crawshaw M (2020) Multi-task learning with deep neural networks: A survey. arXiv preprint doi:10.48550/arXiv.2009.09796

    2020 doi arXiv

  6. [14]

    Crowder M (1992) Bayesian priors based on a parameter transformation using the distribution function. Ann. Inst. Stat. Math. 44:405--416. doi:10.1007/BF00050695

    1992 doi

  7. [15]

    Bayesian Anal

    Denham R, Mengersen K (2007) Geographically assisted elicitation of expert opinion for regression models. Bayesian Anal. 2(1):99--136. doi:10.1214/07-BA205

    2007 doi

  8. [16]

    Depaoli S, Winter SD, Visser M (2020) The importance of prior sensitivity analysis in bayesian statistics: demonstrations using an interactive shiny app. Front. Psychol. 11:608045. doi:10.3389/fpsyg.2020.608045

    2020

  9. [17]

    Dickey J, Lindley DV, Press SJ (1985) Bayesian estimation of the dispersion matrix of a multivariate normal distribution. Commun. Stat. Theory Methods 14(5):1019--1034. doi:10.1080/03610928508828960

    1985 doi

  10. [18]

    arXiv preprint doi:10.48550/arXiv.1711.10604

    Dillon JV, Langmore I, Tran D, et al (2017) Tensorflow distributions. arXiv preprint doi:10.48550/arXiv.1711.10604

    2017 doi arXiv

  11. [19]

    In: ICLR Workshop, doi:10.48550/arXiv.1410.8516

    Dinh L, Krueger D, Bengio Y (2014) Nice: Non-linear independent components estimation. In: ICLR Workshop, doi:10.48550/arXiv.1410.8516

    2014 doi arXiv

  12. [20]

    arXiv preprint doi:10.48550/arXiv.1605.08803

    Dinh L, Sohl-Dickstein J, Bengio S (2016) Density estimation using real nvp. arXiv preprint doi:10.48550/arXiv.1605.08803

    2016 doi arXiv

  13. [21]

    arXiv preprint doi:10.48550/arXiv.2402.06578

    Draxler F, Wahl S, Schn \"o rr C, et al (2024) On the universality of coupling-based normalizing flows. arXiv preprint doi:10.48550/arXiv.2402.06578

    2024 doi arXiv

  14. [22]

    Durkan C, Bekasov A, Murray I, et al (2019) Neural spline flows. Adv. Neural. Inf. Process. Syst. 32. doi:10.5555/3454287.3454962

    2019

  15. [23]

    Falconer JR, Frank E, Polaschek DL, et al (2022) Methods for eliciting informative prior distributions: A critical review. Decis. Anal. 19(3):189--204. doi:10.1287/deca.2022.0451

    2022

  16. [24]

    Fern \'a ndez C, Steel MF (1998) On bayesian modeling of fat tails and skewness. J. Am. Stat. Assoc. 93(441):359--371. doi:10.2307/2669632

    1998 doi

  17. [25]

    In: The 22nd International Conference on Artificial Intelligence and Statistics, PMLR, pp 2681--2690, ://proceedings.mlr.press/v89/feydy19a.html

    Feydy J, S \'e journ \'e T, Vialard FX, et al (2019) Interpolating between optimal transport and mmd using sinkhorn divergences. In: The 22nd International Conference on Artificial Intelligence and Statistics, PMLR, pp 2681--2690, ://proceedings.mlr.press/v89/feydy19a.html

    2019

  18. [26]

    Figurnov M, Mohamed S, Mnih A (2018) Implicit reparameterization gradients. Adv. Neural. Inf. Process. Syst. 31. doi:10.5555/3326943.3326984

    2018

  19. [27]

    Gabry J, Simpson D, Vehtari A, et al (2019) Visualization in bayesian workflow. J. R. Stat. Soc. Ser. A Stat. Soc. 182(2):389--402. doi:10.1111/rssa.12378

    2019 doi

  20. [28]

    Garthwaite PH, Kadane JB, O'Hagan A (2005) Statistical methods for eliciting probability distributions. J. Am. Stat. Assoc. 100(470):680--701. doi:10.1198/016214505000000105

    2005 doi

  21. [29]

    Garthwaite PH, Al-Awadhi SA, Elfadaly FG, et al (2013) Prior distribution elicitation for generalized linear and piecewise-linear models. J. Appl. Stat. 40(1):59--75. doi:10.1080/02664763.2012.734794

    2013

  22. [30]

    Entropy 19(10):555

    Gelman A, Simpson D, Betancourt M (2017) The prior can often only be understood in the context of the likelihood. Entropy 19(10):555. doi:10.3390/e19100555

    2017 doi

  23. [32]

    arXiv preprint doi:10.48550/arXiv.2011.01808

    Gelman A, Vehtari A, Simpson D, et al (2020) Bayesian workflow. arXiv preprint doi:10.48550/arXiv.2011.01808

    2020 doi arXiv

  24. [33]

    Gokhale D, Press SJ (1982) Assessment of a prior distribution for the correlation coefficient in a bivariate normal distribution. J. R. Stat. Soc. Ser. A Stat. Soc. 145(2):237--249. doi:10.2307/2981537

    1982 doi

  25. [34]

    MIT press

    Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT press

    2016

  26. [35]

    Elicitation: The science and art of structuring judgement pp 61--93

    Gosling JP (2018) Shelf: the sheffield elicitation framework. Elicitation: The science and art of structuring judgement pp 61--93

    2018

  27. [36]

    Gretton A, Borgwardt K, Rasch M, et al (2006) A kernel method for the two-sample-problem. Adv. Neural. Inf. Process. Syst. 19. doi:10.5555/2976456.2976521

    2006

  28. [37]

    Nature 585(7825):357--362

    Harris CR, Millman KJ, van der Walt SJ, et al (2020) Array programming with NumPy . Nature 585(7825):357--362. doi:10.1038/s41586-020-2649-2

    2020 doi

  29. [38]

    In: Peters J, Sontag D (eds) Proc.Conf.UAI , PMLR, pp 1129--1138, ://proceedings.mlr.press/v124/hartmann20a.html

    Hartmann M, Agiashvili G, B \"u rkner P, et al (2020) Flexible prior elicitation via the prior predictive distribution . In: Peters J, Sontag D (eds) Proc.Conf.UAI , PMLR, pp 1129--1138, ://proceedings.mlr.press/v124/hartmann20a.html

    2020

  30. [39]

    Johnson SR, Tomlinson GA, Hawker GA, et al (2010) Methods to elicit beliefs for Bayesian priors: a systematic review . J. Clin. Epidemiol. 63(4):355--369. doi:10.1016/j.jclinepi.2009.06.003

    2010 doi

  31. [40]

    Statistician 47(1):3--19

    Kadane J, Wolfson LJ (1998) Experiences in elicitation. Statistician 47(1):3--19. doi:10.1111/1467-9884.00113

    1998

  32. [41]

    Kadane JB, Dickey JM, Winkler RL, et al (1980) Interactive elicitation of opinion for a normal linear model. J. Am. Stat. Assoc. 75(372):845--854. doi:10.1080/01621459.1980.10477562

    1980

  33. [42]

    Advances in neural information processing systems 31

    Kingma DP, Dhariwal P (2018) Glow: Generative flow with invertible 1x1 convolutions. Advances in neural information processing systems 31

    2018

  34. [43]

    stat 1050:1

    Kingma DP, Welling M (2014) Auto-encoding variational bayes. stat 1050:1. doi:10.48550/arXiv.1312.6114

    2014 doi arXiv

  35. [44]

    IEEE Trans

    Kobyzev I, Prince SJ, Brubaker MA (2020) Normalizing flows: An introduction and review of current methods. IEEE Trans. Pattern Anal. Mach. Intell. 43(11):3964--3979. doi:10.1109/TPAMI.2020.2992934

    2020

  36. [45]

    arXiv preprint doi:10.48550/arXiv.2210.02747

    Lipman Y, Chen RT, Ben-Hamu H, et al (2022) Flow matching for generative modeling. arXiv preprint doi:10.48550/arXiv.2210.02747

    2022 doi arXiv

  37. [46]

    In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1871--1880, doi:10.1109/CVPR.2019.00197

    Liu S, Johns E, Davison AJ (2019) End-to-end multi-task learning with attention. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 1871--1880, doi:10.1109/CVPR.2019.00197

    2019

  38. [47]

    arXiv preprint doi:10.48550/arXiv.2303.08528

    Manderson AA, Goudie RJ (2023) Translating predictive distributions into informative priors. arXiv preprint doi:10.48550/arXiv.2303.08528

    2023 doi

  39. [48]

    Martin TG, Burgman MA, Fidler F, et al (2012) Eliciting expert knowledge in conservation science. Conserv. Biol. 26(1):29--38. doi:10.1111/j.1523-1739.2011.01806.x

    2012

  40. [49]

    In: Proceedings of the 9th Python in Science Conference, Austin, TX, pp 51--56, doi:10.25080/Majora-92bf1922-00a

    McKinney W, et al (2010) Data structures for statistical computing in python. In: Proceedings of the 9th Python in Science Conference, Austin, TX, pp 51--56, doi:10.25080/Majora-92bf1922-00a

    2010 doi

  41. [50]

    Bayesian Anal

    Mikkola P, Martin OA, Chandramouli S, et al (2023) Prior knowledge elicitation: The past, present, and future . Bayesian Anal. 1(1):1--33. doi:10.1214/23-BA1381

    2023 doi

  42. [51]

    arXiv preprint doi:10.48550/arXiv.2410.08710

    Mikkola P, Acerbi L, Klami A (2024) Preferential normalizing flows. arXiv preprint doi:10.48550/arXiv.2410.08710

    2024 doi arXiv

  43. [52]

    Morris DE, Oakley JE, Crowe JA (2014) A web-based tool for eliciting probability distributions from experts. Environ. Model. Softw. 52:1--4. doi:10.1016/j.envsoft.2013.10.010

    2014 doi

  44. [53]

    Nemani V, Biggio L, Huan X, et al (2023) Uncertainty quantification in machine learning for engineering design and health prognostics: A tutorial. Mech. Syst. Signal Process. 205:110796. doi:10.1016/j.ymssp.2023.110796

    2023

  45. [54]

    Biometrika 94(2):427--441

    Oakley JE, O'Hagan A (2007) Uncertainty in prior elicitations: a nonparametric approach. Biometrika 94(2):427--441. doi:10.1093/biomet/asm031

    2007 doi

  46. [55]

    O'Hagan A, Oakley JE (2004) Probability is perfect, but we can't elicit it perfectly. Reliab. Eng. Syst. Saf. 85(1-3):239--248. doi:10.1016/j.ress.2004.03.014

    2004 doi

  47. [56]

    John Wiley & Sons

    O'Hagan A, Buck CE, Daneshkhah A, et al (2006) Uncertain judgements: eliciting experts' probabilities. John Wiley & Sons

    2006

  48. [57]

    O’Hagan A (2019) Expert knowledge elicitation: subjective but scientific. Am. Stat. 73(sup1):69--81. doi:10.1080/00031305.2018.1518265

    2019 arXiv

  49. [58]

    Papamakarios G, Nalisnick E, Rezende DJ, et al (2021) Normalizing flows for probabilistic modeling and inference. J. Mach. Learn. 22(57):1--64. ://jmlr.org/papers/v22/19-1028.html

    2021

  50. [59]

    OSF preprint doi:10.31219/osf.io/tq3an

    Perepolkin D, Lindsr \"o m E, Sahlin U (2023) Quantile-parameterized distributions for expert knowledge elicitation. OSF preprint doi:10.31219/osf.io/tq3an

    2023 doi

  51. [60]

    Perepolkin D, Goodrich B, Sahlin U (2024) Hybrid elicitation and quantile-parametrized likelihood. Stat. Comput. 34(1):11. doi:10.1007/s11222-023-10325-0

    2024 doi

  52. [61]

    Peterson C, Miller A (1964) Mode, median, and mean as optimal strategies. J. Exp. Psychol. 68(4):363. doi:10.1037/h0040387

    1964 doi

  53. [62]

    IEEE Trans

    Radev ST, Mertens UK, Voss A, et al (2020) Bayesflow: Learning complex stochastic models with invertible neural networks. IEEE Trans. Neural. Netw. Learn. Syst. 33(4):1452--1466. doi:10.1109/TNNLS.2020.3042395

    2020

  54. [63]

    Radev ST, Schmitt M, Schumacher L, et al (2023) Bayesflow: Amortized bayesian workflows with neural networks. J. Open Source Softw. 8(89):5702. doi:https://doi.org/10.21105/joss.05702

    2023 doi

  55. [64]

    Regenwetter M, Cavagnaro DR (2019) Tutorial on removing the shackles of regression analysis: How to stay true to your theory of binary response probabilities. Psychol. methods 24(2):135. doi:10.1037/met0000196

    2019 doi

  56. [65]

    Bayesian Anal

    lgorzata Roos M, Martins TG, Held L, et al (2015) Sensitivity analysis for bayesian hierarchical models. Bayesian Anal. 10(2):321--349. doi:10.1214/14-BA909

    2015 doi

  57. [66]

    Scott JG (2017) Prior specification is engineering, not mathematics. Stat. Sci. 32(1):29--32. doi:1214/16-STS576

    2017

  58. [67]

    Preprint at https://doi.org/10.48550/arXiv.1910.12263

    da Silva EdS, Ku\' s mierczyk T, Hartmann M, et al (2019) Prior specification via prior predictive matching: Poisson matrix factorization and beyond . Preprint at https://doi.org/10.48550/arXiv.1910.12263

    2019 doi arXiv

  59. [68]

    Simpson D, Rue H, Riebler A, et al (2017) Penalising model component complexity: A principled, practical approach to constructing priors. Stat. Sci. 32(1):1--28. doi:10.1214/16-STS576

    2017 doi

  60. [69]

    Stefan AM, Evans NJ, Wagenmakers EJ (2022) Practical challenges and methodological flexibility in prior elicitation. Psychol. Methods 27(2):177--197. doi:10.1037/met0000354

    2022 doi

  61. [70]

    Van Dongen S (2006) Prior specification in bayesian statistics: three cautionary tales. J. Theor. Biol. 242(1):90--100. doi:10.1016/j.jtbi.2006.02.002

    2006 doi

  62. [71]

    Wagenmakers EJ, Farrell S (2004) AIC model selection using akaike weights. Psychon. Bull. Rev. 11:192--196. doi:10.3758/BF03206482

    2004 doi

  63. [72]

    Springer

    Wang L, Ng AH, Deb K (2011) Multi-objective evolutionary optimisation for product design and manufacturing. Springer

    2011

  64. [73]

    Winkler RL (1967) The assessment of prior distributions in bayesian analysis. J. Am. Stat. Assoc. 62(319):776--800. doi:10.1080/01621459.1967.10500894

    1967

  65. [74]

    , " * write output.state after.block = add.period write newline

    ENTRY address archive author booktitle chapter doi edition editor eid eprint howpublished institution journal key keywords month note number organization pages publisher school series title type url volume year archivePrefix primaryClass adsurl adsnote version label extra.labe...

  66. [75]

    write newline

    " write newline "" before.all 'output.state := FUNCTION add.period duplicate empty 'skip "." * add.blank if FUNCTION if.digit duplicate "0" = swap duplicate "1" = swap duplicate "2" = swap duplicate "3" = swap duplicate "4" = swap duplicate "5" = swap duplicate "6" = swap dupl...

Pith tools

Reviewed August 12, 2026 · model on record in the stance chip above.