{"id":"28ae945c-4be6-4d4f-af85-e45b32c423a1","arxiv_id":"2505.18077","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"A Bayesian deep learning architecture for discrete choice yields calibrated credible intervals for marginal rates of substitution and collapses to linear utility models when data are limited.","lead":"This paper proposes a deep learning model for discrete choice that combines a standard linear utility component with two regularized nonlinear blocks, and trains it with Stochastic Gradient Langevin Dynamics to obtain posterior intervals for economic quantities.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Posterior is undefined: Eq. (11) treats sigma_IIA/sigma_nonIIA as regularized parameters while Sec. 5.2 fixes them as hyperparameters; Table 3 coverage cannot be interpreted as Bayesian until this is resolved.","rationale":"The reader correctly flags the missing SGLD convergence diagnostics, but the more specific, load-bearing gap is that the paper never fixes the target distribution: sigma_IIA/sigma_nonIIA are simultaneously regularized parameters (Eq. 11, Section 5) and fixed hyperparameters (Section 5.2). Because Eq. (14) for the marginal-utility effect explicitly depends on these scales, the posterior whose quantiles produce the MRS intervals in Table 3 is ambiguous. This is an internal consistency issue, not merely a missing diagnostic, and it directly affects the central coverage claim. The concern is addressable: rerun the simulation with sigma either sampled or provably fixed, and state which is the case. The behavioral VOTT results and out-of-sample accuracy comparisons are useful evidence but do not resolve the interval-validity question. Therefore the appropriate verdict is unchanged from the reader's CONDITIONAL.","tokens_in":18114,"tokens_out":4982,"duration_ms":40240,"concrete_test":"Reproduce the Monte Carlo study with two well-specified versions: (A) sigma_IIA/sigma_nonIIA fixed at Step-1 point estimates (as Section 5.2 suggests) and (B) sigma_IIA/sigma_nonIIA included as sampled parameters with Gaussian priors N(0, lambda^-1) as Eq. (11) implies, using identical SGLD schedule and warm start. Compare the empirical coverage of the average-MRS credible intervals in Table 3. If version B yields materially different coverage (e.g., below 90% or above 98%) or materially wider intervals than reported, the ambiguity is load-bearing and the current Table 3 does not establish the claim as written; if the two versions match, the fixed-sigma treatment is a benign simplification and the paper should say so explicitly.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central claim—empirical coverage consistent with the 95% credibility level (Table 3)—rests on the claim that SGLD draws from the posterior p(theta|D). But the paper never specifies the posterior it is sampling. Section 5 (around Eq. 11) defines the training objective as a negative log-posterior, where the L2 penalties on sigma_IIA and sigma_nonIIA are interpreted as Gaussian priors on those scale parameters, and beta/gamma get 'infinite variance' (improper uniform) priors. This makes sigma random quantities whose posterior should be explored. Section 5.2, however, defines theta = (beta, Theta_f, Theta_g), explicitly excludes sigma_IIA and sigma_nonIIA from 'all model parameters', and states they are 'fixed because they are treated as hyperparameters during learning' (Eq. 14 note). The two-step procedure ('freeze nonlinear layers', then 'unfreeze' and run SGLD) does not say whether sigma are re-estimated or resampled in step 2, nor whether they are fixed at values from step 1. If sigma are fixed, the 'prior' in Eq. (11) is not a prior on sampled parameters, the Bayesian interpretation is misleading, and the credible intervals for MRS (which depend on sigma through Eq. 14) are conditional on point-estimated scale parameters, ignoring their uncertainty. If sigma are sampled, Eq. (14) and the pushforward in Eq. (15) omit part of the parameter vector, and the reported intervals are constructed from a posterior that integrates out the wrong variables. Either way, the coverage numbers in Table 3 cannot be interpreted as Bayesian credible intervals without additional clarification. This is not a question of convergence diagnostics; it is a question of what distribution SGLD is even targeting.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a deep learning architecture for discrete choice that combines a knowledge-informed linear utility component, an IIA nonlinear block, and a non-IIA nonlinear block, with scale parameters sigma_IIA and sigma_nonIIA controlling the contribution of the nonlinearities. Training uses a two-step procedure: first optimize the log-posterior with the nonlinear layers frozen, then run Stochastic Gradient Langevin Dynamics after unfreezing. The authors evaluate the method in a Monte Carlo study with simulated data, reporting empirical coverage of credible intervals for marginal rates of substitution (Table 3: 94-99% for their model at N=1000 and N=10000), and present two case studies in which their model yields behaviorally plausible values of travel time savings. The central claim is that this architecture, combined with SGLD, provides valid approximate Bayesian inference for economic quantities while retaining competitive predictive accuracy.","tokens_in":18529,"tokens_out":5748,"duration_ms":55885,"significance":"If the central claim holds, the paper addresses a genuine gap: deep learning models for discrete choice have so far offered point predictions but not a demonstrated route to interval estimation for quantities like marginal rates of substitution. The architecture's built-in collapse to a linear, behaviorally informed model when nonlinearities are unsupported is a useful inductive bias, and the simulation study against a fully connected neural network and a conditional logit model is informative. The empirical coverage results in Table 3 are a strong piece of evidence, and the case studies illustrate that the method can produce economically plausible estimates in real applications. However, the significance is currently conditional on resolving a load-bearing ambiguity in the definition of the posterior and on providing evidence that the SGLD sampler actually targets that posterior.","major_comments":[{"comment":"The target posterior is not defined consistently. Equation (11) explicitly interprets the L2 penalties on sigma_IIA and sigma_nonIIA as Gaussian priors, which makes these scale parameters random quantities, while Equation (14) defines theta as (beta, Theta_f, Theta_g), excludes sigma_IIA and sigma_nonIIA, and states that they 'are fixed because they are treated as hyperparameters during learning.' If the sigmas are fixed at estimates from the first step, then the objective in Eq. (11) is a penalized likelihood conditional on those estimates, not a negative log-posterior over the sampled parameters, and the intervals in Eqs. (17)-(19) are conditional on point estimates of sigma, ignoring their uncertainty. If the sigmas are instead sampled, then Eqs. (14)-(15) omit part of the parameter vector and the pushforward is computed from the wrong posterior. The paper must specify which case holds and adjust the Bayesian interpretation and coverage claims accordingly, otherwise Table 3 cannot be read as evidence of Bayesian credible intervals.","section":"Section 5, Eqs. (11) and (14)"},{"comment":"The paper provides no convergence diagnostics or implementation details for the SGLD sampler, and this is load-bearing because the coverage results in Table 3 are the main evidence for the method. The only information given is 'SGLD with 5,000 epochs'; there is no step-size schedule, no burn-in or thinning description, no number of retained samples, and no trace plots or other diagnostics demonstrating that the iterates approximate draws from p(theta|D). In addition, the architecture uses batch-normalization layers with batch statistics, and it is not explained how the batch-dependent normalization affects either the gradient in the SGLD update of Eq. (8) or the derivative computation m_ijk(theta) in Eq. (14). The authors should report the exact SGLD configuration, provide diagnostics or sensitivity checks, and clarify whether batch or running statistics are used when evaluating m_ijk, or the coverage numbers cannot be attributed to correct posterior sampling.","section":"Section 5.2 and Table 3"},{"comment":"The simulation study is underreported to the point of not being reproducible. The data-generating parameters beta, gamma, and Phi are not given; the statement that Phi was selected to give 'around 70% weighted out-of-sample accuracy' is not a substitute for reporting the parameter values. The authors also do not define precisely how the 'average marginal rate of substitution' intervals are constructed: for each simulated dataset, is the average taken over individuals within each posterior draw before computing quantiles, or are individual-level posterior means averaged? The SGLD hyperparameters needed to reproduce Table 3 (step-size schedule, batch size, burn-in, thinning, number of retained samples) are absent. These details should be added so that the empirical coverage results can be independently verified.","section":"Sections 6 and 8.1"}],"minor_comments":[{"comment":"The marginal rate of substitution is conventionally defined with a negative sign (dx_k/dx_l = - (du/dx_l)/(du/dx_k)); the sign convention used here should be stated explicitly, especially since the case studies report positive values of travel time savings.","section":"Eq. (13)"},{"comment":"The abstract mentions 'out-of-sample balanced accuracy,' but the results report 'weighted out-of-sample accuracy'; these terms should be reconciled and the metric defined precisely.","section":"Abstract and Tables 4-6"},{"comment":"The 'SOTA' rows in Tables 5 and 6 are not tied to a specific reference in the text; the footnote markers appear to be missing, so the reader cannot tell which published model the comparison is against.","section":"Tables 5 and 6"},{"comment":"The statement that the two-step procedure 'ensures' the model defaults to the behaviorally informed hypothesis is stronger than what is demonstrated; it would be more accurate to say that the procedure is designed to do so and that the simulations support this behavior.","section":"Section 5.1"},{"comment":"The notation in Eq. (6) conditions on y on both sides of the equation, which appears to be a typo; the predictive density should condition on the observed training data and covariates only.","section":"Eq. (6)"},{"comment":"The VOTT histograms are posterior-mean summaries and are presented without credible intervals; adding interval summaries would make the inference in the case studies more complete and better aligned with the paper's emphasis on uncertainty quantification.","section":"Figures 14-17"}],"recommendation":"major_revision","confidential_remarks":"The paper is likely to be of interest to the discrete choice and Bayesian deep learning communities, and the empirical coverage results are promising. The main risk is that the ambiguity about sigma_IIA and sigma_nonIIA, together with the absence of SGLD convergence diagnostics, undermines the Bayesian interpretation of the headline numbers. I would not reject on novelty grounds, but I would want the authors to either reformulate the method as conditional on point-estimated scale parameters or provide a fully Bayesian treatment, and to add enough experimental detail for the simulation study to be reproduced."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague,\n\nThe paper is worth a look if you work on uncertainty quantification in discrete choice. It proposes a deep choice architecture with a linear utility term plus two regularized nonlinear blocks (IIA and non-IIA), trained with SGLD in a two-step procedure. The key empirical claim—that the model gives credible intervals for marginal rates of substitution with coverage near 95% in simulations, and plausible VOTT estimates in two case studies—is genuinely new for this literature. Prior deep choice work was mostly point estimation; this is the first attempt I know of to produce Bayesian intervals for economic quantities from a deep model.\n\nWhat the paper does well: the architecture is thoughtfully designed, with scale parameters on the nonlinear blocks that can shrink to a linear model under strong regularization; the simulation study compares against a fully connected NN and a linear logit, and the coverage differences are stark. The case studies show their model gives behaviorally plausible values of travel time while a fully connected BNN gives nonsense. That is a practically useful result.\n\nThe soft spot is load-bearing. The paper is not clear about the status of σ_IIA and σ_nonIIA. Equation (11) states the loss is a negative log-posterior with Gaussian priors on those scales. But Section 5.2 defines the parameter vector θ as excluding them and says they are fixed hyperparameters. If they are fixed, the prior in (11) does not act on sampled parameters, and the credible intervals for MRS are conditional on point estimates of the scales, ignoring their uncertainty. If they are sampled, then (14)–(15) omit part of the parameter vector. Either way, Table 3's coverage numbers are not straightforwardly Bayesian credible intervals without extra assumptions. The paper needs to resolve this—either treat the scales as parameters with priors and sample them, or present the method as empirical Bayes and discuss what the coverage guarantees mean under that interpretation.\n\nA second concern, less fundamental but still present: the SGLD implementation has no convergence diagnostics or step-size schedule, and the simulation details (Φ values, regularization constants, network sizes beyond 512/2) are incomplete. No code is released. These are reproducibility gaps, not fatal flaws.\n\nOverall, the core idea is sound and the empirical pattern is suggestive. The posterior ambiguity is fixable with a clearer formulation and perhaps a small simulation showing whether sampling the scales changes coverage. I'd send it to review, but I'd ask for a major revision clarifying the target distribution and backing it with more complete simulation details.\n\nRegards.","headline":"Useful architecture and promising coverage results, but the posterior is not properly defined — fixed vs sampled scale parameters needs clarification before the Bayesian claims hold.","tokens_in":19027,"tokens_out":4213,"would_cite":false,"duration_ms":32551,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["62F15","62M45"],"pacs":[],"model":"deepseek-v4-flash","headline":"A deep learning architecture for discrete choice that keeps 95% credible intervals honest while matching predictive baselines and producing behaviorally plausible value-of-travel-time estimates.","keywords":["Bayesian deep learning","discrete choice","stochastic gradient Langevin dynamics","Markov chain Monte Carlo","marginal rates of substitution","value of travel time","credible intervals","batch normalization"],"falsifier":"Run the Monte Carlo study with a different SGLD step-size schedule or chain length and compute trace-based convergence diagnostics; if empirical coverage of the marginal-rate-of-substitution intervals drops materially, the reported 94-99% coverage is an artifact of the specific sampler settings. A sharper version: on one simulated dataset, replace SGLD with a well-mixing Hamiltonian Monte Carlo sampler and compare the resulting credible intervals for the four MRS values; if the intervals differ substantially, the SGLD chain is not sampling the intended posterior.","tokens_in":17927,"feed_emoji":"🚆","tokens_out":6314,"duration_ms":61689,"temperature":0.7,"pith_summary":"Discrete choice analysis cares about inference on economic quantities such as marginal rates of substitution and willingness to pay, not just prediction, and deep learning models have been kept out because they lack interpretability, stable estimates, and interval methods. This paper proposes a deep learning architecture built for approximate Bayesian inference: a knowledge-informed linear utility term plus separately scaled nonlinear IIA and non-IIA blocks, trained in two steps and then sampled with Stochastic Gradient Langevin Dynamics. The central claim is that this combination delivers credible intervals for marginal rates of substitution whose empirical coverage matches the stated 95% level even with only 1,000 observations, while a plain fully connected network undercovers at that sample size. In two case studies, the model yields positive, behaviorally plausible values of travel time savings and matches or beats predictive baselines.","feed_headline":"Neural choice model hits 95% coverage on economic estimates","feed_subtitle":"A two-step Bayesian architecture yields value-of-travel-time intervals that match their stated credibility level.","key_machinery":"The load-bearing object is the representative-utility specification $v_{ij} = \\alpha_j + x_{ij}^\\top\\beta + q_{ij}^\\top\\gamma + \\sigma_{\\mathrm{IIA}}\\,\\mathrm{BatchNorm}[f_{\\Theta_f}(x_{ij}, q_{ij})] + \\sigma_{\\mathrm{nonIIA}}\\,\\mathrm{BatchNorm}[g_{\\Theta_g}(x_i, q_i)_j]$, where the BatchNorm layers are non-affine (no learnable shift or scale) and $\\sigma_{\\mathrm{IIA}}$, $\\sigma_{\\mathrm{nonIIA}}$ are scale hyperparameters penalized by $\\ell^2$ terms. These scales control how much the nonlinear components can contribute relative to the linear knowledge-informed part, so the model collapses to a standard linear discrete choice model when the data do not support nonlinear structure. The two-step training procedure first optimizes the log-posterior with the nonlinear layers frozen, then unfreezes them and runs SGLD to sample from the posterior; this steers the sampler toward modes where simple behavioral hypotheses have high posterior mass.","core_discovery":"The authors claim that a deep learning discrete choice model can be made safe for economic inference by designing the architecture around a knowledge-informed component and by controlling the nonlinear blocks with non-affine batch normalization and scale hyperparameters. With their two-step procedure, fit the linear part with nonlinearities frozen and then run SGLD, the posterior credible intervals for marginal rates of substitution achieve empirical coverage of 94-99% in a 100-dataset Monte Carlo study at both N=1000 and N=10000 (Table 3), versus 79-84% for a fully connected neural network at N=1000 and 24-54% for a conditional logit model on the nonlinear MRS. The same setup produces value-of-travel-time estimates that are positive and behaviorally plausible in NYC and Swiss train data, with out-of-sample accuracy competitive with or better than baselines.","pith_inferences":["Editorial inference: a testable extension is to monitor the posterior mass on $\\sigma_{\\mathrm{IIA}}$ and $\\sigma_{\\mathrm{nonIIA}}$; if the collapse-to-linear mechanism works, these scales should shrink toward zero when data are scarce and grow when nonlinear structure is present.","Editorial inference: the two-step warm-start is doing substantial work, so an SGLD run from a random initialization might explore behaviorally implausible modes; a counterfactual study with random initialization would isolate how much of the coverage and VOTT behavior comes from the initialization rather than the sampler.","Editorial inference: the architecture's scaling mechanism is generic and could be combined with other approximate inference methods such as SWAG, deep ensembles, or variational inference, and with other economic quantities such as elasticities and willingness-to-pay distributions, provided the same two-step stabilization is used.","Editorial inference: because no convergence diagnostics are reported, the coverage numbers are the only evidence that the sampler is targeting the intended posterior; applying standard MCMC diagnostics would either strengthen or qualify the central claim."],"forward_implications":["Credible intervals for economic quantities such as marginal rates of substitution and value of travel time can be obtained from a deep learning model by pushing SGLD posterior samples through the marginal-utility function.","The model's empirical coverage stays near the nominal 95% level at both N=1000 and N=10000, while a fully connected neural network undercovers at N=1000 and a conditional logit model misses the nonlinear MRS values.","The posterior credible bands for latent utilities narrow as data grow, which the authors read as evidence that their SGLD implementation captures epistemic uncertainty.","The model matches or beats baselines on out-of-sample accuracy, improving by 8 percentage points in simulations as N grows and outperforming the state-of-the-art benchmark by 6 percentage points on the Swiss route choice data.","Stronger $\\ell^2$ penalties on the nonlinear scale parameters push the model toward the linear, behaviorally informed specification, and the model produces positive value-of-travel-time estimates even with no penalty."],"supporting_citations":[{"why":"Supplies the SGLD algorithm that generates the approximate posterior samples used for all interval estimates.","marker":"[15]"},{"why":"Batch normalization is the mechanism that stabilizes the nonlinear blocks and fixes the location and scale of latent utilities.","marker":"[28]"},{"why":"The authors' prior architecture that the proposed model builds on, supplying the foundations for embeddings, residual connections, and IIA structure.","marker":"[10]"},{"why":"Shows how to extract complete economic information from choice networks, setting the goal of interpretability the model is designed to meet.","marker":"[2]"},{"why":"Introduces the hybrid interpretable deep-learning discrete choice model that this architecture extends and the baseline for predictive performance.","marker":"[9]"},{"why":"Documents the underspecification and unstable parameter estimates problem that motivates the Bayesian treatment.","marker":"[3]"},{"why":"Provides the Monte Carlo simulation design used to evaluate empirical coverage of interval estimates.","marker":"[30]"},{"why":"Supplies the NYC revealed-preference mode choice dataset and the reference range of value-of-travel-time estimates used for comparison.","marker":"[31]"},{"why":"Supplies the Swiss stated-preference train route choice dataset and the reference VOTT values used for comparison.","marker":"[32]"},{"why":"Motivates the embedding layer over shared individual characteristics for identifiability and interpretability.","marker":"[18]"}],"fun_headline_variants":["Bayesian deep choice model hits 95% coverage","Deep choice model: 95% coverage for economics","Neural choice model with reliable intervals","SGLD choice model achieves nominal coverage","Deep choice model: calibrated intervals, robust predictions"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The coverage results rest on the assumption that SGLD, after the two-step warm-start and with non-affine BatchNorm and fixed scale parameters, actually draws samples from the intended posterior over all model parameters; the paper supplies no convergence diagnostics, step-size schedule, or proof of this, and if the sampler targets a different distribution, the claimed 95% credible intervals are not supported.","fun_headline_variants_meta":{"raw":{"variants":["Bayesian deep choice model hits 95% coverage","Deep choice model: 95% coverage for economics","Neural choice model with reliable intervals","SGLD choice model achieves nominal coverage","Deep choice model: calibrated intervals, robust predictions"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001109,"raw_usage":{"total_tokens":4638,"prompt_tokens":977,"completion_tokens":3661,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":593,"completion_tokens_details":{"reasoning_tokens":3605}},"tokens_in":593,"tokens_out":3661,"duration_ms":22237,"temperature":1.0,"reasoning_tokens":3605,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-07T14:35:22.587795+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the Monte Carlo study with a different SGLD step-size schedule or chain length and compute trace-based convergence diagnostics; if empirical coverage of the marginal-rate-of-substitution intervals drops materially, the reported 94-99% coverage is an artifact of the specific sampler settings. A sharper version: on one simulated dataset, replace SGLD with a well-mixing Hamiltonian Monte Carlo sampler and compare the resulting credible intervals for the four MRS values; if the intervals differ substantially, the SGLD chain is not sampling the intended posterior.","supporting_citations":[{"cited_title":"Ioffe, C","cited_arxiv_id":null,"evidence_quote":"Batch normalization is the mechanism that stabilizes the nonlinear blocks and fixes the location and scale of latent utilities."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Shows how to extract complete economic information from choice networks, setting the goal of interpretability the model is designed to meet."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces the hybrid interpretable deep-learning discrete choice model that this architecture extends and the baseline for predictive performance."},{"cited_title":"Sarrias, R","cited_arxiv_id":null,"evidence_quote":"Provides the Monte Carlo simulation design used to evaluate empirical coverage of interval estimates."},{"cited_title":"Vrtic, K","cited_arxiv_id":null,"evidence_quote":"Supplies the Swiss stated-preference train route choice dataset and the reference VOTT values used for comparison."},{"cited_title":"Arkoudi, R","cited_arxiv_id":null,"evidence_quote":"Motivates the embedding layer over shared individual characteristics for identifiability and interpretability."}],"review_version":1}