{"id":"40d85532-8e75-4884-9465-55a332036220","arxiv_id":"2607.04278","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.5,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"CEL jointly learns value, policy, and certainty-equivalent networks for recursive-utility dynamic programming and reports out-of-sample Bellman/Euler residuals of about 1e-4–1e-3 on closed-form, VFI, DSGE, and portfolio benchmarks.","lead":"The paper introduces Certainty Equivalent Learning (CEL), a mesh-free deep-learning method that solves high-dimensional discrete-time dynamic programs with recursive utility by training a neural network for the nonlinear certainty-equivalent term. If reliable, it would let economists and finance researchers solve Epstein–Zin, robust-control, and portfolio problems that grid methods cannot reach.","discovery_kind":"new_method","skeptic_critique":{"model":"grok-4.5","headline":"The residual-size claim that certifies CEL rests on biased Monte Carlo CE targets without a bias-control or welfare-error map.","rationale":"The reader correctly isolates the missing convergence theory and the paper’s own bias observation as the soft spot under the residual-based accuracy claim. My concern sharpens that observation into a concrete, falsifiable gap: the training losses deliberately use the biased plug-in CE estimator that Section 3.1 flags, and residual size alone is not known to bound value or policy error for nonlinear recursive utility. The closed-form LEQG and homothetic robust-control cases already supply the ground truth needed for the proposed G-sweep; if that sweep shows residuals remaining small while CE/value error stays large, the strongest empirical claim weakens and the CONDITIONAL verdict is reinforced rather than overturned. No stronger internal inconsistency appears; the algorithmic novelty relative to expected-utility deep DP and to Friedl et al. stands. Hence the verdict stays CONDITIONAL, with the same practical remedies (code, hyperparameter tables, and preferably a shared-instance comparison) plus the bias audit above.","tokens_in":42615,"tokens_out":687,"duration_ms":7647,"concrete_test":"On the homothetic robust-control closed form (σ=50 and σ=200), retrain the four-network CEL with G\\in{8,32,128,512} while holding all other hyperparameters fixed; report max |Vc^G – Vc^*| and relative value/policy error versus the analytical solution on the VFI capital grid. If either error plateaus above ~1e-3 (or fails to improve with G) while Bellman residuals stay in [1e-4,1e-3], the residual-size claim does not certify solution accuracy.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The paper’s central accuracy claim—that out-of-sample Bellman errors and Euler/FOC residuals of order 1e-4–1e-3 certify that the learned (V,c,Vc) solve the true recursive-utility Bellman fixed point—depends on the training targets for the certainty-equivalent network being faithful. Section 3.1 (third challenge) itself shows that the plug-in estimator f^{-1}(G^{-1}\\sum f(V(s'))) is biased for Vc whenever f^{-1} is nonlinear, with leading bias (f^{-1})''(M)·Var(cM)/2. Yet the three- and four-network losses (28), (32), (44), (49)–(50) minimize exactly that plug-in residual (or its Ve–D decomposition) with finite G and no bias-correction term, importance sampling, or G-scaling study. Target-network Polyak averaging and delayed policy updates stabilize bootstrapping but do not remove the Jensen bias. Consequently residual diagnostics of size ~1e-3 can be consistent with a systematically wrong CE surface, especially under high risk aversion / robustness (σ=50,200; γ=20) where curvature of f is large. Without a bias audit or a mapping from residual size to value/policy error (cf. Santos 2000), the reported residual band does not by itself underwrite the claim that CEL recovers the true solution.","agreement_with_reader":"agree"},"referee_report":{"model":"grok-4.5","summary":"The paper proposes Certainty Equivalent Learning (CEL), claimed as the first deep-learning algorithm for high-dimensional discrete-time infinite-horizon dynamic programming with recursive utility. CEL represents the value function, policy, and (state-control) certainty-equivalent map by neural networks (three-, four-, or two-network variants), trains them by alternating simulation-based least-squares fitting of the CE operator, policy improvement of the Bellman objective, and value fitting, and stabilizes training with exploratory control noise, Polyak target networks, and delayed policy updates. It requires only forward simulation of the transition and does not use Euler equations, first-order conditions, or differentiability of the law of motion. Expected utility is recovered as a special case. Numerical experiments cover discounted linear-exponential-quadratic-Gaussian control (closed-form, including ns=100, nc=50), small-noise robust control (VFI and closed-form homothetic benchmarks), an Epstein–Zin DSGE with stochastic volatility, and multivariate strategic asset allocation; reported out-of-sample Bellman errors and Euler/FOC residuals typically lie in [1e-4,1e-3].","tokens_in":43061,"tokens_out":1297,"duration_ms":22471,"significance":"If the numerical claims hold, CEL would be a practically useful mesh-free tool for a class of models (Epstein–Zin, risk-sensitive/robust control, long-horizon portfolio choice) that are central in macro-finance yet remain hard for grid-based methods once state or control dimension exceeds a handful of variables. The modular CE-network idea, the explicit comparison with closed-form LEQG and homothetic robust-control solutions, and the out-of-sample residual diagnostics are genuine strengths relative to much of the existing deep-learning DP literature. The careful differentiation from Friedl et al. (2023) and the demonstration that the same framework also covers expected utility further increase potential impact.","major_comments":[{"comment":"Section 3.1 (third challenge) correctly notes that the plug-in estimator f^{-1}((1/G) sum f(V(s'))) is biased for the true certainty equivalent whenever f^{-1} is nonlinear, with leading bias proportional to (f^{-1})''(M) Var(cM)/2. Yet the training losses for the three- and four-network architectures—Eqs. (28), (32), (44), (49)–(50)—minimize exactly this plug-in residual (or its Ve–D decomposition) with finite G and no bias-correction, importance sampling, or G-scaling study. Target-network Polyak averaging and delayed policy updates stabilize bootstrapping but do not cancel the Jensen bias. Consequently the reported residual band of 1e-4–1e-3 (especially under high curvature: sigma=50/200, gamma=20) does not by itself certify that the learned (V,c,Vc) solve the true Bellman fixed point. A bias audit (e.g., G-refinement, antithetic or control-variate estimators, or a Santos-style residu","section":null},{"comment":"The central scalability claim rests on the LEQG experiment with ns=100, nc=50 (Section 4.1.3). While relative value errors <1e-4 and small policy errors are impressive against the closed form, the paper supplies no analogous external benchmark for the non-quadratic applications once dimension rises. In the DSGE (3 states, 2 controls) and multivariate portfolio problems the only diagnostics are internal residuals recomputed by nested Monte Carlo. Without either a high-accuracy reference solution or a quantitative residual-to-welfare/policy-error bound, it remains unclear whether residual levels of 1e-3 translate into economically acceptable approximation error once the CE surface is high-dimensional and strongly curved.","section":null},{"comment":"No convergence theory, approximation-rate result, or even a systematic hyper-parameter sensitivity study is provided for the alternating non-convex scheme (Algorithms 3–5). The free parameters (network widths, tau, d, zeta, G, Adam step counts, exploration method) are numerous; the paper’s own discussion of feedback loops between CE learning and policy improvement (Section 3.1, fifth challenge) makes the absence of such analysis material. At minimum, a short ablation on G, tau and the exploration scale, together with multiple random seeds and reported standard errors on the residual tables, is required to support the claim that the method “remains effective in high-dimensional problems.”","section":null}],"minor_comments":[{"comment":"Abstract and several figure captions contain missing spaces (“accuracycomparabletoVFI”, “out-of-sampleBellmanerrors”). A careful copy-edit pass is needed.","section":null},{"comment":"Table 1 and the network-architecture subsections would benefit from an explicit statement of layer widths, activations and output constraints used in each experiment; currently these details are scattered or omitted.","section":null},{"comment":"The comparison with Friedl et al. (2023) is thorough, yet the claim “the first deep learning algorithm” should be qualified more carefully in the abstract and introduction once the differences (quadrature vs. learned CE, residual system vs. direct Bellman maximization) have been stated.","section":null},{"comment":"Appendix A.2 reports matrices only for the ns=8 case; the high-dimensional LEQG matrices are not supplied, hindering exact replication.","section":null}],"recommendation":"major_revision","confidential_remarks":"The numerical evidence against closed forms is stronger than the skeptic’s note suggests, so a pure reject would be too harsh; major revision focused on the bias audit and residual-to-error mapping should suffice. Code release would substantially raise confidence. Fit for a computational-methods journal is good; for a top general-interest outlet the missing theory and limited external benchmarks in the non-LQ applications would be more problematic."},"author_rebuttal":null,"desk_editor":{"model":"grok-4.5","letter":"This is a usable methods paper for people who actually solve recursive-utility models. The new piece is not “deep learning for DP” in general—that literature is already thick—but a clean design choice: treat the state-control certainty equivalent as its own learnable object, improve the policy by maximizing the Bellman objective with that object, and avoid Euler/FOC training and transition differentiability. Relative to Friedl et al.’s residual DEN + low-dim quadrature, that is a genuine alternative, and the modular three/four/two-network split is practical engineering rather than decoration.\n\nWhat they do well is the validation stack. LEQG has closed form and they hit it hard, including ns=100, nc=50 with relative value error below 1e-4. Homothetic robust control lets them beat or match VFI against analytics; the general nonlinear robust case is close to VFI with comparable Euler/Bellman residuals. DSGE and multivariate portfolio results look coherent (smooth policies, residuals mostly 1e-4–1e-3). Out-of-sample MC recomputation of residuals, not training loss, is the right diagnostic habit. Citations are in the right neighborhood.\n\nThe soft spot the stress-test flags is real but not fatal. Section 3.1 itself notes plug-in CE bias under nonlinear f, and the training losses still fit finite-G plug-in targets. Target nets and delayed policy updates stabilize bootstrapping; they do not cancel Jensen bias. So residual size alone does not fully certify the true fixed point under high curvature (large σ, γ=20). They partly offset this with closed-form and VFI anchors, which is why I do not treat the paper as circular. Missing code, thin hyperparameter reporting, and no head-to-head on a shared Friedl-style instance are the other practical gaps.\n\nWho it is for: computational macro/finance people stuck on Epstein–Zin, robust control, or multi-asset recursive portfolio problems. Math is standard Bellman/CE; evidence is numerical and mostly honest. I would send it to referees. Engage if you care about high-dim recursive DP solvers; demand code and a bias/G-scaling check before you trust the residual band as a certificate.","headline":"Solid computational methods paper: CEL is a real design for recursive-utility DP, with strong closed-form and VFI evidence, but residual claims need a bias audit and code.","tokens_in":43676,"tokens_out":567,"would_cite":true,"duration_ms":12947,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"grok-4.5","headline":"A new deep-learning method solves high-dimensional dynamic programs with recursive utility by learning the certainty equivalent itself.","keywords":["recursive utility","deep learning","dynamic programming","certainty equivalent","neural networks","portfolio choice","strategic asset allocation","Epstein–Zin preferences"],"falsifier":"On the closed-form Gaussian linear-exponential-quadratic problem with 100 states and 50 controls, recompute the learned policy and value on a fresh out-of-sample state set; if relative value error exceeds 10^{-3} or absolute policy error exceeds 10^{-2} relative to the analytic solution, the accuracy claim fails.","tokens_in":43464,"feed_emoji":"📈","tokens_out":683,"duration_ms":6909,"temperature":0.7,"pith_summary":"Many economic and financial decisions use recursive utility, which separates risk attitudes from how agents trade off consumption over time. That flexibility makes the Bellman equation hard to solve: the future value enters through a nonlinear certainty equivalent that has no closed form and cannot be estimated by a simple sample average. This paper introduces Certainty Equivalent Learning (CEL), a mesh-free algorithm that trains neural networks for the value function, the policy, and the certainty-equivalent function jointly from simulated paths. Because it never grids the state space and never needs Euler equations or differentiable transitions, it can handle large state and control dimensions. On closed-form linear-exponential-quadratic problems, robust-control models, Epstein–Zin DSGE economies, and multivariate portfolio choice, CEL recovers accurate value and policy functions, including a case with 100 states and 50 controls, with out-of-sample residuals typically between 10^{-4} and 10^{-3}.","feed_headline":"Deep nets learn the certainty equivalent, not just the value","feed_subtitle":"Mesh-free algorithm solves recursive-utility dynamic programs up to 100 states and 50 controls","key_machinery":"Certainty Equivalent Learning (CEL): neural networks jointly approximate the value, the policy, and the certainty-equivalent map Vc(s,c)=f^{-1}(E[f(V(s'))|s,c]), updated by alternating Monte-Carlo regression, Bellman maximization, and (optional) value fitting, with target networks and delayed policy updates for stability.","core_discovery":"The paper claims that treating the state-control certainty equivalent as a separate learnable neural object, then alternating between certainty-equivalent fitting, policy improvement, and value fitting, yields the first practical deep-learning solution method for high-dimensional discrete-time dynamic programming with recursive utility, and that the resulting solutions match closed-form and value-function-iteration benchmarks with residual accuracy of order 10^{-4}–10^{-3}.","pith_inferences":[],"forward_implications":[],"fun_headline_variants":["Deep nets jointly learn value, policy and certainty equivalents","CEL algorithm solves recursive DP via certainty-equivalent nets","Mesh-free deep learning for high-dim recursive-utility programs","First deep method for high-dimensional recursive-utility DP","CEL fits certainty equivalents to solve recursive utility DP"],"cache_read_input_tokens":34432,"weakest_assumption_plain":"The method assumes that nonconvex alternating training of the three networks, fed by Monte Carlo targets that are known to be biased for nonlinear certainty equivalents, still converges to a solution of the true Bellman equation that residual diagnostics of size 10^{-4}–10^{-3} can certify.","fun_headline_variants_meta":{"raw":{"variants":["Deep nets jointly learn value, policy and certainty equivalents","CEL algorithm solves recursive DP via certainty-equivalent nets","Mesh-free deep learning for high-dim recursive-utility programs","First deep method for high-dimensional recursive-utility DP","CEL fits certainty equivalents to solve recursive utility DP"]},"model":"grok-4.5","effort":"low","cost_usd":0.004956,"raw_usage":{"total_tokens":1428,"prompt_tokens":807,"num_sources_used":0,"completion_tokens":81,"cost_in_usd_ticks":49560000,"prompt_tokens_details":{"text_tokens":807,"audio_tokens":0,"image_tokens":0,"cached_tokens":256},"completion_tokens_details":{"audio_tokens":0,"reasoning_tokens":540,"accepted_prediction_tokens":0,"rejected_prediction_tokens":0}},"tokens_in":807,"tokens_out":81,"duration_ms":4388,"temperature":1.0,"reasoning_tokens":540,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-07-11T20:25:16.004899+00:00","model_set":{"reader":"grok-4.5"},"falsifier":"On the closed-form Gaussian linear-exponential-quadratic problem with 100 states and 50 controls, recompute the learned policy and value on a fresh out-of-sample state set; if relative value error exceeds 10^{-3} or absolute policy error exceeds 10^{-2} relative to the analytic solution, the accuracy claim fails.","supporting_citations":[],"review_version":1}