{"id":"687c7351-1ac9-44e9-8aa4-55b5249cbc3b","arxiv_id":"2411.13219","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"For backward stochastic control problems with entropy-regularized costs, the optimal relaxed control is characterized by a variational inequality and, in the linear-quadratic case, is a Gaussian distribution with explicitly computable mean and covariance.","lead":"This paper derives optimality conditions for a control problem where the state is a backward stochastic differential equation and the cost includes an entropy penalty that favors randomized controls. It proves a maximum principle and shows that in the linear-quadratic case the optimal randomized control is Gaussian and unique.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Riccati equation (5.8) is dimensionally inconsistent as printed: the term (R_t+σ²/2 I_p)^{-1}B_t is not an n×n matrix, so existence of Θ and the explicit Gaussian optimal control in Theorem 5.5 are unsupported as stated.","rationale":"The paper's program is coherent and much of the maximum-principle development is standard; I see no circular dependence or fitted parameters. The explicit LQ construction, however, is the centerpiece: Theorem 5.5 claims uniqueness of the Hamiltonian-system solution and Corollary 5.6 claims the Gaussian control is the unique optimal control. Both depend on the decoupling identity and on the Riccati equation. The printed equation is not just notationally sloppy; a scalar test shows the wrong factor changes Θ_0 by a factor of B (here 2), so the claimed explicit formulas fail as stated. This is exactly the reader's weakest assumption, so I agree. The appropriate disposition is unchanged CONDITIONAL: the mathematical idea is sound and the fix, inserting the missing B and B^⊤ around the inverse with correct signs, is likely to restore the theorem, but the proof must be corrected before the theorem can be relied on.","tokens_in":25000,"tokens_out":12843,"duration_ms":121366,"concrete_test":"Set n=p=1, A=C=H=N=0, B=2, R=1, G=1, ξ=1, T=1, σ=1. Solve the Hamiltonian system (5.3) directly: P_t≡-Y_0, so v_t=-(3/2)^{-1}2P_t=4Y_0/3, and the BSDE gives Y_t=1-8Y_0(1-t)/3, hence Y_0=3/11. Decoupling Y_0=Θ_0 P_0+φ_0 with P_0=-Y_0 and φ_0=1 forces Θ_0=8/3. Eq. (5.8) as printed gives -Θdot=(3/2)^{-1}B=4/3, i.e. Θ_0=4/3, contradicting the direct solution; with B(R+σ²/2 I)^{-1}B^⊤ in place of (R+σ²/2 I)^{-1}B, Θ_0=8/3 and the decoupling matches.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The load-bearing weakness is Eq. (5.8), on which Theorem 5.5 and Corollary 5.6 rest. The decoupling ansatz Y=ΘP+φ is legitimate, but the Riccati equation printed there cannot be a well-formed equation for Θ∈R^{n×n}. With B∈L∞(0,T;R^{n×p}) and R∈L∞(0,T;S^p_+), the term (R_t+σ²/2 I_p)^{-1}B_t has the wrong shape: if B_t is n×p, the product is undefined; if B_t is taken as B_t^⊤ under the paper's transpose-omitting convention, it is p×n, not n×n. Re-deriving the coefficient of P_t in the λ-equation from (5.3)–(5.6) gives a quadratic term B_t(R_t+σ²/2 I_p)^{-1}B_t^⊤ (one B on each side), not (R_t+σ²/2 I_p)^{-1}B_t. Since the claimed unique Θ, the Gaussian mean v_t^μ=-(R_t+σ²/2 I_p)^{-1}B_t^⊤ P_t^μ, and the uniqueness argument all depend on this equation, the central LQ existence/uniqueness claim is not proven as printed. The error is likely a repairable typo/omission, but it is load-bearing.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies an entropy-regularized backward stochastic control problem in which the state is a controlled BSDE (1.1) and the cost functional contains a relative-entropy penalty with respect to a reference measure e^{-U}, as in (1.2). The control is measure-valued (relaxed), and the authors prove a stochastic maximum principle by the convex variation method (Theorem 3.7), a sufficient condition (Theorem 4.2), and an implicit form of the optimal relaxed control (4.7). In the linear-quadratic case, with standard normal reference measure, the paper claims that the stochastic Hamiltonian system (5.3) has a unique solution, that the optimal control is Gaussian with covariance Sigma_t^mu = (sigma^2/2)(R_t + (sigma^2/2)I_p)^{-1} and mean v_t^mu = -(R_t + (sigma^2/2)I_p)^{-1} B_t^T P_t^mu, and that this control is the unique optimal control (Theorem 5.5 and Corollary 5.6). The proof uses a decoupling ansatz Y_t = Theta_t P_t + phi_t and a Riccati equation for Theta.","tokens_in":25276,"tokens_out":9911,"duration_ms":104369,"significance":"If the linear-quadratic result is correct, the paper provides the first explicit form of an entropy-regularized optimal relaxed control for a backward stochastic system, thereby extending the exploratory-control methodology of Wang, Zariphopoulou and Zhou (2020) and Siska and Szpruch (2024) from forward to backward dynamics. The strengths of the paper are the detailed variation estimates, the derivation of the Gaussian form from the first-order condition rather than from an ad hoc ansatz, and the Itô-based uniqueness argument. However, the central LQ existence/uniqueness claim is not supported as printed because Eq. (5.8), the Riccati equation on which Theorem 5.5 and Corollary 5.6 depend, is dimensionally inconsistent. The issue appears to be a repairable transposition/dimensional typo, so the manuscript should be revised rather than rejected.","major_comments":[{"comment":"Eq. (5.8) is dimensionally inconsistent as printed. With B_t in L^infty(0,T; R^{n x p}) and R_t in S^p_+, the term (R_t + (sigma^2/2) I_p)^{-1} B_t is not an n x n matrix: the product is undefined if B_t is n x p, and it is p x n if B_t is read as B_t^T under the paper's transpose-omitting convention. Re-deriving the coefficient of P_t in the lambda equation from (5.3) and (5.5) gives B_t (R_t + (sigma^2/2) I_p)^{-1} B_t^T P_t, with one B on each side; the correct Riccati term should therefore be B_t (R_t + (sigma^2/2) I_p)^{-1} B_t^T, not (R_t + (sigma^2/2) I_p)^{-1} B_t. Since the claimed unique Theta, the explicit Gaussian formulas in Theorem 5.5, and the uniqueness argument in Corollary 5.6 all rest on this equation, the existence and uniqueness results are not proven as stated. The authors should correct the equation and either supply a proof or a precise citation for the existence of a bounded solution Theta for the corrected equation.","section":"§5, Eq. (5.8)"},{"comment":"Remark 5.3 takes U identically equal to 0, but Definition 2.1 requires e^{-U} to be a density on R^p; e^0 = 1 is not integrable over R^p, so the reference measure e^{-U} is not a probability density in that case. The cost-of-exploration computation in Remark 5.3 is therefore outside the framework defined in Section 2. This does not affect the main theorem, which uses the standard normal reference measure, but the remark should be corrected, for example by explicitly extending the definition to allow sigma-finite reference measures or by normalizing U appropriately.","section":"§5, Remark 5.3"}],"minor_comments":[{"comment":"The assertion that the pointwise variational inequality (3.17) is equivalent to the minimization problem (4.4) requires convexity of H^sigma in m. As printed, the equivalence appears immediately after Theorem 4.2, but the text should explicitly state that this step uses Assumption 4.1; without that assumption, (3.17) is only a first-order variational inequality.","section":"§4, Eq. (4.4)"},{"comment":"In several places in Section 5, the notation B_t P_t should be B_t^T P_t (and P_t B_t should be P_t^T B_t). The paper states that transpose symbols are omitted unless necessary, but this convention is especially dangerous in Section 5, where matrix dimensions and quadratic forms are central; the dimensional error in Eq. (5.8) is partly a product of this convention. The authors should clarify the convention at the start of Section 5 or restore the transposes in all dimension-sensitive formulas.","section":"§5, Eqs. (5.5), (5.10)"},{"comment":"In Eq. (4.5), the argument l(t,Y_t^mu,Z_t^mu,P_t^mu,m) includes P_t^mu, but the running cost l does not depend on the adjoint variable; this should be l(t,Y_t^mu,Z_t^mu,m).","section":"§4, Eq. (4.5)"},{"comment":"There are numerous typographical issues, including 'theoretical depict' in the abstract and inconsistent spelling of author names in the references; a careful language and proofreading pass is needed.","section":"Throughout"}],"recommendation":"major_revision","confidential_remarks":"The central defect is localized: Eq. (5.8) is dimensionally inconsistent, and the fix appears to be a transposition/correction in one term. If the authors correct the Riccati equation, re-derive the lambda equation with explicit transposes, and justify existence of Theta for the corrected equation, the main LQ result is likely salvageable. The Remark 5.3 issue with U=0 is also worth fixing. I do not see problems of novelty or attribution beyond the acknowledged dependence on [20], [28], [26], and [14]."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear [Name],\n\nShort version: this is the first maximum principle and LQ existence/uniqueness result for entropy-regularized backward stochastic control. The forward counterpart is known ([28], [26], [12]), but the backward setting is genuinely new here. The proof structure is classical—convex variation, flat derivatives, decoupling—but it is executed cleanly and the entropy term is handled honestly. I would take the paper seriously; it has two repairable slips that need to be fixed before the results are used.\n\nWhat is good: the maximum principle (Theorem 3.7) is derived with the right convex-variation estimates and the lower-semicontinuity of the entropy is handled without hand-waving. The sufficient condition (Theorem 4.2) is the natural one and the Hamiltonian convexity conditions are stated clearly. The LQ section properly derives the Gaussian control from the first-order condition rather than assuming it, and the exploration cost computation in Remark 5.3 is a useful extra.\n\nSoft spots:\n\nFirst, Remark 5.3 sets U≡0. With Definition 2.1, U must satisfy that e^{-U} is a density on R^p; U≡0 gives e^0=1, which is not integrable. This looks like a careless shorthand for a constant shift, but as printed it contradicts the standing assumptions.\n\nSecond, and more seriously, the Riccati equation (5.8) is dimensionally inconsistent as printed. With B in R^{n×p} and R in S^p_+, the term (R_t+σ²/2 I_p)^{-1}B_t is not an n×n matrix. Eliminating v from the coupled system gives B_t(R_t+σ²/2 I_p)^{-1}B_t^T (one B on each side). The same transpose omission appears in the mean formula (5.5) and in the uniqueness proof. The decoupling strategy is legitimate, but the stated equation cannot be the equation for Θ, and the existence and uniqueness of Θ together with the Gaussian optimal control and the uniqueness argument all rest on it. This is almost certainly a typo, but it is load-bearing; the paper cannot be accepted as is.\n\nMinor: Section 4's exploratory BSDE derivation is informal, but it is clearly motivational and does not affect the main theorems.\n\nWho this is for: researchers working on relaxed controls, entropy regularization, or backward stochastic control and its financial applications. The paper is a theory paper; no numerics, but the proofs are the content.\n\nRecommendation: send it to peer review. The referee should ask for the LQ section to be re-derived with careful dimensions and for Remark 5.3 to be reconciled with the definitions. With those changes the paper should be solid.","headline":"First entropy-regularized maximum principle and LQ result for backward stochastic control; correct in architecture but needs a dimensions-fix in the Riccati equation and a cleanup of Remark 5.3.","tokens_in":25797,"tokens_out":4565,"would_cite":true,"duration_ms":45153,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["93E20","93C15"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper proves that in the entropy-regularized backward linear-quadratic control problem with a standard normal reference measure, the unique optimal relaxed control is Gaussian and is given explicitly by its covariance and mean.","keywords":["backward stochastic differential equations","entropy regularization","relaxed control","stochastic maximum principle","linear-quadratic control","exploration-exploitation","Gaussian optimal control"],"falsifier":"Take $n=2$, $p=1$ and set $B_t$ to a nonzero two-dimensional column vector with $A=C=H=N=0$. Then the Riccati equation (5.8) as printed adds a vector to $2\\times2$ matrix terms, so no $2\\times2$ solution $\\Theta$ can exist; testing whether the corrected term $B_t(R_t+(\\sigma^2/2)I_p)^{-1}B_t^\\top$ gives a unique solution would settle whether the claimed uniqueness is recoverable.","tokens_in":24773,"feed_emoji":"🎲","tokens_out":7625,"duration_ms":69459,"temperature":0.7,"pith_summary":"This paper establishes a stochastic maximum principle for backward stochastic control systems whose state equation is a controlled backward stochastic differential equation (BSDE) and whose control is a probability-measure-valued relaxed control penalized by relative entropy. It shows that an optimal relaxed control must satisfy a variational inequality involving the flat derivative of the Hamiltonian, that this condition is also sufficient when the Hamiltonian is convex, and that any optimal control has an implicit Gibbs-type density. For the linear-quadratic specialization with a standard normal reference measure, it proves that the stochastic Hamiltonian system has a unique solution and that the unique optimal control is Gaussian, with covariance $\\Sigma_t^\\mu = (\\sigma^2/2)(R_t + (\\sigma^2/2)I_p)^{-1}$ and mean $v_t^\\mu = -(R_t + (\\sigma^2/2)I_p)^{-1}B_tP_t^\\mu$. The result matters because it turns exploration into a tractable explicit object for backward systems, the kind of system used in financial replication and in reinforcement-learning algorithms.","feed_headline":"Optimal relaxed control in backward LQ systems is Gaussian","feed_subtitle":"Maximum principle plus decoupling yield a unique Gaussian optimal control with explicit mean and covariance.","key_machinery":"The load-bearing objects are the admissible set $\\mathcal{A}$ of measure-valued relaxed controls with finite relative entropy, the flat (linear functional) derivative $\\delta F/\\delta m$ used to differentiate the cost and generator with respect to measures, and the stochastic Hamiltonian system (5.3), which couples the controlled BSDE for $(Y^\\mu,Z^\\mu)$, the adjoint SDE for $P^\\mu$, and the first-order optimality condition that pins down the control density. The existence proof for the LQ case runs through the decoupling ansatz $Y_t^\\mu = \\Theta_t P_t^\\mu + \\varphi_t$, which turns the Hamiltonian system into a Riccati equation for $\\Theta$ and a linear BSDE for $\\varphi$; the Gaussian form of the control then follows by completing the square in the exponent of the Gibbs density.","core_discovery":"The central claim is that entropy-regularized exploratory control of BSDEs is not merely a formal device: in the linear-quadratic case with standard normal reference measure, the optimal relaxed control exists, is unique, and is Gaussian. The paper derives this by combining a convex-variation maximum principle with the decoupling ansatz $Y_t^\\mu = \\Theta_t P_t^\\mu + \\varphi_t$, which reduces the stochastic Hamiltonian system to a Riccati equation for $\\Theta$ and a linear BSDE for $\\varphi$. The resulting optimal policy has covariance $\\Sigma_t^\\mu = (\\sigma^2/2)(R_t + (\\sigma^2/2)I_p)^{-1}$ and mean $v_t^\\mu = -(R_t + (\\sigma^2/2)I_p)^{-1}B_tP_t^\\mu$, and as $\\sigma \\to 0$ it concentrates on the strict control $v_t^\\mu$ with exploration cost $\\sigma^2 p T/4$. The paper also gives the implicit form of the optimal control for the general non-LQ problem as a fixed point of a Gibbs-type density, and proves the maximum principle by convex variation for systems with random coefficients.","pith_inferences":["A natural testable extension is to replace the standard normal reference by a general log-concave $e^{-U}$; the Gibbs form (4.7) suggests the optimal control remains Gaussian only when $U$ is quadratic, with covariance depending on the curvature of $U$.","Because the maximum principle handles random coefficients, the same entropy-regularized backward framework could be applied to hedging or portfolio replication under Knightian uncertainty, where the relaxed control represents a family of beliefs rather than a single strategy.","The decoupling method in Section 5 gives a template for proving existence and uniqueness in related exploratory backward problems, such as mean-field or time-inconsistent objectives; the key step would be a well-posed Riccati equation."],"forward_implications":["For the LQ problem with standard normal reference, the optimal exploration policy is fully explicit: a Gaussian with covariance $(\\sigma^2/2)(R_t + (\\sigma^2/2)I_p)^{-1}$, so exploration shrinks as the entropy weight $\\sigma$ decreases or the running cost weight $R$ increases.","The cost of exploration is $\\sigma^2 pT/4$, and as $\\sigma \\to 0$ the relaxed optimal control converges weakly to a Dirac measure at the strict control, recovering the classical backward LQ optimum.","The necessary and sufficient maximum principle gives a concrete fixed-point equation for the optimal control density in the non-LQ case, which can serve as the basis for successive-approximation algorithms.","With convexity of the Hamiltonian, any control satisfying the variational inequality (3.17) is optimal, so the condition is necessary and sufficient in the LQ setting.","The unique solvability of the stochastic Hamiltonian system provides a clean characterization of the optimal triple $(Y^\\mu,Z^\\mu,P^\\mu)$ together with the Gaussian control density."],"supporting_citations":[{"why":"Supplies the decoupling technique and the backward-LQ Riccati uniqueness result that Theorem 5.5 relies on.","marker":"[20]"},{"why":"Gives the classical well-posedness result for BSDEs used to solve the state equation and the linear BSDE for $\\varphi$.","marker":"[22]"},{"why":"Motivates the entropy-regularized relaxed-control formulation and supplies the Gaussian optimal control and cost-of-exploration pattern that the backward case mirrors.","marker":"[28]"},{"why":"Provides the prior-reference relative entropy and the maximum-principle treatment of measure-valued controls used in Sections 3 and 4.","marker":"[26]"},{"why":"Justifies enlarging the admissible set to all probability measures and gives the fixed-point form of the optimal density used in the implicit-form discussion.","marker":"[14]"},{"why":"Provides the flat-derivative calculus and the convexity of the admissible set on which the variation arguments rest.","marker":"[18]"}],"fun_headline_variants":["Entropy-regularized backward control yields Gaussian optimal policy","Gaussian optimal control emerges from entropy-regularized BSDEs","Maximum principle plus decoupling: Gaussian control for backward LQ","Entropy makes backward control unique and Gaussian","Backward control with entropy: explicit Gaussian solution"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The proof that an optimal control exists in the LQ case depends on the decoupling assumption $Y = \\Theta P + \\varphi$ and on the Riccati equation having a unique bounded solution; the printed equation has a term whose dimensions do not match the others, so this existence claim rests on a correction or a fuller derivation.","fun_headline_variants_meta":{"raw":{"variants":["Entropy-regularized backward control yields Gaussian optimal policy","Gaussian optimal control emerges from entropy-regularized BSDEs","Maximum principle plus decoupling: Gaussian control for backward LQ","Entropy makes backward control unique and Gaussian","Backward control with entropy: explicit Gaussian solution"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000183,"raw_usage":{"total_tokens":1297,"prompt_tokens":913,"completion_tokens":384,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":529,"completion_tokens_details":{"reasoning_tokens":305}},"tokens_in":529,"tokens_out":384,"duration_ms":4333,"temperature":1.0,"reasoning_tokens":305,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T16:43:50.846719+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take $n=2$, $p=1$ and set $B_t$ to a nonzero two-dimensional column vector with $A=C=H=N=0$. Then the Riccati equation (5.8) as printed adds a vector to $2\\times2$ matrix terms, so no $2\\times2$ solution $\\Theta$ can exist; testing whether the corrected term $B_t(R_t+(\\sigma^2/2)I_p)^{-1}B_t^\\top$ gives a unique solution would settle whether the claimed uniqueness is recoverable.","supporting_citations":[{"cited_title":"Lim and X","cited_arxiv_id":null,"evidence_quote":"Supplies the decoupling technique and the backward-LQ Riccati uniqueness result that Theorem 5.5 relies on."},{"cited_title":"Pardoux and S","cited_arxiv_id":null,"evidence_quote":"Gives the classical well-posedness result for BSDEs used to solve the state equation and the linear BSDE for $\\varphi$."},{"cited_title":"W ang, T","cited_arxiv_id":null,"evidence_quote":"Motivates the entropy-regularized relaxed-control formulation and supplies the Gaussian optimal control and cost-of-exploration pattern that the backward case mirrors."},{"cited_title":"ˇSiˇska and L","cited_arxiv_id":null,"evidence_quote":"Provides the prior-reference relative entropy and the maximum-principle treatment of measure-valued controls used in Sections 3 and 4."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Justifies enlarging the admissible set to all probability measures and gives the fixed-point form of the optimal density used in the implicit-form discussion."}],"review_version":1}