{"id":"db09ae89-eae2-4ce3-98e4-7dea0c6b98ce","arxiv_id":"2509.03251","paper_version":1,"verdict":"REJECT","confidence":"HIGH","novelty_score":4.0,"correctness_risk":"high","formal_verification":"none","parameter_count":3,"one_line_summary":"The paper derives and tests an RL-based mean-variance strategy for multi-period asset-liability management with hidden bull/bear regimes, but its filtering step is not valid.","lead":"This paper derives a reinforcement-learning portfolio rule that manages liabilities in a market switching between bull and bear phases. It claims the rule hits a target wealth with lower variance than classical methods in simulations and on S&P 500 data, but the partial-information filter that drives the rule is flawed.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Lemma 3.1's deterministic filter contradicts its sigma-field equality claim, so Theorem 3.2 and the PoEMV-1 algorithm rest on an invalid separation argument.","rationale":"The reader's weakest_assumption is exactly Lemma 3.1, and I agree this is the load-bearing defect. It is not a matter of consensus or parameter choice; the proof is internally invalid. The deterministic recursion (3.8) conflicts with the sigma-field equality asserted in the same lemma. Even if p_hat were reinterpreted as an unconditional regime probability, the 'filtered' moments in (3.14)-(3.17) would be unconditional moments, and the optimal policy in Theorem 3.2 would not respond to observed returns and liabilities beyond the state variables x and l. The separation principle cited from Xiong and Zhou (2007) does not apply, because that work uses a genuine likelihood-based filter, whereas the filter here ignores the observation densities. The numerical simulations and empirical tests in Section 5 cannot validate the central claim because the simulator and the learned policy both use the same flawed p_hat recursion. I find no independent support that would soften the reject verdict: the complete-information derivation is a reasonable extension of Cui et al. (2023), and the actor-critic machinery is standard, but the paper's advertised partial-information solution is not established. No ad hominem is intended; the issue is a mathematical gap in the proof.","tokens_in":38300,"tokens_out":4995,"duration_ms":42887,"concrete_test":"Recompute the filter update. For a two-regime Markov chain with transition matrix P and observation density f(x_{t+1}, l_{t+1} | epsilon_{t+1}=i), the correct posterior is p_hat_{t+1} = [p_pred f(x_{t+1}, l_{t+1} | epsilon_{t+1}=1)] / [p_pred f(x_{t+1}, l_{t+1} | epsilon_{t+1}=1) + (1-p_pred) f(x_{t+1}, l_{t+1} | epsilon_{t+1}=2)], with p_pred = P21(1-p_hat_t) + P11 p_hat_t. Use the paper's simulation parameters (e0_t(1)=1.2, e0_t(2)=1.05, q_t(1)~N(0.05, 0.1), q_t(2)~N(0.01, 0.2), p_hat_0=0.3) and one simulated path with x_1 clearly in the low-return regime. Compare the resulting posterior with the deterministic value P21 + p_hat_0(P11-P21) ≈ 0.30756 from (3.8). A material difference confirms that (3.8) is not the conditional filter. Also verify that under the paper's recursion x_1 is not sigma(p_hat_0, p_hat_1)-measurable, contradicting Lemma 3.1.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The partial-information contribution hinges on Lemma 3.1 (Section 3.2). Equation (3.8) gives p_hat_{t+1} = P21 + p_hat_t(P11 - P21), and the lemma claims sigma(x_s, l_s, s <= t) = sigma(p_hat_s, s <= t). Because p_hat_0 is known and P is known, the recursion makes p_hat_t a deterministic function of t, so sigma(p_hat_s, s <= t) is trivial, while x_t and l_t are random. The sigma-field equality cannot hold unless the model is degenerate. The proof's fourth equality cancels P(x_{t+1}, l_{t+1} | epsilon_{t+1}=1, F_t^{x,l}) against P(x_{t+1}, l_{t+1} | F_t^{x,l}), asserting that epsilon_{t+1} cannot improve knowledge of x_{t+1}, l_{t+1}. This is false: the conditional law of x_{t+1}, l_{t+1} depends on the regime through e_t(epsilon_t) and q_t(epsilon_t). A correct hidden-Markov filter would multiply the prediction by the observation likelihood, making p_hat_{t+1} depend on realized x_{t+1} and l_{t+1}. Since Theorem 3.2, the separation principle, and Algorithm 2 (PoEMV-1) all use p_hat from (3.8), the central claim—an RL solution for unobservable regimes—is unsupported. The complete-information Theorem 3.1 appears to be a plausible extension of Cui et al. (2023), but it does not rescue the partial-information result.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies a multi-period mean-variance asset-liability management problem in a regime-switching market, with an entropy-regularized exploratory formulation solved by reinforcement learning. Under complete information, Theorem 3.1 gives a Bellman-derived Gaussian optimal policy and quadratic value function. Under partial information, the paper introduces a filtering process for the hidden regime, states a separation principle in Lemma 3.1, and obtains an analogous optimal policy in Theorem 3.2. Based on these policies, the authors propose three algorithms: CoEMV (complete information), PoEMV-1 (filtered regime learning), and PoEMV-2 (regime estimated by its unconditional expectation), together with a self-correcting Lagrange multiplier update for learning the target wealth constraint. Numerical simulations and an S&P 500 empirical study report that PoEMV-1 performs best in mean terminal wealth, variance, and Sharpe ratio.","tokens_in":38625,"tokens_out":3479,"duration_ms":36301,"significance":"If the main derivation were valid, the paper would make a useful contribution by extending the exploratory mean-variance RL framework of Cui et al. to include uncontrollable liabilities and regime-switching markets. The complete-information derivation in Appendix A.1 is detailed and appears internally consistent as an extension of known results, and the policy-improvement theorems in Appendix A.2 provide a structured convergence argument. The paper also makes a genuine effort at algorithmic implementation and out-of-sample testing. However, the partial-information contribution, which is the paper's central claimed novelty, rests on Lemma 3.1. That lemma asserts a deterministic filter that ignores observations and an impossible sigma-field equality. Since Theorem 3.2, Algorithm 2, and all of the PoEMV-1 results are built on this lemma, the partial-information results are not supported. In addition, the numerical evidence that PoEMV-1 achieves the target mean is largely a consequence of the self-correcting Lagrange multiplier update rather than independent evidence of optimality.","major_comments":[{"comment":"Lemma 3.1 is the load-bearing step of the partial-information analysis, and it is incorrect. Equation (3.8) gives p_hat_{t+1} = P21 + p_hat_t(P11 - P21), a recursion that depends only on the known initial distribution p_hat_0 and the known transition matrix P. Hence p_hat_t is a deterministic function of t, so the sigma-field sigma(p_hat_s, s <= t) is trivial. The lemma also asserts sigma(x_s, l_s, s <= t) = sigma(p_hat_s, s <= t), which is impossible because x_t and l_t are random while p_hat_t is deterministic. The proof's fourth equality cancels the density of (x_{t+1}, l_{t+1}) given (epsilon_{t+1}=1, F_t^{x,l}) against the density given F_t^{x,l}, claiming that epsilon_{t+1} cannot improve knowledge of x_{t+1} and l_{t+1}. That claim is false: the conditional law of the asset return e_t(epsilon_t) and liability return q_t(epsilon_t) depends on the regime, so the observation likelihood must enter the filter update. A correct hidden-Markov filter would make p_hat_{t+1} depend on the realized x_{t+1} and l_{t+1} through the likelihood, not merely on the transition matrix.","section":"Section 3.2, Lemma 3.1"},{"comment":"Because Lemma 3.1 is invalid, the separation principle stated in Section 3.2 and Theorem 3.2 are unsupported. The proof of Theorem 3.2 in Appendix A.1 is explicitly only a substitution of hatted quantities into the complete-information formulas; it does not supply an independent argument that the filtered dynamics (3.11) correctly represent the conditional law of the surplus under F^{x,l}. Consequently, the optimal policy (3.21), the PoEMV-1 algorithm in Section 4.2.2, and the simulation and empirical results labeled PoEMV-1 all rest on an unjustified reduction of a partial-information problem to a complete-information problem. Even if the complete-information Theorem 3.1 is correct, it does not rescue the paper's main claimed contribution of an RL solution for unobservable regimes.","section":"Theorem 3.2 and Algorithm 2"},{"comment":"The self-correcting update w <- w - alpha(average terminal surplus - d) forces the mean terminal surplus toward the target d by construction. After enough updates, any policy of sufficiently flexible form will have sample mean terminal surplus near d, regardless of whether the policy is optimal for the constrained mean-variance problem. The paper uses mean terminal wealth close to d as a headline result in Table 1 and Table 2, but this is not independent evidence of optimality or of superior performance. The meaningful comparison would need to hold fixed the way the target is enforced across algorithms and report results without the self-correcting mean-matching update, or otherwise demonstrate that the learned multiplier solves the original constraint rather than merely matching the sample average.","section":"Section 4.2.1, Lagrange multiplier update"}],"minor_comments":[{"comment":"The notation S_t^pi for the surplus and later ^S_t^pi for the filtered surplus is introduced but not used consistently; the paper switches between x_t, l_t and ^x_t, ^l_t without always stating which process is meant, which makes the filtering section hard to follow.","section":"Section 2.2.2, Eq. (2.6)"},{"comment":"PoEMV-2 is trained for 4,000 iterations while PoEMV-1 and CoEMV are trained for 10,000 iterations; the claim of 'faster convergence' for PoEMV-1 relative to PoEMV-2 is confounded by the unequal training budgets, so the comparison should be repeated with matched iteration counts.","section":"Section 5.1, Table 1"},{"comment":"The sentence 'From Lemma 3.1 we have ... E_{t,x,l}[S_{t+1}^pi | F_t^{x,l}] = E_{t,x,l,^p}[S_{t+1}^pi] = ^S_{t+1}^pi' is not justified: replacing unobservable random returns by their conditional expectations inside the nonlinear surplus dynamics changes the conditional expectation, and the equality would require a linearity argument that is absent.","section":"Section 3.2, Eq. (3.11)"}],"recommendation":"reject","confidential_remarks":"The paper has a clear, load-bearing error in Lemma 3.1 that invalidates the partial-information contribution. A correct filter would lead to observation-dependent updates and would require redoing Theorems 3.2, the policy improvement result, and the numerical implementation, which is well beyond a routine revision. The complete-information part may be salvageable as a separate contribution, but as submitted the manuscript's main claims are not supportable. I would not encourage the editor to seek a quick revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The advertised contribution—an RL solution for regime-switching asset-liability management with unobservable regimes—is not supported. Lemma 3.1 in Section 3.2 asserts that the filter p_hat follows the deterministic recursion (3.8) and that sigma(x_s, l_s, s<=t) equals sigma(p_hat_s, s<=t). That cannot be right: p_hat_0 and the transition matrix are known, so p_hat_t is a deterministic function of t, while x_t and l_t are random. The sigma-field equality would make the observations deterministic. The proof's fourth equality cancels the conditional density of (x_{t+1}, l_{t+1}) given epsilon_{t+1}=1 and the history against the unconditional density, which is false because the law of x and l depends on the regime. A correct hidden-Markov filter would update p_hat using the observation likelihood, making p_hat genuinely random and dependent on the data. Since Theorem 3.2, the separation principle, and Algorithm 2 all use p_hat from (3.8), the partial-information contribution is built on a false premise. The reader's stress-test note lands; I agree with the rejection.\n\nWhat the paper does well: the complete-information derivation in Theorem 3.1 is a legitimate extension of Cui et al. (2023) to a two-regime market with uncontrollable liabilities, giving a closed-form Gaussian exploratory policy and value function. The policy improvement and convergence theorems in Section 4 are standard but carefully executed. The simulation framework is described in enough detail to be reproduced, though no code or error bars are provided.\n\nA second soft spot: the self-correcting Lagrange multiplier update w := w - alpha(avg(x_T - l_T) - d) mechanically drives the average terminal surplus toward d. Reporting mean terminal wealth close to d after training is therefore by construction, not independent evidence of optimality. The variance and Sharpe comparisons are still informative, but the target-attainment claim should be downweighted.\n\nThe numerical comparisons show PoEMV-1 beating PoEMV-2 and CoEMV on in-sample and out-of-sample metrics, but because the filter used to compute p_hat is wrong, those gains cannot be attributed to learning the hidden regime. Replacing Lemma 3.1 with a proper likelihood-based filter might change the verdict, but as it stands the central claim is unsupported.\n\nRecommendation: this deserves a serious referee, and an editor should send it out rather than desk-reject—the complete-information section is nontrivial and the partial-information flaw is concrete and fixable in principle. But the current version should be rejected unless Lemma 3.1 is replaced and Theorem 3.2 and Algorithm 2 are reworked accordingly.","headline":"The partial-information result rests on a filter that ignores the data, so the paper's main claim collapses; the complete-information extension is real but does not rescue it.","tokens_in":39188,"tokens_out":1514,"would_cite":false,"duration_ms":17107,"reading_group":"no","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["91B28","93E11","93E20"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper derives a Gaussian optimal policy for mean-variance asset-liability management with hidden bull-bear regimes, and claims an RL algorithm learns it.","keywords":["mean-variance portfolio selection","uncontrollable liability","exploratory reinforcement learning","asset-liability management","regime-switching market","partial information","stochastic filtering","precommitted strategy"],"falsifier":"Simulate the paper's model while recording the true regime alongside $x_t$ and $l_t$, and at each step compute the Bayes posterior $P(\\varepsilon_{t+1}=1\\mid x_{t+1},l_{t+1},x_t,l_t)$ from the model's likelihoods. Compare it with the deterministic value $P_{21}+\\hat p_t(P_{11}-P_{21})$; if the two diverge on paths with unusually high or low excess returns — as they will whenever returns are regime-informative — the filter, the separation argument, and the claimed optimal policy all fail.","tokens_in":38016,"feed_emoji":"📈","tokens_out":10070,"duration_ms":85606,"temperature":0.7,"pith_summary":"This paper seeks to establish that multi-period mean-variance asset-liability management, with a hidden two-regime market and uncontrollable liabilities, has an explicitly solvable exploratory solution. It claims that once the hidden regime is filtered, the optimal policy is Gaussian: the risky-asset holding is linear in current wealth, the filtered return forecasts, and the liability-growth term, while the exploration variance is set by the exploration weight and the filtered second moment of the excess return. It further claims that an actor-critic reinforcement-learning algorithm, PoEMV-1, learns this policy and, in simulations and on S&P 500 data, lands terminal wealth close to the target $d=8$ with smaller variance than a no-regime-learning variant or a complete-information benchmark. A sympathetic reader would care because the paper offers a direct path from a realistic partial-information ALM model to an implementable rebalancing rule, with the parameter estimation burden shifted onto the learning algorithm.","feed_headline":"Hidden-regime portfolios reduce to one Gaussian rule","feed_subtitle":"Multi-period mean-variance with liabilities has a closed-form Gaussian policy; the RL version reaches target wealth with small variance.","key_machinery":"The engine of the paper is a Bellman recursion for a value function that is quadratic in wealth $x$ and liability $l$, together with an entropy-regularized policy improvement step. At each period the minimizing control is a Gaussian density: the mean cancels the linear terms built from the filtered moments $\\hat A_{0,t},\\hat A_{1,t},\\hat A_{2,t}$ and the products $\\hat F_{1,k}/\\hat B_{1,k}$, and the variance is the exploration temperature $\\lambda$ divided by the filtered second moment of the excess return, times a product of ratios $\\hat B_{1,k}/\\hat F_{1,k}$. The recursive filter for the hidden regime probability, $\\hat p_{t+1}=P_{21}+\\hat p_t(P_{11}-P_{21})$, is what converts the partial-information problem into a complete-information problem in $\\hat p$, and the same quadratic structure yields finite-step policy improvement and convergence theorems for both the observable and filtered cases.","core_discovery":"On its own terms, the paper's central discovery is Theorem 3.2: for fixed Lagrange multiplier $w$, the partially observable exploratory mean-variance problem with liabilities in a two-regime Markov market has value function (3.18) and an optimal feedback control (3.21), which is Gaussian. The mean of that Gaussian is the negative of a linear combination of wealth $x$, the filtered first moments of asset and liability returns, and the product term $(w+l\\prod_{k=t}^{T-1}\\hat A_{2,k})$; the variance is $\\frac{\\lambda}{2\\hat B_{1,t}}\\prod_{k=t+1}^{T-1}\\frac{\\hat B_{1,k}}{\\hat F_{1,k}}$. The route to the theorem is Lemma 3.1, which asserts that the hidden-regime posterior follows the deterministic recursion $\\hat p_{t+1}=P_{21}+\\hat p_t(P_{11}-P_{21})$ and that the observable filtration is generated by $\\hat p$, so the partial-information control problem becomes a complete-information problem in the belief variable. The paper also claims that its PoEMV-1 actor-critic algorithm converges to this policy in finitely many improvement steps and, in both simulation and out-of-sample S&P 500 tests, gives mean terminal wealth close to $d=8$ with lower variance than PoEMV-2 and CoEMV.","pith_inferences":["Editorial inference: if the deterministic filter of Lemma 3.1 were replaced by a likelihood-based Bayesian update using observed returns and liabilities, the optimal policy would generically depend on the latest observations; comparing these two filters on the paper's own simulated paths would isolate whether regime learning or deterministic belief motion drives the reported gains.","Editorial inference: the self-correcting Lagrange multiplier is a heuristic average over the most recent terminal surpluses; its convergence is demonstrated empirically but not proved, so a rate or almost-sure guarantee would be a natural extension.","Editorial inference: the same quadratic value function and Gaussian policy family should extend to more than two regimes and to multiple risky assets, with a multivariate Gaussian policy and positive-definite filtered second-moment matrices playing the role of $\\hat B_{1,t}$.","Editorial inference: the entropy-regularized formulation suggests a direct test of the exploration-exploitation tradeoff: varying $\\lambda$ should shift terminal wealth variance monotonically, and the paper's closed form predicts exactly how."],"forward_implications":["If Theorem 3.2 is right, an investor who never observes the market regime can still rebalance by a closed-form Gaussian rule that depends only on wealth, liabilities, and filtered moments of returns.","The PoEMV-1 algorithm should converge to that rule after at most $T-t$ policy-improvement iterations for a fixed Lagrange multiplier $w$, so the learning step is a finite procedure rather than a heuristic search.","The self-correcting update of $w$ drives terminal surplus toward the target; the reported simulation mean terminal wealth is $7.9985$ and the S&P 500 out-of-sample mean is $8.0625$, both near $d=8$.","Ignoring regime learning (PoEMV-2) leaves mean terminal wealth at $6.8491$ in simulation, well below target, so the filtered belief rather than the unconditional regime expectation is what carries performance.","The complete-information benchmark CoEMV reaches the target but with terminal-wealth variance $0.8199$ versus $0.0094$ for PoEMV-1, indicating that the paper's claimed advantage of filtering is not merely matching an omniscient policy."],"supporting_citations":[{"why":"Supplies the precommitted mean-variance formulation and the Lagrange multiplier method that the EMV objective (2.7) extends.","marker":"Li and Ng (2001)"},{"why":"Gives the multi-period regime-switching MV-ALM analytical solution that this paper extends to exploratory policies.","marker":"Chen and Yang (2011)"},{"why":"Provides the hidden Markov filtering machinery from which Lemma 3.1's filter recursion is deduced.","marker":"Elliott et al. (1997)"},{"why":"Cited for the separation principle that licenses solving the partial-information MV problem as a complete-information control problem.","marker":"Xiong and Zhou (2007)"},{"why":"Supplies the discrete-time exploratory mean-variance RL framework, Gaussian policy parameterization, and the baseline that Remark 3.2 reduces to.","marker":"Cui et al. (2023)"},{"why":"Provides the martingale-loss policy evaluation and policy gradient method used by the actor-critic algorithms.","marker":"Jia and Zhou (2022)"},{"why":"Supplies the continuous-time MV RL formulation and the universal training method used to pool stock paths in the empirical study.","marker":"Wang and Zhou (2020)"},{"why":"Provides the regime-switching RL framework, the sample-average size $N=10$, and the daily time step $1/252$ used in the experiments.","marker":"Wu and Li (2024)"}],"fun_headline_variants":["RL asset-liability policy collapses to Gaussian form","Hidden-regime RL: one Gaussian rule for portfolios","Gaussian policy emerges from regime-switching RL","Partially observable RL yields Gaussian portfolio rule","Two-regime market: RL discovers Gaussian policy"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that observing asset prices and liabilities adds no information about the next market regime beyond the current belief, so the posterior $\\hat p$ evolves by the deterministic recursion $\\hat p_{t+1}=P_{21}+\\hat p_t(P_{11}-P_{21})$ with no likelihood update; if returns and liabilities do inform the regime, the closed-form Gaussian policy of Theorem 3.2 is not the optimal policy claimed.","fun_headline_variants_meta":{"raw":{"variants":["RL asset-liability policy collapses to Gaussian form","Hidden-regime RL: one Gaussian rule for portfolios","Gaussian policy emerges from regime-switching RL","Partially observable RL yields Gaussian portfolio rule","Two-regime market: RL discovers Gaussian policy"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.00058,"raw_usage":{"total_tokens":2786,"prompt_tokens":1051,"completion_tokens":1735,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":667,"completion_tokens_details":{"reasoning_tokens":1664}},"tokens_in":667,"tokens_out":1735,"duration_ms":10708,"temperature":1.0,"reasoning_tokens":1664,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T16:32:31.507542+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simulate the paper's model while recording the true regime alongside $x_t$ and $l_t$, and at each step compute the Bayes posterior $P(\\varepsilon_{t+1}=1\\mid x_{t+1},l_{t+1},x_t,l_t)$ from the model's likelihoods. Compare it with the deterministic value $P_{21}+\\hat p_t(P_{11}-P_{21})$; if the two diverge on paths with unusually high or low excess returns — as they will whenever returns are regime-informative — the filter, the separation argument, and the claimed optimal policy all fail.","supporting_citations":[],"review_version":2}