{"id":"ff203386-ae1e-444e-92aa-dc6ddc68ce01","arxiv_id":"2411.14481","paper_version":2,"verdict":"REJECT","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"high","formal_verification":"none","parameter_count":8,"one_line_summary":"Bank deposit rate competition is cast as a major-minor mean-field game with impulse adjustment costs, and a deep Q-network with fictitious play yields low-rate strategies in simulation.","lead":"The paper models how banks choose deposit interest rates as a game between one big bank and many small banks, where changing rates costs money. It trains a neural network to find strategies that trade market share against interest expenses, and reports a simulation in which all banks keep rates low.","discovery_kind":"new_application","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The paper never verifies the Nash property: the sole convergence evidence is a self-referential Bellman residual, and the returned policies are not tested against unilateral deviations, so the abstract's convergence and Nash-equilibrium claims are unsupported.","rationale":"The reader's REJECT verdict is warranted because the paper's central claim is a numerical convergence claim to a Nash equilibrium, and the evidence cannot discriminate between a true equilibrium and a self-consistent but non-equilibrium fixed point. I agree with the reader's weakest_assumption: minimizing a Bellman residual whose target is generated by the same network does not establish that the resulting strategies are optimal against each other. The self-referential target is the right focal point because Algorithm 1, lines 6-7, uses the identical network on both sides of the Bellman equation, and the paper supplies no theorem, no frozen-target ablation, and no unilateral-deviation check. Section 3.4 defines the correct criterion, but Section 5 never instantiates it. The proposed test, performing best-response backward induction for one player at a time, directly checks Definition 3.1 and is computationally feasible at the paper's own scale (T=5, three central-bank states, and a discretized action grid), so its absence is not a resource limitation. I do not claim the model is impossible to solve; the issue is that this manuscript does not demonstrate that its algorithm solves it. Hence the verdict should remain REJECT, unchanged from the reader.","tokens_in":14920,"tokens_out":8091,"duration_ms":86618,"concrete_test":"Fix the exact Section 5 parameters, train Algorithm 1 to completion, and freeze the output policies \\hat u^0 and \\hat u. For the major player, hold \\hat u and the induced mean-field flow fixed and solve the single-agent best-response problem by backward induction over the discretized action grid U^0 = {2.50%, 2.55%, ..., 3.50%} with T=5 and |R^c|=3; compute the value J^0(\\tilde u^0, \\hat u) of the best deviation \\tilde u^0 and compare it with J^0(\\hat u^0, \\hat u). Repeat symmetrically for the representative minor player, holding \\hat u^0 fixed and allowing only that player's own actions to deviate. If the best unilateral deviation improves the value by more than a small tolerance (e.g., 10^-3 relative), the profile is not a Nash equilibrium under Definition 3.1 and the central claim fails.","verdict_should_be":"UNCHANGED","load_bearing_attack":"To support the abstract's claim, the paper must establish that the control profile returned by Algorithm 1 satisfies Definition 3.1: no unilateral deviation by the major player, and no unilateral deviation by a representative minor player, improves discounted P&L. This is never tested. The only numerical evidence is Figure 2, which reports the training loss from Algorithm 1, lines 6-7, namely L(ω) = (1/B) Σ || Q(θ_i;ω) − S_Q(θ_i;ω) ||². The target S_Q is built from the same network Q(·;ω) being optimized, not from a delayed target network or from independent policy evaluation. A low value of this self-referential residual only shows that the network is close to a fixed point of its own Bellman operator; it does not measure whether the derived greedy policies are optimal against each other. In addition, the fictitious-play averaging rule (11) has no stated convergence theorem in this major-minor, continuous-state setting, and no implementation is provided. Therefore the claim 'converges, offering a solution ... under the Nash Equilibrium' rests on an untested fixed-point assumption rather than on evidence that the output actually satisfies the equilibrium inequalities of Section 3.4.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes a major-minor mean-field game model for interbank deposit-rate competition, incorporating impulse controls for rate adjustment costs. It formulates a discrete-time version with one major bank and a continuum of minor banks, defines a Nash equilibrium in Definition 3.1, and presents a deep Q-network algorithm with fictitious-play averaging of Q-functions by interpreting one-hidden-layer networks as measures. Numerical experiments report decreasing Bellman losses and simulated trajectories of the major bank's rate and the minor players' mean-field distribution.","tokens_in":15210,"tokens_out":12910,"duration_ms":126631,"significance":"If the algorithm really converged to a Nash equilibrium of a continuous-state major-minor impulse-control mean-field game, the paper would be a useful contribution: the application domain is novel, and the neuron-as-measure averaging trick in Eq. (10)-(11) is an elegant workaround for averaging neural-network Q-functions in fictitious play. The paper also correctly identifies that existing major-minor MFG algorithms are mostly restricted to finite state spaces. However, the current validation is not sufficient: the only convergence evidence is a self-referential Bellman loss, the Nash property is never tested against unilateral deviations, there is no baseline comparison, and no code is provided. The significance of the contribution is therefore currently a proof-of-concept rather than an established numerical method.","major_comments":[{"comment":"The abstract's convergence-to-Nash claim is not established by the evidence. The loss minimized in Algorithm 1 (lines 6-7) is L(ω) = (1/B) Σ ||Q(θ_i;ω) - S_Q(θ_i;ω)||², where S_Q is constructed from the same network Q(·;ω) via Eqs. (8)-(9); there is no target network, no independent policy evaluation, and the sampling distribution for the batches in line 5 is not specified. A small residual only shows that the network is close to a fixed point of its own Bellman operator on those batches; it does not show that the greedy policies returned in line 9 satisfy the two inequalities in Definition 3.1. No unilateral-deviation test for the major player or for a representative minor player is reported, and no convergence theorem for the fictitious-play averaging (11) is given. To support the claim, the authors should evaluate the returned policies against best responses computed with frozen opponent policies and report the resulting P&L differences, not just the training loss.","section":"§4, Algorithm 1; §5.2, Fig. 2"},{"comment":"The mean-field derivation contains notational and mathematical ambiguities that prevent verification. In Eq. (3), the terms involving minor players mix r_t, δ^{(i)}, and p_t^{(i)} even when i indexes a major bank; in Eq. (4), the integral is written as ∫ κ(...)_+ ar p_t μ^N_t(dar p_t,dr_t) although μ^N_t is already an atomic measure on (p,r), so the role of the factor ar p_t is unclear; and Eq. (5) introduces μ_t as Law(ar p_t,r_t) only after Eq. (4) has used a similar expression. Eq. (6) then states the conservation law Σ_{i=1}^M ar p^{(i)}_t + ∫ ar p_t μ_t = 1 without specifying whether μ_t is the normalized empirical measure of the rescaled minor proportions or a probability law, and without indicating how this survives the N→∞ limit. These points should be clarified or corrected; as written, the model behind the algorithm is not well-defined.","section":"§3.2, Eqs. (3)-(6)"},{"comment":"There is an internal inconsistency in the Bellman target for the major player. In Eq. (7), the next mean field is T^u_t(x^0,u^0,r^c,μ), using the current central-bank rate r^c. In Eq. (8), the same position is written as T^u_t(x^0,u^0,\\tilde r^c,A(μ)), using the next-period rate \\tilde r^c, while Eq. (9) uses r^c. Since T^u_t is defined in Section 3.3 with the current r^c and the minor controls at time t depend on the current r^c, the version in Eq. (8) changes the information structure of the trained Q-function. Please fix Eq. (8) or explain why the mean-field transition should be evaluated at \\tilde r^c.","section":"§3.3, Eq. (8)"},{"comment":"The transition kernel for the central-bank rate is not a probability kernel. The paper defines P^c(r^c_{t+1}=r^{c,i'}|r^c_t=r^{c,i}) = λΔt 1_{i=i'} + (1-λΔt)1_{i≠i'} with λΔt=0.2 and three states; summing over the three states i' gives 0.2 + 2×0.8 = 1.8 instead of 1. The intended kernel is probably (1-λΔt) on the diagonal and λΔt/(|R|-1) off-diagonal, or an equivalent, but as written the numerical experiments simulate an ill-defined stochastic process. This directly affects Figure 2 and the reported trajectories, so the experiments need to be rerun with a corrected kernel.","section":"§5.1, central bank transition"},{"comment":"The Nash equilibrium definition is incomplete. In the objective for a deviating minor player, J(u^0,u,\\hat u) = E[Σ γ^t R_t(x^0_t,u^0_t,\\hat x_t,\\hat u_t,μ_t)], but the running reward R_t was defined earlier as a function of (x_t,u_t,r^c_t,μ_t); no dynamics for the deviating state \\hat x_t are given, and the dependence on r^c_t is dropped. The major-player condition is only checked against fixed minor policies, which may be deliberate, but the paper should state the information structure and specify how \\hat x_t evolves, otherwise the equilibrium conditions that Algorithm 1 is supposed to compute are not well-posed.","section":"§3.4, Definition 3.1"}],"minor_comments":[{"comment":"The text says the minor state evolves as x_{t+1}=P(...), but the operator P is never defined; it should presumably be K as defined in the same paragraph.","section":"§3.3"},{"comment":"The grid formulas appear to have index errors: p^{(i)} = 20% + 4%·i with i∈[15] and r^{(j)} = 2.5% + 0.2%·j with j∈[5] give p starting at 24% and r starting at 2.7%, missing the stated lower endpoints 20% and 2.5%. Please check the intended indexing.","section":"§5.1"},{"comment":"The bound (12) is |b|≤4.5%, but the text immediately says 'This implies that the proportion ... will change by at most 2%'; with Δt=1 the bound should be 4.5 percentage points. The two statements should be reconciled.","section":"§5.1, Eq. (12)"},{"comment":"The caption reports standard errors 'after 10000 iterations' while the horizontal axis extends to 20000 training loops; please specify which iterations are used and whether the plotted losses are computed on the same batches used for training.","section":"§5.2, Fig. 2"},{"comment":"Several references are incompletely identified: [16], [21], [30], [36], [37], and [41] carry '[n. d.]' or lack venue and year, and [16] is cited without its publication details. This should be completed before resubmission.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"This is a borderline case. The paper has a nice modelling idea and one genuinely interesting algorithmic trick, but the central claim is currently supported only by a self-referential training loss, the central-bank transition kernel in the experiments is invalid as written, and the model equations contain ambiguities. I would be inclined to reject if the revision does not add a credible Nash verification (e.g., unilateral-deviation tests with frozen opponent policies), correct the transition kernel and rerun the experiments, and fix the mean-field derivation. The paper is written in an extended-abstract style; for a journal in mathematical finance or control, much more rigour is expected."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"The paper is worth a look for its modeling idea, but the headline claim is not backed by the evidence. It combines major-minor mean-field games, impulse control, and a deep Q-network with measure-averaged fictitious play, and applies it to bank deposit rate setting. That combination is new relative to the cited literature, and the measure-averaging trick for neural networks is a sensible workaround for the usual difficulty of averaging in fictitious play. The bank-rate application is also timely, and the model setup is explicit enough that someone could build on it.\n\nThe soft spots are real. The abstract says the algorithm converges and gives a Nash equilibrium, but there is no theorem and the numerical evidence is just a decreasing Bellman loss computed against targets generated by the same network being trained. That only shows self-consistency, not convergence to an equilibrium. The paper never checks whether either player could improve by a unilateral deviation, which is exactly what Definition 3.1 requires. The fictitious-play averaging rule in (11) also has no convergence justification in this continuous-state, major-minor setting. On top of that, the mean-field derivation in equations (3)-(5) has notational slips and ambiguities that made me re-read several times, and no code or baseline comparisons are provided.\n\nThe reader's take is fair. I agree that the convergence and Nash-equilibrium claims are unsupported as written. The stress-test note correctly identifies the missing unilateral-deviation test. Still, this is not a crank paper. The authors know the relevant literature and are attempting a real algorithmic contribution, but they overstate what they have shown. A revised version with a proper convergence theorem for the averaged Q-learning, a correction of the mean-field equations, and at least one baseline comparison (e.g., against the finite-state method of Cui et al.) could be a solid contribution.\n\nI would send this to a serious referee, but with the expectation of major revision. The core idea deserves engagement; the current claims do not. I would not cite it in its present form, though I would keep an eye on a revised version.","headline":"Novel mix of major-minor MFG, impulse control, and DQN for bank rates, but the central convergence claim rests on self-referential loss and no Nash verification.","tokens_in":15741,"tokens_out":1612,"would_cite":false,"duration_ms":18486,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["91A16","49N25","93E20"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper establishes that bank deposit-rate competition can be modeled as a major-minor mean-field game with impulsive rate changes, and that a deep Q-network algorithm with fictitious-play averaging converges to a Nash equilibrium of…","keywords":["Bank Interest Rate on Deposits","Competition in Proportion","Major-Minor Players Mean-Field Games","Impulsive Control","Fictitious Play Algorithm","Reinforcement Learning","Deep Q-Network","Nash Equilibrium"],"falsifier":"Hold the trained major and minor strategies fixed, then compute the best response of one bank by high-accuracy dynamic programming over a fine grid of states, rates, and mean-field distributions, and compare its expected discounted profit with what the trained strategy earns; if either the major bank or any minor bank can improve by more than the algorithm's own loss tolerance, the computed profile is not a Nash equilibrium.","tokens_in":1527,"feed_emoji":"🏦","tokens_out":1590,"duration_ms":74517,"temperature":0.7,"pith_summary":"The paper tries to establish that the long-standing problem of how a bank sets deposit rates while competitors respond can be framed as a game with one dominant bank and many small identical banks, and that the equilibrium strategies of that game can actually be computed. It builds a mean-field model in which the small banks are represented by a distribution, rate changes are costly and therefore impulsive, and the central bank rate provides common noise. It then argues that a deep Q-network algorithm, which averages past Q-functions in the manner of fictitious play, converges to a Nash equilibrium of this game in continuous state and action spaces. If true, this would give banks and regulators a quantitative way to see how deposit rates, market shares, and adjustment costs interact under competition.","feed_headline":"Algorithm finds Nash-equilibrium bank rates via deep Q-networks","feed_subtitle":"A deep Q-network with memory averaging handles continuous states, yielding strategies for one big bank facing many small ones.","key_machinery":"The central object is the parameterized action-value (Q) function of the major bank and of the representative minor bank, defined through the projected Bellman equations (8)--(9). A compactly supported mean-field measure is replaced by its projection onto a finite grid, so each Q-function takes a vector of grid masses as part of its input. The carrying identity is the representation of a one-hidden-layer network as an expectation under an empirical measure over parameters (equation (10)); this turns the fictitious-play moving average of neural networks into a convex combination of measures (equation (11)), which can be implemented by randomly merging neurons. The equilibrium notion is Definition 3.1: no unilateral deviation by the major bank or by an individual minor bank improves expected discounted profit, so the algorithm is searching for a fixed point of the best-response map.","core_discovery":"The central claim is that the inter-bank deposit-rate game has a Nash equilibrium in which each bank's rate policy is optimal given every other bank's policy, and that this equilibrium can be reached by a concrete iterative algorithm. The discovery is computational: a deep Q-network can parameterize action-value functions whose inputs include a projection of the mean-field measure, and the fictitious-play averaging step can be performed by averaging the parameter measures that represent the networks rather than by averaging the outputs. With this machinery the paper reports Bellman residuals below $1\\times 10^{-7}$ after about half of training, stable across repeated runs, and it reads the resulting policies under the tested parameters---all banks keep rates low and market shares nearly flat---as an approximate Nash equilibrium.","pith_inferences":["The same machinery should carry over to other pass-through settings where one dominant firm prices against a competitive fringe with adjustment costs, such as mortgage rates, deposit sweeps, or retail lending.","A sharper test of the Nash claim would be to audit the final strategies for unilateral deviations on a coarse action grid; the paper does not report such a check, so the equilibrium conclusion is only as strong as the Bellman-residual fit.","One could extend the model to heterogeneous minor banks by replacing the single representative minor with several populations, at the cost of a higher-dimensional mean-field vector.","A formal convergence guarantee would likely need a contraction or monotonicity condition; without it, the observed decay of training loss is an empirical, not a proven, convergence result."],"forward_implications":["The algorithm supplies an approximate Nash equilibrium for one major bank against many homogeneous minor banks under impulse control, in continuous state and action spaces, where tabular methods would fail.","Under the tested parameters, the equilibrium behavior is non-competitive: all banks set low deposit rates, and market shares remain roughly constant.","The measure-averaging trick makes fictitious-play averaging of neural networks implementable for major-minor mean-field games.","The framework extends to multiple major banks by increasing the number $M$ of major players in the dynamics (5)--(6), with the discrete-time experiment specialized to $M=1$.","Because the central bank rate is modeled as a jump process, the equilibrium rate policies can in principle trace how banks respond to policy-rate shocks."],"supporting_citations":[{"why":"Supplies the closest prior algorithm for learning discrete-time major-minor mean-field games with finite state spaces, which this paper extends to continuous spaces.","marker":"[16]"},{"why":"Provides the representation of shallow neural networks as measures that the paper uses to average Q-functions in fictitious play.","marker":"[14]"},{"why":"Identifies the difficulty of averaging network outputs in mean-field reinforcement learning, motivating the measure-averaging technique.","marker":"[30]"},{"why":"Establishes fictitious play for mean-field games, the update scheme that Algorithm 1 adapts.","marker":"[37]"},{"why":"Connects impulse control problems to quasi-variational inequalities, giving theoretical grounding for the impulsive rate-change model.","marker":"[38]"},{"why":"Formulates impulse control and jump-diffusion dynamics, used for the central bank rate process and the control setup.","marker":"[35]"},{"why":"Introduces the deep Q-network method that the paper's algorithm builds on.","marker":"[32]"},{"why":"Provides the probabilistic theory of mean-field games and common noise, underpinning the major-minor formulation.","marker":"[11]"}],"fun_headline_variants":["Deep Q-learning finds Nash equilibrium for bank rates","Major-minor bank game solved by deep Q-networks","Impulse control mean-field game finds bank rate equilibrium","Deep Q-network balances bank rates via Nash equilibrium"],"cache_read_input_tokens":17792,"weakest_assumption_plain":"The algorithm's claim of reaching a Nash equilibrium rests on the premise that pushing the Q-networks' prediction error to a tiny value against targets produced by the same networks yields the true action-value functions of the game; the experiments only show that this training loss decreases, never whether any player could profitably deviate from the final strategies.","fun_headline_variants_meta":{"raw":{"variants":["Deep Q-learning finds Nash equilibrium for bank rates","Major-minor bank game solved by deep Q-networks","Impulse control mean-field game finds bank rate equilibrium","Deep Q-network balances bank rates via Nash equilibrium"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000576,"raw_usage":{"total_tokens":2671,"prompt_tokens":848,"completion_tokens":1823,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":464,"completion_tokens_details":{"reasoning_tokens":1761}},"tokens_in":464,"tokens_out":1823,"duration_ms":13327,"temperature":1.0,"reasoning_tokens":1761,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T17:14:44.517570+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Hold the trained major and minor strategies fixed, then compute the best response of one bank by high-accuracy dynamic programming over a fine grid of states, rates, and mean-field distributions, and compare its expected discounted profit with what the trained strategy earns; if either the major bank or any minor bank can improve by more than the algorithm's own loss tolerance, the computed profile is not a Nash equilibrium.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the closest prior algorithm for learning discrete-time major-minor mean-field games with finite state spaces, which this paper extends to continuous spaces."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the representation of shallow neural networks as measures that the paper uses to average Q-functions in fictitious play."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Identifies the difficulty of averaging network outputs in mean-field reinforcement learning, motivating the measure-averaging technique."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Establishes fictitious play for mean-field games, the update scheme that Algorithm 1 adapts."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Connects impulse control problems to quasi-variational inequalities, giving theoretical grounding for the impulsive rate-change model."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Formulates impulse control and jump-diffusion dynamics, used for the central bank rate process and the control setup."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Provides the probabilistic theory of mean-field games and common noise, underpinning the major-minor formulation."}],"review_version":1}