{"id":"2b847c8f-dc04-4e9a-8136-eda3481892b1","arxiv_id":"2507.14040","paper_version":1,"verdict":"CONDITIONAL","confidence":"HIGH","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"Linear-response optimization algorithms are derived for entropy, KL divergence, and entropy production on Markov chains, with a drift-reconstruction protocol linking matrix perturbations to vector field forcing.","lead":"This paper derives linear algebra algorithms that find the small perturbation of a Markov chain which most changes entropy, Kullback-Leibler divergence, or entropy production. It also gives a data-driven recipe for turning such matrix perturbations back into vector field forcings, which could help design external forcing in climate and turbulence models without knowing the underlying equations.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The entropy-production minimizer's linear system (Eqs. 36–37) is invalid: the q_i coefficient in Eq. 34 depends on the row k, so it cannot collapse to a rank-one 1γ^T term, and the sign is also inconsistent. Equation (30) is thus unsupported for sparse Markov matrices.","rationale":"The reader's verdict was CONDITIONAL, and I agree that the paper should not be accepted as-is. However, the most load-bearing concern is not the drift-reconstruction ansatz in Eq. (43), which the authors explicitly label as a heuristic; it is the derivation of the entropy-production minimizer in Section 3.3. The linear system for the Lagrange multipliers q is obtained by replacing a row-dependent coefficient matrix Γ with a rank-one term 1γ^T, with no supporting assumption, while the sparsity constraint (C3) makes such a reduction generically false. The printed sign in Eq. (37) is also inconsistent with Eq. (36). Since the entropy-production minimization with fixed invariant measure is the genuinely new algorithmic contribution, this error directly undermines the central claim. The numerical comparison in Appendix B may indicate that a working method exists, but it does not validate the equations as written. A conditional verdict remains appropriate: the paper's ideas are promising and the numerical evidence is suggestive, but Section 3.3 must be corrected or restructured before the claimed algorithm is supported. My concern differs from the reader's named weakest assumption, though the reader's rationale did mention the rank-one reduction and sign inconsistency, so my agreement is partial.","tokens_in":23781,"tokens_out":10104,"duration_ms":113814,"concrete_test":"Take the 3-state column-stochastic matrix M = [[0.5, 0.2, 0], [0.4, 0.5, 0.6], [0.1, 0.3, 0.4]] and compute its positive invariant vector u. Form the coefficient matrix Γ from Eq. (34) with Γ_{ki} = Σ_{j: M_{kj}M_{ij}>0} u_j^2 / |Z^c_j|. Verify whether Γ is a rank-one matrix of the form 1γ^T (it will not be, because Γ has row-dependent off-diagonal entries). Then implement Eq. (30) exactly as written and compare it with the Appendix B projection method on the same matrix and on a random 50×50 sparse Markov matrix; if the two perturbations differ, the published formula is not the implemented minimizer and the derivation must be corrected.","verdict_should_be":"UNCHANGED","load_bearing_attack":"In Section 3.3, Eq. (34) contains the double sum Σ_{j:(k,j)∈Z} β_j Σ_{i:(i,j)∈Z} q_i. For a fixed k this is Σ_i Γ_{ki} q_i with Γ_{ki} = Σ_{j: M_{kj}M_{ij}>0} u_j^2 / |Z^c_j|, which depends on k through the row support {j : M_{kj}>0}. Eq. (36) replaces Γ_{ki} by a k-independent γ_i and asserts Σ_k γ_k = 1; this is only true under a full-support-type condition that contradicts the explicit sparsity constraint (C3). Moreover, Eq. (36), α_k + Σ_i γ_i q_i - ξ_k q_k = 0, implies (Ξ - 1γ^T)q = α, not (Ξ + 1γ^T)q = α as printed. Hence the Sherman-Morrison inversion and the resulting P_ij in Eq. (30) do not follow. The Appendix B comparison does not fix the written algebra unless the code solves a different system; as published, the main new optimization algorithm is not derived. Since the paper's central claim includes 'entropy production minimized ... by adding a suitable perturbation', this is a load-bearing correctness gap. The drift-reconstruction ansatz (Eq. 43) is admittedly heuristic, but the EP minimizer is the mathematical core.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper develops a linear-response framework for optimizing three functionals — Shannon entropy, Kullback-Leibler divergence, and entropy production — of the invariant measure or transition matrix of a finite Markov chain, by adding a small perturbation P subject to norm, stochasticity, and sparsity constraints. The authors derive an explicit formula (Eq. 17) for entropy optimization, a singular-value-decomposition approach for KL divergence (Section 3.2), and a Lagrange-multiplier formula (Eq. 30) for entropy-production minimization. They apply the algorithms to Ulam discretizations of the Lanford map, double-well SDEs, the Lorenz 63 system, and a periodic-orbit reduced model of Kolmogorov flow, and propose a numerical protocol to reconstruct vector-field perturbations from matrix perturbations (Eq. 43).","tokens_in":24069,"tokens_out":12907,"duration_ms":131778,"significance":"If the results are correct, the paper provides a useful toolbox for optimal forcing in data-driven dynamical systems, with potential applications in climate science and Markov chain Monte Carlo, extending the line of work by Antown et al. The numerical experiments are rich and the connection between matrix-level and vector-field-level perturbations is appealing. However, the derivation of the entropy-production optimizer contains a load-bearing algebraic gap, and the drift-reconstruction step is heuristic; both need to be addressed before the central claims can be accepted. The paper does not provide machine-checked proofs or publicly available code, but the numerical studies are described in sufficient detail to be reproducible in principle.","major_comments":[{"comment":"The reduction of Eq. (34) to the rank-one form in Eq. (36) is invalid for sparse Markov matrices. For fixed k, the double sum in Eq. (34) equals Σ_i Γ_{ki} q_i with Γ_{ki} = Σ_{j : (k,j)∈Z, (i,j)∈Z} β_j, which depends on k through the row support {j : M_{kj} > 0}; the replacement of Γ_{ki} by a k-independent γ_i is only justified under a full-support condition that contradicts the sparsity constraint (C3). In addition, Eq. (36) gives α_k + Σ_i γ_i q_i − ξ_k q_k = 0, which implies (Ξ − 1γ^T)q = α, not (Ξ + 1γ^T)q = α as written in Eq. (37); the Sherman–Morrison inversion and the closed-form expression for P_ij in Eq. (30) therefore do not follow from the displayed equations. The numerical comparison in Appendix B does not repair the written algebra unless the implemented code solves a different linear system.","section":"Section 3.3, Eqs. (34)–(38)"},{"comment":"The drift reconstruction formula (43) is called an ansatz in the text, but it is a central pillar of the paper's claim to provide a numerical link between matrix perturbations and vector-field perturbations. No error analysis is provided, and the only validation is a single one-dimensional double-well example; no dependence on box size N, transition time τ, or noise amplitude is reported, and the text itself acknowledges spiky artifacts. The authors should provide error bounds under suitable hypotheses on the Ulam discretization, or explicitly delimit the regime in which the reconstruction is quantitatively reliable. Without this, the equation-free physical interpretation of the optimized perturbations remains unsupported.","section":"Section 4.1, Eq. (43)"},{"comment":"The claimed equivalence between Method 1 (Eq. (30)) and Method 2 (projection onto the nullspace of the constraint operator) is not established by the presented numerical comparison. The histograms show only that both methods reduce the entropy-production functional relative to the unperturbed matrix; they do not compare the resulting perturbation matrices against the exact optimizer of the constrained problem, and the non-zero relative difference in Fig. 11(B) is attributed to 'numerical issues' without a convergence test as tolerances are varied. Since the main text presents Eq. (30) as the exact solution, this equivalence needs a quantitative check, such as the residual of the Lagrangian stationarity conditions.","section":"Appendix B.1, Fig. 11"}],"minor_comments":[{"comment":"In the definition of α_k, the term C_{ij} should be C_{kj} (with a fixed row index k); the same index error appears in Eq. (33).","section":"Eq. (35a) and Eq. (33)"},{"comment":"The indices in the displayed sum are inconsistent: the sum over j uses n instead of N, and the factor P_{kj} should be P_{\\ell j} to match the matrix product D^{-1}GPu.","section":"Eq. (19b)"},{"comment":"In the text and caption, the entropy-production curves are both labeled s(M+εPs); the second curve should be s(M+εPr).","section":"Section 5.3 and Fig. 5 caption"},{"comment":"The definition of G(s) uses s both as the upper limit and as the dummy summation index; it should be G(s) = Σ_{t=0}^s M^t − u1^T.","section":"Section 2, Eq. (8)"},{"comment":"The name is spelled 'Kullback-Liebler' throughout; it should be 'Kullback-Leibler'.","section":"Abstract and Section 3.2"},{"comment":"The conclusion refers to 'Eq. (60)', but that equation appears in Appendix B and is not part of the main text; the constraints should be referenced by their labels (C1)–(C4).","section":"Section 6"},{"comment":"The phrase 'can be writen' contains a typo and should be 'can be written'.","section":"Section 3.2"}],"recommendation":"major_revision","confidential_remarks":"The paper is within the scope of the journal and the topic is timely. The main concern is the validity of Section 3.3; I recommend requiring a corrected derivation, either by replacing the rank-one reduction with a full linear solve or by promoting the projection method of Appendix B to the primary derivation and demoting Eq. (30) to a numerical shortcut with a verified validity condition. The drift-reconstruction section also needs either a more cautious statement of what is proven or additional systematic validation. The numerical examples are otherwise well designed and informative."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nHere's my read on arXiv:2507.14040. The paper does three things: optimizes entropy and KL divergence for Markov-chain perturbations (mostly reusing Antown et al. 2018), derives a new entropy-production minimizer with fixed invariant vector, and proposes a heuristic to map matrix perturbations back to vector-field perturbations. The first and third are fine; the second has a real problem in the written derivation.\n\nThe entropy and KL parts are straightforward applications of the Antown framework. The paper correctly vectorizes the KL objective, uses SVD with the sparsity constraints, and demonstrates the results on Lanford, double well, and Lorenz. Those experiments look solid and the linear response checks are convincing. I'd also credit the drift reconstruction: it's explicitly called an ansatz, and the 1D validation shows it works reasonably well, spikes and all. That's honest and useful.\n\nThe soft spot is Section 3.3. The Lagrange multiplier calculation leading to Eq. (30) is not valid for sparse matrices. In Eq. (34), the double sum over j and i does not collapse to a term gamma^T q independent of k. The coefficient of q_i depends on k through the row support of row k, so the matrix is not rank-one in general. Even if it were, the sign in Eq. (37) is wrong: Eq. (36) gives (Xi - 1 gamma^T)q = alpha, not (Xi + 1 gamma^T)q = alpha. The Sherman-Morrison inversion that produces the explicit P_ij therefore does not follow. The Appendix B comparison doesn't cure this; it just shows that some numerically implemented method reduces entropy production. Possibly the code solves a different, correct system, but the paper as written doesn't derive it.\n\nThat's a load-bearing gap, because the entropy-production minimizer is one of the two genuinely new contributions. The drift reconstruction is the other, and it's honestly labeled as heuristic. So the paper's central claim—providing linear optimization algorithms for entropy production—isn't supported by the algebra as presented.\n\nI don't think this is fatal to the whole project. The numerical results in Section 5.3 suggest the method works in practice, and the framework is worth preserving. But the authors need to redo the derivation carefully, either by justifying the rank-one reduction under stated conditions or by giving the correct linear system and its solution. The sign error alone is fixable.\n\nShould we referee it? Yes, I'd send it to a serious referee if I were an editor. The paper is useful, well-motivated, and the numerical work is extensive. The referee will need to check the linear algebra carefully, but that's exactly what referees are for. A major revision with a corrected Section 3.3 could make this a solid contribution.\n\nReading group? Maybe—the flawed derivation is instructive, and the drift reconstruction is thought-provoking. I wouldn't cite it in my own work until the EP part is fixed.","headline":"The paper's entropy-production minimizer has a load-bearing algebraic error; the rest is a solid application of existing optimal-response methods, but the main new formula needs fixing before it can be trusted.","tokens_in":24612,"tokens_out":5218,"would_cite":false,"duration_ms":50856,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["37A30","37M25","60J10"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper gives linear algorithms that, for a given Markov matrix, find the perturbation of fixed norm which maximizes or minimizes entropy, Kullback-Leibler divergence, or entropy production, and a numerical recipe to read that matrix…","keywords":["Markov chain perturbation","linear response theory","entropy functional","Kullback-Leibler divergence","entropy production","Ulam method","drift reconstruction","transfer operators"],"falsifier":"Simulate a system with known drift and diffusion, build Ulam Markov matrices at several box sizes and transition times, reconstruct the drift with Eq. (43), and measure the error against the true drift; if the error does not shrink as the partition refines, the matrix-to-vector-field link fails. The same test can be run on the optimal perturbation: apply the reconstructed forcing to the original SDE and check whether the entropy or KL change matches the value predicted by the linear response formulas.","tokens_in":23517,"feed_emoji":"🎲","tokens_out":7191,"duration_ms":77070,"temperature":0.7,"pith_summary":"This paper is trying to establish that the problem of finding the external forcing that most changes a system's entropy-like statistics reduces to a linear algebra problem on a Markov matrix. Given a transition matrix $M$ with invariant vector $u$, the authors derive perturbation matrices $P$ that, to first order in the perturbation strength, maximize or minimize entropy, Kullback-Leibler divergence from the unperturbed state, and entropy production, under constraints that preserve the Markov property, the norm, and the sparsity pattern of $M$. Because such Markov matrices arise from Ulam discretizations of transfer operators, the same algorithms apply to discrete maps, stochastic differential equations, chaotic flows, and a periodic-orbit reduced model of turbulence. The paper also proposes a numerical protocol that reads the optimal matrix perturbation back as a vector-field perturbation, using the logarithm of the Markov matrix, so the forcing can be interpreted physically without knowing the underlying equations. If the claims hold, linear response theory becomes a design tool: one computes the forcing that produces a desired response in entropy-related observables instead of only predicting the response to a given forcing.","feed_headline":"Markov-matrix tweaks that force optimal entropy responses","feed_subtitle":"A small perturbation recipe maximizes or minimizes entropy, KL divergence, and entropy production in one go.","key_machinery":"The generalized inverse $G=(I_N - M + u\\mathbf{1}^\\top)^{-1}$ is the core object: it converts any admissible perturbation $P$ into the first-order change $G P u$ of the stationary measure, and every functional considered is a linear or quadratic function of this vector. The constraints $\\|P\\|_F=1$, $\\mathbf{1}^\\top P=0$, $P_{ij}=0$ where $M_{ij}=0$, and (for entropy production) $P u=0$ define the admissible set; the Lagrange multiplier solution for entropy production, Eq. (30), is the explicit minimizer. The return path from matrices to vector fields is carried by the ansatz $L=(1/\\tau)\\log M$ together with the first-moment formula $F(c_i)\\approx\\sum_j L_{ji}(c_j-c_i)$.","core_discovery":"The central claim is that for a mixing Markov matrix $M$, the first-order change in the stationary measure due to a perturbation $\\varepsilon P$ is $v_1 = G P u$, with $G=(I_N - M + u\\mathbf{1}^\\top)^{-1}$, and that entropy, KL divergence, and entropy production are respectively linear, quadratic, and linear functionals of this $v_1$. Consequently, constrained optimization over $P$ with $\\|P\\|_F=1$, row sums zero, and the sparsity mask of $M$ fixed becomes a solvable linear problem: an explicit Lagrange multiplier formula for entropy, an SVD of a Kronecker-structured matrix for KL divergence, and another explicit formula with a Sherman-Morrison step for entropy production. The same formalism extends to the change in expectation of any observable, so the paper's formulas directly apply to mixing acceleration and to energy maximization in the turbulent-flow model. The paper further claims that the optimal matrix perturbation can be translated into a drift perturbation of the underlying continuous system through $F(c_i)\\approx \\sum_j L_{ji}(c_j-c_i)$ with $L=(1/\\tau)\\log M$, and validates this reconstruction on a one-dimensional double-well potential.","pith_inferences":["A direct test the paper does not perform is to apply the reconstructed optimal forcing to the full stochastic or chaotic system and compare the measured entropy change with the linear prediction; this would separate the validity of the optimization formulas from the validity of the drift-reconstruction ansatz.","The drift-reconstruction formula uses only first moments of the flux matrix, so any inferred forcing is a drift-type forcing; it cannot recover or optimize the noise amplitude (diffusion) of the SDE, a limitation one should keep in mind in equation-free applications.","Relaxing the sparsity constraint (C3) may open genuinely new transitions; in MCMC or network settings, where there is no continuity constraint, this could give substantially better optima, and the paper's linear framework is a natural starting point for testing that.","Second-order corrections to KL divergence and entropy production would show how large $\\varepsilon$ can be before the optimal perturbation must be re-solved; a possible extension is to compare the first-order optimal P with a numerical nonlinear optimization of the exact functionals at moderate $\\varepsilon$."],"forward_implications":["For any Ulam-type Markov matrix, the small forcing that maximizes or minimizes the entropy of the stationary distribution is obtained in closed form, so no forced simulations are needed to design the forcing.","The KL maximizer gives a data-driven perturbation that makes the forced statistics maximally distinguishable from the unforced one, the quantity used in predictability or climate-change detection studies.","Minimizing entropy production under the fixed-invariant-measure constraint yields a perturbation that counteracts non-conservative forces, as shown by the reconstructed clockwise field opposing the counterclockwise rotation in the double-well-with-rotation example.","The same entropy-production formula solves other linear objectives, including accelerating mixing while keeping the invariant measure unchanged, because the spectral-gap response has the same structure (as noted in Remark 3.3).","In the periodic-orbit reduced model of turbulence, maximizing kinetic energy translates into a specific repopulation of the periodic orbits, and the opposite-sign perturbation minimizes it, since linearity makes the minimizer $-P$."],"supporting_citations":[{"why":"Supplies the Lagrange multiplier and SVD constraint framework that the paper extends to entropy, KL divergence, and entropy production.","marker":"[2]"},{"why":"Provides the perturbative expansion of the stationary distribution and the generalized inverse G used in all linear response formulas.","marker":"[52]"},{"why":"Establishes the finite-state Markov chain response operator viewpoint that connects the matrix perturbations to statistical physics linear response.","marker":"[31]"},{"why":"Shows how Ulam-type Markov matrices are used to compute linear response in continuous dynamical systems, the numerical baseline for the paper's experiments.","marker":"[48]"},{"why":"Introduces Ulam's method for discretizing transfer operators into Markov matrices, the starting point for the matrix representations.","marker":"[60]"},{"why":"Extends Ulam approximations to higher-dimensional systems and physical invariant measures, supporting the box discretizations used here.","marker":"[14]"},{"why":"Motivates maximizing KL divergence as an information-theoretic detection problem and provides the leading-order KL expansion used in Section 3.2.","marker":"[36]"},{"why":"Defines entropy production for Markov chains via the log-ratio of fluxes, the functional minimized in Section 3.3.","marker":"[51]"},{"why":"Provides the unstable periodic orbit reduced Markov model of turbulent Kolmogorov flow used in Section 5.5.","marker":"[62]"}],"fun_headline_variants":["Markov-matrix perturbations optimize entropy and entropy production","Linear recipes to tune entropy, KL divergence, and production","Data-driven matrix tweaks for optimal entropy responses","Perturb Markov chains to steer entropy and observables","Optimal entropy via matrix perturbations: a linear approach"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the logarithm of a coarse-grained Ulam Markov matrix, divided by the transition time, faithfully represents the Fokker-Planck generator through the first-moment formula (43), so the reconstructed vector-field perturbation is the true physical forcing; the paper labels this an ansatz and supplies no error analysis.","fun_headline_variants_meta":{"raw":{"variants":["Markov-matrix perturbations optimize entropy and entropy production","Linear recipes to tune entropy, KL divergence, and production","Data-driven matrix tweaks for optimal entropy responses","Perturb Markov chains to steer entropy and observables","Optimal entropy via matrix perturbations: a linear approach"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001061,"raw_usage":{"total_tokens":4447,"prompt_tokens":938,"completion_tokens":3509,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":554,"completion_tokens_details":{"reasoning_tokens":3432}},"tokens_in":554,"tokens_out":3509,"duration_ms":28387,"temperature":1.0,"reasoning_tokens":3432,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-06T16:11:49.505960+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Simulate a system with known drift and diffusion, build Ulam Markov matrices at several box sizes and transition times, reconstruct the drift with Eq. (43), and measure the error against the true drift; if the error does not shrink as the partition refines, the matrix-to-vector-field link fails. The same test can be run on the optimal perturbation: apply the reconstructed forcing to the original SDE and check whether the entropy or KL change matches the value predicted by the linear response formulas.","supporting_citations":[{"cited_title":"Network theory of microscopic and macroscopic behavior of master equation systems","cited_arxiv_id":null,"evidence_quote":"Defines entropy production for Markov chains via the log-ratio of fluxes, the functional minimized in Section 3.3."},{"cited_title":"Optimal Linear Responses for Markov Chains and Stochastically Perturbed Dynamical Systems","cited_arxiv_id":null,"evidence_quote":"Supplies the Lagrange multiplier and SVD constraint framework that the paper extends to entropy, KL divergence, and entropy production."},{"cited_title":"Perturbation theory and finite Markov chains","cited_arxiv_id":null,"evidence_quote":"Provides the perturbative expansion of the stationary distribution and the generalized inverse G used in all linear response formulas."},{"cited_title":"Response Operators for Markov Processes in a Finite State Space: Radius of Convergence and Link to the Response Theory for Axiom A Systems","cited_arxiv_id":null,"evidence_quote":"Establishes the finite-state Markov chain response operator viewpoint that connects the matrix perturbations to statistical physics linear response."},{"cited_title":"Response and Sensitivity Using Markov Chains","cited_arxiv_id":null,"evidence_quote":"Shows how Ulam-type Markov matrices are used to compute linear response in continuous dynamical systems, the numerical baseline for the paper's experiments."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Introduces Ulam's method for discretizing transfer operators into Markov matrices, the starting point for the matrix representations."},{"cited_title":"Approximating physical invariant measures of mixing dynamical systems in higher dimensions","cited_arxiv_id":null,"evidence_quote":"Extends Ulam approximations to higher-dimensional systems and physical invariant measures, supporting the box discretizations used here."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Motivates maximizing KL divergence as an information-theoretic detection problem and provides the leading-order KL expansion used in Section 3.2."},{"cited_title":"Coarse Graining the State Space of a Turbulent Flow Using Periodic Orbits","cited_arxiv_id":null,"evidence_quote":"Provides the unstable periodic orbit reduced Markov model of turbulent Kolmogorov flow used in Section 5.5."}],"review_version":1}