{"id":"5532d66c-5a24-4a0f-99a1-5c8b656cc8f6","arxiv_id":"2505.06419","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"Gradient descent on randomly initialized matrix product states gets stuck in a causal trap that ignores boundary correlations, but natural gradient descent or a TTNS-Sketch warm start avoids the trap.","lead":"This paper shows that training random matrix product states with gradient descent fails to capture long-range correlations in periodic spin models, and proposes two fixes: a natural gradient method and a smart initialization from a sketching algorithm. The work matters for quantum-inspired generative modeling and quantum state tomography, where such training failures are common.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The GD 'causality trap' is inferred from flat NLL curves over 1000 iterations, but the paper never shows the trained model is at a local minimum; the plateau could be a saddle or a slow region, making the central failure claim premature.","rationale":"The reader's flagged weak assumption concerns whether the aggregate NLL/MI diagnostics in Appendix A uniquely identify pcausal. That concern is partially mitigated by Figure 4, which directly computes TV distance for n=16 and shows the trained model is closer to pcausal than to p*; thus the empirical identification of the trap does not rely solely on the approximate diagnostics. A more fundamental gap is that the paper never demonstrates that the observed plateau is a true local minimum of the GD dynamics. The authors assert 'local minima issues' (Sections II and III) and interpret the flat NLL curves as convergence to a suboptimal model, but they report no gradient norm, no Hessian information, no convergence test, and no long-run experiments. The toy model in Appendix B explicitly shows that GD on a multilinear MPS can have exponentially small effective learning rates, so flatness over 1000 iterations is exactly what one would expect far from a critical point. If so, the 'causality trap' may be an artifact of the optimization horizon, and the conclusion that randomly initialized BMs are unreliable for periodic distributions is premature. NGD escaping from the same point is not evidence of a local minimum, because NGD follows a different gradient flow. The concrete test—longer runs and gradient-norm monitoring—would settle whether the plateau is a stable attractor of GD. This is a single, clean check that directly tests the central claim, and it should be added before the conclusions are fully accepted.","tokens_in":15341,"tokens_out":11130,"duration_ms":116134,"concrete_test":"Re-run the Section IIB experiment (n=16, beta=1, rmax=20) for 10^5 gradient steps with several learning rates and random seeds, recording the gradient Frobenius norm and the NLL gap at the final parameters. If the gap drops below 0.33 or the gradient norm remains substantially above zero at the plateau, the 'causality trap' is an artifact of the optimization horizon and the local-minimum interpretation fails.","verdict_should_be":"CONDITIONAL","load_bearing_attack":"The central negative result (Sections IIB, IIIB) is that randomly initialized MPS models trained by gradient descent converge to suboptimal models—specifically, in the Born machine case, to pcausal, the cycle-Ising model with the boundary edge removed (Eq. 5). The evidence is Figures 3 and 4: after 1000 iterations the NLL plateau is approximately 0.33 above the global minimum and the trained model is closer to pcausal in TV distance. But the paper never establishes that this plateau is a local minimum. It reports no gradient norms, no Hessian information, no convergence criterion, and no longer runs. This matters because the paper's own Appendix B shows a mechanism by which GD on an MPS can have exponentially vanishing effective learning rates: in the toy model, the GD update multiplies the gradient by (product of other coefficients)^2, which can be exponentially small. In such a regime, a flat NLL curve over 1000 iterations is consistent with the model being far from any critical point, moving at an exponentially slow speed, rather than having converged to a local minimum. If the plateau is transient, the 'causality trap' is a misdiagnosis of a poorly conditioned optimization trajectory, and the conclusion that randomly initialized BMs are unreliable for periodic spin distributions would not be supported. The existence of a local minimum is asserted (e.g., 'local minima issue' in Sections II and III) but never verified. The NGD escape from the same plateau does not distinguish a local minimum from a saddle, since NGD changes the optimization dynamics.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper studies training of matrix product state (MPS) probabilistic models, both Born machines (BM) and general MPS quantum state tomography (QST). It reports that gradient descent (GD) from random initialization gets stuck in suboptimal solutions, specifically a 'causality trap' in which the trained BM approximates a path-graph Ising model that omits the boundary edge of the true cycle graph. The paper proposes two fixes: a natural gradient descent (NGD) method derived from a projected gradient flow on the MPS manifold, and a warm-start initialization based on the TTNS-Sketch density estimator. Numerical experiments on periodic Ising and Heisenberg models show that GD fails while NGD, especially with line search, converges in very few iterations, and that warm-started GD avoids the causality trap even with few sketching samples.","tokens_in":15636,"tokens_out":6718,"duration_ms":69567,"significance":"If the reported failure mode is real, the paper identifies a practically relevant pathology of MPS-based generative models and offers two actionable remedies, one of which (NGD) is derived cleanly from a gradient-flow perspective and connects to the TDVP literature. The KL decomposition in Appendix A and the NGD derivation in Appendix C are mathematically sound, and the warm-start strategy leverages a published sample-complexity guarantee for TTNS-Sketch. However, the central negative claim depends on interpreting flat NLL curves as convergence to local minima, and the empirical support consists entirely of single-run experiments without convergence diagnostics or error bars. With appropriate strengthening, the manuscript would be a useful contribution to the numerical analysis of tensor-network training.","major_comments":[{"comment":"The claim that GD converges to a suboptimal local minimum (or to the causality trap) is not substantiated because no convergence diagnostics are reported. The flat NLL curves after 1000 iterations are interpreted as evidence of a local minimum, yet the paper provides no gradient norms, no Hessian information, no convergence criterion, and no longer runs. This matters directly because Appendix B shows that GD on a multilinear MPS ansatz can have an effective learning rate multiplied by (prod_{k≠i} c_k)^2, which can be exponentially small; a plateau over 1000 iterations is fully consistent with an exponentially slow transient rather than a critical point. Please report gradient-norm trajectories, continue until a first-order optimality condition is met, or otherwise demonstrate that the iterates are at a local minimum. Without this, the 'causality trap' as a convergence phenomenon is not established.","section":"Section II B, Figures 3-5"},{"comment":"The proposed diagnostic for the causality trap is logically incomplete. The conditions D_w|z(pθ)≈0 and D_z(pθ)≈I_{X~p*}(X1;Xn) are claimed to be verifiable from the NLL gap and the mutual information I_{pθ}(X1;Xn) being near zero. However, from DKL(p*||pθ) = D_z(pθ)+D_w|z(pθ) and small I_{pθ}(X1;Xn), one cannot conclude D_z(pθ)≈I_{p*}(X1;Xn): pθ_{x1,xn} could be independent yet have marginals that differ substantially from p*_{x1,xn}, making D_z(pθ) arbitrarily different from the mutual information of p*. The paper uses this diagnostic to classify finite-sample training runs as trapped, so the inference needs to be justified, e.g., by directly estimating the conditional KL terms or by showing that the trained marginal pθ_{x1,xn} is close to that of pcausal.","section":"Appendix A, Eqs. (A1)-(A2)"},{"comment":"The headline quantitative claim that 'NGD with line search can converge to the global minimum in only a few iterations' is supported only by single-run experiments with no error bars, no repeated initializations, and no stated stopping tolerance. Given that the paper's central contrast is between GD failure and NGD success, it is important to show that the results are robust across random initializations and over a reasonable range of the learning rate η and line-search settings. Please provide multiple runs (e.g., mean and standard deviation over seeds) or at least report the specific hyperparameters and tolerances used.","section":"Section IV C, Figures 6-8"},{"comment":"The proof of Proposition 1 uses Π_i = M(M^T M)^{-1} M^T, which requires M to have full column rank. This is not guaranteed for arbitrary tensor components, and the paper itself notes that trained MPS models exhibit rank degeneracy (Section II B). At a rank-deficient M, the NGD step is undefined by this expression. Please either state a regularity assumption that is maintained by the algorithm (e.g., via gauge fixing or truncation) or replace the inverse with a pseudo-inverse and discuss the rank-deficient case.","section":"Appendix C, Eq. (C1)"}],"minor_comments":[{"comment":"The text says 'Theorem 1 is directly related' and refers to 'Theorem 1' twice, but no Theorem 1 is stated; the statement appears to be Proposition 1. Please renumber or cross-reference consistently.","section":"Section IV B"},{"comment":"Appendix E refers to 'Theorem 2' and 'Theorem 3' without having introduced them in the main text; the main text only states Proposition 2. Please align the numbering between the main text and the appendix.","section":"Appendix E"},{"comment":"The statement that 'running Algorithm 1 has only a time complexity of O(n)' is ambiguous: it is unclear whether this is the cost per site update, per full sweep, or per iteration, and the dependence on the bond dimension r is omitted. Please clarify the complexity statement.","section":"Section IV A"},{"comment":"The experimental sections do not report the learning rate η used for GD, NGD, or the 2-site DMRG method, nor do they discuss sensitivity to this hyperparameter. Reporting these values would improve reproducibility.","section":"General"},{"comment":"In Figure 3(b), the y-axis is on a logarithmic scale with values down to 10^{-14}; the caption does not mention the log scale. Please note this in the caption for readability.","section":"Figure 3"}],"recommendation":"major_revision","confidential_remarks":"The paper has a clean core derivation (Proposition 1, KL decomposition) and a sensible proposed remedy, but the central negative result rests on convergence claims that are not verified. The 'causality trap' diagnosis also has a logical gap in Appendix A. These are fixable with additional experiments and a revised inference argument, so I recommend major revision rather than rejection. The authors should also be encouraged to temper the 'converges in only a few iterations' phrasing unless they provide statistical support."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Colleague, here's my read. The paper's real contribution is the causality trap: a named, analyzable failure mode in which GD-trained Born machines on periodic Ising data collapse to a path model that drops the boundary edge. The KL decomposition in Appendix A makes that diagnosis clear, and the TTNS-Sketch warm start is a sensible use of prior sample-complexity guarantees. I also want to credit them for Appendix D, where they state plainly that NGD is equivalent to 1-site DMRG under mixed canonical gauge. That disclosure means the optimization half of the paper is a reframing, not a new algorithm.\n\nThe soft spots are real but not fatal. The stress-test note has it right: the central negative claim is read off flat NLL curves over 1000 iterations, with no gradient norms, no evidence the model is at a stationary point, and no long runs. Their own toy example in Appendix B shows GD on MPS can have exponentially small effective learning rates. A flat curve is then consistent with a slow region far from a critical point. So 'converges to a local minimum' is not actually demonstrated. That matters because the causality trap is the paper's headline finding. I would soften the claim to 'GD gets stuck or moves extremely slowly near a non-global model' until they show gradient norms or run longer.\n\nSecond, everything rests on three models and single runs, no error bars, no code. The abstract's 'converge to the global minimum in only a few iterations' is stronger than what the figures show. Third, the NGD equivalence cuts both ways: they disclose it, but it means the demonstrated payoff is a comparison with 2-site DMRG, not a new optimizer family.\n\nWho's this for? Anyone doing Born machine training or MPS tomography will want the causality trap characterization and the warm-start trick. The math derivations are clean and the appendix is honest. I'd send it to review, but I'd require convergence diagnostics and ideally code or at least error bars before accepting the failure claim. Worth a serious referee.","headline":"Useful diagnosis of a real Born-machine training failure, but the 'trap' claim needs longer runs and convergence evidence; the optimization remedy is honestly reframed as 1-site DMRG.","tokens_in":16214,"tokens_out":1610,"would_cite":true,"duration_ms":15922,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["65K10"],"pacs":[],"model":"deepseek-v4-flash","headline":"Born machines trained by gradient descent converge to a causal model that omits the boundary interaction; natural-gradient descent and a sketch-based warm start both escape this trap.","keywords":["Born machine","matrix product state","causality trap","natural gradient descent","quantum state tomography","TTNS-Sketch","tensor network generative models","negative log-likelihood"],"falsifier":"Train a randomly initialized Born machine with gradient descent on the periodic Ising distribution with $n=16$, $\\beta=1$, and $|T|=2^{15}$; if the trained model's total variation distance to $p^\\star$ is smaller than its distance to $p_{\\mathrm{causal}}$, or if the NLL gap is not approximately $I(X_1;X_n)\\approx 0.33$, the causality-trap claim fails. A finite-$n$ version with all $2^n$ configurations enumerated would settle the same question exactly without sampling noise.","tokens_in":15109,"feed_emoji":"⚛️","tokens_out":7489,"duration_ms":69246,"temperature":0.7,"pith_summary":"The paper argues that randomly initialized matrix product state (MPS) models trained by gradient descent can fail at a basic probabilistic modeling task: learning a periodic spin distribution. In the Born machine setting, the trained model converges not to the true distribution $p^\\star\\propto\\exp(-\\beta\\sum_{(i,j)\\in\\mathrm{cycle}(n)}x_ix_j)$ but to a causal model $p_{\\mathrm{causal}}\\propto\\exp(-\\sum_{(i,j)\\in\\mathrm{path}(n)}x_ix_j)$ that omits the boundary edge between sites 1 and $n$, and this failure persists at internal rank 20. The paper identifies this as a causality trap, gives a diagnostic for detecting it from the negative log-likelihood gap and the mutual information of the boundary pair, and proposes two cures: a natural gradient descent method that approximates projected gradient flow on the tensor manifold, and a TTNS-Sketch warm initialization. If correct, this means standard gradient training of MPS-based generative models is unreliable for periodic or long-range-correlated targets, and that the failure is curable by the right optimization geometry or a good starting point.","feed_headline":"Gradient descent makes Born machines forget the periodic edge","feed_subtitle":"With natural-gradient training or a warm start, MPS models keep the boundary correlation.","key_machinery":"The load-bearing object is the causality-trap diagnostic: a trained Born machine is classified as trapped when its NLL gap over the optimum equals the mutual information $I(X_1;X_n)$ of the target distribution and the mutual information of the same pair under the model is near zero, conditions equivalent to $D_{w|z}(p_\\theta)\\approx 0$. The paper's cure is a natural gradient step whose curvature term measures change in the tensor $q_\\theta$ rather than in the parameters, which makes the update an approximate projection of $\\nabla_q F$ onto the tangent space of one tensor component; Algorithm 1 sweeps sites forward and backward and is equivalent to gradient descent in a mixed canonical form, i.e. a 1-site DMRG update. The warm start uses the TTNS-Sketch algorithm to obtain a direct MPS density estimate $\\hat p_{TS}$, takes its positive square root, interpolates it by TT-cross, and fits the resulting MPS to the Born machine ansatz.","core_discovery":"The central claim is that the training dynamics of a Born machine under gradient descent on a periodic Ising target converge to the wrong model class: the learned density is close to $p_{\\mathrm{causal}}$ in total variation, not to $p^\\star$. The paper characterizes the trap through the decomposition $D_{\\mathrm{KL}}(p^\\star\\|p)=D_z(p)+D_{w|z}(p)$ with $z=(x_1,x_n)$ and $w=(x_2,\\ldots,x_{n-1})$; trained models satisfy $D_{w|z}(p_\\theta)\\approx 0$ while $D_z(p_\\theta)\\approx I_{X\\sim p^\\star}(X_1;X_n)$, so the NLL gap equals the boundary mutual information. The same local-minimum phenomenon appears in MPS quantum state tomography on a periodic transverse-field Ising ground state. Natural gradient descent, defined as the site-wise minimization of $\\langle\\nabla_\\theta L,\\delta\\theta\\rangle + \\tfrac{1}{2\\eta}\\|q_{\\theta_t+\\delta\\theta}-q_{\\theta_t}\\|_F^2$, is shown to be a discretization of a projected gradient flow and reaches the global minimum in both settings, with line search needing only a few iterations; warm-starting gradient descent with a TTNS-Sketch density estimate avoids the trap even when that estimate uses only 128 samples.","pith_inferences":["If the causality-trap diagnosis is as sharp as the paper suggests, the same NLL-gap-equals-mutual-information test could be used as a stopping or rejection criterion in any MPS training run, flagging models whose boundary correlations are unreliable before expensive total-variation checks.","The trap is formulated for the cycle graph, but the mechanism—a chain-like approximation that drops the closing edge—is plausible for any high-dimensional distribution with one strong long-range or non-local interaction; testing on other graphical models with a single non-local bond would show whether the phenomenon is generic.","The warm-start result hints that a cheap direct-MPS density estimate plus local fine-tuning could replace expensive random-restart training across tensor-network generative models, but extending it to general quantum state tomography needs a density-estimation subroutine for observables other than computational-basis samples, which the paper leaves open."],"forward_implications":["For periodic spin targets, a randomly initialized Born machine trained by gradient descent should not be assumed to capture long-range boundary correlations; the NLL gap and the boundary-pair mutual information are enough to tell whether it has fallen into the trap.","Natural gradient descent, implemented as single-tensor-component updates with a function-space curvature term, reaches the global minimum in the Born machine and tomography experiments, and with line search does so within a few iterations, so the local-minimum problem is an artifact of the optimization geometry, not of the MPS ansatz.","Over-parameterization alone does not cure the trap: increasing $r_{\\max}$ from 2 to 20 leaves the trained model at the causal NLL level.","A warm initialization obtained from a cheap density estimator, even one fed only 128 samples, is enough for gradient descent to escape the trap, so initialization quality matters more than sample-hungry pretraining.","Because the natural gradient derivation is not tied to gauge choices, the same update rule extends to tensor network families where gauge transformations are unavailable, such as positive MPS and locally purified states."],"supporting_citations":[{"why":"introduces the Born machine maximum-likelihood training scheme whose gradient-based local minima and causality trap are the paper's main object of study","marker":"[9]"},{"why":"provides the Born machine MPS ansatz, the positivity-preserving structure, and the implementation used in the gradient descent experiments","marker":"[14]"},{"why":"supplies the TTNS-Sketch algorithm with sample-complexity guarantees that underlies the warm-start initialization and Proposition 2","marker":"[11]"},{"why":"defines DMRG, the MPS optimization template from which the compared 2-site method and the mixed-canonical NGD update both derive","marker":"[13]"},{"why":"provides the TT-cross interpolation routine used to fit the square root of the sketched density into an MPS for warm starting","marker":"[36]"},{"why":"the time-dependent variational principle whose projected gradient-flow picture the natural gradient method is shown to discretize","marker":"[34]"}],"fun_headline_variants":["Natural gradient escapes the Born machine causality trap","Warm start or natural gradient: fix for MPS training failure","Why gradient descent misses boundary correlations in MPS models","Causality trap in Born machines: two fixes from optimization and initialization","Periodic spin chain MPS training: avoid the trap with better updates"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The whole diagnosis rests on the assumption that two easy-to-measure aggregate statistics, the NLL gap and the mutual information of the boundary pair, are a faithful stand-in for the harder structural condition that the model has learned the wrong causal graph; that equivalence is proven only in the infinite-sample limit and is approximate, yet it is used to classify the finite-sample runs.","fun_headline_variants_meta":{"raw":{"variants":["Natural gradient escapes the Born machine causality trap","Warm start or natural gradient: fix for MPS training failure","Why gradient descent misses boundary correlations in MPS models","Causality trap in Born machines: two fixes from optimization and initialization","Periodic spin chain MPS training: avoid the trap with better updates"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000306,"raw_usage":{"total_tokens":1803,"prompt_tokens":1046,"completion_tokens":757,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":662,"completion_tokens_details":{"reasoning_tokens":686}},"tokens_in":662,"tokens_out":757,"duration_ms":6587,"temperature":1.0,"reasoning_tokens":686,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-15T22:42:40.838176+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Train a randomly initialized Born machine with gradient descent on the periodic Ising distribution with $n=16$, $\\beta=1$, and $|T|=2^{15}$; if the trained model's total variation distance to $p^\\star$ is smaller than its distance to $p_{\\mathrm{causal}}$, or if the NLL gap is not approximately $I(X_1;X_n)\\approx 0.33$, the causality-trap claim fails. A finite-$n$ version with all $2^n$ configurations enumerated would settle the same question exactly without sampling noise.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"introduces the Born machine maximum-likelihood training scheme whose gradient-based local minima and causality trap are the paper's main object of study"},{"cited_title":"Glasser, R","cited_arxiv_id":null,"evidence_quote":"provides the Born machine MPS ansatz, the positivity-preserving structure, and the implementation used in the gradient descent experiments"},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"supplies the TTNS-Sketch algorithm with sample-complexity guarantees that underlies the warm-start initialization and Proposition 2"},{"cited_title":"Oseledets and E","cited_arxiv_id":null,"evidence_quote":"provides the TT-cross interpolation routine used to fit the square root of the sketched density into an MPS for warm starting"}],"review_version":1}