{"id":"2a76f38f-b4c4-4b0a-a0b3-1cedb85c6bf3","arxiv_id":"1909.02659","paper_version":3,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":0,"one_line_summary":"A new backpropagation formula for complex-valued SVD is derived, with the key contribution a novel diagonal term absent from the real SVD case.","lead":"The authors derive the reverse-mode automatic differentiation (backpropagation) formula for complex-valued SVD, adding a new diagonal term that has no real-valued counterpart. The formula fills a gap needed to use automatic differentiation on complex tensor networks and complex linear algebra.","discovery_kind":"first_principles","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Repeated or zero singular values make Eq. (1) undefined; the paper never states this, and Eq. (13) is already inconsistent at degeneracy, so the unqualified central claim is too broad.","rationale":"The reader's weakest assumption identifies the same load-bearing restriction: distinct and nonzero singular values are silently required. My independent check of the degenerate case confirms that this is not just a numerical precision issue; the underlying perturbation equation (13) is algebraically inconsistent for a generic perturbation at a repeated singular value, so the formula genuinely has no value there. The derivation is otherwise coherent for the nondegenerate case, and it matches manual checks for simple gauge-invariant losses such as Im(A_00) and Re(A_00). There is no evidence of a sign or normalization error in the claimed new term. The paper does lack a numerical gradient check and does not carefully delimit against references [7] and [8], but those are secondary. Because the defect is a missing domain restriction rather than a contradiction in the main nondegenerate result, the conditional verdict from the reader is appropriate; I would not move to reject or accept without the authors stating and testing the degeneracy caveat.","tokens_in":4679,"tokens_out":32393,"duration_ms":330988,"concrete_test":"Analytical check: set A = diag(1,1,2), U = V = I, and take an infinitesimal perturbation dA with dA_01 = 1 and dA_10 = 0. Insert dP = dA into Eq. (13). The (0,1) and (1,0) entries force dC_01 - dD_01 = 1 and dC_10 - dD_10 = 0. Using skew-Hermiticity dC_10 = -conj(dC_01), dD_10 = -conj(dD_01), the second equation becomes -conj(dC_01) + conj(dD_01) = 0, which implies dC_01 - dD_01 = 0, contradicting the first equation. This shows Eq. (13) is unsolvable at a repeated singular value, so Eq. (1) cannot be an unqualified backpropagation formula. If the authors intend a restricted domain, the test at a nearby nondegenerate matrix, e.g. diag(1+epsilon,1,2) with epsilon > 0, should reproduce the finite-difference gradient to order epsilon; the exact epsilon = 0 case will remain singular.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that Eq. (1) is the reverse-mode AD formula for complex SVD is only valid for matrices with distinct, nonzero singular values, and this restriction is never stated. The formula itself contains F_ij = 1/(s_j^2 - s_i^2) and S^{-1}, so it is undefined when any two singular values coincide or any singular value is zero. The failure is not merely a removable numerical artifact: at a repeated singular value the derivation's own perturbation equation breaks algebraically. Take A = diag(1,1,2), U = V = I, and an infinitesimal off-diagonal perturbation dA with dA_01 = 1 and dA_10 = 0. Then dP = U^T dA V = dA. Equation (13), dP = dC S + dS - S dD, requires dP_01 = dC_01 - dD_01 = 1 and dP_10 = dC_10 - dD_10 = 0. Since dC and dD are skew-Hermitian, dC_10 = -conj(dC_01) and dD_10 = -conj(dD_01), so the second equation is -conj(dC_01) + conj(dD_01) = 0, i.e. conj(dP_01) = 0, contradicting dP_01 = 1. Thus no first-order SVD factors exist for a generic perturbation at a repeated singular value, and Eq. (1) has no well-defined value. The paper's proof also only treats diagonal-phase gauge invariance, Eq. (31), whereas degeneracy enlarges the gauge freedom to continuous unitary rotations in the degenerate subspace. Consequently, the claim of a complete AD formula for complex SVD is conditional on a domain restriction that the manuscript never articulates.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper derives a reverse-mode backpropagation formula for the complex-valued singular value decomposition (SVD), for the case of a real, gauge-invariant scalar loss function L depending on the SVD factors. The main result, Eq. (1), expresses the conjugate gradient of L with respect to the input matrix A in terms of U, S, V and the reverse-mode signals for U, S, V. The first three terms parallel the real SVD derivation; the new contribution, Eq. (2), is a term proportional to U S^{-1}(L^\\dagger - L) V^\\dagger that arises from the imaginary diagonal parts of dC = U^\\dagger dU and dD = V^\\dagger dV. The derivation is a direct chain-rule computation using the gauge invariance of the loss under diagonal phase transformations, and the paper also sketches the rectangular extension via two additional terms.","tokens_in":5024,"tokens_out":10576,"duration_ms":99050,"significance":"If valid, the result fills a genuine gap in complex-valued automatic differentiation, with direct applications to tensor-network algorithms where complex SVD is a primitive operation. The paper is explicit and does not rely on fitted parameters or curve fitting: the missing term is obtained from the stated gauge-invariance condition, and the derivation is internally consistent under the implicit assumptions of distinct nonzero singular values. A GitHub implementation and a TensorFlow merge are mentioned, which strengthens the practical relevance. The main caveat is that the domain of validity is narrower than the unqualified statement of the result, and the paper would benefit from stating that restriction clearly.","major_comments":[{"comment":"The central formula is stated without the domain restriction required by its own ingredients: F_{ij}=1/(s_j^2-s_i^2) and S^{-1} are undefined when any two singular values coincide or when any singular value is zero. This is not merely a numerical caveat; the derivation's perturbation equation (13) has no first-order solution at a repeated singular value. For example, with A=diag(1,1,2), U=V=I, and an infinitesimal dA with dA_{01}=1 and dA_{10}=0, Eq. (13) gives dP_{01}=dC_{01}-dD_{01}=1 and dP_{10}=-conj(dC_{01})+conj(dD_{01})=0, which are contradictory because dC and dD are skew-Hermitian. The paper must state explicitly that Eq. (1) is valid only for distinct, nonzero singular values, and it should discuss the near-degenerate case where the gradient is large or undefined.","section":"Results, Eq. (1)"},{"comment":"The claimed extension to rectangular matrices is not fully specified. In a full SVD of an m by n matrix A, the matrix S is m by n and S^{-1} is undefined; the extension (36) is only meaningful if one adopts a thin SVD in which S is square, or if S^{-1} is defined as a pseudo-inverse. The paper should state the dimensions of U, S, V in Eq. (36) and justify how the square formula (1) is applied in that case.","section":"Discussions, Eq. (36)"},{"comment":"The notation in Eq. (7) appears to identify \\bar{A}=∂L/∂A with \\bar{A}=2∂L/∂A^* = ∇_A L, but these are different objects. The derivation up to Eq. (35) is carried out for derivatives, while the final formula (1) is stated for gradients, and the conversion is only alluded to with 'Note how J,K,L changed accordingly'. Please write out this conversion explicitly so that the relation between Eq. (35) and Eq. (1) can be verified directly.","section":"Notation, Eqs. (7), (35), and (1)"},{"comment":"The gauge-invariance condition (31) is derived for diagonal phase transformations Λ only. If degenerate singular values were included in the domain, the gauge group would be larger (unitary rotations mixing the degenerate subspace), so Eq. (31) would not be the full condition. Since the formula is only valid for simple singular values, this issue is already covered by the domain restriction, but the paper should state that the derivation assumes simple singular values and that the diagonal-phase gauge is the only relevant gauge freedom in that case.","section":"Gauge freedom, Eq. (31)"}],"minor_comments":[{"comment":"There are several typos: 'Namley' after Eq. (21), 'imiginary' after Eq. (12), and 'Beijin g' in the affiliation. These should be corrected.","section":"Throughout"},{"comment":"The symbol L is overloaded: it denotes the loss function and also the matrix I∘(V^† \\bar{V}) in the definition after Eq. (1). Rename the matrix (e.g., M) to avoid confusion.","section":"Eq. (1)"},{"comment":"The paper would benefit from a short numerical verification, such as a finite-difference check of Eq. (1) on a small non-diagonal example with a gauge-invariant loss depending on both U and V; this would guard against sign and transpose convention errors.","section":"Abstract and Introduction"}],"recommendation":"major_revision","confidential_remarks":"The core derivation appears sound under the distinct-nonzero-singular-values assumption, and the result is publishable if the domain restriction is stated and the derivative-to-gradient conversion is clarified. The main reason for major revision rather than rejection is that the unqualified central claim is too broad and the rectangular extension is underspecified; both are fixable within the scope of the paper."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Dear colleague,\n\nThis note gives the reverse-mode AD formula for complex-valued SVD, with one genuinely new term beyond the real case. The derivation is explicit chain-rule work, the gauge-invariance handling is clear, and they ship a TensorFlow implementation that was merged upstream. The math in the paper is right for what it actually proves.\n\nThe soft spot is that the formula is only defined for matrices with distinct, nonzero singular values, and the paper never says that. Eq. (1) contains F_ij = 1/(s_j^2 - s_i^2) and S^{-1}; the derivation's own perturbation equation (13) has no solution for a generic direction at a repeated singular value, as a simple 2x2 counterexample shows. So the claim “the backprop formula for complex valued SVD” is too broad; it should be “for complex SVD with distinct nonzero singular values.” This is not numerical pedantry: in tensor networks degenerate spectra do occur, and users need to know the gradient is undefined there.\n\nA second, smaller issue: the paper doesn't explicitly say whether ref [8] already contains the term. They claim it's missing, but a sentence auditing [8] would close that gap. Also, the gauge-invariance argument only treats diagonal phase freedom; in degenerate subspaces the gauge group is larger, which reinforces the same caveat.\n\nThese are fixable with a stated restriction and a short discussion. I don't see a load-bearing flaw in the math itself; the four-term formula follows from the chain rule under the stated assumptions. The paper would benefit from a numerical gradient check, but the derivation is clean enough that I don't consider its absence disqualifying.\n\nThis is for anyone doing AD through complex SVD, especially tensor-network people. It's a small but needed primitive. As a referee I would ask for the degeneracy caveat and the [8] audit, then accept. If those are added, I'd cite it. Yes, send it to peer review.","headline":"A correct and useful complex-SVD backprop formula, but the paper overstates its domain by omitting the distinct-nonzero-singular-value condition.","tokens_in":5554,"tokens_out":2091,"would_cite":true,"duration_ms":20944,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["15A18","65F15"],"pacs":[],"model":"deepseek-v4-flash","headline":"This paper derives the reverse-mode automatic differentiation formula for complex-valued SVD, adding a fourth term with no real-valued counterpart that couples U and V in the gradient.","keywords":["automatic differentiation","complex SVD","reverse mode","backpropagation","tensor networks","Wirtinger calculus","singular value decomposition","gradient"],"falsifier":"Take a random 2x2 complex matrix with two distinct nonzero singular values and the gauge-invariant loss L = Re(U00 V*_00), compute the central finite-difference gradient of L with respect to the real and imaginary parts of A, and compare with Eq. (1). Any mismatch for such a matrix would refute the formula; this is the kind of test the paper itself recommends.","tokens_in":4465,"feed_emoji":"🧮","tokens_out":6633,"duration_ms":59190,"temperature":0.7,"pith_summary":"Automatic differentiation (AD) needs a reverse-mode formula for every primitive operation, and for complex numbers the singular value decomposition was the last missing linear-algebra piece. This paper derives that formula: given a real loss function that depends on the SVD factors U, S, V of a complex matrix A, the gradient with respect to A is a four-term expression. Three terms mirror the known real-valued SVD backpropagation formula, but the fourth term is new and has no real-valued counterpart. The new term is what makes differentiation through complex SVD correct whenever the loss couples U and V, which is precisely the case in tensor network computations. If the formula is right, AD over complex linear algebra is complete, and gradient-based optimization of complex tensor networks becomes straightforward.","feed_headline":"Missing term found for complex SVD backprop","feed_subtitle":"The new term couples U and V in the gradient, unlocking automatic differentiation through complex tensor networks.","key_machinery":"The machinery is the Wirtinger calculus setup (derivatives with respect to z and z*, with gradient \\bar A = 2 ∂L/∂A*), combined with the differential relations dC = U†dU, dD = V†dV, and the algebraic identities (8)-(9) for elementwise products. The proof resolves dA into four pieces — diagonal of S, off-diagonal of C and D, and the pure-imaginary diagonals of C and D — and uses gauge invariance, equation (31), to show that the naive diagonal contributions cancel, leaving the new $S^{{-1}}$ term. The matrix F = 1/($s_j^{2}$ - $s_i^{2}$) carries the coupling between different singular vectors and is the same object that appears in the real-valued SVD gradient.","core_discovery":"The paper's claim is that reverse-mode differentiation of a complex-valued SVD, A = U S V†, for a real gauge-invariant loss L is given by a closed-form expression with four contributions. The first three contributions are the complex analogues of the real SVD gradient, built from F_ij = 1/($s_j^{2}$ - $s_i^{2}$) for i≠j and zero on the diagonal. The genuinely complex addition is \\bar A_d = (1/2) U $S^{{-1}}$(L† - L) V†, where L = I ∘ (V† \\bar V), a term required because the diagonal blocks of the differentials dC = U†dU and dD = V†dV are pure imaginary rather than zero. The derivation uses gauge invariance of the loss under phase transformations of U and V — equation (31) — to cancel the ambiguous diagonal contributions, and it extends to rectangular matrices by adding two standard projection terms. The upshot: for any real loss that is invariant under the SVD phase gauge and depends on both U and V, the gradient of the loss with respect to the input matrix can be computed exactly.","pith_inferences":["The appearance of 1/(s_j^2 - s_i^2) means gradients amplify strongly when singular values are close; near-degenerate spectra in tensor network optimization could produce large or unstable gradient updates, so practitioners may need to regularize or avoid SVD layers at near-degenerate points.","Because the new term vanishes for losses that depend only on U or only on V, existing AD libraries with a complex SVD might have passed correctness checks without exercising the true complex contribution; a correct test must couple U and V, as the paper notes.","The same differential-geometric treatment of the SVD fiber (the phase gauge) could be adapted to derive reverse-mode formulas for other factorizations with gauge freedom, such as the eigenvalue decomposition of non-Hermitian matrices or polar decomposition.","The gauge-invariance condition (31) effectively tells us that the gradient of a gauge-invariant loss is a well-defined function on the quotient space of SVD factors, which may matter for optimization landscapes of tensor networks."],"forward_implications":["Complex SVD joins the list of primitives that can be differentiated in reverse mode, closing the explicit gap noted in prior work on complex-valued automatic differentiation.","Tensor network algorithms that use complex SVD for truncation or canonicalization can now be optimized end-to-end by gradient descent on complex tensors.","Any automatic differentiation library can implement Eq. (1) directly; the new fourth term is zero when the loss depends on only U or only V, but indispensable when the loss couples both.","The rectangular extension in Eq. (36) makes the formula applicable to non-square matrices, covering typical uses of SVD in machine learning and physics.","The derivation's gauge-freedom argument shows why a correct complex SVD backprop must respect phase gauge invariance, not merely holomorphic derivatives."],"supporting_citations":[{"why":"A tensor-network AD study that motivates the need for complex SVD backpropagation as a primitive operation.","marker":"[2]"},{"why":"Provides the real-valued SVD reverse-mode differentiation proof that the first three terms of Eq. (1) generalize.","marker":"[5]"},{"why":"Identifies reverse-mode complex SVD as a missing piece of complex AD infrastructure, the gap this paper fills.","marker":"[7]"},{"why":"Supplies the Wirtinger calculus definitions for complex derivatives and gradients used throughout the derivation.","marker":"[9]"},{"why":"A reference implementation that confirms the derived formula in practice and demonstrates its use in an AD framework.","marker":"[10]"}],"fun_headline_variants":["Complex SVD gradient gets missing U-V coupling term","Missing term couples U,V in complex SVD backprop","Exact complex SVD gradient found with U-V coupling","U-V coupling term completes complex SVD auto-diff"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The formula assumes the loss is real and invariant under the phase-gauge freedom of the SVD (multiplying U and V by the same diagonal phase matrix), and that all singular values are distinct and nonzero; if any of these fails, the gradient either depends on the arbitrary gauge choice or the expression contains divisions by zero.","fun_headline_variants_meta":{"raw":{"variants":["Complex SVD gradient gets missing U-V coupling term","Missing term couples U,V in complex SVD backprop","Exact complex SVD gradient found with U-V coupling","U-V coupling term completes complex SVD auto-diff"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.001095,"raw_usage":{"total_tokens":4499,"prompt_tokens":799,"completion_tokens":3700,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":415,"completion_tokens_details":{"reasoning_tokens":3634}},"tokens_in":415,"tokens_out":3700,"duration_ms":23403,"temperature":1.0,"reasoning_tokens":3634,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-14T05:11:21.495314+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Take a random 2x2 complex matrix with two distinct nonzero singular values and the gauge-invariant loss L = Re(U00 V*_00), compute the central finite-difference gradient of L with respect to the real and imaginary parts of A, and compare with Eq. (1). Any mismatch for such a matrix would refute the formula; this is the kind of test the paper itself recommends.","supporting_citations":[{"cited_title":"Wavefunction positivization via automatic differentiation","cited_arxiv_id":"1906.04654","evidence_quote":"Provides the real-valued SVD reverse-mode differentiation proof that the first three terms of Eq. (1) generalize."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Identifies reverse-mode complex SVD as a missing piece of complex AD infrastructure, the gap this paper fills."},{"cited_title":"Use and implementation of autodifferentiation in tensor network methods with complex scalars","cited_arxiv_id":"1907.13422","evidence_quote":"Supplies the Wirtinger calculus definitions for complex derivatives and gradients used throughout the derivation."},{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"A reference implementation that confirms the derived formula in practice and demonstrates its use in an AD framework."}],"review_version":1}