{"paper":{"title":"Fast and Stable Gradient Approximation for Bilinear Forms of Hermitian Matrix Functions","license":"http://creativecommons.org/licenses/by/4.0/","headline":"A forward-only gradient approximation reuses the Lanczos pass to stably differentiate bilinear forms u^T f(A(θ))v for Hermitian A with error proportional to the residual norm.","cross_cats":["cs.NA"],"primary_cat":"math.NA","authors_text":"Kipton Barros, Navjot Singh, Xiaoye Sherry Li","submitted_at":"2026-05-12T22:40:16Z","abstract_excerpt":"Objectives involving bilinear forms $u^\\top f(A(\\theta))v$ for Hermitian $A$ arise widely in scientific computing and probabilistic machine learning. For large matrices, Lanczos efficiently approximates these quantities, but differentiating them with respect to $\\theta$ is challenging. Existing approaches either backpropagate through the Lanczos recurrence, requiring reorthogonalization for stability, or apply Arnoldi to an augmented block matrix of twice the original size. Both introduce extra computation and orthogonalization costs that can limit performance on modern hardware. We propose a "},"claims":{"count":4,"items":[{"kind":"strongest_claim","text":"We propose a forward-only gradient approximation that reuses the Lanczos pass and adds very minimal overhead in most cases. We prove that its error is proportional to the Lanczos residual norm, the same quantity controlling the forward approximation. Whereas a traditional adjoint-based calculation would be unstable without reorthogonalization, the new method appears unconditionally stable in our tests.","source":"verdict.strongest_claim","status":"machine_extracted","claim_id":"C1","attestation":"unclaimed"},{"kind":"weakest_assumption","text":"The approach assumes A(θ) remains Hermitian so that Lanczos produces real eigenvalues and orthogonal vectors, and that the residual norm from the forward Lanczos pass is a reliable error indicator for both the value and the gradient; stability is reported from tests but the abstract does not specify the range of matrices or functions f where this holds without additional conditions.","source":"verdict.weakest_assumption","status":"machine_extracted","claim_id":"C2","attestation":"unclaimed"},{"kind":"one_line_summary","text":"A forward-only Lanczos gradient approximation for Hermitian matrix function bilinear forms whose error scales with the same residual norm as the forward approximation and appears stable without reorthogonalization.","source":"verdict.one_line_summary","status":"machine_extracted","claim_id":"C3","attestation":"unclaimed"},{"kind":"headline","text":"A forward-only gradient approximation reuses the Lanczos pass to stably differentiate bilinear forms u^T f(A(θ))v for Hermitian A with error proportional to the residual norm.","source":"verdict.pith_extraction.headline","status":"machine_extracted","claim_id":"C4","attestation":"unclaimed"}],"snapshot_sha256":"099caa02bb0260f37ec9f62ee35a526db3c5ef6ba5d70226b323ed21b8e36031"},"source":{"id":"2605.12801","kind":"arxiv","version":1},"verdict":{"id":"eafc3d2f-9fd8-4753-8a33-c67a8dea1976","model_set":{"reader":"grok-4.3"},"created_at":"2026-05-14T19:28:58.810999Z","strongest_claim":"We propose a forward-only gradient approximation that reuses the Lanczos pass and adds very minimal overhead in most cases. We prove that its error is proportional to the Lanczos residual norm, the same quantity controlling the forward approximation. Whereas a traditional adjoint-based calculation would be unstable without reorthogonalization, the new method appears unconditionally stable in our tests.","one_line_summary":"A forward-only Lanczos gradient approximation for Hermitian matrix function bilinear forms whose error scales with the same residual norm as the forward approximation and appears stable without reorthogonalization.","pipeline_version":"pith-pipeline@v0.9.0","weakest_assumption":"The approach assumes A(θ) remains Hermitian so that Lanczos produces real eigenvalues and orthogonal vectors, and that the residual norm from the forward Lanczos pass is a reliable error indicator for both the value and the gradient; stability is reported from tests but the abstract does not specify the range of matrices or functions f where this holds without additional conditions.","pith_extraction_headline":"A forward-only gradient approximation reuses the Lanczos pass to stably differentiate bilinear forms u^T f(A(θ))v for Hermitian A with error proportional to the residual norm."},"references":{"count":42,"sample":[{"doi":"10.1137/080716426","year":2009,"title":"Al-Mohy and Nicholas J","work_id":"256900b9-9dc5-4470-a0e1-8b1b1c68468a","ref_index":1,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2023,"title":"Learning quantum systems.Nature Reviews Physics, 5:141–156, 2023","work_id":"0af8d3eb-3b7c-4734-9ac1-646988e2fd47","ref_index":2,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"10.1137/080741744","year":2009,"title":"Error estimates and evaluation of matrix functions via the Faber transform.SIAM Journal on Numerical Analysis, 47(5):3849–3883, 2009","work_id":"d470a323-11f4-40ba-ab88-10f3b8842689","ref_index":3,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"","year":2020,"title":"Matrix functions in network analysis.GAMM-Mitteilungen, 43 (3):e202000012, 2020","work_id":"1e835a40-5358-41e4-b572-0dbedcc11934","ref_index":4,"cited_arxiv_id":"","is_internal_anchor":false},{"doi":"10.1137/22m1526357","year":2023,"title":"Krylov-aware stochastic trace estimation.SIAM Journal on Matrix Analysis and Applications, 44(3):1218–1244, 2023","work_id":"a1573fb9-1aa6-47d8-b0e6-0c41c145bca3","ref_index":5,"cited_arxiv_id":"","is_internal_anchor":false}],"resolved_work":42,"snapshot_sha256":"25bad2aa1de10dc1098ee862598fbc580db58620f9c4f09f8b8e7e1a2d30bfaa","internal_anchors":0},"formal_canon":{"evidence_count":2,"snapshot_sha256":"4532b64119779b7eadd845e43a9029610bd5b1bf930968469979a04a88cbbca8"},"author_claims":{"count":0,"strong_count":0,"snapshot_sha256":"258153158e38e3291e3d48162225fcdb2d5a3ed65a07baac614ab91432fd4f57"},"builder_version":"pith-number-builder-2026-05-17-v1"}