{"id":"53f6108b-1381-43dd-9f3e-65af9bfe0f13","arxiv_id":"2411.13906","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":3,"one_line_summary":"A modified Adam optimizer that works directly on the Stiefel manifold makes symplectic autoencoder training for Hamiltonian model reduction faster per update and, in most tested settings, more accurate than the homogeneous-space Adam baseline.","lead":"This thesis modifies the Adam optimizer to run directly on a curved matrix space, the Stiefel manifold, and uses it to train a symplectic autoencoder that reduces high-dimensional Hamiltonian wave simulations. In tests on the 1D wave and sine-Gordon equations, the new optimizer is much faster per step and often more accurate than the previous homogeneous-space Adam approach.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"StiefelAdam's pseudo second-moment cache (Alg. 7) has no descent guarantee; without the decay chosen after V5 diverged the method explodes, so claimed gains are not shown to be intrinsic.","rationale":"Read in good faith: the thesis clearly describes the StiefelAdam update, provides Julia code, and reports a convincing O(Nn^2) speed advantage for the PSDLayer update step. That part of the central claim is supported. The fragile point is the validity of the pseudo second-moment cache in Algorithm 7. Adam's second moment is a moving average of squared gradients; here B2 is rebuilt each step from the squared current first-moment cache plus squared current gradient, and then used for elementwise normalization after splitting the cache into skew and orthogonal parts. No argument shows this yields a descent direction or preserves the adaptive-scaling properties of Adam. The paper's Figure 16 is direct evidence: without decay, V5's loss explodes after three epochs; with η·0.9995, V6 converges. Because the decay was introduced specifically in response to that failure and is not derived from the optimizer, the reported accuracy gains of V6–V9 are conditional on that tuning choice. Single-run comparisons and the contradicting N=256 results further weaken the accuracy claim. The proposed test would distinguish between 'pseudo-cache is invalid' and 'fixed learning rate too large', which is the key uncertainty. Given the code availability and otherwise reasonable numerical study, CONDITIONAL remains the right verdict; no change from the reader's verdict is needed.","tokens_in":54449,"tokens_out":4336,"duration_ms":45995,"concrete_test":"On the 1D wave equation setup of Runs 5/6 (N=128, n=15, same data and batch size), run StiefelAdam without decay (V5) for 100 epochs with fixed learning rates η = 1e−3, 1e−4, 1e−5, each over 10 random initializations, and record epoch loss and reduction error. Also run a variant in which B2 in Algorithm 7 is replaced by a properly transported second-moment cache (transporting Zt⊙Zt along the same vector transport as B1). If V5 diverges for all fixed rates while the transported-cache variant remains stable without decay, the pseudo-cache in Algorithm 7 is the load-bearing flaw; if some fixed rate matches V6, the instability is a step-size issue and the decay is a legitimate tuning choice.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that StiefelAdam is a faster and often more accurate replacement for homogeneous-space Adam rests on Algorithm 7's pseudo second-moment cache. In step 1, B2 is formed as sqrt((β2−β2^t)/(1−β2^t)·(B1^t⊙B1^t) + ((1−β2)/(1−β2^t))·Zt⊙Zt + δ), and steps 5–6 divide the skew and orthogonal parts of B1^t by this matrix. This is not Adam's transported second-moment estimate, and no convergence or descent-direction argument is given. The paper's own Figure 16 shows that V5 (StiefelAdam without decay) has a strong loss decrease for three epochs and then the epoch loss suddenly increases again and explodes, while V6 only becomes usable after adding the hand-picked decay η ← η·0.9995 per iteration. That makes the reported accuracy gains empirically inseparable from a decay schedule that was selected after observing the divergence, rather than a property of the Stiefel manifold update. Additionally, all accuracy comparisons are single runs, and the N=256 wave-equation run (Run 13) shows V9 losing to PSD for n=17,...,20, contradicting the broad accuracy claim. The efficiency claim O(Nn^2) per PSDLayer update is well supported by the speed test, but it does not by itself establish cheaper end-to-end training, since the GradientLayers cost O(N^2) per Adam step and are not included in Table 10.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript is a master's thesis on structure-preserving model reduction of Hamiltonian systems via a symplectic deep convolutional autoencoder from [BK23]. Its central contribution is a modified Adam optimizer, StiefelAdam, that performs the PSDLayer parameter update directly on the compact Stiefel manifold St(n,N) using a pseudo second-moment cache (Algorithm 7) and a vector-transported first-moment cache, in place of the homogeneous-space detour over O(N) used by [BK23]. The thesis also introduces several network and training modifications (reference-state ROM, normalized snapshots, epoch-wise training, an alternative loss), tabulates the per-update-step complexity of both approaches (O(N n^2) versus O(N^2 (N-n) + N n^2)), and reports numerical experiments for the 1D wave equation and the sine-Gordon equation (single soliton and soliton-soliton doublets), comparing variants V1-V10 against the PSD method and the homogeneous-space Adam baseline. The claimed results are that the StiefelAdam variants are substantially faster per manifold update step and often more accurate, particularly for larger parameter values; the thesis candidly reports that the no-decay variant diverges after a few epochs (Figure 16) and that the N=256 wave-equation run degrades for n=17,...,20 (Figure 18).","tokens_in":54845,"tokens_out":19497,"duration_ms":177067,"significance":"Taken at face value, the per-step complexity improvement for the PSDLayer update (Tables 4-6) is real, and the speed test in Table 10 is consistent with it; if the gain survived in full-network training, it would reduce the cost of high-dimensional symplectic autoencoder training, which is the main bottleneck the thesis targets. The experimental design is meaningful: the PSD method is a standard linear baseline, the homogeneous-space Adam of [BK23]/[Bra23] is the right structural baseline, and there is no circular dependence of the optimizer on the results it is meant to produce. The thesis reports negative results honestly (V5 divergence in Figure 16, N=256 failures in Figure 18, integration-time gap in Table 14), the accompanying Julia code is public, and the complexity tables are transparent. I confirm the main skeptical concern from the review note: the stability and accuracy of StiefelAdam are currently tied to a decay schedule selected after the divergence of V5 was observed.","major_comments":[{"comment":"The specification of the core StiefelAdam step is internally inconsistent. Algorithm 7 step 5 normalizes the skew-symmetric part using W / sqrt(B2^T B2), i.e., an n x n matrix right-division by the 'square root' of the Gram matrix of B2, whereas the shipped implementation in Figure 5 divides W elementwise by B2n = C.B2[1:n,:], the first n rows of the N x n pseudo-cache B2; step 6 divides the orthogonal part elementwise by the full B2. These are different operations, and neither is justified for a general X_t in St(n,N): the X_t-component of the cache is X_t W with W = X_t^T B1^cache, so its second moment has no reason to coincide with the first n rows of the elementwise square of Z_t (it would only do so at the reference point E = [I_n; 0] of (40)). As written, the skew and orthogonal components are normalized by inconsistently chosen quantities, and the step is not invariant under orthogonal changes of coordinates of St(n,N). Because Algorithm 7 is the entire content of the new optimizer, this contradiction between pseudocode and code makes the method under-specified and directly weakens the claim that StiefelAdam is a well-defined replacement for the homogeneous-space Adam step.","section":"Section 3.2.3, Algorithm 7 and Figure 5"},{"comment":"The thesis reports that V5 (StiefelAdam without decay) shows a massive loss decrease for about four epochs and then 'suddenly increases again and explodes', and that V6 is only usable after adding the per-iteration decay eta <- eta * 0.9995, introduced in Section 3.3.3. No descent-direction or convergence argument is given for the pseudo second-moment cache of Algorithm 7, and no sensitivity study for the decay rate or for smaller fixed step sizes is reported. The claimed accuracy advantage of StiefelAdam over homogeneous-space Adam is therefore currently inseparable from a decay schedule selected after the divergence was observed, i.e., it is not shown to be a property of the Stiefel-manifold update itself. I would like to see (i) a decay sweep on the n=15 wave-equation case (e.g., no decay with eta in {1e-3, 3e-4, 1e-4}, and decay rates 0.999, 0.9995, 0.9999, with several seeds each), and (ii) either a theoretical statement about when the pseudo-cache provides a descent direction or an explicit statement that the method currently requires the decay to be stable.","section":"Section 4.2.4, Figure 16"},{"comment":"Every numerical comparison in the thesis is based on a single training run per variant, with random initialization and random batch draws. The observed differences between variants (e.g., V6-V9 better than V3 on average, V8/V9 slightly ahead of V6/V7, or the comparison to PSD) are therefore not distinguishable from seed-to-seed noise. This matters for the central 'often more accurate' claim: at least three to five seeds with mean and range (or individual points) should be reported for the key comparisons on each example, or the accuracy claims should be explicitly labeled as single-run observations. This also applies to the loss curves in Figures 16, 17, 22 and 26.","section":"Section 4, Tables 13, 16 and 17"},{"comment":"The speed test measures only a single PSDLayer manifold update step with a synthetic update vector. The thesis itself notes (end of Section 3.3.3) that each GradientLayer with upscaling factor L = 5N costs O(N^2) per Adam iteration, so the per-iteration cost of the full network remains O(N^2)-dominated; the StiefelAdam speed-up in Table 10 therefore does not by itself establish cheaper end-to-end training. The abstract's claim of 'computational efficiency of the presented learning algorithms' is broader than the evidence presented. Please either report wall-clock training time per epoch (or per iteration) for the full network for the compared variants, or explicitly scope the efficiency claim to the PSDLayer update step (and correspondingly adjust the conclusion).","section":"Section 4.1, Table 10 and Section 3.3.3"},{"comment":"In the N=256 wave-equation run, the V9 autoencoder 'can keep up' with PSD only for n=10,...,16, loses to PSD for n=17,...,20, shows exploding reduction errors for some (n, mu), and the ROM integration time is roughly 20-25 times that of PSD at the largest reduced dimensions. These results are reported honestly, but they are not a minor caveat: model reduction is most relevant at high dimension, so this is a direct limitation on the abstract's statement that the experiments 'demonstrate the improved accuracy' of the presented algorithms. The accuracy claim should be restated with the observed domain of validity (N=128 across the tested n, and N=256 only for n <= 16), and the failure mode at large n should be discussed (e.g., a hypothesis, or at least a statement of which of the two issues observed in Figure 16 - the initial fast decrease and the subsequent instability - is at play here).","section":"Section 4.2.4, Run 13, Figures 18-19 and Table 14"}],"minor_comments":[{"comment":"Many appendix plot labels contain corrupted characters or missing letters ('reduced dimensi n n', 'err r', 'Reducti n err rs', '/uni03BC' in axis titles, and 'Vμ' in place of 'V3'); the figures should be regenerated with correct fonts and complete labels.","section":"Appendix A, Figures 13-27"},{"comment":"The text states n_training_iterations = n_epochs * (#snapshots)/batch_size, but the shown code uses dl.n_params in that formula; if n_params is the number of network parameters rather than the number of snapshots, the non-epoch-wise reproduction does not implement the stated iteration rule and this should be clarified.","section":"Section 3.3.4, Figure 7"},{"comment":"The phrase 'the (discretized) FOM solution for nu = 0 is constant (int)' appears to mean 'constant in time'; as printed, the parenthetical is not meaningful and should be corrected.","section":"Section 4.3.4"},{"comment":"Reference [Hal00] is cited as an arXiv preprint; the published version (B. Hall, Lie Groups, Lie Algebras, and Representations, Springer GTM 222, 2003) would be more appropriate for a journal version.","section":"References"},{"comment":"To position the claimed novelty, the thesis should briefly contrast StiefelAdam with other Riemannian adaptive optimizers in the literature beyond [Bra23] (e.g., Riemannian Adam variants for the Stiefel manifold), even if the comparison is only a remark.","section":"Section 3.2.3"}],"recommendation":"major_revision","confidential_remarks":"This is a submitted arXiv master's thesis with an unusually transparent experimental record: the code is public, the negative results (V5 divergence, N=256 degradation, integration-time gap) are reported in the main text, and the conclusion is more cautious than the abstract. The main obstacles to publication in a journal are the post-hoc decay selection, the single-run statistics, and - most concretely - the mismatch between Algorithm 7 and the implementation in Figure 5 for the skew-part normalization. The novelty relative to [BK23] and [Bra23] is incremental but real, and the contribution would best fit a venue that values careful numerical studies over theoretical guarantees. I have no concerns about plagiarism or proper attribution; the thesis credits [BK23], [Bra23], and [Gao+21] appropriately. My recommendation is major revision with the specific requests listed in the report."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"StiefelAdam is a plausible speed-up for the PSDLayer update, and the thesis is honest about where it fails. The per-iteration cost claim O(N n^2) versus O(N^2(N-n)+N n^2) is backed by a convincing speed test: for N=10,000, n=10, StiefelAdam is roughly 200x faster, and it avoids the GPU out-of-memory failure of HomogeneousAdam at N=20,000. The reference-state plus normalized-data training protocol is a legitimate improvement over the [BK23] baseline and is well-motivated.\n\nThe soft spot is the accuracy story. The pseudo second-moment B2 in Algorithm 7 is not a transported second-moment estimate, and no descent or convergence argument is given. The paper's own Figure 16 shows V5 without decay explodes after a few epochs, and the decay rate 0.9995 was selected after observing that divergence. All accuracy comparisons are single training runs, and the one high-dimensional run (N=256) shows V9 losing to PSD for n=17-20, contradicting the broad accuracy claim. The end-to-end training cost is also not established, since the GradientLayers cost O(N^2) per Adam step and are excluded from Table 10.\n\nThat said, the thesis is refreshingly transparent. It reports the V5 explosion, the strong slowdown after a few epochs, and the integration time penalty. The comparisons to PSD and to the homogeneous-space baseline are meaningful, and the code is available.\n\nFor a reader in structure-preserving model reduction, the efficiency result is the takeaway; the accuracy improvements are suggestive but not yet proven. A serious referee should ask for multi-run statistics with variance, a closer look at the pseudo-cache (even a local descent result), and an ablation separating the decay from the Stiefel manifold update.\n\nI would send this to peer review, expecting major revision. The optimizer is novel, the speed test is reproducible, and the author's own discussion of limitations is useful.","headline":"Speed claim holds; accuracy claim needs more evidence before it can be taken as established.","tokens_in":55314,"tokens_out":2456,"would_cite":true,"duration_ms":24499,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":["65P10","68T07","37M05"],"pacs":[],"model":"deepseek-v4-flash","headline":"The paper's StiefelAdam updates weights directly on the compact Stiefel manifold, cutting a symplectic-autoencoder training step from O(N²(N−n)+Nn²) to O(Nn²) and making training several times faster and often more accurate.","keywords":["symplectic autoencoder","model reduction","Hamiltonian systems","Stiefel manifold","Adam optimizer","manifold optimization","Cayley retraction","sine-Gordon equation"],"falsifier":"Two concrete checks settle the claims. First, time one PSD-layer update at $N=20000$, $n=10$ on hardware with enough memory: if the direct StiefelAdam step is not markedly cheaper than the homogeneous-space step, the cost claim fails. Second, rerun the wave-equation training at $N=128$, $n=15$ with the no-decay StiefelAdam for 100 epochs while computing offline, from the same gradient history, what Adam's exact transported second-moment cache would be; if the epoch loss explodes as the paper reports and the angle between the pseudo-cache direction and the exact-moment direction diverges before the explosion, the pseudo-cache is the load-bearing failure point.","tokens_in":54260,"feed_emoji":"⚙️","tokens_out":24122,"duration_ms":175607,"temperature":0.7,"pith_summary":"This paper claims that the Adam optimizer can be reworked into a StiefelAdam step that updates the autoencoder's dimension-changing layer directly on the compact Stiefel manifold, the set of matrices with orthonormal columns, instead of taking a detour through a homogeneous space. In the symplectic autoencoder used for structure-preserving model reduction of Hamiltonian systems, each such update costs $O(N n^2)$ with $N$ the full and $n$ the reduced dimension, compared with $O(N^2(N-n) + N n^2)$ for the earlier homogeneous-space approach. The reported speed measurements put every StiefelAdam variant multiple times faster than the homogeneous Adam step, and on the 1D wave equation and the sine-Gordon equation the direct updates often reach lower reduction and projection errors, especially for the hardest tested system parameters. The paper also proposes supporting changes — normalized training data, a reference state in the reduced model, and epoch-wise training — that improve accuracy independently of the optimizer choice.","feed_headline":"Direct StiefelAdam step cuts symplectic autoencoder training cost","feed_subtitle":"Updating weights on the manifold cuts each training step from O(N²) to O(Nn²), making high-dimensional reduction affordable.","key_machinery":"The load-bearing mechanism is Algorithm 7, the StiefelAdam one-cache update on the tangent space $T_X\\mathrm{St}(n,N)$. It splits the transported first moment $B_1 = X_tW + X_t^\\perp K$ into a skew-symmetric part $X_tW$ with $W \\in S_{\\mathrm{skew}}(n)$ and an orthogonal part $X_t^\\perp K$, rescales each component against the pseudo second moment $B_2$ (the skew part through an $n\\times n$ block of $B_2$, the orthogonal part by elementwise division), multiplies by the negative step size $-\\eta$, and maps the result back to the manifold with the Cayley retraction $R_X^{\\mathrm{cay}}(Z) = \\mathrm{cay}(\\tfrac12 A_{X,Z})X$, computed through the Sherman–Morrison–Woodbury formula in $O(N n^2)$. The companion piece is the vector transport of the single cache $B_1$ along the Cayley retraction, available in a submanifold (projection) version and a differentiated version, which substitutes for Adam's transported second-moment cache.","core_discovery":"The central discovery is a modified Adam optimizer, named StiefelAdam, that performs the PSD-layer weight update directly on the compact Stiefel manifold $\\mathrm{St}(n,N)$ instead of routing it through the homogeneous space $O(N)/\\mathrm{St}(n,N)$. On a tangent space, ordinary Adam's caches break down: the elementwise square of a tangent vector leaves the tangent space, and there is no canonical connection for transporting a second-moment cache between iterates. The paper's fix keeps a single transported cache $B_1$ and manufactures a pseudo second moment $B_2 = \\sqrt{\\frac{\\beta_2-\\beta_2^t}{1-\\beta_2^t}(B_1\\odot B_1) + \\frac{1-\\beta_2}{1-\\beta_2^t}(Z_t\\odot Z_t) + \\delta}$ from the square of that cache plus the square of the current Riemannian gradient; the update then rescales the skew-symmetric part $X_tW$ and the orthogonal part $X_t^\\perp K$ of $B_1$ separately against the pseudo cache $B_2$, reassembles them, retracts to the manifold with the Cayley retraction, and transports $B_1$ along Cayley with either the submanifold or the differentiated vector transport. Because the QR-based section and the lift into the global tangent-space representation are no longer needed, one full update step costs $O(N n^2)$ rather than $O(N^2(N-n) + N n^2)$.","pith_inferences":["A natural next experiment is to replace the pseudo-cache with a genuinely transported second-moment cache on the Stiefel manifold and compare convergence; if that removes the need for an external learning-rate decay, the one-cache design is the limiting factor, and if it does not, the pseudo-cache is not the bottleneck.","The paper reports that the loss decrease slows sharply after a few epochs, which suggests that step-size scheduling, not manifold geometry, limits the method; an adaptive decay driven by the transported cache could recover the fast early descent.","Because the symplectic Stiefel manifold contains the compact Stiefel manifold as an immersed subset through $X \\mapsto \\mathrm{diag}(X,X)$, the same direct-update idea transfers to other optimization problems posed on the symplectic Stiefel manifold, and also to the GradientLayers whose classical Adam step costs $O(N^2)$ per iteration at the chosen upscaling parameter.","At the larger full dimension $N=256$ the learned decoder is overtaken by the linear PSD method near reduced dimension 20, and ROM integration with the nonlinear decoder is an order of magnitude slower; this suggests the direct optimizer pays off most at moderate reduced dimensions, and further gains would require cheaper decoder Jacobians or richer training data."],"forward_implications":["One PSD-layer update with StiefelAdam costs $O(N n^2)$ instead of $O(N^2(N-n) + N n^2)$; in the paper's speed test at $N=10000$, $n=10$ the homogeneous Adam step took about 13.8 seconds while every StiefelAdam variant took under 0.065 seconds, and at $N=20000$ the homogeneous step ran out of GPU memory while StiefelAdam still completed.","With the learning-rate decay $\\eta \\leftarrow 0.9995\\,\\eta$, the StiefelAdam variants show a fast early loss decrease that then continues monotonically, and in the numerical tests they attain lower reduction and projection errors than the homogeneous-space baseline for the most demanding system parameters ($\\mu = 0.625$ for the wave equation, $\\nu = -0.97$ and $\\nu = 0.97$ for the sine-Gordon equa","The combination of normalized training data, a reference state in the reduced-order model, and epoch-wise training (variants V3 onward) improves accuracy over the original unnormalized setup and is carried into all StiefelAdam runs.","All tested StiefelAdam constellations — canonical or Euclidean metric, submanifold or differentiated vector transport — behave similarly in accuracy, with the Euclidean-metric variants slightly ahead on the wave equation, so the direct-manifold update is not sensitive to the choice of metric and transport."],"supporting_citations":[{"why":"introduces the symplectic autoencoder architecture and the homogeneous-space Adam update that StiefelAdam replaces and is benchmarked against.","marker":"[BK23]"},{"why":"supplies the homogeneous-space retraction, the lift, and the global tangent-space representation whose O(N²(N−n)+N n²) cost the direct update avoids.","marker":"[Bra23]"},{"why":"provides the retraction and vector-transport definitions and the manifold-optimization framework used by both update steps.","marker":"[AMS08]"},{"why":"gives the low-rank Cayley retraction on the compact Stiefel manifold and its O(N n²) Sherman–Morrison–Woodbury computation.","marker":"[WY13]"},{"why":"introduces the GradientLayer symplectic building block that supplies the nonlinear pre- and post-processing of the encoder and decoder.","marker":"[Jin+20]"},{"why":"is the source for the symplectic Stiefel manifold properties and the PSD reduction method used as the comparison baseline.","marker":"[Gao+21]"},{"why":"motivates the normalized training data and the reference-state reduced model used in the improved variants.","marker":"[BGH21]"}],"fun_headline_variants":["StiefelAdam trains symplectic autoencoders in O(Nn²)","Direct manifold Adam cuts Hamiltonian reduction cost","Symplectic autoencoder training faster with StiefelAdam","Structure-preserving model reduction gets cheaper optimizer","Hamiltonian systems simulation speedup via StiefelAdam"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that the pseudo second-moment cache in Algorithm 7, formed from the squared transported first-moment cache plus the squared current gradient, reliably replaces Adam's true second-moment cache when it rescales the skew and orthogonal parts of the update; the paper itself reports that without a learning-rate decay the epoch loss of this variant explodes after a few epochs, which is the failure mode one would expect if that premise gave way.","fun_headline_variants_meta":{"raw":{"variants":["StiefelAdam trains symplectic autoencoders in O(Nn²)","Direct manifold Adam cuts Hamiltonian reduction cost","Symplectic autoencoder training faster with StiefelAdam","Structure-preserving model reduction gets cheaper optimizer","Hamiltonian systems simulation speedup via StiefelAdam"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000981,"raw_usage":{"total_tokens":4284,"prompt_tokens":1184,"completion_tokens":3100,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":800,"completion_tokens_details":{"reasoning_tokens":3020}},"tokens_in":800,"tokens_out":3100,"duration_ms":64468,"temperature":1.0,"reasoning_tokens":3020,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-12T15:46:12.415062+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Two concrete checks settle the claims. First, time one PSD-layer update at $N=20000$, $n=10$ on hardware with enough memory: if the direct StiefelAdam step is not markedly cheaper than the homogeneous-space step, the cost claim fails. Second, rerun the wave-equation training at $N=128$, $n=15$ with the no-decay StiefelAdam for 100 epochs while computing offline, from the same gradient history, what Adam's exact transported second-moment cache would be; if the epoch loss explodes as the paper reports and the angle between the pseudo-cache direction and the exact-moment direction diverges before the explosion, the pseudo-cache is the load-bearing failure point.","supporting_citations":[],"review_version":1}