PySHRED: A Python package for SHallow REcurrent Decoding for sparse sensing, model reduction and scientific discovery

David Ye; David Zoro; Jan Williams; J. Nathan Kutz; Mars Gao; Matteo Tomasetto; Stefano Riva

REVIEW 2 major objections 5 minor 1 cited by

PySHRED: A Python package for SHallow REcurrent Decoding for sparse sensing, model reduction and scientific discovery

T0 review · 2 major / 5 minor · reviewed 2026-08-06 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read PySHRED unifies SHRED sensing, reduced-order modeling, and discovery in one Python package.

desk verdict PySHRED is a genuinely useful software release, but Appendix B's noise demonstration has a code-ordering bug that makes Table 1 meaningless as a noise test. read the letter →

arxiv 2507.20954 v1 pith:74WJPWOP submitted 2025-07-28 cs.LG cs.CEmath.DSnlin.CD

David Ye , Jan Williams , Mars Gao , Stefano Riva , Matteo Tomasetto , David Zoro , J. Nathan Kutz This is my paper

classification cs.LGcs.CEmath.DSnlin.CD

keywords SHallowRecurrentDecodersparsesensingreduced-ordermodelingscientificdiscoverySINDyPythonpackagespatiotemporaldatacompression

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper introduces PySHRED, an open-source Python package built around the SHallow Recurrent Decoder (SHRED) architecture, and makes the case that the method has matured from a research idea into a usable tool. The package implements the original sensing formulation plus three extensions—SHRED-ROM for parametric reduced-order modeling, SINDy-SHRED for discovering sparse latent dynamics, and multi-field SHRED for coupled fields—behind a single DataManager–SHRED–SHREDEngine interface. It also supports training on compressed representations, which the paper presents as the key to laptop-level training on high-dimensional data. The paper's aim is to establish that a broad set of SHRED tasks can be performed with one easy-to-install, documented, and modular codebase.

What carries the argument

The load-bearing object is the SHRED model, written as $H(\{s_i\}_{i=t-k}^t) = F(G(\{s_i\}_{i=t-k}^t; W_{RN}); W_{SD})$, where $G$ is a sequence network (LSTM, GRU, or Transformer) that maps $k$ lagged sensor readings to a latent vector and $F$ is a fully connected decoder that maps the latent vector back to the high-dimensional state. Around this sit three modular components: a DataManager that handles lag selection, train/validation/test splits, sensor assignment, and optional compression via randomized SVD or a user-supplied basis such as Fourier modes; a SHRED object whose sequence, decoder, and optional latent forecaster can be mixed and matched; and a SHREDEngine that performs sensor-to-latent encoding, latent forecasting, decoding, and end-to-end evaluation. The argument for why this works rests on the time-delay embedding theorem, which justifies using a window of past sensor measurements, and on the decoding-only strategy, which avoids explicitly inverting a learned encoder.

What would settle it

A concrete check: install the package with a pinned environment, run the three Appendix workflows end-to-end (SST sensing, double-gyre SHRED-ROM with POD and Fourier compression, and SINDy-SHRED on SST), and compare the output figures and the reported mean relative test errors (0.0347/0.0379 for POD $u/v$, 0.0403/0.0609 for Fourier) against the paper's Table 1 values. If any workflow fails through the public API or the errors diverge well beyond run-to-run randomness, the paper's central usability claims would need qualification.

Watch

Extended reading notes

Core claim

The central claim is that the SHRED architecture—encoding time-lagged sensor measurements with a sequence model into a low-dimensional latent space, then decoding back to the full spatial field with a feedforward network—is general enough to cover sensing, model reduction, and physics discovery, and that all of these uses can live in one package. The paper works through concrete cases: weekly sea-surface temperature reconstructed from mobile and stationary sensors, the double-gyre flow modeled as a parametric reduced-order system under both POD and Fourier compression, with relative test errors reported for each, and global sea-surface temperature dynamics discovered as a linear latent ODE by the SINDy forecaster. In each case the same three-module pipeline is used, differing only in configuration, which is the evidence the paper offers for the package's modularity and breadth.

Load-bearing premise

The load-bearing premise is that the PySHRED code in the public repository actually runs as documented in a fresh environment, so the example scripts produce the reconstructions, error tables, and forecast figures shown in the paper.

Editorial extensions

If this is right

New users can go from raw spatiotemporal data to a trained SHRED model with a handful of API calls, lowering the barrier for sensor-based reconstruction.
Compressive training means the same code runs on a laptop for fields whose full state would normally require a GPU, as long as a low-rank representation exists.
Parametric problems (varying parameters, noisy measurements) are handled by the same interface, so a practitioner can build a reduced-order model without writing custom training loops.
Because the latent forecaster is pluggable, the package connects reconstruction to scientific discovery: SINDy-SHRED turns the latent dynamics into an explicit ODE that can be integrated forward for forecasting.
The modular design makes it straightforward to swap sequence models or decoders, so new architectural variants can be tested without refactoring the data pipeline.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

A natural next test not reported here is a systematic sweep of the `compress` parameter to map how reconstruction error degrades as fewer SVD modes are kept; the package's interface makes this a one-line experiment.
The same latent-forecaster slot could host equation-learning methods other than SINDy, and the paper's claimed extensibility implies such a swap should require no changes outside the forecaster module.
If the paper's usability claims hold, PySHRED could serve as a common benchmark harness for comparing sensor placement, lag length, and architecture choices across SHRED variants.
The reported SINDy-discovered SST ODE is specific to the 50 randomly chosen sensors; an implicit open question the paper leaves is how sensitive the discovered coefficients are to sensor choice.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

Colleague,

Quick take: this is a legitimate software release paper for PySHRED, the unified Python implementation of SHRED and its ROM/SINDy/multi-field variants. If you work on sparse sensing or reduced-order modeling with sensor data, this package is worth knowing about: clean API, DataManager -> SHRED -> SHREDEngine pipeline, compression built in, MIT license. The paper itself introduces no new method or result; novelty is the engineering. That's fine for a software paper, and the code listings make the intended usage reasonably clear.

What the paper does well: the architecture is modular, the design choices are sensible, and the examples cover sensing, parametric ROM, and SINDy-based discovery with enough detail to follow. The appendices read like real tutorials.

Where it gets soft. First, the reader's instincts are right: the reproducibility evidence is thin. No commit hash, no pinned dependencies, no test suite, no archived artifact; the paper says "easy to install" but nothing demonstrates it. That alone would keep me from fully verifying the package from the paper.

Second, the stress-test note lands on a real internal inconsistency in Appendix B. Listing 6 calls manager_pod.prepare() and binds train_dataset, val_dataset, test_dataset. Listing 7 then mutates manager_pod.sensor_measurements_df by adding Gaussian noise. Listing 8 fits shred_pod on the datasets from Listing 6. Under any natural contract where prepare() materializes the datasets from the sensor measurements, the added noise never reaches the training objects. So Table 1's errors say nothing about noise robustness; the text claims the noise matters, but the code as printed makes it inert. That's a concrete, load-bearing flaw in the paper's only noise demonstration. Fix: inject noise before prepare() and verify the table changes.

Also note: the appendices don't seed sensor selection or network initialization, so exact reproductions are impossible anyway. That's a lesser but real issue.

Is the core claim salvageable? Yes. The package almost certainly works; the examples are typical SHRED usage, and the results match earlier published behavior. But the paper as written doesn't give enough to independently confirm it. The noise-ordering bug should be fixed and the reproducibility story (test suite, pinned environment, archive) tightened before publication.

Who's this for: practitioners wanting a ready SHRED implementation, and the SHRED research group's own users. A serious referee should engage with it; desk rejection would be wrong. I'd recommend conditional acceptance: require the Appendix B fix, a runnable test or smoke test, and a specific commit hash. With those, the paper becomes an honest, useful software contribution.

Recommendation: send to peer review.

Referee Report

2 major / 5 minor

Summary. The paper introduces PySHRED v1.0, an open-source Python package implementing SHRED (SHallow REcurrent Decoder) and its extensions for sparse sensing, reduced-order modeling, and physics discovery. It describes the modular pipeline (DataManager, SHRED, SHREDEngine), presents examples for sea-surface-temperature sensing (Appendix A), SHRED-ROM with POD and Fourier compression (Appendix B, Table 1), and SINDy-SHRED latent dynamics discovery (Appendix C), and claims that the package is easy to install, thoroughly documented, and modularly extensible. The manuscript is primarily a software description; it contains no new algorithmic derivation, but it does include several code listings and a small numerical evaluation table.

Significance. If the package works as described, it would provide a unified, potentially valuable tool for a community that uses SHRED-based methods for sparse sensing, ROM, and scientific discovery. The modular design, support for compressed training representations, and the illustrated SINDy forecaster are real strengths, and the accompanying example gallery could lower the barrier to adoption. However, the manuscript's evidence is limited: there is no test suite, no pinned environment or commit hash, no seed control, and the only explicit noise-robustness demonstration in Appendix B appears to be internally inconsistent. The paper would be strengthened by a small, reproducible verification harness (e.g., a unit test or a script that generates Table 1) and by fixing the example ordering so that the reported results actually reflect the described pipeline. As written, the central claims of maturity and ease of use are plausible but not yet substantiated to the standard expected for a software-release paper.

major comments (2)

[Appendix B, Listings 6-8] The noise-robustness example is internally inconsistent. Listing 6 calls manager_pod.prepare() and unpacks train_dataset, val_dataset, and test_dataset; Listing 7 then adds Gaussian noise to manager_pod.sensor_measurements_df; Listing 8 calls shred_pod.fit(train_dataset, val_dataset, ...). Under the DataManager contract described in Section 3.1 ('Once all data has been added, call prepare to obtain the train, validation, and test datasets'), prepare() materializes the training datasets at call time, so the noise added in Listing 7 cannot influence the datasets passed to fit() in Listing 8. Consequently, Table 1 and Figure 5 do not, as printed, demonstrate any noise robustness. If prepare() instead returns live views into the DataFrame, that contract is undocumented and the example is misleading. The fix is to inject the noise before calling prepare() (or to re-run prepare() after adding the noise) and to confirm that the reported table actually reflects the noisy measurements.
[Abstract and Section 3.4 / Appendices A-C] The manuscript provides no reproducibility artifacts: no commit hash, no pinned dependency versions, no seed values, and no test suite, and the examples use unseeded random sensor selection (e.g., Listing 3 uses `random=3`, Appendix C uses `random=50`) and unseeded network initialization. As a result, the exact figures and Table 1 cannot be regenerated from the paper alone. This is load-bearing for the abstract's claims that the package is 'easy to install, thoroughly-documented, supplemented with extensive code examples': a software paper of this kind should include at least an installation command, a minimal smoke test, instructions for setting seeds, and a versioned release identifier. The repository may already contain these, but the manuscript must state them and ideally archive a snapshot for review.

minor comments (5)

[Section 4, first sentence] The sentence 'The PySHRED package in an open-source project' should read 'The PySHRED package is an open-source project'.
[Appendix A, Listing 1 paragraph] 'We being by loading the data' should be 'We begin by loading the data.'
[Code listings throughout] Several code listings contain visible artifacts such as `D a t a M a n a g e r` and `S H R E D E n g i n e`; these should be fixed so that the code can be copy-pasted directly.
[Section 2, paragraph after Eq. (2)] The statement that the SHRED loss landscape 'has been observed to be globally convex' cites an arXiv preprint [5]; this is a strong claim and should be either qualified as an empirical observation for specific architectures or omitted, as it is not central to the package description.
[Appendix C, paragraph on SINDy thresholding] The claim that the thresholding procedure is 'equivalent to ℓ0 regression with ℓ2 regularization [10, 11]' is stated without proof or precise conditions; it would be helpful to cite the exact theorem or otherwise soften the equivalence claim.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: PySHRED is an implementation of previously published SHRED methods, not a derivation that reduces to its own inputs.

full rationale

PySHRED is a software/implementation paper rather than a derivation of new scientific results. Its central claims are that the package implements SHRED and its extensions (SHRED-ROM, SINDy-SHRED, multi-field SHRED), is installable, documented, and modular. These claims are externally checkable against the public repository and do not rest on a fitted parameter being renamed as a prediction. The mathematical grounding cited in Section 2 (separation of variables, Takens embedding, SVD compression, and the l0/l2 equivalence for sparse deep learning) comes from established literature or from prior method papers that are implemented rather than re-derived here. The appendices demonstrate the cited methods on SST and double-gyre data; the reported reconstructions and the discovered SINDy model are outputs of those methods, not quantities defined in terms of the package's own outputs. Self-citations to the original SHRED, SHRED-ROM, and SINDy-SHRED papers are lineage references for the algorithms being packaged and are not load-bearing circularity, because the package's contribution is the implementation itself. One internal inconsistency does exist: in Appendix B, Gaussian noise is added to manager_pod.sensor_measurements_df in Listing 7 after prepare() has already created train_dataset, val_dataset, and test_dataset in Listing 6, so as printed the noise may not affect the SHRED-ROM training that produces Table 1. This is a reproducibility/correctness flaw, not a circularity of the derivation, and it does not change the circularity score.

Assumptions & free parameters 6 free parameters · 6 assumptions · 0 invented entities

The package itself is engineering built on previously published methods, so most ledger entries are example-level hyperparameters and background assumptions rather than new postulates. No new physical entity, force, or latent structure is introduced beyond the already-published SHRED framework.

free parameters (6)

SINDy latent ODE coefficients = x0_dot = 0.048 - 0.122 x0 - 0.279 x1 - 0.103 x2, x1_dot = 0.012 + 0.066 x0 + 0.036 x1 + 0.070 x2, x2_dot = -0.165…
Fitted to SST sensor data in Appendix C; they are outputs of the package example, not inputs to the package's core claim.
POD retained modes = 4 per scalar field
Chosen by hand in Appendix B; compression rank affects the ROM reconstruction errors reported in Table 1.
SINDy sparsity threshold = 0.05
Default threshold for zeroing coefficients in Appendix C; changes the discovered equation.
SINDy regularization strength = 1
Controls the trade-off between reconstruction fidelity and latent ODE consistency in Appendix C.
User-set hyperparameters = lags=52 or 25, sensors=3 or 50, train/val/test split 0.8/0.1/0.1
These choices shape the reported example results in Appendices A through C.
Noise standard deviation = 0.005
Added to synthetic sensor data in Appendix B; affects the reported reconstruction error.

assumptions (6)

standard math Takens embedding theorem: time-delayed sensor measurements form a diffeomorphic copy of the original state space.
Invoked in Section 2 as a core justification for mapping sensor windows to full states.
domain assumption Separation of variables and a decoding-only strategy avoid the ill-conditioned inverse of encoder-decoder pairs.
Stated in Section 2 as a mathematical motivation for the SHRED architecture.
standard math Randomized SVD provides a low-rank compressed representation with controllable reconstruction error.
Used in the DataManager compress argument and in POD compression for Appendix B.
domain assumption Thresholding coefficients during SINDy training approximates l0 regression with l2 regularization.
Appendix C relies on this equivalence, citing references [10, 11], to justify the sparse discovery procedure.
domain assumption The SHRED loss landscape is globally convex, as reported in a prior paper by the same group.
Section 2 repeats this observation from reference [5]; it is not proved or re-tested in this release paper.
domain assumption The double-gyre analytical velocity field is a valid ground-truth model for the ROM demonstration.
Appendix B uses this model to generate synthetic snapshots and evaluate reconstruction errors.

how reviews work

0 comments

Cite this review

Pith. "Pith review of PySHRED: A Python package for SHallow REcurrent Decoding for sparse sensing, model reduction and scientific discovery." pith.science (2026). https://pith.science/paper/74WJPWOP

@misc{pith2026250720954,
  author       = {Pith},
  title        = {Pith review of: PySHRED: A Python package for SHallow REcurrent Decoding for sparse sensing, model reduction and scientific discovery},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/74WJPWOP}},
  note         = {Machine review of arXiv:2507.20954}
}

read the original abstract

SHallow REcurrent Decoders (SHRED) provide a deep learning strategy for modeling high-dimensional dynamical systems and/or spatiotemporal data from dynamical system snapshot observations. PySHRED is a Python package that implements SHRED and several of its major extensions, including for robust sensing, reduced order modeling and physics discovery. In this paper, we introduce the version 1.0 release of PySHRED, which includes data preprocessors and a number of cutting-edge SHRED methods specifically designed to handle real-world data that may be noisy, multi-scale, parameterized, prohibitively high-dimensional, and strongly nonlinear. The package is easy to install, thoroughly-documented, supplemented with extensive code examples, and modularly-structured to support future additions. The entire codebase is released under the MIT license and is available at https://github.com/pyshred-dev/pyshred.

Figures

Figures reproduced from arXiv: 2507.20954 by the authors.

**Figure 2.** Overview of a basic PySHRED workflow, including the DataManager for preprocessing, [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗

**Figure 3.** Available combinations across different models and architectures. As of PySHRED v1.0 [PITH_FULL_IMAGE:figures/full_fig_p004_3.png] view at source ↗

Figures from the paper (6 more)

**Figure 4.** Figure 4: SHRED reconstruction of SST data. 11 id="SST ", 12 mobile = mobile , 13 compress = False 14 ) Listing 3: Define sensor locations. Now that we’ve defined our training and testing splits as well as sensor locations, we can prepare our datasets and train the SHRED model. …

**Figure 5.** Figure 5: Test vorticity fields and velocity contour plots (first column), corresponding SHRED [PITH_FULL_IMAGE:figures/full_fig_p012_5.png]

**Figure 6.** Figure 6: SINDy-SHRED reconstruction and forecast of SST dynamics. [PITH_FULL_IMAGE:figures/full_fig_p013_6.png]

**Figure 7.** Figure 7: Sensor to latent pipeline: sensor measurements are transformed into latent representa [PITH_FULL_IMAGE:figures/full_fig_p014_7.png]

**Figure 8.** Figure 8: Forecasting pipeline: a seed sequence of latent vectors is propagated forward in time using [PITH_FULL_IMAGE:figures/full_fig_p014_8.png]

**Figure 9.** Figure 9: Decoding pipeline: latent trajectories are mapped back to the full high-dimensional state [PITH_FULL_IMAGE:figures/full_fig_p015_9.png]

Discussion (0). Sign in to comment.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

A Shallow Recurrent Decoder for Dynamic State Estimation with a Limited Number of PMUs in Power Systems
eess.SY 2026-06 unverdicted novelty 4.0 of 10

SHRED reconstructs full power system state from limited PMU data, outperforming a shallow decoder benchmark on the IEEE 39-bus system under nonlinear disturbances.

Reference graph

Works this paper leans on

12 extracted references · 7 canonical work pages · cited by 1 Pith paper

[1]

J. P. Williams, O. Zahn, and J. N. Kutz , Sensing with shallow recurrent decoder networks , Proceedings of the Royal Society A, 480 (2024), p. 20240054

work page 2024
[2]

J. J. D. CROZ and N. J. Higham , Stability of methods for matrix inversion , IMA Journal of Numerical Analysis, 12 (1992), pp. 1--19

work page 1992
[3]

Takens , Detecting strange attractors in turbulence , in Dynamical Systems and Turbulence, Warwick 1980, Lecture Notes in Mathematics, vol

F. Takens , Detecting strange attractors in turbulence , in Dynamical Systems and Turbulence, Warwick 1980, Lecture Notes in Mathematics, vol. 898, Springer Berlin Heidelberg, 1981, pp. 366--381. https://doi.org/10.1007/BFb0091924 https://doi.org/10.1007/BFb0091924

doi:10.1007/bfb0091924 1980
[4]

Sauer, J

T. Sauer, J. Yorke, and M. Casdagli , Embedology , Journal of Statistical Physics, 65 (1991), pp. 579--616. https://doi.org/10.1007/BF01053745 https://doi.org/10.1007/BF01053745

doi:10.1007/bf01053745 1991
[5]

M. L. Gao, J. P. Williams, and J. N. Kutz , Sparse identification of nonlinear dynamics and koopman operators with shallow recurrent decoder networks , arXiv preprint arXiv:2501.13329, (2025)

arXiv 2025
[6]

Faraji, M

F. Faraji, M. Reza, and J. N. Kutz , Shallow recurrent decoder for reduced order modeling of e b plasma dynamics , Machine Learning: Science and Technology, 6 (2025), p. 025024

work page 2025
[7]

J. N. Kutz , Data-driven modeling & scientific computation: methods for complex systems & big data , OUP Oxford, 2013

work page 2013
[8]

Tomasetto, J

M. Tomasetto, J. P. Williams, F. Braghin, A. Manzoni, and J. N. Kutz , Reduced order modeling with shallow recurrent decoder networks , arXiv preprint arXiv:2502.10930, (2025)

arXiv 2025

Show all 12 references

[9]

S. C. Shadden, F. Lekien, and J. E. Marsden , Definition and properties of lagrangian coherent structures from finite-time lyapunov exponents in two-dimensional aperiodic flows , Physica D: Nonlinear Phenomena, 212 (2005), pp. 271--304

2005
[10]

Zheng, Y

Z. Zheng, Y. Fan, and J. Lv , High dimensional thresholded regression and shrinkage effect , Journal of the Royal Statistical Society Series B: Statistical Methodology, 76 (2014), pp. 627--649

2014
[11]

L. Gao, U. Fasel, S. L. Brunton, and J. N. Kutz , Convergence of uncertainty estimates in ensemble and bayesian sparse model discovery , arXiv preprint arXiv:2301.12649, (2023)

2023 arXiv
[12]

write newline

" write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTION or pop #1 'skip if FUNCTION new.block.check...

Pith tools

Reviewed August 6, 2026 · model on record in the stance chip above.

[1] [1]

J. P. Williams, O. Zahn, and J. N. Kutz , Sensing with shallow recurrent decoder networks , Proceedings of the Royal Society A, 480 (2024), p. 20240054

work page 2024

[2] [2]

J. J. D. CROZ and N. J. Higham , Stability of methods for matrix inversion , IMA Journal of Numerical Analysis, 12 (1992), pp. 1--19

work page 1992

[3] [3]

Takens , Detecting strange attractors in turbulence , in Dynamical Systems and Turbulence, Warwick 1980, Lecture Notes in Mathematics, vol

F. Takens , Detecting strange attractors in turbulence , in Dynamical Systems and Turbulence, Warwick 1980, Lecture Notes in Mathematics, vol. 898, Springer Berlin Heidelberg, 1981, pp. 366--381. https://doi.org/10.1007/BFb0091924 https://doi.org/10.1007/BFb0091924

doi:10.1007/bfb0091924 1980

[4] [4]

Sauer, J

T. Sauer, J. Yorke, and M. Casdagli , Embedology , Journal of Statistical Physics, 65 (1991), pp. 579--616. https://doi.org/10.1007/BF01053745 https://doi.org/10.1007/BF01053745

doi:10.1007/bf01053745 1991

[5] [5]

M. L. Gao, J. P. Williams, and J. N. Kutz , Sparse identification of nonlinear dynamics and koopman operators with shallow recurrent decoder networks , arXiv preprint arXiv:2501.13329, (2025)

arXiv 2025

[6] [6]

Faraji, M

F. Faraji, M. Reza, and J. N. Kutz , Shallow recurrent decoder for reduced order modeling of e b plasma dynamics , Machine Learning: Science and Technology, 6 (2025), p. 025024

work page 2025

[7] [7]

J. N. Kutz , Data-driven modeling & scientific computation: methods for complex systems & big data , OUP Oxford, 2013

work page 2013

[8] [8]

Tomasetto, J

M. Tomasetto, J. P. Williams, F. Braghin, A. Manzoni, and J. N. Kutz , Reduced order modeling with shallow recurrent decoder networks , arXiv preprint arXiv:2502.10930, (2025)

arXiv 2025