Pith. sign in

REVIEW 3 major objections 6 minor 15 references

ChemReservoir -- An Open-Source Framework for Chemically-Inspired Reservoir Computing

T0 review · 3 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read This paper introduces ChemReservoir, an open-source framework for chemically inspired reservoir computing, and reports that changing the inflow signal more often—smaller step sizes—substantially improves memory-task performance in…

desk verdict ChemReservoir is a real software contribution whose empirical claims about step size and memory are not yet supported due to single-seed runs and an underspecified GA fitness split. read the letter →

arxiv 2506.04249 v1 pith:ZEFLXIJN submitted 2025-05-31 cs.CE cs.ETcs.LG

Mehmet Aziz Yirik , Jakob Lykke Andersen , Rolf Fagerberg , Daniel Merkle This is my paper
classification cs.CEcs.ETcs.LG
keywords reservoircomputingchemicalreactionnetworkschemicallyinspiredechostatepropertymemorycapacitygeneticalgorithmGillespiesimulationopen-sourcesoftware
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

ChemReservoir is a new open-source framework for building and testing reservoirs made from abstract chemical reaction networks, where pseudo-molecules are nodes and pseudo-reactions are edges. The framework uses a two-level genetic algorithm to search over reservoir topologies and simulation parameters, then trains only a linear readout. The paper's central empirical claim is that the temporal pattern of the input signal matters: reducing the step size, so the inflow changes more frequently, markedly improved short-term memory performance, with NRMSE falling from 7.342 at step size 25 to 0.348 at step size 2. The paper also reports that local connection density affects short-term memory and that longer past time lags make long-term memory tasks harder, consistent with fading memory. The contribution is a reusable, reproducible tool plus these observations about input dynamics and topology.

What carries the argument

The central object is a cycle-based abstract reaction network used as the reservoir topology: pseudo-molecules serve as nodes, pseudo-reactions as edges, and added chords provide local feedback connections. Dynamics are generated by stochastic Gillespie simulation of these networks, with molecule counts over time serving as reservoir states, and a ridge-regression readout maps those states to target outputs. A two-level genetic algorithm carries the argument: the outer level optimizes topology parameters such as node count, chord length, and chord step, while the inner level optimizes simulation parameters such as inflow, reaction rates, and outflow rates, all scored by NRMSE on memory tasks.

What would settle it

Rerun the same short-term memory experiments with several random seeds beyond seed 1, reporting the distribution of NRMSE for each step size; if step size 2 does not consistently outperform step size 25, or if the optimal chord step is not consistently the increasing parameter, the paper's central empirical claims would be unsupported.

Watch

Extended reading notes

Core claim

On the paper's own terms, ChemReservoir establishes that chemically inspired reservoirs built on cycle-based abstract reaction networks can satisfy the echo state property when evaluated with short- and long-term memory capacity tasks. The key quantitative result is that the frequency of input changes is a dominant factor: for the short-term memory task, decreasing the step size from 25 to 2 reduced the normalized root mean square error from 7.342 to 0.348. In the topology optimization, the chord step—the spacing between local feedback connections—was the only topological parameter that consistently increased, which the authors interpret as chord density having the main structural impact on learning performance. In the long-term memory task, increasing the past time lag from 6 to 18 raised the error from 0.313 to 0.410, with a slight drop at tau 24 attributed to the symmetric structure of the target data.

Load-bearing premise

The empirical conclusions rest on stochastic simulations and genetic-algorithm searches run with a single fixed random seed and single-run error values, so the comparative claims about step size and chord density assume that the reported results are stable across randomness.

Editorial extensions

If this is right

  • If the central claim holds, chemically inspired reservoir performance depends strongly on how often the input signal is perturbed rather than only on network topology, so input encoding should be a first-class design choice in chemical reservoir computing.
  • Cycle-based abstract reaction networks with chords can serve as echo-state-property-compliant reservoirs, supporting the further use of chemically inspired graph structures in machine learning.
  • Chord density appears to be a more influential topological lever than cycle length or inflow amount for short-term memory, giving a concrete design heuristic for future chemically inspired reservoirs.
  • The framework provides a reproducible, actively maintained open-source alternative to earlier DNA-specific chemical reservoir tools, allowing systematic benchmarking that those tools could not support.
  • Long-term memory degrades with increasing past time lag in these reservoirs, matching the fading-memory property expected of echo state networks.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • Editorial inference: Because the reported NRMSE values come from stochastic simulations and a single fixed random seed, the quantitative comparisons should be treated as preliminary; a multi-seed replication study would be needed before relying on the exact error rankings.
  • Editorial inference: The step-size effect suggests a testable extension in real chemical reservoirs: if an in-vitro reaction network is driven by periodic inflows, increasing the frequency of concentration changes should improve its short-term memory capacity, analogous to the simulated result.
  • Editorial inference: The framework's generality means it could be applied beyond cycle-based topologies—for example, to random or scale-free abstract reaction networks—without changing the simulation or readout machinery, although the paper does not report such experiments.
  • Editorial inference: The anomalous tau 24 result, attributed to target-data symmetry, flags that benchmark design itself can confound memory measurements; future studies using multiple target formulations would clarify how much of the measured fading memory is intrinsic to the reservoir versus an artifact of the task.
Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

3 major / 6 minor

Summary. The paper introduces ChemReservoir, an open-source framework for chemically inspired reservoir computing built on MØD-StochSim's Gillespie simulation. It constructs cycle-based reservoir topologies with chords, feeds step-function random inputs through the reservoir, and trains a ridge-regression readout on short- and long-term memory tasks. A two-level genetic algorithm optimizes topology and simulation parameters, and the authors report NRMSE values for varying input step sizes and varying delay (tau) values. The paper's main empirical claim is that reducing the step size (i.e., increasing the frequency of input changes) improves memory-task performance (Table 2), with additional observations about chord density and long-term memory degradation.

Significance. If the empirical claims hold, ChemReservoir fills a real gap: it is an actively maintained, open-source, general-purpose tool for chemically inspired reservoir computing, in contrast to the earlier DNA-focused tools of Yahiro and Nguyen. The code is publicly available and the fixed-seed setup is a step toward reproducibility. The specific observations about input step size, chord density, and fading memory are potentially useful for future in-silico and real-chemistry reservoir designs. However, the empirical foundation is currently thin: the central step-size claim rests on single-seed, single-run NRMSE values, and the GA fitness definition is ambiguous with respect to train/test leakage. These issues must be addressed before the quantitative conclusions can be accepted.

major comments (3)
  1. [Section 2.4 and Table 2] The central claim that reducing the step size improves performance rests on single NRMSE values obtained with one fixed random seed (seed 1). The Gillespie simulation and both levels of the genetic algorithm are stochastic, so each reported NRMSE is one realization of the input signal, the reservoir dynamics, and the search trajectory. The monotonic trend in Table 2 (7.342, 0.936, 0.532, 0.348) could be due to run-to-run variability rather than the step-size effect. Please provide repeated-seed runs (e.g., 5-10 seeds) with means and variances, or per-seed scatter plots, for Tables 2 and 3. The same issue affects the chord-density observation in Figure 4.
  2. [Sections 2.3 and 2.4] It is unclear whether the NRMSE used as the genetic-algorithm fitness is computed on the training partition or the test partition. The text states that the data are split 70/30 and that the reported errors are test-set errors, but the fitness definition in Section 2.4 simply says 'Individuals are evaluated based on the fitness score which is the NRMSE.' If the GA selects individuals on test-set NRMSE, the reported test errors are optimistically biased and the comparisons in Tables 2 and 3 are invalid. Please specify explicitly which partition is used for fitness evaluation; if the test set was used during selection, re-run the optimization using training/validation splits and report only held-out test errors.
  3. [Section 4 and Abstract] The conclusion that cycle-based chemically inspired reservoirs 'meet the echo state property' is not supported by the experiments. The echo state property is a formal dynamical-system property, whereas NRMSE on memory-capacity tasks is an empirical proxy. The text should be revised to state that the results are consistent with the fading-memory behavior expected from the echo state property, rather than claiming that the property is demonstrated or met.
minor comments (6)
  1. [Section 2.1] The Gillespie algorithm is attributed to reference [13], but that reference appears to be a paper on a biology-inspired recurrent oscillator network, not the original Gillespie algorithm or MØD-StochSim documentation. Please cite the correct source.
  2. [Section 2.4] There is a typo in 'For these two levels of the the genetic algorithm'; remove the duplicated 'the'.
  3. [Section 3 and Figure 4] The sentence 'the decrease in chord density, the chord step value, had the main impact on learning performance' is unclear. Since a larger chord step corresponds to fewer chords, the text should say that the chord step increased (i.e., chord density decreased) in the best-performing topologies.
  4. [Section 3 and Table 3] The explanation for the tau=24 decrease, attributed to 'the symmetric structure of the target data (Fig. 6)', is not substantiated. Please provide a quantitative analysis of the symmetry or remove the claim.
  5. [Abstract] The abstract states that the tool 'demonstrated stable performance across a range of configurations,' but Table 2 includes an NRMSE of 7.342 for step size 25. Please qualify what is meant by 'stable' in light of that value.
  6. [Section 2.4] The population and elite sizes in the genetic algorithm are both very small (4 and 2). A brief justification for this choice, or a citation to prior work using similar settings, would help the reader assess the search's reliability.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the evaluation is data-driven and independent of its inputs by construction.

full rationale

The paper's core claim is that ChemReservoir is an open-source framework and that reducing input step size improves memory capacity performance. The derivation chain is empirical: reservoir topologies are constructed with MØD, simulated with a Gillespie implementation, fed with a random step input, and evaluated by training a ridge regression readout on short- and long-term memory tasks. The target data are functions of the input signal by design, but this is a standard benchmark construction, not a circular reduction: the influx node is explicitly excluded from training, so the readout cannot trivially copy the target. The genetic algorithm optimizes topologies and parameters against NRMSE on the same tasks, but the paper never misrepresents this optimization search as an independent prediction; it reports the resulting test-set NRMSE values as empirical observations. The conclusions about step size and chord density are data-driven findings from those runs, not consequences of a self-citation or a uniqueness theorem. The paper's references to prior work on cycle-based reservoirs are external support, not self-citation load-bearing arguments. The fixed-seed, single-run issue noted by the reader is a legitimate concern about statistical robustness, but it is not a circularity per the review rubric, since no equation is defined in terms of an output, no fitted parameter is renamed as a prediction, and no load-bearing claim reduces to a self-citation. On the evidence quoted, the derivation is self-contained and non-circular.

Assumptions & free parameters 8 free parameters · 3 assumptions · 2 invented entities

The framework relies on standard stochastic simulation and regression methods, plus domain assumptions about abstract reaction networks and memory tasks. The GA-optimized parameters are free parameters fitted to the benchmark tasks. No new physical entities are introduced beyond the pseudo-molecules and pseudo-rules, which are explicitly abstract.

free parameters (8)
  • Number of nodes (cycle length) = 80 (short-term best); 90 (long-term best)
    Optimized by the outer-level genetic algorithm within bounds (50,300); the best topology reported is 80 nodes for short-term and 90 for long-term.
  • Inflow amount = 60 (short-term); 70 (long-term)
    Constant inflow scaled by random signal; optimized by genetic algorithm within bounds (50,200).
  • Chord length = 15 (short-term); 13 (long-term)
    Optimized within bounds (5,25); affects feedback connectivity.
  • Chord step = 11 (short-term); 9 (long-term)
    Interval between chords; optimized within bounds (5,25); the paper identifies chord density as influential.
  • Input rate scaling
    Inner GA parameter with bounds (1,10); final fitted value not reported in the paper.
  • Reaction rate scaling
    Inner GA parameter with bounds (1,10); final fitted value not reported.
  • Outflow rate
    Inner GA parameter with bounds (0.05,1.0); final fitted value not reported.
  • Reaction rate
    Inner GA parameter with bounds (0.1,1.0); final fitted value not reported.
assumptions (3)
  • standard math The Gillespie stochastic simulation algorithm correctly produces the molecule-count time series for the abstract reaction networks.
    The framework's reservoir states are generated by MØD-StochSim, described in Section 2.1; the paper relies on this simulation without derivation.
  • domain assumption Pseudo-molecules and pseudo-rules in MØD provide a faithful abstract model of chemical reaction networks for the purpose of reservoir dynamics.
    Section 2.1 defines the reservoir topology using abstract analogs; the paper explicitly states that modeling real chemistry is outside the scope, so this is an assumption about the abstraction's usefulness.
  • domain assumption The short- and long-term memory tasks defined by Eqs. (1) and (2) are valid proxies for the echo state property.
    Section 2.3 defines the benchmarks and Section 4 interprets NRMSE results as evidence that the reservoir 'meets the echo state property'; no direct ESP verification is provided.
invented entities (2)
  • Pseudo-molecule nodes
    purpose: Serve as the nodes of the reservoir topology in ChemReservoir.
    Introduced in Section 2.1 as abstract analogs of molecules; the paper disclaims real chemistry, so these entities have no independent experimental handle.
  • Pseudo-reaction rule edges
    purpose: Serve as edges connecting reservoir nodes, analogous to chemical reactions.
    Same section; abstract constructs with no empirical grounding beyond the simulation itself.

how reviews work

0 comments
Cite this review

Pith. "Pith review of ChemReservoir -- An Open-Source Framework for Chemically-Inspired Reservoir Computing." pith.science (2026). https://pith.science/paper/ZEFLXIJN

@misc{pith2026250604249,
  author       = {Pith},
  title        = {Pith review of: ChemReservoir -- An Open-Source Framework for Chemically-Inspired Reservoir Computing},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/ZEFLXIJN}},
  note         = {Machine review of arXiv:2506.04249}
}
read the original abstract

Reservoir computing is a type of a recurrent neural network, mapping the inputs into higher dimensional space using fixed and nonlinear dynamical systems, called reservoirs. In the literature, there are various types of reservoirs ranging from in-silico to in-vitro. In cheminformatics, previous studies contributed to the field by developing simulation-based chemically inspired in-silico reservoir models. Yahiro used a DNA-based chemical reaction network as its reservoir and Nguyen developed a DNA chemistry-inspired tool based on Gillespie algorithm. However, these software tools were designed mainly with the focus on DNA chemistry and their maintenance status has limited their current usability. Due to these limitations, there was a need for a proper open-source tool. This study introduces ChemReservoir, an open-source framework for chemically-inspired reservoir computing. In contrast to the former studies focused on DNA-chemistry, ChemReservoir is a general framework for the construction and analysis of chemically-inspired reservoirs, which also addresses the limitations in these previous studies by ensuring enhanced testing, evaluation, and reproducibility. The tool was evaluated using various cycle-based reservoir topologies and demonstrated stable performance across a range of configurations in memory capacity tasks.

Figures

Figures reproduced from arXiv: 2506.04249 by the authors.

Figure 1
Figure 1. Reservoir network with 30 nodes, chord length 10, and chord step size 5. Cycle edges are colored blue, chords [PITH_FULL_IMAGE:figures/full_fig_p003_1.png] view at source ↗
Figure 2
Figure 2. The ChemReservoir Workflow. 2.3 Memory Capacity Tasks Memory capacity tasks are widely used benchmarks in reservoir computing, especially the simplified versions of the NARMA benchmark, such as short-memory and long-memory tasks [11]. These tasks evaluate the memory capacity of reservoir models based on their learning performance in the regression step. This also provides a way to assess the fading memory of the res… view at source ↗
Figure 3
Figure 3. The inflow values over 50 s [PITH_FULL_IMAGE:figures/full_fig_p006_3.png] view at source ↗
Figures from the paper (5 more)
Figure 4
Figure 4. Figure 4: Minimum fitness scores across generations for the topology optimization in the short-term memory task. [PITH_FULL_IMAGE:figures/full_fig_p007_4.png]
Figure 5
Figure 5. Figure 5: Minimum fitness scores across generations for the network parameter optimization of the most optimal [PITH_FULL_IMAGE:figures/full_fig_p007_5.png]
Figure 6
Figure 6. Figure 6: Long-term memory target data for tau 24. [PITH_FULL_IMAGE:figures/full_fig_p008_6.png]
Figure 7
Figure 7. Figure 7: Minimum fitness scores across generations for the topology optimization for tau 6 in the long-term memory [PITH_FULL_IMAGE:figures/full_fig_p008_7.png]
Figure 8
Figure 8. Figure 8: Minimum fitness scores across generations for the network parameter optimization of the most optimal [PITH_FULL_IMAGE:figures/full_fig_p009_8.png]

Discussion (0). Sign in to comment.

Reference graph

Works this paper leans on

15 extracted references · 14 canonical work pages

  1. [1]

    Prediction of chaotic time series using recurrent neural networks and reservoir computing techniques: A comparative study.Machine learning with applications, 8:100300, 2022

    Shahrokh Shahi, Flavio H Fenton, and Elizabeth M Cherry. Prediction of chaotic time series using recurrent neural networks and reservoir computing techniques: A comparative study.Machine learning with applications, 8:100300, 2022

    work page 2022

  2. [2]

    Temporal information transformed into a spatial code by a neural network with realistic properties.Science, 267(5200):1028–1030, 1995

    Dean V Buonomano and Michael M Merzenich. Temporal information transformed into a spatial code by a neural network with realistic properties.Science, 267(5200):1028–1030, 1995

    work page 1995

  3. [3]

    Citeseer, 2002

    Herbert Jaeger.Tutorial on training recurrent neural networks, covering BPPT, RTRL, EKF and the echo state network approach, volume 5. Citeseer, 2002

    work page 2002

  4. [4]

    A survey on reservoir computing and its interdisciplinary applica- tions beyond traditional machine learning.IEEE Access, 11:81033–81070, 2023

    Heng Zhang and Danilo Vasconcellos Vargas. A survey on reservoir computing and its interdisciplinary applica- tions beyond traditional machine learning.IEEE Access, 11:81033–81070, 2023

    work page 2023

  5. [5]

    Reservoir computing approaches to recurrent neural network training

    Mantas Lukoševiˇcius and Herbert Jaeger. Reservoir computing approaches to recurrent neural network training. Computer science review, 3(3):127–149, 2009

    work page 2009

  6. [6]

    Complex chemical reaction networks for future information processing

    Katja-Sophia Csizi and Emanuel Lörtscher. Complex chemical reaction networks for future information processing. Frontiers in Neuroscience, 18:1379205, 2024

    work page 2024

  7. [7]

    Chemical reservoir computation in a self-organizing reaction network.Nature, 631(8021):549–555, 2024

    Mathieu G Baltussen, Thijs J de Jong, Quentin Duez, William E Robinson, and Wilhelm TS Huck. Chemical reservoir computation in a self-organizing reaction network.Nature, 631(8021):549–555, 2024

    work page 2024

  8. [8]

    Reservoir computing with random chemical systems

    Hoang Nguyen, Peter Banda, Darko Stefanovic, and Christof Teuscher. Reservoir computing with random chemical systems. InArtificial Life Conference Proceedings 32, pages 491–499, Cambridge, MA, USA, 2020. MIT Press

    work page 2020

Show all 15 references
  1. [9]

    A reservoir computing approach for molecular computing

    Wataru Yahiro, Nathanael Aubert-Kato, and Masami Hagiya. A reservoir computing approach for molecular computing. InArtificial Life Conference Proceedings, pages 31–38, Cambridge, MA, USA, 2018. MIT Press

    2018

  2. [10]

    Minimum complexity echo state network.IEEE transactions on neural networks, 22(1):131–144, 2010

    Ali Rodan and Peter Tino. Minimum complexity echo state network.IEEE transactions on neural networks, 22(1):131–144, 2010

    2010

  3. [11]

    Reservoir computing benchmarks: a tutorial review and critique.International Journal of Parallel, Emergent and Distributed Systems, pages 1–39, 2025

    Chester Wringe, Martin Trefzer, and Susan Stepney. Reservoir computing benchmarks: a tutorial review and critique.International Journal of Parallel, Emergent and Distributed Systems, pages 1–39, 2025

    2025

  4. [12]

    A software package for chemically inspired graph transformation

    Jakob L Andersen, Christoph Flamm, Daniel Merkle, and Peter F Stadler. A software package for chemically inspired graph transformation. InGraph Transformation: 9th International Conference, ICGT 2016, in Memory of Hartmut Ehrig, Held as Part of STAF 2016, Vienna, Austria, July...

    2016

  5. [13]

    A biology-inspired recurrent oscillator network for computations in high-dimensional state space, 2022

    Felix Effenberger, P Carvalho, I Dubinin, and W Singer. A biology-inspired recurrent oscillator network for computations in high-dimensional state space, 2022

    2022

  6. [14]

    Scikit-learn: Machine learning in python

    Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. Scikit-learn: Machine learning in python. the Journal of machine Learning research, 12:2825–2830, 2011

    2011

  7. [15]

    Deap: Evolutionary algorithms made easy.The Journal of Machine Learning Research, 13(1):2171–2175, 2012

    Félix-Antoine Fortin, François-Michel De Rainville, Marc-André Gardner Gardner, Marc Parizeau, and Christian Gagné. Deap: Evolutionary algorithms made easy.The Journal of Machine Learning Research, 13(1):2171–2175, 2012. 10

    2012

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.