{"id":"a72dfba1-80b1-460b-87c0-fb07071b2cf4","arxiv_id":"2608.07043","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":7,"one_line_summary":"The tnkm JAX library provides a unified, modular implementation of tensor network kernel machines and reports competitive benchmark accuracy with low training cost.","lead":"A new open-source Python library, tnkm, packages tensor network kernel machines in JAX with interchangeable feature maps, tensor network backbones, and optimizers. The authors report competitive accuracy on nonlinear system identification benchmarks with very fast training, though the baseline comparisons and single-run results need scrutiny.","discovery_kind":"extension","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Benchmark comparability is the crux: website baseline RMSEs may not share TNKM's normalization, initialization, or simulation protocol.","rationale":"The reader's weakest assumption correctly identifies the comparability of benchmark baselines as the linchpin of the paper's central empirical claim. My stress-test pass found no internal inconsistency in the TNKM formulation or the software architecture; the mathematics (CP/TT decompositions, ALS updates, Fourier/polynomial features) is standard and clearly presented. The code is promised to be public, which strengthens reproducibility, though no pinned environment or commit hash is given. The single most load-bearing issue is that Tables 2–4 mix independently measured TNKM numbers with website-reported baseline numbers, and the equivalence of the underlying evaluation protocols is asserted rather than demonstrated. This is addressable by a re-run with official evaluation scripts, so the appropriate verdict remains CONDITIONAL as the reader concluded. Since my concern coincides with the reader's identified weakest assumption, no adjustment to the verdict is needed (UNCHANGED).","tokens_in":12724,"tokens_out":4709,"duration_ms":42088,"concrete_test":"Download the official Nonlinear Benchmark datasets and evaluation scripts from nonlinearbenchmark.org. Re-run the TNKM models from Section 4.2 through the official evaluation pipeline, using the website's prescribed normalization, initialization samples, and free-run simulation function, rather than the in-house protocol. Independently re-run one or two baseline methods (e.g., dynoNet and GPNARX) on the same data with the same official evaluation code. Then compare the resulting RMSE values with both the paper's Tables 2–4 and the website's tables. If the TNKM RMSEs reproduce the paper's numbers and the baseline RMSEs reproduce the website's numbers under the identical protocol, the concern is resolved. If the numbers shift, the comparative claim is unsupported.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The central claim that TNKM is 'comparable to the best-performing methods' rests entirely on Tables 2–4 of Section 4.2. TNKM results are generated in-house with a specific protocol: min–max normalization computed on training data only, NARX lags chosen heuristically, and recursive free-run simulation seeded with the first 10 or 50 measured output samples. Baseline RMSE and training-time values, however, are taken directly from the Nonlinear Benchmark website without re-running the baselines under the same protocol. The paper does not show that the website's baseline numbers were produced with the same normalization, the same train/test split, the same NARX initialization length, or the same free-run simulation mode. If, for example, the official benchmark used a different number of initialization samples or a one-step-ahead evaluation for some methods, the reported TNKM RMSEs would not be commensurate with the baseline values. Since almost all empirical support for the framework's competitiveness depends on these cross-method comparisons, this is the single most load-bearing assumption in the paper. The concern is not that the authors intentionally misled; it is that the missing protocol match is a plausible source of spurious 'competitiveness' and remains untested in the manuscript.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The manuscript presents tnkm, an open-source JAX-based Python library for tensor-network kernel machines (TNKMs), with modular feature maps, CP/TT parameterizations, and ALS/gradient optimizers. It evaluates the library on an Airfoil regression task comparing ALS and Adam against KRR, and on three nonlinear system identification benchmarks (Coupled Electric Drives, Cascaded Tanks, Silverbox) using NARX models with recursive simulation. The paper claims competitive accuracy with compact parameters and training times on the order of seconds or less, based on RMSE comparisons with baseline results taken from the official Nonlinear Benchmark website.","tokens_in":12966,"tokens_out":5908,"duration_ms":52717,"significance":"If the reported accuracy and speed hold, tnkm provides a practical, extensible framework that complements existing tensor-network software. The code is publicly available under a permissive license, and the modular design is a genuine contribution. The main load-bearing issue is the protocol comparability of the baseline numbers; without resolving it, the empirical claims are not yet fully supported.","major_comments":[{"comment":"The central claim that TNKM is 'comparable to the best-performing methods' is based on RMSEs copied from the official Nonlinear Benchmark website, while the TNKM results are produced in-house with min-max normalization, heuristic NARX lags, and recursive simulation initialized with the first 10 or 50 measured output samples. The manuscript provides no evidence that the website baseline RMSEs were computed under the same normalization, train/test split, initialization length, or recursive simulation mode; if any baseline used one-step-ahead prediction or a different output scaling, the RMSE values would not be commensurate. Please report the exact website evaluation protocol and verify that it matches the TNKM protocol, or re-run the baselines under the same protocol; otherwise the accuracy claims should be restricted to internal TNKM comparisons.","section":"Section 4.2, Tables 2-4"},{"comment":"The training-time comparisons are not apples-to-apples: TNKM times are measured on a Dell Latitude laptop, while baseline times are reported as broad ranges ('1-60 sec', '1-60 min') taken from the Nonlinear Benchmark website, with no statement about the hardware, software, or implementation used for those baselines. Thus the headline 'less than one second' versus '1-60 sec' may reflect hardware differences rather than algorithmic efficiency. Please report measured training times for all methods on the same hardware, or explicitly qualify the comparison as indicative only.","section":"Section 4.2, Tables 2-4"},{"comment":"All experimental results consist of single runs. In Table 1, the ALS and Adam comparisons use fixed seeds, and Tables 2-4 report one RMSE value per test set, with no variance across random initializations, data resampling, or hyperparameter perturbations. It is therefore unclear whether observed differences (for example, TNKM 0.347 versus CT-SUBNET 0.306 in Table 3) are meaningful. Please provide means and standard deviations over at least several independent runs, and for the Airfoil experiment over multiple random train/validation splits.","section":"Sections 4.1 and 4.2"}],"minor_comments":[{"comment":"ALS achieves a validation MSE of 0.102, which is lower than the KRR baseline of 0.15, even though the text describes KRR as the unconstrained solution of the underlying regression problem; please clarify the setting (for example, different regularization, feature dimension, or data split) that makes this comparison meaningful.","section":"Section 4.1, Table 1"},{"comment":"Hyperparameter selection is described only as 'heuristically based on partial autocorrelation' and 'cross-validation on the training (validation) data'; please report the cross-validation folds, search grid, criterion, and the exact chosen values for ranks, feature degrees, lag sets, and regularization parameters for each benchmark so that the results are reproducible.","section":"Section 4.2"},{"comment":"There is a typo in the configuration description: 'R=ℓ 2 regularization of the tensor cores v' should read 'with ℓ2 regularization of the tensor cores v'.","section":"Section 4.1"},{"comment":"The TNKM training time is listed as '1-60 sec', unlike '<0.1 sec' in Tables 2 and 3, while the text says 'only several seconds'; please clarify the actual measured training time and why it differs from the other benchmarks.","section":"Table 4"}],"recommendation":"major_revision","confidential_remarks":"The manuscript is not circular: hyperparameters are selected on validation data and comparisons are made against external baselines. The heavy reliance on the authors' own prior TNKM publications is disclosed and is not a concern. The main risk for the editor is that the headline 'competitive' claim could dissolve if the Nonlinear Benchmark website baselines used different evaluation protocols, so I would insist on protocol verification or re-running the baselines under the same pipeline before publication."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"This is a software contribution, not a new method. The TNKM equations are the authors' prior work, and the paper says so openly. What's new is the unified JAX implementation, the modular API, and the benchmark evaluation. On the merits, that's a useful and honest contribution.\n\nThe library design is clean: feature maps, CP/TT backbones, and ALS and gradient optimizers are separated behind a common interface. The Airfoil optimization study is a nice sanity check—ALS beats Adam on time and matches KRR. The system identification results on Coupled Electric Drives, Cascaded Tanks, and Silverbox are competitive with published baselines, and the training times are genuinely impressive (<0.1 sec to a few seconds).\n\nThe main weakness is benchmark comparability. TNKM numbers come from the authors' own protocol—min-max normalization on training data, heuristic NARX lags, free-run simulation seeded with 10 or 50 measured outputs—while the baseline numbers are copied from the Nonlinear Benchmark website. The paper doesn't demonstrate that the website baselines used the same normalization, the same simulation initialization, or the same evaluation mode. If the protocols differ, the RMSE comparisons in Tables 2–4 are not apples-to-apples. This is a load-bearing assumption, and I think the stress-test note is right to flag it. Also, all results are single runs with no error bars; for a library paper that claims robustness, that's a minor but real gap. The hyperparameters are selected on validation data, which is fine, but the lag sets are heuristic—fine for a demo, less fine for a strong claim.\n\nI don't think the circularity concern holds: they restate their own prior formulation, but that's disclosed and expected for a software paper. The citation pattern is fine. The code is promised under MIT with an experiments directory; no commit hash, but that's a minor reproducibility nit.\n\nThe paper is for people who want to use or extend TNKM without reimplementing it from scratch. It's a fair software paper, worth refereeing. I'd like to see the authors either re-run the baselines under their protocol or show that the website protocol matches, and I'd want error bars or at least multiple seeds. Send it to peer review; it deserves a serious referee who will check the benchmark protocol.","headline":"A solid, well-scoped software paper for a JAX TNKM library; the empirical claims are plausible but the benchmark comparisons are only as good as the external baselines they inherit.","tokens_in":13512,"tokens_out":1413,"would_cite":true,"duration_ms":13327,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"An open-source Python library makes tensor network kernel machines competitive with established nonlinear system identification baselines while training in under a second to a few seconds.","keywords":["tensor network kernel machines","nonlinear system identification","JAX","tensor-train decomposition","canonical polyadic decomposition","alternating least squares","NARX models","Python library"],"falsifier":"Run the released code on the three benchmarks while recomputing all baseline RMSE values under the exact min-max normalization, lag sets, and recursive free-run simulation protocol stated in Section 4.2; if any baseline moves enough to push TNKM outside the reported competitive band, the central claim fails.","tokens_in":12495,"feed_emoji":"⚙️","tokens_out":12070,"duration_ms":95201,"temperature":0.7,"pith_summary":"Tensor network kernel machines predict a target by combining per-input nonlinear features through a tensor product and storing the resulting weight vector as a low-rank tensor network. This paper introduces tnkm, an open-source Python library that unifies the components of this model family—polynomial, Fourier, and Volterra feature maps, CP and tensor-train parameterizations, and alternating least squares or gradient-based training—under one interface. The paper's central claim is that these implemented models achieve predictive accuracy comparable to established black-box and gray-box methods on the Coupled Electric Drives, Cascaded Tanks, and Silverbox benchmarks, while training in under a second to a few seconds on a laptop. If correct, this gives practitioners a reproducible and customizable starting point for tensor-network-based nonlinear modeling and system identification.","feed_headline":"Tensor network kernels rival nonlinear benchmarks in seconds","feed_subtitle":"An open-source JAX library trains compact CP and tensor-train models to competitive RMSE on three benchmarks.","key_machinery":"The load-bearing object is the TNKM itself: a model $f(x)=\\phi(x)^\\top w$ in which $\\phi$ is a tensor product of one-dimensional feature maps and $w$ is reinterpreted as a $D$-way tensor stored in either CP or tensor-train form. This storage is what converts an exponential parameter count into one that grows linearly in $D$, and the same multilinear structure is what alternating least squares exploits by solving one core at a time. On the benchmarks the model is trained as a NARX system with min-max normalized inputs and evaluated by recursive free-run simulation, so the reported RMSEs are over multi-step predicted trajectories rather than one-step-ahead errors.","core_discovery":"The paper's central claim is that a unified software implementation makes tensor network kernel machines practical and competitive for nonlinear regression and system identification. A TNKM is the linear-in-features model $f(x)=\\phi(x)^\\top w$ with $\\phi(x)=\\phi^{(D)}(x_D)\\otimes\\cdots\\otimes\\phi^{(1)}(x_1)$, and $w$ stored as a CP or tensor-train decomposition, so a parameter vector that would otherwise grow exponentially is compressed to one that grows linearly in the number of inputs. On the Coupled Electric Drives benchmark the CP model with rank 15 and polynomial degree-3 features reaches RMSE 0.085/0.077 in under 0.1 s; on Cascaded Tanks the same backbone with rank 13 and degree-2 features reaches RMSE 0.347 in under 0.1 s; on Silverbox the tensor-train model with rank 4 and degree-2 features reaches $\\mathrm{RMSE}\\times 10^3$ values of 0.402/0.881/0.368 in one to sixty seconds. These numbers sit close to GPNARX and SUBNET and slightly behind dynoNet and physics-informed NL-LFR, while training orders of magnitude faster. The paper also shows ALS converging close to a kernel ridge regression reference on Airfoil Self-Noise in roughly a third of the time Adam needs.","pith_inferences":["The competitive numbers depend on the official benchmark website's baselines using the same normalization, lag sets, and recursive-simulation protocol; if they do not, the comparison would need to be redone under a shared protocol.","Since feature maps are predefined rather than learned, TNKM accuracy likely transfers best to systems whose nonlinearities resemble polynomial, Fourier, or Volterra structure; a stress test on discontinuous or highly oscillatory dynamics would clarify the boundary.","The reported training times come from one laptop CPU, so scaling to GPUs, very large datasets, or high tensor ranks is not established by this paper.","The modular design invites testing additional backbones such as tensor ring, Tucker, and hierarchical Tucker, which the paper lists as future work rather than demonstrated capabilities."],"forward_implications":["On the three benchmarks, TNKM reaches RMSE comparable to GPNARX and SUBNET while training on a laptop in less than a second to a few seconds, making broad hyperparameter search affordable.","Because the interface separates features, tensor backbones, and optimizers, a practitioner can switch between polynomial, Fourier, and Volterra features or between CP and tensor-train models without rewriting the training loop.","ALS offers a fast deterministic default for least-squares TNKM training, approaching the kernel ridge regression reference on Airfoil Self-Noise in about one third of the time Adam requires; gradient-based methods remain available for other losses.","The tensor-train model on Silverbox maintains stable free-run predictions over roughly 35,000 simulation steps, indicating the compact rank-4 parameterization does not accumulate excessive simulation error.","Releasing the code and experiments means the reported tables can be reproduced and extended by independent users."],"supporting_citations":[{"why":"Introduces Fourier-feature tensor network kernel machines, the learning framework this library generalizes.","marker":"Wesel and Batselier, 2021"},{"why":"Establishes low-rank tensor decompositions as a tool for nonlinear system identification.","marker":"Batselier, 2022"},{"why":"Supplies the tensor-train format used for one of the two supported parameterizations.","marker":"Oseledets, 2011"},{"why":"Supplies the CP decomposition and general tensor algebra used in the library.","marker":"Kolda and Bader, 2009"},{"why":"Provides the alternating least squares update strategy for tensor core optimization.","marker":"Comon et al., 2009"},{"why":"Defines the Cascaded Tanks benchmark and its evaluation setup.","marker":"Schoukens and Noël, 2017"},{"why":"Defines the Silverbox benchmark and its test sequences.","marker":"Wigren and Schoukens, 2013"},{"why":"Defines the Coupled Electric Drives benchmark and reference models.","marker":"Wigren and Schoukens, 2017"},{"why":"Provides the dynoNet architecture used as a baseline in all three benchmark comparisons.","marker":"Forgione and Piga, 2021"},{"why":"Provides the SUBNET deep encoder baseline used in the Silverbox and Cascaded Tanks comparisons.","marker":"Beintema et al., 2021"}],"fun_headline_variants":["Tensor kernels: fast, compact, competitive","JAX library for tensor network kernels","Train compact kernel models in under a second","Open-source TNKM: fast nonlinear system ID","Compact tensor kernels rival benchmarks"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The entire competitive-accuracy claim depends on the baseline RMSE and training-time values taken from the official Nonlinear Benchmark website being directly comparable to the TNKM results, under identical data splits, preprocessing, and evaluation protocol.","fun_headline_variants_meta":{"raw":{"variants":["Tensor kernels: fast, compact, competitive","JAX library for tensor network kernels","Train compact kernel models in under a second","Open-source TNKM: fast nonlinear system ID","Compact tensor kernels rival benchmarks"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000166,"raw_usage":{"total_tokens":1267,"prompt_tokens":969,"completion_tokens":298,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":585,"completion_tokens_details":{"reasoning_tokens":235}},"tokens_in":585,"tokens_out":298,"duration_ms":3757,"temperature":1.0,"reasoning_tokens":235,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-10T15:49:42.889344+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run the released code on the three benchmarks while recomputing all baseline RMSE values under the exact min-max normalization, lag sets, and recursive free-run simulation protocol stated in Section 4.2; if any baseline moves enough to push TNKM outside the reported competitive band, the central claim fails.","supporting_citations":[{"cited_title":null,"cited_arxiv_id":null,"evidence_quote":"Supplies the tensor-train format used for one of the two supported parameterizations."}],"review_version":1}