{"id":"1cca0fa3-b2d7-4e04-9429-ea74be8a1423","arxiv_id":"2607.15916","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"Writing both feature maps and polynomial coefficient tensors as MPOs yields feature-order-independent polynomial models that beat CPD/TT polynomial baselines on most benchmarks.","lead":"(MPO)^2 builds polynomial models for machine learning by writing both the feature transform and the polynomial weights as matrix product operators (tensor networks), so the model is independent of the order the input features are listed. Across tabular regression/classification and image benchmarks it often beats existing tensor-network polynomial models and gives an interpretable alternative to deep nets.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Reported gains over TT-based TNML baselines rest on omitted failures and absent permutation-ablation, so the central 'improves over existing' claim is not yet established.","rationale":"The central claim is the empirical superiority of (MPO)^2 over existing tensor-decomposed polynomial models. That claim cannot be assessed if the primary TT baseline (TNML-P/F) is not successfully run: Table II omits six 'F' results, and Section IV discloses training instability 'across all seeds.' The authors' own explanation for the gains is feature order independence (Conclusions), but no permutation-ablation confirms that TNML is order-sensitive while (MPO)^2 is not. The reader's weakest_assumption about masking/ring bijections is a legitimate secondary gap, but those variants (M-MPO, Ring) are not the main empirical basis for the headline. The right verdict remains CONDITIONAL: the authors should transparently report all baseline trials and provide a direct permutation test. This does not change the reader's verdict, hence UNCHANGED.","tokens_in":18462,"tokens_out":19647,"duration_ms":216279,"concrete_test":"Re-run Tables II and III for TNML-P and TNML-F using the original authors' implementations and hyperparameter budgets from [26] and [30], with a search budget at least as large as that used for (MPO)^2, and report all runs including large negative R^2 or low accuracy. Then, on a dataset where TNML-P trains successfully (e.g., EE), perform a permutation-ablation: shuffle feature order across at least 10 random permutations, retrain TNML-P and G-(MPO)^2, and compare test R^2 distributions. If TNML-P becomes competitive or is insensitive to feature ordering, the reported advantage is not robust.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The abstract claims (MPO)^2 improves over existing tensor decomposition based polynomial models, and the Conclusions attribute this to feature order independence. The empirical support for the TT comparison is incomplete. In Table II, six of ten TNML regression results are flagged 'F' and omitted from the table, with the caption stating these are removed 'for visual clarity ... and lack of significance.' Section IV discloses missing TNML results are 'due to training instability across all seeds for the model specifications we iterated over.' Thus the headline comparison against TT/MPS-based baselines is conducted against a baseline that frequently fails outright in the authors' framework. A baseline that does not run is not a fair yardstick for measuring architectural advantage. Moreover, no permutation-ablation is reported: the paper never permutes input feature ordering to show TNML is order-sensitive while (MPO)^2 is not. The reader's flagged masking/ring bijection proofs (Sec. II-D3, II-E) are also unproved, but they affect secondary M-MPO and Ring variants; the TNML baseline issue directly undermines the primary empirical claim.","agreement_with_reader":"partial"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper proposes (MPO)^2, a tensor-network parameterization for multivariate polynomial regression and classification. The model represents both the polynomial coefficient tensor and an input-feature transformation as matrix product operators (MPOs), with optional structured operators for linear projections, convolutions, and masking of monomial redundancies. The authors claim the resulting polynomial is feature-order independent, more expressive than CPD-based polynomial models, and empirically superior to existing CPD/TT/MPS-based tensor polynomial baselines. Experiments cover 10 regression and 10 classification UCI datasets, plus MNIST/Fashion-MNIST and CIFAR image experiments, with comparisons to TeMPO, CPD, TNML-P/TNML-F, GP, XGBoost, and MLP.","tokens_in":18790,"tokens_out":6024,"duration_ms":54020,"significance":"If the central claims hold, the paper is a useful contribution: it offers a flexible MPO-based architecture with learned feature transforms, a second-order blockwise optimization scheme, and a public codebase. The range of ablations and the inclusion of several structured MPO variants are strengths. However, the headline empirical claim of 'improvement over existing tensor decomposition based polynomial models' is weakened by the frequent failure of the TT/MPS baselines in the reported tables, and the claimed feature-order independence is never directly tested. The unproved masking and ring constructions affect advertised framework components and need rigorous justification.","major_comments":[{"comment":"The comparison against TNML baselines is incomplete. In Table II, six of ten TNML-P regression results are flagged 'F' and omitted, and TNML-F reports only three non-F values. Section IV discloses that missing/flagged results are 'due to training instability across all seeds for the model specifications we iterated over.' A baseline that fails to run on most datasets cannot support the conclusion that (MPO)^2 'improves over existing tensor decomposition based polynomial models.' Please report all seeded outcomes (including large negative values), or restrict the claim to datasets where the baseline is stable, and discuss why instability occurs.","section":"Table II and Section IV"},{"comment":"Feature order independence is a central claimed advantage, but no permutation-ablation experiment is provided. The paper never permutes input feature order for TNML-P/TNML-F or for (MPO)^2 to demonstrate that the former is order-sensitive while the latter is not. The comparison across different datasets is indirect evidence at best. Add experiments that randomly permute feature columns and report test metrics for each ordering, ideally over several permutations.","section":"Section I and Conclusions"},{"comment":"The masking MPO construction is asserted rather than proved. The text states that the Heaviside/hyperdiagonal construction 'can be rewritten' and then 'extract[s] an MPO,' but the claimed one-to-one correspondence between ordered monomials and coefficient tensor entries is not proven for general MPO rank. If the mask duplicates or drops monomials, the parameter-count and expressivity claims for M-(MPO)^2 fail. Please provide a formal proof, or state precisely the conditions under which the construction works, and clarify the boundary bond indices (the current notation is ambiguous).","section":"Section II-D3, Eqs. (17)-(20)"},{"comment":"The claim that 'the coefficients of the optimal solution are permutation invariant with respect to a basis change, belonging to the fully symmetric space' is not justified. Equation (4) is not permutation invariant for arbitrary \\tilde T; the optimal solution need not be fully symmetric unless the data/target has that symmetry, and the linearity of the model alone does not imply it. Moreover, the ring with identical blocks and periodic boundary conditions is only cyclic-invariant, not fully symmetric, and the additional condition that all matrices commute is stated without proof. This underpins the expressivity claim for the Ring variant and needs a corrected argument or an explicit assumption.","section":"Section II-E, Eq. (22)"}],"minor_comments":[{"comment":"There are numerous typos and formatting errors, e.g., 'enalbing', 'whit', inconsistent use of r vs. R as ranks, and the notation in Eq. (15) uses b,n in a context where d,n would be expected. A careful proofreading is needed.","section":"General"},{"comment":"The definition of multivariate polynomial in Eq. (2) uses ordered index ranges d_N ≥ d_{N-1} ≥ ... ≥ d_1, but the subsequent Type I and Type II formulations in Eqs. (3)-(4) sum over all d(N). Clarify how the ordered form relates to the unordered tensor representation, especially when discussing symmetric coefficients.","section":"Section II-B, Eq. (2)"},{"comment":"The complexity analysis would benefit from a summary table with parameter counts for each variant, not just arithmetic costs. Currently the reader must reconstruct the block dimensions from the prose.","section":"Section II-G"},{"comment":"The image experiments report only average accuracy curves without error bars or standard deviations; reporting variability across seeds would strengthen the comparison.","section":"Figure 5"}],"recommendation":"major_revision","confidential_remarks":null},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"I'll keep this short. The (MPO)^2 idea is the real thing: putting a learned MPO between inputs and an MPO coefficient tensor is a natural extension of CPD/TT polynomial regression, and the masking/convolution/ring variants give it a range that CPD and plain MPS do not have. I believe the architecture is novel relative to TeMPO and TNML. The paper also does solid empirical work in one respect: ten regression plus ten classification datasets, several seeds, validation-based hyperparameter search, and a code link that suggests reproducibility.\n\nBut the central comparison to TT/MPS is not yet fair. In Table II, six of ten TNML-P regression results and most TNML-F results are 'F' or missing, and the caption says they were removed for visual clarity and lack of significance; Section IV blames training instability across all seeds. A baseline that fails to train in your framework is not evidence your model is better. The classification table gives TNML numbers and they are clearly worse, which helps, but the paper cherry-picks by omitting regression failures. Also, feature order independence is asserted as the explanation for gains, but there is no permutation ablation showing TNML degrades under feature permutation while (MPO)^2 does not. That claim is load-bearing and it is testable.\n\nThe masking MPO and the ring are weaker parts. The masking construction (Equations 17-20) is stated but not proved to enforce exactly one coefficient per monomial at general rank. The ring relies on an unproved claim that the optimal coefficient tensor is in the fully symmetric space and that the cyclic ring retains it without loss. I suspect both are salvageable, but as written they are assertions. These affect the secondary variants more than the main architecture.\n\nI do not see a circularity problem. MPO expressivity is imported from Oseledets, and the empirical comparisons are direct; the gains are not baked into the construction. The authors disclose limitations honestly, including no rank learning, no Bayesian uncertainty, and no nonlinearities.\n\nWho is this for? People working on tensor-parameterized polynomials and low-rank function approximation, and to a lesser extent tensor-network machine learning. I would not take the abstract's 'improves over existing' at face value, but the framework deserves serious referee time. My recommendation: send it to review, but require the authors to report all TNML outcomes, including failures, add a permutation ablation, and either prove the masking and ring claims or mark them as conjectures and soften the claims.","headline":"The architecture is genuinely new and worth reviewers' time, but the headline 'improves over TT/MPS' claim currently rests on a benchmark where the TT baseline often fails to run and on an unverified order-independence assumption.","tokens_in":19229,"tokens_out":2095,"would_cite":true,"duration_ms":24353,"reading_group":"yes","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"Modeling both polynomial coefficients and feature embeddings as matrix product operators yields feature-order-independent polynomial regressors that beat existing tensor-decomposition baselines.","keywords":["tensor networks","matrix product operators","polynomial regression","multivariate polynomial","feature order independence","tensor decomposition","natural gradient","symmetry mask"],"falsifier":"Enumerate all monomials for a small configuration (e.g., 4 features, degree 3), construct the masking MPO from Θ and I, and check that contracting it with a generic coefficient tensor yields a distinct output index for every non-decreasing multi-index and zero elsewhere. A single duplicate or missing monomial refutes the central construction.","tokens_in":18411,"feed_emoji":"🧮","tokens_out":4398,"duration_ms":45422,"temperature":0.7,"pith_summary":"The paper introduces (MPO)^2, a framework that parameterizes both the polynomial coefficient tensor and a learned linear transformation of the input features as matrix product operators (MPOs). Because the two MPO layers couple to every feature rather than to feature-specific blocks, the resulting polynomial is independent of feature ordering, unlike tensor-train/matrix-product-state baselines. The framework subsumes earlier polynomial-tensor models and adds structured MPOs for projections, convolutions, and symmetry masking. On tabular regression and classification benchmarks, (MPO)^2 outperforms existing CPD- and TT/MPS-based polynomial models on most datasets, while a convolutional variant achieves strong image-classification accuracy with fewer parameters.","feed_headline":"Beats CPD and TT baselines with two stacked MPOs","feed_subtitle":"Feature-order-independent polynomial regression gains on tabular and image benchmarks via learnable projections, convolutions, and masks.","key_machinery":"The matrix product operator (MPO) — a chain of tensors contracted along one shared 'bond' dimension. The polynomial coefficient tensor T is written as one MPO, and the learned feature embedding A as another, so a degree-N polynomial becomes a contraction of the two chains with the input tensor. The masking MPO, built from the Heaviside matrix Θ and the hyperdiagonal tensor I, enforces the ordered sum over non-decreasing feature indices, so each monomial is meant to receive exactly one coefficient.","core_discovery":"The central claim is that an MPO formulation of the full polynomial regression problem — two MPOs, one for coefficients and one for an input-space linear map — is simultaneously more expressive than CPD, free of the feature-order dependence of TT/MPS, and general enough to absorb structured operators as MPO blocks. The authors construct MPO representations for convolutions and a monomial-symmetry mask, and empirically show that the resulting models outperform prior tensor-based polynomial regressors on most of the tested datasets.","pith_inferences":["If the masking-MPO bijection holds rigorously, the construction provides an exact, compact parameterization of the symmetric monomial space; that could be reused as a regularizing vocabulary in other polynomial-function learners, such as kernel methods or symbolic-regression priors.","The invariant ring, invariant to cyclic permutations, could serve as an inductive bias for exchangeable or rotation-symmetric data where full permutation symmetry is too expensive to enforce.","The two-layer MPO recipe generalizes beyond convolutions: any linear operation that commutes with the feature mode can likely be folded into the embedding MPO, suggesting extensions to graph filters, random features, or wavelet transforms.","A direct test of the expressivity claim: on a synthetic high-degree polynomial of known low rank, compare M-(MPO)^2 against an unconstrained MPO with the same parameter budget to see whether the mask restores accuracy without rank inflation."],"forward_implications":["On the tabular benchmarks reported, (MPO)^2 achieves the best or near-best test metrics among all tensor-network polynomial models on most datasets, with CPD and TT/MPS variants clearly behind.","Feature order independence removes the need for manual feature-ordering heuristics that TT/MPS models require.","The framework supports three structured MPO blocks — linear projection, convolution, and symmetric masking — and thereby accommodates inductive biases such as translation invariance.","Both second-order (alternating natural gradient) and first-order (AdamW) optimizers are viable; the paper reports gradient descent as faster and more memory-efficient with comparable accuracy.","On MNIST and Fashion-MNIST, the convolutional (MPO)^2 reaches high test accuracy with substantially fewer parameters than compared tensor-network and CNN-MLP models."],"fun_headline_variants":["Two stacked MPOs beat CPD and TT polynomial baselines","Feature-order-independent polynomial regression via MPO^2","MPO^2: Stacked MPOs make polynomial learning order-agnostic","Beat CPD and TT with feature-order-free MPO^2","MPO^2: Order-independent polynomials with structured operators"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The masking MPO is asserted to give exactly one coefficient per unique monomial at general rank, but this one-to-one mapping is not proven; if it is off, the parameter-count and expressive-power advantages of the masked variant collapse.","fun_headline_variants_meta":{"raw":{"variants":["Two stacked MPOs beat CPD and TT polynomial baselines","Feature-order-independent polynomial regression via MPO^2","MPO^2: Stacked MPOs make polynomial learning order-agnostic","Beat CPD and TT with feature-order-free MPO^2","MPO^2: Order-independent polynomials with structured operators"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000341,"raw_usage":{"total_tokens":1665,"prompt_tokens":646,"completion_tokens":1019,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":390,"completion_tokens_details":{"reasoning_tokens":931}},"tokens_in":390,"tokens_out":1019,"duration_ms":8083,"temperature":1.0,"reasoning_tokens":931,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T21:56:01.094520+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Enumerate all monomials for a small configuration (e.g., 4 features, degree 3), construct the masking MPO from Θ and I, and check that contracting it with a generic coefficient tensor yields a distinct output index for every non-decreasing multi-index and zero elsewhere. A single duplicate or missing monomial refutes the central construction.","supporting_citations":[],"review_version":1}