pith. sign in

arxiv: 2605.09609 · v2 · pith:PUMJ7S2Nnew · submitted 2026-05-10 · 💻 cs.LG · math.AG

Minimal Filling Architectures of Polynomial Neural Networks: Counterexamples, Frontier Search, and Defects

Kevin Dao , Jose Israel Rodriguez This is my paper

Pith reviewed 2026-06-30 22:35 UTC · model grok-4.3

classification 💻 cs.LG math.AG
keywords polynomial neural networksminimal filling architecturesunimodal conjectureneurovarietiesdefectsfrontier searchpower activationscounterexamples
0
0 comments X

The pith

Counterexamples disprove the unimodal minimal filling architecture conjecture for polynomial neural networks.

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper aims to refute the idea that every minimal filling architecture for polynomial neural networks with power activations must have unimodal hidden layer widths. A sympathetic reader would care because confirming the conjecture would have allowed designers to restrict their search for efficient networks to a simpler class of shapes. Instead, the authors construct explicit counterexamples by combining a frontier search over possible width sequences with recursive bounds on the dimensions of the neurovarieties realized by each architecture. They further observe that some related architectures display large defects, where the dimension of the realized function space is much smaller than expected.

Core claim

Fixing input and output widths, there exist minimal filling architectures for polynomial neural networks with power activations whose sequences of hidden widths are not unimodal. The counterexamples are located by a frontier search that enumerates architectures while pruning via recursive dimension bounds on neurovarieties, with symbolic computation used to confirm the dimensions and defects.

What carries the argument

Frontier search over candidate architectures, guided by recursive dimension bounds on neurovarieties, which systematically identifies minimal filling architectures and measures their defects.

If this is right

  • Minimal filling architectures can have non-unimodal hidden layer widths.
  • Some subarchitectures exhibit large defects unlike the small defects seen in earlier examples.
  • The combination of frontier search and dimension bounds can locate such counterexamples efficiently.
  • The unimodal conjecture does not hold in general for power activation functions.

Where Pith is reading between the lines

These are editorial extensions of the paper, not claims the author makes directly.

  • Similar counterexamples may exist for other activation functions if the search method is extended.
  • The presence of large defects suggests that dimension alone may not capture all aspects of network expressivity.
  • Network designers may need to explore a wider range of width configurations when seeking minimal architectures.

Load-bearing premise

The frontier search with recursive dimension bounds on neurovarieties correctly identifies all minimal architectures without overlooking smaller ones or miscalculating dimensions.

What would settle it

An explicit smaller architecture than one of the counterexamples that still fills the target function space, or a direct computation showing that one of the reported examples fails to achieve the claimed filling dimension.

Figures

Figures reproduced from arXiv: 2605.09609 by Jose Israel Rodriguez, Kevin Dao.

Figure 1
Figure 1. Figure 1: The first counterexample to the minimal unimodal conjecture ( [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 2
Figure 2. Figure 2: A MFA with d0 = dL = 2 (r = 2) References [1] Y. Arjevani, J. Bruna, J. Kileel, E. Polak, and M. Trager, Geometry and Optimization of Shallow Polynomial Networks, SIAM J. Appl. Algebra Geom., 10 (2026), pp. 174–209. [DOI]. [2] M.-C. Brandenburg, G. Loho, and G. Montufar, The real tropical geometry of neural networks for binary classification, Transactions on Machine Learning Research, (2024). [URL]. [3] G.… view at source ↗
read the original abstract

We provide counterexamples to the unimodal minimal filling architecture conjecture for polynomial neural networks (PNNs) with power activation functions. Fixing the input and output widths, the conjecture states that any minimal filling architecture has unimodal widths for the hidden layers. We found counterexamples via a frontier search, recursive dimension bounds on neurovarieties, and symbolic computation. Notably, several subarchitectures of our main example exhibit large defect, in contrast with the predominantly small-defect behavior observed in prior literature.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit. Tearing a paper down is the easy half of reading it; the pith above is the substance, this is the friction.

Referee Report

1 major / 2 minor

Summary. The paper claims to provide counterexamples to the unimodal minimal filling architecture conjecture for polynomial neural networks (PNNs) with power activation functions. Fixing input and output widths, the conjecture asserts that minimal filling architectures have unimodal hidden-layer widths; the authors identify non-unimodal minimal examples via frontier search, recursive dimension bounds on neurovarieties, and symbolic defect computation, and report that several subarchitectures of their main example exhibit large defects (contrasting prior small-defect observations).

Significance. If the computational identification of minimal architectures is correct, the counterexamples are significant because they directly refute the unimodal conjecture and the large-defect observation challenges the small-defect pattern reported in earlier literature on neurovarieties. The frontier-search plus recursive-bound methodology is a concrete contribution that could be reused for other architecture-minimality questions in algebraic machine learning.

major comments (1)
  1. [Section describing the recursive dimension bounds and frontier search (near the main counterexample)] The central claim that the reported architectures are minimal (and hence constitute counterexamples) rests on the recursive dimension bounds on neurovarieties being tight and the frontier search being exhaustive. The manuscript must supply explicit verification—e.g., a table of achieved versus bounded dimensions for the main example and its subarchitectures, or a proof that the recursion attains the actual dimension—that no smaller candidate was overlooked; without this, the counterexamples are not established.
minor comments (2)
  1. Notation for neurovariety dimension and defect should be introduced once with a short table of symbols; repeated re-definition across sections reduces readability.
  2. The symbolic computation pipeline for defects is mentioned but not accompanied by a short pseudocode or reference to the exact computer-algebra commands used; adding this would aid reproducibility.

Simulated Author's Rebuttal

1 responses · 0 unresolved

We thank the referee for the constructive report and for highlighting the need for stronger verification of minimality. We address the single major comment below and will revise the manuscript accordingly.

read point-by-point responses

  1. Referee: [Section describing the recursive dimension bounds and frontier search (near the main counterexample)] The central claim that the reported architectures are minimal (and hence constitute counterexamples) rests on the recursive dimension bounds on neurovarieties being tight and the frontier search being exhaustive. The manuscript must supply explicit verification—e.g., a table of achieved versus bounded dimensions for the main example and its subarchitectures, or a proof that the recursion attains the actual dimension—that no smaller candidate was overlooked; without this, the counterexamples are not established.

    Authors: We agree that the current presentation leaves the tightness of the recursive bounds and exhaustiveness of the search implicit, which weakens the claim that the reported architectures are minimal. In the revision we will add an explicit table (in the section on the main counterexample) that lists, for the primary architecture and all its subarchitectures, (i) the dimension obtained by symbolic computation, (ii) the recursive upper bound, and (iii) the gap. We will also insert a short paragraph immediately after the description of the frontier-search procedure explaining why the combination of the recursive bound and the enumeration order guarantees that every candidate with smaller total width has already been ruled out. These additions will make the minimality argument self-contained. revision: yes

Circularity Check

0 steps flagged

No circularity: computational counterexamples stand independently

full rationale

The paper's core result consists of explicit computational counterexamples to the unimodal minimal filling architecture conjecture, obtained via frontier search, recursive dimension bounds, and symbolic defect computation. No quoted equations, definitions, or claims reduce any reported architecture or defect value to a fitted parameter, self-citation chain, or input by construction. The methods are presented as external verification tools rather than tautological renamings or ansatzes smuggled from prior self-work; the finding therefore remains self-contained against external benchmarks.

Axiom & Free-Parameter Ledger

0 free parameters · 1 axioms · 0 invented entities

The central claim rests on the correctness of recursive dimension bounds and the completeness of the frontier search; these are domain assumptions whose verification requires the full manuscript.

axioms (1)
  • domain assumption Recursive dimension bounds on neurovarieties accurately upper-bound the dimensions of the varieties arising from PNN architectures.
    Invoked to certify minimality of the discovered architectures.

pith-pipeline@v0.9.1-grok · 5603 in / 1183 out tokens · 32605 ms · 2026-06-30T22:35:21.694152+00:00 · methodology

Review history (2 revisions) →

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Forward citations

Cited by 1 Pith paper

Reviewed papers in the Pith corpus that reference this work. Sorted by Pith novelty score.

  1. Singular Learning and Occam's Razor in Deep Monomial Networks

    cs.LG 2026-06 unverdicted novelty 5.0

    For large monomial activation degree, critical points in deep fully-connected networks coincide exactly with subnetwork configurations where neurons are inactive or redundant.

Reference graph

Works this paper leans on

16 extracted references · 10 canonical work pages · cited by 1 Pith paper

  1. [1]

    Arjevani, J

    Y. Arjevani, J. Bruna, J. Kileel, E. Polak, and M. Trager , Geometry and O ptimization of S hallow P olynomial N etworks , SIAM J. Appl. Algebra Geom., 10 (2026), pp. 174--209. https://doi.org/10.1137/25M1732994 DOI

    work page doi:10.1137/25m1732994 2026

  2. [2]

    Brandenburg, G

    M.-C. Brandenburg, G. Loho, and G. Montufar , The real tropical geometry of neural networks for binary classification , Transactions on Machine Learning Research, (2024). https://openreview.net/forum?id=I7JWf8XA2w URL

    2024

  3. [3]

    G. G. Chrysos, S. Moschoglou, G. Bouritsas, J. Deng, Y. Panagakis, and S. Zafeiriou , Deep polynomial neural networks , IEEE Trans. Pattern Anal. Mach. Intell., 44 (2022), p. 4021–4034. https://doi.org/10.1109/TPAMI.2021.3058891 DOI

    work page doi:10.1109/tpami.2021.3058891 2022

  4. [4]

    Finkel, J

    B. Finkel, J. I. Rodriguez, C. Wu, and T. Yahl , A ctivation degree thresholds and expressiveness of polynomial neural networks , Algebraic Statistics, 16 (2025), pp. 113--130. https://doi.org/10.2140/astat.2025.16.113 DOI

    work page doi:10.2140/astat.2025.16.113 2025

  5. [5]

    Goyal, R

    M. Goyal, R. Goyal, and B. Lall , I mproved P olynomial N eural N etworks with N ormalised A ctivations , in 2020 International Joint Conference on Neural Networks (IJCNN), 2020, pp. 1--8. https://doi.org/10.1109/IJCNN48605.2020.9207535 DOI

    work page doi:10.1109/ijcnn48605.2020.9207535 2020

  6. [6]

    Henry, G

    N. Henry, G. L. Marchetti, and K. Kohn , Geometry of lightning self-attention: Identifiability and dimension , in International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu, eds., vol. 2025, 2025, pp. 14400--14416. https://proceedings.iclr.cc/paper_files/paper/2025/file/259e59fe23ebd09252647fed42949182-Paper-Conferen...

    2025

  7. [7]

    F. A. Hossain and T. Rahman , A T raining F ramework for O ptimal and S table T raining of P olynomial N eural N etworks , 2025. https://arxiv.org/abs/2505.11589 arXiv: 2505.11589

    work page arXiv 2025

  8. [8]

    Kileel, M

    J. Kileel, M. Trager, and J. Bruna , O n the E xpressive P ower of D eep P olynomial N eural N etworks , in Advances in Neural Information Processing Systems, vol. 32, 2019. https://proceedings.neurips.cc/paper/2019/file/a0dc078ca0d99b5ebb465a9f1cad54ba-Paper.pdf URL

    2019

  9. [9]

    K. Kohn, T. Merkh, G. Mont\'ufar, and M. Trager , Geometry of linear convolutional networks , SIAM J. Appl. Algebra Geom., 6 (2022), pp. 368--406. https://doi.org/10.1137/21M1441183 DOI

    work page doi:10.1137/21m1441183 2022

  10. [10]

    Kubjas, J

    K. Kubjas, J. Li, and M. Wiesmann , G eometry of P olynomial N eural N etworks , Algebraic Statistics, 15 (2024), pp. 295--328. https://doi.org/10.2140/astat.2024.15.295 DOI

    work page doi:10.2140/astat.2024.15.295 2024

  11. [11]

    G. L. Marchetti, V. Shahverdi, S. Mereta, M. Trager, and K. Kohn , P osition: A lgebra U nveils D eep L earning -- A n I nvitation to N euroalgebraic G eometry , in Proceedings of the 42nd International Conference on Machine Learning, PMLR, 2025. https://doi.org/10.48550/arXiv.2501.18915 DOI

    work page doi:10.48550/arxiv.2501.18915 2025

  12. [12]

    The alexander-hirschowitz theorem for neu- rovarieties.arXiv:2511.19703,

    A. Massarenti and M. Mella , The A lexander- H irschowitz theorem for neurovarieties , arXiv preprint arXiv:2511.19703, (2025). https://doi.org/10.48550/arXiv.2511.19703 DOI

    work page doi:10.48550/arxiv.2511.19703 2025

  13. [13]

    Usevich, R

    K. Usevich, R. Borsoi, C. D\' e rand, and M. Clausel , Identifiability of deep polynomial neural networks , in Advances in Neural Information Processing Systems, vol. 38, Curran Associates, Inc., 2025, pp. 81809--81858. https://openreview.net/forum?id=MrUsZfQ9pC URL

    2025

  14. [14]

    Zhang, G

    L. Zhang, G. Naitzat, and L.-H. Lim , Tropical geometry of deep neural networks , in Proceedings of the 35th International Conference on Machine Learning, J. Dy and A. Krause, eds., vol. 80 of Proceedings of Machine Learning Research, PMLR, 10--15 Jul 2018, pp. 5824--5832. https://proceedings.mlr.press/v80/zhang18i.html URL

    2018

  15. [15]

    J. Zhou, H. Qian, X. Lu, Z. Duan, H. Huang, and Z. Shao , Polynomial activation neural networks: M odeling, stability analysis and coverage BP -training , Neurocomputing, 359 (2019), pp. 227--240. https://doi.org/10.1016/j.neucom.2019.06.004 DOI

    work page doi:10.1016/j.neucom.2019.06.004 2019

  16. [16]

    write newline

    " write newline "" before.all 'output.state := FUNCTION fin.entry add.period write newline FUNCTION new.block output.state before.all = 'skip after.block 'output.state := if FUNCTION not #0 #1 if FUNCTION and 'skip pop #0 if FUNCTION or pop #1 'skip if FUNCTION new.block.checka empty 'skip 'new.block if FUNCTION field.or.null duplicate empty pop "" 'skip ...