Pith. sign in

REVIEW 4 major objections 5 minor 28 references

MOGNET: A Mux-residual quantized Network leveraging Online-Generated weights

T0 review · 4 major / 5 minor · reviewed 2026-08-10 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗ Review JSON

Pith's one-line read MOGNET claims a sub-2-Mb quantized network beats larger compressed models on CIFAR.

desk verdict The architecture is real and the paper is honest, but the fixed CA-generated weights are never tested, and single-run numbers can't carry a 1% claim. read the letter →

arxiv 2501.09531 v1 pith:DI4ABPTF submitted 2025-01-16 cs.LG cs.AR

Van Thien Nguyen , William Guicquero , Gilles Sicard This is my paper · ORCID
classification cs.LGcs.AR
keywords quantizedneuralnetworkscellularautomatonweightsternaryquantizationmodelcompressionhardware-efficientdeeplearningCIFAR-10CIFAR-100edgeinference
verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

This paper proposes MOGNET, a convolutional architecture designed to run image classification on chips with very little memory. The central claim is that a sub-2-Mb model can match or beat recent compressed networks: the full configuration reports 92.12% on CIFAR-10 and 68.80% on CIFAR-100 at 1.72 Mb, roughly one point above the SLBF-ResNet-18 baseline at a similar size. The savings come from generating part of the weights on the fly with a cellular automaton, so they are never stored, and from a multiplexer-based residual path that keeps activations at 3 bits without extra hardware. The paper also introduces a balanced ternary quantization scheme that sets its step size from the tertiles of the weight distribution during training.

What carries the argument

The CFLOG factorization is the backbone: two pointwise convolutions around a grouped convolution, with the second pointwise kernel fixed and generated online by a one-dimensional cellular automaton (rule 30), so the compression rate in Eq. (4) counts those weights as free. The MUX Residual Block (MRB) is an attention-like skip: a channel-wise binary signal from thresholded global average pooling chooses, per channel, between the residual input and the block output, and a BitShift halves the sum so the activation stays in k-bit range. Balanced Ternary Quantization (BTQ) sets the ternary step size each epoch to |q1|+|q2|, the sum of the absolute tertiles of the proxy weights, pushing the quantized levels toward equal occupancy.

What would settle it

Train the same MOGNET configuration with the CA kernel replaced by (1) a learned 1x1 kernel of equal size, (2) a fixed random matrix initialized from a seeded RNG, and (3) a different CA rule or seed; if (2) or (3) match Rule 30's accuracy at 1.72 Mb, the specific CA mechanism is not load-bearing, and if (1) clearly exceeds it, the fixed projection is costing accuracy that a stored kernel would recover.

Watch

Extended reading notes

Core claim

MOGNET replaces every internal convolution except the first and last with a CFLOG block: a 1x1 projection to a low dimension, a grouped convolution, then a 1x1 projection back to full width whose kernel is not trained but produced in real time by Wolfram's rule-30 cellular automaton from a seed. Because this kernel is fixed and generated, its parameters cost no storage, which is what brings the model under 2 Mb. Residual connections are integrated through a multiplexer steered by thresholded global average pooling and a bit-shift rescale, so activations stay at k bits and the operation maps to simple hardware. With 3-bit activations, binarized pointwise and ternarized grouped weights, the paper reports 92.12% on CIFAR-10 and 68.80% on CIFAR-100 at 1.72 Mb, compared with 91.70% and 67.89% for SLBF-ResNet-18 at a similar reported size. It also finds that shrinking activations to 1 or 2 bits costs several points of accuracy.

Load-bearing premise

The load-bearing premise is that fixed, untrained Rule-30 CA-generated projection weights are expressive enough to replace a learned final pointwise convolution; the paper reports no experiment that varies the seed, rule, or a learned substitute.

Editorial extensions

If this is right

  • At 1.72 Mb with 3-bit activations, the n=128, g=4 MOGNET reaches 92.12% on CIFAR-10 and 68.80% on CIFAR-100, beating SLBF-ResNet-18 by about a point at similar size.
  • The CA-generated final pointwise kernel removes an entire class of stored parameters from each block, so the memory saving scales with the number of blocks.
  • Reducing activation precision to 2 bits or 1 bit degrades accuracy substantially (91.16% and 88.99% on CIFAR-10), so 3-bit activations appear necessary for the claimed accuracy.
  • Beyond n=128, MOGNET's accuracy-versus-size curve falls below SLBF-RN18, so the reported advantage is specific to the sub-2-Mb regime.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

  • The paper does not ablate the CA kernel choice: a fixed random matrix or another CA rule might give the same accuracy, which would mean the specific cellular-automaton mechanism is not the source of the gain; a direct comparison would separate the storage trick from the projection's representational quality.
  • Because BTQ's tertile rule depends on the proxy weights staying roughly symmetric around zero, it may transfer to other ternary networks as a drop-in quantizer, independent of the CA-generated weights.
  • The MUX-plus-bitshift residual path is a candidate building block for any quantized architecture targeting integer-only hardware, since it replaces an addition with a 2-input mux and a shift.
Share X Bluesky LinkedIn Reddit HN

Signed reviews

No signed human review yet.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Referee Report

4 major / 5 minor

Summary. The paper proposes MOGNET, a compact CNN for resource-limited hardware, combining MUX residual blocks with bitshift rescaling, a convolution factorization (CFLOG) whose final pointwise weights are generated online by a Rule-30 cellular automaton (so they are not stored), and a balanced ternary quantization scheme (BTQ). The authors report CIFAR-10 and CIFAR-100 accuracy and model-size numbers in Table II, claiming sub-2Mb models with a roughly 1% accuracy gap over SLBF/ETD baselines at similar or lower model size. The derivation of the CFLOG compression rate is internally consistent if the displayed formulas are read as containing squared quantities, but the empirical support for the headline claim is incomplete: there are no multi-run statistics, no ablation of the CA-generated weights, and no full architecture specification.

Significance. If the results are reproducible, the idea of replacing a stored weight matrix with online-generated CA states is a genuine and creative contribution to ultra-low-memory inference, and the hardware-oriented MRB design with multiplexer-based residual connections is interesting. The paper provides a clean parameter-count formula, reports comparisons against published methods on two standard benchmarks, and explicitly targets a practical sub-2Mb regime that is relevant to edge-AI hardware. However, the absence of seed-sensitivity studies, ablations, and multi-run statistics currently leaves the central accuracy claim unproven; the method is of moderate significance and its empirical evidence must be hardened before the claim can be accepted.

major comments (4)
  1. [Section III-B, Eq. (4), Fig. 2] The memory savings and the accuracy claim depend on treating the CA-generated pointwise weights as zero-cost parameters, yet the paper provides no evidence that these fixed Rule-30 projections are as expressive as learned or random projections of the same shape. Please add (i) an ablation replacing the CA kernel with a learned ternary kernel of identical dimensions, (ii) at least one alternative random projection, and (iii) a scan over CA seeds and initial conditions, reporting accuracy variation. Without these experiments, the reported gap may be an artifact of a particular CA seed rather than a property of the architecture.
  2. [Section IV-B, Table II] All accuracy numbers in Table II appear to come from a single run per configuration, and the headline comparison on CIFAR-100 (68.80% versus 67.89%) is a 0.91-percentage-point difference. For small quantized CNNs, run-to-run variance is often of this order, so the claim of a "clear gap" is not statistically supported. Please report the mean and standard deviation over at least three independent training runs for each configuration, or otherwise demonstrate that the gap exceeds the noise floor.
  3. [Section III, Fig. 1] The manuscript does not specify the complete network configuration: the number of MRB/CFLOG blocks, the channel schedule, the placement of max pooling, the exact first and last layers, and the CA seed/initial state are not given. This makes the experiments irreproducible and prevents independent verification of the model-size calculations in Eq. (4) and Table II. A layer-by-layer specification and the exact CA setup should be included in the revised manuscript.
  4. [Section IV, Table II] The term "model size" appears to count only weight memory, while the abstract and introduction claim reductions in "on-chip required memory." Because activations also occupy memory in a hardware implementation, and MOGNET uses 3-bit activations while the baselines use 32-bit activations, the total-memory comparison may differ materially from the weight-only comparison. Please state explicitly whether Table II excludes activation memory, and if the claim is about total on-chip memory, provide an estimate that includes activation buffers.
minor comments (5)
  1. [Section III-B, Eq. (4)] The displayed formula appears to have lost superscripts: "32m^2/g" and "32CiCo" should likely read "3^2 m^2/g" and "3^2 C_i C_o" (i.e., 9) for a 3x3 convolution, and the simplified expression on the right is consistent with this corrected reading; the typesetting should be fixed.
  2. [Section III-C, Eq. (6)] The symmetry assumption q1 ≈ -q2 is central to the BTQ step-size update, but no empirical validation (e.g., plots of proxy-weight histograms during training) is provided; adding such evidence would strengthen the novelty claim.
  3. [Section III-A] The symbol m is used both as the latent dimension in CFLOG and as the maximum value in the TGAP threshold; this overloaded notation is confusing and should be disambiguated.
  4. [Table I] The learning-rate schedules are underspecified (e.g., whether the exponential decay is applied per epoch or per iteration); please clarify the schedule details.
  5. [General] The arXiv header includes the IEEE AICAS 2022 copyright notice and DOI; the authors should state explicitly what, if anything, has been updated in this arXiv version relative to the published conference paper.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the reported results are empirical comparisons against external baselines, and the memory savings follow from parameter-count arithmetic plus fixed, untrained CA-generated weights.

full rationale

The paper's central claims are empirical: MOGNET's accuracy is measured on CIFAR-10/100 and compared against published results from SLBF, ETD, and other external methods. No quantity predicted by the paper is defined in terms of the target accuracy or fitted to it. The compression rate in Eq. (4) is a direct parameter-count formula comparing CFLOG to a regular convolution; it is derived algebraically from the stated layer shapes (Ci, Co, m, g), not from accuracy data. The CA-generated pointwise weights are generated by Wolfram rule 30 with a fixed seed and are explicitly not trained ('fixed during training and generated in real-time by a CA'), so the storage saving is not a hidden fit. The BTQ step size in Eq. (6) is a heuristic updated from tertiles of proxy-weight histograms during training; it is a training procedure, not a prediction that is then re-derived from the result. The MUX residual block and CFLOG are presented as architectural designs with hardware motivations, and the comparisons in Table II are against externally reported numbers. There is no self-citation chain supporting the main result: the method is evaluated directly by experiments, and no uniqueness theorem or load-bearing prior claim from the same authors is invoked. The absence of an ablation or seed-sensitivity study for the CA-generated weights is a legitimate evidence gap, but it is not circularity under the stated criteria; the weights are fixed inputs to training rather than quantities fitted to the final accuracy.

Assumptions & free parameters 5 free parameters · 5 assumptions · 0 invented entities

The ledger contains five hand-chosen architecture hyperparameters (n, g, k, m, CA seed) because the central sub-2Mb accuracy claim is demonstrated only at the reported values. Five axioms are explicit or implicit assumptions: STE gradients, ternary histogram symmetry, expressiveness of untrained CA weights, usefulness of TGAP, and the validity of weight-only memory as the footprint metric. No invented physical entities are introduced; the novel components are algorithmic constructions, handled in the axioms and red flags.

free parameters (5)
  • n (CFLOG output feature width) = 128
    Hand-chosen architecture width in reported experiments; Fig. 4 shows accuracy drops for larger n, so the headline sub-2Mb result depends on this value.
  • g (number of groups in grouped convolution) = 4 (best), 8
    Reported results use g=4 for best accuracy and g=8 for smaller size; the comparison depends on this selection.
  • k (activation bitwidth) = 3 for headline
    Accuracy degrades substantially at k=1 or k=2 in Table II, so the claimed gap depends on 3-bit activations.
  • m (latent dimension in CFLOG) = Ci/2
    Set to half the input channels in Eq. (4); no ablation shows this is optimal.
  • CA seed or initial configuration = not reported
    Rule 30-generated weights depend on the initial CA state; without a stated seed, exact replication is impossible.
assumptions (5)
  • domain assumption Straight-through estimators for QReLU and Bitshift give useful gradients for training quantized networks.
    Invoked after Eq. (1) and Eq. (2); this is a standard approximation, not a proven convergence guarantee, and the paper does not validate it with ablations.
  • ad hoc to paper Ternary proxy-weight histograms are symmetric around zero, so tertiles satisfy q1 approximately -q2 and s = |q1| + |q2|.
    Section III-C and Algorithm 1; if this symmetry fails for some layers, step sizes are miscalibrated. No layer-by-layer evidence is provided.
  • ad hoc to paper Untrained Rule-30 CA-generated weights form an expressive projection for the final pointwise convolution.
    Section III-B and Fig. 2; all memory savings depend on replacing a learned pointwise layer with a fixed generator, but no theoretical or ablation support is given.
  • domain assumption Global average pooling thresholded at 0.5m yields a useful channel-attention-like control signal.
    Section III-A, TGAP definition; the paper asserts this balances large and small activations but provides no quantitative analysis.
  • domain assumption Weight-only model size is the correct proxy for on-chip memory, and CA-generated weights can be ignored in the footprint.
    Table II compares weight model sizes while the text claims on-chip memory reduction; total memory including activation buffers is not reported.

how reviews work

0 comments
Cite this review

Pith. "Pith review of MOGNET: A Mux-residual quantized Network leveraging Online-Generated weights." pith.science (2026). https://pith.science/paper/DI4ABPTF

@misc{pith2026250109531,
  author       = {Pith},
  title        = {Pith review of: MOGNET: A Mux-residual quantized Network leveraging Online-Generated weights},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/DI4ABPTF}},
  note         = {Machine review of arXiv:2501.09531}
}
read the original abstract

This paper presents a compact model architecture called MOGNET, compatible with a resource-limited hardware. MOGNET uses a streamlined Convolutional factorization block based on a combination of 2 point-wise (1x1) convolutions with a group-wise convolution in-between. To further limit the overall model size and reduce the on-chip required memory, the second point-wise convolution's parameters are on-line generated by a Cellular Automaton structure. In addition, MOGNET enables the use of low-precision weights and activations, by taking advantage of a Multiplexer mechanism with a proper Bitshift rescaling for integrating residual paths without increasing the hardware-related complexity. To efficiently train this model we also introduce a novel weight ternarization method favoring the balance between quantized levels. Experimental results show that given tiny memory budget (sub-2Mb), MOGNET can achieve higher accuracy with a clear gap up to 1% at a similar or even lower model size compared to recent state-of-the-art methods.

Figures

Figures reproduced from arXiv: 2501.09531 by the authors.

Figure 1
Figure 1. Top-level architecture description of MOGNET with [PITH_FULL_IMAGE:figures/full_fig_p002_1.png] view at source ↗
Figure 3
Figure 3. Balanced ternary quantization with histogram bin [PITH_FULL_IMAGE:figures/full_fig_p003_3.png] view at source ↗
Figure 2
Figure 2. CFLOG description with CA-generated weights. [PITH_FULL_IMAGE:figures/full_fig_p003_2.png] view at source ↗
Figures from the paper (1 more)
Figure 4
Figure 4. Figure 4: Test accuracy of different compression method-model [PITH_FULL_IMAGE:figures/full_fig_p004_4.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

28 extracted references · 18 canonical work pages

  1. [1]

    Xception: Deep learning with depthwise separable convo- lutions,

    F. Chollet, “Xception: Deep learning with depthwise separable convo- lutions,” in CVPR, 2017

    work page 2017

  2. [2]

    Quantized neural networks: Training neural networks with low preci- sion weights and activations,

    I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y . Bengio, “Quantized neural networks: Training neural networks with low preci- sion weights and activations,” ArXiv, vol. abs/1609.07061, 2017

    arXiv 2017

  3. [3]

    Learning both weights and connections for efficient neural networks,

    S. Han, J. Pool, J. Tran, and W. J. Dally, “Learning both weights and connections for efficient neural networks,” CoRR, 2015

    work page 2015

  4. [4]

    Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding,

    S. Han, H. Mao, and W. J. Dally, “Deep compression: Compressing deep neural network with pruning, trained quantization and huffman coding,” in ICLR, 2016

    work page 2016

  5. [5]

    Scalable model compression by entropy penalized reparameterization,

    D. Oktay, J. Ball ´e, S. Singh, and A. Shrivastava, “Scalable model compression by entropy penalized reparameterization,” in ICLR, 2020

    work page 2020

  6. [6]

    Deep residual learning for image recognition,

    K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in CVPR, 2016

    2016

  7. [7]

    Residual attention network for image classification,

    F. Wang, M. Jiang, C. Qian, S. Yang, C. Li, H. Zhang, X. Wang, and X. Tang, “Residual attention network for image classification,” in CVPR, 2017

    work page 2017

  8. [8]

    Squeeze-and-excitation networks,

    J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in CVPR, 2018

    2018

Show all 28 references
  1. [9]

    Ternary weight networks,

    F. Li and B. Liu, “Ternary weight networks,” ArXiv, vol. abs/1605.04711, 2016

    2016 arXiv

  2. [10]

    Linear symmetric quantization of neural networks for low-precision integer hardware,

    X. Zhao, Y . Wang, X. Cai, C. Liu, and L. Zhang, “Linear symmetric quantization of neural networks for low-precision integer hardware,” in ICLR, 2020

    2020

  3. [11]

    Learned step size quantization,

    S. K. Esser, J. L. McKinstry, D. Bablani, R. Appuswamy, and D. S. Modha, “Learned step size quantization,” in ICLR, 2020

    2020

  4. [12]

    Sifret, Rigid-Motion Scattering For Image Classification

    L. Sifret, Rigid-Motion Scattering For Image Classification . Ecole Polytechnique, CMAP PhD thesis, 2014

    2014

  5. [13]

    Mobilenets: Efficient convolutional neural networks for mobile vision applications,

    A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” ArXiv, vol. abs/1704.04861, 2017

    2017 arXiv

  6. [14]

    Rethinking depthwise separable convolu- tions: How intra-kernel correlations lead to improved mobilenets,

    D. Haase and M. Amthor, “Rethinking depthwise separable convolu- tions: How intra-kernel correlations lead to improved mobilenets,” in CVPR, 2020

    2020

  7. [15]

    Shufflenet: An extremely efficient convolutional neural network for mobile devices,

    X. Zhang, X. Zhou, M. Lin, and J. Sun, “Shufflenet: An extremely efficient convolutional neural network for mobile devices,” in CVPR, 2018

    2018

  8. [16]

    Aggregated residual transformations for deep neural networks,

    S. Xie, R. Girshick, P. Doll ´ar, Z. Tu, and K. He, “Aggregated residual transformations for deep neural networks,” in CVPR, 2017

    2017

  9. [17]

    Wolfram, A New Kind of Science

    S. Wolfram, A New Kind of Science. Champaign, Ilinois, USA: Wolfram Media Inc., 2002

    2002

  10. [18]

    Chaotic cellular automaton for generating measure- ment matrix used in cs coding,

    J. Liu and Q. Sun, “Chaotic cellular automaton for generating measure- ment matrix used in cs coding,” IET Signal Process. , vol. 11, 2017

    2017

  11. [19]

    Reservoir computing using cellular automata,

    ¨O. Yılmaz, “Reservoir computing using cellular automata,” ArXiv, vol. abs/1410.0162, 2014

    2014 arXiv

  12. [20]

    Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation,

    Y . Bengio, N. L ´eonard, and A. Courville, “Estimating or Propagating Gradients Through Stochastic Neurons for Conditional Computation,” arXiv:1308.3432 [cs], Aug. 2013

    2013 arXiv

  13. [21]

    Binarized neural networks,

    I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y . Bengio, “Binarized neural networks,” in NeurIPS, 2016

    2016

  14. [22]

    Tensorflow: Large-scale machine learning on heteroge- neous distributed systems,

    M. A. et al., “Tensorflow: Large-scale machine learning on heteroge- neous distributed systems,” ArXiv, vol. abs/1603.04467, 2016

    2016 arXiv

  15. [23]

    Cellpylib: A python library for working with cellular automata,

    L. M. Antunes, “Cellpylib: A python library for working with cellular automata,” Journal of Open Source Software , vol. 6, no. 67, 2021

    2021

  16. [24]

    Learning Multiple Layers of Features from Tiny Im- ages,

    A. Krizhevsky, “Learning Multiple Layers of Features from Tiny Im- ages,” 2009

    2009

  17. [25]

    Adam: A method for stochastic optimization,

    D. P. Kingma and J. Ba, “Adam: A method for stochastic optimization,” CoRR, vol. abs/1412.6980, 2015

    2015 arXiv

  18. [26]

    Compressing deep convolutional neural networks by stacking low-dimensional binary convolution filters,

    W. Lan and L. Lan, “Compressing deep convolutional neural networks by stacking low-dimensional binary convolution filters,” in AAAI, 2021

    2021

  19. [27]

    Very deep convolutional networks for large-scale image recognition,

    K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” CoRR, vol. abs/1409.1556, 2015

    2015 arXiv

  20. [28]

    Towards efficient tensor decomposition-based DNN model compression with optimization frame- work,

    M. Yin, Y . Sui, S. Liao, and B. Yuan, “Towards efficient tensor decomposition-based DNN model compression with optimization frame- work,” in CVPR, 2021

    2021

Pith tools

Reviewed August 10, 2026 · model on record in the stance chip above.