Pith. sign in

REVIEW 2 major objections 4 minor 31 references

Training the wiring of logic-gate and lookup-table networks cuts gate count nearly fifty-fold while matching MNIST accuracy.

Reviewed by Pith at T0; open to challenge. T0 means a machine referee read the full paper against a public rubric. the ladder, T0–T4 →

T0 review · grok-4.5

2026-07-13 03:17 UTC pith:R3AGOSIO

load-bearing objection Useful systems result: joint connection+gate training cuts LGN size by nearly 50× at matched MNIST accuracy, with honest ablations, though the exact headline numbers come from shallow-only recipes. the 2 major comments →

arxiv 2607.09399 v1 pith:R3AGOSIO submitted 2026-07-10 cs.LG cs.AI

Fully Trainable Deep Differentiable Logic Gate Networks and Lookup Table Networks

classification cs.LG cs.AI
keywords logic gate networkslookup table networkstrainable connectionsstraight-through estimatorbinary neural networksFPGAedge computingdiscretization gap
verification ladder T0 review T1 audit T2 compute T3 formal T4 reserved

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

This paper shows that the connections inside deep differentiable logic-gate networks and lookup-table networks can be learned together with the gate types or table entries, rather than being fixed at random. By keeping a soft probability over a pool of candidate wires for each input pin and then hard-selecting the best wire, the networks reach the same accuracy as far larger fixed-wiring baselines while using only a fraction of the gates. On MNIST a single fully trainable layer of 8000 gates already hits 98.45 percent—matching a 384 000-gate fixed network—and two such layers reach 98.92 percent; the same idea yields 98.88 percent with two layers of 2000 six-input LUTs. Stability to ten layers is obtained by a higher learning rate, straight-through estimators, and the removal of constant-output gates. The practical payoff is far smaller, FPGA-friendly binary models that still train with ordinary back-propagation.

Core claim

Jointly training a soft distribution over candidate connections for every gate or LUT input pin, while simultaneously learning gate types or LUT entries, produces binary networks that match the accuracy of fixed-connection baselines with nearly fifty times fewer gates (one layer of 8000 gates reaches 98.45 percent on MNIST versus 98.47 percent with 384 000 fixed gates) and remain stable up to ten layers once straight-through estimators, a high learning rate and constant-gate removal are used.

What carries the argument

A temperature-controlled softmax over a pool of candidate wires per input pin, hardened by a straight-through estimator so that the forward pass selects only the highest-probability connection while the backward pass still receives gradients from the soft mixture; gate types or LUT entries are learned in parallel by the same soft-to-hard schedule.

Load-bearing premise

That forcing each pin to a single hard wire (and each LUT entry to a binary value) after soft training leaves only a small accuracy drop—the discretization gap stays negligible even for deeper six-input LUT networks.

What would settle it

Train the reported fully connection-trainable LGN and 6-LUTN architectures, measure the test accuracy of the continuous model just before hard selection versus the fully binarized network after hard selection; if the gap grows beyond a few tenths of a percent for networks deeper than four layers, the claim that the final binary model retains the continuous accuracy fails.

Watch this falsifier — get emailed when new claim-graph text bears on it.

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, simulated authors' rebuttal, and a circularity audit.

Referee Report

2 major / 4 minor

Summary. The paper proposes a method to jointly train gate/LUT types and their input connections in deep differentiable logic gate networks (LGNs) and lookup table networks (LUTNs). Connections are selected from a per-pin pool of Nc candidates via a softmax (or STE/argmax) over learned weights, while gate types or LUT entries are optimized in parallel. On Yin-Yang, MNIST and Fashion-MNIST the connection-optimized models are reported to match or exceed fixed-connection baselines with far fewer gates/LUTs (e.g., one layer of 8000 gates at 98.45 % and two layers at 98.92 % on MNIST, claimed nearly 50 imes smaller than Petersen et al. 2022; two layers of 2000 6-LUTs at 98.88 %). Stability to 10 layers for LGNs is obtained by raising the learning rate, applying straight-through estimators, and removing constant-output gate types; an improved sigmoid-annealed LUT neuron is shown to train stably to 6 layers and to need fewer parameters than the original LGN formulation.

Significance. If the gate-count reductions hold under a single consistent training recipe, the work would be a clear advance for resource-constrained and FPGA-targeted inference: it shows that the random fixed wiring used in prior LGN/LUTN literature is a major source of inefficiency and that co-optimizing topology can shrink models by an order of magnitude while preserving accuracy. The multi-benchmark evaluation, 95 % confidence intervals, systematic ablations (learning-rate, STE on gates vs. connections, constant-gate removal, residual init), and the GEMM reformulation of the LUT neuron are concrete engineering contributions that make the approach more practical. The discretization-gap analysis and the explicit comparison of full-precision versus binarized networks further strengthen the empirical foundation.

major comments (2)
  1. [§4.2, abstract, Table 1, Fig. 6] The headline gate-reduction claim (abstract and §4.2: 98.45 % with one layer of 8000 gates, 98.92 % with two layers, “almost 50 times fewer” than Petersen et al.’s 384 k-gate fixed-connection network) is obtained with the non-default recipes ste/c and highLR. The same paragraph explicitly states that these recipes “performed worse on networks beyond two layers deep,” while the only recipe shown to remain stable to 10 layers (ste/gc/no01/res) is the one used for Table 1, whose 8 k-gate entries are 98.3 % (1 layer) and 98.7 % (2 layers). Consequently the 50 imes claim is not demonstrated by the depth-stable method that the paper otherwise promotes. A single consistent recipe that simultaneously reaches the claimed shallow accuracy and remains stable when depth is increased must be reported, or the abstract/§4.2 numbers must be clearly caveated and replaced by the Table-1 figures.
  2. [§5.1.2, §5.3, Table 4, Fig. 8b] For 6-LUTNs the fully-binarized accuracy collapses with depth (Table 4: 98.88 % at 2 layers with Nc=all falls to 93.7 % at 6 layers). The paper correctly notes the growing discretization gap (Fig. 8b, §5.1.2–5.3), yet still presents the shallow 98.88 % figure as a principal result. Either a method that keeps the gap small for deeper 6-LUTNs must be supplied, or the claim of “stable training \ldots up to 6-layer deep networks” should be restricted to the full-precision (non-hardware) model and the binarized numbers reported with the same caveats applied to the LGN headline results.
minor comments (4)
  1. [Table 1] Table 1 reports only the ste/gc/no01/res recipe; the abstract numbers obtained with other recipes never appear in any table. Adding a row or footnote that lists the shallow highLR/ste/c results would remove ambiguity.
  2. [§2.2, §3.2, §4] The temperature schedules for Tc and Tg, the precise residual-init value, and the β-annealing schedule for LUTNs are described only in prose; a short hyper-parameter table would aid reproducibility.
  3. [Fig. 6c] Figure 6c caption defines the discretization gap as max full-precision accuracy minus max binarized accuracy “even if they do not occur on the same epoch.” Clarifying whether early-stopping or last-epoch selection is used would avoid confusion.
  4. [front matter, references] A few typographical inconsistencies appear (e.g., “Peinlaan” vs. “Pleinlaan”, occasional missing spaces around citations). A light copy-edit pass is sufficient.

Circularity Check

0 steps flagged

Empirical methods paper: accuracies measured on external held-out benchmarks; no derivation reduces to its inputs by construction.

full rationale

The paper proposes a training procedure (softmax over candidate connections per pin, jointly with gate/LUT parameters; STE hard selection; temperature annealing; constant-gate trimming; sigmoid annealing of LUT entries) and reports test-set accuracies on Yin-Yang, MNIST and Fashion-MNIST. Those accuracies are not algebraically forced by the training objectives or by any fitted scalar that is later re-used as a “prediction.” Comparisons to fixed-connection LGNs cite Petersen et al. (independent authors) and to the authors’ own prior LUTN model (Mommen et al. 2026) only as baselines that are re-implemented and re-measured; the load-bearing claims are the new experimental numbers obtained with the connection-training algorithm. No uniqueness theorem, ansatz, or self-definitional identity is invoked. Selective reporting of shallow-only recipes for the headline 50 imes claim is a transparency/correctness issue, not circularity. Hence score 0 and empty steps.

Axiom & Free-Parameter Ledger

7 free parameters · 6 axioms · 2 invented entities

The central claims rest on standard differentiable relaxations of Boolean gates/LUTs plus several hand-chosen training knobs (learning rate 0.1, STE, temperature floors, residual weight 5, Nc pool sizes, sigmoid β annealing). No new physical entities are postulated. Domain assumptions include that continuous softmax/sigmoid training followed by hard selection yields useful binary hardware nets, and that MNIST-style binarized-pixel benchmarks are informative for edge/FPGA deployment.

free parameters (7)
  • learning_rate (fully trainable LGNs) = 0.1
    Raised from the Petersen default 0.01 to 0.1; paper states this was critical for deep fully trainable training (Section 4.2).
  • connection_softmax_temperature_schedule Tc = 1 → 1e-4
    Annealed from 1 to 1e-4 over chosen epoch windows (e.g. epochs 60–80 Yin-Yang; 160–200 MNIST partial connections).
  • gate_softmax_temperature_schedule Tg = 1 → 1e-4
    Annealed after connections for partial-connection runs; not used the same way for all-connection STE runs.
  • Nc (candidate connections per input pin) = 8 / 16 / all
    Hyperparameter controlling partial connection search (8, 16, or all); directly affects capacity and reported accuracy.
  • residual_init_pass_through_weight = 5
    Pass-through gate type initialized to weight 5, others 0 (~91–92% pass-through prior) following Petersen et al. 2024.
  • sigmoid_annealing_beta_schedule (LUTNs) = increased over second half of training
    Scalar β increased during training to binarize LUT entries; schedule (e.g. epochs 100–200) is hand-chosen.
  • layer_width / depth choices = e.g. 8000 gates/layer; 2000 6-LUTs/layer
    Reported SOTA-style numbers use specific widths (8000 gates, 2000 6-LUTs) selected for the comparison tables.
axioms (6)
  • domain assumption Boolean 2-input gates and N-input LUTs can be relaxed to differentiable mixtures (softmax over 16 gate types; MUX/Boolean product form with sigmoid entries) and trained by backpropagation.
    Inherited from Petersen et al. 2022 and Mommen et al. 2026; foundational to all training claims (Sections 2–3).
  • domain assumption Straight-through estimators that use argmax in the forward pass and softmax gradients in the backward pass yield useful discrete networks.
    Invoked for deep fully trainable LGNs and connection training (Section 2.3); standard STE assumption, not proved.
  • domain assumption After training, selecting the highest-probability connection and gate/LUT configuration produces a hardware-implementable binary network whose accuracy is close to the continuous model.
    Load-bearing for all hardware-oriented claims; challenged by the paper’s own deep 6-LUT discretization gap.
  • domain assumption Binarized-pixel MNIST/Fashion-MNIST and Yin-Yang are adequate proxies for the value of smaller logic/LUT networks on edge/FPGA targets.
    All quantitative claims use these benchmarks; no FPGA power/latency measurements are reported.
  • standard math Softmax, LogSumExp, and matrix calculus identities used in the GEMM LUT reformulation are valid.
    Appendix A derivations; standard analysis.
  • ad hoc to paper Removing constant-output gate types from hidden layers improves information flow without harming expressivity enough to erase the accuracy gains.
    Empirical design choice in Section 4.2 (ste/gc/no01); constants kept only in the last layer as biases for group sums.
invented entities (2)
  • Connection pool with per-pin softmax probabilities (partial/full connection training) no independent evidence
    purpose: Makes inter-layer wiring trainable by maintaining Nc weighted candidates per gate/LUT input and hard-selecting the best at the end.
    Core methodological object of the paper; not a physical entity, but a new training construct relative to fixed-connection LGNs.
  • Sigmoid-annealed LUT neuron (improved LUTN model) no independent evidence
    purpose: Keeps LUT entries in (0,1) and anneals them toward binary values for stable deep LUT training and hardware mapping.
    Reformulation of the authors’ prior MUX-based LUT neuron; independent evidence would be external FPGA deployments not provided here.

pith-pipeline@v1.1.0-grok45 · 29931 in / 4362 out tokens · 48879 ms · 2026-07-13T03:17:39.441901+00:00 · methodology

0 comments
read the original abstract

We introduce a novel method for both partial and full optimization of the connections in deep differentiable logic gate networks (LGNs) and lookup table networks (LUTNs). Our training method utilizes a probability distribution over a set of connections per gate/lookup table (LUT) input pin, selecting the connection with highest merit, all whilst the optimal gate types or LUT-entries are learned in parallel. We show that the connection-optimized LGNs outperform standard fixed-connection LGNs on the Yin-Yang, MNIST Handwritten Digits and Fashion-MNIST benchmarks, while requiring only a fraction of the number of logic gates. We achieve 98.92% on the MNIST dataset with two layers of 8000 gates. With only one layer of 8000 gates, we obtain 98.45%, showing that our method requires almost 50 times fewer gates compared to fixed-connection LGNs. Training stability up to ten layers has been ensured by employing a high learning rate, straight-through estimators and trimming constant-output gate types. Additionally, we present a LUT neuron description that enables stable training with backpropagation, tested up to 6-layer deep networks. The model requires four times fewer trainable parameters and still achieves a higher accuracy compared to the fixed-connection LGN training algorithm. Our connection-training algorithm also works well for the LUTNs, achieving an accuracy of 98.88% for two layers of 2000 6-input LUTs.

Figures

Figures reproduced from arXiv: 2607.09399 by Lars Keuninckx, Matthias Hartmann, Piet Wambacq, Werner van Leekwijck, Wout Mommen.

Figure 1
Figure 1. Figure 1: Each first input pin with input value a of gate k in layer l has Nc number of possible connections to gate output pins in the previous layer l − 1. We will call these the pool of possible connections for that input pin with input value a of gate k in layer l. Each connection in the pool takes the output value of gate i in the previous layer, namely g l−1 i , and multiplies it with a probability value (pa) … view at source ↗
Figure 2
Figure 2. Figure 2: The test set of the Yin-Yang data set using a 2-layer network of 500 [PITH_FULL_IMAGE:figures/full_fig_p006_2.png] view at source ↗
Figure 5
Figure 5. Figure 5: Accuracy on the Fashion-MNIST data set as a function of number [PITH_FULL_IMAGE:figures/full_fig_p007_5.png] view at source ↗
Figure 6
Figure 6. Figure 6: Accuracy obtained on the MNIST Handwritten Digits data set as a [PITH_FULL_IMAGE:figures/full_fig_p008_6.png] view at source ↗
Figure 7
Figure 7. Figure 7: (a) Accuracy and (b) training time of the 2-LUTNs (cyan) compared [PITH_FULL_IMAGE:figures/full_fig_p010_7.png] view at source ↗
Figure 9
Figure 9. Figure 9: Accuracy on the MNIST Handwritten Digits data set of the 6-LUTN [PITH_FULL_IMAGE:figures/full_fig_p011_9.png] view at source ↗
Figure 8
Figure 8. Figure 8: (a) Comparison of accuracies obtained from the method of [PITH_FULL_IMAGE:figures/full_fig_p011_8.png] view at source ↗

discussion (0)

Sign in with ORCID, Apple, or X to comment. Anyone can read and Pith papers without signing in.

Reference graph

Works this paper leans on

31 extracted references · 9 canonical work pages · 2 internal anchors

  1. [1]

    Bacellar, Alan T. L. and Susskind, Zachary and Jr, Mauricio Breternitz and John, Eugene and John, Lizy K. and Lima, Priscila M. V. and Fran. Differentiable. doi:10.48550/arXiv.2410.11112 , urldate =

  2. [2]

    Trends in

    Desislavov, Radosvet and. Trends in. Sustainable Computing: Informatics and Systems , volume =. doi:10.1016/j.suscom.2023.100857 , urldate =

  3. [3]

    and Pedroni, Bruno U

    Diehl, Peter U. and Pedroni, Bruno U. and Cassidy, Andrew and Merolla, Paul and Neftci, Emre and Zarrella, Guido , year = 2016, month = jul, pages =. 2016. doi:10.1109/IJCNN.2016.7727758 , urldate =

  4. [4]

    and Ward, Max and Neftci, Emre O

    Eshraghian, Jason K. and Ward, Max and Neftci, Emre O. and Wang, Xinxin and Lenz, Gregor and Dwivedi, Girish and Bennamoun, Mohammed and Jeong, Doo Seok and Lu, Wei D. , year = 2023, month = sep, journal =. Training. doi:10.1109/JPROC.2023.3308088 , urldate =

  5. [5]

    Quantized

    Hubara, Itay and Courbariaux, Matthieu and Soudry, Daniel and. Quantized. Journal of Machine Learning Research , volume =

  6. [6]

    Spiking-

    Kim, Seijoon and Park, Seongsik and Na, Byunggook and Yoon, Sungroh , year = 2020, month = apr, journal =. Spiking-. doi:10.1609/aaai.v34i07.6787 , urldate =

  7. [7]

    Kriener, Laura and G. The. doi:10.48550/arXiv.2102.08211 , urldate =

  8. [8]

    1998 , howpublished =

    MNIST Database of Handwritten Digits , author =. 1998 , howpublished =

  9. [9]

    Stochastic

    Lee, Yang Yang and Halim, Zaini Abdul and Wahab, Mohd Nadhir Ab and Almohamad, Tarik Adnan , year = 2024, month = mar, journal =. Stochastic. doi:10.34133/research.0307 , urldate =

  10. [10]

    Convolutional

    Petersen, Felix and Kuehne, Hilde and Borgelt, Christian and Welzel, Julian and Ermon, Stefano , year = 2024, month = dec, journal =. Convolutional

  11. [11]

    Petersen, Felix and Borgelt, Christian and Kuehne, Hilde and Deussen, Oliver , year = 2022, month = dec, journal =. Deep

  12. [12]

    Putra, Rachmad Vidya Wicaksana and Shafique, Muhammad , year = 2021, month = jul, pages =. Q-. 2021. doi:10.1109/IJCNN52387.2021.9534087 , urldate =

  13. [13]

    Computer

    Rastegari, Mohammad and Ordonez, Vicente and Redmon, Joseph and Farhadi, Ali , editor =. Computer. doi:10.1007/978-3-319-46493-0_32 , abstract =

  14. [14]

    doi:10.1109/TCAD.2018.2819366 , urldate =

    Rathi, Nitin and Panda, Priyadarshini and Roy, Kaushik , year = 2019, month = apr, journal =. doi:10.1109/TCAD.2018.2819366 , urldate =

  15. [15]

    Nature Computational Science , volume =

    Opportunities for Neuromorphic Computing Algorithms and Applications , author =. Nature Computational Science , volume =. doi:10.1038/s43588-021-00184-y , abstract =

  16. [16]

    Sevilla, Jaime and Heim, Lennart and Ho, Anson and Besiroglu, Tamay and Hobbhahn, Marius and Villalobos, Pablo , year = 2022, month = jul, pages =. Compute. 2022. doi:10.1109/IJCNN55064.2022.9891914 , abstract =

  17. [17]

    LUTNet: Rethinking Inference in FPGA Soft Logic

    Wang, Erwei and Davis, James J. and Cheung, Peter Y. K. and Constantinides, George A. , year = 2019, month = apr, number =. doi:10.48550/arXiv.1904.00938 , urldate =

  18. [19]

    Fashion-

    Xiao, Han and Rasul, Kashif and Vollgraf, Roland , year = 2017, month = sep, number =. Fashion-. doi:10.48550/arXiv.1708.07747 , urldate =

  19. [20]

    IEEE Access , author =

    Deep. IEEE Access , author =. 2023 , pages =. doi:10.1109/ACCESS.2023.3328622 , abstract =

  20. [21]

    Inter-patient

    Mommen, Wout and Keuninckx, Lars and Detterer, Paul and Colpaert, Achiel and Wambacq, Piet , month = jan, year =. Inter-patient. doi:10.48550/arXiv.2601.11433 , abstract =

  21. [22]

    Recurrent

    Bührer, Simon and Plesner, Andreas and Aczel, Till and Wattenhofer, Roger , month = aug, year =. Recurrent. doi:10.48550/arXiv.2508.06097 , abstract =

  22. [23]

    LUTNet: Learning FPGA Configurations for Highly Efficient Neural Network Inference

    Wang, Erwei and Davis, James J. and Cheung, Peter Y. K. and Constantinides, George A. , month = mar, year =. doi:10.48550/arXiv.1910.12625 , abstract =

  23. [24]

    doi:10.48550/arXiv.2510.15655 , abstract =

    Gerlach, Lino and Våge, Liv and Gerlach, Thore and Kauffman, Elliott , month = oct, year =. doi:10.48550/arXiv.2510.15655 , abstract =

  24. [25]

    2020 30th

    Umuroglu, Yaman and Akhauri, Yash and Fraser, Nicholas James and Blott, Michaela , month = aug, year =. 2020 30th. doi:10.1109/FPL50879.2020.00055 , abstract =

  25. [26]

    , month = dec, year =

    Andronic, Marta and Constantinides, George A. , month = dec, year =. 2023. doi:10.1109/ICFPT59805.2023.00012 , abstract =

  26. [27]

    , month = sep, year =

    Andronic, Marta and Constantinides, George A. , month = sep, year =. 2024 34th. doi:10.1109/FPL64840.2024.00028 , abstract =

  27. [28]

    Nazemi, Mahdi and Pasandi, Ghasem and Pedram, Massoud , month = jan, year =. Energy-. 2019 24th. doi:10.1145/3287624.3287722 , abstract =

  28. [29]

    Neural Networks , author =

    On training networks of monostable multivibrator timer neurons , volume =. Neural Networks , author =. 2026 , keywords =. doi:10.1016/j.neunet.2025.108092 , abstract =

  29. [30]

    Evolution

    Salimans, Tim and Ho, Jonathan and Chen, Xi and Sidor, Szymon and Sutskever, Ilya , month = sep, year =. Evolution. doi:10.48550/arXiv.1703.03864 , abstract =

  30. [31]

    and Clune, Jeff , month = apr, year =

    Such, Felipe Petroski and Madhavan, Vashisht and Conti, Edoardo and Lehman, Joel and Stanley, Kenneth O. and Clune, Jeff , month = apr, year =. Deep. doi:10.48550/arXiv.1712.06567 , abstract =

  31. [32]

    Evolution

    Sarkar, Bidipta and Fellows, Mattie and Duque, Juan Agustin and Letcher, Alistair and Villares, Antonio León and Sims, Anya and Wibault, Clarisse and Samsonov, Dmitry and Cope, Dylan and Liesen, Jarek and Li, Kang and Seier, Lukas and Wolf, Theo and Berdica, Uljad and Mohl, Valentin and Goldie, Alexander David and Courville, Aaron and Sevegnani, Karin and...