{"id":"0330e09c-72f6-4027-9ecd-b116807a0e71","arxiv_id":"2607.21633","paper_version":2,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":6.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":5,"one_line_summary":"An input-anchored topology in which every logic gate combines a private hidden spine with a direct input connection lets fixed-width logic gate networks keep improving with depth to 150 layers.","lead":"This paper identifies why deep logic gate networks stop improving with depth and proposes a simple topology — each gate reads the original input alongside its private hidden chain — that keeps accuracy climbing to 150 layers. It reports consistent depth gains on MNIST, CIFAR-10, and CIFAR-100, with diagnostics for why random wiring loses useful learning signals.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"Shared-hyperparameter comparison is under-specified: τ (Table 4) varies per dataset/width with no sensitivity analysis for RWLGN, so the claimed topology-driven depth-scaling gap could be a hyperparameter artifact.","rationale":"The reader's weakest assumption identifies the τ hyperparameter as the main threat, and I agree. The most load-bearing condition for the central claim is not merely that IALGN improves with depth (which could be due to extra trainable anchor logits), but that RWLGN's failure is not an artifact of an unfavorable shared hyperparameter. The fixed-anchor kx=1 experiment in Fig 7 provides some evidence against the pure-capacity confound, even though it changes nb and lacks a fully reported RWLGN control. No comparable control exists for τ: Table 4 introduces a dataset/width-specific logit scale without definition or sensitivity analysis. Because τ directly controls the relaxation used for STE gradients, it can determine whether deep RWLGNs learn. A reviewer cannot distinguish the paper's topology claim from a poorly-tuned baseline without a τ sweep. The verdict CONDITIONAL is appropriate; our concern strengthens the condition rather than overturning the verdict.","tokens_in":13455,"tokens_out":11627,"duration_ms":129262,"concrete_test":"On CIFAR-10 at W=12k, run RWLGN and LDLGN-reimpl at depths 4, 50, 100, 150 with τ ∈ {1,3,10,30,100,300} (defining τ precisely and keeping all other settings from Table 3). Plot accuracy vs depth for each τ. Also state how the default τ values in Table 4 were selected: per-method validation, per-dataset, or from prior work. If any τ produces a positive RWLGN depth slope comparable to IALGN's (≈+2.9 accuracy from D=4 to D=150), the comparative claim fails. As a secondary check, run IALGN with RWLGN's best τ to confirm the trend persists.","verdict_should_be":"UNCHANGED","load_bearing_attack":"The paper's central empirical claim is that IALGN's depth-accuracy scaling is due to its input-anchored topology, while RWLGN saturates/degrades under 'identical' optimization settings. That fairness claim is under-supported. Appendix Table 4 lists a per-dataset/per-width 'logit scaling τ' (values 10, 30, 100) whose meaning is never defined in the main text, and the paper reports no sensitivity analysis for RWLGN over τ. If τ governs the softmax temperature or logit initialization of the Boolean relaxation, it is a critical hyperparameter for training deep LGNs; a value tuned for IALGN (or for the shallow regime) could suppress RWLGN's depth trend. Since the abstract attributes depth scalability to topology, a demonstration that the RWLGN depth curve stays flat/negative across a range of τ is necessary. Without it, the observed gap could be a hyperparameter artifact.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"The paper argues that deep Logic Gate Networks (LGNs) fail to benefit from depth for two distinct reasons: optimization collapse, which can be mitigated by skip-biased initialization and straight-through estimation, and topology-induced degradation of output-specific credit, which persists even when training is stabilized. The authors propose Input-Anchored Logic Gate Networks (IALGNs), where each gate takes one input from a private hidden spine (usually the same-index previous-layer gate) and one input directly from the original input. They report credit-assignment diagnostics (path sharing, gradient coverage, gradient purity) showing that random wiring dilutes or conflicts output-specific gradients, whereas IALGN preserves coherent private credit. Empirically, on MNIST, CIFAR-10, and CIFAR-100, IALGN shows consistent fixed-width accuracy improvements with depth up to 150 layers, while RWLGN and a reimplemented LDLGN baseline saturate or degrade. Additional analyses (linear probes, kh ablations, operation-aware effective depth) are used to support the claim that IALGN's depth scaling arises from state preservation and sparse input-conditioned refinement rather than raw path length or generic wiring flexibility.","tokens_in":13709,"tokens_out":4447,"duration_ms":47773,"significance":"If the main claim holds, the paper offers a simple, credit-preserving topology that makes depth useful in differentiable Boolean circuits, a problem that has limited LGN research. The paper's strengths include multi-seed results with standard deviations, controlled comparisons under shared optimization settings, several ablations (kh, kx, nb, initialization/gradient-estimator variants), and explicit diagnostics that go beyond raw accuracy. The main result is not circular: accuracy is measured against independent baselines. However, the supporting evidence has gaps that need to be addressed before the claim is fully convincing: the logit-scaling hyperparameter τ is not defined and not sensitivity-checked, the D=150 result rests on a single configuration, the LDLGN baseline is a self-described reimplementation without code, and the credit-diagnostic sampling protocol may favor the proposed architecture. These issues are fixable, so the paper is a promising candidate for major revision rather than rejection.","major_comments":[{"comment":"The logit-scaling hyperparameter τ is listed per dataset/width but is never defined in the main text, and no sensitivity analysis over τ is reported for RWLGN or LDLGN. Since the central claim is that IALGN's depth-scaling gap arises from topology under identical optimization settings, an unexamined τ that varies per experiment leaves open the possibility that the baselines are suppressed by a hyperparameter choice. Please define τ, state how the values were selected, and provide a sweep of τ for RWLGN (and LDLGN) at representative depths to show that the qualitative depth-scaling gap is robust.","section":"§6, Table 4, Appendix A"},{"comment":"The abstract claims 'consistent fixed-width depth–accuracy scaling up to 150 layers,' but D=150 appears for only one configuration (CIFAR-10, W=12k). Other sweeps stop at D=80 (CIFAR-10 W=128k, CIFAR-100) or D=100 (MNIST). At the single D=150 point, the improvement over D=100 is 0.09±0.20 (56.50±0.11 vs 56.41±0.17), which is not significant. Either extend the depth sweeps to at least one more width/dataset, or qualify the 'up to 150 layers' claim as preliminary.","section":"Abstract, Table 1"},{"comment":"The LDLGN baseline is explicitly described as a reimplementation 'not a reproduction of the original reported results,' and no code is provided. Since this baseline is used in the headline comparison (and in Figure 8), a reader cannot verify that the reimplementation faithfully represents the original method. Please release code (or a detailed implementation specification) and, if possible, validate the reimplementation against published numbers under the same no-augmentation protocol, or remove LDLGN from the central comparison and rely on RWLGN as the main baseline.","section":"Table 1, LDLGN reimpl."},{"comment":"The paired hidden-output sampling procedure uses identical sampled indices for hidden gates and output gates. The text argues this avoids bias against private-chain architectures, but it may instead favor IALGN: if an IALGN hidden gate's chain output index is in the output sample, that gate is guaranteed to have a reachable sampled output, whereas a randomly wired hidden gate's sampled output descendants are not guaranteed. This could inflate IALGN's coverage or purity relative to RWLGN. The authors should report what happens under independent uniform sampling of hidden and output gates, or justify why the paired scheme does not introduce a systematic bias. The credit diagnostics are the main mechanistic evidence, so this point is important.","section":"§3, Fig. 2, Appendix B"}],"minor_comments":[{"comment":"The caption should explicitly state which curves use which initialization/STE combination, especially the RWLGN curve, to avoid confusion with the four IALGN settings.","section":"Figure 1 caption"},{"comment":"The probe depth labels (0,1,10,...,100) are not specified in the caption; please state that depth 0 refers to the binarized input before any logic layer, as mentioned in the body text.","section":"§5, Figure 4"},{"comment":"There is a stray period in the caption ('... benefit deep models. Topology...'). Also, the table caption should clarify that 'Relaxed source' refers to the pin that is relaxed, not the output source.","section":"Table 2 caption"},{"comment":"The STE equations use exsoft and exhard; it would be helpful to define the softmax temperature or scaling used to compute exsoft from the anchor-selection logits. This is related to the τ question and should be made explicit.","section":"Appendix C"},{"comment":"The reference to the AAAI copyright line in the header is unusual; please ensure the venue and year are correct.","section":"References"}],"recommendation":"major_revision","confidential_remarks":"The paper addresses a timely problem and the core idea is simple and plausible. The main empirical claim is probably correct, but the missing sensitivity analysis for τ and the single-configuration D=150 result are exactly the kind of gaps that should be closed before publication. The LDLGN baseline is a weak point because it is a self-reported reimplementation; the authors should either make code available or de-emphasize that comparison. The credit-diagnostic sampling concern is subtle and deserves a careful response."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Interesting but not yet fully verified. The paper proposes a genuinely simple topology for logic gate networks — a fixed same-index hidden spine plus a direct input anchor at every gate — and shows that, under fixed width and shared training hyperparameters, IALGN accuracy rises with depth on MNIST, CIFAR-10/100 while randomly wired LGNs saturate or drop. That pattern is interesting and the supporting diagnostics are thoughtful. The credit-preservation story is plausible: random wiring merges output paths, while the input-anchored spine keeps one private lineage per output. The hidden-wiring ablation (k_h) is a nice control: adding hidden flexibility to IALGN helps at depth 4 but hurts at depth 50, suggesting input anchoring rather than generic wiring flexibility drives the gain.\n\nThe main soft spot is the comparison fairness. Table 4 lists a per-dataset/per-width 'logit scaling τ' (values 10, 30, 100) that is never defined in the main text, and there is no sensitivity analysis for RWLGN over τ. If τ controls the softmax temperature of the Boolean relaxation, it is a critical training hyperparameter; a value tuned for IALGN could suppress RWLGN's depth trend. The authors state optimization settings are identical, but without a τ sweep for the baselines, the abstract's topology-attribution claim is under-supported. The paper also promises code only upon acceptance, and the LDLGN comparison uses a self-reported reimplementation (though clearly labelled). And the headline 'up to 150 layers' rests on one configuration (CIFAR-10, W=12k); other datasets stop at 80 or 100 layers.\n\nNone of these are fatal. The core empirical pattern is plausible and the diagnostics are honest about their limitations (the effective-depth section explicitly says it does not measure credit quality). The paper deserves serious peer review, but the authors should be asked to define τ, provide a τ sensitivity check for RWLGN, and release code. If that check holds, this is a solid within-subfield advance for LGN researchers. Not a broad-ML result, but a real one for the community.","headline":"Plausible input-anchored topology for LGNs, but the τ hyperparameter and missing code leave the depth-scaling claim conditional.","tokens_in":14165,"tokens_out":3135,"would_cite":true,"duration_ms":30795,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"To make logic gate networks scale with depth, each gate must keep a private hidden spine and a direct input anchor; with this wiring, accuracy keeps improving to 150 layers.","keywords":["Logic Gate Networks","depth scaling","credit assignment","input anchoring","Boolean circuits","differentiable logic gates","straight-through estimation","representation learning"],"falsifier":"Run RWLGN and IALGN on CIFAR-10 with identical width and depth and a shared grid of τ values (including each method's best τ), then compare the depth–accuracy curves. If some τ setting gives RWLGN the same upward scaling as IALGN, the central claim fails.","tokens_in":13363,"feed_emoji":"⚙️","tokens_out":5807,"duration_ms":54000,"temperature":0.7,"pith_summary":"The paper argues that logic gate networks (LGNs) fail to scale with depth for two distinct reasons: gradients vanish, and random wiring merges the output-specific computational paths, so hidden layers lose usable credit. It shows that even when optimization is stabilized with skip-biased initialization and straight-through estimation, randomly wired LGNs still saturate or degrade with depth. The proposed topology, Input-Anchored Logic Gate Networks (IALGNs), gives each gate one private hidden input (a spine) and one direct input-bit anchor, preserving a separate lineage to each output while keeping input access at every layer. On MNIST, CIFAR-10, and CIFAR-100, IALGN shows consistent fixed-width accuracy gains up to 150 layers, whereas alternative topologies do not. If correct, this separates credit-preserving topology from mere trainability and yields a constructive recipe for deep Boolean circuits.","feed_headline":"Private spine plus input anchor lets logic nets scale with depth","feed_subtitle":"Each gate keeps a hidden lineage and a direct input bit; accuracy keeps rising to 150 layers, while random wiring plateaus.","key_machinery":"The carrying mechanism is the IALGN wiring rule: h_i^ℓ = g_i^ℓ(h_i^{ℓ−1}, x_{b_{ℓ,i}}), where one input pin (the spine) passes the same-index hidden feature from the previous layer and the other pin (the anchor) is a directly selected bit of the original binarized input. This two-pin structure prevents output-path merging while keeping input access at every layer. The paper's diagnostics—path sharing, descendant-conditioned gradient coverage, and gradient purity—show that random wiring merges output paths and dilutes credit, whereas IALGN maintains one private lineage per output with coherent gradients. Random-k_x anchor relaxation adds a learned soft selection among k_x candidate input anch","core_discovery":"The central discovery is that depth scalability in logic gate networks requires both stable optimization and a topology that preserves output-specific credit. Randomly wired LGNs that are successfully stabilized with skip-biased initialization and straight-through estimation still fail to improve with depth because early-layer gates become ancestors of nearly every output, diluting or canceling the gradient contributions. IALGN prevents this by giving each gate a private one-to-one hidden spine (so each output keeps its own lineage) and a direct anchor to the original input (so each layer has task-relevant information without routing through another hidden unit). The resulting networks exhib","pith_inferences":["The IALGN wiring is a discrete analogue of residual or dense connectivity: the spine acts as a learned state, the anchor as a fixed input skip. It could be transplanted into other discrete or quantized architectures (lookup-table networks, binarized networks) to test whether depth scaling reappears there.","Because each gate reads only one input bit, trained circuits are extremely input-sparse; a natural test is whether the accuracy scaling survives when anchors are limited to a task-relevant input region, and whether the learned anchor selection consistently picks interpretable bits.","The paper stops short of budget-matched comparisons; a width-versus-depth study under a fixed total-gate budget would clarify how much of the gain is topology versus extra parameters.","The observed identity-heavy circuits suggest a cheap optimization trick: regularize gate logits toward pass-through operations during training, which might accelerate convergence or improve scaling in other LGN variants."],"forward_implications":["Fixed-width IALGNs improve classification accuracy with depth up to 150 layers on MNIST, CIFAR-10, and CIFAR-100, while randomly wired LGNs and the reimplemented baseline saturate or degrade.","The depth gains come from input anchoring, not generic wiring flexibility: relaxing the hidden spine (k_h > 1) helps at depth 4 but hurts at depth 50.","Trained IALGN circuits mostly preserve a private hidden state (about 88% spine-preserving operations) with sparse anchor-conditioned updates, so nominal depth is not the same as functional path length.","Optimization fixes alone do not confer depth scaling: under the strongest shared protocol, the random-wired network still declines with depth.","At inference each gate keeps a single selected anchor, so the trained model is a discrete two-input Boolean circuit with no train-versus-inference gap."],"fun_headline_variants":["Private spine and input anchor stop logic-net depth plateau","Deep logic nets stay accurate when each gate keeps a private spine","Random wiring kills deep logic nets; input anchors save them","Logic nets scale to 150 layers if each gate has a private spine","Stop gradient dilution: private spines let logic nets go deep"],"cache_read_input_tokens":2304,"weakest_assumption_plain":"The comparison assumes the logit scale τ (and the shared optimization settings) is equally fair to every topology; if τ was tuned toward IALGN rather than swept for each baseline, the depth-scaling gap could be a hyperparameter artifact.","fun_headline_variants_meta":{"raw":{"variants":["Private spine and input anchor stop logic-net depth plateau","Deep logic nets stay accurate when each gate keeps a private spine","Random wiring kills deep logic nets; input anchors save them","Logic nets scale to 150 layers if each gate has a private spine","Stop gradient dilution: private spines let logic nets go deep"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000558,"raw_usage":{"total_tokens":2475,"prompt_tokens":711,"completion_tokens":1764,"prompt_tokens_details":{"cached_tokens":256},"prompt_cache_hit_tokens":256,"prompt_cache_miss_tokens":455,"completion_tokens_details":{"reasoning_tokens":1680}},"tokens_in":455,"tokens_out":1764,"duration_ms":12387,"temperature":1.0,"reasoning_tokens":1680,"cache_read_input_tokens":256,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-01T18:04:21.539257+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Run RWLGN and IALGN on CIFAR-10 with identical width and depth and a shared grid of τ values (including each method's best τ), then compare the depth–accuracy curves. If some τ setting gives RWLGN the same upward scaling as IALGN, the central claim fails.","supporting_citations":[],"review_version":1}