{"id":"c47a3435-e3f1-4d3c-a77b-d4f4bbae975f","arxiv_id":"2504.20079","paper_version":1,"verdict":"CONDITIONAL","confidence":"MODERATE","novelty_score":5.0,"correctness_risk":"medium","formal_verification":"none","parameter_count":4,"one_line_summary":"FX-DARTS removes cell-topology and two-input discretization priors from DARTS and stabilizes the enlarged search space with entropy-based super-network shrinking, achieving 76.4% top-1 ImageNet accuracy.","lead":"This paper proposes FX-DARTS, a differentiable architecture search method that drops DARTS's rules that all cells of a type share one topology and that each node keeps exactly two inputs. It uses an entropy-based, progressively shrinking super-network to keep search stable and reports competitive image classification results at low search cost.","discovery_kind":"new_method","skeptic_critique":{"model":"deepseek-v4-flash","headline":"The method's search signal is never isolated: no random-search or supernet-to-pruned-accuracy control shows that ESS's entropy-thresholding, rather than the enlarged space and evaluation protocol, produces the reported accuracy.","rationale":"The paper has honest positives: multi-dataset evaluations, component ablations, entropy curves, and an explicit conclusion that flexible architectures are not yet overwhelmingly better. Those support a plausible method, not a proven one. The load-bearing gap is that the empirical success is never attributed to the search signal. Because the search space is larger and DARTS is already known to be vulnerable to random search (cited in [40]), a random-search control is the minimal experiment that would make the claim land. Similarly, the supernet-to-pruned accuracy gap is the internal validation the method's discretization step needs. These checks are straightforward and do not require new theory. If they pass, the conditional verdict can become an accept; if they fail, the central claim is in doubt.","tokens_in":23034,"tokens_out":13253,"duration_ms":148845,"concrete_test":"Run a random-search control in the same unconstrained search space: sample 50 architectures by independently selecting operators and edges with the same marginal counts as FX-DARTS-O1(48E), train them under the exact Table I protocol (17 cells, 150 epochs, same augmentation), and compare the CIFAR-10/CIFAR-100 accuracy distribution to the FX-DARTS checkpoint. Also, for that checkpoint, measure super-network validation accuracy before and after applying Algorithm 2, with and without renormalizing remaining weights. If random architectures match FX-DARTS, or if the pruned supernet loses more than roughly 0.5% relative to the full supernet, the entropy-shrinking mechanism is not established as the source of the reported performance.","verdict_should_be":"UNCHANGED","load_bearing_attack":"FX-DARTS's central claim is that removing DARTS priors while using ESS yields competitive architectures. The key unverified link is attribution: the paper never shows the entropy-threshold selection (Eq. 12, Eq. 14, Algorithm 2) picks architectures that are better than comparably sized random or thresholded subgraphs in the same unconstrained space. Section II-A even cites [40] showing random search is competitive in the original DARTS space, so this control is particularly important. Theorem 1 and Corollary 2 only prove that sparsity entropy decreases for sufficiently large lambda; they say nothing about validation accuracy of the pruned architecture or about whether the retained operators are the ones responsible for accuracy. Section IV-B(g) asserts that pruning at epsilon=0.02 has minimal impact, but no supernet-vs-pruned accuracy numbers are reported, and because Algorithm 1 executes Algorithm 2 during architecture optimization (line 18), the pruning criterion is not even a clean final discretization: after deletion, Eq. (10)'s denominator should be recomputed, a detail the paper leaves unspecified. With baselines re-evaluated at 17 cells and without drop-path, and with no random-search baseline, the reported gains could reflect the favorable evaluation protocol or the space itself rather than the ESS search.","agreement_with_reader":"agree"},"referee_report":{"model":"deepseek-v4-flash","summary":"FX-DARTS proposes to relax two standard DARTS priors: the requirement that cells of the same type share one topology and the rule that each intermediate node keeps exactly two operators from two distinct source nodes. The method uses node-wise normalized architectural parameters, an entropy-based sparsity penalty with feedback-adapted per-cell coefficients, a warm-up/architecture-optimization split, cyclic reinitialization of model parameters, and threshold-based pruning of low-weight operators. The authors report competitive CIFAR-10/100 results, multi-task results on TinyImageNet/SVHN/Flowers102, and ImageNet-1K top-1 accuracies around 76% at search costs of a few GPU-hours, and they claim that the entropy-based shrinking framework makes this feasible in the enlarged, less constrained space.","tokens_in":23259,"tokens_out":4040,"duration_ms":46434,"significance":"If the empirical claims hold, the paper addresses a question of genuine interest to the NAS community: whether differentiable search can be extended to a substantially less constrained cell topology space without suffering instability and without losing accuracy relative to strong DARTS-style priors. The paper is transparent about its main limitation in the conclusion, provides ablations for several components, and evaluates across multiple datasets and operator spaces. The theoretical contribution is modest: Theorem 1 and Corollary 2 establish that the sparsity entropy can be driven down by gradient descent under a sufficiently large adaptive coefficient, which is a statement about the optimization objective, not about the quality of the pruned architecture. The central performance claim is therefore empirical, and the missing attribution controls described below are the main obstacle to accepting that claim at face value.","major_comments":[{"comment":"The paper never isolates the effect of the ESS search signal. Section II-A itself cites Li and Talwalkar [40], who show that random search is competitive in the original DARTS space. Since FX-DARTS enlarges the space and also changes the evaluation protocol (17 cells, no drop-path, adjusted channels), a random-search or random-thresholding baseline in the same unconstrained space is needed to attribute the reported gains to ESS rather than to the enlarged search space and the evaluation setup. Without such a control, the central claim that entropy-based shrinking produces better architectures than comparably sized arbitrary subgraphs in the same space is unsupported.","section":"§II-A, §IV-D, Table I"},{"comment":"The related-work section explicitly discusses GOLD-NAS and DNAD as methods that also operate in unconstrained or flexible spaces, but neither appears in any experimental comparison. Because the paper's contribution is specifically about searching in a topology-unconstrained space, omitting the two most directly comparable unconstrained-search methods prevents the reader from assessing whether FX-DARTS advances the state of the art for flexible architectures. At minimum, the authors should add these comparisons or justify their absence with concrete experimental or computational reasons.","section":"§II-B, Tables I, III, V"},{"comment":"Algorithm 2 is executed inside the architecture-optimization loop (Algorithm 1, line 18), but the paper does not specify what happens to the normalization in Eq. (10) after an operator is pruned. If the operator is simply removed from the softmax denominator, the remaining weights change value; if it is masked but kept in the denominator, the entropy loss and gradient computations are inconsistent with the actual discrete architecture. This matters because the same thresholding operation is used both as a search-time regularizer and as the final discretization. The authors should state the exact bookkeeping rule for pruned operators and report whether the super-network accuracy before and after final thresholding differs, since Section IV-B(g) asserts that pruning at epsilon=0.02 has minimal impact but no such numbers are given.","section":"Algorithm 1 line 18, Algorithm 2, Eq. (10)"},{"comment":"The experimental protocol creates a potential selection bias. The baselines are re-evaluated with 17 cells and without drop-path under a protocol chosen by the authors, yet there is no evidence that this protocol is neutral or that the baselines' hyperparameters were re-tuned under it. In addition, the hyperparameters of FX-DARTS (search epochs, shrinking coefficients, expected entropy reduction) appear to be selected using the same multi-task benchmark that is later reported as the headline multi-task result in Table III, with Table IV reporting ablations on that same benchmark. The authors should either use a held-out validation split for hyperparameter selection or explicitly report the selection procedure, so that the reported margins (e.g., 0.86% on TinyImageNet over P-DARTS) are not the result of tuning on the test tasks.","section":"§IV-D-1-b, Tables III and IV"}],"minor_comments":[{"comment":"The section ends with the incomplete sentence 'There are' immediately before Section III; this appears to be a missing passage or a formatting error and should be completed or removed.","section":"§II-B, last sentence"},{"comment":"The text says lambda_k for k = 1,...,N, but the sum is over L cells; the index set should be k = 1,...,L.","section":"Eq. (14)"},{"comment":"The formula for lambda can be negative when the angle theta between the two gradients is small and the performance-gradient term dominates; if lambda is intended to be a positive regularization coefficient, the authors should state the required conditions and how the adaptive mechanism handles this case.","section":"Corollary 2"},{"comment":"The top-5 accuracy of FX-DARTS-O1 (Tiny, 64E) is listed as 93.7%, which is higher than that of the stronger top-1 configuration FX-DARTS-O3 (Tiny, 64E) at 93.0%; this inconsistency is not discussed and should be explained or corrected.","section":"Table V"},{"comment":"The naming convention 'FX-DARTS-O1 (48E)' is defined in the text as 3 x Tsearch, but the figure and table captions do not repeat this definition; adding a brief parenthetical in the table caption would improve readability.","section":"§IV-D-1-a and Fig. 2"}],"recommendation":"major_revision","confidential_remarks":"The omission of GOLD-NAS and DNAD from the experiments is particularly conspicuous because both are named in the related-work section as flexible/unconstrained DARTS variants. The authors should also be encouraged to provide a random-search control and to release code for reproducibility; the current manuscript does not state any code availability. These issues seem fixable within the scope of a revision, but they are central to the paper's attribution claim, so I recommend major revision rather than minor revision."},"author_rebuttal":null,"desk_editor":{"model":"deepseek-v4-flash","letter":"Quick take: FX-DARTS is a legitimate step toward topology-unconstrained differentiable NAS, with a concrete mechanism and honest evaluation, but the paper never shows that the entropy shrinking is the reason the searched architectures are good.\n\nThe genuinely new pieces are the cell-unique architecture parameters, the node-wise normalization in Eq. (10), and the adaptive entropy-based shrinking with cyclic reinitialization. The authors also report a wide set of experiments, including CIFAR-100, TinyImageNet, and ImageNet, and they include ablations that isolate some components. The search cost is low, and the claim that one search can yield several architectures with different compute budgets is nice. Credit is due for the honest conclusion admitting that flexible architectures have not shown overwhelming advantages over constrained ones.\n\nThe soft spots are real but not fatal. The most important is attribution: there is no random-search baseline in the enlarged space, and no comparison with GOLD-NAS or DNAD, the closest flexible-NAS methods. Since the paper itself cites work showing random search is competitive in the DARTS space, the missing control is significant. Theorem 1 proves that entropy decreases under gradient descent for large enough lambda; it does not connect sparsity to accuracy. The pruning threshold (epsilon=0.02) is applied during search (Algorithm 1 calls Algorithm 2), so the discretization is entangled with training, and the paper never reports supernet-vs-pruned accuracy. The unspecified renormalization after pruning is a minor but real detail. On the experimental side, ImageNet results are single-seed, and hyperparameters like c1/c2 and Delta E appear to be tuned on the same multi-task benchmark used for headline results, with no separate validation split described. Re-evaluating baselines at 17 cells without drop-path may also shift comparisons.\n\nThese are standard revision requests for a NAS paper, not a fatal flaw. The core idea is sound and the paper is readable. It deserves peer review; reviewers should push on the missing baselines and the attribution question. If the authors add a random-search control and compare with GOLD-NAS and DNAD, the claim would be much stronger.","headline":"FX-DARTS is a credible attempt to remove DARTS priors, with a concrete mechanism and honest evaluation, but the paper never isolates whether entropy shrinking is what makes the searched architectures good.","tokens_in":23806,"tokens_out":1998,"would_cite":false,"duration_ms":19412,"reading_group":"maybe","serious_thinker":"yes","would_accept_peer_review":true},"rs_alignment":null,"lean_confirmation":null,"pith_extraction":{"msc":[],"pacs":[],"model":"deepseek-v4-flash","headline":"FX-DARTS shows neural architectures can be searched without cell-sharing or two-operator priors, matching constrained NAS at 76.4% top-1 on ImageNet.","keywords":["differentiable architecture search","neural architecture search","topology-unconstrained search space","super-network shrinking","entropy regularization","cell-based architectures","image classification","adaptive coefficient feedback"],"falsifier":"Retrain each threshold-pruned FX-DARTS architecture and the un-pruned super-network under identical budgets on the same validation split; if the pruned models do not consistently match or beat the super-network's accuracy, the entropy-shrinking route to discrete architectures fails.","tokens_in":22804,"feed_emoji":"🧠","tokens_out":7229,"duration_ms":71380,"temperature":0.7,"pith_summary":"This paper tries to establish that neural architecture search need not rely on two of DARTS' strongest priors: cells of the same type sharing one topology, and each intermediate node keeping exactly two operators from two distinct nodes. To make the enlarged search space tractable, it softens architectural parameters node-wise and adds an entropy term that progressively shrinks the super-network, with per-cell weights adjusted by feedback and the model weights periodically reinitialized. If the claim holds, a single search can return a family of flexible architectures with different compute budgets at search costs of a few GPU-hours, matching or slightly beating topology-constrained competitors (78.32% on CIFAR-100, 76.4% top-1 on ImageNet-1K). The paper's own conclusion concedes that the advantage over constrained-space methods is not overwhelming.","feed_headline":"No-prior DARTS matches constrained search at 76.4% accuracy","feed_subtitle":"FX-DARTS drops shared-cell and two-operator rules, then shrinks the super-network with entropy to keep the search stable.","key_machinery":"The central object is the node-wise softmax-normalized architectural weight $a^o_{(i,j)}$ from Eq. (10), which sums over all incoming edges and operators and thereby removes the 'two operators from two distinct nodes' rule. The shrinking mechanism is the cell-level sparsity entropy $H^{(k)}_{\\mathrm{cell}}(\\alpha)=\\sum_{2<j<N} H^{(k,j)}_{\\mathrm{node}}(\\alpha)$, added to the cross-entropy loss as $\\sum_k \\lambda_k H^{(k)}_{\\mathrm{cell}}$; the $\\lambda_k$ are adaptively increased or decreased by a feedback rule depending on whether the actual entropy reduction meets the expected $\\Delta E$. Cyclic reinitialization of the model parameters and threshold-based discretization (Algorithm 2) complete the pipeline. Theorem 1 guarantees monotone decrease of the sparsity entropy under a sufficiently large $\\lambda$, assuming bounded gradients of the cross-entropy term.","core_discovery":"The paper's central claim is that DARTS can be made topology-unconstrained: every cell may have a unique structure, and a node may retain any number of operators from any previous nodes, without sacrificing search stability. The Entropy-based Super-network Shrinking (ESS) framework is what makes this possible, by normalizing architectural parameters node-wise, adding a cell-level sparsity entropy to the loss, adaptively adjusting the entropy coefficients, and periodically reinitializing model parameters to decouple architecture optimization from super-network training. The resulting architectures reach 78.32% top-1 on CIFAR-100 and 76.4% top-1 on ImageNet-1K with search costs between 1.6 and 4.3 GPU-hours. The paper itself notes that the advantage over constrained-space methods is not overwhelming.","pith_inferences":["If every constrained architecture is contained in the unconstrained space, the constrained-space accuracy is a lower bound on what a perfect unconstrained search could achieve; the moderate gains reported here likely reflect optimizer limits rather than a ceiling of the space itself.","The entropy-shrinking feedback loop is a generic sparsification controller: replacing sparsity entropy with latency, memory, or energy proxies could yield hardware-diverse architectures from the same search framework.","A direct transfer test would run the same ESS shrinking on a non-cell-based differentiable search space, such as transformer depth or mixed-precision configurations, to see whether the stability benefits generalize.","The reported sensitivity to the hand-set coefficients $c_1,c_2$ and to the expected entropy reduction $\\Delta E$ suggests that automatic tuning of the shrinking schedule could close the remaining gap to constrained-space methods."],"forward_implications":["A single FX-DARTS run can return several discrete architectures spanning different parameter and FLOP counts, so practitioners can trade accuracy for cost without rerunning the search.","Searched networks are no longer limited to the shared normal/reduction cell pattern, expanding the set of candidate architectures available for a given task.","The search cost of 1.6 to 4.3 GPU-hours keeps topology-unconstrained search in the same practical range as DARTS itself.","Architectures found on TinyImageNet transfer to ImageNet-1K at 76.0 to 76.4% top-1 accuracy, indicating the flexible search is not tied to the CIFAR proxy."],"supporting_citations":[{"why":"Supplies the differentiable search formulation and the constrained cell-based search space that FX-DARTS relaxes.","marker":"[18]"},{"why":"Shows progressive super-network pruning in a flexible search space; ESS extends this idea with entropy-based control.","marker":"[30]"},{"why":"Demonstrates an alternative unconstrained-space search via distillation, whose complicated hyper-parameter selection FX-DARTS aims to avoid.","marker":"[52]"},{"why":"Shows randomly wired flexible networks already beat DARTS on ImageNet, motivating the flexible-search direction.","marker":"[51]"},{"why":"Provides the topology-constrained PC-DARTS baseline that FX-DARTS matches or beats on CIFAR and ImageNet.","marker":"[53]"},{"why":"Documents how evaluation protocols dominate reported NAS accuracy, justifying FX-DARTS' re-evaluation methodology without drop-path.","marker":"[41]"}],"fun_headline_variants":["DARTS without priors: 76.4% ImageNet via entropy shrinking","Topology-free DARTS: stable search, 76.4% top-1","FX-DARTS: unconstrained cells, entropy-stabilized search","No shared cells, no two-operator rule: DARTS still works","Entropy shrinks super-network for unconstrained DARTS"],"cache_read_input_tokens":3200,"weakest_assumption_plain":"The load-bearing premise is that forcing each node's operator weights to become sparse, then deleting everything below a 0.02 threshold, yields a discrete architecture whose final accuracy tracks what the super-network promised.","fun_headline_variants_meta":{"raw":{"variants":["DARTS without priors: 76.4% ImageNet via entropy shrinking","Topology-free DARTS: stable search, 76.4% top-1","FX-DARTS: unconstrained cells, entropy-stabilized search","No shared cells, no two-operator rule: DARTS still works","Entropy shrinks super-network for unconstrained DARTS"]},"model":"deepseek-v4-flash","effort":"low","cost_usd":0.000555,"raw_usage":{"total_tokens":2632,"prompt_tokens":923,"completion_tokens":1709,"prompt_tokens_details":{"cached_tokens":384},"prompt_cache_hit_tokens":384,"prompt_cache_miss_tokens":539,"completion_tokens_details":{"reasoning_tokens":1608}},"tokens_in":539,"tokens_out":1709,"duration_ms":11182,"temperature":1.0,"reasoning_tokens":1608,"cache_read_input_tokens":384,"cache_creation_input_tokens":0},"cache_creation_input_tokens":0},"created_at":"2026-08-16T10:22:16.474110+00:00","model_set":{"reader":"deepseek-v4-flash"},"falsifier":"Retrain each threshold-pruned FX-DARTS architecture and the un-pruned super-network under identical budgets on the same validation split; if the pruned models do not consistently match or beat the super-network's accuracy, the entropy-shrinking route to discrete architectures fails.","supporting_citations":[{"cited_title":"DARTS: Differentiable architecture search,","cited_arxiv_id":null,"evidence_quote":"Supplies the differentiable search formulation and the constrained cell-based search space that FX-DARTS relaxes."},{"cited_title":"GOLD-NAS: Gradual, One-Level, Differentiable","cited_arxiv_id":"2007.03331","evidence_quote":"Shows progressive super-network pruning in a flexible search space; ESS extends this idea with entropy-based control."},{"cited_title":"DNAD: Differentiable neural architecture distillation,","cited_arxiv_id":null,"evidence_quote":"Demonstrates an alternative unconstrained-space search via distillation, whose complicated hyper-parameter selection FX-DARTS aims to avoid."},{"cited_title":"Exploring randomly wired neural networks for image recognition,","cited_arxiv_id":null,"evidence_quote":"Shows randomly wired flexible networks already beat DARTS on ImageNet, motivating the flexible-search direction."},{"cited_title":"Partially-connected neural architecture search for reduced computational redundancy,","cited_arxiv_id":null,"evidence_quote":"Provides the topology-constrained PC-DARTS baseline that FX-DARTS matches or beats on CIFAR and ImageNet."},{"cited_title":"NAS evaluation is frustratingly hard,","cited_arxiv_id":null,"evidence_quote":"Documents how evaluation protocols dominate reported NAS accuracy, justifying FX-DARTS' re-evaluation methodology without drop-path."}],"review_version":1}