A Topological Improvement of the Overall Performance of Sparse Evolutionary Training: Motif-Based Structural Optimization of Sparse MLPs Project

Hongyun Liu; Seyed Sahand Mohammadi Ziabari; Xiaotian Chen

REVIEW 5 major objections 6 minor 30 references

A Topological Improvement of the Overall Performance of Sparse Evolutionary Training: Motif-Based Structural Optimization of Sparse MLPs Project

T0 review · 5 major / 6 minor · reviewed 2026-08-07 · deepseek-v4-flash

Challenge this review Re-run · record.json Download PDF Read on arXiv ↗

Pith's one-line read By grouping neurons into motifs of size 2 with shared weights, Sparse Evolutionary Training of MLPs cuts training time by 30–43 percent while losing only 1–4 percent accuracy.

desk verdict A clearly written but internally inconsistent empirical note; the central efficiency claim is not supported by the numbers as printed. read the letter →

arxiv 2506.09204 v1 pith:GAVPIWAN submitted 2025-06-10 cs.NE cs.AI

Xiaotian Chen , Hongyun Liu , Seyed Sahand Mohammadi Ziabari This is my paper · ORCID

classification cs.NEcs.AI

keywords SparseNeuralNetworksEvolutionaryTrainingMotif-BasedOptimizationMulti-layerPerceptronWeightSharingEfficiency–AccuracyTrade-offTopology

verification ladder T0 review T1 audit T2 compute T3 formal

The pith

A machine-rendered reading of the paper's core claim, the machinery that carries it, and where it could break.

The reading

The paper argues that Sparse Evolutionary Training of multilayer perceptrons (SET-MLP) can be made substantially cheaper by grouping neurons into fixed-size "motifs" whose weights are initialized, propagated, and updated together. Testing motif sizes 1 (the standard SET), 2, and 4 on Fashion MNIST and a lung X-ray dataset, it finds that motif size 2 gives the best efficiency–accuracy trade-off: 43.3% faster training with a 3.7% accuracy drop on Fashion MNIST, and 30.4% faster with a 1.2% drop on Lung. A weighted comprehensive score that values accuracy at 90% and efficiency at 10% puts motif size 2 ahead of the SET baseline on both datasets. If the result holds beyond these two benchmarks, it offers a cheap way to speed up sparse MLP training for latency- and energy-sensitive applications.

What carries the argument

The load-bearing mechanism is motif-based structural optimization: weights are initialized and updated per fixed-size group of neurons rather than per neuron, so each forward and backward pass operates on motifs while the network still uses Erdős–Rényi sparse connectivity and He-initialized weights. The paper couples this with a comprehensive score $S = 0.1 R_r + 0.9(1 - A_r)$ that converts runtime reduction $R_r$ and accuracy loss $A_r$ into a single number, giving accuracy nine times the weight of efficiency. That score is what makes "motif size 2 is best" a quantitative conclusion rather than a visual impression.

What would settle it

For any new dataset, the paper's claim that motif size 2 is optimal implies the inequality $R_r > 9 A_r$—runtime reduction must be more than nine times the accuracy loss because the score weights accuracy nine times more heavily than efficiency. A single dataset where a motif size 2 run satisfies $R_r \le 9 A_r$, or where motif size 1 beats it on the comprehensive score under the reported 90/10 weighting, would falsify the central conclusion.

Watch

Extended reading notes

Core claim

On the paper's own terms, the discovery is that a topological reorganization of SET-MLP—replacing per-neuron weight handling with per-motif weight handling, where a motif is a fixed block of adjacent neurons sharing one set of weights—improves efficiency without sacrificing the accuracy advantage of sparse training. For Fashion MNIST, moving from motif size 1 (the SET benchmark) to motif size 2 cut total training time from 25,236.2 s to 14,307.5 s, a 43.3% reduction, while accuracy fell from 0.7610 to 0.7330; motif size 4 was faster still (9,209.3 s) but lost 9.7% accuracy. On the Lung dataset, motif size 2 reduced runtime by 30.4% with only a 1.2% accuracy loss. Using the comprehensive score $S = 0.1 R_r + 0.9(1 - A_r)$, where $R_r$ is fractional runtime reduction and $A_r$ is fractional accuracy loss, the paper concludes that motif size 2 is the optimal configuration among those tested.

Load-bearing premise

The argument stands on the assumption that tying together the weights of a fixed block of neurons preserves enough representational capacity that the faster training justifies the accuracy loss; the paper offers no theoretical or ablation evidence for this, so if weight sharing within motifs destroys capacity faster than it saves computation, the claimed trade-off collapses.

Editorial extensions

If this is right

Motif size 2 can be adopted directly in SET-MLP implementations as a cheap speed-up: roughly 30–43% less wall-clock training time for a 1–4% accuracy drop on the two tested benchmarks.
Pushing motif size to 4 is not worthwhile under the paper's 90/10 accuracy/efficiency weighting, since the extra speed is outweighed by the larger accuracy loss.
Once efficiency receives any weight above 0.1 in the evaluation score, the paper's trade-off analysis shows motif-based models outperforming the standard SET baseline.
The optimal motif size depends on dataset and architecture, so the paper's recipe is to tune motif size per application rather than assume size 2 is universal.

Reading between the lines

Editorial extensions of the paper, not claims the author makes directly.

The reported wall-clock gains are measured on the authors' implementation, where per-motif loops shrink the number of update steps; a vectorized GPU implementation might shrink or erase the speed advantage, so the 43.3% figure should be seen as implementation-level until reproduced on other hardware.
A direct test of capacity would vary motif size while holding total parameter count or total FLOPs fixed; if motif size 2 still wins, the benefit is structural, but if it only wins when the update loop is smaller, the benefit is mostly computational bookkeeping.
The paper's own future-work suggestion of adaptive motif size points to a cheap test: start training with motif size 1 and coarsen to size 2 later; if early coarse grouping causes the accuracy loss, late coarsening could preserve accuracy while still saving time.
The stated efficiency gains are relative to SET's baseline, not to a dense MLP; for practitioners coming from dense networks, the appropriate comparison is SET-MLP with motif size 2 versus a dense MLP of equal accuracy, which the paper does not report.

Share X Bluesky LinkedIn Reddit HN

Editorial analysis

A structured set of objections, weighed in public.

Desk editor's note, referee report, and a circularity audit.

Desk Editor's Note

read the letter

The one thing you should know: this paper is not ready for serious review. The method is block-wise weight sharing applied to SET-MLP, and the reported efficiency numbers do not add up. The stress-test note holds up. Table 2 gives total runtime 25236.2 s and average 17.73 s/epoch for the SET baseline; 17.73 × 300 = 5319, not 25236.2. That is a factor of about 4.75, which means one of those columns describes a different run or a different model. The prose says motif size 4 improves efficiency by 73.7%, but Table 2 gives 9209.3 s versus 25236.2 s, which is 63.5%. Eq. (6) uses R_r = 0.637 and prints S4 = 0.8864, but 0.1×0.637 + 0.9×0.903 = 0.8764. These are not cosmetic typos; the final recommendation that motif size 2 is optimal rests on margins like 0.9100 vs 0.9000, from single runs with no error bars, no seeds, and no code.

What is actually new: very little. Grouping neurons into fixed-size blocks and sharing weights is an old idea (tied weights, group convolutions, parameter sharing). The paper relabels this as “motif-based structural optimization” but performs no network-motif analysis and proposes no new mechanism. It tests block sizes 2 and 4 on two datasets, which is a routine extension. To be fair, the paper does a few things well: the algorithm is described clearly in prose and pseudocode, the control-variable setup is sensible, and the authors acknowledge that results are dataset-dependent and that the composite score’s weights are a choice. That level of honesty is commendable.

The soft spots are load-bearing. The composite score uses hand-picked weights (0.9 accuracy, 0.1 runtime), and the paper’s own Figures 8 and 9 show that other weightings change the winner, so the “motif size 2 is optimal” conclusion is largely an artifact of that choice. The internal arithmetic errors make the headline 43.3% and 30.4% gains unreliable. There is no external reproducibility evidence: no code, no data release, no repeated runs, no error bars. The central trade-off claim is therefore not demonstrated.

Who is this for? A reader curious about whether simple block-wise weight sharing can speed up SET might skim Section 3, but they should not rely on the numbers. It does not deserve a serious referee in its current form; the inconsistencies need fixing and the method needs comparison to existing parameter-sharing baselines before it is a candidate for publication.

Recommendation: desk reject. If the authors fix the arithmetic, report seeds and variance, and compare against standard tied-weight baselines, it could become a minor workshop-level contribution.

Referee Report

5 major / 6 minor

Summary. The paper proposes a motif-based structural optimization of Sparse Evolutionary Training (SET) MLPs, in which neurons are grouped into fixed-size motifs with shared weights, and claims this improves training efficiency at a small accuracy cost. Experiments on Fashion-MNIST and Lung datasets compare motif sizes 1, 2, and 4 and use a composite score S = 0.1*R_r + 0.9*(1 - A_r) to select the best configuration. The central claim is that motif size 2 is optimal, giving a 43.3% efficiency improvement with a 3.7% accuracy drop on FMNIST and a 30.4% efficiency gain with a 1.2% accuracy drop on Lung.

Significance. If the claimed gains were reliable, the idea of improving SET-MLP efficiency by grouping neurons into motifs would be a simple and potentially useful contribution for efficiency-critical applications. The paper is clearly organized, and Figures 8 and 9 attempt to probe the sensitivity of the composite score, which is a strength. However, the empirical basis is not currently reliable: the reported timing data are internally inconsistent, the composite-score arithmetic contains errors, and there are no repeated runs or code/seed information. Because the headline numbers drive every conclusion in the paper, the contribution cannot be evaluated without substantially redoing the experiments.

major comments (5)

[Table 2 and Section 5.1.1] Table 2 reports total running times and average times per epoch for 300 epochs, but the two columns are mutually inconsistent: for motif size 1, 17.73 * 300 = 5319 s, not 25236.2 s; for motif size 2, 9.14 * 300 = 2742 s, not 14307.5 s; and for motif size 4, 6.74 * 300 = 2022 s, not 9209.3 s. Since the runtime reductions used in the composite score are computed from the total running times, this discrepancy means the central efficiency measurements are not self-consistent, and the reported gains cannot be trusted without clarification.
[Section 5.1.1, Eq. (6)] The prose states that motif size 4 improves efficiency by 73.7%, but Eq. (6) uses R_r = 0.637, and the printed times give (25236.2 - 9209.3)/25236.2 = 0.635. In addition, substituting R_r = 0.637 and A_r = 0.097 into Eq. (6) gives S_4 = 0.8764, not the printed 0.8864. These errors matter because the selection of motif size 2 depends on small score differences, such as S_2 = 0.9100 versus S_1 = 0.9000 on FMNIST.
[Section 4.2, Eq. (1); Figures 8 and 9] The composite score uses hand-picked weights of 0.9 for accuracy and 0.1 for runtime with no principled justification. Figures 8 and 9 are said to show that the ranking changes when the efficiency weight exceeds 0.1, which means the paper's conclusion that motif size 2 is 'most optimal' is an artifact of the chosen weights. The manuscript needs either a principled justification for the specific weights or a sensitivity analysis showing that the conclusion is stable over a reasonable weight range.
[Section 4.2, Table 2] All results are based on single runs with no error bars, no seeds, and no code. The decisive score differences are small, for example 0.9100 versus 0.9000 on FMNIST and 0.9199 versus 0.9089 on Lung, so without run-to-run variance it is impossible to tell whether the differences are significant. The paper states that its setup 'ensures reproducibility,' but it provides none of the materials needed to reproduce the experiments.
[Section 3.1 and Section 3.2] The paper assumes that grouping neurons into fixed-size motifs with shared weights preserves enough representational capacity to keep accuracy close to the ungrouped SET baseline, but no theory, ablation, or capacity analysis is provided to support this. Since increasing motif size reduces the number of free parameters, the observed speedups may simply reflect a smaller effective model rather than a 'topological' benefit; the manuscript should test this by comparing against SET baselines with matched parameter counts.

minor comments (6)

[Tables 1 and 2] Tables 1 and 2 present the same FMNIST results; one should be removed.
[Section 4.1] The Lung dataset is not described with a source or reference; add a citation and details of the train/test split and class distribution.
[Section 5.1.1] The simplified model with two 1000-neuron layers is mentioned, but its results are never reported in the text; clarify whether Table 2's per-epoch column refers to the simplified model.
[Figure 1] Figure 1 is reproduced from reference [15]; a permission statement or an original figure is needed.
[References] Reference [17] has an incomplete bibliographic entry, and several other references lack page numbers or venues; the bibliography needs to be cleaned up.
[Equations (1)-(3)] The definitions of R_r and A_r appear after the composite score is introduced; reorder the equations so that all symbols are defined before use.

Circularity Check

0 steps flagged · score 0.0 of 10

No significant circularity: the paper's claims rest on external benchmark measurements and a transparent composite score, not on self-citations or fitted predictions.

full rationale

The derivation chain is not circular. The SET baseline and motif-based variants are implemented from external sources (Mocanu et al. [16], Kichler [12], Milo et al. [15]) and evaluated on external datasets (FMNIST and Lung). The claimed efficiency gains are arithmetic ratios of measured runtimes (e.g., 43.3% from Table 2), not fitted parameters disguised as predictions. The composite score S is an author-chosen evaluation metric, not an input that the data is forced to reproduce; the paper explicitly acknowledges that the weighting is not universally accepted (Section 8) and reports sensitivity to the weighting in Figures 8 and 9. There are no load-bearing self-citations: the cited prior works share no authors with this paper, and no uniqueness theorem or ansatz is smuggled in from the authors' own prior work. The internal inconsistencies in Table 2 (total runtime versus per-epoch runtime) and the arithmetic error in Eq. 6 are correctness concerns, not circularity, and do not change the circularity score.

Assumptions & free parameters 3 free parameters · 4 assumptions · 1 invented entities

The central claim depends on an arbitrary composite score, a domain assumption that weight sharing within blocks preserves accuracy, and a redefinition of motif that carries no independent structural significance. No parameters are fitted to data; the listed free parameters are design choices.

free parameters (3)

Composite score weights = 0.9 for accuracy, 0.1 for runtime
Chosen by hand from management literature [20,21]; the conclusion that motif size 2 is best depends directly on this weighting.
Sparsity epsilon = 0.1
Fixed for all runs and not varied; standard hyperparameter.
Motif sizes tested = 1, 2, 4
A hand-picked grid; the paper selects the best from this grid rather than deriving an optimal size.

assumptions (4)

standard math Backpropagation and stochastic gradient descent update rules for sparse MLPs are correct and standard.
Section 3.2 uses standard forward and backward equations; no proof of convergence or correctness is given, but none is needed for standard MLP training.
domain assumption Weight sharing within fixed-size groups of neurons preserves enough accuracy for the trade-off to hold.
Section 3.1 assumes grouping nodes into motifs and assigning shared weights improves efficiency without a large accuracy drop; no theoretical or empirical justification is provided.
domain assumption Erdős-Rényi sparse topology and SET evolution (pruning and re-adding weights) remain beneficial when applied to motif groups.
The paper carries SET's evolution mechanism over to the motif-based model without analyzing interactions between grouping and evolutionary rewiring.
ad hoc to paper The weighted composite score S = 0.1 R_r + 0.9 (1 - A_r) is an appropriate measure of overall performance.
Introduced in Section 4.2 with weights chosen by hand; the best configuration is defined by this score, so the conclusion is sensitive to the choice.

invented entities (1)

Motif as a fixed-size group of neurons with shared weights
purpose: Reduce parameter count and training time in SET-MLP
The paper borrows the term from network motifs (Milo et al.) but redefines it as an arbitrary block of neurons; no motif-significance testing or structural analysis is performed, so the entity has no independent evidence.

how reviews work

0 comments

Cite this review

Pith. "Pith review of A Topological Improvement of the Overall Performance of Sparse Evolutionary Training: Motif-Based Structural Optimization of Sparse MLPs Project." pith.science (2026). https://pith.science/paper/GAVPIWAN

@misc{pith2026250609204,
  author       = {Pith},
  title        = {Pith review of: A Topological Improvement of the Overall Performance of Sparse Evolutionary Training: Motif-Based Structural Optimization of Sparse MLPs Project},
  year         = {2026},
  howpublished = {\url{https://pith.science/paper/GAVPIWAN}},
  note         = {Machine review of arXiv:2506.09204}
}

read the original abstract

Deep Neural Networks (DNNs) have been proven to be exceptionally effective and have been applied across diverse domains within deep learning. However, as DNN models increase in complexity, the demand for reduced computational costs and memory overheads has become increasingly urgent. Sparsity has emerged as a leading approach in this area. The robustness of sparse Multi-layer Perceptrons (MLPs) for supervised feature selection, along with the application of Sparse Evolutionary Training (SET), illustrates the feasibility of reducing computational costs without compromising accuracy. Moreover, it is believed that the SET algorithm can still be improved through a structural optimization method called motif-based optimization, with potential efficiency gains exceeding 40% and a performance decline of under 4%. This research investigates whether the structural optimization of Sparse Evolutionary Training applied to Multi-layer Perceptrons (SET-MLP) can enhance performance and to what extent this improvement can be achieved.

Figures

Figures reproduced from arXiv: 2506.09204 by the authors.

**Figure 3.** Concept of motif-based SET training The motif-based concept refers to a specific type of structural topology or network pattern, as illustrated in [PITH_FULL_IMAGE:figures/full_fig_p002_3.png] view at source ↗

**Figure 4.** Sample from FMNIST Dataset [PITH_FULL_IMAGE:figures/full_fig_p004_4.png] view at source ↗

Figures from the paper (2 more)

**Figure 7.** Figure 7: FMNIST Efficiency (First 30 Epochs) [PITH_FULL_IMAGE:figures/full_fig_p004_7.png]

**Figure 8.** Figure 8: Efficiency vs. Accuracy Weight Ratio and Comprehensive Score (Lung) [PITH_FULL_IMAGE:figures/full_fig_p005_8.png]

Discussion (0). Continue with ORCID to comment.

Reference graph

Works this paper leans on

30 extracted references · 14 canonical work pages

[1]

N. S. Altman. [n. d.]. An Introduction to Kernel and Nearest-Neighbor Nonpara- metric Regression. 46, 3 ([n. d.]), 175–185. https://doi.org/10.1080/00031305.1992. 10475879

arXiv 1992
[2]

Arash Ardakani, Carlo Condo, and Warren J. Gross. [n. d.]. Sparsely-Connected Neural Networks: Towards Efficient VLSI Implementation of Deep Neural Net- works. https://doi.org/10.48550/arXiv.1611.01427 arXiv:1611.01427 [cs]

work page Pith review arXiv doi:10.48550/arxiv.1611.01427
[3]

Zahra Atashgahi, Ghada Sokar, Tim van der Lee, Elena Mocanu, Decebal Con- stantin Mocanu, Raymond Veldhuis, and Mykola Pechenizkiy. [n. d.]. Quick and Robust Feature Selection: the Strength of Energy-efficient Sparse Training for Autoencoders. ([n. d.]). https://doi.org/10.48550/ARXIV.2012.00560

work page Pith review arXiv doi:10.48550/arxiv.2012.00560
[4]

Albert-Laszlo Barabasi and Reka Albert. [n. d.]. Emergence of Scaling in Random Networks. 286, 5439 ([n. d.]), 509–512. https://doi.org/10.1126/science.286.5439. 509

doi:10.1126/science.286.5439
[5]

Guillaume Bellec, David Kappel, Wolfgang Maass, and Robert Legenstein. [n. d.]. Deep Rewiring: Training very sparse deep networks. https://doi.org/10.48550/ arXiv.1711.05136 arXiv:1711.05136 [cs, stat]

arXiv doi:10.48550/arxiv.1711.05136
[6]

Zhang Bin, Gan Zhi-chun, Chen Wen, Hu Qiang-qiang, and Hou Jian-feng. 2019. Topology Optimization of Complex Network based on NSGA-II. In 2019 IEEE 4th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC). IEEE, Chengdu, China, 1680–1685. https://doi.org/10.1109/IAEAC47372. 2019.8997597

work page arXiv 2019
[7]

Ed Bullmore and Olaf Sporns. 2009. Complex brain networks: graph theoretical analysis of structural and functional systems. Nature Reviews Neuroscience 10, 3 (March 2009), 186–198. https://doi.org/10.1038/nrn2575

doi:10.1038/nrn2575 2009
[8]

Soravit Changpinyo, Mark Sandler, and Andrey Zhmoginov. 2017. The Power of Sparsity in Convolutional Neural Networks. https://doi.org/10.48550/arXiv.1702. 06257 arXiv:1702.06257 [cs]

arXiv doi:10.48550/arxiv.1702 2017

Show all 30 references

[9]

Song Han, Huizi Mao, and William J. Dally. 2016. Deep Compression: Compress- ing Deep Neural Networks with Pruning, Trained Quantization and Huffman Coding. https://doi.org/10.48550/arXiv.1510.00149 arXiv:1510.00149 [cs]

2016 doi arXiv
[10]

Hassibi, D.G

B. Hassibi, D.G. Stork, and G.J. Wolff. 1993. Optimal Brain Surgeon and general network pruning. In IEEE International Conference on Neural Networks . 293–299 vol.1. https://doi.org/10.1109/ICNN.1993.298572

1993
[11]

Tomohiro Hayase and Ryo Karakida. [n. d.]. MLP-Mixer as a Wide and Sparse MLP. ([n. d.]). https://doi.org/10.48550/ARXIV.2306.01470

doi arXiv
[12]

Neil Kichler. [n. d.]. Robustness of sparse MLPs for supervised feature selection . https://essay.utwente.nl/86886/
[13]

Yann LeCun, John Denker, and Sara Solla. 1989. Optimal Brain Dam- age. In Advances in Neural Information Processing Systems , Vol. 2. Morgan-Kaufmann. https://proceedings.neurips.cc/paper/1989/hash/ 6c9882bbac1c7093bd25041881277658-Abstract.html

1989
[14]

Shiwei Liu, Tim Van der Lee, Anil Yaman, Zahra Atashgahi, Davide Ferraro, Ghada Sokar, Mykola Pechenizkiy, and Decebal Constantin Mocanu. [n. d.]. Topological Insights into Sparse Neural Networks. https://doi.org/10.48550/arXiv.2006.14085 arXiv:2006.14085 [cs, stat]

doi arXiv
[15]

R. Milo, S. Shen-Orr, S. Itzkovitz, N. Kashtan, D. Chklovskii, and U. Alon. [n. d.]. Network Motifs: Simple Building Blocks of Complex Networks. 298, 5594 ([n. d.]), 824–827. https://doi.org/10.1126/science.298.5594.824

doi
[16]

Nguyen, Madeleine Gibescu, and Antonio Liotta

Decebal Constantin Mocanu, Elena Mocanu, Peter Stone, Phuong H. Nguyen, Madeleine Gibescu, and Antonio Liotta. [n. d.]. Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science. 9, 1 ([n. d.]), 2383. https://doi.org/10.1038/...

doi
[17]

Hesham Mostafa and Xin Wang. [n. d.]. Parameter efficient training of deep convolutional neural networks by dynamic sparse reparameterization. ([n. d.]). https://openreview.net/forum?id=S1xBioR5KX
[18]

Jinu Sophia and T

J. Jinu Sophia and T. Prem Jacob. 2024. A Comprehensive Analysis of Exploring the Efficacy of Machine Learning Algorithms in Text, Image, and Speech Analysis. Journal of Electrical Systems 20, 2s (April 2024), 910–921. https://doi.org/10.52783/ jes.1688

2024
[19]

Youcheng Sun, Xiaowei Huang, Daniel Kroening, James Sharp, Matthew Hill, and Rob Ashmore. 2019. Testing Deep Neural Networks. https://doi.org/10. 48550/arXiv.1803.04792 arXiv:1803.04792 [cs]

2019 doi arXiv
[20]

Morgan Swink, Srinivas Talluri, and Temyos Pandejpong. 2006. Faster, better, cheaper: A study of NPD project efficiency and performance tradeoffs. Journal of Operations Management 24, 5 (Sept. 2006), 542–562. https://doi.org/10.1016/j. jom.2005.09.004

2006 doi
[21]

Justin Tan and Liang Wang. 2010. Flexibilityefficiency tradeoff and performance implications among Chinese SOEs. Journal of Business Research 63, 4 (April 2010), 356–362. https://doi.org/10.1016/j.jbusres.2009.04.016

2010 doi
[22]

The Theano Development Team, Rami Al-Rfou, Guillaume Alain, Amjad Alma- hairi, Christof Angermueller, Dzmitry Bahdanau, Nicolas Ballas, Frederic Bastien, Justin Bayer, Anatoly Belikov, Alexander Belopolsky, Yoshua Bengio, Arnaud Bergeron, James Bergstra, Valentin Bisson, Josh ...

2016 doi arXiv
[23]

Gomez, Lukasz Kaiser, and Illia Polosukhin

Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2023. Attention Is All You Need. https://doi.org/10.48550/arXiv.1706.03762 arXiv:1706.03762 [cs]

2023 doi arXiv
[24]

Mowei Wang, Yong Cui, Shihan Xiao, Xin Wang, Dan Yang, Kai Chen, and Jun Zhu. 2018. Neural Network Meets DCN: Traffic-driven Topology Adaptation with Deep Learning. Proceedings of the ACM on Measurement and Analysis of Computing Systems 2, 2 (June 2018), 1–25. https://doi.org/...

2018 doi
[25]

Zi Wang, Jihye Choi, Ke Wang, and Somesh Jha. 2024. Rethinking Diversity in Deep Neural Network Testing. https://doi.org/10.48550/arXiv.2305.15698 arXiv:2305.15698 [cs]

2024 doi arXiv
[26]

Han, Kilian Q

Felix Wu, Kwangyoun Kim, Jing Pan, Kyu J. Han, Kilian Q. Weinberger, and Yoav Artzi. [n. d.]. Performance-Efficiency Trade-Offs in Unsupervised Pre-Training for Speech Recognition. In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (...

2022
[27]

Han Xiao, Kashif Rasul, and Roland Vollgraf. 2017. Fashion-MNIST: a Novel Image Dataset for Benchmarking Machine Learning Algorithms. https://doi. org/10.48550/arXiv.1708.07747 arXiv:1708.07747 [cs, stat]

2017 doi arXiv
[28]

Zi Yin and Yuanyuan Shen. 2018. On the Dimensionality of Word Embedding. https://doi.org/10.48550/arXiv.1812.04224 arXiv:1812.04224 [cs, stat]

2018 doi arXiv
[29]

Shijin Zhang, Zidong Du, Lei Zhang, Huiying Lan, Shaoli Liu, Ling Li, Qi Guo, Tianshi Chen, and Yunji Chen. 2016. Cambricon-X: An accelerator for sparse neural networks. In 2016 49th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO). 1–12. https://doi.org/10...

2016
[30]

Shuai Zhang, Bo Yin, Weiyi Zhang, and Yu Cheng. 2022. Topology Aware Deep Learning for Wireless Network Optimization. IEEE Transactions on Wireless Communications 21, 11 (Nov. 2022), 9791–9805. https://doi.org/10.1109/TWC. 2022.3179352

2022

Pith tools

Reviewed August 7, 2026 · model on record in the stance chip above.

[1] [1]

N. S. Altman. [n. d.]. An Introduction to Kernel and Nearest-Neighbor Nonpara- metric Regression. 46, 3 ([n. d.]), 175–185. https://doi.org/10.1080/00031305.1992. 10475879

arXiv 1992

[2] [2]

Arash Ardakani, Carlo Condo, and Warren J. Gross. [n. d.]. Sparsely-Connected Neural Networks: Towards Efficient VLSI Implementation of Deep Neural Net- works. https://doi.org/10.48550/arXiv.1611.01427 arXiv:1611.01427 [cs]

work page Pith review arXiv doi:10.48550/arxiv.1611.01427

[3] [3]

Zahra Atashgahi, Ghada Sokar, Tim van der Lee, Elena Mocanu, Decebal Con- stantin Mocanu, Raymond Veldhuis, and Mykola Pechenizkiy. [n. d.]. Quick and Robust Feature Selection: the Strength of Energy-efficient Sparse Training for Autoencoders. ([n. d.]). https://doi.org/10.48550/ARXIV.2012.00560

work page Pith review arXiv doi:10.48550/arxiv.2012.00560

[4] [4]

Albert-Laszlo Barabasi and Reka Albert. [n. d.]. Emergence of Scaling in Random Networks. 286, 5439 ([n. d.]), 509–512. https://doi.org/10.1126/science.286.5439. 509

doi:10.1126/science.286.5439

[5] [5]

Guillaume Bellec, David Kappel, Wolfgang Maass, and Robert Legenstein. [n. d.]. Deep Rewiring: Training very sparse deep networks. https://doi.org/10.48550/ arXiv.1711.05136 arXiv:1711.05136 [cs, stat]

arXiv doi:10.48550/arxiv.1711.05136

[6] [6]

Zhang Bin, Gan Zhi-chun, Chen Wen, Hu Qiang-qiang, and Hou Jian-feng. 2019. Topology Optimization of Complex Network based on NSGA-II. In 2019 IEEE 4th Advanced Information Technology, Electronic and Automation Control Conference (IAEAC). IEEE, Chengdu, China, 1680–1685. https://doi.org/10.1109/IAEAC47372. 2019.8997597

work page arXiv 2019

[7] [7]

Ed Bullmore and Olaf Sporns. 2009. Complex brain networks: graph theoretical analysis of structural and functional systems. Nature Reviews Neuroscience 10, 3 (March 2009), 186–198. https://doi.org/10.1038/nrn2575

doi:10.1038/nrn2575 2009

[8] [8]

Soravit Changpinyo, Mark Sandler, and Andrey Zhmoginov. 2017. The Power of Sparsity in Convolutional Neural Networks. https://doi.org/10.48550/arXiv.1702. 06257 arXiv:1702.06257 [cs]

arXiv doi:10.48550/arxiv.1702 2017